Fisheye to Spherical Conversion using After Effects

fisheye-to-spherical-conversion-of-kodak-pixpro-sp360-4k-footage

Lately I’ve been shooting with the Kodak PIXPRO SP360 4k camera for both VR and the planetarium dome. Shooting with dual cameras is great for capturing 360°, but shooting with a single camera is sometimes easier since the footage doesn’t need to be stitched. Also a single camera captures 235° which is a surprisingly huge FOV. Yet it’s necessary to warp the footage from fisheye to spherical so that it can be experienced in VR or Youtube 360.

The ‘Pixpro SP360 4k’ software is actually capable of warping a single camera from fisheye to spherical. But it’s not intuitive (here is a tutorial) and the Kodak software can only export footage to MP4… And seeing as how the raw camera footage is an already heavily compressed MP4, I wasn’t thrilled about this added lossy step. So I figured out a simple technqiue.

If you’re wondering… the terms Spherical, Equirectangular, and LatLong refer to the EXACT same thing.


Tutorial using After Effects without plugins

— This technique is a hack and the warping isn’t ideal for all occasions. Yet it really depends on whether your fisheye lens is equidistant or equisolid angle. Equidistant fisheye lenses can get a near perfect conversion using this technique. But equisolid angle fisheye lenses are unique and therefore this technique cannot provide an accurate conversion. The technique still works for equisolid angle fisheye lenses but parts of the image will look slightly stretched or squashed vertically when viewed in VR or Youtube 360. For instance, the SP360 4k camera has an equisolid angle fisheye lens and yet it’s the camera I used in this tutorial and achieved decent results. On that note, I actually haven’t been able to confirm from any official Kodak specs that the SP360 4k camera indeed uses a equisolid angle fisheye lens, but it seems pretty obvious when comparing renders from this After Effects technique against the ‘Pixpro SP360 4k’ warping software.
— You cannot use gaussian blur, sharpen, or such effects since they would create very obvious seams when viewed in VR or Youtube 360. But you could instead use the Skybox 360 Post FX since they are seamless VR effects.
— If you need to adjust the horizon level, then you’ll need to instead use the RE:Lens plugin or Skybox Studio V2 which provides much better controls and proper conversion tools.

DOWNLOAD: Fisheye FOV Guide
required to complete tutorial

convert-fisheye-to-spherical-using-after-effects-no-plugins




Spherical to Fisheye Conversion

Or if you need to convert spherical to fisheye (so it can be watched in a dome) then the same process can be applied in reverse. But you’ll need to scale up the footage to crop out some of the unwanted FOV and unfortunately it’s a lossy conversion since it’s being uprezzed. Also you cannot change the FOV accurately, it’s just a very basic conversion. Although a perfect conversion can be achieved using a plugin from this list.

15 thoughts on “Fisheye to Spherical Conversion using After Effects

  1. This tutorial is phenomenal! One question- your footage is captured pointing straight up, any way to apply these same functions to a video capture pointing forward?

    • Good point! If you’re shooting with a fisheye camera pointing forward, then you would want the 360 footage also be pointing forward and not straight up.

      The tutorial above can take any fisheye footage, no matter what orientation it was originally shot, and then convert it to spherical. But the resulting 360 video will always have the fisheye footage oriented straight up. This technique is a hack since we are tricking After Effects into creating a 360 video.

      This is frustrating because we are so close! We already have the footage converted to spherical and now we simply need to level the horizon. But there isn’t a way to achieve this without using a custom plugin and they are not cheap. (SkyBox Rotate Sphere, 360VR Reorient Sphere, GoPro VR Horizon)

      Honestly you are much better off using the RE:Lens plugin. Not only will it convert fisheye footage to spherical, but it will also allow you to level the horizon. Below are some example screenshots using a demo of RE:Lens.

  2. Hi Jason!

    I’ve been trying to edit some footage of my Samsung Gear 360 following your tutorial but I got stuck in step 3 since the result of the conversion makes my footage looks really weird… It doesn’t look like your example at all but totally deformed with some “arcs” in the upper part (I wish I could post a screenshot here so you could see what I mean).
    Any idea how can I correct this? I’ve tried plenty of times with different parameters but nothing seems to work 😦

    Would be great if you could give me a hand with this.

    Thanks in advance!

    • Hi Lucia! I’m not quite sure what problem you’re experiencing. Maybe you didn’t pre-comp the footage layer before applying the Polar Coordinates effect. Why don’t you upload a screenshot to imgur.com and then I can take a look.

  3. Hi Jason, this is awesome and I am giving it a go now, as an interim solution before I buy Mettle’s plug in. I think I’m about to hit the “camera was facing forward when we shot” problem, but let’s see once I press export anyway!

  4. This is a wonderful tutorial! Thank you so much for the comprehensive and clear write-up!

    I was wondering, will this only work if you’re pointing the camera upwards? Or would it be possible to film forwards so the field of view wraps around a viewer who’s facing mostly forward? And if that is possible, what steps in your tutorial might differ?

    Sorry, super noob here, but super excited! For the first time what I’m looking to achieve is looking possible!

    • When finished working on the 360 video, then you’re ready to render the movie from After Effects.

      You can do this by opening up your final Comp, and then going to the top toolbar and selecting: Composition / Make Movie

      This will create a render job within the Render Queue.

      Then you just need to edit the ‘Output Module’ settings and also tell the file where to be rendered by editing the ‘Output To’ location. Below are some good settings for rendering a 4k movie for Youtube. (Also these compression settings are useful.)

      Render!

      Important last step. After the render is completed, then you need to add a special 360 metatag into the MP4. Youtube has created a 360 metadata insertion tool just for this. (The free tool can be found within step 2.)

      Done. The MP4 with injected 360 metadata is ready for upload to Youtube for a interactive 360 experience.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s