Lately I’ve been shooting with the Kodak PIXPRO SP360 4k camera for both VR and the planetarium dome. Shooting with dual cameras is great for capturing 360°, but shooting with a single camera is sometimes easier since the footage doesn’t need to be stitched. Also a single camera captures 235° which is a surprisingly huge FOV. Yet it’s necessary to warp the footage from fisheye to spherical so that it can be experienced in VR or Youtube 360.
The ‘Pixpro SP360 4k’ software is actually capable of warping a single camera from fisheye to spherical. But it’s not intuitive (here is a tutorial) and the Kodak software can only export footage to MP4… And seeing as how the raw camera footage is an already heavily compressed MP4, I wasn’t thrilled about this added lossy step. So I figured out a simple technqiue.
If you’re wondering… the terms Spherical, Equirectangular, and LatLong refer to the EXACT same thing.
Tutorial using After Effects CS6 without plugins
— This technique is a hack and the warping isn’t ideal for all occasions. Yet it really depends on whether your fisheye lens is equidistant or equisolid angle. Equidistant fisheye lenses can get a near perfect conversion using this technique. But equisolid angle fisheye lenses are unique and therefore this technique cannot provide an accurate conversion. The technique still works for equisolid angle fisheye lenses but parts of the image will look slightly stretched or squashed vertically when viewed in VR or Youtube 360. For instance, the SP360 4k camera has an equisolid angle fisheye lens and yet it’s the camera I used in this tutorial and achieved decent results. On that note, I actually haven’t been able to confirm from any official Kodak specs that the SP360 4k camera indeed uses a equisolid angle fisheye lens, but it seems pretty obvious when comparing renders from this After Effects technique against the ‘Pixpro SP360 4k’ warping software.
— You cannot use gaussian blur, sharpen, or such effects since they would create very obvious seams when viewed in VR or Youtube 360. But you could instead use the Skybox 360 Post FX since they are seamless VR effects.
— If you need to adjust the horizon level, then you’ll need to instead use the RE:Lens plugin or Skybox Studio V2 which provides much better controls and proper conversion tools.
DOWNLOAD: Fisheye FOV Guide
required to complete tutorial
Spherical to Fisheye Conversion
Or if you need to convert spherical to fisheye (so it can be watched in a dome) then the same process can be applied in reverse. But you’ll need to scale up the footage to crop out some of the unwanted FOV and unfortunately it’s a lossy conversion since it’s being uprezzed. Also you cannot change the FOV accurately, it’s just a very basic conversion. Although a perfect conversion can be achieved using a plugin from this list.
Update: After Effects – VR Tools
The latest version of After Effects now has VR tools built-in. It’s simply to work with and also gives you much better controls. I don’t have the latest version installed, but the basic setup goes like this:
— Apply the ‘VR Converter’ effect to your fisheye comp
— Input: Fisheye (FullDome)
— Output: Equirectangular 2:1
This tutorial is phenomenal! One question- your footage is captured pointing straight up, any way to apply these same functions to a video capture pointing forward?
Good point! If you’re shooting with a fisheye camera pointing forward, then you would want the 360 footage also be pointing forward and not straight up.
The tutorial above can take any fisheye footage, no matter what orientation it was originally shot, and then convert it to spherical. But the resulting 360 video will always have the fisheye footage oriented straight up. This technique is a hack since we are tricking After Effects into creating a 360 video.
This is frustrating because we are so close! We already have the footage converted to spherical and now we simply need to level the horizon. But there isn’t a way to achieve this without using a custom plugin and they are not cheap. (SkyBox Rotate Sphere, 360VR Reorient Sphere, GoPro VR Horizon)
Honestly you are much better off using the RE:Lens plugin. Not only will it convert fisheye footage to spherical, but it will also allow you to level the horizon. Below are some example screenshots using a demo of RE:Lens.
Hi Jason!
I’ve been trying to edit some footage of my Samsung Gear 360 following your tutorial but I got stuck in step 3 since the result of the conversion makes my footage looks really weird… It doesn’t look like your example at all but totally deformed with some “arcs” in the upper part (I wish I could post a screenshot here so you could see what I mean).
Any idea how can I correct this? I’ve tried plenty of times with different parameters but nothing seems to work 😦
Would be great if you could give me a hand with this.
Thanks in advance!
Hi Lucia! I’m not quite sure what problem you’re experiencing. Maybe you didn’t pre-comp the footage layer before applying the Polar Coordinates effect. Why don’t you upload a screenshot to imgur.com and then I can take a look.
Hi Jason, this is awesome and I am giving it a go now, as an interim solution before I buy Mettle’s plug in. I think I’m about to hit the “camera was facing forward when we shot” problem, but let’s see once I press export anyway!
Check out the RE:Lens plugin. It allows you to change the camera orientation to match the way it was actually shot. http://revisionfx.com/products/relens/
Here are some example screenshots using the RE:Lens plugin.
https://thefulldomeblog.com/2017/01/24/fisheye-to-equirectangular-conversion-using-after-effects/#comment-6705
Thanks man, I’m definitely on this march now. Do you know if Skybox Studio V2 does the same? I can’t nail down whether it handles single camera wide angle.
Ah interesting, I didn’t realize that Skybox Studio V2 added the ability to convert fisheye footage to equirectangular. It also allows to you to change the camera orientation to match the way it was actually shot and input the FOV of the actual fisheye lens. Here is a video demo of the feature.
This is a wonderful tutorial! Thank you so much for the comprehensive and clear write-up!
I was wondering, will this only work if you’re pointing the camera upwards? Or would it be possible to film forwards so the field of view wraps around a viewer who’s facing mostly forward? And if that is possible, what steps in your tutorial might differ?
Sorry, super noob here, but super excited! For the first time what I’m looking to achieve is looking possible!
Sorry, ignore me. Just saw the other comments. For some reason they didn’t load until I made mine.
No problem. As mentioned above, you’ll need to use either RE:Lens or Skybox Studio V2.
Just wanted to say thanks for this – it was very helpful in my attempts to manually stitch shots from my Samsung Gear 360 (2017) where I was finding other methods were falling short. It was the note above from Jason Fletcher (https://thefulldomeblog.com/2017/01/24/fisheye-to-equirectangular-conversion-using-after-effects/#comment-6866) that got me 99% of the way there; from that point it was simple.
Much appreciated!
Great to hear the info was helpful! Cheers.
This was great!!!!! What do you do after step 13 to make it into an actual 360 video for Youtube?
When finished working on the 360 video, then you’re ready to render the movie from After Effects.
You can do this by opening up your final Comp, and then going to the top toolbar and selecting: Composition / Make Movie
This will create a render job within the Render Queue.
Then you just need to edit the ‘Output Module’ settings and also tell the file where to be rendered by editing the ‘Output To’ location. Below are some good settings for rendering a 4k movie for Youtube. (Also these compression settings are useful.)
Render!
Important last step. After the render is completed, then you need to add a special 360 metatag into the MP4. Youtube has created a 360 metadata insertion tool just for this. (The free tool can be found within step 2.)
Done. The MP4 with injected 360 metadata is ready for upload to Youtube for a interactive 360 experience.