The Reefs of Belize – Fulldome & VR Short

In January 2016, Allan Adams and Keith Ellenbogen took a group of MIT students scuba diving in Belize as part of a college course on underwater conservation photography. Coral reefs worldwide are deteriorating due to changes in our climate and so it’s important to document both the beauty of our oceans and what’s happening to them. Capturing this moment in time is important for future generations to learn from, be immersed in, and be inspired from.

Keith Ellenbogen is an acclaimed underwater photographer and videographer who focuses on environmental conservation. Ellenbogen documents marine life to showcase its beauty and to elicit an emotional connection to the underwater world. He aims to inspire social change and action toward protecting the marine environment.

Over the past few years, Ellenbogen has collaborated with MIT theoretical physicist Allan Adams who is focuses on the intersection of art, science, and cutting-edge technology. During his residency, they worked with Edgerton Center Associate Director Jim Bales to explore new high-speed photography and other underwater imaging techniques. They also developed an ‘Underwater Conservation Photography’ course taught at MIT and challenged students to push technical and aesthetic boundaries in the pursuit of compelling images of marine conservation.


Domemasters Freely Available

  • The Reefs of Belize, Still Shot 1, and Still Shot 2 are available for planetarium use. Please contact me to obtain a download link.
  • 4k domemaster frames, 30fps, stereo audio
  • 2k MOV or MP4
  • 1k MOV or MP4

Terms: permission to freely screen to the public in planetariums as you see fit. You must screen the short in full and unedited. Not to be used in other shows without permission.


Behind the Scenes

Allan and Keith approached the Museum of Science’s planetarium team because of its expertise in 360° video. It was a perfect meeting of minds and collaboration started immediately to fully test the equipment and plan for the dive. 360° video is very challenging to begin with and it’s even more difficult underwater, so I’ve documented some of the important things we learned.

From the very beginning we were aiming to use the immersive scuba footage for a live lecture in the planetarium. It was only after throwing this event that we realized other planetariums and the VR community might be interested. We should note that this was our first underwater project and we have learned a ton along the way. So some of the shots are a little shaky, lighting isn’t ideal, footage contrasty, and no underwater audio was recorded. Shooting underwater is difficult and you simply cannot improvise with shot techniques in the same way as a 360° shoot on land. But that’s hindsight and so we decided to share the best shots edited into a short film, even if it doesn’t reach the high bar we’ve set for ourselves. Because what’s the use of it keeping it private? We are proud of this project and hope it can inspire others to remember the hidden beauty of the ocean.

360Abyss-RigBut you might be wondering, how do you capture underwater 360° video? It’s possible through the use of 6 GoPro cameras and the specially designed 360Rize 360Abyss scuba rig. Since it’s going underwater, it needs to be watertight and also use domes for the camera porthole due to water refraction.

Prior to the expedition, we needed to test the 360° camera rig underwater and preferably not just in an old bucket. Luckily Keith is good friends with the New England Aquarium and so our first tests were within the Giant Ocean Tank, a gigantic cylindrical aquarium in the center of the aquarium. We were instantly excited about the results. During this time students were practicing shooting still photography within an olympic-size pool.

There are so many worrying factors when pairing scuba diving with photography. You need to keep track of oxygen levels, focus and expose your camera, be careful of sea life, keep the group together, track the boat, and the list goes on. So being prepared mentally, physically, and technically is important.


Glover’s Reef Research Station

Their expedition took them to the Glover’s Reef Research Station in Belize, which is operated by the Wildlife Conservation Society. They worked with the research station staff to carefully dive in the conserved coral reefs and shoot underwater photography. The WCS mission is to save wildlife and wild places worldwide through science, conservation action, education, and inspiring people to value nature. They envision a world where wildlife thrives in healthy lands and seas, valued by societies that embrace and benefit from the diversity and integrity of life on earth.

Glover’s Reef is a partially submerged atoll located off the southern coast of Belize, approximately 45km from the mainland. It forms part of the outermost boundary of the Belize Barrier Reef. It harbors one of the greatest diversity of reef types in the western Caribbean. A large spawning site for the endangered Nassau grouper is located at the northeastern end of the atoll. It has been identified as one of only two viable sites remaining for the species, of nine originally known locations. In 2002, it was declared a special marine reserve, permanently closed to fishing.


Credits

A co-production by the Massachusetts Institute of Technology (MIT) and the Charles Hayden Planetarium, Museum of Science, Boston

Underwater 360º Photography:
Keith Ellenbogen – MIT CAST Visiting Artist / Assistant Professor Photography SUNY/FIT
Allan Adams, Associate Professor – MIT Dept of Physics

Post-Production:
Stitched and edited by Jason Fletcher
Charles Hayden Planetarium, Museum of Science

Special Thanks:
The MIT Edgerton Center and Jim Bales
The Roy Little Fund at MIT
The MIT Alumni Class Fund
Wildlife Conservation Society, Glover’s Reef Research Station, Belize


Screenings

Conferences & Festivals
— Further Fest 2017 (Nashville, TN)

International Planetariums
— ESO Supernova Planetarium (Garching, Germany)
— Portable Planetarium (Karnataka, India)
— Portable Planetarium (Huelva, Spain)
— Melbourne Planetarium, Scienceworks (Melbourne, Australia)
— CCAF Observatory & Planetarium (Farra d’Isonzo, Italy)
— Pro Planetario Movel (Curitiba, Brazil)

USA Planetariums
— Museum of Science, Charles Hayden Planetarium (Boston, MA)
— Slippery Rock University Planetarium (Slippery Rock, PA)
— Ho Tung Visualization Laboratory & Planetarium (Hamilton, NY)
— Collier County Public Schools, Portable Planetarium (Naples, FL)
— The College of Southern Nevada, Planetarium (Las Vegas, NV)
— Dreyfuss Planetarium, Newark Museum (Newark, NJ)

Distributors
— ESO Fulldome Archive
— Dome Club (UK)
— British Fulldome Institute
— Kosmos Scientific de México
— British Fulldome Institute

Fisheye to Spherical Conversion using After Effects

fisheye-to-spherical-conversion-of-kodak-pixpro-sp360-4k-footage

Lately I’ve been shooting with the Kodak PIXPRO SP360 4k camera for both VR and the planetarium dome. Shooting with dual cameras is great for capturing 360°, but shooting with a single camera is sometimes easier since the footage doesn’t need to be stitched. Also a single camera captures 235° which is a surprisingly huge FOV. Yet it’s necessary to warp the footage from fisheye to spherical so that it can be experienced in VR or Youtube 360.

The ‘Pixpro SP360 4k’ software is actually capable of warping a single camera from fisheye to spherical. But it’s not intuitive (here is a tutorial) and the Kodak software can only export footage to MP4… And seeing as how the raw camera footage is an already heavily compressed MP4, I wasn’t thrilled about this added lossy step. So I figured out a simple technqiue.

If you’re wondering… the terms Spherical, Equirectangular, and LatLong refer to the EXACT same thing.


Tutorial using After Effects without plugins

— This technique is a hack and the warping isn’t ideal for all occasions. Yet it really depends on whether your fisheye lens is equidistant or equisolid angle. Equidistant fisheye lenses can get a near perfect conversion using this technique. But equisolid angle fisheye lenses are unique and therefore this technique cannot provide an accurate conversion. The technique still works for equisolid angle fisheye lenses but parts of the image will look slightly stretched or squashed vertically when viewed in VR or Youtube 360. For instance, the SP360 4k camera has an equisolid angle fisheye lens and yet it’s the camera I used in this tutorial and achieved decent results. On that note, I actually haven’t been able to confirm from any official Kodak specs that the SP360 4k camera indeed uses a equisolid angle fisheye lens, but it seems pretty obvious when comparing renders from this After Effects technique against the ‘Pixpro SP360 4k’ warping software.
— You cannot use gaussian blur, sharpen, or such effects since they would create very obvious seams when viewed in VR or Youtube 360. But you could instead use the Skybox 360 Post FX since they are seamless VR effects.
— If you need to adjust the horizon level, then you’ll need to instead use the RE:Lens plugin which provides much better controls and proper conversion tools.

DOWNLOAD: Fisheye FOV Guide
required to complete tutorial

convert-fisheye-to-spherical-using-after-effects-no-plugins




Spherical to Fisheye Conversion

Or if you need to convert spherical to fisheye (so it can be watched in a dome) then the same process can be applied in reverse. But you’ll need to scale up the footage to crop out some of the unwanted FOV and unfortunately it’s a lossy conversion since it’s being uprezzed. Also you cannot change the FOV accurately, it’s just a very basic conversion. Although a perfect conversion can be achieved using a plugin from this list.

NASA Grant – Jackson Hole Finalist – Boston Globe Article

NASA Grant: Planetarium Show

A while back we applied for a NASA grant and we’ve been patiently waiting to hear the results… And we are thrilled to announce that we have been selected! We are one of nine organizations chosen from 73 applications through a peer-reviewed process.

The two-year NASA grant will enable us to capture the excitement of the next generation’s moonshot — the human journey to Mars and back. We will create two educational experiences that will bring to life the engineering skills and team spirit fueling this historic pioneering endeavor. So stay tuned…

AS12-49-7278-webImage Source: NASA Project Apollo Archive


Jackson Hole Science Media Awards 2016

The Jackson Hole Science Media Awards has announced finalists in 22 categories spanning content, program, and craft in the science media field. This competition celebrates the world’s most effective science storytellers and stories. This year’s competition saw more than 500 entries competing for 25 special awards. More than 115 international judges screened an aggregated 2,100 hours in order to select the finalists.

I am pleased to announce that our show From Dream to Discovery: Inside NASA is a finalist within the Immersive Cinema category.


Boston Globe: Write-up on SubSpace Project

Over the last few months we have been focusing on SubSpace Project. This is an experimental playground for developing fresh and original social experiences for adults. Art, science, and technology collide to create a new wave of intelligent nightlife that is provocative and one-of-a-kind. From musical tributes in the Planetarium to performance art installations and beyond, SubSpace Project is an ever-evolving laboratory for Boston’s most intriguing and immersive experiences.

Our current series of experiences is dedicated to amazing musicians such as David Bowie, Prince, Björk, Beyoncé, Radiohead, Tom Waits, and Lady Gaga. And our 3D animators have been creating dome visuals with the aim of making an intense music show experience.

So it was a pleasant surprise when The Boston Globe was interested in writing an article about the work we’ve been doing. Check out the online article written by Sophie Haigney.

Globe-Newspaper-Scan_Charles-Hayden-Planetarium

“I am in a tunnel of blue light that is also sort of a hurricane. I am approaching its mouth, or its eye. Fish spiral toward me. “I thrive best hermit style, with a beard and a pipe and a parrot on each side,” Björk croons, in the background. Then I am in a watery place where silver orbs are bouncing.”

“Actually, I am in the Charles Hayden Planetarium at the Museum of Science. This show is part of their SubSpace Project, which features dynamic visual interpretations of music. Harnessing the planetarium’s immersive dome-theater space and highly advanced audiovisual technology — usually used for shows like “Moons: Worlds of Mystery” — the staff is now paying tribute to musicians. In this case, it’s Björk, the avant-garde Icelandic singer who has inspired a cult following and even a mid-career survey at New York’s Museum of Modern Art.”

Interviews at IMERSA 2016 – Recent Challenges

It’s been fascinating to see IMERSA evolve and mature over the last few years. And since things are moving so fast, I wanted to document the challenges being faced in the immersive community with a series of interviews.

Last year at IMERSA there were nervous murmurings of VR, but this year there is clear excitement. It’s particularly interesting to see fulldome producers realize that they already possess the tools, skills, and ultra high resolution workflows to create polished VR experiences.

IMERSA 2016: View Presentation Recordings

Select presentations available on the IMERSA Vimeo page. Below are my favs:
How Are Museums & Educators Using VR-AR Today / The VR and AR Explosion
Immersion Expanding: New Opportunities for Immersive Experiences
Challenges and Strategies for Producers
The Future of Immersion
Ambisonics Sound Technology
What’s Never Been Seen: Successful Visualizing for Fulldome Storytelling
Shooting 360 Trials and Successes
Proven Methods for a Faster Render
Visual Immersion for Greater Learning Gains in Digital Domes
Let’s Play: Using Games to Entertain and Educate Audiences in the Planetarium
Real Developments in Virtual Reality

Interviewees
— Jenny Carden / Zenka.org
— Greg Downing / xRez Studio
— Troy Whitmer / Sky-Skan
— Jay Heinz / Morehead Planetarium
— David Merrell / Clark Planetarium
— Ken Ackerman / California Academy of Sciences
— Dan Neafus / Denver Museum of Nature and Science
— Orion McCaw / Roundhouse Productions
— Mark Petersen / Loch Ness Productions
— Jay Lamm / Louisiana Art & Science Museum
— Annette Sotheran-Barnett / Sky-Skan

The Dome Dialogues – Andrew Hazelden

An interview with the man that needs no introduction! Andrew Hazelden and I discuss the many vital production tools that he has been creating for fulldome and VR. We discuss in-depth tools such as: Domemaster3D, PlayblastVR, RocketComp, Domemaster Fusion Macros.

Chapters
0m 6s – Intro
3m 53s – RocketComp
5m 58s – PlayblastVR
17m 8s – Andrew’s History
20m 14s – Domemaster3D
41m 8s – Domemaster Fusion Macros
1h 27m 41s – Maxwell Render Toolbox

Bio
Andrew Hazelden is a visual effects artist and co-founder of Dover Studios. He regularly develops tools, tutorials, and documentation for VR/fulldome production, photography, visual effects, and electronics. He has passion for sharing knowledge and also enjoys writing about hobby experiments he does on the weekends and the tools he uses everyday. A few examples of his wide range of interests include building an underwater ROV, flying a model airplane, compiling a mental ray shader, creating a time-lapse video, or doing stereoscopic 3D photography.

Utilities: Converting Timecode & Frames – Estimate Job Render Time on a Farm

All too often I need to precisely convert between timecode and frames. Or I’ll want a rough time estimate of a render job on the farm. Yet I needed a utility that could allow for intuitive interactivity, GUI creation, and be instantly editable. So I’m using a simple open source solution: Pure Data

Pure Data (aka Pd) is a realtime graphical programming environment which is mainly used in live music performances. But its realtime aspect makes it ideal for my needs. So with some careful planning I applied the math nodes to create tools which automate the conversion process. Basically I’ve created a calculator tailored for animators and filmmakers.

DOWNLOAD
collection of Pd patches

Instructions
— Requires Pure Data to be installed. (supports: Windows, Mac, Linux)
— It’s easy to use. Just click and drag on the number boxes to input your timecode or frames.


Convert Timecode into Frames /// Convert Frames into Timecode
— Assumes you’re working at 30fps.
convert-frames-timecode


Calculate Shot Duration: Input as Timecode or Frames
— Especially useful when adjusting timings between two versions of the animatic and need to figure out the exact amount of time added or removed.
— Assumes you’re working at 30fps.

convert-duration-frames-timecode


Estimate Job Render Time on a Farm
— Most useful for estimating the min/max time untill the render is complete. Which is especially important if you’re worried about bottlenecking your farm and need to prioritize for deadlines.
estimate-render-time


How Does it Work? Lets see the Breakdown
— All the source code is included (within subpatches) and can be edited. So if you’re instead working at 60fps, then you can alter it to your needs.

IMERSA Summit 2016: Presentations We’re Giving

imersa-logo-squareWe are going to be at the upcoming IMERSA Summit and sharing several presentations. With so much that’s been happening lately in the immersive community, it’s bound to be an exciting conference this year. David, Heather, and I will each be on different panels and giving presentations. Hope you can check out what we’ve been working on! More info below.


Panel: Challenges and Strategies for Producers
Thursday, March 17 at 10:45 AM
Update: Watch a video recording of this talk
A team of panelists will discuss questions of importance to producers: What are the biggest obstacles to creating content for immersive media, specifically fulldome? Our panel of producers, with lots of help from the audience, will consider the answers and propose solutions. We want to hear from you! In this lively audience led discussion, we will explore your greatest successes and failures in creating and experiencing immersive media.

— Moderator: David Rabkin (Museum of Science)
— Panelists: Robin Sip (Mirage3D), Annette Sotheran-Barnett (Sky-Skan), Mark Webb (Adler Planetarium), Chris Lawes (Fulldome.pro)


What’s Never Been Seen – Successful Visualizing for Fulldome Storytelling
Thursday, March 17 at 04:15 PM
Update: Watch a video recording of this talk
The script is written. The storytelling is effective. But there are calls for visuals of things that don’t exist yet, or real data representations that have never been visualized. Storytelling for immersive fulldome environments has required producers to take the idea of the “artist concept” to new levels. How do we ensure a successful pipeline between the left-brain expert supplying the input and the right-brain creative implementing the visuals to achieve the desired⎯but most important, accurately told story for the educational goals? Examples of what works… and sometimes what doesn’t.

— Presented by Tom Casey (Home Run Pictures), Jason Fletcher (Museum of Science), Carolyn Sumners (Houston Museum of Natural Science)


Let’s Play: Using Games to Entertain and Educate Audiences in the Planetarium
Saturday, March 19 at 10:15 AM
Update: Watch a video recording of this talk
Experiments with gaming recently done by the Charles Hayden Planetarium at the Museum of Science in Boston, will be presented as well as some plans for the future. We’ll talk about the partnerships we’ve made, the logistics of planning for these events, and some of the technology behind it all. We welcome discussion on what other planetariums have tried, what has worked, and, of course, what hasn’t.

— Presented by Heather Fairweather (Museum of Science)


IMERSA Summit 2016: View Entire Agenda

davidrabkin-jasonfletcher-heatherfairweather
David Rabkin (Planetarium Director)
Jason Fletcher (Science Visualizer)
Heather Fairweather (Science Visualizer)

IMERSA Summit 2016: Announcing Slack

slack-in-the-charles-hayden-planetarium-cropped

Recently I’ve been brainstorming how to better connect the IMERSA crowd. Part of what makes going to the summit so great is the random people you meet. And so we are experimenting with how we can better connect people of similar interests. We are hoping that smartphones can be part of the equation.

So we are using “Slack” to continue the amazing conversations that happen at the IMERSA Summit. It’s both an icebreaker into the community and way for you to keep your ear to the ground. And also, it’s free.

To join the IMERSA Slack group, please request an invitation.


Slack-logo-transparent
After you’ve registered, then be sure to install the Slack app on your smartphone (iPhone or Android). But you can also use Slack on any web browser. Then join some channels relevant to your interests and start by sharing a recent project you’ve worked on.

IMERSA.slack.com

Upcoming Special Events in the Planetarium


EinsteinsPlayground-ASlowerSpeedofLight
Image Source: A Slower Speed of Light

Einstein’s Playground

— Thursday, February 11 /// 7:15pm
— Admission $10
— Gerd Kortemeyer, PhD, associate professor of physics at Michigan State University

Have you ever wanted to experience the complete distortion of time and space as we know it? The Charles Hayden Planetarium has partnered with the MIT Game Lab to immerse you in a virtual special relativity playground where you’ll witness the laws of physics in a completely new way. Using the power of video games, we’ll turn Einstein’s most famous theory from an abstract concept into something you can encounter yourself right here at the Museum of Science. Experience the effects of movement, time, and space as you’ve never been able to before!

Tickets on sale beginning January 28 /// (January 26 for Museum members)



AWorldUnderwater-TheReefsofBelize-KeithEllenbogen_5309952
Image Source: Keith Ellenbogen

A World Underwater: The Reefs of Belize

— Thursday, March 24 / 7:00pm
— Admission Free
— Keith Ellenbogen, award-winning underwater photographer and 2015-16 CAST Visiting Artist at MIT | Allan Adams, PhD, theoretical physicist, associate professor of physics and member of the Creative Art Council at MIT

Take an underwater journey to Glover’s Reef Research Station in Belize and immerse yourself in coral reefs! With images and cutting-edge immersive video captured during their January 2016 expedition, Keith and Allan will tell the story of the Mesoamerican reef ecosystem, the researchers working hard to conserve it, and the innovative MIT course behind the expedition in which students from across the institute (chemists, civil engineers, historians, physicists, and poets) learned the art, technique, and technology of underwater conservation photography. Under the Planetarium’s fulldome expanse, experience the thrills, challenges, and serendipity of wildlife photography and explore the role of visual culture as a catalyst for positive social change on our tiny blue planet.

Advance registration beginning March 10 /// (March 8 for Museum members)



StoriesUnderTheStars-Ari-Daniel-Hubble-2013-17-a-large_web-cropped
Image Source: NASA, ESA, CXC and the University of Potsdam, JPL-Caltech, and STScI

Stories Under the Stars

— Wednesday, April 20 / 7:30pm & 9:00pm
— Admission $12
— Ari Daniel, science reporter

Come to the Charles Hayden Planetarium for an evening of live storytelling, radio, and music under the stars. You’ll hear true stories, both personal and inspired by science, that explore the theme of “Light in the Dark,” all unfolding beneath the canopy of our cosmos. Join the search for light during the earliest moments of your life and from the outer reaches of our universe to the inner reaches of the human heart.

Tickets on sale beginning January 28 /// (January 26 for Museum members)
Hosted by science reporter Ari Daniel and co-produced by Ari and the Museum of Science as part of the Cambridge Science Festival.



SpaceStation-ISSCupola-cropped
Image Source: NASA

Space Station

— Thursday, April 21 / 7:30pm
— Admission $10
— Jared Sorensen, game designer

You wake up inside the cramped confines of a cryosleep chamber. You feel weak and dizzy from a prolonged period in cryonic suspension. What will you do next? Join game designer Jared Sorensen and the Charles Hayden Planetarium team as we break new ground in the Planetarium dome. Inspired by the text-parsing games of the ’80s, Space Station allows the entire audience to play a single character trying to survive a dangerous situation… in space! Give commands, explore rooms, examine objects, and try to escape the Space Station, if you can!

Check out the Parsely website for more information about their series of text-based adventure games.

Tickets on sale beginning January 28 /// (January 26 for Museum members)
Part of the Cambridge Science Festival.



CosmicLoops-IanEthanCase
Image Source: David Rabkin

Cosmic Loops

— Wednesday, May 18 / 7:15pm
— Admission $15
— Ian Ethan Case, acoustic double-neck guitars, fretless guitar, live looping | Stephanie Case, live sound design | Bertram Lehmann, percussion | Jeff Willet, gongs and percussion

As you soar through nebulas, galaxies, and star systems in the immersive space under the dome of the Charles Hayden Planetarium, live music with simple beginnings builds layer upon layer into an intricate universe of musical loops created by masters of an evocative style. Acoustic double-neck guitarist Ian Ethan fluidly combines a staggering variety of self-invented playing techniques necessitated by his multilayered compositions, further expanded using real-time live looping technology. Indulge in this rare quartet performance in which gongs and exotic percussion instruments from around the world take Ian’s latest compositions into new dimensions, with the Planetarium team’s transcendent visions overhead.

Tickets on sale beginning January 28 /// (January 26 for Museum members)

Using Real Pluto Imagery – From Dream to Discovery: Inside NASA

In producing From Dream to Discovery: Inside NASA we made the exciting but perilous decision to include the New Horizons mission within our story. So we made an early bet that the mission would be a success…

As you well know, New Horizons has given us an amazing close-up look at Pluto. And so we are excited to announce that we have updated the show to include the latest real images of Pluto and Charon!