A couple of months ago, I found myself in the fortunate situation of shooting virtual reality footage with a prototype VR camera...in Tahiti. Needless to say, that was an insane experience that I will never ever forget. If you want to read about that experience, check out this post, but here, I'm going to lay out a few things Skye Mayring, my partner in Superswell VR, and I, learned about shooting virtual reality, because, let me tell you, it's nothing like shooting regular video.
I have spent the better portion of the last two decades working in the media industry, and a LOT of that time has been spent producing, shooting, and editing videos. Everything from Fortune 500 corporate packages to music videos to broadcast television packages and nature films, you name it. The process has become almost robotic in the sense that you know what to expect, you know how to set things up, you know what you need to do to get the material you need...basically, you've gone through the true, tried and tested formulas to the point where the whole procedure becomes quite instinctual.
Well, with VR...forget all that! Essentially, you're learning from scratch, and everything you thought you knew...throw it right out the window. Heck, it might be almost easier for someone that doesn't come from a background of video production, because they don't have to focus on re-training their brain...it'd all be new to them, eventually becoming instinctual.
With every video I've ever worked on, and anyone has ever worked on - whether it be film, television, internet, whatever - there is always a barrier between the storyteller, the production itself, and what the viewer ends up seeing on the other end. With this, fuggedabadit. That barrier completely disappears. This was no longer just finding a way to tell a story to someone, this was a way of that someone experiencing the story with you.
For the first time.
Ever.
And because of that, the approach had to be completely altered. There was definitely a learning curve to the process, and I'll lay out a few of those key things we learned. If you jump down the VR rabbit hole (and it's quite an exciting rabbit hole to jump into right now), hopefully some of them might help you save some time and sanity in your own productions.
1. Create Movement To Guide The Viewer
When most people first put the headset on, they're not sure what to do. Essentially they look right, they look left, they look ahead, sometimes they look up, but it doesn't occur to them that they can literally look EVERYWHERE, all around them. And when they finally realize this, that's when the 'aha' moment kicks in, the jaw drops, and an expression along the lines of 'Oh wow! That's amazing!' uncontrollably oozes out of their psyche.
As 360 degree Virtual Reality filmmakers, your job is to find ways to utilize that space, otherwise, why shoot in VR? An easy way to do this is to create moment in order to guide the viewers head.
For example, while we were in Tahiti, one of the videos we shot was a motu picnic experience, in which a local guide prepares a dish of poisson cru, the national dish of Tahiti. It consists of fresh fish, citrus, vegetables, mixed with coconut water and shavings. The first couple of takes, we had him prepare the entire dish while standing in one position. Something wasn't clicking. We realized that we weren't giving the viewer a single reason to move their heads. That's when it hit us. So we moved the coconut he was going to shuck to the other end of the beach and we placed the table where he would make the final preparations 180 degrees to the right of him. This way, he'd have to walk down the beach to the left to pick up the coconut, and then back to the right to the shucking station, and then from there, he'd walk 90 degrees to the right to the table to finish the preparation. This forces the viewer to turn their head completely with him, naturally guiding them to take full advantage of the immersive view.
Another example was on the Island of Huahine, another French Polynesian island of Tahiti. This island is famous for its ancient marea temples and rich archaeological history, so we had a local tour guide there take us around the island to various locations. While he was guiding us in several scenes, we had him walk around the camera as he pointed out various landmarks and aspects of the tour. This gives the viewer the reason to turn and look at what he's pointing at.
2. Treat Your 360 Camera As You'd Like To Be Treated
While not as important in regular video, as viewers have their entire periphery as a frame of reference, nothing is more important to making your viewers feel comfortable, especially when experiencing fully immersed VR in a headset, than being able to relate to the point-of-view. Therefore, treat the 360 camera as another person. For example, if the scene is a situation in which everyone is standing up, have the camera level at an average height of a human being. If the scene is a situation in which everyone is sitting down, have the camera at the height of everyone sitting down. Imagine standing with a group and looking up at everyone if the camera is too low, or looking down on everyone if the camera is too high - it can be very disorienting - so this is super important to remember in order to help minimize any sort of unnatural sensations, greatly reducing the risk of nausea. Especially in VR, nausea is a no-no.
3. Use Sound As A Cue.
With the nature of VR being a completely immersive experience, things that were impossible before, become very possible now, including the effective use of sound as a 'visual' cue. For example, if someone is talking behind you, you can now turn around and see that person talking behind you. The surround sound experience is now enhanced by surround view. While shooting that same poisson cru preparation discussed in #1 above, we had two elements of sound that helped guide the story. One, we had live ukulele playing, and it was completely opposite of the focus of the scene. If you turned around, you'd see another local Tahitian man sitting on the picnic table in the water, playing the ukulele. Additionally, when our main guide was ready to move from one step of preparing the dish to the next, he walked through the water, roughly to a point 45 degrees away from where he was, and as he walked through the water, you hear the water splashing in the direction he is walking, prompting you to turn your head and follow him.
There are several ways to play with this new story telling technique. Imagine a horror film where someone suddenly screams at you from right behind you and you turn your head...finding yourself face to face with a monster!
4. Learn to Let Go - Remember the 'Reality' in 'Virtual Reality'
When you’re used to shooting in a 2D space, whether still photography or film, you’re used to having a certain amount of control of every element in the frame. From blocking to positioning to crew. It took quite a bit of self-control to realize that, when shooting in 360, you don’t have all that control, and once you realize that you don’t have that control, you have to learn to let go and embrace it. You can’t really predict what’s going to happen in the ‘background,’ and you have to remember that’s a good thing. We had kayakers, swimmers, people walking on the beach, doing what you might consider ordinary and unremarkable things, but those are the very things that lend to the immersive nature of the medium. It helps add a new element of approachability that can often work in your favor, because, after all, life is not perfect, and if everything was perfect, you’d lose the ‘reality’ in ‘virtual reality.’
Also, another lesson along these lines is that what may seem boring in real life, may not be so boring in VR. We were with the engineer of the Project Beyond camera we were using, and he insisted we shoot a bit of footage between a bunch of food vendors on the side of the street. We weren’t quite sure why, it made no sense as there didn't seem to be anything compelling other than the delicious aromas, but once we saw the footage, it was surprisingly engaging. We realized that because it’s such a real scene that we’re used to and know, we were immediately able to relate. At the time we weren’t focused on getting release forms as we didn’t think we’d use that footage, but, well, now I wish we had.
5. It's Not About You!
Again, remember, this is 360, so EVERYTHING will be seen. Everything. So be aware of what's in focus because you’re gonna see anyone and everyone in the frame. And with this, people’s faces will be in focus. You will need to sort out any release form situations before you get to the location. Otherwise, it will be madness trying to get everyone to sign them. Trust me. Oh, PLEASE...trust me...
6. Wait What? There IS A Front Of The Camera!??
This was a weird one to learn...as it was a lesson that wasn't apparent until post production. We were editing a scene in the interior of one of the overwater bungalows where a couple was supposed to be front and center as the focal point. Well, when I imported the clip, the boring screen of a TV was at the center of the frame. No bueno!
It should become a simple habit while shooting to use a piece of tape or a marker to learn which set of lenses you will use as the front, and always use that marker to distinguish where the main focal point of action is located, especially when shooting with a very multi-camera array like we were (17 cameras). That front will be the center of your stitch, so it’s useful to know which of the cameras is #1 and which is #16, as it makes for great practice to keep that focal point of action away from the stitching point of the vertical planes between the first camera and the last camera, which, ideally, would be behind the head when viewing in a viewer. So trying to keep the focus between cameras between cameras #8 & #9 would be ideal (or whatever the middle cameras in your array are, for example, in a 6 camera array, use lenses 3&4, and so forth). By keeping the action in the center frame of your VR array, you'll help assure that the action remains in front of the viewer in the safe zone.
Thankfully, there are now plugins for Premiere and After Effects made my Mettle where you can seamlessly shift that plane of view, which are INCREDIBLY useful and a must for anyone working in VR. Additionally, if you're using Adobe's Creative Cloud, the latest version of Premiere CC includes built in VR tools, one of which is the ability to shift the plane of view in post.
7. Clean Every Lens Every Shot
This is EXTREMELY important. Any fingerprint or smudge on one, or any part of, a lens has the very real potential of ruining an entire shot. Have lens wipes in all your pockets at all times.
8. Slate Everything!
The importance of this will become incredibly apparent come the post production phase for two distinct reasons:
A. Since you have to run and hide while shooting, if you don't have a wifi or bluetooth monitoring solution or connection, most times you are unable to see all the action and hear everything that’s happening, so to have a visual cue of each take becomes super important.
B. You will most likely have to sync audio from a separate audio device or recorder, just like in regular video or film productions, and since so many of the takes will be similar with just minor variants, you’ll have to be sure to use the right audio, primarily when shooting in situations in which the environment isn't completely controlled. You don’t want to hear a random motorboat cruise by if there wasn’t one there while shooting in 360 because, well, the viewer can turn around and see that there is no motorboat. Jus' sayin...
9. Keep It Short & Sweet
As best you can, never record long clips for a few reasons. First of all, it gets boring for the viewer after an average of around 20-30 seconds. Second of all, cameras like Project Beyond uses 17 cameras, each with their own sensor, capturing over 35 megapixels per frame, at an average of 30-60 frames per second...well...you do the math. This becomes quite the issue once we get to post production. Large files equals large amounts of data equals computer wants to poo.
10.Learn To Offline Edit!
If you've never done it before, you're going to want to learn to offline edit. Unless you have a super powerful super computer, chances are you're going to have an insanely frustrating troublesome time editing these large files on your computer. Essentially, offline editing is creating low resolution versions of the files, while retaining the exact same names and naming structure, editing with those, and then connecting to the high resolution files once you have picture locked with final edits. Lucky for Adobe Creative Cloud users, the latest version of Premiere includes an automatic function to create proxies (low resolution offline versions) that you can switch back and forth from with the high resolution original with the click of a button. This is a game changer. Seriously.
11. Take Your Sweet Time.
Let the shots ring out. Even when you thought you let the shot ring out, you haven't. Let it ring out even more. In regular video or film setting, you'll cut to the next scene once the action or dialogue is done in the prior scene because there's nothing else to see or hear. In 360, you want to give the viewer adequate time to explore the entire scene as there is much more to see.
Our first two edits, while cutting in Premiere, it seemed like we were definitely staying on shots long enough, almost too long. But when I put it in the Gear VR to review, I was longing for more time to explore the scene. I had to go back in and literally almost double the length of each shot before I was comfortable I had enough time to explore.
12. Get Your Extensions Right!
Learn the naming conventions for uploading and viewing on GearVR. I had to test and try and figure out several different ways of encoding in order for the videos to play, and on top of that, you have to be cognizant of the ending of the name (TB_BT_3DV, etc) to play in different viewers, apps, and programs. There's a great post on the Oculus support pages that will help with this: https://goo.gl/y4NzYZ
13. Get Some Mettle Plugins
Just do it. Trust me. A must. Just do it. Mettle. Now. Here.
14. Be Prepared To Show Off!
Take a GearVR or some viewer with you everywhere you go because everybody thinks VR is amazing. Everybody. From the grandma in Tahiti to the GM of a Bora Bora resort to a child in Los Angeles. My mom, dentist, friends, and random strangers in random coffee shops. Everyone. And if you're in the business of VR, even better, as it sells itself. People see it, and their neurons immediately fire.
With all that being said, the industry and the technologies are moving at such an incredible rate, that new stuff is available literally every single minute. It's almost impossible to keep up with. But that's a big part of the excitement of the VR space right now...there is so much experimentation and so much to figure out, as there are no real precedents set just yet, which leaves room for a LOT of us to get crazy.
Well, that's it for now. I'm learning more with each project and with each day and perhaps I'll update this post with more or write a new post later, but these tips will hopefully get you on the right path with your VR project. And if you have any questions, feel free to contact me, I'll do my best to answer any questions.
All the screenshots you see throughout this post are from the footage we shot in Tahiti. If you want to check out those videos, keep an eye on SuperswellVR. We'll be posting them there soon. Very very soon...
And if you so wish, keep up with the rest of my what-have-yooz and thinga-what-nots, find me at:
instagram: @wasimofnazareth
ello: @wasimofnazareth
google+: +WasimMuklashy
And if you enjoy these posts, please feel free to subscribe to my mailing list at top right of this page, and I'll deliver each post to you as I publish them, and if you support the kind of work I’m doing and the things I’m sharing, please consider ‘subscribing to me’ on Patreon.com. Yup, subscribing to ‘me.’ You can even set the price. Literally, $1 per month if you want. No joke, your little helps me a LOT.