Posts

Getting the Most Out of Your 360 Degree Video at Live Events

Georgia Tech’s TaQuon Marshall coming right at you for his third touchdown of
the game. Be sure to spin this 360 degree video around to see the
fans in Mercedes Benz Stadium going nuts!

The Chick-fil-A Kickoff Game between Tennessee and Georgia Tech is officially in the books. And our Foundry 45 team was there to capture all of the highlights in 360 degree video for the Georgia Tech Athletics Department.

The second game at Atlanta’s new Mercedes Benz Stadium gave fans what they paid for including a fourth quarter comeback that turned into a double-overtime thriller! The atmosphere inside the sold-out stadium was electric, with most of the fans experiencing for the first time one of 13 truly game-changing stadiums built in the last two decades.

Over the years, our team has captured some truly amazing sporting content for Georgia Tech (check out our Georgia Tech Case Study for more info). We have used footage across multiple divisions but mainly to elevate the university’s student recruiting efforts. Some highlights include the “Miracle on Techwood Drive” in 2015 when the Yellow Jackets miraculously blocked a kick and returned for a touchdown against the Florida State Seminoles.

WSBTV News Story on the GT Football VR Experience

Or, earlier this year when Georgia Tech’s basketball team stunned Notre Dame with a last-second layup. Check out this related interview on 11alive’s Atlanta Tech Edge.

While some of this can be chalked up to “being at the right place at the right time”, it’s with proper planning and a whole lot of coordination that we were able to record many of these amazing game moments.

Below we’ve compiled 5 things you can do to get the most out of your live 360 degree video event session.

  1. Location, Location, Location. Work closely with your VR team to scout multiple filming locations in advance. If shooting with one camera, find a suitable spot so that VR viewers feel closer to the action than those in attendance.
  2. Be flexible, and stay mobile. While stationary cameras are essential, have team members use mobile video recording setups as well. It’ll pay dividends in post-production and will also result in more B-roll footage.
  3. Pre-event, post-event, (locker room). If you’re recording a sports game, capture the pep-rally, stay for the post-game party (in case that happens), and check with your client if you can get permission to set up a 360 camera in the team’s locker room for the pre-game pep talk.
  4. Check, double-check, and triple-check your equipment. You want to be absolutely sure that your cameras are ready to capture the potential play of the season. This includes everything from back-up batteries, charging stations and equipment tests pre-event.
  5. It’s all about the story. Your ultimate goal is to tell a compelling story that moves beyond the traditional experience of going to the stadium. Having a detailed shot list in advance and working with your VR team to come up with mandatory items and extras keeps everyone focused and ensures you get the right footage.

Keeping these points in mind will put you in a much better position to create an engaging and memorable 360-video experience for viewers. And if you want to know more about how to best distribute 360-video content, check out one of our previous posts here.

ABOUT FOUNDRY 45

Foundry 45 is The Business VR Leader. We create immersive Virtual Reality (VR) experiences for mobile platforms including HTC Vive, Oculus Rift, Samsung Gear VR, iOS Cardboard Headsets, Android Cardboard Headsets. Foundry 45 helps businesses step into their brand story by using dynamic 360-degree video and VR app creation. Our marketing and recruiting approach captures customer experience desires and emphasize product value. We also partner with existing video production studios, advertising agencies, and other content creators to serve as their VR technical consultants and developer shop.

To learn more about Foundry 45, visit us at www.foundry45.com.

For Immersive VR inquiries, contact Managing Partner:

Dave Beck, dave@foundry45.com
Follow Us!

Linkedin Twitter Facebook Instagram

360 Video Camera Comparisons: GoPro, Ricoh Theta, Gear 360, Kodak PixPro SP360 4k

At Foundry 45, many of the Virtual Reality apps we develop rely on 360 video. While we directly shoot 360 footage for some projects, other clients ask for suggestions on 360 video cameras they can use to shoot video themselves. This blog post compares 4 popular 360 rigs, on the cheaper end of the spectrum, capable of recording spherical 360 video (both vertical and horizontal).

The video below shows camera output comparisons as well as a lot of our learnings from using the different rigs. You can also directly download the 4k 360 comparison mp4 (2.5 GB!) to view the video on Gear VR headsets.

Overall, the GoPro ball provides the best output resolution, but it comes at the expense of time and money. The GoPro rig not only costs the most, but also requires a lot of extra hardware, a decent computer, expensive stitching software, and time to learn the more sophisticated stitching. Also, with seven independent cameras, there are more things that can go wrong, like a single battery dying early, an error on one of the SD cards, or one of the cameras overheating.

While certain specific use cases will alter this advice, our general recommendation is the Kodak SP360 4ks. These have better image quality than the Gear 360, especially in lower light (not outside). However, the Gear 360 has a lot of advantages over the Kodaks. It’s less than half the price, nearly 4k, incredibly easy to use (just hit “save” in the phone app), and lets you preview a full 360 from the app. And since the cameras are perfectly sync’d, you don’t run into stitching issues with moving scenes.

With either the Gear360 or Kodaks, it’s easy to orient the stitch line away from the action, avoiding some of the more difficult stitching issues entirely.

360 Video Camera Comparison Chart

360_video_cameras_GoPro_Kodak_pixpro_sp360_gear360_ricoh_theta

Overheating and Battery Life

Sadly, we’ve had the GoPro rig, Kodak PixPros and even the Gear 360 overheat during shoots. In this regard, the Ricoh Theta seems the most reliable in our experience. If shooting with any of these rigs, we highly recommend at least one backup camera if the shoot is important. And, if at all possible, plan to only record for 5 min at a time so the cameras have time to cool off between takes. While these rigs theoretically have enough battery life for almost an hour of shooting, we’ve had fully charged GoPro batteries die in 15-20 minutes.

Parallax and Stitch Lines

Parallax is when the scene looks different from one camera to the next due to the cameras being at different locations. There is no way the differences can be blended together to recover from this, and the effect gets worse as objects get closer to the cameras.

Parallax effect causes differences between cameras. Credit bucknell.edu

There are parallax effects at 8-10 ft for the GoPros, Kodak PixPros and Gear 360, but there is only one line to worry about with the Kodaks and Gear 360. Even if you have issues with the one line, you can usually position the camera so that nothing important is on that line. For instance, have one camera face the performer, and the other face the audience.

The small parallax issues on the Ricoh Theta are its best feature. There are almost no issues beyond 3 feet away since the cameras are so close to each other.

You’ll see stitch lines from this parallax effect, and also from a lack of synchronization in the source files. The GoPro and Kodak remotes don’t start the cameras at exactly the same time, and there’s no electronic syncing of the start of each frame. Stitching software can line up the footage based on audio or visual motion, and is usually pretty good. But if you have lots of motion, it’s sometimes extraordinarily difficult to get rid of the lines. We just did a test shoot with the Kodak PixPros mounted underneath the wing of a plane, and one of the cameras was vibrating slightly more than the other. This led to a very obvious stitch line that would take a lot of post production to clean up. Since the Gear 360 and Ricoh Theta S are electronically synchronized at the hardware level, there’s no need to do an imperfect sync of the resulting footage in software.

ABOUT FOUNDRY 45

Foundry 45 is The Business VR Leader. We create immersive Virtual Reality (VR) experiences for mobile platforms including HTC Vive, Oculus Rift, Samsung Gear VR, iOS Cardboard Headsets, Android Cardboard Headsets. Foundry 45 helps businesses step into their brand story by using dynamic 360-degree video and VR app creation. Our marketing and recruiting approach captures customer experience desires and emphasize product value. We also partner with existing video production studios, advertising agencies, and other content creators to serve as their VR technical consultants and developer shop.

To learn more about Foundry 45, visit us at www.foundry45.com.

For Immersive VR inquiries, contact Managing Partner:

Dave Beck, dave@foundry45.com

Follow Us!

Linkedin Twitter Facebook Instagram

How to Take Screenshots in Gear VR

What good is revolutionary technology if you can’t show it off to your friends? As kindergarten taught me, the more you share your new toy, the more fun you’ll have with it. So if you’ve recently purchased a cutting-edge Gear VR headset, you too may wish to share your experience by capturing screenshots to distribute digitally. You’ll also need screenshots to submit apps to the Oculus Store. We at Foundry 45 have uncovered many different methods of acquiring screenshots, each with their own strengths and limitations.

Method 1: Bluetooth Keyboard Option

The most optimal option is quite self-explanatory. Connecting your phone to any Bluetooth Keyboard and pressing the Print Screen Key (or Function + Print Screen in our case) while the phone is in the VR headset will snap a screenshot. Pictures will be placed in your phone’s gallery and accessible via USB. At Foundry 45, we recently purchased this Bluetooth keyboard from Amazon, and have discovered this to be the most user-friendly method available for obtaining screenshots. The high resolution pictured quality combined with the ease of the process dignify this method over any other current solution. We officially give this option the Foundry 45 Seal of Approval.

keyboard

Limitations:

After buying a Gear VR headset and a VR compatible phone, you might not want to rush to purchase another gadget. Especially when the same quality screenshots can be captured through free, albeit more difficult methods.  If you do buy one, make sure that the keyboard you purchase has a Print Screen button! If it doesn’t, you might end up hating your keyboard more than this guy.

keyboard gifVideo Source: https://www.youtube.com/watch?v=BugexqConq8

(Note: You may be able to use a regular keyboard with a USB adapter. Though we have not tested this method, we did come across this tweet that details the process. Looks like this method requires a cable with a male USB end and a female USB end.)

Method 2: Native Video Recording / Screenshot function.

Oculus responded to their users’ communal requests by launching an update to all Android 5.0 phones which enables them to record and screenshot within the Gear VR headset.
In order take screenshots using this method:

1. While viewing inside the headset, hold the back button to bring up the Universal Menu.

2. Select Utilities from the Menu (seen below)

mhm

3. Select the Screenshot option.

4. You will return to whatever application or menu you previously using.

5. You are given 5 seconds to set up the shot. A small red dot on the screen indicates when the picture will be taken.

screen7 edited

6. After you channel your inner photographer, you should hear a shutter flash. The screenshot will be stored in your phone’s gallery and the Oculus folder in your phone’s internal storage.

Final Product

com.oculus.home-20160607-143826Sooooo prrrreeeeeettttyy

Limitations
The screenshots from this method are unfortunately low resolution (1024x1024px). Also, while this method works great for snapping pictures of apps run through the Gear VR Menu, some VR experiences don’t accommodate this function as the Universal Menu is not accessible while the app is running.

On the video side of things, there are also a few hindrances that keep the process from being entirely intuitive.

  • This function only records video, not audio!
  • Unlike the red dot from the screenshot option, there is no recording indicator when capturing video in this method.
  • Video recording will stop any time an app is exited, opened, or when the Universal Menu is opened.
  • Any streaming service like Netflix will completely negate any recording function from the VR headset. Pirates be warned!

painty vr lolARGH!

Method 3: Android SDK tools/ Adb route.
Despite its moderate learning curve, this process has proven to be the most effective way for us to capture screenshots from the phone within the Gear VR headset without the purchase of additional hardware. If you’re not afraid of the command prompt, prepare to have your mind blown.

vertigo finalVideo source: https://www.youtube.com/watch?v=4WAxDlUOw-w

Step 1) Download the Android SDK Toolkit on your PC/Mac
https://developer.android.com/studio/index.html

Install using these instructions!

Step 2) Enable USB Debugging on your Android
a.) Select the Settings Cog from the top swipe-down menu

b.) Select About Device

c.) Tap “Build Number” 7 times to enable Developer Mode

d.) Go back to the previous menu

e.) Select Developer Options

f.) Enable USB Debugging

Step 3) Connect to your Android through Wifi
a.) Connect Android to PC via USB

b.) Open Command Prompt on the PC

c.) Ensure Device is connected by typing: adb devices

d.) Should see Phone serial listed

e.) Turn on Phone Wifi

f.) Find Out Phone IP address by typing in PC: adb shell ip addr show wlan0

g.) Among all the information, locate the IP address.

h.) Should be in format: 192.168.1.111

i.) Type in: adb tcpip 5555

j). Disconnect the USB cable

k.) Type in this sequence with your IP address: adb connect 192.168.1.111

l.) Ensure Device is connected by typing: adb device

Step 4) Snap Some Shots Baby!

Dancing Wedding Photographer | Foundry 45

Video Source: https://www.youtube.com/watch?v=PiccRcnjPtE

a.) Place the Phone in the VR Headset and run any app you want.

b.) Record your first screenshot by typing and entering in the PC: adb shell screencap /sdcard/screen1.png

c.) Record your second screenshot by typing and entering in the PC: adb shell screencap /sdcard/screen2.png

d.) Continue this process until your heart’s content.

e.) To access these files, simply reconnect your Phone to Computer via USB! They’ll be in DCIM/Screenshots

f.) To disconnect Phone from Wifi, type: adb disconnect

Limitations:
Obviously, technical expertise will benefit you while accomplishing this task, but if you stick closely to this guide, you can complete every step without any prior knowledge. Though being user friendly isn’t this method’s strong suit, learning something never hurt anyone (Except maybe Oedipus). Be conscious of how your headset is tilted while you enter the screenshot function because it will affect the picture. Because this process isn’t optimized for VR quite yet, screenshots will be doubled (like below) as this is what the actual phone screen is displaying. The same is the case for the Bluetooth Keyboard method, but simply cropping the image can solve this problem!

screen1
Feel free to come to us for all questions and needs related to this cool Virtual Reality tech!

Oracle Pictures   You can call us the Ora-cool

If you’re interested in creating your own Virtual Reality application for your trade show or recruiting efforts, be sure to contact Foundry 45 for premiere content!

Article Sources + Related Reading
https://twitter.com/AndrewDickerson/status/618842069734305792

http://uploadvr.com/take-screenshot-gear-vr/

http://www.sammobile.com/2016/01/06/samsung-gear-vr-screenshot/

http://www.roadtovr.com/natively-record-video-take-screenshots-gear-vr/

ABOUT FOUNDRY 45

Foundry 45 is The Business VR Leader. We create immersive Virtual Reality (VR) experiences for mobile platforms including HTC Vive, Oculus Rift, Samsung Gear VR, iOS Cardboard Headsets, Android Cardboard Headsets. Foundry 45 helps businesses step into their brand story by using dynamic 360-degree video and VR app creation. Our marketing and recruiting approach captures customer experience desires and emphasize product value. We also partner with existing video production studios, advertising agencies, and other content creators to serve as their VR technical consultants and developer shop.

To learn more about Foundry 45, visit us at www.foundry45.com.

For Immersive VR inquiries, contact Managing Partner:

Dave Beck, dave@foundry45.com

Follow Us!

Linkedin Twitter Facebook Instagram

Implementing 360 Video in Unity for Gear VR and Cardboard

This entry will describe our efforts to get 360 video working in Unity for apps running on Gear VR and Cardboard (both iOS and android). This is more of a work in progress than a full guide, but I hope it helps. Feel free to leave questions or suggestions!

The overall process is:

  1. Start with google cardboard camera or oculus camera demo scene.
  2. Add a sphere with an equirectangular UV mapping and inward facing normals around the camera.
  3. Purchase a plugin to play a movie on that sphere’s texture. (Note: if you just want to run on Gear VR, you can adapt their movie example to play on the inside of a sphere. I’m not sure how to extract their movie playing code to a cardboard android app, though).
  4. Use mp4s or ogg vorbis files that are compatible with the platform. Some resolutions and frame rates don’t work for me.

Movie & Video Plugins for Unity and Android / iOS / Gear VR.

The most success so far has been achieved using the Easy Movie Texture plugin for Unity, currently $45. It works on android (utilizes Android MediaPlayer) and iOS and the developer gives great support. It even supports streaming and playing from the SD card. Note that it doesn’t work in the Editor, so you have to build to see the results.

The best part of this plugin is that it comes with a demo scene including a sphere to play back equirectangular videos. Getting a suitable sphere is a little tricky, since the default sphere in Unity has normals facing outwards rather than in, and doesn’t have the right UV mapping.

I also tried prime31’s iOS video plugin, LiveTexture, which was $75. This one was nice because videos would play inside the Editor, but I had to provide my own sphere and it doesn’t work on android. AND, sounds don’t play. The plugin provides a way to sync an audio track along with the video, but this seems likely to get out of sync. I had trouble with larger format videos being very choppy. A 2048×1024 was unwatchable on iPhone 5S, which is my lower end target for playing back 360 video with cardboard. Anything below 1000px of vertical resolution starts too look poor under a magnified 360 view.

Another one that’s also $75, but free to try, is Mobile Movie Texture. This one requires ogg vorbis files, and also doesn’t play video at the same time. It does support android and iOS.

One other plugin I tried plays back a sequence of images: Universal Video Texture ($20). This also doesn’t sync audio, but has the potential for extreme hi-def playback at the cost of extreme app size–not ideal for a mobile app. I believe a lot of the compression in an mp4 or ogg vorbis file is due to the compression algorithm knowing that a certain pixel doesn’t change from one frame to the next.

A final plugin that was recently pulled from the store by the developer (too much work to maintain maybe?) was Video Texture Pro 2. Here’s an interesting forum post by its author about video support in Unity.

I just came across a forum post on reddit talking about SPlugins Movie Texture for Mobile ($30) that may also work on android and iOS with sound, but haven’t tried it yet.

360 Equirectangular Sphere for VR Movie Playback in Unity

If you’d like a sphere to use with LiveTexture or one of the image sequence players, check out the roundscreen.obj (has holes at poles) here or an inside facing normals sphere with 32×16 vertices I made in blender. I’m not sure how the number of vertices impacts the resulting quality vs performance. Lower poly spheres seem to look very bad near the poles. Here are the instructions I followed to create the UV wrapped sphere if you’d like to try out much higher vertex counts. If you find a sphere with the normals facing out rather than in, you can run this reverse normals script on the object.

(With LiveTexture, I also had some strange wrapping issues with non-powers-of-2 video resolutions (ie, not 1024,2048, etc). I got around those by commenting out the line including “updateMaterialUVScaleForTexture” in the plugin code.)

In addition to reversing the normals, you may also need to mirror the UV map–ie, it might playback flipped. The Easy Movie Texture plugin flips the UV y coordinates (on iPhone only?) by iterating through all the UVs and reversing the y:

vec2UVs[i] = new Vector2(vec2UVs[i].x, 1.0f -vec2UVs[i].y);

Video Resolution, File Location, Frame Rate and Other Considerations for VR on iOS and Android

I was able to play 4096×2048 mp4s on Android, however files with a 50 fps frame rate stalled repeatedly on a Galaxy Note 4, whereas 25 fps files played perfectly (Using Easy Movie Texture plugin). Reducing the bitrate in half or the resolution had no effect on playability, only the frame rate mattered. I’m still trying to wrap my head around what frame rate I see given that the Unity app is running at one frame rate and the video at another.

One tip for experimenting with different files is to play off the SD card on android. This way you don’t have to import the videos into Unity. Placing an mp4 into the Streaming Assets folder in a Unity project causes unity to try to import it to a ogg vorbis file, which can take hours for videos over a few minutes, and is completely unnecessary since the plugin plays the original mp4. You can also make the extension a bogus name like .mp42 to prevent unity from converting the video. This worked for me on Android, but not iOS.

To play a file from the SD card, simply give the Str File Name variable in Easy Movie Texture the following full path:

file:///storage/extSdCard/myvideo.mp4

Note that the path to the sdcard may be different on different phones. The above is only for the Galaxy Note 4. You can find out your path with the following command:

adb shell 'echo ${SECONDARY_STORAGE%%:*}'

Note on iOS, be sure to run one of the patches depending on your version of Unity inside the EasyMovieTexture folder: Unity5_Patch_IOS or Unity463_Patch_IOS.

Another big restriction on iOS, at least on an iPhone 5s is that the max file size is 1920×1080. So for 2×1 equirectangular movies, this is 1920×960, almost half the size of the android. I believe the iPhone 6 can play larger files, but have not tested yet.

Open Questions

  • What is the limit of playback fps and resolution on both iOS and Android? I know Gear VR supports 4096×2048 at 60fps, but is this possible from within Unity?

Looking to create custom, native VR Apps without having to learn any of the above or touch a single line of code? Our VR Forge system might be just what you need to get going quickly and profitably. Take a look to see if it’s right for you!

ABOUT FOUNDRY 45

Foundry 45 is The Business VR Leader. We create immersive Virtual Reality (VR) experiences for mobile platforms including HTC Vive, Oculus Rift, Samsung Gear VR, iOS Cardboard Headsets, Android Cardboard Headsets. Foundry 45 helps businesses step into their brand story by using dynamic 360-degree video and VR app creation. Our marketing and recruiting approach captures customer experience desires and emphasize product value. We also partner with existing video production studios, advertising agencies, and other content creators to serve as their VR technical consultants and developer shop.

To learn more about Foundry 45, visit us at www.foundry45.com.

For Immersive VR inquiries, contact Managing Partner:

Dave Beck, dave@foundry45.com

Follow Us!

Linkedin Twitter Facebook Instagram