Posts

Embodied Cognition: Thinking with Your Whole Body

As human beings, we are always learning. Every day our brains process and store new information learned from experiences that can be used to further direct our lives.

But despite what you may think, it’s not just your brain doing the thinking.

The theory of embodied cognition states that your entire body is involved in thinking and learning. It’s one of many reasons that training through virtual reality is so incredibly powerful.

What is it?

Differing from the new theory of embodied cognition, the theory of dualism states that the brain does all the thinking while merely directing the body to function.

The theory of embodied cognition, on the other hand, paints a more connected picture of the brain and body. Many today believe that we process information about our world through the context of our bodies. George Lakoff and Mark Johnson’s pioneering work Metaphors We Live By applied this theory to language, finding that people across cultures connected the same concepts with similar phenomena related to the body. For example, we tend to think of the past as being behind us, and the future as in front of us.

Learning from the Outside In

Another way to think of embodied cognition is by considering the many ways we learn and understand. A child learning about apples might hear the word ‘apple,’ but will also see real apples as well as pictures of them; they may learn where apples are kept in the kitchen, and how an apple tastes. All of these pieces of information work together to form the idea of an apple in the brain.

All of this is to say that we learn more effectively when we can think and interact with our subjects in many different ways. The problem comes when what we have to learn cannot be interacted with. That’s where virtual reality can help.

Researchers Susan Jang and Jonathan Vitale demonstrated this by using VR to teach anatomy to medical students. They made a 3D model of the inner ear, a complex and hidden anatomical structure, and let one-half of their study participants manipulate it in VR while the other half watched 2D video replays from the VR sessions. The VR group scored an average 20% higher when tested on the structure of the inner ear, regardless of how much they knew before the study.

Foundry 45’s immersive learning experiences put the theory of embodied cognition into action in order to assist in employee training. As part of our recent collaboration with Delta Air Lines, we simulated a full runway environment for users to explore. Our experience lets trainees learn runway safety and pre-flight procedures with their whole bodies, picking up debris and pointing out airplane damage.

A Piece of the Puzzle

Does this mean that books and classroom teaching are dead as a method of learning? Of course not. Written material will always be an essential part of learning. But new methods, such as virtual reality and embodied cognition, can definitely have a positive impact as a complement to go in tandem with traditional methods of education.

VR is not a panacea for education but instead a supplemental tool to deepen understanding and engage learners in a whole new way.

Dr. Lindsay Portnoy, Digital Culturist

Micro-Learning & VR: A Promising Solution to a Daunting Issue

Why Create a Virtual Reality App from Your 360 Video?

While creating your 360 story is an accomplishment unto itself, for most people, it’s not the end goal. You’ve likely created your immersive story to share with others — whether that’s with a specific audience, at a trade show, with sales prospects, or the wider Internet audience in general. You now have to figure out how to get your immersive story into the hands of your audience, and make sure whatever approach you take actually works for them!

One potentially valuable approach is to use the 360 Video as the anchor for a full Virtual Reality app, rather than simply offering the footage itself for viewing. There are multiple advantages to such an approach, and creators can often drive more profitable engagements by including such initiatives in their overall project to their clients.

7K0A0032Let me give you more money please!

Delivers Presence With a Virtual Reality Headset
There is a concept in the VR world of “presence” — when the user actually feels, to some extent, as if they are there. In this regard, there is a very big difference between watching 360 video that you can move around and virtual reality from within a headset. If you’ve never used a VR headset (Samsung Gear VR, Oculus Rift, Google Cardboard, etc.), this statement will likely ring hollow. But, I cannot overstate this — using a Virtual Reality headset is a completely different experience from watching a 360 video on a flat screen.

Viewing a 360 video on a desktop computer is really just watching a flat video, with the added feature that they can use a mouse to control what direction the flat video is pointing. Someone experiencing your content through a headset is transported inside your story, connecting with the experience on a completely different emotional level. Watching a YouTube 360 video on a computer is cool — you can see from the perspective of a skydiver and pan around to see what is around. Watching the same content through a headset gets your heart racing, and I’ve seen more than one person pull their feet up at the end as they’re about to “hit the ground” — I’ve never seen such a reaction from a user watching a flat video.

Unfortunately, current 360 viewers do not offer a simple, broadly supported approach for this type of experience. YouTube 360 does have the ability to switch into Cardboard mode and use it with a headset — but only if they are using an Android phone. If the user has an iPhone (often a majority of your target audience), they’re out of luck and are back to watching a flat video for which they can control the perspective.

By packaging and releasing an app, you can make sure that your audience can get that all important sense of presence whether they’re using iOS or Android.

Provides a Stronger Branding Opportunity
We’ve found that having a branded app, that has the logo and connection to you or your customer and the content you’ve created, is powerful by itself. By releasing a branded app for your customer, you are delivering a higher end, more comprehensive experience that can deliver more value to your customer (at a higher price point) beyond the 360 video production.

Makes Getting Started Easier
No matter the use case, creating a standalone app makes it much easier for the intended audience to get started. It’s better than expecting/hoping end users will follow the steps necessary to install a viewer, download your content, load your content into the viewer (often in a special directory for it to work), launch the viewer app, and select your 360 video in the generic viewer. One misstep and the end user can easily get frustrated and give up — it’s much easier to install a branded app via a link or keyword search term in the public app stores.

Allows for Interactive Calls to Action
By releasing a full native app, clear calls to action (Contact Us, Share with a Friend, Schedule a Demo, Visit Our Website, etc.) can be integrated and presented to your audience at appropriate times. While this can be integrated directly into the immersive experience, it can also be done before and after, as part of the traditional app interface. You are in control of the audience experience, rather than dropping your content into someone else’s viewer ecosystem.

7K0A0674

Hello, I just used your app, and I had to contact you. I can bearly contain myself.

Limits Audience Exposure to Competitive Messages
Delivering content as a custom app also keeps end users from going to something like YouTube 360 where, after seeing your content, they are then bombarded with “similar” (potentially competitive) video options from other creators. You have complete control over the experience, and aren’t subjecting your audience to advertisements and other imagery over which you have little to no control.

Provides for Ongoing Engagement with Audience
If you are telling a story over a series of content releases, a native app is a great way to push content to your users over time. After they install the base app, they can download later experiences or chapters as you release them. These content releases do not have to go through the app store review process again; rather, the user simply gets the new content by running the app. The application on your audience members’ phones serves as a bookmark to remind them when new content is available, rather than relying on them to seek you back out online to check for updates.

Strengthens Copyright Protection
As many content creators have discovered, videos posted to services like YouTube and Facebook can be easily downloaded by anyone and re-purposed for unlicensed uses. A simple search on Google returns pages of results for solutions to allow even non-technical users the ability to download YouTube videos. Once downloaded, they can post the video, edited or unedited, on their own site or post right back on the same sites, but under their name and account.  There is nothing more frustrating than finding your work posted on another site, being used to sell someone else’s services.

lol2

A native app, however, can be constructed to keep users from being able to download the raw content. The footage can be bundled into the app and only accessible via API with authorized keys. This allows the end user to still access it easily, but only in the manner in which you are offering it.

Are you a professional 360 photographer or videographer wanting to create custom, native VR Apps for your clients without having to learn any of the above or touch a single line of code? Our VR Launch system might be just what you need to get going quickly and profitably. Take a look to see if it’s right for you!

ABOUT FOUNDRY 45

Foundry 45 is The Business VR Leader. We create immersive Virtual Reality (VR) experiences for mobile platforms including HTC Vive, Oculus Rift, Samsung Gear VR, iOS Cardboard Headsets, Android Cardboard Headsets. Foundry 45 helps businesses step into their brand story by using dynamic 360-degree video and VR app creation. Our marketing and recruiting approach captures customer experience desires and emphasize product value. We also partner with existing video production studios, advertising agencies, and other content creators to serve as their VR technical consultants and developer shop.

To learn more about Foundry 45, visit us at www.foundry45.com.

For Immersive VR inquiries, contact Managing Partner:

Dave Beck, dave@foundry45.com

Follow Us!

Linkedin Twitter Facebook Instagram

Implementing 360 Video in Unity for Gear VR and Cardboard

This entry will describe our efforts to get 360 video working in Unity for apps running on Gear VR and Cardboard (both iOS and android). This is more of a work in progress than a full guide, but I hope it helps. Feel free to leave questions or suggestions!

The overall process is:

  1. Start with google cardboard camera or oculus camera demo scene.
  2. Add a sphere with an equirectangular UV mapping and inward facing normals around the camera.
  3. Purchase a plugin to play a movie on that sphere’s texture. (Note: if you just want to run on Gear VR, you can adapt their movie example to play on the inside of a sphere. I’m not sure how to extract their movie playing code to a cardboard android app, though).
  4. Use mp4s or ogg vorbis files that are compatible with the platform. Some resolutions and frame rates don’t work for me.

Movie & Video Plugins for Unity and Android / iOS / Gear VR.

The most success so far has been achieved using the Easy Movie Texture plugin for Unity, currently $45. It works on android (utilizes Android MediaPlayer) and iOS and the developer gives great support. It even supports streaming and playing from the SD card. Note that it doesn’t work in the Editor, so you have to build to see the results.

The best part of this plugin is that it comes with a demo scene including a sphere to play back equirectangular videos. Getting a suitable sphere is a little tricky, since the default sphere in Unity has normals facing outwards rather than in, and doesn’t have the right UV mapping.

I also tried prime31’s iOS video plugin, LiveTexture, which was $75. This one was nice because videos would play inside the Editor, but I had to provide my own sphere and it doesn’t work on android. AND, sounds don’t play. The plugin provides a way to sync an audio track along with the video, but this seems likely to get out of sync. I had trouble with larger format videos being very choppy. A 2048×1024 was unwatchable on iPhone 5S, which is my lower end target for playing back 360 video with cardboard. Anything below 1000px of vertical resolution starts too look poor under a magnified 360 view.

Another one that’s also $75, but free to try, is Mobile Movie Texture. This one requires ogg vorbis files, and also doesn’t play video at the same time. It does support android and iOS.

One other plugin I tried plays back a sequence of images: Universal Video Texture ($20). This also doesn’t sync audio, but has the potential for extreme hi-def playback at the cost of extreme app size–not ideal for a mobile app. I believe a lot of the compression in an mp4 or ogg vorbis file is due to the compression algorithm knowing that a certain pixel doesn’t change from one frame to the next.

A final plugin that was recently pulled from the store by the developer (too much work to maintain maybe?) was Video Texture Pro 2. Here’s an interesting forum post by its author about video support in Unity.

I just came across a forum post on reddit talking about SPlugins Movie Texture for Mobile ($30) that may also work on android and iOS with sound, but haven’t tried it yet.

360 Equirectangular Sphere for VR Movie Playback in Unity

If you’d like a sphere to use with LiveTexture or one of the image sequence players, check out the roundscreen.obj (has holes at poles) here or an inside facing normals sphere with 32×16 vertices I made in blender. I’m not sure how the number of vertices impacts the resulting quality vs performance. Lower poly spheres seem to look very bad near the poles. Here are the instructions I followed to create the UV wrapped sphere if you’d like to try out much higher vertex counts. If you find a sphere with the normals facing out rather than in, you can run this reverse normals script on the object.

(With LiveTexture, I also had some strange wrapping issues with non-powers-of-2 video resolutions (ie, not 1024,2048, etc). I got around those by commenting out the line including “updateMaterialUVScaleForTexture” in the plugin code.)

In addition to reversing the normals, you may also need to mirror the UV map–ie, it might playback flipped. The Easy Movie Texture plugin flips the UV y coordinates (on iPhone only?) by iterating through all the UVs and reversing the y:

vec2UVs[i] = new Vector2(vec2UVs[i].x, 1.0f -vec2UVs[i].y);

Video Resolution, File Location, Frame Rate and Other Considerations for VR on iOS and Android

I was able to play 4096×2048 mp4s on Android, however files with a 50 fps frame rate stalled repeatedly on a Galaxy Note 4, whereas 25 fps files played perfectly (Using Easy Movie Texture plugin). Reducing the bitrate in half or the resolution had no effect on playability, only the frame rate mattered. I’m still trying to wrap my head around what frame rate I see given that the Unity app is running at one frame rate and the video at another.

One tip for experimenting with different files is to play off the SD card on android. This way you don’t have to import the videos into Unity. Placing an mp4 into the Streaming Assets folder in a Unity project causes unity to try to import it to a ogg vorbis file, which can take hours for videos over a few minutes, and is completely unnecessary since the plugin plays the original mp4. You can also make the extension a bogus name like .mp42 to prevent unity from converting the video. This worked for me on Android, but not iOS.

To play a file from the SD card, simply give the Str File Name variable in Easy Movie Texture the following full path:

file:///storage/extSdCard/myvideo.mp4

Note that the path to the sdcard may be different on different phones. The above is only for the Galaxy Note 4. You can find out your path with the following command:

adb shell 'echo ${SECONDARY_STORAGE%%:*}'

Note on iOS, be sure to run one of the patches depending on your version of Unity inside the EasyMovieTexture folder: Unity5_Patch_IOS or Unity463_Patch_IOS.

Another big restriction on iOS, at least on an iPhone 5s is that the max file size is 1920×1080. So for 2×1 equirectangular movies, this is 1920×960, almost half the size of the android. I believe the iPhone 6 can play larger files, but have not tested yet.

Open Questions

  • What is the limit of playback fps and resolution on both iOS and Android? I know Gear VR supports 4096×2048 at 60fps, but is this possible from within Unity?

Looking to create custom, native VR Apps without having to learn any of the above or touch a single line of code? Our VR Forge system might be just what you need to get going quickly and profitably. Take a look to see if it’s right for you!

ABOUT FOUNDRY 45

Foundry 45 is The Business VR Leader. We create immersive Virtual Reality (VR) experiences for mobile platforms including HTC Vive, Oculus Rift, Samsung Gear VR, iOS Cardboard Headsets, Android Cardboard Headsets. Foundry 45 helps businesses step into their brand story by using dynamic 360-degree video and VR app creation. Our marketing and recruiting approach captures customer experience desires and emphasize product value. We also partner with existing video production studios, advertising agencies, and other content creators to serve as their VR technical consultants and developer shop.

To learn more about Foundry 45, visit us at www.foundry45.com.

For Immersive VR inquiries, contact Managing Partner:

Dave Beck, dave@foundry45.com

Follow Us!

Linkedin Twitter Facebook Instagram