Motion Capture and More: How Apple Is Revolutionizing Augmented Reality
Augmented Reality (AR) has exploded in the past few years, mainly due to the easy access provided by our mobile devices. Digital images once reserved for high-cost headsets are now seen through the camera on our phones and tablets. And thanks to Pokemon Go and Snapchat filters, we’re finally witnessing the use of AR on a mass scale.
In the past few years, Apple has taken an especially aggressive approach to the new functionality of augmented reality with the release of its ARKit, a tool that let’s app developers use the tech in their creations. Google quickly followed suit with the release of ARCore.
Today’s AR is much more than scanning images on trigger images or floating objects in the space in front of us. Our devices are using the camera to detect distances, scanning surfaces to place objects in a realistic position and even adjusting the light of the AR to fit in our real world. The experiences are much more realistic and captivating for the user—and when these resources are in our classrooms, our learners are blown away.
Last year, Apple released ARKit 2 for developers, offering features useful for education apps, including group work in augmented reality and the ability for users to connect in a digital space across the world. The tracking also improved and it was easier to navigate in the space around the 3D objects.
ARKit 3 is the latest update by Apple and was released to developers this summer, meaning it’s still pretty new. The next breakthrough will bring new functionality in the classroom using the most current technology on the newest devices. The feature that I am most excited about is one called “people occlusion,” which is the ability to see augmented reality interwoven in our world, both in front and behind people. It’s an incredible leap forward in our experiences.
Another big feature in ARKit 3 is motion capture. The device will have the ability to track body position and movement, mimicking the movement of a person and replicating it with a digital AR character. I used an app called PuppetMaster to demonstrate the motion capture feature and then placed my animated puppet outside.
I try to connect with education AR developers on a regular basis to collaborate on the latest features, compatibility, implementation and ideas. The new ARKit 3 options have inspired these companies to consider ways to bring the previously impossible learning experiences into your classroom through easy to use apps. Here’s what Franklin Lyons, CEO of Merge had to say:
Merge is always looking at the new features available in Apple ARKit and how we can use them to create magical learning experiences for the classroom. For instance, we can use the collaborative session feature to allow everyone in the class to share in learning together, and this is just scratching the surface of what we’d like to do. It’s safe to say that you’ll see many of these new capabilities show up in future versions of Merge EDU.
The goal of these education companies is to build tools that give students a digital pencil to virtually design, create and collaborate across the world in augmented reality. When considering the new release, they are challenged to find ways to give your students ownership of their learning using AR.
The greatest benefit of using augmented reality in the classroom is enabling creativity. There are a lot of tools out there that provide visually impressive one-shot experiences, but the true power of AR lies in creating content yourself and learning while you do that. ARKit 3 will enable people occlusion which will bring digital storytelling in classrooms to the next level in immersion and feeling. Body tracking and motion capture not only helps to tell your stories more immersively but also enables “programming” motion-triggered actions in your scenes. – Jussi Kajala, 3DBear CEO
A12 Chip Devices
- iPad Mini
- iPad Air
- 11-inch iPad Pro
- 12.9-inch iPad Pro
- iPhone XS
- iPhone XS Max
- iPhone XR
Many companies are determining the benefits and limitations of ARKit 3, and this is especially important in the education market. One limitation: Apps that use ARKit 3 require newer devices. While I’m thrilled about the new technology, I’m discouraged that the majority of classrooms don’t have the compatible devices to begin using these new features. In the past few months, I’ve finally started to see more districts with the right devices for ARKit 1 and 2 apps, but this won’t be true for ARKit 3 as they require an A12 chip or above to function properly.
A13 Chip Devices
- iPhone 11 Pro
- iPhone 11 Pro Max
- iPhone 11
If you’re ready to test out these new features on a compatible device, get started with the JigSpace app, which lets you create augmented reality experiences called Jigs. JigSpace recently updated the app to use people occlusion. Another big announcement is the ability to create your own Jigs with their Workshop app, now available on the iPhone. The JigSpace Workshop app provides the freedom to create your own Jigs. As you can see in my photo, my body is in from of the corral, while my hand is behind it.
Apple released iOS 13 and iPadOS 13 last week, making these new functions available now on the specific devices are shown above. While we didn’t hear too much in the Apple event this month, there’s definitely an upward momentum for more AR experiences on our mobile devices for Apple, Google and Microsoft, among others. It’s rumoured that the release of Apple headwear is possible next year, and if that’s true, our classrooms are in for a special AR treat in 2020. Well, a girl can dream!
News Source:
Image Source:
