One of the show stoppers at WWDC was when Apple announced a new core technology that is intended to position Apple firmly in the world of AR (Augmented Reality). Utilizing the iOS device hardware, and a new developer kit known as ARKit, developers are now able to merge what’s real with what’s not.
Apple’s Senior Vice President of Software Engineering, Craig Federighi ran us through a demonstration. It begins with a demo of a developer app that uses the iPhone’s camera. The camera recognizes the surface of a table on the stage and establishes the plane with boundaries. Then Craig places a virtual steaming cup of coffee on the table that is properly proportioned to the size of the table. He then placed a virtual lamp on the table and demonstrated how the two virtual objects interact. By moving one object or the other the shadows on the cup of coffee behaved as they would in the real world.
How do they do that?
By using the hardware, CPU, GPU, motion sensors, already built into the iPhone or iPad and utilizing ARKit, which provides all the parameters to make this happen which includes:
- Fast, stable motion tracking
- Plane estimation with basic boundaries
- Ambient lighting estimation
- Scale estimation
- Support for Unity, Unreal, SceneKit
- Xcode App Templates
- Integrated third party frameworks to help with rendering
Because there are millions of iPhones and iPads in the world already. Apple will have the largest AR platform in the world, right out of the gate.
Imagine using the camera in your iOS device to map out the rooms in your home, then placing virtual copies of IKEA furniture in your home so you can see what they will look like before buying.
Wingnut AR Director Peter Jackson’s company is dedicated to producing AR content. Alasdair Coull, Creative Director at Wingnut AR did a mind-blowing demo of AR on a real world table. It begins with turf unrolling over the entire surface of the table. Structures popup and virtual people are walking on this set of an outpost somewhere in the universe. Airships arrive bringing supplies. A moment later raider ships swoop in and attack the outpost. Chaos, fire, and mayhem occur and a battle follows. You have to force yourself to realize that you are looking at all of this taking place on a real table top, on a stage, in an auditorium, in San Jose, CA on planet Earth. The experience really draws you in. Wingnut AR will be releasing an Air Experience in the App Store later this year.
Already there are a number of developer videos on YouTube going through some paces with ARKit. Detecting planes and building a virtual city for example. Apple hit a home run with ARKit and it will be a source of exciting apps to come.