If you need evidence that Apple is working on a mixed-reality headset, take a spin with the HoloKit X. Created by Botao Amber Hu, a developer who has worked at companies like DJI, Google, and Twitter and is now CEO and founder of Holo Interactive, this headset relies entirely on existing capabilities of the iPhone to create interactive hands-free augmented reality experiences. It’s a powerful showcase of what’s possible if Apple ever made a headset using the tech already embedded in its smartphone.
Any such headset to come out of Cupertino would almost certainly cost more than a thousand dollars. (This is Apple, after all.) Look at Meta’s newest mixed-reality headset for reference; it starts at $1,499. Headsets in Microsoft’s XR platform cost between $600 and $1,000. These high prices are why the HoloKit X exists. Hu, who has long had a special interest in future computing and new media art, says he wants to “democratize” the world of mixed reality. As such, the HoloKit X costs $129, and all you need is a recent iPhone (excluding iPhone Mini and iPhone SE models) to power it.
The HoloKit X is a plasticky headset with optical lenses inside. There’s no technology here (save for an NFC sensor, but more on that later). Just think of it as a viewer, not unlike old-school View-Masters. Similar to mobile virtual reality headsets like Google Cardboard, Lenovo’s AR set for Star Wars games, or the now-defunct Google Daydream, you need to mount an iPhone onto the HoloKit X.
Unlike VR headsets, you’re not staring at a screen. The iPhone is mounted up and away from your eyes. Instead, you’re looking through the glass in a 60-degree field of view and can see the physical world as well as the people around you. The iPhone’s screen, while using the rear cameras to manage these AR experiences, is mirrored in stereoscopic vision to the lenses, making it so that you can effectively see virtual 3D objects embedded in the real world.
Exactly what you can do with the HoloKit X is limited right now. There are just a handful of experiences—what Hu calls “Realities”—in the HoloKit app, one of which is a multiplayer dueling game where you cast spells at an enemy. The visuals are clear, colorful, and pretty sharp, and the platform supports six degrees of freedom via Apple’s ARKit framework. Because of this, you can move around virtual objects and they will stay anchored in the real-world places where you position them. And when you’re playing a game, you can even duck to dodge blasts. The “enemy” can be another person using a HoloKit X in a shared space, a virtual character, or even a character controlled by someone with just an iPhone.
Since it’s entirely powered by an iPhone, the HoloKit app is leveraging existing technologies. The ability to play a game with other HoloKit X users, for example, doesn’t rely on cellular data or Wi-Fi, but rather the local networking technology that powers AirDrop. This is also what powers “Spectator View," which allows anyone to use an iPhone and the HoloKit app to view your augmented reality experience in real time by pointing their phone at the scene. (You can record and share this to social media, or cast it via AirPlay to a TV for others to see.) Hu says Holo Interactive is also working on a Puppeteer mode that would enable someone else to direct your AR experience.
There are a few ways to interact with the augmented reality experience. The HoloKit app uses Apple’s Vision framework technology to identify and track your hand. I didn’t see a demo of this, but the idea is that you can just use your hands to interact with objects and the iPhone’s cameras will recognize your hand movements. Hu says HoloKit also supports any Bluetooth device that can connect to the iPhone, like PlayStation controllers.
What I did demo was the ability to use an Apple Watch’s gyroscope as a motion controller, just like a Wiimote. Hu strapped an Apple Watch to my wrist (it works with Watch Series 4 and newer) with the HoloKit watch app installed and running, and gave me a wand purely so I could feel like I was using it to shoot out spells. Lo and behold, I was able to cast spells with mere gestures or a flick of the wrist. I could even point my wand downward to load a charging bar and trigger a more powerful spell. Aiding the immersion is the use of spatial audio via any of Apple’s headphones that support that feature, so you can hear a spell whizzing past your right ear. The iPhone’s haptic vibration adds another layer of sensory input, but since the phone is mounted in the headset, it’s only vibrating up near your forehead, so you may not immediately sense it.
You can use the HoloKit X with an iPhone XS, XS Max, iPhone 11, iPhone 11 Pro and 11 Pro Max, iPhone 12, iPhone 12 Pro and 12 Pro Max, iPhone 13, and iPhone 13 and 13 Pro Max, iPhone 14, and iPhone 14 and 14 Pro Max. (You’ll need to take off your case so it will fit.) You’ll get the best experience with an iPhone that has a lidar sensor, which became a staple on the Pro models—starting with the iPhone 12 series.
The dueling app doesn’t use the lidar sensor, but there’s another AR experience that showcases virtual objects filling up a space and interacting with objects and people. The lidar sensor helps the phone and app detect your environment, and the Neural Engine in the iPhone’s processors helps with object and human detection.
What’s really impressive is just how comfortable HoloKit X is to wear. It’s even quite stylish, resembling a visor hat. There’s a plush band you can tighten over your head to keep it snug, and these bands are interchangeable, but I was more impressed that the headset didn’t interfere with my eyeglasses at all. Throw a splash of color on the whole thing, and I could see it as a gadget straight from a sci-fi show like Cowboy Bebop. The only tech in the headset is an NFC sensor, which Hu says can be used to track anonymous data, like which device is in the headset and which AR apps were used the most.
The hardware is final, but the HoloKit app is still very much in a controlled beta. You can buy a headset right now, but you’ll only get access to a TestFlight version of the HoloKit app—Hu says his team (of around 10 people) expects to launch it in the App Store by the end of November with additional Realities. And building the right Realities is the crux.
“The killer app for StAR [Stereoscopic AR] has not been made yet,” Hu says. "Everyone is waiting for Apple or Meta—but for what? Everyone is talking about the hardware, but no one is talking about the app. Everyone is talking about productizing workers or weaponizing soldiers. We don’t believe that’s the future. People talk about putting notifications on your face, but that’s an Apple Watch—we already have that.”
Hu says his company won’t be releasing a software development kit at the moment. Instead, his team of software experts, who have a background in content creation and new media, will craft the initial Realities, particularly because they’re using so many new technologies that it’s difficult to educate other developers on how to use them. It’s also because Hu has a vision of what his “RealVerse” will look like. “Why did I make HoloKit? I’m an artist. I dreamed of something like it, so I made HoloKit so everyone can share my dream.” When the technology and the market mature, Hu expects Holo Interactive will release HoloKit under an open source software license.
The platform will still heavily rely on partnerships and collaborations. Holo Interactive has already partnered with the Web3 Gallery in New York City, where you’ll be able to walk in and demo the HoloKit X. Eventually you’ll be able to see NFTs and avatars come alive in the real world.
Much of HoloKit feels as though it’s designed to be sitting at museums and spaces like the Web3 Gallery as a means for creative professionals and marketers alike to showcase their art or brand experiences in mixed reality, jumping on the chatter of the metaverse. But Hu wants there to be a community too. That’s why it’s so easy to share and see someone’s HoloKit X experience with Spectator Mode.
However, there’s a whole slice of the market that HoloKit will not work for: Android users. Hu says everything the team has built relies on specific technologies in the iPhone, like lidar, which has yet to debut on an Android phone. “For AR, you really need hardware, software, and silicon working together,” Hu says, a feat that Google has only recently accomplished. “We won’t support Android until WebXR gets mature,” he says, referring to the platform that lets you create mixed-reality experiences accessible from a browser.
We’ve seen Apple show off various augmented-reality experiences every time its developer conference rolls around, but it usually involves a person holding their phone up to their face as they walk around a space. Hu hardly thinks that’s interactive. Bundling the existing technology into a system that lets you plonk the iPhone on your head—leaving your hands free—without pulling you into a virtual world has the potential to elevate Apple’s work and give even more people with an iPhone their first taste of this new reality.
