AR in phone
AR apps are going to boom.
Saying it is a golden age of AR may be an overstatement. But after hands-on experience with the new iPhone X. I view iPhone X’s impacts as the following:
- Depth sensors will become a standard for all phones
- Unleash app developers’ creativity on AR apps
Depth Sensor
There are many depth sensing technologies. Ranging from the most expensive and accurate laser-based (like the LiDAR) to cheaper IR camera-based and software-based (mostly depending on machine learning algorithm to analyzing one or more image taken at an object). Here I only focus on what iPhone X uses.
Note: cheap here only refers to cost per user excluding the development cost.
iPhone X uses a technology first pioneered by Microsoft in their game console Xbox. The device is called Kinect (which Microsoft just killed it).
Although the underlying technology is the same, Apple has two significant contributions.
- An IR camera small to fit into an iPhone size device.
Kinect is a 11″ x 2.5″ x 1.5″ device and iPhone is 5.65" x 2.79" x 0.3". The actual “haircut” bar that fits the dot projector and IR camera is only about 1.4" x 0.3". That is a huge success to engineer the sensors into such a compact space, and it works great! - Outdoor usage.
Kinect does not work well in outdoor, where there is IR interference. iPhone X also shows some unreliable Face ID recognition, but it seems to be better than Kinect. I do not know whether the improvement is coming from a more robust hardware or some software error correction. I will wait for someone to better explain this than me. In general, iPhone X’s Face ID is usable with a bit of effort. Still a big plus here.
AR application
The most prominent impact of the technology does not come from the technology itself, but from its applications.
Side note: The list of technologies Microsoft pioneered and did well in other’s product worths another article about it.
Apple demoed some exciting applications (for example, the Animoji)
The following is another cool application I found called Clip. It has a 360 degree AR selfie (excuse me about the noise and the light on my face is not great. I recorded it when I was lining up for Boba tea :D)
Although we can still see some artifacts around my head, the result is good enough for a lot of applications. For example, trying out new glasses, putting on makeups, test different hairstyles and more.
Future
One thing that is missing is the rear facing depth sensor. With that, the possibility will be huge. It will also come with more challenge on the depth sensor. When the dot projector need to project the dots further away, interference will become a problem. Engineers might need to use a different technology to achieve the goal. I am looking forward to the day with depth sensor built into all devices.