A developer has demonstrated the use of ARKit and Vision for performing 3D finger painting in augmented reality.
The developer, Osama Abdel-Karim, demonstrates how a user could draw in the air similar to as if they were holding a pen. The ink will then magically appear on the surface. You can even add an effect to make the drawing into a 3D object.
Of course, it’s not actual magic powering the experience, but rather a form of wizardry from Apple’s software. In particular, the ARKit framework paired with an iOS 11 tool called Vision.
“I believe AR has always been missing two key technology leaps to make it useful: usability and immersion,” says Abdel-Karim. “We are heading towards a new AR hype peak very soon.”
You’re probably aware of ARKit as it’s been demonstrated on several occasions:
The difference with this demo is the use of the Vision library which tracks the user’s fingertip to create the illusion of finger painting in thin air.
Writing about his development process, Abdel-Karim wrote:
“One of the cool libraries that Apple introduced in iOS 11 is Vision Framework. It provides some computer vision techniques in a pretty handy and efficient way. In particular, we are going to use the object tracking technique. Object tracking works as follows: First, we provide it with an image and coordinates of a square within the image boundaries for the object we want to track. After that we call some function to initialize tracking. Finally, we feed in a new image in which the position of that object changed and the analysis result of the previous operation. Given that, it will return for us the object’s new location.”
You can find his full write-up and source code here.
Have you tried ARKit yet? Let us know your thoughts in the comments.
