We’ve found a solution to one of the main problems in computer vision with white surfaces.

After two years, we are ready to show you one of the most critical stages in the development of SketchAR. Also, in computer vision in general.

You already know how we’ve developed some of our methods in computer vision(CV). These methods help to secure virtual objects on surfaces, even on flat surfaces.

The best marker is no marker! The complexity of the development has been exacerbated by a lack of information in the CV area. The most sophisticated CV development so far has occurred in the sphere of face-recognition. Also headway has been made with texts, signs, and objects, but not a lot regarding flat surfaces. So, we have to find, step by step, solutions to these fundamental problems.

Sketch AR-Anchors.

SketchAR Tracking Method Without Markers is able to detect flat surfaces. Moreover, it remains steady even in extreme conditions, such as the camera not seeing the environment clearly when zoomed in and only looking at a white surface.

How does it work?

Do you remember the movie ’50 shades of gray’? Even if not, then the title is enough to understand that we have enabled the recognition of more than ’50 shades of white’. The camera of a smartphone can recognize many tones of white. Do not forget that the task is more complicated when a hand is always crossing the surface. Besides this, we use machine learning and neural networks to analyze the environment. We collect all data as connections and relationships between the anchor points and then analyze the position of the virtual object relative to the real surface.

Check out the example below where ‘anchors’ on the paper are losing their points, because of moving their hands over the paper.

Problems of ARKit/ARCore: a virtual image flies away if the camera close to a flat surface.

Our algorithm solves the problem better than ARKit or ARCore, which give low accuracy on a flat surface and on a virtual object moving away from the current position.

You can see in the example how the virtual image is shaking a lot, because the camera doesn’t see any environment, only white flat surfaces.The problem of virtual objects moving away every time they are placed in front of any flat surfaces is a huge problem for any company who works with augmented reality.

SketchAR’s Tracking Method Without Markers is based on matrix of tranfarmation(+ML and NN). Connections between the points do not allow the virtual drawing to shift (the virtual sketch is not display on the video). Moreover, the reinitialization occurs and the points in the grid return to their original position. When a hand leaves the frame or moves to another square, the desired pixel of their place is re-recognized.

SketchAR’s Method vs. ARKit/ARCore

With the STMWM a virtual image(frog) is stable even if the hand crosses the surface more than 60%. (ARKit/ARCore is not involved).

We’ve solved this problem, and implementing the technology is an ongoing process. We will continue to use our previous solution with markers until STMWM is able to do all jobs by itself later this year.

Thank you,

SketchAR Team