SketchAR Facade Detector

To make it possible to place AR content on real-world objects in video, such objects must be accurately tracked in each frame in real time. AR anchors could be planar or curved surfaces. Examples of planar ones: a table top, paper list, wall, building facade. Determining the spatial configuration of the anchoring surface is crucial for AR content to be correctly displayed. In contrast to other surfaces, such as a sheet of paper, facade boundaries may be uncertain or overlapped by nearby objects (trees, cars). Therefore, facade boundaries alone cannot be used to define the configuration of the facade plane.

Comparison of analogs:

• AR Cloud requires scanning the environment with a specific software and hardware before filling it with virtual objects.
• LIDAR works effectively only within 5 meters and only on the latest Apple smartphones. SketchAR Facade Detector works with any camera 50+ meters away.

We propose a new method for facade plane detection, in which the detection is based on the assumption of the predominance of vertical and horizontal lines on the facade surface in the physical world. Thus, perspective transformation can be found by detecting the distribution of horizontal and vertical lines in the image.

The facade plane detection method comprises the following algorithms:
• edge detection;
• line detection and filtering;
• approximation of line distributions;
• determination of boundaries.

Download SketchAR for iOS and test the new Facade Detector feature.

Feel free to drop us a line at hi@sketchar.tech.