フレクトのクラウドblog re:newal

http://blog.flect.co.jp/cloud/からさらに引っ越しています

Use indoor positioning system (IPS) on AR Remote Instructions base on ARKit

研究開発室の馮 志聖(マイク)です。

Content

  • Introduction
  • IPS (Indoor Positioning System)
  • Solution
  • Conclusion
  • Other
  • Reference

Introduction

When we want to go to an unknown location, we will use maps to navigate. In the era when the global positioning system was not popular, people used paper maps. With the development of science and technology, almost all modern navigation systems use the Global Navigation Satellite System (GNSS), and many countries have developed their own positioning systems, among which Global Positioning System (GPS) is more commonly used. GPS will use trilateration to locate the user's current location and set the path to the destination. GPS positioning is used when outdoors.

Why we need indoor positioning system (IPS)?

However, in the indoor environment, it is difficult for GPS to provide reliable positioning information due to signal attenuation and blocking caused by buildings and construction materials. Thus the location-based apps such as Navigation don’t work inside airports, malls, subways, etc. GPS doesn’t provide information about the floor and building level where the user is located. These factors led to the need for a system where a user can be located inside a building accurately and precisely to make positioning and navigation inside indoors as easy as outdoors.

GNSS https://en.wikipedia.org/wiki/Satellite_navigation

GPS https://en.wikipedia.org/wiki/Global_Positioning_System

Trilateration https://en.wikipedia.org/wiki/True-range_multilateration

Association of indoor positioning system (IPS) and AR remote instructions system.

This is the previous blog about AR remote instruction.

AR Remote Instructions base on ARKit - フレクトのクラウドblog re:newal

In AR remote instruction, the supporter or the client will have a problem, that is very often unable to find the target location and cannot provide assistance. Almost all locations are indoor environments, such as warehouses or offices.

Therefore, we searched some existing research information to investigate which research methods are suitable for AR remote instruction.

IPS (Indoor Positioning System)

Purpose

We must have the Navi include (our / target) position. Sample use case:

  • Supporter wants to teach the client.
  • But the client has no idea which machine.
  • So we create a positioning system to help the client find the target.

f:id:fengchihsheng:20210325120734p:plain
Sample use case

IPS is keep growing. We will create IPS for AR Remote Instruction.

Function of IPS

  • Locating and positioning
  • Locating and tracking
  • Identification and segregation

Positioning methods

Proximity positioning

Proximity positioning is based on direct contact or on proximity between a receiver and a device, such as a Beacon device and a smartphone.

In this system, the method of determining the user's position is based on which beacon signal range is closest. In this picture, we can see that the user is closest to beacon1 and then is associated with the location set by this beacon1, so we can conclude that the user is in the beacon1 room rather than in other beacon rooms.

f:id:fengchihsheng:20210329155028p:plain
Positioning methods (Proximity positioning)

https://www.researchgate.net/figure/Combined-BLE-and-Wi-Fi-fingerprint-based-indoor-positioning_fig2_315839931 https://www.avsystem.com/blog/indoor-navigation-and-indoor-positioning/

Solution

We will use iBeacon as the solution.

There are many limits to the use of iBeacon.

A. Obstacle

For example, in the figure below, the figure on the left shows that the beacon signal received by the user's mobile phone is shielded by the wall. The figure on the right shows that the signal of the beacon is also affected by the human body or other objects.

f:id:fengchihsheng:20210325141351p:plain
iBeacon (Ranging issue)

https://developer.apple.com/ibeacon/Getting-Started-with-iBeacon.pdf

B. Distance

This picture analyzes the degree of signal interference, you can see that the signal value decreases with distance, and the farthest distance is about three meters.

f:id:fengchihsheng:20210325141551p:plain
Beacon (RSSI issue)

https://shizuk.sakura.ne.jp/bluetooth/beacon/proximity.html

Plan

Because the signal range of beacon is relatively small, we will set up multiple devices to achieve the goal of positioning and navigation in the environment. Like this image:

f:id:fengchihsheng:20210325142046p:plain
Proximity-based Systems (Beacon)

System Overview

In this system, we will first create a Beacon map to correspond to the relationship of storing ARWorldMap data.

During the saving process, the signal strength of a certain Beacon in the map can be used to understand the current user's location when storing.

During the loading process, the correspondence between ARWorldMap and the storage location can be load through the data in the cloud, and then the data can be used to locate and navigate to the destination.

Like this image:

f:id:fengchihsheng:20210325142141p:plain
system overview

Save ARWorldMap

If the user saves the file in the beacon9 area, the association between the beacon and the file will be stored in the cloud, and the beacon map will calculate the relative direction and position of other beacon devices.

f:id:fengchihsheng:20210325142412p:plain
Saving

Load ARWorldMap

The next time the user loads the data, the direct relationship between the target location and the beacon device will be displayed to assist the user in reaching the destination.

f:id:fengchihsheng:20210325142428p:plain
Loading

Demo

In this demo, you can see the upper right corner is the relative distance (cm) between the user and the three iBeacons, the upper left corner is the photo of iBeacon location, and the plane in the middle shows the direction to the next iBeacon.

f:id:fengchihsheng:20210325143411g:plain
demo

Conclusion

First of all, we use iBeacon technology to locate and navigate. By setting up multiple devices in the environment, we can assist each other to locate and navigate to achieve the goal.

Next, we tried to use the most common Wi-Fi technology to locate and navigate, but based on the strict regulations of ios, we could not achieve the goal immediately. I will explain the detail next time.

Future work

As far as we know, Apple has launched new products containing LiDAR sensors, which will bring us a new experience. The above systems require time and money to set up wireless devices. We will develop SLAM based on LiDAR devices to save costs. In SLAM, only use the camera to achieve the navigation function.

LiDAR sensor

LiDAR is a method for determining ranges (variable distance) by targeting an object with a laser and measuring the time for the reflected light to return to the receiver.

https://en.wikipedia.org/wiki/Lidar

f:id:fengchihsheng:20210325144954p:plain
LiDAR

https://www.apple.com/jp/

Simultaneous Localization and Mapping (SLAM)

In computational geometry and robotics, simultaneous localization and mapping (SLAM) is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it.

https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping

Type of SLAM

Visual SLAM

As the name suggests, Visual SLAM is a SLAM technology that focuses on images from cameras and image sensors.

Lidar SLAM

This method mainly uses a laser sensor (distance sensor) called Lidar (Light Detection and Ranging).

With the advancement of technology, LiDAR used to be a very expensive and huge device. Now it can be made as small as a mobile phone, and the price is no longer so expensive.

f:id:fengchihsheng:20210325144443p:plain
Comparison of different SLAM

On the left side of the figure, you can see the two-dimensional map and path created by LiDAR SLAM, and the right one is the three-dimensional map.

f:id:fengchihsheng:20210325145111p:plain
Other use case use LiDAR SLAM result

https://jp.mathworks.com/discovery/slam.html

Other

Tell us what do you think about our result or anything else that comes to mind.

We welcome all of your comments and suggestions.

Reference

ARKit
ARKit Compass Rose
ARKitCompass