RGBD SLAM using Intel RealSense camera

From RobotinoWiki

Simultaneous Localization and Mapping (SLAM) (see also on Wikipedia) is used in buildings, which have no previous mapping and do not receive a global position with GPS.

Nearly every sensor, which is able to detect the surrounding area, can be used for SLAM. Laser scanner or cameras are a few examples. In this paragraph we are using a 3D camera, which provides a visual picture (RGB) and the depth (D) picture. The depth information can be determined through stereo-processes, through sensors or through triangulation. Well known 3D cameras are the Kinect camera and the Intel RealSense camera series. Here, we are using an Intel RealSense R200 camera, which provides depth information through a combination of a stereo camera and an infrared (IR) laser projector system.

The homepage openSLAM.org is a good choice to get an overview of different SLAM processes. In this article we are using rtabmap. This algorithm is actively further developed and is supported by all present ROS (Robotic Operating System). The winner of the IROS 2014 Microsoft Connect Challenge used this particular algorithm.


Preconditions

  • Ubuntu 14.04 or 16.04
  • Intel RealSense R200 camera (has to be plugged in an USB 3 port!)
  • existing internet connection
  • Successful installation of Intel RealSense driver according to the manual Driver installation for Intel RealSense camera

Installation

The assembled packet makes the installation more simple, it supports a RealSense R200 camera with an installation script for a rtabmap in ROS. The installation follows these steps:

  • Installation of Robotino API2 with the packet source http://packages.openrobotino.org, if they are not yet installed.
  • Installation of the official ROS packets from http://packages.ros.org/ros/ubuntu.
  • Creation of a sandbox rtabmap4robotino-ros in the user file system with the following elements:
    • ROS driver of the Intel RealSense R200 camera
    • ROS link to Robotino with the packet robotino_node as well a patch to build the robotino_node on recent ROS variations
    • an up to date Robotino Model.
    • The ROS paket rtabmap4robotino which provides launch files for ROS.
    • Script enter_env.sh to set the new environment variables in ROS in a new shell.


For the following installation process, an existing internet connection is required. The following steps need to be executed in a terminal: 

wget http://doc.openrobotino.org/download/jebens/rtabmap4robotino.tar.gz
tar xfz rtabmap4robotino.tar.gz
cd rtabmap4robotino
./install_rtabmap.sh

After a successful installation, the ROS sandbox is located in the directory rtabmap4robotino-ros: 

cd ~/rtabmap4robotino-ros

First steps in ROS sandbox

The script enter_env.sh is installed to the ROS sandbox directory for an easier usage. This script sets the required environment variables, enables ROS commands and their tab completion.

To activate the ROS sandbox, the following commands need to be executed: 

cd ~/rtabmap4robotino-ros
./enter_env.sh

Now the sandbox environment will appear in the terminal. Notice that the prompt of the shell will change to a label with [ROS].

Different launch files to start ROS have been prepared for this environment. These contain the ROS system configuration, which consists of multiple parts in general. The individual launch files are part of the corresponding ROS packet. In this case the ROS packet is called rtabmap4robotino. For example, with the following command, you can start the Intel RealSense R200 camera and show a 3D point cloud with the program rviz: 

roslaunch rtabmap4robotino realsense_live_display.launch

Tab completion helps choosing the right ROS packets (rtabmap4robotino) and also the corresponding launch files (realsense_live_display.launch). To get a name completion press tab once, to get a list with all possibilities press tab twice.

Provided ROS configurations

The ROS packet rtabmap4robotino contains ROS configurations for different areas of the rtabmap software. These are further explained in the next sections.


Usage without a robot

To get to know the software, the following commands can be used:

  • realsense_live_display.launch
  • rtabmap_start_mapping.launch
  • rtabmap_continue_mapping.launch
  • rtabmap_localization.launch

They use information from the Intel RealSense R200 camera. A connection to the robot is not required. The required odometry for the SLAM algorithm is calculated from the camera pictures.

The first configuration realsense_live_display.launch starts the camera and a rviz display, which visualizes the data. This configuration is for the first test of the Intel RealSense camera.

The configuration rtabmap_start_mapping.launch starts also the camera and the rtabmap software which creates a new map (Attention: Your existing map can be overwritten!). During capturing there is a graphical view which shows you the current status of the SLAM algorithm. Also the current 3D-Map, possible loop-closures and the status of the visual odometry are shown.

To close the configuration, just press Ctrl-c. The created map will be saved to ~/.ros/rtabmap.db automatically.

To continue the mapping, use the configuration rtabmap_continue_mapping.launch. This configuration differs to the configuration rtabmap_start_mapping.launch, by not deleting the previously generated map during start. The newly added pictures are automatically updated to the old map, if the loop-closure has been successful.

To use the localization without updating the map, use the configuration rtabmap_localization.launch. To see the old map in the visualization, select it in the menu Edit -> Download.


Usage of Robotino’s odometry

The configurations

  • robotino_realsense_live_display.launch
  • robotino_rtabmap_start_mapping.launch
  • robotino_rtabmap_continue_mapping.launch
  • robotino_rtabmap_localization.launch

use the odometry provided by Robotino, which is usually more precise than the visual odometry based on the Intel RealSense R200 camera. Due to the Robotino API2 the configuration can be directly started on Robotino and as well on a different computer, which is connected and also receives information from Robotino. For this use the environment variable ROBOTINO_IP. Before starting the ROS configuration, this variable has to be set: 

export ROBOTINO_IP=172.26.1.1
roslaunch rtabmap4robotino robotino_rtabmap_start_mapping.launch

In case the environment variable is not set, the connection will be set to the local host (127.0.0.1).

Separating camera and SLAM calculation

If the capturing and SLAM calculation is desired to run on two different computers, then the following camera computer configuration can be used:

  • realsense_sender.launch

On the receiving computer any one of the following commands can be used:

  • remote_camera_rtabmap_start_mapping.launch
  • remote_camera_rtabmap_continue_mapping.launch
  • remote_camera_rtabmap_localization.launch

The transmitting side is reducing the frame rate and the receiving side receives only compressed files.

To connect the transmitting and receiving side, the environment variable ROS_MASTER_URI needs to be set on any side. For example, if the camera computer IP is 192.168.1.1 and receiving computer IP is 192.168.1.2, then the camera computer ROS can be started as follows: 

roslaunch rtabmap4robotino realsense_sender.launch

On the receiving computer first the environment variable ROS_MASTER_URI has to be set, followed by the desired configuration: 

export ROS_MASTER_URI=http://192.168.1.1:11311
roslaunch rtabmap4robotino remote_camera_rtabmap_start_mapping.launch

Playback of ROS-Bags

The configurations

  • playback_realsense_display.launch
  • playback_rtabmap_start_mapping.launch
  • playback_rtabmap_continue_mapping.launch
  • playback_rtabmap_localization.launch

are similar to the configuration of the chapter „Usage without a robot“. However, they do not start the Intel RealSense R200 camera. Here it is expected that the picture information is provided by a record of ROS, which has to be started in parallel to the configuration.

Inspection of the card

rtabmap has a tool rtabmap-databaseViewer to inspect the card, which will be started with the command: 

rtabmap-databaseViewer ~/.ros/rtabmap.db
Retrieved from "https://wiki.openrobotino.org/index.php?title=RGBD_SLAM_using_Intel_RealSense_camera&oldid=3822"