Kalibr installation tutorial

I was confused about installing Kalibr, but there is no even one hint in README.md. I just put them in the catkin_ws, in which so many ROS packages are also there. Unsuccessfully, it can't be compiled one by one package by the command catkin_make -DCATKIN_WHITELIST_PACKAGE="PACKAGE_NAME". It means a good choice is to build another ROS workspace in case of rebuilding others in the same workspace.

Resiquite:

sudo apt-get install python-setuptools python-rosinstall ipython libeigen3-dev libboost-all-dev doxygen libopencv-dev ros-kinetic-vision-opencv ros-kinetic-image-transport-plugins ros-kinetic-cmake-modules python-software-properties software-properties-common libpoco-dev python-matplotlib python-scipy python-git python-pip ipython libtbb-dev libblas-dev liblapack-dev python-catkin-tools libv4l-dev

sudo pip install python-igraph --upgrade

Warning: If having done catkin_make at first then must run the following command.

catkin clean -bdy

cd ~

mkdir -p kalibr_ws/src

cd ~/kalibr_ws

source /opt/ros/kinetic/setup.bash

catkin init

catkin config --extend /opt/ros/kinetic

catkin config --merge-devel # Necessary for catkin_tools >= 0.4. catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release

cd ~/kalibr_ws/src

git clone https://github.com/ethz-asl/kalibr.git

cd ..

catkin build -DCMAKE_BUILD_TYPE=Release -j4

Output seems like this:

Finished  <<< kalibr                                        [ 16.1 seconds ]

[build] Summary: All 37 packages succeeded!

[build]   Ignored:   None.

[build]   Warnings:  21 packages succeeded with warnings.

[build]   Abandoned: None.

[build]   Failed:    None.

[build] Runtime: 14 minutes and 53.4 seconds total.

[build] Note: Workspace packages have changed, please re-source setup files to use them.

source ~/kalibr_ws/devel/setup.bash

Update:

Traceback (most recent call last):

  File "../python/kalibr_calibrate_cameras", line 6, in <module>

    import sm

ImportError: No module named sm

**Solution: **

sudo pip install sm

then rebuild kalibr.

References:

[1] kalibr教程

[2] Installing and Configuring Your ROS Environment

[3] ethz-asl/kalibr

[4] catkin_make vs catkin build

[5] https://github.com/ethz-asl/kalibr/wiki/installation

[6] 完整版用kalibr标定 camera imu

Multiple camera calibration

roslaunch realsense2_camera rs_camera.launch

rosrun topic_tools throttle messages /camera/color/image_raw 4.0 /color

rosbag record -O rs_cam_hz4 /color

Which distortiong model should be choose for Realsense D435i? From all I know, a factory calibration setup of D435i looks like: (You can /usr/local/bin/rs-sensor-control, type 0, 1, 2, 91 etc to see)

Principal Point         : 322.424, 237.813

Focal Length            : 617.521, 617.576

Distortion Model        : Brown Conrady

Distortion Coefficients : [0,0,0,0,0]

And according to the dorodnic, of course a equidistant distortion model could be used. (But r1 & r2 are needed in realsense comfig in vins. So the best distortion model must be radial-tangential (radtan))

Yes, these are supposed to be zero for the D400. We consider adding coefficient estimation to the RGB calibration to reduce the distortion (by about 1 pixel at extremes), but at the moment projection without coefficients is the most accurate you can do (we are not calibrating and then ignoring the coefficients, we estimate fx, fy, ppx and ppy without them)

cd ~/kalibr_ws/src/kalibr/aslam_offline_calibration/kalibr/data

../python/kalibr_calibrate_cameras --target april_6x6_50x50cm.yaml --bag rs_cam_hz4.bag --models pinhole-equi --topics /color

Note that in the bag file there are up to 800 images, but it only 39. Maybe that's enough for calibration?

Output:

Calibration complete.

[ WARN] [1556719991.003758]: Removed 26 outlier corners.

Processed 826 images with 39 images used

Camera-system parameters:

	cam0 (/color):

	 type: <class 'aslam_cv.libaslam_cv_python.EquidistantDistortedPinholeCameraGeometry'>

	 distortion: [  0.3044413    2.04741574 -11.06112629  18.6743852 ] +- [ 0.0320288   0.46759766  2.76374537  5.41971393]

	 projection: [ 604.9671891   602.10506316  325.8395051   238.35406753] +- [ 10.62286295  10.41921913   1.68531874   1.43868064]

	 reprojection error: [-0.000000, -0.000000] +- [0.153693, 0.138547]

Results written to file: camchain-rs_cam_hz4.yaml

  Detailed results written to file: results-cam-rs_cam_hz4.txt

Result:

camchain-rs_cam_hz4.yaml

cam0:

  cam_overlaps: []

  camera_model: pinhole

  distortion_coeffs: [0.3044413037380324, 2.0474157424478348, -11.061126286843251,

    18.67438520203368]

  distortion_model: equidistant

  intrinsics: [604.9671890973748, 602.1050631617551, 325.83950509989114, 238.35406753467785]

  resolution: [640, 480]

  rostopic: /color

Compared to the default settings, assumes our result is accurate. The reprojection error seems like good too.

Reference:

[1] Multiple camera calibration

[2] [相机标定]RealSense D435i相机标定

[3] rs2_intrinsics coeffs[] all 0 by default #1430

[4] Camera models

IMU calibration

imu_utils from HKUST

Protecting from error:

CMake Warning at /opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:76 (find_package):

Could not find a package configuration file provided by "code_utils" with

any of the following names:

code_utilsConfig.cmake

code_utils-config.cmake

Add the installation prefix of "code_utils" to CMAKE_PREFIX_PATH or set

"code_utils_DIR" to a directory containing one of the above files. If

"code_utils" provides a separate development package or SDK, be sure it has

been installed.

Put code_utils in the workspace, catkin_make first.

Then do the same for imu_utils.

Result (BMI055 is the IMU D435i is using):

BMI055_imu_param.yaml

%YAML:1.0

---

type: IMU

name: BMI055

Gyr:

   unit: " rad/s"

   avg-axis:

      gyr_n: 6.0673370376614875e-03

      gyr_w: 3.6211951458325785e-05

   x-axis:

      gyr_n: 5.4501442406047970e-03

      gyr_w: 4.0723401163659986e-05

   y-axis:

      gyr_n: 5.9380128602687073e-03

      gyr_w: 2.9388325769986972e-05

   z-axis:

      gyr_n: 6.8138540121109601e-03

      gyr_w: 3.8524127441330383e-05

Acc:

   unit: " m/s^2"

   avg-axis:

      acc_n: 3.3621979208052800e-02

      acc_w: 9.8256589971851467e-04

   x-axis:

      acc_n: 3.6095477320173631e-02

      acc_w: 9.6831827726998488e-04

   y-axis:

      acc_n: 3.4696437020780901e-02

      acc_w: 1.3092042863834673e-03

   z-axis:

      acc_n: 3.0074023283203882e-02

      acc_w: 6.7017513550209160e-04

[1] imu标定 imu_tk

[2] Imu_tk算法流程

[3] catkin_make failed #3

[4] imu_utils

[5] code_utils

camera/IMU calibration

roscd realsense2_camera/

roslaunch realsense2_camera rs_camera.launch

rostopic hz /camera/imu

rostopic hz /camera/color/image_raw

rosrun topic_tools throttle messages /camera/color/image_raw 20.0 /color

rosrun topic_tools throttle messages /camera/imu 200.0 /imu

Some problem:

In the rs_camera.launch, but when I check the frequency: IMU is 150 Hz and the camera is 15FPS. It can't be slow down to the frequency needed.

  <arg name="color_fps"           default="30"/>

  <arg name="gyro_fps"            default="200"/> <!-- 200 or 400-->

  <arg name="accel_fps"           default="250"/> <!--  63 or 250-->

The best frequency is 200 Hz and 30 Hz. Of course, others are still good.

rosbag record -O rs_cam15hz_imu150hz.bag /color /imu

camchain-rs_cam_hz4.yaml

cam0:

  cam_overlaps: []

  camera_model: pinhole

  distortion_coeffs: [0.3044413037380324, 2.0474157424478348, -11.061126286843251,

    18.67438520203368]

  distortion_model: equidistant

  intrinsics: [604.9671890973748, 602.1050631617551, 325.83950509989114, 238.35406753467785]

  resolution: [640, 480]

  rostopic: /color

imu.yaml

rostopic: /imu

update_rate: 150.0 #Hz

accelerometer_noise_density: 3.3621979208052800e-02 #continous

accelerometer_random_walk: 9.8256589971851467e-04

gyroscope_noise_density: 6.0673370376614875e-03 #continous

gyroscope_random_walk: 3.6211951458325785e-05

roscd kalibr

cd data

cp ~/catkin_ws/src/realsense/realsense2_camera/rs_cam15hz_imu150hz.bag .

 ../python/kalibr_calibrate_imu_camera --target april_6x6_50x50cm.yaml --cam camchain-rs_cam_hz4.yaml --imu imu-BMI055.yaml --bag rs_cam15hz_imu150hz.bag

Note that when something is wrong with the input data in bagfile, just record another one bagfile.

Initializing

Optimization problem initialized with 101968 design variables and 1079428 error terms

The Jacobian matrix is 2310198 x 458841

[0.0]: J: 1.35165e+06

Exception in thread block: [aslam::Exception] /home/william/kalibr_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [1.55677e+09 <= 1.55677e+09 < 1.55677e+09]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!

Exception in thread block: [aslam::Exception] /home/william/kalibr_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [1.55677e+09 <= 1.55677e+09 < 1.55677e+09]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!

Exception in thread block: [aslam::Exception] /home/william/kalibr_ws/src/kalibr/aslam_nonparametric_estimation/aslam_splines/src/BSplineExpressions.cpp:447: toTransformationMatrixImplementation() assert(_bufferTmin <= _time.toScalar() < _bufferTmax) failed [1.55677e+09 <= 1.55677e+09 < 1.55677e+09]: Spline Coefficient Buffer Exceeded. Set larger buffer margins!

[ERROR] [1556773048.921808]: Optimization failed!

Traceback (most recent call last):

  File "../python/kalibr_calibrate_imu_camera", line 236, in <module>

    main()

  File "../python/kalibr_calibrate_imu_camera", line 206, in main

    iCal.optimize(maxIterations=parsed.max_iter, recoverCov=parsed.recover_cov)

  File "/home/william/kalibr_ws/src/kalibr/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccCalibrator.py", line 179, in optimize

    raise RuntimeError("Optimization failed!")

RuntimeError: Optimization failed!

Result looks like this:

After Optimization (Results)

==================

Normalized Residuals

----------------------------

Reprojection error (cam0):     mean 0.169417479013, median 0.154212672023, std: 0.0973946838993

Gyroscope error (imu0):        mean 0.18574054756, median 0.159830346682, std: 0.115913332564

Accelerometer error (imu0):    mean 0.169497068217, median 0.145829709726, std: 0.10939033445

Residuals

----------------------------

Reprojection error (cam0) [px]:     mean 0.169417479013, median 0.154212672023, std: 0.0973946838993

Gyroscope error (imu0) [rad/s]:     mean 0.013802268496, median 0.0118768970357, std: 0.00861345010194

Accelerometer error (imu0) [m/s^2]: mean 0.0697960902289, median 0.0600502633182, std: 0.0450451310679

Transformation T_cam0_imu0 (imu0 to cam0, T_ci):

[[ 0.01542341 -0.99976267  0.01538561  0.00713584]

 [ 0.03147917 -0.01489429 -0.99939343 -0.03487332]

 [ 0.9993854   0.01589838  0.03124198 -0.05266484]

 [ 0.          0.          0.          1.        ]]

cam0 to imu0 time: [s] (t_imu = t_cam + shift)

0.0334634768386

IMU0:

----------------------------

  Model: calibrated

  Update rate: 150.0

  Accelerometer:

    Noise density: 0.0336219792081

    Noise density (discrete): 0.411783466011

    Random walk: 0.000982565899719

  Gyroscope:

    Noise density: 0.00606733703766

    Noise density (discrete): 0.0743093991988

    Random walk: 3.62119514583e-05

  T_i_b

    [[ 1.  0.  0.  0.]

     [ 0.  1.  0.  0.]

     [ 0.  0.  1.  0.]

     [ 0.  0.  0.  1.]]

  time offset with respect to IMU0: 0.0 [s]

  Saving camera chain calibration to file: camchain-imucam-rs_cam15hz_imu150hz.yaml

  Saving imu calibration to file: imu-rs_cam15hz_imu150hz.yaml

  Detailed results written to file: results-imucam-rs_cam15hz_imu150hz.txt

Generating result report...

/home/william/kalibr_ws/src/kalibr/Schweizer-Messer/sm_python/python/sm/PlotCollection.py:57: wxPyDeprecationWarning: Using deprecated class PySimpleApp.

  app = wx.PySimpleApp()

  Report written to report-imucam-rs_cam15hz_imu150hz.pdf

References:

[1] Kalibr 标定双目内外参数以及 IMU 外参数

[2] [相机标定]RealSense D435i相机标定

[3] Problem with single imu and single cam Optimization failed #223