Robotics Perception

课程链接：https://www.coursera.org/learn/robotics-perception/

Week 1

Pinhole Model

Assuming image plane is in front of the lens

$$y=f\frac{Y}{Z}$$

Vanishing Points

• Any two parallel lines have the same vanishing point
• The ray from C through v point is parallel to the lines
• An image may have more than one vanishing point

Vanishing Lines

• Any set of parallel lines on the plane define a vanishing point
• The union of all of these vanishing points is the horizon line (also called vanishing line)
• Note that different planes define different vanishing lines

Point and Line Duality

• Line: $l=x\times x^{\prime }$
• Point: $x=l\times l^{\prime }$
• Points and lines are dual in projective space: given any formula, can switch the meanings of points and lines to get another formula

Point at Infinity

Line $l=(a,b,c{)}^T$ intersects at $(b,-a,0{)}^T$.

Perspective Projection

1. Camera Projection Matrix

$$Z_c\left[\begin{array}{c}{x}^{\prime }\\ y^{\prime }\\ 1\end{array}\right]=\left[\begin{array}{cccc}f& 0& 0& 0\\ 0& f& 0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{c}{X}_c\\ Y_c\\ Z_c\\ 1\end{array}\right]$$

2. Intrinsic Camera Parameters

$$\left[\begin{array}{c}x\\ y\\ z\end{array}\right]=\left[\begin{array}{ccc}{\alpha }_x& s& p_x\\ 0& {\alpha }_y& p_y\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}{x}^{\prime }\\ y^{\prime }\\ z^{\prime }\end{array}\right]$$

• from optical world ( $x^{\prime },y^{\prime },z^{\prime }$) to pixel world ($x,y,z$)
• ${\alpha }_x,{\alpha }_y$: pixel scaling factor
• $p_x,p_y$: principle point
• $s$: slant factor

3. 3rd Person to 1st Person 3D Mapping (World to Camera)

$$X_c=\left[\begin{array}{cc}{R}_{3\times 3}& t_{3\times 1}\\ 0& 1\end{array}\right]X$$

World coordinates to camera coordinates

$$x=K_{3\times 3}[I;0{]}_{3\times 4}{X}_c$$

Camera coordinates to pixel coordinates

$$x=K[R,t]X$$

Combine 1, 2, 3 here, we have

$$Z\left[\begin{array}{c}{U}_{img}\\ V_{img}\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& s& p_x\\ & f_y& p_y\\ & & 1\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_1\\ r_{21}& r_{22}& r_{23}& t_2\\ r_{31}& r_{32}& r_{33}& t_3\end{array}\right]\left[\begin{array}{c}X\\ Y\\ Z\\ 1\end{array}\right]$$

or

$$\left[\begin{array}{c}x\\ 1\end{array}\right]=L\left(\begin{array}{c}K\left[Rt\right]\left[\begin{array}{c}X\\ 1\end{array}\right]\end{array}\right)$$

Camera Calibration

Calibration estimates intrinsic parameters

• $f$ focal length
• $(u_0,v_0)$ image center
• $k_1,k_2,\cdots$ radical distortion parameters

Week 3

RANSAC

Problem of Minimum Square Error: outliers will affect result significantly.