Definition
Def Optical Flow Optical flow is the pattern of apparent motion of image objects between two consecutive frames caused by the movement of object or camera. It is 2D vector field where each vector is a displacement vector showing the projection of velocity of points in the real world to the image plane.
Theorem
Thrm Brightness Constancy Equation The brightness constancy equation states that the intensity of a pixel does not change between two consecutive frames:
Under first order Taylor expansion, we have: Therefore, we have the brightness constancy equation: where is the optical flow vector.
Algorithm
Algorithm Lucas–Kanade’s Algorithm Assume constant small motion within a small neighbourhood, and brightness constancy is satisfied. Then a
image patch gives us brightness constancy equations. We can stack them into a matrix form: Then the least square solution is given by:
Remark
Indeed, we saw the matrix
in the Harris corner detector. Harris corners are where eigenvalues and are both big, this is also when Lucas–Kanade optical flow estimation works best. It implies that corners are good places to compute optical flow.
Algorithm
Algorithm Coarse-to-Fine Optical Flow Estimation The coarse-to-fine optical flow estimation is based on Gaussian pyramid. The algorithm is as follows:
- Construct Gaussian pyramid for both images.
- Start from the coarsest level, estimate optical flow.
- Upsample the optical flow to the next finer level.
- Refine the optical flow estimate at the finer level.
- Repeat until the original image level is reached.
- The final optical flow is the refined estimate at the original image level.