Introduction

We analyze the discontinuity preserving problem in TV-L¹ optical flow methods. This kind of methods typically creates rounded effects at flow boundaries, which usually do not coincide with object contours. A simple strategy to overcome this problem consists in inhibiting the diffusion at high image gradients. In this work, we first introduce a general framework for TV regularizers in optical flow and relate it with some standard approaches. Our survey takes into account several methods that use decreasing functions for mitigating the diffusion at image contours. Consequently, this kind of strategies may produce instabilities in the estimation of the optical flows. Hence, we study the problem of instabilities and show that it actually arises from an ill-posed formulation. From this study, it is possible to come across with different schemes to solve this problem. One of these consists in separating the pure TV process from the mitigating strategy. This has been used in another work and we demonstrate here that it has a good performance. Furthermore, we propose two alternatives to avoid the instability problems: (i) we propose a fully automatic approach that solves the problem based on the information of the whole image; (ii) we derive a semi-automatic approach that takes into account the image gradients in a close neighborhood adapting the parameter in each position. In the experimental results, we present a detailed study and comparison between the different alternatives. These methods provide very good results, especially for sequences with a few dominant gradients. Additionally, a surprising effect of these approaches is that they can cope with occlusions. This can be easily achieved by using strong regularizations and high penalizations at image contours.

In this page you can find the code for the implementation of a robust variational optical flow method that allow the use of five regularization strategies.

Full text: Monzón López, N, Salgado de la Nuez, A.J. & Sánchez Pérez, J. (2013) Discontinuity Preserving in TV-L¹ Optical Flow Methods.CTIM Technical Report.(5).Universidad de Las Palmas de Gran Canaria.

Theoretical Framework

Given two images in a sequence, I₁, I₂:Ω: ℜ²→ℜ, of gray level values, the optical flow w = (u(x),v(x))^T, puts in correspondence the pixels of the first image with the pixels of the second, with x = (x,y)^T. Functions u(x) and v(x) are the horizontal and vertical displacements, respectively. Variational optical flow techniques rely on a global energy functional. The minimization of this functional provides a solution to the optical flow problem between two frames. Standard approaches typically include an attachment and a regularization term, as follows:
Many different alternatives have been proposed in the literature for each of these terms. In our case, we use L¹ norms in a similar way as in the Brox et al. model. Then, our attachment term reads as


with and ε = 0.0001 a small constant. Henceforth, we will use this scheme for the attachment term. This will remain unaltered in the experimental results and we will enable or disable the second term for convenience.
The regularization term is responsible for the continuity of the computed optical flow. It can be generalized using a tensor matrix as

or

with a convex function and D a 2x2 matrix that allows to steer the diffusion process depending on the structure of the image. The first alternative yields a rotational invariant regularizer, while the second produces a system of decoupled diffusion equations.
The behavior of the smoothing strategy depends on the choices for and D. Next table shows the five strategies implemented.

Table 1. Regularization Strategies

Examples

This subsection shows several tests with using synthetic sequences from two main datasets: a set of images with geometric figures, like circles, squares, stars or rectangles, presenting pronounced discontinuities and large displacements and sequences from the Middlebury benchmark database. These sequences are Grove2 and Hydrangea. The optical flows are represented using the color scheme shown in Fig. 1. The color represents the orientation and the intensity the magnitude, similar to the vector field on the left.

Fig. 1. Color scheme used to represented the optical flows. The colors represent the orientation and magnitude of the flow field.

The next figures show the solutions obtained with the different methods using several geometric and Middlebury sequences. The aim of the experiments is to analyze the influence of the regularization in the preservation of discontinuities. Therefore, our study will basically concentrate on the α and λ parameters. In the majority of the experiments, γ has been set to zero. We show three results for each method in these figures: first, a typical solution obtained with a regular configuration of the α and λ parameters; then, a good solution looking for better parameters; and, finally, a limiting situation where we observe the appearance of outliers or very regularized solutions (large values of α and λ).

Fig. 2. From top to bottom: Solutions of the Brox et al., Exponential, Exponential Β, Exponential λ-Global and Exponential λ-Local methods, respectively. The columns show increasing values for α and λ.

Fig. 2. From top to bottom: Solutions of the Brox et al., Exponential, Exponential Β, Exponential λ-Global and Exponential λ-Local methods, respectively. The columns show increasing values for α and λ.

Fig. 3. From top to bottom: Solutions of the Brox et al., Exponential, Exponential Β, Exponential λ-Global and Exponential λ-Local methods, respectively. The columns show increasing values for α and λ.

Fig. 4. From top to bottom: Solutions of the Brox et al., Exponential, Exponential Β, Exponential λ-Global and Exponential λ-Local methods, respectively. The columns show increasing values for α and λ.

References

1. Javier Sánchez, Agustín Salgado and Nelson Monzón. Preserving Accurate Motion Countours with Reliable Parameter Selection IEEE International Conference on Image Processing (ICIP2014), pages 209-213.
2. Nelson Monzón, Javier Sánchez and Agustín Salgado. Efficient Mechanism for Discontinuity Preserving in Optical Flow Methods 6th International Conference, ICISP 2014, Cherbourg, France, June 30 – July 2, 2014. pages 425-432.
3. Nelson Monzón, Javier Sánchez and Agustín Salgado. Optic Flow: Improving Discontinuity Preserving 14th International Conference, Las Palmas de Gran Canaria, Spain, February 10-15, 2013. pages 117-124.
4. Javier Sánchez, Nelson Monzón and Agustín Salgado. Robust Optical Flow Estimation Image Processing On Line (IPOL), 3 (2013), pp. 252–270. http://dx.doi.org/10.5201/ipol.2013.21
5. Javier Sánchez, Agustín Salgado and Nelson Monzón. Optical flow estimation with consistent spatio-temporal coherence models. In International Conference on Computer Vision Theory and Applications (VISAPP), pages 366–370. Institute for Systems and Technologies of Information, Control and Communication, 2013.

Acknowledgements

This work has been partially supported by the Spanish Ministry of Science and Innovation through the research project TIN2011-25488 and the grant ULPGC011-006.