Abstract
This paper presents the theory of motion-compensated spatiotemporal filtering of time-varying imagery. The properties of motion trajectories and their relation to displacement fields and velocity fields are presented. The constraints that image motion places on the time-varying image in both the spatiotemporal domain and in the frequency domain are described, along with the implications of these results on motion-compensated filtering and on sampling. An iterative method for estimating motion which generalizes many pixel-oriented and block-oriented methods is presented. Motion-compensated filtering is then applied to the problems of prediction, interpolation, and smoothing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
E. Dubois, “The Sampling and Reconstruction of Time-Varying Imagery with Application in Video Systems,”Proceedings of the IEEE, vol. 73, 1985, pp. 502–522.
B. Horn and B. Schunck, “Determining Optical Flow,”Artificial Intelligence, vol. 17, 1981, pp. 185–203.
R. Paquin and E. Dubois, “A Spatiotemporal Gradient Method for Estimating the Displacement Field in Time-Varying Imagery,”Computer Vision, Graphics and Image Processing, vol. 21, 1983, pp. 205–221.
L. Jacobson and H. Wechsler, “Derivation of Optical Flow using a Spatiotemporal-Frequency Approach,”Computer Vision, Graphics and Image Processing, vol. 38, 1987, pp. 29–65.
L. Cohen, “Time-Frequency Distributions—A Review,”Proceedings of the IEEE, vol. 77, 1989, pp. 941–981.
B. Girod and R. Thoma, “Motion-Compensating Field Interpolation from Interlaced and Non-interlaced Grids,” inSPIE Vol. 594 Image Coding, 1985, pp. 186–193.
M. Bertero, T. Poggio, and V. Torre, “Ill-Posed Problems in Early Vision,”Proceedings of the IEEE, vol. 76, 1990, pp. 869–889.
J. Konrad and E. Dubois, “Estimation of Image Motion Fields: Bayesian Formulation and Stochastic Solution,” inProceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 1988, pp. 1072–1075.
J. Konrad and E. Dubois, “Bayesian Estimation of Discontinuous Motion in Images Using Simulated Annealing,” inProceedings on Vision Interface, 1989, pp. 354–363.
C. Bergeron and E. Dubois, “Gradient-Based Algorithms for Block-Oriented Map Estimation of Motion and Application to Motion-Compensated Temporal Interpolation,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 1, 1991, pp. 72–85.
J. Jain and A. Jain, “Displacement Measurement and its Application in Interframe Image Coding,”IEEE Transactions on Communications, vol. COM-29, 1981, pp. 1799–1808.
S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-6, 1984, pp. 721–741.
J. Konrad and E. Dubois “Comparison of Stochastic and Deterministic Solution Methods in Bayesian Estimation of 2D Motion,”Image and Vision Computing, vol. 8, 1990, pp. 304–317.
T. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, “Motion-Compensated Interframe Coding for Video Conferencing,” inConf. Rec. National Telecommunications Conference, 1981, pp. G5.3.1–G5.3.5.
P. Gill, W. Murray, and M. Wright,Practical Optimization, New York: Academic Press, 1981.
R. Keys, “Cubic Convolution Interpolation for Digital Image Processing,”IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. ASSP-29, 1981, pp. 1153–1160.
G. Golub and C. van Loan,Matrix Computations, Baltimore, MD: The Johns Hopkins University Press, 1985.
P. Anandan, “A Unified Perspective on Computational Techniques for the Measurement of Visual Motion,” inProceedings of the IEEE International Conference on Computer Vision ICCV '87, 1987, pp. 219–230.
F. Glazer, G. Reynolds, and P. Anandan, “Scene Matching by Hierarchical Correlation,” inProceedings of the IEEE Conference on Computer Vision Pattern Recognition CVPR '83, 1983, pp. 432–441.
W. Enkelmann, “Investigations of Multigrid Algorithms for the Estimation of Optical flow Fields in Image Sequences,”Computer Vision, Graphics, and Image Processing, vol. 43, 1988, pp. 150–177.
F. Glazer,Hierarchical Motion Detection, Ph.D. thesis, University of Massachusetts, 1987.
C. Cafforio and F. Rocca, “Methods for Measuring Small Displacements of Television Images,”IEEE Transactions on Information Theory, vol. IT-22, 1976, pp. 573–579.
A. Netravali and J. Robbins, “Motion-Compensation Television Coding: Part I,”Bell System Technical Journal, vol. 58, 1979, pp. 631–670.
H. Nguyen and E. Dubois, “Representation of Motion Fields for Image Coding,” inProceedings of the 1990 Picture Coding Symposium, 1990, pp. 8.4-1–8.4-5.
T. Reuter, “Standards Conversion Using Motion Compensation,”Signal Processing, vol. 16, 1989, pp. 73–82.
R. Depommier, “Estimation de Mouvement Considérant les Phénomènes d'Occlusion pour le Codage Interpolatif des Séquences d'Images,” Master's thesis, INRS-Telecommunications, 1990.
E. Dubois and S. Sabri, “Noise Reduction in Image Sequences Using Motion-Compensated Temporal Filtering,”IEEE Transactions on Communications, vol. COM-32, 1984, pp. 826–831.
Author information
Authors and Affiliations
Additional information
Invited Paper
Rights and permissions
About this article
Cite this article
Dubois, E. Motion-compensated filtering of time-varying images. Multidim Syst Sign Process 3, 211–239 (1992). https://doi.org/10.1007/BF01942043
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01942043