Scale space theory is a framework for multi-scale signal representation developed by the computer vision, image processing and signal processing communities. It is a formal theory for handling image structures at different scales in such a way that fine-scale features can be successively suppressed and a scale parameter $t$ can be associated with each level in the scale-space representation.

The notion of scale-space is general and applies in arbitrary dimensions. For simplicity of presentation, however, we here describe the most commonly used case with two-dimensional images. For a given image $f(x,\; y)$, its linear scale-space representation is a family of derived signals $L(x,\; y,\; t)$ defined by convolution of $f(x,\; y)$ with the Gaussian kernel

$G(x,\; y,\; t)\; =\; \backslash frac\; \{1\}\{2\{\backslash pi\}\; t\}e^\{-(x^2+y^2)/2t\}.\backslash ,$

such that

$L(x,\; y,\; t)\backslash \; =\; g(x,\; y,\; t)\; *\; f(x,\; y).$

where $t\; =\backslash sigma^2$ is the variance of the Gaussian. Equivalently, the scale-space family can be generated from the solution of the heat equation

$\backslash partial\_t\; L\; =\; \backslash frac\{1\}\{2\}\; \backslash nabla^2\; L$

with initial condition $L(x,\; y,\; 0)\; =\; f(x,\; y)$. Several different derivations have been expressed showing that this is the canonical way to generate a linear scale-space, based on the essential requirement that new structures must not be created from a fine scale to any coarser scale Witkin, A. P. "Scale-space filtering", Proc. 8th Int. Joint Conf. Art. Intell., Karlsruhe, Germany,1019--1022, 1983.Koenderink, Jan "The structure of images", Biological Cybernetics, 50:363--370, 1984Lindeberg, Tony, Scale-Space Theory in Computer Vision, Kluwer Academic Publishers, 1994, ISBN 0-7923-9418-6Florack, Luc, Image Structure, Kluwer Academic Publishers, 1997.Sporring, Jon et al (Eds), Gaussian Scale-Space Theory, Kluwer Academic Publishers, 1997.Romeny, Bart ter Haar, Front-End Vision and Multi-Scale Image Analysis, Kluwer Academic Publishers, 2003.. Conditions,referred to as scale-space axioms, that have been used for deriving the uniqueness of the Gaussian kernel include linearity, shift-invariance, semi-group structure, non-enhancement of local extrema, scale invariance and rotational invariance.

The motivation for generating a scale-space representation of a given data set originates from the basic fact that real-world objects are composed of different structures at different scales. This implies that real-world objects, in contrast to idealized mathematical entities such as points or lines, may appear in different ways depending on the scale of observation. For example, the concept of a "tree" is appropriate at the scale of meters, while concepts such as leaves and molecules are more appropriate at finer scales. For a machine vision system analysing an unknown scene, there is no way to know a prioiri what scales are appropriate for describing the data. Hence, the only reasonable approach is to consider descriptions at all scales simultaneously.

From the scale-space representation, a large variety of image processing and computer vision operations can be expressed, such as feature detection, feature classification, image segmentation, image matching, motion estimation and computation of shape cues, based on (possibly non-linear) combinations of Gaussian derivatives at multiple scales

$L\_\{x^m\; y^n\}(x,\; y,\; t)\; =\; \backslash partial\_\{x^m\; y^n\}\; \backslash left(\; g(x,\; y,\; t)\; *\; f(x,\; y)\; \backslash right).$

A highly useful property of scale-space representation is that image representations can be made invariant to scales, in order to handle size variations that arise from objects of different size or varying distances between the object and the camera. Scale invariance can be achieved by performing scale selection Lindeberg, Tony "Feature detection with automatic scale selection", International Journal of Computer Vision, 30, 2, pp 77--116, 1998.Lindeberg, Tony "Edge detection and ridge detection with automatic scale selection", International Journal of Computer Vision, 30, 2, pp 117--154, 1998. based on local maxima over scales of normalized derivatives

$L\_\{\backslash xi^m\; \backslash eta^n\}(x,\; y,\; t)\; =\; t^\{(m+n)\; \backslash gamma/2\}\; L\_\{x^m\; y^n\}(x,\; y,\; t)$

where $\backslash gamma\; \backslash in*$ is a parameter that is related to the dimensionality of the image feature. Following this approach of gamma-normalization, different types of scale adaptive and scale invariant feature detectors can be expressed for tasks such as blob detection, corner detection, ridge detection and edge detection. In particular, the scale levels obtained from automatic scale selection can be used for determining regions of interest for subsequent affine shape adaptation Lindeberg, T. and Garding, J.: Shape-adapted smoothing in estimation of 3-D depth cues from affine distortions of local 2-D structure, Image and Vision Computing, 15,~415--434, 1997. to obtain affine invariant interest points Baumberg, A.: Reliable feature matching across widely separated views, Proc. Computer Vision Pattern Recognition, I:1774--1781, 2000.Mikolajczyk, K. and Schmid, C.: Scale and affine invariant interest point detectors, Int. Journal of Computer Vision, 60:1, 63 - 86, 2004., or for determining scale levels for computing associated image descriptors, such as locally scale adapted N-jets. Recent work has shown that also more complex operations, such as scale-invariant object recognition can be performed in this way, by computing local image descriptors (N-jets or local histograms of gradient directions) at scale-adapted interest points obtained from scale-space maxima of the normalized Laplacian operator (see also scale-invariant feature transformLowe, D. G., “Distinctive image features from scale-invariant keypoints”, International Journal of Computer Vision, 60, 2, pp. 91-110, 2004.).

The notion of scale-space representation has also been frequently used for expressing coarse-to-fine methods in particular for tasks such as image matching and for multi-scale image segmentation. For technical details when implementing scale-space smoothing in practice, please see the article on scale-space implementation.

Pyramid representation is a predecessor to scale-space representation, constructed by simultaneously smoothing and subsampling a given signalBurt, Peter and Adelson, Ted, "The Laplacian Pyramid as a Compact Image Code", IEEE Trans. Communications, 9:4, 532--540, 1983.Crowley, Jim and Sanderson, A. C. "Multiple resolution representation and probabilistic matching of 2-D gray-scale shape", IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(1), 113-121, 1987.. In this way, computationally highly efficient algorithms can be obtained. In a pyramid, however, it is ususally algorithmically harder to relate structures at different scales, due to the discrete nature of the scale levels. In a scale-space representation, the existence of a continuous scale parameter makes it conceptually much easier to express this so-called deep structure. For features defined as zero-crossings of differential invariants, the implicit function theorem directly defines trajectories across scales, and at those scales where bifurcations occur, the local behaviour can be modelled by singularity theory.

There are interesting relations between scale-space representation and biological vision. Neurophysiological studies have shown that there are receptive field profiles in the mammalian retina and visual cortex, which can be well modelled by linear or non-linear scale-space operatorsYoung, R. A. "The Gaussian derivative model for spatial vision: Retinal mechanisms", Spatial Vision, 2:273--293, 1987.DeAngelis, G. C., Ohzawa, I., and Freeman, R. D., "Receptive-field dynamics in the central visual pathways", Trends Neurosci. 18: 451-458, 1995..

Extensions of linear scale-space theory concern the formulation of non-linear scale-space concepts more committed to specific purposesRomeny, Bart (Ed), Geometry-Driven Diffusion in Computer Vision, Kluwer Academic Publishers, 1994.Weickert, J Anisotropic diffusion in image processing, Teuber Verlag, Stuttgart, 1998.. There are strong relations between scale-space theory and wavelet theory, although these two notions of multi-scale representation have been developed from somewhat different premises. There has also been work on other multi-scale approaches, such as pyramids and a variety of other kernels, that do not exploit or require the same requirements as true scale-space descriptions do.

References

---

See also

---

• scale-space axioms
• scale-space implementation
• scale-space segmentation
• multi-scale approaches
• Gaussian function

• computer vision
• image processing
• wavelets
• edge detection
• blob detection
• corner detection
• ridge detection
• pyramids, image processing
• smoothing

External links

---

• Lindeberg, Tony, "Scale-space: A framework for handling image structures at multiple scales", In: Proc. CERN School of Computing, Egmond aan Zee, The Netherlands, 8-21 September, 1996 (online web tutorial)
• Lindeberg, Tony: Scale-space theory: A basic tool for analysing structures at different scales, in J. of Applied Statistics, 21(2), pp. 224--270, 1994 (longer pdf tutorial)
• Powers of ten interactive Java tutorial at Molecular Expressions website
• On-line resource with space-time receptive fields of visual neurons provided by Izumi Ohzawa at Osaka University
• Lecture on scale-space at the University of Massachusetts (pdf)

skalrum

Image processing | Computer vision