The Uncertainty Principle consists of videos and an interactive application about information and uncertainty in the domain of image analysis.

The various elements of the work are produced by applying certain filters, known as 2D Gabor filters, to images. Gabor filters identify various kinds of edges -- horizontal, vertical, and diagonal – by responding to specific visual frequencies in an image.

Videos

The following three videos were made using Gabor filters. (It is recommended that the videos should be viewed in Vimeo using full-screen and in a relatively dark environment). The first video illustrates one possible function of the Gabor filters, whereas the second and third videos are more self-reflexive, and calls attention to the filters themselves as artistic media. The first video has sound but the second and third are silent.

The first video was made by identifying rain streaks in the opening sequence of Wong Kar-Wai's film The Grandmaster. The Gabor filters helped to detect visual frequencies corresponding to vertical edges on the image. Most other details in the sequence were removed.

The second video was made by repeatedly processing a short segment from Ingmar Bergman’s film Persona (1966) using different Gabor image filters individually or in combination. The results are visualized by highlighting those parts of the image with a high response to the filter being applied, while darkening low-response pixels.

The repetitive structure of the video is meant to encourage viewers to pay close attention, both to the texture of the image and to the distinctive effects of the different edge-detectors. Whereas mainstream narrative cinema encourages the viewer to focus on what happens next, this project draws attention to the details of the individual frame.

The third video applies a set of Gabor filters to every frame in a sequence from the Japanese film The 47 Ronin, made in 1941 by Mizoguchi Kenji. The filters are designed to respond to different kinds of edges (horizontal, vertical, diagonal) in an image. The application of these filters to Mizoguchi's work foregrounds this director's assertively geometric compositions.

 

Interactive Application

The interactive application allows people to explore and experiment with the application of different filters to images in a database. The interactor can choose which filter to apply, and observe the resulting visual description of the image.

The installation consists of two screens.

A selection of filters with different orientations is visible on the right screen. The interactor can select or deselect individual filters by clicking on them. The currently chosen filters become brighter.

The left screen has several parts. On the left is an image from the database. On the right is the activation pattern of the image based on the filters currently selected. The interactor can browse the database and choose other images.

The system is meant to encourage exploratory interaction.

The images selected all contain a high degree of visual clutter. The Gabor filters make manifest different aspects of the graphical composition. For instance, this image shows a Picasso picture where the foreground figure is almost concealed in the background configuration of lines. Choosing different filters affords exploration of different configurations detectable in the image.

The following examples show results of applying different filters to a frame from the film Tom, Tom, The Piper’s Son (Billy Bitzer, 1905).

This film is particularly important because of the history of its subsequent reception by artists and scholars. Avant-garde filmmaker Ken Jacobs made a found-footage work in 1969 by repeatedly re-photographing frames, and upscaling small details of frames, from the original film. The composition of the shot is so rich in detail that it affords close visual exploration. Historians like Noel Burch regard the mise-en-scene of the 1905 film as an alternative to the mainstream Hollywood system of representation. The cluttered composition makes it difficult to identify the central action.

Like the Picasso picture, the frame is rich in visual information.

The application also enables the user to select various parts of the frame and apply different filters to each of them.

Setup

There are different setup possibilities, depending on the resources available.

The work can be shown as three standalone videos. The videos must be shown in a very dark environment, for instance a darkened room, with a projector (minimum: 300 ANSI Lumens).

The three videos can be shown together, as illustrated by the following diagrams:

The application can be exhibited as a two-screen installation (with two monitors or two projectors) together with a mouse on a pedestal.

It is possible to exhibit the videos and the application together as a single work. The following image illustrates one of the possible ways.

If there is available space, some of the images shown in the Background page can also be displayed as digital prints outside the room where the video is displayed.

 

 

A 3-screen exhibtion setup in Hidden Variables, from Writing Machine Collective edition 6. Hong Kong. Sep-Oct 2018.

Background

Several tasks in image processing and analysis, such as for instance segmentation, classification, or edge detection, can be tackled by applying various filers to the image under consideration. The filters are designed to respond to the presence of the features of interest (e.g. edges or corners) in the image.

Just as an audio waveform can be decomposed into elementary sinusoids (simple sine and cosine waves) of specific frequencies, an image can be decomposed into elementary patterns of bands (2D sinusoids). The weights of these different frequency components in any given image account for many visual properties of the image. Edge detection filters can be designed to respond to oriented elements (e.g., horizontal, diagonal, or vertical boundaries) and so can be used to analyze the visual structure of the image.

The following filter, for example, detects vertical edges.

A question that arises in this context is the identification of the best filters for one or more given tasks. A widely used criterion for a good filter is that it should minimize the uncertainty of the description that results from applying it to any given image.

What does uncertainty mean here? In everyday life, we might say things like “Pedro arrived at 6 O’clock, give or take ten minutes”. The expression “give or take” indicates the uncertainty of our knowledge. We do not give a specific time but rather a range or interval. We can think of the word “uncertainty” as an expression of this “give or take” property that pervades our knowledge of the physical world. To speak of uncertainty is to speak of limits to the achievable resolution in our information about a specific kind of signal.

In the case of images, we are interested in identifying edges and their orientations. This aim is accomplished by describing the visual frequencies detectable in specific locations in the image. To tackle this problem is to think of an image from two different viewpoints. First of all, the image is viewed as a set of discrete positions (pixels), each with a certain brightness. Secondly, the image is also viewed as a combination of sinusoidal plane waves. We want to represent the image in both the space domain and the frequency domain, and so to adopt both viewpoints at once. Thus we wish to obtain information with the least possible uncertainty concerning frequency and position.

Frequency uncertainty occurs when instead of detecting the intended frequency we detect a range of frequencies. Position uncertainty occurs, for instance, when an image is blurred, and so the brightness of a given pixel p is replaced with a weighted average of the brightnesses of pixels in a region centered on p.

In general, an uncertainty principle asserts the existence of limits to the precision that can be attained in our knowledge of a complementary pair of physical properties, such as the momentum and position of a particle. The uncertainty principle for still images asserts the existence of a lower bound to the joint position-frequency uncertainty in the representation of an image. Once the theoretical lower bound is attained, any further improvements in one domain (frequency or position) must be paid for with a loss of resolution in the other domain.

The so-called Gabor image filters are designed to minimize the joint position-frequency uncertainty. Different filters attain a better position resolution by sacrificing frequency resolution, or vice versa.

Every Gabor image filter has two components. The first is a blurring filter consisting of a discrete kernel that approximates a continuous Gaussian function. Its uncertainty is determined by the standard deviation of the Gaussian function (and so, by the radius of the corresponding filter). The second filter is a quadrature pair of oriented two-dimensional sinusoids that respond to certain orientations (horizontal, vertical, diagonal) and wavelengths within the image. Its uncertainty depends on the effective bandwidth of the filter. The combined uncertainty is the product of the uncertainties in the two domains. We wish to minimize this combined uncertainty.

The Gaussian components of the following horizontal filters have (from right to left) standard deviation 22.4, 11.2, 5.6, 2.8 and their respective effective bandwidths are 0.02, 0.04, 0.07, 0.14 Hz (cycles per pixel). The lower bandwidths require a higher standard deviation, and vice versa.

If we wish to identify precisely certain frequency-domain properties in an image, we need to sacrifice resolution in the position-domain, i.e., blur the image. We might then know more precisely which frequencies make a strong contribution globally to the image, but we lose precision in our knowledge of the areas of the image wherein those frequencies are active. If we wish to gain more precise location information, we need to sacrifice frequency resolution, i.e, use a filter that responds to a wider range of frequencies.

The following image shows the position-frequency trade-off. The source image is a Siemens Star, often used in video adjustment charts. As the spatial resolution of the image is decreased (as we blur the image), its Fourier representation becomes better localized in frequency.


The top row of this diagram shows the result of applying different Gaussian filters to the image.

Each column shows the application of complete Gabor filters with the same standard deviation, but with different orientations.

The second (fourth, fifth) row then visualizes the Fourier domain decomposition of the result of applying the filter to the Siemens Star picture. In general, the Fourier decomposition is more concentrated as the standard deviation of the Gaussian increases (i.e., the resolution of the decomposition improves as we move towards the right, i.e., as the resolution of the spatial information, the sharpness of the image, decreases).

The third (fifth, seventh) row is the activation pattern that results from applying a filter with a specific orientation to the source image. We can think of this as the result of the analysis.

A note on visualization. The Gabor filters are complex filters. In displaying the results, we show the amplitude spectrum, i.e., the square root of the sum of the real parts squared.

The following diagram shows several frames from Ingmar Bergman’s film Persona and the amplitude spectrum of the result of applying various Gabor filters (with different orientations and bandwidths) to those frames.

 

 

_____________

reference

John G. Daugman, "Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters", Journal of the Optical Society of. America A, 2, 1160-1169 (1985).

Gallery


 


All images used are from the following sources:

https://art.famsf.org/otto-n%C3%BCckel/eighteenth-image-eighteen-burden-chapter-14-book-destiny-novel-pictures-otto-n%C3%BCckel-new

https://shinkwangho.deviantart.com/art/107-Untitled-Charcoal-On-Canvas-53-2-X-41-Cm-376856553

https://shrineodreams.files.wordpress.com/2013/02/ward_woodblock.jpg

https://art.famsf.org/k%C3%A4the-kollwitz/die-freiwilligen-volunteers-second-plate-series-krieg-war-19608524

http://donnamj.dmjstudio.com/wp-content/uploads/2013/08/kollwi02.jpg

http://my.uarts.edu/blog/libraries/files/2011/08/mike-storyteller.jpg

https://www.are.na/block/812078

https://modernism.com/items/1277/art-prints/6463-4292-der-general-der-kaschemme-rudolf-schlicter

http://www.tate.org.uk/art/artists/pierre-alechinsky-639

http://pallant.org.uk/exhibitions/past-exhibitions/2011/delonghi-print-room/german-expressionist-prints

http://www.heyapathy-comics-art.com/pen-artwork.html

http://www.blooddirtandangels.com/index.php/2010/10/24/lynd-ward-six-novels-in-woodcuts/

http://4.bp.blogspot.com/_R-wZWQCr-x8/ShhsUyiuXoI/AAAAAAAABNU/O8xcTEKVttQ/s1600-h/tiempo+de+luto.jpg

https://upload.wikimedia.org/wikipedia/commons/c/cd/Erasmus_censored.png

http://josedanielespejo.lagallaciencia.com/2014/10/

https://ceclirevista.wordpress.com/2015/09/21/pequena-galeria-de-tachaduras/

https://www.wikiart.org/en/pablo-picasso/portrait-of-ambroise-vollard-1910

CREDITS


ARTISTIC & TECHNICAL DIRECTOR

HECTOR RODRIGUEZ



PROGRAMMING

SAM CHAN

with

HECTOR RODRIGUEZ

PHILIP KRETSCHMANN



MATHEMATICAL ADVISOR

FELIPE CUCKER



TEXT

HECTOR RODRIGUEZ



WEB DESIGN

SAM CHAN

HUGO YEUNG

PHILIP KRETSCHMANN