The Uncertainty Principle consists of a video and an interactive application about information and uncertainty in the domain of image analysis.

The various elements of the work are produced by applying certain filters, known as 2D Gabor filters, to images. Gabor filters identify various kinds of edges -- horizontal, vertical, and diagonal – by responding to specific visual frequencies in an image.

Video

The video is made by repeatedly processing a short segment from Ingmar Bergman’s film Persona (1966) using different Gabor image filters individually or in combination. The results are visualized by highlighting those parts of the image with a high response to the filter being applied, while darkening low-response pixels.

The repetitive structure of the video is meant to encourage viewers to pay close attention, both to the texture of the image and to the distinctive effects of the different edge-detectors. Whereas mainstream narrative cinema encourages the viewer to focus on what happens next, this project draws attention to the details of the individual frame.

Interactive Application

The interactive application allows people to explore and experiment with the application of different filters to images in a database. The interactor can choose which filter to apply, and observe the resulting visual description of the image.

The installation consists of two screens.

A selection of filters with different orientations is visible on the right screen. The interactor can select or deselect individual filters by clicking on them. The currently chosen filters become brighter.

The left screen has several parts. On the left is an image from the database. On the right is the activation pattern of the image based on the filters currently selected. The interactor can browse the database and choose other images.

The system is meant to encourage exploratory interaction.

The images selected all contain a high degree of visual clutter. The Gabor filters make manifest different aspects of the graphical composition. For instance, this image shows a Picasso picture where the foreground figure is almost concealed in the background configuration of lines. Choosing different filters affords exploration of different configurations detectable in the image.

The following examples show results of applying different filters to a frame from the film Tom, Tom, The Piper’s Son (Billy Bitzer, 1905).

This film is particularly important because of the history of its subsequent reception by artists and scholars. Avant-garde filmmaker Ken Jacobs made a found-footage work in 1969 by repeatedly re-photographing frames, and upscaling small details of frames, from the original film. The composition of the shot is so rich in detail that it affords close visual exploration. Historians like Noel Burch regard the mise-en-scene of the 1905 film as an alternative to the mainstream Hollywood system of representation. The cluttered composition makes it difficult to identify the central action.

Like the Picasso picture, the frame is rich in visual information.

The application also enables the user to select various parts of the frame and apply different filters to each of them.

Setup

There are different setup possibilities, depending on the resources available.

The work can be shown as a standalone video. The video must be shown in a very dark environment, for instance a darkened room, with a projector (minimum: 300 ANSI Lumens).

The application can be exhibited as a two-screen installation (with two monitors or two projectors) together with a mouse on a pedestal.

It is possible to exhibit the video and the application together as a single work. The following image illustrates one of the possible ways.

If there is available space, some of the images shown in the Background page can also be displayed as digital prints outside the room where the video is displayed.

Background

Several tasks in image processing and analysis, such as for instance segmentation, classification, or edge detection, can be tackled by applying various filers to the image under consideration. The filters are designed to respond to the presence of the features of interest (e.g. edges or corners) in the image.

Just as an audio waveform can be decomposed into elementary sinusoids (simple sine and cosine waves) of specific frequencies, an image can be decomposed into elementary patterns of bands (2D sinusoids). The weights of these different frequency components in any given image account for many visual properties of the image. Edge detection filters can be designed to respond to oriented elements (e.g., horizontal, diagonal, or vertical boundaries) and so can be used to analyze the visual structure of the image.

The following filter, for example, detects vertical edges.

A question that arises in this context is the identification of the best filters for the task at hand. One criterion for a good filter is that it should minimize the uncertainty of the description that results from applying it to any given image.

What does uncertainty mean here? In everyday life, we might say things like “Pedro arrived at 6 O’clock, give or take ten minutes”. The expression “give or take” indicates the uncertainty of our knowledge. We do not give a specific time but rather a range or interval. We can think of the word “uncertainty” as an expression of this “give or take” property that pervades our knowledge of the physical world. To speak of uncertainty is to speak of limits to the achievable resolution in our information about a specific kind of signal.

In the case of images, we are interested in identifying edges and their orientations. This aim is accomplished by describing the visual frequencies detectable in specific locations in the image. Thus we wish to obtain information with the least possible uncertainty concerning frequency and position.

Frequency uncertainty occurs when instead of detecting the intended frequency we detect a range of frequencies. Position uncertainty occurs, for instance, when an image is blurred.

In general, an uncertainty principle asserts the existence of limits to the precision that can be attained in our knowledge of a complementary pair of physical properties, such as the momentum and position of a particle. The uncertainty principle for still images asserts the existence of a lower bound to the joint position-frequency uncertainty in the representation of an image. Once the theoretical lower bound is attained, any further improvements in one domain (frequency or position) must be paid for with a loss of resolution in the other domain.

The so-called Gabor image filters are designed to minimize the joint position-frequency uncertainty. Different filters attain a better position resolution by sacrificing frequency resolution, or vice versa.

Every Gabor image filter has two components. The first is a blurring filter consisting of a discrete kernel that approximates a continuous Gaussian function. Its uncertainty is determined by the standard deviation of the Gaussian function. The second filter is a quadrature pair of oriented two-dimensional sinusoids that respond to certain orientations (horizontal, vertical, diagonal) and wavelengths within the image. Its uncertainty depends on the effective bandwidth of the filter.

The Gaussian components of the following horizontal filters have (from right to left) standard deviation 22.4, 11.2, 5.6, 2.8 and their respective frequency bandwidths are 0.02, 0.04, 0.07, 0.14 Hz(cycle/pixel).

The uncertainty involved in our analysis of an image consists of a space-domain or position uncertainty (the standard deviation of the Gaussian kernel) and a frequency-domain uncertainty (the bandwidth of the frequencies captured by the analysis).

If we wish to identify precisely certain frequency-domain properties in an image, we need to sacrifice resolution in the position-domain, i.e., blur the image. We might then know more precisely which frequencies make a strong contribution globally to the image, but we lose precision in our knowledge of the areas of the image wherein those frequencies are active. If we wish to gain more precise location information, we need to sacrifice frequency resolution, i.e, use a filter that responds to a wider range of frequencies.

The following image shows the position-frequency trade-off. The source image is a Siemens Star, often used in video adjustment charts. As the spatial resolution of the image is decreased (as we blur the image), its Fourier representation is inversely stretched in frequency.


The top row of this diagram shows the result of applying different Gaussian filters to the image.

Each column shows the application of complete Gabor filters with the same standard deviation, but with different orientations.

The second (fourth, fifth) row then visualizes the Fourier domain decomposition of the result of applying the filter to the Siemens Star picture. In general, the Fourier decomposition is more concentrated as the standard deviation of the Gaussian increases (i.e., the resolution of the decomposition improves as we move towards the right, i.e., as the resolution of the spatial information, the sharpness of the image, decreases).

The third (fifth, seventh) row is the activation pattern that results from applying a filter with a specific orientation to the source image. We can think of this as the result of the analysis.

A note on visualization. The Gabor filters are complex filters. In displaying the results, we show the amplitude spectrum, i.e., the square root of the sum of the real and imaginary parts squared.

The following diagram shows several frames from Ingmar Bergman’s film Persona and the amplitude spectrum of the result of applying various Gabor filters (with different orientations and bandwidths) to those frames.

Gallery


 


All images used are from the followings:

https://art.famsf.org/otto-n%C3%BCckel/eighteenth-image-eighteen-burden-chapter-14-book-destiny-novel-pictures-otto-n%C3%BCckel-new

https://shinkwangho.deviantart.com/art/107-Untitled-Charcoal-On-Canvas-53-2-X-41-Cm-376856553

https://shrineodreams.files.wordpress.com/2013/02/ward_woodblock.jpg

https://art.famsf.org/k%C3%A4the-kollwitz/die-freiwilligen-volunteers-second-plate-series-krieg-war-19608524

http://donnamj.dmjstudio.com/wp-content/uploads/2013/08/kollwi02.jpg

http://my.uarts.edu/blog/libraries/files/2011/08/mike-storyteller.jpg

https://www.are.na/block/812078

https://modernism.com/items/1277/art-prints/6463-4292-der-general-der-kaschemme-rudolf-schlicter

http://www.tate.org.uk/art/artists/pierre-alechinsky-639

http://pallant.org.uk/exhibitions/past-exhibitions/2011/delonghi-print-room/german-expressionist-prints

http://www.heyapathy-comics-art.com/pen-artwork.html

http://www.blooddirtandangels.com/index.php/2010/10/24/lynd-ward-six-novels-in-woodcuts/

http://4.bp.blogspot.com/_R-wZWQCr-x8/ShhsUyiuXoI/AAAAAAAABNU/O8xcTEKVttQ/s1600-h/tiempo+de+luto.jpg

https://upload.wikimedia.org/wikipedia/commons/c/cd/Erasmus_censored.png

http://josedanielespejo.lagallaciencia.com/2014/10/

https://ceclirevista.wordpress.com/2015/09/21/pequena-galeria-de-tachaduras/

https://www.wikiart.org/en/pablo-picasso/portrait-of-ambroise-vollard-1910

CREDITS


ARTISTIC & TECHNICAL DIRECTOR

HECTOR RODRIGUEZ



PROGRAMMING

SAM CHAN

with

HECTOR RODRIGUEZ

PHILIP KRETSCHMANN



MATHEMATICAL ADVISOR

FELIPE CCUCKER



TEXT

HECTOR RODRIGUEZ



WEB DESIGN

SAM CHAN

HUGO YEUNG

PHILIP KRETSCHMANN