Approximation Theory is an art-research project in visual mathematics and data aesthetics. It consists of a series of prints and videos that visualize the mathematical idea of approximation.

 

Development of this project was partly funded by City University research grants 9610322 and 7004725.

About

This project is a collaboration between Felipe Cucker & Hector Rodriguez.

The methodology used in the work involves the choice of a set of fixed dictionaries or databases of images. Each dictionary has its own distinctive quality. For instance, some consist of linear or curved elements generated procedurally.

Figure 1. A few images from the rectilinear dictionary


Figure 2. A few images from the curvilinear dictionary


Other dictionaries might consist of more complicated images, such as frames from movies or Chinese characters.

Figure 3. A few images from the Chinese dictionary


Any other image can then be reconstructed as a weighted superposition of all or some of the images in the dictionary. This set of images is called a basis. For instance, this source image from the 1936 Summer Olympics can be reconstructed by superposing images from a basis of 3840 elements (each of them of the "linear" dictionary).

Figure 4. An image of a swimmer and a reconstruction of it


There exist mathematical procedures that identify how to superpose a given subset of images from the dictionary in a manner that most closely reconstructs or "approximates" the source image.

The character of the approximation depends on two kinds of factors: qualitative and quantitative.

The quantitative aspect has to do with the number of images that are used in the reconstruction; i.e., the size of the basis. The sequence of digital prints in Figure 5 shows several reconstructions of one source image using progressively larger bases from the same dictionary.

Figure 5. Reconstructions using 30, 60, 120, 360, 720, and 1920 images from the dictionary


The qualitative aspect has to do with the character of the images in the dictionary, for instance whether they are linear or curved. Figure 6 shows an image and three reconstructions obtained using bases of the same size from the three dictionaries shown in Figures 1-3. While the three results resemble the original, each has its own distinctive personality.

Figure 6. Three reconstructions with different dictionaries

This video illustrates the qualitative aspect of approximation theory. It shows reconstructions of short clips done with the three dictionaries above. In all three cases the size of the basis is moderate (250 images). This allows the viewer to focus on the reconstruction of the motion rather than on the accuracy of detail.


The following video encapsulates the fundamental character of the project in the form of a grid. Qualitative aspects of approximation theory are shown along the vertical axis: the same images are approximated using four qualitatively different dictionaries. The dictionaries consist of straight lines, curves, frames from Godard's movie Alphaville, Chinese characters. Quantitative aspects of approximation theory are shown along the horizontal axis. The same images are approximated using more and more elements from each of the four elements. As more elements are used, the reconstruction is closer to the original image but the distinctive character of the dictionary becomes less marked.


Our work uses source images drawn from the 1936 Summer Olympics, as shot by director Leni Riefenstahl. These images express a fascination with the human form, and one of our main concerns has to to do with the representation of the body in an algorithmic age. The images are also drawn from an age where the threat of fascism was very real, and manifested itself in the cult of the body. We are also confronting a growing sense of populist authoritarianism around the world. Our response to this threat is to reassert the value of rational analysis and the integration of art, science, and cultural critique. An additional reason to choose Riefenstahl's movie is that it naturally provides a diverse, yet thematically coherent, collection of short clips, rich in movement. The reconstruction of these clips with different dictionaries allows for a visual comparison of the dictionaries' characters.

This project can be exhibited as a set of prints that explore different aspects (both qualitative and quantitative) of mathematical approximation theory, together with videos that compare different reconstructions. The number of prints and videos can be adjusted to take into account venue and equipment constraints.

One of the strengths of this project is the close connection between art, mathematics, and philosophy.

The philosophical aspects have to do with aesthetics. Philosopher Richard Wollheim claims that the experience of seeing a representational image, such as a painting, requires or invites a “twofoldness of attention”(1)(2)(3)(4). The viewer must attend to:

1. The design properties of the physical medium or support (configurational fold), and
2. The scene or content represented (recognitional fold) by means of those properties.

These two folds are aspects of a single experience, which characterizes representational painting. We see the design properties as responsible for the appearance of the scene depicted. In other words, the recognitional fold is experienced as dependent on the configurational fold.

In some works of visual art, however, attention is devoted mainly to the scene rather than to the manner in which the scene is depicted. The recognitional fold then gains primacy over the configurational fold. For example, the ideal of perspective painting as a transparent window onto the world expresses the primacy of the recognitional over the configurational. In its most extreme version, for instance in photorealist painting, the medium becomes transparent and we see the scene through it. Other artists, however, emphasize the role of the medium. Impressionist and cubist painting can be viewed in this light. Chinese ink painting also highlights the configurational properties of the medium. In this case, we see the scene in, rather than through, the medium.

This distinction between seeing-through and seeing-in provides an important motivation for Approximation Theory.

The quantitative aspects of approximation determine the relationship between the configurational and recognitional folds in the reconstruction of an image. As the size of the chosen basis increases, the image becomes more and more immediately recognizable. The element of seeing-through eventually takes over and we perceive the scene rather than the medium. If the size of the basis becomes very large, then the qualitative character of the dictionary becomes irrelevant and all reconstructions tend to look alike, regardless of the dictionary used.

If the basis is very small, the content is not recognizable, and one can hardly speak of a representational image at all. But if the basis is neither too small nor too large, we see both the medium (the qualitative character of the images in the dictionary) and the content (the image being reconstructed). More crucially, we see the medium as responsible for our seeing the content. We see the content in the medium.

The mathematical concept of the work is in this sense intimately connected to a philosophical reflection on the nature of artistic representation.

 

 

_____________

(1) Richard Wollheim, “Seeing-as, Seeing-in, and Pictorial Representation”, in Art and Its Objects (Cambridge University, 1980), 205-226.

(2) Richard Wollheim, “On Pictorial Representation”, The Journal of Aesthetics and Art Criticism (1998), 56:217-226.

(3) Richard Wollheim, Painting as an Art (Princeton, 1987).

(4) Robert Schroer uses the expression “phenomenological doubleness”. See: Robert Schroer, “The woman in the painting and the image in the penny: an investigation of phenomenological doubleness, seeing-in, and ‘reversed seeing-in’” Philosophical Studies (2008) 139:329-341.

Preferred Setup

Please note that we are giving here the original dimensions of the images. These can be adjusted to fit the dimensions of exhibition area.

 

Part one - Quantitative

Show a sequence of reconstructions of one image. Each reconstruction is made with a greater number of elements in the dictionary. This illustrates the quantitative aspects of approximation theory.

Figure 8. 3-by-3 setup of the Owens series

The recommended dimensions the image set are as follows:

Dimensions for each of the 9 prints: 45cm wide x 30 cm high

 

Part two – Qualitative

Show the image of a diver reconstructed with three different dictionaries, to illustrate the qualitative aspects of approximation theory.

Figure 7. 2-by-2 qualitative set of diving images

The recommended dimensions the image set are as follows:

Dimensions for each of the 4 prints: 32cm wide x 40 cm high

 

Part three - Video

In addition, the room should contain a monitor (52" full HD 1920x1080p resolution recommended) that displays a clip illustrating different sets of pre-rendered moving images from the film Olympia (1938). The video clip is on .mov format and can be played from a VLC media player. The video can be found in the about page.

 

Actual setup in the 14th Athens Digital Arts Festival, May 2018


Mathematical framework

The idea of approximation underpins many aspects of digital technology, including machine learning, computer vision, and digital image and sound processing. In many of these applications, it is vital to replace a complicated function (for instance, one that is expensive to compute or tedious to describe) by a simpler one. Digital data compression, for instance, often involves the replacement of image or audio data by a shorter description. Approximation theory aims to find the best approximation of the desired input.

The context of approximation theory is the following: there is a target function f which we want to approximate with another function g to be selected from within a set S of approximant functions. The main problem is that of estimating how good this g can be, that is, how small can the error of replacing f by g be. This problem presupposes a way of measuring this error (usually a distance in a space of functions where both f and g live). In addition, it depends on both the target function f and the set of approximant functions S. This dependence has both quantitative and qualitative aspects.

On the quantitative side there are parameters, such as the dimension of the linear space spanned by S (typically, when S spans a finite dimensional linear space) or the radius of S (typically, when S is a closed ball in a linear space, possibly of infinite dimension).

Our project is an experimental exploration of the quantitative and qualitative aspects of approximation.

Intuitively, a digital image can be viewed as a position or a point in a space of possible images. More precisely, an image of N (grayscale or "black-and-white") pixels is identified with a vector of N real numbers in the interval [0, 1], with 0 being black and 1 being white. Our target function (for instance a photograph that we wish to reconstruct) is a point in this N-dimensional vector space. A set S of approximant functions is a subset of the unit cube in this space which we will assume to be closed under weighted sums. In particular, S is a set of N-pixel images (the "dictionary").

Any target image is to be approximately reconstructed as a weighted sum (linear combination) of the elements of a basis (a selection of images from the dictionary). Intuitively, we can think of this procedure as a superposition or mix of brightened or darkened versions of the various images in the basis. By changing the intensities of the images in the basis in appropriate ways, the resulting combination will resemble approximately the target image. The "appropriate ways" of mixing the images in the basis are determined automatically by mathematically projecting the target image onto the subspace spanned by the images in the basis. The measure of error implicitly used is Euclidean distance.

The quantitative side mainly involves the number of images to be used in any approximation, i.e., the size of the basis. We might choose to employ only a relatively small subset of images in the dictionary. As the number of images employed decreases, the fidelity of the resulting reconstruction also decreases, and vice versa.

The qualitative side involves the character of the images in the dictionary at hand. Our project explores how the use of different kinds of images (for instance geometrically abstract frames consisting of lines or checkerboards, photographs of different locations in the world or still frames from classical films) affects the visual quality of the reconstruction.

We are currently exploring how the use of advanced mathematical techniques, such as "L1-minimization", may affect the visual character of the result.

CREDITS


ARTISTS

FELIPE CUCKER

HECTOR RODRIGUEZ



PROGRAMMING

HUGO YEUNG

with

HECTOR RODRIGUEZ

FELIPE CUCKER

PHILIP KRETSCHMANN



WEB DESIGN

SAM CHAN

HUGO YEUNG

PHILIP KRETSCHMANN