§68. Sight Seeking Sight

The modern theory of vision begins with Kepler’s discovery of the retinal image in 1604. Before that, the eye was conceived as a sensitive body that was somehow affected by its object and took on its form.

Kepler envisioned light painting an inverted picture on the retina, as on the back of the camera obscura, reproducing all the visual features of the scene before the eye. He named the retinal projection pictura, likening it to a pic­ture, and spoke of pencilli, the word for an artist’s brush, to describe how by passing through the lens all rays from one point form a focused image on the retina. “The retina is painted with the colored rays of visible things.” “Vision occurs through a picture of the visible thing on the white, concave surface of the retina.” The image being inverted baffled him though. On the one hand, “Geometrical laws leave no choice in the matter”; but how the image gets righted was beyond his competence as a mathematician. No purely optical action takes place beyond the retina, no optical phenomena—no image, no light—passes through the optical nerve.¹⁷¹

Kepler obviously assumes that the inverted retinal projection is itself seen.

A fascinating implication of these assumptions is that neither distance nor volume is a visual perception. The geometrical assumptions make both qualities invisible, as Locke’s friend William Molyneux demonstrated in Dioptrika Nova (1692), the first optical treatise written in English. “For dis­tance of it self, is not to be perceived; for ’tis a line (or a length) presented to our eye with its end toward us, which must therefore be only a point, and that is invisible.” What we see must therefore be as flat as a photograph; like a painting, all that vision presents is light and color. Hence Berkeley: “What we strictly see are not solids, nor yet planes variously coloured: they are only diversity of colour.” We imagine or, as these authors say, judge the rest. We have to learn, not to see distance and dimension, which is impossible, but to infer them. Since we are typically insensible of such judgments, Helmholtz described them as unconscious inferences. The experience of touch teaches what Berkeley calls “outness,” inviting us to agree that an “unbodied spirit” who sees perfectly but has no touch would not have “any idea of distance, outness, or profundity.” As Condillac said, touch unfurls the universe.¹⁷³

This entire line of thought depends on the specious analogy between vi­sion and a camera (or camera obscura). These are quite different systems and unsuited to model each other.

The eye-camera comparison assumes that nothing gets into the eye except light in the form of rays or wave fronts. That was Kepler’s new idea, replacing the medieval multiplication-of-species theory. All we ever see directly is what stimulates the retina, which is light. In the early nineteenth century, Johannes Muller translated this assumption into the terms of an emer­ging science of neurophysiology. According to his so-called specific energy theory, the direct, immediate objects of vision are nerves stimulated by the pattern of light falling on the retina. “The immediate objects of the percep­tion of our senses are merely particular states inducted in the nerves, and felt as sensations.” All the brain receives is nerve energy organized by the organ, the distal cause being merely a trigger. Each sense organ has its own specific energy, and produces the same effect regardless of how it is stimulated. Long discredited, the idea survives among philosophers in the fantasy of the “brain in a vat.” Provided stimulation achieves the “specific energy” of the organ, we have no way to distinguish normal perception from what happens to this dis­embodied, massively deluded brain, which could be us for all we can tell.¹⁷⁴

Helmholtz understood Muller’s specific energy theory to imply that the material in the afferent nerves does not image or depict reality but rather constitutes a natural system of signs that organisms learn to interpret, a thesis he identified with “empiricism.” Sensations are mental correlates of activated afferent nerves, and perceptions are states of consciousness interpreting these sensations as signs of external realities. Berkeley had already described sensations as natural signs, as had Descartes.

The assumption of this entire line of thought, from Kepler to Helmholtz, is that we perceive our stimuli. Wolfgang Kohler called this the “experience error,” when “characteristics of sensory experience are inadvertently attrib­uted to the mosaic of stimuli.” The optical projection of a retinal image does not enjoy the centrality that the modern theory supposes it must. We do not see by inspecting what is on the retina. Seeing is the function of a higher- order visual system that integrates many streams of stimuli into the act of vision. Kepler discovered the lens, but he knew nothing about the screen on which its image supposedly focuses, including whether it is even suitable to receive an image of projected light, which it is not. Cells of the inner layer of the retina, whose fibers project into the brain, do not respond to light as such but rather to differences of intensity of light between parts of their recep­tive field. What forms on the retina is not an image; it is a map of invariants. Projecting a light-image onto such a surface would be like projecting on a sieve. Achieving an image on the retina therefore does not have the teleolog­ical gravitas that comparison with a camera (whose work is finished with the registration of an image) wrongly suggests.¹⁷⁶

What are eyes adapted to perceive? Not light.

The object of vision is a layout of surfaces, which we see as the visual system detects invariants by exploratory changes to its array. Vision is ravenous for these invariants, which it does not passively receive but actively orchestrates in a way Helmholtz and Mach likened to experiment, making vision and ex­perimental action inseparable. The camera is again a poor model. To reg­ister a good image the camera has to be motionless, but treat eyes that way and vision disappears. Vision requires the visual tremor, the saccadic move­ment of the eyeball some fifty times each second. The tremors ensure that the fovea, the locus of greatest acuity, rapidly scans the retinal data, exploring invariants. “We can understand the point of the visual tremor, once we cease to think of the retinal image as moving over the retina, but of the retina run­ning over and exploring the visual image.”¹⁷⁸

Vision does not derive from the patterns of light that fall on the retina. It derives from continuous modifications brought on by retinal movements coordinated with an ever-changing body posture scanning rapidly for invariants. This exploration is typically a whole-body movement. A frozen form is ambiguous and not a typical stimulus; only when the perspectives flow do we detect the invariants that we see as solid objects, surfaces, and their features.

Those “trigger features” are what psychologist of perception J. J. Gibson calls affordances. When Gibson began research in the 1950s, experimental psychology tended to assume that perceptual objects were sets of qualities, somewhat as Russell and Carnap proposed. Gibson thought the task of per­ception is not to combine lots of individual sensations into extensive wholes, but rather to break up the initial continuity of the sensory field, introducing borders and segregating substructures, following lines of perceptual invari­ance, offering the organism what Gibson termed affordances. The principle by which vision analyzes its stimulus is a search for qualities that afford ac­tion, affordances being the qualities things offer for an organism’s action. To see something is to see how to approach it, how it is likely to change, or re­spond to change. “The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill.” The stimulus information of vision is not the quality or intensity of a sensation; it is the affordances that get attended to, affordances for manipulation, locomotion, nutrition, and social interaction. That, and not light on the retina, is what we see at the primary level of vision.¹⁸⁰

The concept of affordance highlights the complementarity of organism and environment. Habitat is the locale where an organism lives; niche describes how it lives there, and it describes a symbiotic economy of affordances. These are not representations. They are how the given is given, which is not by luminous presence, simple sensation, impression, or representation, but virtual interaction. We perceive environments in terms of potential engage­ment, discernment registering tendencies for which it is adaptive to be pre­pared. Neither are perceptual affordances interpretations. Gibson’s idea is not that the mind interprets perceptions in terms of how the body can interact. Vision, for instance, is not seeing something as an affordance. Interpretation implies a text. If affordances were interpretations, they could not be what they are, which is how the given is given.

Gibson did not so much invent this theory as assemble a scientific argu­ment for it. Similar ideas had been advanced by Bergson in his thesis of per­ception as virtual action (§77); by ecologist Jakob von Uexkull: “An animal is able to distinguish as many objects as it can carry out actions in its envi­ronment”; and by gestalt psychologist Wolfgang Metzger, who said that “not shape and color, but rather such forces as entice and urge certain specific be­havior are perhaps the earliest characteristics of all perception.” It is behav­ioral affordances, the proximal antecedents of action, and not “information” or “sense data” that perception attends to. We perceive in terms of poten­tial interaction, and only subsequently, if at all, attend to isolated qualities. Gibson used the terminology of “direct perception” and this misled some readers. His point is that the perception of affordances does not organize per­ceptually simpler, more immediately apprehended elements. The causation of the perception is of course complex, but the perception of affordances is psychologically direct and simple in not being composed of simpler, more immediately intuited components.¹⁸¹

Empiricism is not irremediably bound to the atomistic psychology that prominent modern empiricists (Gassendi, Locke, Berkeley, Hume, Mill, Spencer) adhered to. Some empiricists may have thought we perceive sensations, but the assumption is not, so to say, built in to empiricism as a philosophy of nature and its knowledge. What is more nearly built in is the idea of an artful use of experience for the enhancement of interactive know­ledge. Gibson's theory of visual perception seems like exemplary empiricism (I haven't described his many experiments), and there is no reason why em­pirical philosophy cannot learn from it and criticize defective psychology in its own tradition. It remains an empiricist insight that even the most abstract concept has to start from sensory perception. Every concept is abstract, and it is perception from which concepts are abstracted and to which they re­main keyed. Concepts arise from perceptions because concepts respond to perceptions, conceptualizing their affordances and categorizing potential ac­tion, which contributes to adaptive response as well (in humans) as to the smooth implementation of tools and other artifacts.¹⁸²

Helmholtz and Mach seem right to regard experimentation as continuous with evolved perception, a view Dewey shares. Experiments systematically seek the invariant in the sea of noise, and that is exactly what perception does. We are neurologically tuned by evolution to attend to the invariant in sensory modalities, and experiments are just our learning to be more ingen­ious and systematic about it, relying on experimental art to disconnect the perception of invariance from service to need. Experimental science uses adaptive perceptual powers of invariance detection for new purposes play­fully disengaged from their adaptive functionality.