An introduction to auditory objects, events, figures, images and scenes
From CNBH Acoustic Scale Wiki
|
Hommage à René Magritte: An introduction to auditory objects, events, figures, images and scenes
Roy Patterson , Tom Walters
This page presents a discussion of (1) how we should describe sounds as they occur in the world and representations of those sounds as they occur in the head. There is a tendency to use the phrase "auditory object" to refer to anything in the head, and even sounds out in the world. It is argued that we should distinguish "auditory objects" from "auditory images", "auditory figures", "auditory events" and "auditory scenes" and establish what we mean by each of these terms. The discussion below began with a talk presented at the Auditory Objects Meeting organized by Tim Griffiths, Maria Chait and Dave McAlpine [hosted by the Novartis Foundation in London] 1-2 October 2007. The terminology has evolved as this followup page was written.
The talk began with the presenter (RP) saying -
Bonjour mes amis. Aujourd’hui, je veux vous présentez une Hommage à René Magritte et son peinture:
- Ceci n’est pas une pipe.
- Et, ceci n’est pas le mot pipe.
Figure 2: A simulation of the auditory images produced by the words in "Ceci n’est pas le mot pipe." Loading the player... Download CeciNestPasLeMotPipe.mov [7.47 MB] |
- Mais ça, c’est une pipe.
En Anglais, the pipe in the hand is a pipe (Figure 3). The 'pipe' in Figure 1 is not actually a pipe. It is a painting of a pipe -- a painting by René Magritte which includes the famous inscription intended to prompt discussion about art and illusion. The visual part of the video in Figure 2 is not a visual image of a pipe; it is a video that presents a simulation of the activity that occurs in the auditory system in response to the sounds that correspond to the words on the audio track. The space of auditory perception that appears in the video is referred to as the auditory image (Patterson et al., 1992; Patterson, 1994), and the structures that appear in the image are referred to as auditory figures and/or auditory events (Patterson et al., 1992). The question is whether we might want to call portions of the neural activity simulated in the video auditory objects? In short, is the internal, auditory representation of a word an auditory object? And if so, what are the properties that make it an object as distinct from other auditory perceptions that are not auditory objects?
The purpose of the talk is to
- describe the analogy between visual and auditory objects, as we find it in the literature,
- develop some terminology and notation for the analogy, and
- demonstrate a visual representation of the auditory representation of sound that is intended to assist in understanding and distinguishing auditory images, auditory figures, auditory events, and auditory objects as they occur in auditory scenes.
The basic problem is to relate the external world of objects to the internal world of visual and auditory representations of objects. As a starting point, it is assumed that:
- A Visual Object is the visual part of your experience of an external object, or event, including visual knowledge associated with the object. And, by analogy,
- An Auditory Object is the auditory part of your experience of an external, acoustic event involving an object, including auditory knowledge associated with the object.
There are a wealth of concepts involved in the description of perceptual objects, and the concepts are described by a variety of words. So, the purpose of the talk is to
- list the important concepts,
- set out the relationships between the concepts, and
- see to what extent it might be possible to develop a functional notation for the relationship between these external objects and mental objects.
The topics are:
- Terminology and notation for the perception of visual objects and visual events
- Acoustic events and Auditory Events
- ‘Source’ versus ‘object’ and ‘environment’ versus ‘scene’
- The basic element of the acoustic event and the auditory figure
- Making the auditory figure scale-shift covariant
- Visual objects and Auditory Objects
- Words as auditory objects and visual objects
- The energy source: reflected light; emitted sound
References
- Patterson, R.D. (1994). “The sound of a sinusoid: Time-interval models.” J. Acoust. Soc. Am., 96, p.1419-1428. [1]
- Patterson, R.D., Robinson, K., Holdsworth, J., McKeown, D., Zhang, C. and Allerhand, M. (1992). “Complex Sounds and Auditory Images”, in Auditory Physiology and Perception, Y Cazals L. Demany and Horner, K. editors (Pergamon Press, Oxford). [1] [2]