Synthetic Sensory Systems
Synthetic Sensory Systems: one way forward for neuromorphic systems research.
UNDER CONSTRUCTION (as always!)
Aims
- The aim of this page is to suggest one particular type of application area for neuromorphic systems: namely the development of systems which provide senses for (e.g.) computers, or autonomous robotic systems.
- To help on research in this area by gathering together references and resources on different forms of synthetic sensory systems
What is a sensory system?
Sensory systems are what animals (and even plants) use to find out about their environment. A sense is the capability to interpret some variable flux: from a biological viewpoint, the flux being interpreted must impact on the sensing entities survival (whether through finding food, or a mate, or avoiding prey, or whatever). A flux is simply something which varies (or fluctuates). Examples of biological systems are given in Table 1.
Table 1: some (human) biological senses and their fluxes
| Sense |
Flux |
| Sight |
Electromagnetic waves in the visible spectrum |
| Hearing |
Pressure waves between about 30 and 20000Hz |
| Smell |
Concentration of various airborne molecules |
Taste |
Concentration of various ions etc. in saliva |
| Touch |
Pattern of pressure at points on animal's surface |
| Proprioception |
Pattern of nerve impulses from muscles |
With the exception of proprioception, these senses all tell the animal something about the external world: proprioception tells the animal about the relative position of parts of its body. Note that some, but not all, of the fluxes are patterns of incident energy. Some animals have slightly different senses (for example, the spectrum of electromagnetic waves visible to some birds and insects extends into the ultraviolet (Silberglied R.E., Ann Rev Ecol Syst, 1979, 10:373-398)), or the spectrum of sounds which can be heard extends into the ultrasonic region: some animals have additional senses (for example, the biosonar sense in bats, or the electroreceptive sense in some fish, or the infra-red sensitive sense in certain snakes).
What is a synthetic sensory system?
A synthetic sensory system is a system which detects some (probably external) flux, translating it into internal signals, and interpreting these signals in some way. The interpretation should have some relevance to the reason for the machine's existence, although this really only applies if the system which the sensing is part of is autonomous.
One can consider everyday computers to have a limited set of senses: they can interpret key depressions and mouse movements. More recent machines like the iPhone, iPad, and the multitude of related touch-screen based devices have senses that allow them to detect gestures made on the screen. One aim of the work implied here could be to increase this flexibility, possibly by providing hearing and sight for computer systems. This could certainly increase the range of inputs available to the machines, and conceivably make them easier to use. Clearly, more autonomous machines have more need of senses, if only to permit their (fragile) systems to survive in a hostile environment. On the other hand, Keating [1] makes the point that one needs to match the sensor sophistication to the machine's internal capacity and function.
Why do research into synthetic sensory systems
A great deal of the work on synthetic sensing is motivated by understanding the biological system better: by building a working model of a biological sensing systems we gain insight into the biological system. The other primary motivations are
- the possibility of producing better sensing systems for robots or sessile computer systems
- developing prosthesis for humans who have impaired sensing faculties.
The possible gains from any of these research motivations are huge: by understanding animal sensing we will both improve our understanding of the way in which animals fit their ecological niches and improve the design of artificial prostheses for impaired humans. Better sensing systems for completely synthetic systems (like mobile robots or desktop computers) will allow them to interact much more effectively with their environment. For an autonomous robot, this is the difference between being able to work usefully independently and not being able to do so.
At first sight, one might imagine that adding senses to a desktop computer would not be useful: however, if one compares the sophistication of the display with that of the input devices (keyboard and mouse) one rapidly realises that the input devices lag way behind. In the last 10 years, screens have improved enormously in quality, yet the last improvement in input devices (for desktop computers) was the mouse. (As noted above, palmtop machines now have touch-sensitive displays, and these can respond to gestures, including multiple-touch gestures: yet these have not really made their way in quantity to the realm of desktops, which is perhaps a little strange.) Keyboards have not altered materially in many years. Input based on sensing includes sound input (and that, in turn, includes speech), visual input, and even intelligent usage of keyboard and mouse input (are key-depressions frequent or infrequent, often incorrect or always right, is mouse usage smooth or jerky, etc.). Indeed, one can imagine the desktop machine merging with the mobile robot to produce a synthesis in which the static computer becomes a thing of the past. The limitations on these machines are imposed primarily by our imaginations!
Existing and past work
There is a quite a lot of information both published and on the internet about sensory neuromorphic systems. The Telluride workshops have been running for some years. In recent years there have been workshops on neuromorphic systems in Europe as well at Capocaccia, Sardinia.
I recently wrote a review of Neuromorphic systems for the proceedings of Brain Inspired Cogniive Systems 2008.
(For older material, see the book, Neuromorphic Systems: Engineering Silicon from Neurobiology, World Scientific, 1998: this book developed from the 1st European Workshop on Neuromorphic Systems (Stirling, 29-31 August 1997).)
The biggest group in Europe is in the Institute of Neuroinformatics, jointly run by the University of Zurich, and ETH Zurich. There is a page describing biologically based work at the Dept of Artificial Intelligence at the University of Edinburgh.
My own primary work is on the auditory system. We are curently working with Edinburgh on the development a novel multi-sensor microphone which can detect sounds using mutlitpl independently gain-adjustable sensors based on MEMS technology. In the past I also worked on the statistics of sound signals: this work aimed to look at what regularities there are in sounds, and to use that (eventually) to guide processing.
A page of references and WWW pages etc is under construction.
Back to Professor Smith's home page.
Last updated: Monday, 13-Jun-2011 11:15:18 BST
If you have any difficulties accessing this page, or you have any
queries/suggestions arising from this page, please email:
Prof Leslie S Smith (lss(nospam_please)@cs.stir.ac.uk)
