Artificial Synesthesia: Re-Vision
Description
For expanding horizon of human perceptions, a novel head-mounted device is presented, the device gathers visual information and represents into pure tactile language.What is the Topic?
Synesthesia is a perceptual condition in which the stimulation of one sense triggers an automatic, involuntary experience in another sense. Such as hearing the colours, tasting the shape of alphabets, and seeing colours of music are that perceptions do not have in the objective world. But synesthesia provides us a general idea of how a sense can transferred to other senses. Therefore, "artificial synesthesia" is designed so that people who do not have synesthesic experiences can gain the ability to experience joined perception through human intervention, through it to discover the information that people had not been able to sense.
Why does it look like this?
Humans’ vision involves more complex neural mechanisms and gathers distinctively more information than the other senses, I attempt to translate visual information into tactile information by artificial means. The successful sensory substitution proves, that the information, which received by the natural sensory organs, can be perceived in another form through manual intervention. How people use their own imagination to understand the subjective world, and eventually by means of machine cognition and artificial intelligence to enhance sensory abilities are mainly considerations of my thesis.
What is special?
Such as shapes, colours, and etc., the subjective description of objects, are actually a perceptual language of our brain, there is no natural connection between perceptions of humans and the reality of the objects. Compared to the previous works in sensory substitution, scientists focused mainly on to convey "real" information from one perceptual modality to another, which is expensively to transfer and extremely long-term training required to understand such information. Thus, in this thesis i focus on, to invent a tactile "language," to represent our visual perception in a completely different way. A prototype was made to simulate the tactile "language." It consisted of two parts: 1, the system based on a Kinect camera for gathering the visual data. 2, eleven hub magnets mounted on the annular housing in one row, that can be attached to the forehead, to simulate tactile stimuli.
What is new?
As a head-mounted device, the reception of environmental information is based on the rotation of the head. When the head is no longer towards the target, the stimuli will also be transferred from centre of forehead to the side, when user moves, the change of feedback can intuitively indicate the spatial position of target. After long-term training, subtle variations of signals intensity can even indicate differences in the shape of different objects. With this feature, the height and size of the target can be also identified by different variations of signals intensities.
Link to Video