Knowledge Co-Creation - Profiles of researchers
Where Does the Mind Exist?
Science Clarifies Subjectivity and the Unconscious Mind
Professor, Faculty of Science and Engineering, Waseda University
Looking into the Reality of the Mind
Having interest in the human mind since high school, I majored in psychology at the University of Tokyo’s Faculty of Letters. At first, I just vaguely sought to study clinical psychology, psychoanalysis for example, but experimental psychology was the main focus of the department. I thought it was just gathering test subjects in the laboratory and single-mindedly collecting and analyzing data, but it was very interesting once I tried it. An encounter with a graph showing a psychometric function at that time determined my future studies.
a psychometric function counterposing the objective physical quantity on the horizontal axis and the subjective psychological quantity on the vertical axis
For example, a color is just a psychological variable, not a physical one. A psychometric function is a graph made by counterposing the physical quantity on the horizontal axis and the psychological quantity on the vertical axis (Figure 1). It analyzes how a subjective phenomenon is connected with some physical characteristic, for example, how something "looks red". This method of roughly allocating and comparing a physical (objective) quantity to a psychological (subjective) quantity took me by surprise, and I realized that subjectivity was also within the scope of scientific research.
I entered graduate school and enrolled in a doctoral course at the California Institute of Technology. I did not have the option to study psychology there, so I conducted research on crossmodal interaction (perceptual and inter-sensory integration) by specializing in Computation and Neural Systems, an interdisciplinary field. Specifically, I studied the mutual effects between human perception and senses, such as the mutual interaction between seeing and hearing.
After completing the doctoral program, I became a researcher at several research institutions, including the National Institutes of Health (NIH), and conducted studies on how motivation and rewards translate into actions in the brain in the field of system neuroscience. As such, I completely changed my research field, but direct observation of the activities made by a single neuron was a great experience for me. The attitude of directly observing both physical and psychological phenomena, such as neural activity and subjective experience, has become one of the criteria in expanding my research without losing the sight of reality of the mind.
The Mind coming from the Outside
We inevitably think about the unconscious mind and our feelings when we look at the mind as a real phenomenon. This has been my fundamental interest from the beginning. Many people think that subjective feelings come from the inside, but is that so? This topic has been debated throughout history, and there are two conflicting theories: the Cannon-Bard theory, which advocates that feelings come from inside the mind (central origin), and the James-Lange theory, which states that a body's reactions provoke feelings (peripheral origin). According to the Cannon-Bard theory, people cry because they are sad and according to the James-Lange theory, they are sad because they cry.
DAVID (Da Amazing Voice Inflection Device) has been developed as a digital platform for voice-based emotion induction that can give emotion modulation to voice in real time while someone is talking. It has been released as an open source software.
To come straight to the point, it is believed that most of the phenomena in the mind occur under the influence of outside factors. In recent years, I have been conducting research on self-perception—how a person perceives themselves. For example, development and maintenance of the self is considered to be caused by a spiral of positive or negative feedback; an optimistic person becomes more optimistic because they perceive themselves as optimistic, and a pessimistic person becomes more pessimistic because they perceive themselves as such. We may then intervene in the spiral by processing sensory feedback. As a result of the research based on this hypothesis, we have developed a new method called DAVID (Da Amazing Voice Inflection Device), a voice-based feeling induction system (Figure 2).
With this device, participants read aloud a short story for about ten minutes. Their voices are filtered, processed and transformed into a sad voice, a happy voice, or a scared voice. The voices are played back to the participants in almost real time. By surveying the feelings of the participants through questionnaires before and after the experiment, we found that the person whose voice was changed into a sad voice felt more sad, that the person whose voice was changed into a happy voice felt more happy, and that the person whose voice was changed into a scared voice felt scared. We can clearly say that feelings come not from the inside, but the outside.
It was difficult to objectively assess and determine whether people cry because they are sad or that people are sad because they cry, but by extracting physical putput (vocal information) with DAVID, we can clarify self-perception as having some relationship between the physical quantity and the psychological quantity. We can expect that this device will not only be useful as a basic research methodology but also be applied to mood disorder and PTSD treatment. It may also be effective in improving interpersonal communication and productivity.
We have been conducting my own basic and applied research using this method, and we wanted many people to use DAVID for expanding its potential. After consulting with my peers, we have released DAVID as an open source software on the Internet at the same time we published our paper. I hope that many people will use it (for independent summer research projects and the like) and that the research will further develop in a bottom-up fashion.
Abilities Are Determined by Society and Environment
Conceptual Diagram of the Research Project on the Implicit Ambient Surface Information
Psychologists have found that not only subjective feelings but a person’s performance level also changes under external influences. Even in a simple experiment where the participants were required to push a button as quickly as possible, they pushed faster when the other people move quickly, and they pushed slower when others are slow. The limit of our abilities can change depending on the group, organization and society we are in. In the past, there was a case that competitive swimmers wore a swimsuit called the LZR Racer and dramatically shortened their lap times. Using LZR Racer was then prohibited, but I suppose that even swimmers who didn't wear LZR Racers might have also cut down on some time compared their regular time. The social environment in which swimmers who wore LZR Racers shortening their time might have removed some social constraints on physical performance. In other words, the society and the environment create an insurmountable wall, but at the same time, provide strength to overcome such a wall.
Recently, we are focused on a project called Implicit Ambient Surface Information, paying attention to bring out people’s potential hidden in the environment (ambience), which surrounds us like air (Figure 3). In this project, we scientifically assess, analyze and use the informational atmosphere that people create, as well as the synchronization of interpersonal feelings and actions. We often say, "I don't know why, but I get along well with that person." The purpose of this project is to find out what kind of information in being transmitted, how physiological phenomena such as heart rate, body temperature, perspiration, and muscle tension are being synchronized, and whether or not performance level improves in cases where some kind of synchronization occurs between two persons or groups by using wearable devices to the participants.
For this purpose, we have formed a sport team by ourselves (laugh). In an experiment with participants gathered in the laboratory, motivations and rewards are not so high. Therefore, we have created a social situation and an environment in which we are forced to desperately put our hearts into winning and achieve victory, no matter the cost. All the team members wear the devices and aim to assess the situation of the group as a whole in real time.
With advances in information and communication technology, artificial intelligence will begin to play a role in society. I am interested in not technology itself but how the human mind and ability change through encounters with new technology. If the idea to create the human mind and ability through real-time networks becomes accepted as the norm, our way of life and relations with others would change drastically. I hope that by redefining the human society itself, a symbiotic relationship with the so-called vulnerable, like those with disabilities, will be rebuilt and that a new view of the world enabling social inclusion will be established.
Professor, Faculty of Science and Engineering, Waseda University
Professor Katsumi Watanabe graduated from the Department of Psychology of the Faculty of Letters, the University of Tokyo. He earned a Master’s degree from the Graduate School of the Arts and Sciences, the University of Tokyo, in 1997 with a major in cognitive behavioral science and obtained his Ph.D. in Computation and Neural Systems from California Institute of technology (CalTech) in 2001. He served as a research fellow at the Japan Society for the Promotion of Science, a researcher at National Eye Institute, National Institute of Health, and National Institute of Advanced Industrial Science and Technology, as well as a group leader for Shimojo Implicit Brain Function Project, ERATO, JST (Decision Making Research Group). He then became an Associate Professor of the Research Center for Advanced Science and Technology’s cognitive science section at the University of Tokyo in 2006 before he was appointed his current position in 2015. He is also a Visiting Associate Professor at the University of Tokyo’s Research Center for Advanced Science and Technology. Please refer to the website of the research office for further research.