Halfway through my interview with Louis-Philippe Morency I suddenly felt incredibly self-conscious.
Every nod, every movement of pen to paper, every glance in his eyes made me wonder what I might have been saying without saying anything. Would he catch my eyes straying to his bookshelves or the traffic on the street below and notice my (rare) moments of boredom and feel insulted? Would he detect a hurried, enthusiastic nod and hammer a point home to me? Would he latch onto my fascination to try to spin me?
Nonverbal cues drive human conversation. They signal a speaker to come to a point with an expectant glance or urge a listener to grasp the significance of a message with a well-timed raise of the eyebrows. Beneath the surface of our words we steep our conversations in texture and fill our discussions with broader meaning when we move our hands to the rhythm of our voices, shift our weight nervously, affix our gaze on listeners and alter the pitch of our voice with excitement or trepidation. These “backchannels” direct the flow of social interactions, but they aren’t universal.
Morency completed his Ph. D. at the Massachusetts Institute of Technology and joined a team at the University of Southern California’s Institute for Creative Technology studying how virtual humans — artificially created but independent characters residing in a computer environment and meant to look, move, behave and communicate like real humans — can be taught to interact more plausibly with real humans and even each other using both vocal language and these nonverbal backchannels. Since coming on board at ICT Morency — along with colleagues at ICT — has won a series of awards and other recognition for research in how computers make sense of the visual data they collect.
Backchannels evolve through time, and they are differentiated by culture. They frame our words. But while these backchannels come to us almost as easily as breathing and are as much a product of thousands of years of history as art and music and religion, they’re foreign to computers. Scientists could program the whole of the Oxford English Dictionary and countless combinations of “heuristics” — or problem solving formulas — for proper grammar and machines would still have trouble learning this natural language.
The notion that virtual humans might have unscripted conversations with humans and one another may seem like science fiction. Real humans themselves often struggle to communicate with one another; whether we’re participating in complex international negotiations or wooing a mate we weave a quilt of words and body language meant to express our needs and desires. Computers communicate in strings of ones and zeros, a vocabulary of closed and open circuits determining how they “decide” to run programs. They have no other culture, no thousands of years of history to determine their identity.
Continue reading “Blurring the lines: Virtual human research promises real-world impacts”
