Posters

Poster Dimensions

Gale M. Lucas, Evan Szablowski, Jonathan Gratch, Andrew Feng, Tiffany Huang, Jill Boberg, and Ari Shapiro
Recent advances in scanning technology have enabled the widespread capture of 3D character models based on human subjects. Intuition suggests that, with these new capabilities to create avatars that look like their users, every player should have his or her own avatar to play videogames, especially those simulating real-world activities that could later be performed in the real world. We explicitly test the impact of having one's own avatar (vs. a yoked control avatar) in such a simulation (i.e., maze running task with mines). We test the impact of avatar identity on both subjective (e.g., feeling connected and engaged, liking avatar's appearance, feeling upset when avatar's injured, enjoying the game) and behavioral variables (e.g., time to complete task, speed, number of mines triggered, riskiness of maze path chosen). Results indicate that having an avatar that looks like the user improves their subjective experience, but there is no significant effect on how users behave in the game.
Divesh Lala and Tatsuya Kawahara
Research on embodied teammate agents which use dialog and gesture to coordinate their activities with the user is relatively sparse compared to conversational agents. We propose a dialog management model to handle interactions between user and agent in a virtual basketball environment. The model describes how a joint action should be initialized and executed through dialog, and how it should handle new dialog interruptions. The model also allows the agent to be parameterized to exhibit different combinations of speech and gestural behavior over repeated joint actions. We propose that this model allows us to conduct many several types of unique experiments in this environment.
Tomoko Koda, Masaki Ogura, and Yu Matsui
This paper reports how shy people perceive different amount of gaze from a virtual agent and how their perception of the gaze affects comfortableness of the interaction. Our preliminary results indicate shy people are sensitive to even a very low amounts of gaze from the agent. However, contrary to our expectations, as the amounts of gaze from the agent increases, shy people had more favorable impression toward the agent, and they did not perceive the adequate amount of gaze as most comfortable.
Zhou Yu, Xinrui He, Alan W. Black, and Alexander I. Rudnicky
Human communication literature states that people with different culture backgrounds act differently in conversations. Currently most virtual agents are designed for a single targeted popular culture. Most have a fixed task to complete and are not designed to track users' internal state, such as engagement. We first conducted a comparative behavior analysis and found Americans and Chinese both expressed engagement with longer spoken utterances and faster speech, when interacting with a virtual agent of their own cultural identity. However, they reacted differently towards system interruptions and long system pauses. We built a computational model to automatically predict user engagement with a set of identified behavior cues which were found correlate significantly with engagement in our comparative study. Not separating the two cultures in the engagement modeling process resulted less desirable performance compared to separating them. We built a high performance engagement model for the less-studied culture, Chinese culture, by transferring information from data in a well-studied culture, American culture.
Zhenglin Pan, Mihai Polceanu, and Christine Lisetti
Real time user independent facial expression recognition is important for virtual agents but challenging. Researchers have put great amount of effort into this topic. However, since in real time recognition users are not necessarily presenting all the emotions, some proposed methods are not applicable. In this paper, we present a new approach that instead of using the traditional base face normalization on whole face shapes, performs normalization on the point cloud of each landmark using the CPD algorithm that helps cluster facial expressions. We compare the mean-based and median-based normalization with our approach on CLM processed CK+, BU-3DFE and BU-4DFE datasets. The result shows that our method outperforms the other two when the user input does not contain all six universal emotions.
Joseph Garnier, Jean-Charles Marty, and Karim Sehaba
We propose a computational model for the Component Process Model (CPM) of Scherer, the most recent and the most complete model of emotion in psychology. This one proposes to appraise a stimulus through a sequence of sixteen appraisal variables dealing with a large number of its characteristics. As CPM is very abstract and high level, it is not really used in affective computing and no formal models exist for its appraisal variables. In this paper we propose two mathematical functions for two appraisal variables detecting the relevance of a perceived event, according to the state of the cognitive component of an agent (goals, needs, semantic memory, and episodic memory).
Natasha Jaques, Yoo Lim Kim, and Rosalind Picard
This paper investigates how the personality and attitudes of intelligent agents could be designed to most effectively promote bonding. Observational data are collected from a series of conversations, and a measure of bonding is adapted and verified. The effects of personality and dispositional attitudes on bonding are analyzed, and we find that attentiveness and excitement are more effective at promoting bonding than traits like attractiveness and humour.
James Ryan, Adam Summerville, Michael Mateas, and Noah Wardrip-Fruin
In this paper, we present a novel approach to natural language understanding that utilizes context-free grammars (CFGs) in conjunction with sequence-to-sequence (seq2seq) deep learning. Specifically, we take a CFG authored to generate dialogue for our target application, a videogame, and train a long short-term memory (LSTM) recurrent neural network (RNN) to translate the surface utterances that it produces to traces of the grammatical expansions that yielded them. Critically, we already annotated the symbols in this grammar for the semantic and pragmatic considerations that our game’s dialogue manager operates over, allowing us to use the grammatical trace associated with any surface utterance to infer such information. From preliminary offline evaluation, we show that our RNN translates utterances to grammatical traces (and thereby meaning representations) with great accuracy.
Andrea Bönsch, Tom Vierjahn, and Torsten W. Kuhlen
Computer-controlled virtual humans can serve as assistants in virtual scenes. Here, they are usually in an almost constant contact with the user. Nonetheless, in some applications assistants are required only temporarily. Consequently, presenting them only when needed, i.e, minimizing their presence time, might be advisable. To the best of our knowledge, there do not yet exist any design guidelines for such agent-based support systems. Thus, we plan to close this gap by a controlled qualitative and quantitative user study in a CAVE-like environment. We expect users to prefer assistants with a low presence time as well as a low fallback time to get quick support. However, as both factors are linked, a suitable trade-off needs to be found. Thus, we plan to test four different strategies, namely fading, moving, omnipresent and busy. This work presents our hypotheses and our planned within-subject design.
Franziska Burger, Joost Broekens, and Mark A. Neerincx
Reciprocal self-disclosure is an integral part of social bonding between humans that has received little attention in the field of human-agent interaction. To study how children react to self-disclosures of a virtual agent, we developed a disclosure intimacy rating scale that can be used to assess both the intimacy level of agent disclosures and that of child disclosures. To this end, 72 disclosures were derived from a biography created for the agent and rated by 10 university students for intimacy. A principal component analysis and subsequent k-means clustering of the rated statements resulted in four distinct levels of intimacy based on the risk of a negative appraisal and the impact of betrayal by the listener. This validated rating scale can be readily used with other agents or interfaces.
Anton Bogdanovych and Tomas Trescak
Many modern virtual reality reconstructions of historical and cultural heritage sites are predominantly focused on significant buildings and artefacts, but often ignore the issue of portraying everyday life of the ancient people who populated the reconstructed area. This is mainly due to high costs and complexity of populating such sites with virtual agents. In this paper we show how these costs can be reduced and the process can be automated through the combination of normative multiagent systems and planning. The key focus of this work is on simulating human motivation that helps to automatically supply agents with goals. We explain how large crowds of agents can perform planning around these goals and re-enact complex scenarios in real time.
Moshe Bitan, Galit Nahari, Zvi Nisin, Ariel Roth, and Sarit Kraus
This paper presents a Virtual-Suspect system designed for use in police interrogation training simulations. The system allows users to pre-configure various scenarios based on real cases, as well as different suspect histories and personality types. The responses given by the Virtual-Suspect during the interrogation are selected based on context and the suspect's psychological state, which changes in response to each interrogator's statement. Experiments with 24 subjects have shown that the Virtual-Suspect's responses in an interrogation scenario are similar to those of a human respondent.
Wenjue Zhu, Andry Chowanda, and Michel Valstar
An important aspect of dialogue management is the concept of a topic: virtual humans should stay on topic, or make sensible topic switches if the conversation demands it. In this work we propose a novel data-driven Topic Switch Model based on a cognitive representation of a limited set of topics that are currently in-focus, which determines what utterances are chosen next. After each utterance and consideration of inputs to the virtual human system (e.g. perceived facial expressions or utterances by the agent's interlocutor), the topic model undergoes a transition, with some topics going out-of-focus, while others coming into focus. The transition model is probabilistic and learned from a large set of transcribed dyadic interactions. Results show that using our proposed topic switch model results in interactions that last 2.17 times longer on average compared to the same system without our model.
Sam Thellman, Annika Silvervarg, Agneta Gulz, and Tom Ziemke
Previous work has shown that physical robots elicit more favorable social responses in interaction compared to virtual agents. These effects have often been attributed to the physical embodiment of the robot. However, a recently published meta-analysis by Li [1] casts doubt on this conclusion, suggesting that the benefits of robots should be attributed to their physical presence rather than their physical embodiment. To further explore the importance of presence we conducted a pilot study in order to investigate the relationship between physical presence and social presence. The results suggest that social presence is positively related to social influence, and that social presence and physical presence may be independent from each other.
Jean-Arthur Micoulaud-Franchi, Patricia Sagaspe, Etienne de Sevin, Stéphanie Bioulac, Alain Sauteraud, and Pierre Philip
While the interest of Embodied Conversational Agents (ECA) in health care context increased, the extent to which patients find ECAs acceptable should be more evaluated. Thus, in this study, we evaluated the acceptability of an ECA who conducts a clinical structured interview to make a medical diagnosis, in comparison with the same clinical structured interview presented in written form on a tablet screen. 178 patients participated to the study (102 females (57.3%); Mean age = 46.5 years ± 12.9, range 19-64; Mean educational level = 13.3 years ± 3.1). It was showed that patients perceived globally the acceptability of the ECA higher than the tablet. This higher acceptability was linked rather to higher satisfaction than to higher usability. Moreover, the patients were more satisfied when they repeated the clinical interview with the ECA than with the tablet. Thus ECA usage could avoid the decrease of satisfaction of repeated computerized clinical interviews.
Annika Silvervarg
This paper explores how students perceive the gender of a visually androgynous teachable agent, and if and how the perceived gender relates to the perceived personality traits of the agent. It is shown that the students' perception of the agent's gender was independent of their own gender. There were few significant differences in the perceived personality traits in relation to perceived gender. However, when looking at the perceived degree of gender, the traits of the agent were more positively rated by those students who perceived it as moderately gendered as compared to androgynous or strongly gendered.
Peng Yu, Jiaying Shen, Peter Yeh, and Brian Williams
Intelligent personal assistants, such as Siri and Google Now, are pervasive, and can help users with a variety of requests such as checking the weather, recommending restaurants, getting directions, and more. However, the usefulness of these personal assistants quickly diminish when presented with requests that are no longer limited to a single goal or activity, such as a device command or a straight-forward information lookup. For example, these assistants cannot help users plan a night out after a hard day at work, which may involve a dinner and a movie. They do not understand that these complex requests involve multiple goals and activities with interrelated (and even competing) constraints and preferences that must be met accordingly.In this paper, we present an intelligent personal assistant, called Uhura, that handles requests involving multiple, interrelated goals and activities by efficiently producing a coherent plan. Uhura achieves this by integrating a collaborative dialog manager, a conflict-directed planner with spatial and temporal reasoning capabilities, and a large-scale knowledge graph. We also present a user study that assesses the usefulness of the plans produced by Uhura in urban travel planning, and show that Uhura performs well on a wide range of scenarios.
Huaguang Song and Michael Neff
Gestures can take on complex forms that convey both pragmatic and expressive information. When creating virtual agents, it is necessary to make fine grained manipulations of these forms to precisely adjust the gesture's meaning to reflect the communicative content an agent is trying to deliver, character mood and spatial arrangement of the characters and objects. This paper describes a gesture schema that affords the required, rich description of gesture form. Novel features include the representation of multiphase gestures consisting of several segments, repetitions of gesture form, a map of referential locations and a rich set of spatial and orientation constraints. In our prototype implementation, gestures are generated from this representation by editing and combining small snippets of motion captured data to meet the specification. This allows a very diverse set of gestures to be generated from a small set of input data. Gestures can be refined by simply adjusting the parameters of the schema.
Ameneh Shamekhi, Timothy Bickmore, Anna lestoquoy, Lily Negash, and Paula Gardiner
In this paper we describe a conversational virtual agent that is designed to be used in conjunction with group medical visits to help treat individuals with chronic pain and depression. During group visits, patients learn how to care for themselves using non-medical treatments including yoga, meditation, and self-massage. Patients interact with the virtual agent at home on a touch screen tablet to reinforce what they learned in the group and to guide them through self-practice sessions. Results from two rounds of pilot testing and preliminary results from a clinical trial in progress indicate that patients like the virtual agent and find that it helps them manage their condition.
Zhiqiang Cai, Yan Gong, Qizhi Qiu, Xiangen Hu, and Art Graesser
AutoTutor uses conversational intelligent agents in learning environments. It has long been the major challenge to assess student natural language answers to AutoTutor questions. AutoTutor relies on semantic matching in answer assessment, which involves pre-authored ideal answers. We investigated an AutoTutor dataset with 3358 student answers to 49 AutoTutor questions. Comparing with human ratings, we found that semantic matching works well for some questions but poor for others. We found that the performance of semantic matching can be predicted by a measure called “question uncertainty”, an entropy value on semantic cluster probabilities. Based on this findings, we proposed an iterative AutoTutor script authoring process, which can make AutoTutor agents smarter and smarter by modifying ideal answers and improving assessment models.
Nicole Krämer, Carina Edinger, and Astrid Rosenthal-von der Pütten
In an attempt to replicate earlier research on intelligent agents for robots, we analysed the effects of the presence and absence of a robot´s nonverbal behavior on users´ nonverbal behavior and evaluation with a between subjects experimental study (N = 90). Results demonstrated that when the robot shows nonverbal behavior (head movement and deictic, illustrative and rhythmic gesture) participants evaluated it more positive. Against expectations, however, participants displayed more nonverbal behavior when the robot only used speech.
Sangyoon Lee, Yifan Lu, Apurba Chakraborty, and Mark Dworkin
The study presented in this paper focuses on a elational, educational, and motivational virtual human (VH) mobile phone application for African American men who have sex with men (AAMSM). This project targets on increasing adherence to HIV medication and on improving the proportion of HIV-infected persons engaged in care by developing a theory-based mobile phone intervention that engages young HIV-positive AAMSM. The intervention is aimed to improve the likelihood of compliance with healthy behavior leading to both patient benefits (decreased morbidity, mortality and resistant virus) and population benefits (decreased HIV transmission). The VH encourages interaction with information and functions that promote engagement with the HIV Care Continuum, provide fundamental HIV information, present motivating statements, facilitate interaction with healthcare, visualize laboratory results, and encourage, explain, and illustrate relevant behavioral skills.
Debajyoti Datta, Valentina Brashers, John Owen, Casey White, and Laura Barnes
This paper describes the development of a deep learning methodology for semantic utterance classification (SUC) for use in domain-specific dialogue systems. Semantic classifiers need to account for a variety of instances where the utterance for the semantic domain class varies. In order to capture the candidate relationships between the semantic class and the word sequence in an utterance, we have proposed a shallow convolutional neural network (CNN) that uses domain-specific word embeddings which have been initialized using word2vec for determining semantic similarity of words. Experimental results demonstrate the effectiveness of shallow neural networks for SUC. The methods produce superior classification accuracy comparable to existing benchmarks.
Ulysses Bernardet, Maryam Saberi, and Steve DiPaola
Building virtual humans is a task of formidable complexity. We believe that, especially when building agents that interact with biological humans in real-time over multiple sensorial channels, graphical, data flow oriented programming environments are the development tool of choice. In this paper, we describe a toolbox for the system control and block diagramming environment Simulink that supports the construction of virtual humans. Available blocks include sources for stochastic processes, utilities for coordinate transformation and messaging, as well as modules for controlling gaze and facial expressions.
Seyedeh Zahra Razavi, Mohammad Ali, Tristram Smith, Lenhart Schubert, and Mohammed Ehsan Hoque
We summarize an exploratory investigation into using an autonomous conversational agent for improving the communication skills of teenagers with autism. The system conducts a natural conversation with the user and gives real-time and post-session feedback on the user’s nonverbal behavior. We obtained promising results and ideas for improvements in preliminary experiments with five autism spectrum disorder teens.
Hafdís Erla Helgadóttir, Svanhvít Jónsdóttir, Andri Már Sigurðsson, Stephan Schiffel, and Hannes Högni Vilhjálmsson
We developed a virtual game playing agent with the goal of showing believable non-verbal behavior related to what is going on in the game. Combining the fields of virtual agents and general game playing allows our agent to play arbitrary board games with minimal adaptations for the representation of the game state in the virtual environment. Participants in preliminary user testing report the game to be more engaging and life-like with the virtual agent present.
Kangsoo Kim, Ryan Schubert, and Greg Welch
We explore how and in what ways the surrounding environment can be an important factor in human perception during interactions with virtual humans. We also seek to leverage any such knowledge to increase the sense of Social/Co-Presence with virtual humans. We conducted a user study to explore the influence of environmental events on social interaction between real and virtual humans in a Mixed Reality setting. Specifically we tested two different treatments to see the effects on Social/Co-Presence: (i) enhanced physical-virtual connectivity/influence via a real fan blowing on virtual paper, and (ii) the virtual human's corresponding awareness of the environmental factor as she looks at the fan and holds the fluttering paper. While a statistical analysis for the study did not support the positive effects of the two treatments, we have developed some new insights that could be useful for future studies involving virtual humans.
Reginald L. Hobbs
Information retrieval across disadvantaged networks requires intelligent agents that can make decisions about what to transmit in such a way as to minimize network performance impact while maximizing utility and quality of information (QOI). Specialized agents at the source need to process unstructured, ad-hoc queries, identifying both the context and the intent to determine the implied task. Knowing the task will allow the distributed agents that service the requests to filter, summarize, or transcode data prior to responding, lessening the network impact. This paper describes an approach that uses natural language processing (NLP) techniques, multi-valued logic based inferencing, distributed intelligent agents, and task-relevant metrics for information retrieval.
Masato Fukuda, Hung-Hsuan Huang, Naoki Ohta, and Kazuhiro Kuwabara
no abstract
Matthew Marge, Claire Bonial, Kimberly A. Pollard, Ron Artstein, Brendan Byrne, Susan G. Hill, Clare Voss, and David Traum
The Wizard-of-Oz (WOz) method is a common experimental technique in virtual agent and human-robot dialogue research for eliciting natural communicative behavior from human partners when full autonomy is not yet possible. For the first phase of our research reported here, wizards play the role of dialogue manager, acting as a robot’s dialogue processing. We describe a novel step within WOz methodology that incorporates two wizards and control sessions: the wizards function much like corpus annotators, being asked to make independent judgments on how the robot should respond when receiving the same verbal commands in separate trials. We show that inter-wizard discussion after the control sessions and the resolution with a reconciled protocol for the follow-on pilot sessions successfully impacts wizard behaviors and significantly aligns their strategies. We conclude that, without control sessions, we would have been unlikely to achieve both the natural diversity of expression that comes with multiple wizards and a better protocol for modeling an automated system.
Hung-Hsuan Huang, Yuki Ida, Kohei Yamaguchi, and Kyoji Kawagoe
no abstract
Antonia Hamilton, Xueni Sylvia Pan, Paul Forbes, and Jo Hale
Virtual reality is providing new tools to explore and quantify human social cognition. Here we review some recent studies using virtual characters to study imitation behaviour, with a focus on VR methods. We created virtual characters which demonstrate pointing actions and find that typical adults spontaneously copy action height. In a second study, we are able to create virtual characters which mimic the head movement of a participant in a naturalistic conversation task, but find no evidence for increases in rapport or liking. These studies demonstrate how virtual characters can be used to examine social cognition, and the value of greater interaction between cognitive psychology and computing in future.
Beatriz Bernardo, Patrícia Alves-Oliveira, Maria Graça Santos, Francisco S. Melo, and Ana Paiva
This work explores the use of a social robot as an assistive agent during therapy sessions, in order to assist children with Autism Spectrum Disorder (ASD), through a Tangram game. This experiment has two conditions: the Tutor Mode - the robot gives help whenever the child needs; and the Peer Mode - the robot plays with the child in turn-taking. The results showed that, in the TM, the robot was capable of stimulating children's attention towards the game and to assist them most of the times. In the PM, the robot also stimulated children's attention to the game and was able to establish turns for most participants.