HUSCAP logo Hokkaido Univ. logo

Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >

An Automatic Evaluation Method for Conversational Agents Based on Affect-as-Information Theory

Files in This Item:
JJSF22-1_73-89.pdf668.63 kBPDFView/Open
Please use this identifier to cite or link to this item:http://hdl.handle.net/2115/46974

Title: An Automatic Evaluation Method for Conversational Agents Based on Affect-as-Information Theory
Authors: Ptaszynski, Michal Browse this author
Dybala, Pawel Browse this author
Rzepka, Rafal Browse this author
Araki, Kenji Browse this author →KAKEN DB
Keywords: Affect-as-Information Theory
Conversational Agents
Evaluation Methods
Issue Date: Feb-2010
Publisher: Japan Society for Fuzzy Theory and Intelligent Informatics
Journal Title: Journal of Japan Society for Fuzzy Theory and Intelligent Informatics
Journal Title(alt): 知能と情報
Volume: 22
Issue: 1
Start Page: 73
End Page: 89
Publisher DOI: 10.3156/jsoft.22.73
Abstract: This paper presents a method for automatic evaluation of conversational agents. The method consists of several steps. First, an affect analysis system is used to detect users' general emotional engagement in the conversation and classify their specific emotional states. Next, we interpret this data with the use of reasoning based on Affect-as-Information Theory to obtain information about users' general attitudes to the conversational agent and its performance. The affect analysis system was also enhanced with a procedure for analysis of Contextual Valence Shifters to help determine the semantic orientation of emotive expressions. The method is used as a background procedure during users' conversations with two Japanese-speaking conversational agents. To verify the usability of the method, the users' attitudes to the conversational agents determined automatically during the conversations were compared to the results of a questionnaire taken after the conversations. The results provided by the system revealed similar tendencies to the questionnaire. Therefore we can say that the method is applicable as a means of evaluation for Japanese-speaking conversational agents.
Type: article
URI: http://hdl.handle.net/2115/46974
Appears in Collections:情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)

Submitter: 荒木 健治

Export metadata:

OAI-PMH ( junii2 , jpcoar_1.0 )

MathJax is now OFF:


 

 - Hokkaido University