An Automatic Evaluation Method for Conversational Agents Based on Affect-as-Information Theory

Ptaszynski, Michal; Dybala, Pawel; Rzepka, Rafal; Araki, Kenji

An Automatic Evaluation Method for Conversational Agents Based on Affect-as-Information Theory

Ptaszynski, Michal; Dybala, Pawel; Rzepka, Rafal et al.

Journal of Japan Society for Fuzzy Theory and Intelligent Informatics, 2010, 22(1), 73-89

Permalink : https://hdl.handle.net/2115/46974

このアイテムのアクセス数:1,289件（2026-08-13 15:02 集計）

閲覧可能ファイル

ファイル	フォーマット	サイズ	閲覧回数	説明
JJSF22-1_73-89	pdf	668 KB	1,110

論文情報

ファイル出力 EndNote Basic出力 Mendeley出力

アクセス権	open access
URI	https://hdl.handle.net/2115/46974
タイトル	An Automatic Evaluation Method for Conversational Agents Based on Affect-as-Information Theory
著者	Ptaszynski, Michal ; Dybala, Pawel ; Rzepka, Rafal ; Araki, Kenji
著者名 Ptaszynski, Michal
著者名 Dybala, Pawel
著者名 Rzepka, Rafal
e-Rad_Researcher 50202742 著者名 Araki, Kenji
言語	英語
キーワード	Affect-as-Information Theory
	Conversational Agents
	Evaluation Methods
発行日	2010-02
出版者	Japan Society for Fuzzy Theory and Intelligent Informatics
出版者	日本知能情報ファジィ学会
収録物名	Journal of Japan Society for Fuzzy Theory and Intelligent Informatics
収録物名	知能と情報
巻(号)	22(1)
ページ	73-89
抄録	This paper presents a method for automatic evaluation of conversational agents. The method consists of several steps. First, an affect analysis system is used to detect users' general emotional engagement in the conversation and classify their specific emotional states. Next, we interpret this data with the use of reasoning based on Affect-as-Information Theory to obtain information about users' general attitudes to the conversational agent and its performance. The affect analysis system was also enhanced with a procedure for analysis of Contextual Valence Shifters to help determine the semantic orientation of emotive expressions. The method is used as a background procedure during users' conversations with two Japanese-speaking conversational agents. To verify the usability of the method, the users' attitudes to the conversational agents determined automatically during the conversations were compared to the results of a questionnaire taken after the conversations. The results provided by the system revealed similar tendencies to the questionnaire. Therefore we can say that the method is applicable as a means of evaluation for Japanese-speaking conversational agents.
資源タイプ	学術雑誌論文
出版タイプ	Version of Record
PISSN	1347-7986
EISSN	1881-7203
関連情報 (isIdenticalTo)	出版社版DOI https://doi.org/10.3156/jsoft.22.73