Prof. Perusquia’s following paper was presented at ACM ICMI 2023 (2023 25th ACM International Conference on Multimodal Interaction), held from 9-13 October in Paris, France.
- Deffrennes, A., Vincent, L., Pivette, M., El Haddad, K., Bailey, J.D., Perusquía-Hernández, M., Alarcão, S.M., Dutoit, T. “The Limitations of Current Similarity-Based Objective Metrics In the Context of Human-Agent Interaction Applications“
(Abstract) There are two main ways of evaluating a model generating an interactive virtual agent’s expressions. The first is through subjective perception tests, and the second is through objective metrics, which usually compare the model’s generated expressions to a test set of expressions considered the ground truth. In this work, we argue that using such objective metrics comparing generated expressions, to expressions contained in a test set limits the accuracy of the evaluation by failing to consider expressions that are different from the test set, but are still valid and well-perceived by users. We support this argument through experiments showing that different expression sequences are well perceived as listening responses to the same speaker’s utterance.
