219 / 2018-08-16 23:15:24

Assessing Nonverbal Communication Competence by Using Multimodal Sensing Technologies

nonverbal communication,multimodal sensing,AI technology,public-speaking assessment

全体主题 > 人工智能技术在语言测评中的应用

Non-verbal communication plays an important role in human social interactions. As Abercrombie (1968: 55) said: “We speak with our vocal organs, but we converse with our entire bodies.” Using TED styled public speaking as an example, we can clearly see that a good presenter does not only use his/her verbal communication well, but also is good at orchestrating his/her nonverbal behaviors, e.g., facial expressions, eye contacts, body posture changes, and so on. Therefore, the competence of being able to properly understand and utilize nonverbal communication impacts the overall language competence.

However, in the foreign language assessment area, non-verbal communication has only received limited attention. In most language assessments, the verbal communication, e.g., vocabulary and grammar, is still emphasized. In this talk, we’d like to raise attention to the importance of inclusion of the measurements on nonverbal behaviors in language assessments. We will focus on a public speaking task in our talk given that the public speaking is a very useful skill and a proper assessment needs consider both verbal and nonverbal aspects. We will introduce several assessments developed in different regions and analyze the constructs being used, especially focusing on the measurements of nonverbal cues, e.g., using gestures to support making presentations. Regarding measuring nonverbal behaviors, recent works have emerged of using multimodal sensing technologies, e.g., facial expression recognition, body pose tracking, etc. on providing reliable and accurate automatic measurements. We will review several recent exemplary works to show the audiences what nowadays technologies can provide.

Throughout our talk, we’d like to let the audiences to be aware of the importance of inclusion of nonverbal communication in language assessments, to become familiar with multimodal assessments on public speaking skills, and to know automatic measurements supported by multimodal sensing technologies.