Abstract
This paper presents the development process of an Intelligent Tutoring System that performs emotion recognition on the basis of two modalities: keyboard and microphone. The system uses a multi-criteria theory to combine information about the user emotions through the two modalities. In this paper we focus on presenting and discussing the experimental studies that were conducted for the purposes of the design of the system. The experimental studies yield results concerning the way that human observers perform emotion recognition on other humans and then we show how we use these results to design the reasoning mechanism of the Intelligent Tutoring system that performs emotion recognition.