II.4.5 Measuring Emotions in Meetings
In large companies, employees spend up to 75% of their time in meetings. It is therefore highly relevant to make the time employees spend in meetings as productive as possible. Until now, measuring meeting efficiency and productivity has been mostly done through surveys of meeting participants or by placing observers in the meetings. A third line of research is based on analyzing videotaped recordings of meetings. These videos have then to be manually coded for later analysis. All three approaches involve a lot of manual questioning and video coding. A more automated and scalable approach to study meeting quality, efficiency, and productivity is thus desirable. In our work we analyze the efficiency of meetings through measuring the emotions of participants, as measuring and mirroring the emotions of meeting participants back to them will increase efficiency of meetings.
We created an AI-based system that measures the emotions of meeting participants by automatically recognizing the emotions of their faces and the emotions of their voices, and their happiness and stress with the smartwatches. Their meeting satisfaction is measured using the happimeter (see section III.2.3), and by asking them about their perceived efficiency and productivity of a meeting at the end of a meeting. Using webcams, the emotion of the faces can be tracked using machine learning. Using pre-labeled test data, we found that our system achieved an accuracy of 63% to 80% using facial emotion recognition. Using microphones, a voice emotion recognition system was also integrated, which, again using publicly available test data, was shown to be 62% to 78% accurate.
This system has been tested in a series of meetings in a company setting, the emotions measured through face and voice emotion recognition have been correlated with subjective meeting outcome rated through a survey among meeting participants. We found that happier and less angry faces, and happy speech were positively correlated with self-rated meeting outcome. This means that other than in a jazz concert and theater performance, participants do not want a roller coaster, but prefer a harmonious meeting.
We also used the smartwatch-based happimeter system to predict meeting outcome through the body signals of the meeting participants wearing the happimeter. Meeting participants trained the system using experience-based sampling entering their meeting satisfaction at random times during the meeting. We then trained a machine learning system to automatically predict meeting satisfaction from the body signals, reaching 60% accuracy. In this project we found that the lower the variance in arm movement was, the higher was the productivity of the meeting scored. That means either keeping the arms still, or moving them steadily is an indicator of better meetings, fidgeting around indicates unproductive meetings.