II.2.1 Data Collection
The first step consists of collecting the interaction data. The body response to direct interpersonal interaction can for example be tracked through brainwaves, heartrate, body movement, voice pattern, and facial expression. Brainwaves can be measured with EEG sensors from companies such as Muse, Emotiv, and OpenBCI. In our experiments we used the Muse headband to measure Alpha waves in the frequency range of 7.5–12.5 Hz which occurs during wakeful relaxation with closed eyes, and Beta waves ranging from 12.5 and 30 Hz indicative of normal waking consciousness. However, when comparing Alpha time series of different users of Muse at the same event, we found little correlation among the different users , indicating that the measurements were not very accurate. Also, the Muse SDK (software development kit) is not supported anymore, therefore the Emotiv Epoc or Emotiv Insight might be a better option.
To measure body movement, smartwatches provide built-in sensors such as accelerometer, heartrate sensor, blood pressure sensor, microphone, and light sensors. A smartphone will also include accelerometer, microphone, and Webcam, however the drawback is that the smartphone is not worn on the body most of the time, so it can only collect environmental variables such as GPS coordinates through the location sensor and movement when carried by the owner. Alternatively, body movement as well as facial expressions can be tracked through image recognition with Webcams. As Webcams are part of each laptop and smartphone, this is basically free technology, however the challenge will be to first recognize the person, and then calculate the features (facial expressions, body posture) from the raw picture stream. Voice also delivers input to calculate emotions, microphones are readily available in smartphones, Webcams, and laptops, again a key issue is to assign a voice to the right person if there is more than one person present.
In our most recent research, we discovered that plants respond to human movement through electrical signals and leaf movement. This means that either image recognition to track leaf movement, or highly sensitive electric current measurement devices such as the Backyard Brains plant spikerbox can be used to measure the response of plants such as Mimosa Pudica and Codariocalyx Motorius to humans (see section II.9).
Electronic communication archives such as E-mail and Twitter provide convenient records of inter-human interaction. To construct social networks, communication records need a source or sender, and a target or receiver, plus a timestamp indicating when the interaction happened. Sender and receiver can be Twitter users, E-Mail users, Reddit users, etc. As soon as there is content, such as the e-mail body, or a tweet, it will be possible to compute input features for the machine learning with natural language processing (NLP).