II.9 Animals and Plants Provide a Social Compass

If trees provided Wi-Fi, we would be planting them everywhere. But they only give us the air that we breathe. – Bear LeVangie, Massachusetts arborist 

In our most recent research, we have been analyzing the interaction of humans with other species, such as plants, dogs, and horses. While we humans have a hard time communicating among ourselves, we find it even more difficult to communicate with individuals of other species. The main interaction between species consists of eating members of other species, or being eaten. We humans consume huge amounts of pig, cow, and chicken meat, drink milk, eat eggs, bread, rice, corn and other veggies. However, when we are not eating plants and animals, we also like to talk to them. Some people even talk to their houseplants. And owners of dogs, cats, and horses of course talk to their pets all the time, maybe even considering them their soulmates. But do the plants and animals really understand what we are trying to say to them? And even more important, can we understand what the dog, horse, cat, or a mimosa or basil plant is trying to say to us?

Before we can start talking to others, we need to listen to what they have to say, and try to make sense of their output. Computers and artificial intelligence have made huge progress over the last twenty years helping us both to listen and to talk to each other. Combining machine learning and AI with functional magnetic resonance imaging (fMRI), computers are finally able to read the human brain. Computers are capable of knowing what we are thinking by tracking the activation of combinations of brain cells indicative of certain words. In the currently best implementations  computers can read up to 250 different words at 90 percent accuracy by looking at our neurons. In genetically engineered mice, computers have also spoken to the mice, telling them when to drink water by turning on their neurons, “playing them like a piano” . Google Translate and Deepl have become language geniuses, dynamically translating from English to Chinese, with me talking to my phone in English in the hotel in Beijing, and the phone repeating my sentence to the hotel receptionist in Chinese.

What if we could do the same when talking to a horse, a dog, or a mimosa, with the computer telling the animal or plant what I would like to say, and then the animal or plant talking back to me. Our research group has been studying how humans communicate online and face to face for the last two decades. In our work we have been building many tools for happier, more creative, and more productive collaboration among humans, leveraging the computer and AI to read the "honest signals" and emotions of what a human really wants to say beyond the literal meaning of words. Only recently, in the last two years, have we been starting to do the same for animals and plants.

Figure 64. Emotion recognition of dogs (left) and horses (right) 

In our research, we have been applying these algorithms and technologies to interspecies communication. We have used body sensing technology to better understand communication with horses, and facial emotion recognition to understand emotions of dogs and of horses. Figure 64 illustrates the dog and horse emotion tracking systems. The dog tracking system includes both body posture and facial expression of dogs to identify happiness, anger, fear and relaxation. The horse emotion tracking system has been trained with hundreds of pictures of horse faces labelled by an expert horse trainer with the four emotions relevant for horses, “alarmed’, “curious’, “relaxed”, and “annoyed”.

In other recent projects we are trying to listen to plants. We have put “brain sensors” on mimosas, as they talk back to the outside world quite visibly: when their leaves are touched, they fold them. We found that they seem to sense the electrostatic discharge of human bodies and the rhythm of body movement near them (Figure 65). When putting our sensors on other plants such as basil they show the same response. 

Figure 65. Experimental setting for measuring human emotions through electrostatic body discharge through Mimosa pudica

In a new series of experiments, we found that plants are at least as good as custom-built hardware in measuring body movements . Mimosas and other plants act as bio-sensor tracking the way somebody moves their body, and will thus deduce the same information as can be gleaned from wearing a smartwatch at least as good as the sensors of the smartwatch. The main difference is that there will be a little plant pot on the table to collect emotion data instead of the user having to wear a smartwatch. By measuring the pattern of electrostatic discharge of human bodies together with Mimosa Pudica and other plants in response to the human movement we have been able to recognize individuals based on their distinctive pattern of body movements with 66% accuracy as well as positive or negative mood of people based on their gait characteristics as perceived by the plant with 85% accuracy. As a measuring device we use the Plant SpikerBox , a device that measures the electrical action potential while also measuring the electrostatic discharge between the electrode on the leaves of a plant and the capacitively coupled human body.

We also recorded the leaf movement of the “dancing plant” (Codariocalyx Motorius) in response to human voice and music. Using automatic image recognition, we found that its leaf movement was different in reaction to male and female voices talking nearby .

This means that in the end we envision a scenario where inter-human interaction will be measured not only by computer hardware, but also by living growing sensors in the form of plants. This has the additional advantage of being surrounded by plants, as having plants on our desks and hallways will make us happy and healthy. Being surrounded by plants is one of the best and proven ways of increasing happiness. In a groundbreaking experiment published in the journal Science in 1984, Roger Ulrich found that patients who viewed trees from their hospital rooms recovered 15% faster compared to patients whose windows went to a brick wall. Tree-view patients also needed significantly less painkillers compared to brick-wall patients. As Oliver Sacks says in “The Healing Power of Gardens”  “…even for people who are deeply disabled neurologically, nature can be more powerful than any medication”. Sacks then continues “…Hortophilia, the desire to interact with, manage and tend nature, is also deeply instilled in us. The role that nature plays in health and