Cuescape (2014)
Body language based human-computer interaction
How might we...
...tap into body language to unlock intuitive user-interaction design?
...help people better understand their own nonverbal communication?
The solution...
...a system that tracks, visualises and interprets body language
Technologies
Unity; Human-Computer Interaction
Body language represents a huge area of untapped potential for user interaction design. Our bodies have evolved to be capable of displaying an infinite amount of nuances and expressive fluency, but current technology has us idly poking at screens. If our digital devices could appreciate the nuance of body language, we could open up an entire unexplored domain of interaction: technology could shift in response to our specific body language, responding to the tacit or latent needs that are conveyed by the way we move and act.
Cuescape allows users to gain an immediate understanding of their body language and how it can come across to others
Cuescape analyses, interprets and visualises body language in order to create an overall assessment of subconscious intention and attitude. The prototype consisted of a combination of depth cameras which could pick up on hand gestures as well as the overall posture of the body.
This gestural data is then fed into a custom visualisation engine (made with Unity) to produce a generative map that encodes this rich information communicated by the body, and the potential interpretation that goes along with it. This graphical representation of body language helps users visualise and understand their non-verbal communication.
Example gesture map from "The Godfather" showing seriousness and stern body language, punctuated by moments of gestural tension and emotional significance
Our Library of Gestures: a collection of body language gestures, visual encoding and potential interpretations
Additional information#