The title Inputs/Outputs concerns the interaction between ‘sender’ and ‘receiver’. Inputs can be computer games, immersive theatre productions, novels, music, and classroom lessons; examples of outputs are emotions, memories, neural activities, physiological changes, and motivated behaviours.
The Inputs-Outputs conference brought together an interdisciplinary spectrum of academics from science and media, performance art and game design practitioners to facilitate discussion and collaboration around the subject of engagement:
• Approaches to engendering engagement in the arts and HCI
• The relationship between physical, emotional, and intellectual engagement
• Results from assessment and quantification of engagement in different fields
• Methodologies and modalities for measuring engagement in different fields
Fabian Ramseyer (Bern) opened the plenary session explaining his innovative technology for objectively measuring spontaneous interactional synchrony using image analysis during dyadic interaction; this technology allowed him to show that increased interactional synchrony was associated with improved psychotherapeutic outcomes. This was followed by Colin Nightingale (Punchdrunk), who gave a presentation of Punchdrunk’s work, and their approaches to engendering engagement in immersive theatre. Steve Benford (Nottingham MRL) talked about the more recent innovations in HCI in the Mixed Reality Lab, including the use of challenging emotions to create engagement in the user, followed by Tassos Stevens (Agency of Coney), talking about multi-level narrative progression in Coney’s work. The plenary session very much set the tone for ensuing discussion within the conference, highlighting the need for relational approaches to quantifying engagement, and showing that complex and challenging emotions can be highly immersive.
The following part of the programme had parallel sessions, with Hugo Critchley (BSMS), Effie Lai-Chong Law (Leicester), Judith Good (Sussex), and Alessandro Vinciarelli (Glasgow) in the main lecture theatre, and Louise Fryer (Goldsmiths), John Bonner (Huddersfield), Mariza Dima (Edinburgh) and Kate Genevieve (Brighton) in the adjoining room. Hugo Critchley talked about the neurological basis for reciprocity and embodiment in social and emotional interactions, Effie Lai-Chong Law presented an overview of the ongoing usability/user experience debate, Judith Good gave a talk on engagement in autistic children that reinforced the relational aspect introduced by Fabian Ramseyer in the plenary session, and Alessandro Vinciarelli gave an introduction to social signal processing – the science of encoding and decoding non-verbal social signals. Louise Fryer talked about audio description and presence in the visually impaired, John Bonner gave a talk on innovative approaches to performance and audience engagement, Mariza Dima presented a project where the audience co-created the performance, and Kate Genevieve talked about her project Gather, where virtual reality blends with embodied feedback.
The afternoon started with a series of talks in the main lecture theatre on sound object production and audience engagement by Anthony Churnside (BBC Research & Development), dissociative disorder and its relationship to immersion and presence by Nick Medford (BSMS), and the relationship between devised disorientation in cross-platform (transmedia) experience design, audience engagement and immersion by Carina Westling (Sussex). There was a parallel funding workshop with representatives from EPSRC and AHRC in the adjoining room, which brought out ideas for collaboration and further developments.
In the final session, D. K. Arvind (Edinburgh) presented a talk on Speckled Computing, a wearable, wireless sensor technology developed by his lab, and its use in motion capture, Anna Cox (UCL) talked about immersion in digital games, and Jonathan Freeman (Goldsmiths) gave a talk on the harvesting and processing of big data; its potential and limitations. In a parallel session, Robert Dean (South Wales) gave a presentation on live digital-foley and multimedia accompaniment to moving images.
Harry Witchel chaired the plenary discussion, which brought up the methodological tensions between the creation of engagement in the arts, and the quantitative approach to engagement in computer science, bridged by the research efforts by The Mixed Reality Lab. The prominent conclusions of the conference were the need for broadening the perspective of how engagement might be engendered, with experiences from immersive theatre and augmented reality games indicating that disorientation and challenge can be highly immersive, supported by MRL research, the role of multimodal sensory input in immersive environments, and the exciting potential for progressing quantitative engagement research by inclusion of relational factors, supported by the research of Fabian Ramseyer, Hugo Critchley, and Judith Good.
We are very pleased that so much interest in collaboration emerged, confirmed by the EPSRC and AHRC representatives, and look forward to the future development of key strands identified by the conference. D K Arvind summarised the conference as “an event that will be seen as a pivotal moment in years to come.”
Recordings of the presentations will be made available through this website.
Listen to Radio 4′s Digital Human, where Aleks Krotoski and Harry Witchel from the Inputs-Outputs team discuss the science behind the amazing experiences created by horror film makers and immersive theatre to engage their audiences: