A Materials Perspective on Human-Computer Interaction

Up until now, materiality has largely not been in the focus of computing and human-computer interaction (HCI), although it shapes to an increasing degree what we interact with and how we interact, especially in fields such as tangible user interfaces (TUIs) and embodied interaction. While certain aspects such as using the shape of objects for interaction have been discussed within tangible interaction design, the meaning of material qualities for TUIs has not yet been explored and investigated sufficiently. Moreover, new physical-digital types of materials as well as new material-centered application fields such as personal fabrication arise that demand for novel terminology, approaches and design knowledge. These developments lead to the situation that HCI as a discipline needs to better understand how materials shape interactions on a micro and a macro level and how this knowledge can be applied in order to design appropriate, engaging and meaningful interactions. This thesis contributes to the evolving body of works on materiality and HCI by providing a survey, a theoretical research contribution, as well as artifact and empirical research contributions.

One contribution of this thesis consists in presenting, discussing and extending eight evolving themes on materiality and HCI that encompass different material concepts and together form “a materials perspective on HCI” as a field of research. Furthermore, it presents a theoretical framework to understand and inspire how material aspects shape interaction. We explored selected material aspects presented as part of this framework in five case studies on gestural, tangible and ephemeral user interfaces.

Two of these case studies address gestural interaction with physical artifacts. In these studies, we built prototypes and developed as well as evaluated novel gesture sets. Two dominant ways how materiality shapes gestural interaction were applied: first, directly, by the materials of the artifacts used for the interaction and second, indirectly, in the way the developed gestures were inspired by manipulations of physical objects.

In two further case studies, we focus on tangible user interfaces. One is dedicated to the concept, design and evaluation of an end-user toolkit that allows users to design and customize devices by physical composition. This means, users can select and integrate arbitrary materials for the shell of the device and the control elements and simply link these to functions. The end-user toolkit builds a platform that conceptually enables aspects of sustainable interaction design and explores material aspects of DIY toolkits. In the other case study, we compared the use of a physical tool to a digital tool in a tabletop-based collaborative game. Analyzing the results from a materials perspective that takes the performative roles of materials into account, our study indicated that the physical tool facilitated group awareness to a greater extent than the digital tool.

A further contribution of this thesis lies in the definition, analysis, and exploration of ephemeral user interfaces as a novel user interface concept and field for research. Ephemeral user interfaces contain at least one user interface element that is intentionally created to last for a limited time only and typically incorporate materials that evoke a rich and multisensory perception, such as water, fire, soap bubbles, or plants. Based on a review of existing user interfaces we created a design space for ephemeral user interfaces. Our design space reveals a number of insights how material aspects shape interactions on a micro and a macro level and extends the material canon typically used for user interfaces. Additionally, we conducted an in-depth exploration of soap bubbles for interaction, studying material-based interaction constraints and material-driven user engagement.

As a whole, the prototypes and the empirical work of this thesis exemplarily reveal how material aspects matter for gestural, tangible and ephemeral interaction on different levels. They demonstrate that it is valuable to widen and rethink the canon of typical materials used for interaction in order to design appropriate, engaging and meaningful interactions. Furthermore, the work provides structured approaches revealing how materiality can get more attention in the design and analysis of human-computer interaction.

Thesis online