Embodying Affect: Critical Interventions in Human-Computer Interaction via Embedded and Embodied Design in Extended Reality

Human computer interaction has been significantly advanced by the integration of biometric sensors and artificial intelligence, allowing more intuitive and engaging user experiences. However, there is a lack of critical engagement practices with artificial systems and their implementation in user design. The primary objective of this multimodal dissertation (composed of the virtual reality prototype Us Xtended and the online platform Miro) is to close this gap by proposing a methodology of critical embedded and embodied design via biofeedback in extended reality. It aims to do so by creating a design pathway that fosters embodiment, user agency and responsible consumption of emerging technologies. The methodology involves affect recognition and self-quantification as a critical, analytical and storytelling device. This approach aims to redefine human-computer interaction by embedding criticality and agency within the process. This is applied in the prototype via the materialist and performative aspects of biometric data and affective analysis and represented on a three-dimensional affect scale. This quantification apparatus is integrated into the experience. The critical evaluation lies within ways participants shape their experience through their biometric and behavioral inputs. Via self-reflection and comparison between the systemic analysis and self-evaluation, the project critiques affect recognition practices and stresses ethical considerations in ways biometric data is interpreted by artificial systems, reflecting on the reproduction of systemic biases in affect recognition technologies. At the same time, it highlights the system’s reliance on measurable bodily signals, emphasizing the importance of understanding its limitations. This educational aspect enhances users' ability to navigate new technologies, contributing to responsible consumption of emerging media. In essence, this dissertation advocates for a critical, embodied approach to human-machine collaboration. By exploring the intersections of technology, art, and critical theory, it aims to foster deeper understanding and more responsible engagement with the technologies that increasingly shape our existence in the digital age.