Design and evaluation of human-computer interfaces for expert creative interaction

Digital artists, composers, and musicians form a class of extreme users whose demands of their systems include communication of personal style, mood, and expressive intent, and who are willing to spend countless hours both adapting and becoming accustomed to their tools (Mackay 2000). How to best design such tools is still an open question, and so this research program will address two main questions, which will be studied in parallel: a) how can hardware and software infrastructure support the creation, exploration and iterative development of such novel systems, and b) how to design and evaluate digital systems and interfaces for supporting creative practices by individual expert practitioners?. These infrastructures will not discriminate between designers and end-users, instead enabling users to continuously appropriate and redesign their tools. The first research question will explore the design, use and evaluation of modular, open--source software prototyping tools, building on the connectivity middleware libmapper (see e.g. Malloch, Sinclair, and Wanderley 2014; 2018) and its supporting ecosystem of language bindings and utilities such as session managers, data recorders, and protocol bridges. This base will be used to investigate tools for discovering, understanding and mapping (connecting) "interesting" signals found within large collections of streaming sensor data; integrating concepts from intermedia mapping and scripting tools, and comparing programming representations based on events to those based on streams in the context of live interactivity. Research on the second theme will make use of existing prototyping tools-and those developed by research theme 1 as they become available-to create and evaluate functional prototypes of novel human--computer interfaces for creative interaction with digital systems. These interfaces will be adapted for extreme personalisation to the idiosyncratic needs of the artist(s) or creator(s) involved. Evaluation through prolonged creative use of the interfaces will be valued more heavily than empirical measurements of e.g. speed or ergonomic efficiency.