Instrumental gameplay: towards a performance practice in physics-driven environments

AEAEA is a performance environment conceived as the culmination of a decade-long exploration of musical game-pieces. Developed in Max and Unreal Engine, the system translates real-time analysis of acoustic gesture into a population of audiovisual objects governed by rules and game-engine physics. Rather than treating percussion as a trigger for immediate sound events, AEAEA frames each gesture as the initiation of processes that unfold in space and time, producing a sound world inseparable from the dynamics of a simulated environment.

This article situates AEAEA within a lineage of interactive and game-based practices, from the game-pieces of Zorn, Cage, and Xenakis to interactive ecologies by Lewis and Di Scipio, as well as contemporary uses of game engines in computer music. The system’s design is examined through its consistent multiparametric mappings, its reliance on agent persistence to generate form, and its incorporation of failure states that terminate performance through saturation, immobilization, or silence. The article also proposes the concept of instrumental gameplay to name this practice: a mode of performance in which an instrument functions simultaneously as a source of embodied gesture and as a controller of a rule-bound environment.