Search papers, labs, and topics across Lattice.
Sounding Canvas integrates capacitive sensors and real-time decision models into a touch-responsive installation that transforms painting into an interactive sound experience. By employing a CNN-based offline mapping for sound vocabulary and two online event managers鈥攁 higher-order Markov model and an LSTM-based policy鈥攖he installation enables spatialized sounds to respond to user interactions, fostering a sense of co-authorship among participants. This innovative approach not only makes algorithms perceptible through their behavior but also provokes critical discussions about authorship and agency in the context of embedded algorithmic art.
Touch-responsive paintings can now create immersive soundscapes, transforming solitary art into a collaborative auditory experience.
Sounding Canvas turns painting into a touch-responsive multimodal installation by embedding capacitive sensors, real-time decision models, and networking inside the canvas. Touches trigger spatialised sounds that appear to emanate from the painting itself. The work embeds algorithms physically, as sensing and computation concealed behind the artwork; perceptually, through an offline visual-to-sonic mapping that aligns a painting's features with sound descriptors; and performatively, through online models that shape live interaction with visitors and with remote canvases over a network. We describe the artistic rationale and technical implementation, combining a CNN-based offline mapping that defines the sound vocabulary with two online event managers, a higher-order Markov model and an LSTM-based policy, that balance responsiveness with guided exploration. We discuss how these layers make algorithms perceptible through behaviour rather than code, how networking transforms solitary touch into distributed co-authorship, and how the system raises questions of authorship, agency, and evaluation in embedded algorithmic artworks.
