Autolume: Automating Live Music Visualisation

Autolume is a WIP live music visualiser using Generative Adversarial Networks (GAN). The program explores the latent space of a trained model and fits the movement to the audio it is listening to. In the current state, we use a custom trained styleGAN2 on a corpus of abstract art. The algorithm computes simple audio features, such as amplitude and onset timing to weight both the step size in the latent space and the noise strength used at every resolution level of styleGAN2.

As the processes need to run in a high enough frame-rate we generate the frames at a resolution of 512x512px. For our exhibition at the Distopya sound art festival we added a super-resolution model to the pipeline once the video is completely rendered.

Our next step consists of implementing interactions with the program through a MIDI controller. This would allow users to cooperate with the GAN to create visualisations that they deem fit for their music.

The visualiszer was recently adapted for an installation at the Dystopie Sound Art festival in Istanbul 2021, through a collaboration between Philippe Pasquier and Jonas Kraasch.

Autolume Mzton is a meditation on the theme of dystopia. The piece is figuring generative analogic music and generative AI-driven video. This automated creation process is the paroxysm of media art: when the medium is literally autonomous, and the human creator made remote and removed from the content produced by algorithmic means. In fact, the dynamic of the network production is also reminiscent of cell cultures, and biological growth, adding a layer to this sensation of dystopic, a-human, or post-human autonomy. Yet, musical gestures, patching, training data, and codding are all expressions of human creativity, and the generative visuals are surprisingly referring to horizons and sunsets, new beginnings, and the antonymic notion of utopia!