Pandora hears her dreams, they talk to her in mysterious voices and unknown languages. You are there in the middle of the dark but you don't know how did you get there, are you one of Pandora's dreams? Talk to her, maybe she will answer you...
Pandora's Dream is a versatile live coding playground that opens up a world of possibilities for performers seeking to engage audiences through live coding performances. Designed as a dynamic and interactive platform, Pandora empowers artists to harness the perspective of multiple tools, languages and approaches to create audiovisual experiences in real time.
Built within the Unity engine, Pandora serves as a text editor that transcends conventional coding practices. Its design enables performers to seamlessly weave together various programming languages, including C#, Javascript, GLSL, ChucK, and even a subset of C# called Pandora. This diverse array of languages empowers artists to explore different creative avenues and leverage the unique features and capabilities offered by each language.
Pandora also has an API to create other APIs to communicate and interact with other platforms and tools, and due to this, it was possible to develop an API to handle AI/machine learning algorithms in live coding environments. This is the case for CHAI, part of ChucK language released in version 1.5.x.x.
The integration with CHAI was tested by training an unsupervised model that can be fed in real-time with a live coding improvisation, using the audio output. In this case, the model was trained on a collection of 1000 audio samples mainly part of the Dirt collection (https://github.com/tidalcycles/Dirt-Samples) plus some samples collected by the author. The training time was no more than 10 seconds extracting classical audio features including Centroid, RMS, and Mel Frequency Cepstral Coefficients. Then using Similarity retrieval the algorithm creates an audio layer that fits the improvisation in real time.
Chuck Generated D&B - Drum and Bass
In this case, importance is given to the complementary that the system can give to the performance and not to the algorithm itself. Actually, the features, algorithms, and data used are not the most advanced or sophisticated and in the end, the model is dependent on the performer's decisions. Also, all the training stages can be done and redone during the live coding performance to adjust, limit or expand the model.
As part of this research and development, the author has presented concerts with original pieces. "Invisible Powers - In Pandora's Hands" was presented in Bing Concert Hall (June 10th, 2023, Stanford, CA) in a live coding performance where the performer controls 12 laptops simultaneously plus 124 phones (audience devices). "Pandora's Mycophony" presented in Centro de la Cultura digital (June 16th, 2023, CDMX, Mex) as part of NIME conference, in this performance the system interacts with two live coders in real-time.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)