Differentiable Logic for Interactive Systems and Generative Music

This project aims to enable the development of models that are suitably efficient for use in real-time interactive applications on embedded systems such as the BeagleBone-based Bela. At the project's core is difflogic, a recent technique that employs sparsely-connected network composed of basic logic gates to obtain small models and fast inference. The core goal of the project is to enable a convenient workflow for developing difflogic models and running them on the Bela, with an expected workflow of developing and training models on a larger machine, followed by exporting the model to C and cross-compiling it for the BeagleBone - either the main CPU or the PRUs. To support this, I will develop wrappers for exporting compiled difflogic models for use in the languages supported on Bela (C++, Pure Data, SuperCollider, Csound). I will then build out integrations and applications of difflogic for the rapid development of useful audio models, exploring the possibilities for combining difflogic networks with DDSP (differentiable digital signal processing) and cheap/lightweight techniques and toolkits, "porting" well-known ML architectures to difflogic networks, and training difflogic networks to approximate the behavior of existing neural networks (knowledge distillation). I will develop example applications involving sound analysis, classification, and synthesis, and experiment with interactive machine learning. Finally, I will apply these techniques to a creative application inspired by work on bytebeat and sound matching: generating networks of logic gates to approximate sounds and exploring the space of such sound-generating networks. I will train difflogic gates to reproduce particular sounds, treating the entire network as a bytebeat-style audio-generating function of time. Lastly, I will build an interactive application to explore the space of sound-generating networks (e.g. by mutating or morphing networks) and visualize the execution of logic gate networks.