Julia Dagger - Enhancing Streaming Data Processing for Heterogeneous Computing

This project is aimed at advancing the capabilities of heterogeneous computing environments in Julia through Dagger.jl. We propose to implement a series of enhancements for significantly improving task execution efficiency, data transfer speed and robustness of distributed parallel computing systems. We place special emphasis on further development of streaming data processing for distributed heterogeneous systems with Dagger.jl. Heterogeneous computing environments leverage a combination of resources - such as CPUs, GPUs, DPUs - and face challenges in optimizing task execution and data streaming. Data transfer mechanisms are often inefficient or may lack robustness against system slowdowns or failures. The project will be supporting ongoing radio astronomy and ionospheric RADAR work at MIT and its Haystack Observatory — bringing about real-world impact in the short-term.