SC²I Distinguished Lecture | Distributed ML Applications on Multitier Infrastructure

We consider the design of systems to support the automatic deployment and execution of DNN applications on a multi-tiered infrastructure. We propose an architecture that has three core components that work cohesively to achieve this goal. First, at the core of this architecture is the Kong system that automatically compiles and deploys existing monolithic applications into distributed executable graphs that run across the multi-tiered infrastructure. The system reduces deployment overhead while providing highly efficient distributed execution. Second, to further simplify the application development process, we design AppleSeed, a technique that uses Large Language Models (LLMs) to generate monolithic application programs based on user intent expressed in natural language. The generated programs seamlessly integrate with Kong to enable efficient distributed execution.

Finally, to support more efficient execution, we have designed Murmuration to optimize the DNN models and algorithms for distributed environments. These algorithms, integrated closely with Kong, enable more efficient execution with Service Level Objective (SLO) guarantees in dynamic environments. Kong enables easy programming of distributed multi-tiered infrastructures to support distributed applications that are efficient, customizable, and personalized.