Toward Scalable Generative AI via Mixture of Experts in Mobile Edge Networks

The evolution of generative artificial intelligence (GenAI) has driven revolutionary applications like ChatGPT. The proliferation of these applications is underpinned by the mixture of experts (MoE), which contains multiple experts and selectively engages them for each task to lower operation costs while maintaining performance. Despite MoE's efficiencies, GenAI still faces challenges in resource utilization when deployed on local user devices. Therefore, we first propose mobile edge networks supported MoE-based GenAI. Rigorously, we review the MoE from traditional AI and GenAI perspectives, scrutinizing its structure, principles, and applications. Next, we present a new framework for using MoE for GenAI services in Metaverse. Moreover, we propose a framework that transfers subtasks to devices in mobile edge networks, aiding GenAI model operation on user devices. Moreover, we introduce a novel approach utilizing MoE, augmented with Large Language Models (LLMs), to analyze user objectives and constraints of optimization problems based on deep reinforcement learning (DRL) effectively. This approach selects specialized DRL experts, and weights each decision from the participating experts. In this process, the LLM acts as the gate network to oversee the expert models, facilitating a collective of experts to tackle a wide range of new tasks. Furthermore, it can also leverage LLM's advanced reasoning capabilities to manage the output of experts for joint decisions. Lastly, we insightfully identify research opportunities of MoE and mobile edge networks.