Efficient Inference Scheduling With Edge and Cloud Collaboration for LLMs Under Resource Constraints

Large language models (LLMs) have transformed artificial intelligence (AI) applications but require substantial computational resources, posing challenges for real-time deployment. Cloud computing offers powerful resources but suffers from high latency and bandwidth costs. In contrast, edge computing reduces latency but is limited by its computational capacity for large-scale models. Consequently, edge–cloud collaboration emerges as a critical approach to harness the strengths of both cloud and edge systems, enabling dynamic task allocation to optimize resource utilization and reduce latency. However, deploying LLMs in edge–cloud environments present key challenges, including efficient task offloading, managing communication delays, addressing task dependencies, and balancing performance with energy efficiency, particularly for edge devices. To tackle these challenges, this article proposes a reinforcement learning (RL)-enhanced edge–cloud collaborative framework designed to improve task allocation, optimize resource utilization, and support real-time processing. Key contributions include an RL-based task offloading framework, a Deep Q-Network algorithm for optimizing task assignment, and a task scheduling algorithm that updates server states for continuous optimization. Extensive experimental evaluations demonstrate that the proposed framework outperforms existing methods in terms of success rate, throughput, and energy efficiency, achieving a significant reduction in energy consumption and improved task processing times. This work provides a solution for deploying LLMs in edge–cloud settings, enhancing intelligent application performance and offering insights into optimizing resource allocation and task scheduling

Cloud computing · Edge computing · Enhanced Data Rates for GSM Evolution · Inference · Cloud Computing and Resource Management · Distributed and Parallel Computing Systems · IoT and Edge/Fog Computing