E2LLM

E2LLM: Towards Efficient LLM Serving in Heterogeneous Edge/Fog Environments

arXiv:2606.03770v1 Announce Type: new Abstract: Large Language Models (LLMs) have become integral to modern applications, yet their deployment remains challenging. Beyond executing the models themselves, practical deployment must address cost efficiency, low latency, and optimal resource utilization. Conventional approaches typically assume that an entire model can be hosted on a single device, which does not hold in many real-world scenarios, particularly in Edge and Fog environments where...