Inferencing has become ubiquitous across cloud, regional, edge, and device environments, powering a wide spectrum of AI use cases spanning vision, language, and traditional machine learning applications. In recent years, Large Language Models (LLMs), initially developed for natural language tasks, have expanded to multimodal applications including vision speech, reasoning and planning each demanding distinct service-level objectives (SLOs). Achieving high-performance inferencing for such diverse workloads requires both model-level and system-level optimizations.

This talk focuses on system-level optimization techniques that maximize token throughput , achieve user experience  metrics and inference service-provider efficiency. We review several recent innovations including KV caching, Paged/Flash/Radix Attention, Speculative Decoding, P/D Disaggregation, KV Routing and Parallelism, and explain how these mechanisms enhance performance by reducing latency, memory footprint, and compute overhead. These techniques are implemented in leading open-source inference frameworks such as vLLM, SGLang, Hugging Face TGI, and NVIDIA’s TensorRT-llm, which form the backbone of large-scale public and private LLM serving platforms.

Attendees will gain a practical understanding of the challenges in delivering scalable, low-latency LLM inference, and of the architectural and algorithmic innovations driving next-generation high-performance inference systems.

with John C. Havens, Global Lead, IEEE Planet Positive; Founding E.D., IEEE AI Ethics Initiative and Dr. Maike Luiken, Chair, IEEE Planet Positive 2030

Date/Time: Tuesday, December 9, 2025; 9am - 10 am Pacific Time

Abstract:

For the December 2025 talk, IEEE SusTech Talks online will join the members and supporters of IEEE Planet Positive 2030 for their “End of Year Town Hall Celebration” to share updates from 2025 along with plans for 2026. The growth and evolution of IEEE Planet Positive 2030 Communities efforts beginning in Miami has been a major focus in 2025, and plans are being made to host events and other potential Communities in China, India, Europe and Latin America in 2026. Reports on specific initiatives will include:

• an Education Committee and an Executive Advisory Committee designed to drive the work forward strategically, pragmatically, and at scale.
• Communities ‘toolkits’ that expand on and share the insights in PP2030’s compendium Strong Sustainability by Design
• Accountable Sustainability by Design and data frameworks, utilizing the expertise of the PP2030 metrics experts.
• engaging with like-minded organizations to create Communities.
• identifying new ways to a regenerative and sustainable future.

For more information, see the IEEE Planet Positive 2030 web page.