Edge LLM Inference via Chain of Agents

Jun 1, 2026 · 1 min read
projects

A Chain-of-Agents approach to memory-efficient large language model inference on resource-constrained edge devices. Accepted as a poster at the NeurIPS 2026 Workshop on Machine Learning for Systems (MLForSys): Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell (arXiv:2610.07782). Agent decomposition bounds peak KV cache per query about 60% below single-pass and RAG baselines at accuracy parity; the persistent memory tier costs a little cache and buys no measurable accuracy on single-question benchmarks.

LLM Agents Edge AI Inference Optimization
Hochan Son
Authors
Hochan Son (he/him)
Database Architect · SRE & DevOps Engineer · ML/GenAI Practitioner
Hochan Son is a distributed systems architect and site reliability engineer who builds data infrastructure, ML pipelines, and high-scale distributed systems, working the gap between GenAI/LLM research and the systems that run it in production. His graduate research at the UCLA Trustworthy AI Lab spans synthetic tabular data generation and agentic, intent-conditioned model selection. His background spans Adtech, entertainment, and enterprise — from MySpace and Hallmark Labs to Branch.io and ADP.