https://ohsono.github.io/tags/inference-optimization/Inference OptimizationHugoBlox Kit (https://hugoblox.com)en-usMon, 01 Jun 2026 00:00:00 +0000https://ohsono.github.io/media/icon_hu_eee4a95885829ab2.pnghttps://ohsono.github.io/tags/inference-optimization/https://ohsono.github.io/projects/edge-llm-inference/Mon, 01 Jun 2026 00:00:00 +0000https://ohsono.github.io/projects/edge-llm-inference/<p>A Chain-of-Agents approach to memory-efficient large language model inference on resource-constrained edge devices. Accepted as a poster at the NeurIPS 2026 Workshop on Machine Learning for Systems (MLForSys): <em>Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell</em> ( ). Agent decomposition bounds peak KV cache per query about 60% below single-pass and RAG baselines at accuracy parity; the persistent memory tier costs a little cache and buys no measurable accuracy on single-question benchmarks.</p>