Research
LeanStream
GPU-Centric LLM Runtime for Compute-I/O Scheduling
ACM MobiCom 2026
LeanStream is a GPU-centric LLM inference runtime for models whose weights do not fit in GPU memory. It continuously refines weight priorities from partial GPU results, asynchronously streams high-priority weights from storage, and executes loaded weights as they arrive to overlap I/O with computation. On edge SoCs, fine-grained CPU-GPU coordination through unified memory and custom GPU kernels delivers up to 2.1× higher token-generation throughput and 7.5× lower memory usage. A server-side extension using persistent GPU kernels for remote weight streaming is in development.
Paper
TokenFlow
Responsive LLM Serving via Preemptive Scheduling
EuroSys 2026
TokenFlow is a responsive LLM serving system with buffer-aware preemptive scheduling built on SGLang. Its scheduler tracks each request’s buffered tokens and target consumption rate, pausing requests that are overproducing and resuming those running low to maintain responsive streaming under bursty workloads. Proactive KV cache migration between GPU and CPU memory overlaps cache movement with computation, improving effective throughput by up to 82.5% and reducing P99 time to first token by up to 80.2%.
Paper
DAF
Dynamic-Quantization Training Runtime
ACM MobiSys 2025
DAF is a memory-efficient DNN training runtime that dynamically adjusts activation precision according to runtime importance. Kernel fusion reduces quantization and memory-movement overhead, while a page-based memory manager supports variable-sized quantized activations and reduces training memory usage by up to 22.9×.
Paper
DynaSpa
Dynamic Sparse GPU Runtime
ACM SenSys 2024 · Best Paper Award Nominee
DynaSpa is a dynamic sparse GPU execution framework for convolution and attention. It selects custom GPU kernel variants for input-dependent sparsity patterns and uses sparsity-aware tiling to group irregular active regions into GPU-friendly work tiles, improving shared-memory/L1 reuse and reducing global-memory traffic. The system achieves up to 7.8× operator speedup.
Paper
