perf(nvidia): reduce refactored inference overhead - #565

Merged

voltjia merged 12 commits into

refactor/adopt-modern-infini-stackfrom

perf/cache-default-infiniops-implementation

Sep 9, 2026

Commits on Sep 8, 2026

perf(runtime): make stream access constant time
voltjia
committed
71133ce
perf(mlp): consume packed gate-up output
voltjia
committed
086f4b5
perf(ops): cache default infiniops implementation
voltjia
committed
7989ff9
perf(paged): reuse decode metadata buffers
voltjia
committed
ef62fda
perf(cache): skip unused paged cache scale upload
voltjia
committed
01643d5
perf(graph): avoid eager output snapshots
voltjia
committed
d962c8c
perf(inference): optimize reviewed execution paths
voltjia
committed
c61555f
perf(speculative): reuse graphs for token verification
voltjia
committed
a41debf
perf(paged): capture ChatGLM short decode graph
voltjia
committed
574ee4e
test(runtime): cover optimized inference paths
voltjia
committed
b936677
style: format optimized inference paths
voltjia
committed
2148deb
refactor(graph): centralize reviewed profile checks
voltjia
committed
326cd45