Repository navigation
feat(energy): GPU energy meter with pluggable power models, validated on ML.ENERGY H100 data - #2
Open
therealansh wants to merge 1 commit into
Open
therealansh wants to merge 1 commit into
therealansh wants to merge 1 commit into
Conversation
Adds GPU energy accounting to run and replay. A one-method sim.PowerModel prices each batch sim.EngineDriver times with engine.Engine.StepTime, in watts per GPU, from the step time, prefill/decode tokens, batch size and (when the engine exposes sim.StepBreakdown) per-resource busy time. Time between steps is charged at idle_w. With energy off nothing changes; with it on every Result metric section is unchanged and only input.options records the flags. Models (sim/energy, strict YAML): tdp, utilization (Vidur-Energy, arXiv:2507.11417 Eq. 1-2), resource_share (time-weighted per-resource power; refused on an engine without a breakdown, which the blis-vllm 0.32 engine is). Flags: --energy-config, --energy-report, --energy-trace. Data-parallel groups, P/D and pipeline parallelism are refused rather than under-counted (R1). experiments/energy/run.py scores the models against the ML.ENERGY leaderboard (github.com/ml-energy/leaderboard @ 2bdd0070186956688e1f842dc28077980fe64013, sha256-pinned, fetched at run time, not committed): calibrated on Llama-3.1-8B, scored on Qwen3-14B, Llama-3.1-70B TP4 and Qwen3-30B-A3B. Held-out power MAPE on this base: utilization 13.2%, constant mean 22.4%, TDP 35.8%. The A-vs-B comparison (resource_share 33.3%) ran on the kernel-adapter base 29f783c; both are in experiments/energy/findings.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds GPU energy accounting to
blis runandblis replay. A power model converts every step the engine prices into watts per GPU, and the meter integrates those watts over time. I compared two power models against measured H100 power from the ML.ENERGY benchmark. As per the gate fixed before any prediction, model A (utilization curve, from Vidur-Energy) is selected.Design
sim.PowerModelhas one method,StepPower(StepSample) float64, which returns watts per GPU.StepSamplecarries the step time, prefill tokens, decode tokens, batch size and the per-resource busy time (when the engine gives it).sim.EnergyMetersits insim.EngineDriver.timeBatchand records every batch priced byStepTime. Time between steps is charged atidle_w.sim/energyholds the models and a strict YAML loader. A new model needs only one type and onecase.--energy-config,--energy-report(JSON) and--energy-trace(one CSV row per step).input.optionsrecords the new flags.Energy (J) = Σsteps P × tstep × GPUs + idle_w × GPUs × tidle. Facility energy = PUE × GPU energy, and gCO2 = kWh × grid intensity.
Design comparison
B's weights sum to one over resources, so it cannot show compute and memory both busy in the same step. Even with weights bounded to 0–1000 W (post-hoc, not part of the gate), it needs W_sm = 1000 W, which is above TDP, to follow the measured rise in power.
Method
lm-arena-chat, H100 SXM, vLLM, BF16, steady state [4], [5]. The data is pinned to commit2bdd007with sha256 checks, downloaded at run time and not committed.max_num_seqsand output-length histogram, with a burst arrival. The steady-state window is steps with batch ≥ 0.9 × the measured average batch. Prompt length is 512 tokens, and 256 and 1024 are checked for sensitivity.29f783c7(kernel adapter v0.3.0), where B can be scored. Study 2 ran on this branch (blis-vllm 0.32 engine) and covers A and the baselines.Results
These are study 2 numbers; study 1 is in
experiments/energy/findings.md.The J/token error comes mostly from step time, not power:
Fitted A for H100: P_floor 436 W, P_max 689 W, MFU_sat 0.15, γ 0.75.
Over-provisioning (E3): Llama-3.1-8B, 20 req/s, 4,000 requests, PUE 1.2, 400 gCO2/kWh.
Going from 1 to 8 instances uses 5.8× the energy for a 3.3× better p99 TTFT. Idle gaps stay below 0.2% of energy, because a GPU decoding small batches still draws about 65–70% of TDP.
Parallelism (E4): Llama-3.1-70B, 1,024-request burst.
E2 (workload shape) and the sensitivity tables are in
experiments/energy/findings.md.Not supported yet (refused, not under-counted)
resource_shareon an engine withoutsim.StepBreakdown.Only GPU-board energy is counted. Host CPU, DRAM and network are not counted, and asynchronous KV offload transfers are charged at idle power.
Testing
go test ./...,go vetandgofmtpass. Unit tests cover the energy laws (additive, idle clamp), prefill/decode split, refusals and YAML validation../sim/energyis added to the CI test matrix as a new entry.runtimeblock. With energy on, every metric section matches.python3 experiments/energy/run.py gatereproduces the tables in about 30 s.References
2bdd0070186956688e1f842dc28077980fe64013.Kindly review.
🤖 Generated with Claude Code