I build and test Python tools for AI inference, with a focus on reproducible measurements and clear failure reporting.
A small CLI for measuring non-streaming OpenAI-compatible inference endpoints. It records request latency, successful request throughput, failures, and server-reported token usage, with JSON and CSV exports.
- Quick start and measurement boundaries
- Real-model validation work and its limitations
- Contributing and review expectations
CSV export was contributed by Jeff Walsh. We keep contributor attribution accurate and review changes before merging.
- A2UI: preserve model state after rejected writes
- SageMaker examples: prevent LLMPerf monitor hangs
- SageMaker examples: preserve all XGBoost batch CSV records
These links show the current review and merge status. Forked repositories retain their upstream identity and are separate from my original projects.
I'm interested in inference tooling, reproducible bug reports, and small changes with useful tests. If you try inference-bench, please report your backend and model versions, the command you ran, and the result you expected. Keep API keys and private endpoint details out of public issues.

