A set of Rust crates for building multi-runtime S3 gateway proxies. Multistore provides the components — S3 request parsing, SigV4 authentication, backend resolution, configuration, and middleware — to assemble a proxy that sits between S3-compatible clients and backend object stores. Combine them into a native Tokio/Hyper server, a Cloudflare Worker at the edge, or your own custom runtime.
Warning
This project is in early stages and should be considered experimental. APIs and interfaces may change significantly between versions.
- Unified interface — Single stable URL per dataset regardless of backend provider. Backend migrations are invisible to data consumers.
- Native S3 compatibility — Works with aws-cli, boto3, DuckDB, obstore, GDAL, or any S3-compatible client. Just set the endpoint URL.
- Flexible authentication — Anonymous access, long-lived access keys, or OIDC/STS token exchange for temporary credentials.
- Multi-runtime — Same core logic deploys as a native server in containers or as a Cloudflare Worker at the edge.
- Zero-copy streaming — Presigned URLs enable direct streaming between clients and backends without buffering.
- Extensible — Pluggable traits for request resolution, configuration, and backend I/O.
The workspace is split into reusable libraries (traits and logic) and example runtimes (executable targets). Each crate name links to its API documentation on docs.rs.
| Crate | Path | Description |
|---|---|---|
multistore |
crates/core/ |
Runtime-agnostic core: traits, S3 parsing, SigV4, registries |
multistore-metering |
crates/metering/ |
Usage metering and quota enforcement middleware |
multistore-sts |
crates/sts/ |
OIDC/STS token exchange (AssumeRoleWithWebIdentity) |
multistore-oidc-provider |
crates/oidc-provider/ |
Outbound OIDC provider (JWT signing, JWKS, exchange, backend-cloud STS federation) |
multistore-static-config |
crates/static-config/ |
Static config provider (buckets/roles/credentials) |
multistore-path-mapping |
crates/path-mapping/ |
Hierarchical path-based backend resolution |
multistore-cf-workers |
crates/cf-workers/ |
Cloudflare Workers runtime library (WASM) |
| Crate | Path | Description |
|---|---|---|
multistore-server |
examples/server/ |
Tokio/Hyper native server for container deployments |
multistore-lambda |
examples/lambda/ |
AWS Lambda runtime |
multistore-cf-workers-example |
examples/cf-workers/ |
Cloudflare Workers example for edge deployments |
multistore-static-config is not published to crates.io; follow its source link for documentation. For per-crate responsibilities and the dependency graph, see Crate Layout in the docs.
See the full documentation for configuration, deployment, and usage guides.
See CONTRIBUTING.md for development setup and release process.