AU Consumer Data Right energy reference data research and tooling

---

CDR Energy Research

Comprehensive research and tooling for the Australian Consumer Data Right (CDR) Energy Product Reference Data (PRD) APIs. Used to power PriceHawk (Home Assistant integration) and inform other energy projects.

Prerequisites

• Python 3
• pytest for local tests (pip install -r requirements-dev.txt)

Core commands

python3 scripts/cdr_probe_v1.py
python3 scripts/cdr_full_sweep_v2.py
python3 scripts/build_catalogue.py
python -m compileall -q scripts
python -m pytest tests/ -q

What's here

cdr-energy-research/
├── README.md                      this file
├── CHANGELOG.md
├── docs/
│   ├── shape-catalog-v1.md        first sweep (78 retailers, 10,266 plans, 1,724 sigs)
│   ├── shape-catalog-v2.md        comprehensive sweep (117 retailers, all fields)
│   ├── parser-spec.md             defensive parser implementation contract
│   ├── api-reference.md           endpoints, query params, headers, errors
│   ├── enums-reference.md         spec enum values vs observed (with ⚠ flags)
│   ├── upcoming-changes.md        15 open CDS standards-maintenance Energy issues
│   ├── registry-comparison.md     EME refdata2 vs AER PDF vs jxeeno vs ACCC
│   └── operations.md              runbook for re-running sweeps
├── data/
│   ├── aer-base-uris-jan2026.pdf       authoritative retailer registry
│   ├── aer-fact-sheet-feb2025.docx     AER guide
│   ├── eme-refdata.json                Energy Made Easy refdata2 snapshot
│   ├── retailer-index.json             merged retailer index (script output)
│   └── registry-comparison.json        registry diff (script output)
├── scripts/
│   ├── cdr_probe_v1.py            initial sample probe (5 plans/retailer)
│   ├── cdr_full_sweep_v1.py       first full sweep
│   ├── cdr_full_sweep_v2.py       comprehensive sweep w/ EME refdata2 + brand filtering
│   └── build_catalogue.py         builds compact PriceHawk catalogue from sweep cache
├── tests/                         offline regression tests (pytest)
└── cache/v1 + cache/v2            gitignored symlinks to /tmp/cdr-cache

TL;DR — where to start

1. Read docs/shape-catalog-v2.md for the comprehensive shape inventory.
2. Read docs/parser-spec.md for the implementation contract.
3. Read docs/upcoming-changes.md for what to watch (especially Solar Sharer Offer landing 1 July 2026).
4. Read docs/api-reference.md for the operational HOWTO.
5. Run scripts/cdr_full_sweep_v2.py to refresh data.
6. Run scripts/build_catalogue.py to generate dist/catalogue.json.gz for consumers.

Key endpoints

Purpose	Endpoint	Headers	Notes
Retailer registry	GET https://api.energymadeeasy.gov.au/refdata2?keys=organisations,thirdParties	none	117 orgs + 72 brokers, no auth
Plan list	GET {base}/cds-au/v1/energy/plans?fuelType=ELECTRICITY&type=ALL&effective=CURRENT&page-size=1000&brand={cdrBrand}&updated-since={iso}	x-v: 1	brand={cdrBrand} for shared endpoints; updated-since= for incremental
Plan detail	GET {base}/cds-au/v1/energy/plans/{planId}	x-v: 3	v2 retired Mar 2025

{base} is normally cdr.energymadeeasy.gov.au/<cdrCode>. The sweep's deduplicated view of the committed EME snapshot constructs three shared base URIs hosting nine brands. The authoritative AER registry maps iO Energy to the Radian base URI instead of its earlier EME record's constructed /io-energy/ URI, producing four operational shared base URIs hosting 11 brands.

Headline findings from the sweeps

• 117 committed retailer records in the current generated snapshot; the corrected sweep deduplicates future regeneration by cdrBrand
• 3 shared base URIs hosting 9 brands in generated EME-derived outputs; the authoritative AER mapping adds /radian/, producing 4 shared base URIs hosting 11 brands operationally — use the brand=<cdrBrand> query parameter to isolate a co-hosted brand
• 10,266 residential ELEC plans observed in v1 sweep (78 retailers)
• 1,724 distinct shape signatures — extreme long tail; top 30 sigs cover only 13% of plans
• 0 retailers 404 on plan detail when listed — reliability is excellent
• tariffPeriod[0].dailySupplyCharge is the ONLY observed location for daily charges — other 3 spec locations are 0/10,266
• tariffPeriod[0].dailySupplyCharge is GST-EXCLUSIVE per spec — most other AmountStrings are GST-inclusive
• EV-overlay plans (AGL Night Saver EV, Origin 360 EV, Red EV Saver, Three for Free SA) are mis-classified — they ship as TIME_OF_USE_CONT_LOAD with zero-priced rate rows; spec issue SM#710 tracks the gap
• Solar Sharer Offer (SSO) plans land 1 July 2026 — government-mandated zero-cost consumption window with daily volume cap, no spec value yet (SM#719)
• fit0.scheme=OTHER IS spec-valid (despite informal reports otherwise)
• incentives.category enum is GIFT | ACCOUNT_CREDIT | OTHER (not the often-cited DISCOUNT/BONUS/OTHER)

Sources

Source	URL	Use
AER PDF (authoritative registry)	aer.gov.au/documents/consumer-data-right-energy-retailer-base-uris-and-cdr-brands	Truth source for brand → base URI mapping
EME refdata2 (richest metadata)	api.energymadeeasy.gov.au/refdata2?keys=organisations,thirdParties	117 orgs, ABNs, contacts, logos, bill URLs
CDS spec (canonical schema)	consumerdatastandardsaustralia.github.io/standards/#cdr-energy-api_get-generic-plan-detail	Field types, enums, mandatory/optional
CDS standards-maintenance	github.com/ConsumerDataStandardsAustralia/standards-maintenance/issues?q=label%3A%22Energy%22	In-flight schema changes
jxeeno community scrape	github.com/jxeeno/energy-cdr-prd-endpoints	Auto-updated weekly; drifts from AER; uses ACCC Register + EME refdata2
ACCC Register API	api.cdr.gov.au/cdr-register/v1/energy/data-holders/brands/summary	Per SM#561, unreliable for energy — publicBaseUri flips between PRD and outage URIs based on retailer enrollment status

Operational notes

• API is public, no auth required for PRD (per AER fact sheet).
• Polite usage: 1 req/sec per base URI, 12-way parallel across distinct base URIs is fine.
• v1 sweep took 33.5 min for 78 retailers / 10,266 plans on a residential connection.
• Cache hits make re-runs free. Use ?updated-since=<iso> for incremental sync (5 min instead of 33 min).
• Spec versions: plan list = v1 (x-v: 1), plan detail = v3 (x-v: 3). v2 retired March 2025.

Catalogue output

• Local build output: dist/catalogue.json.gz and dist/manifest.json
• Scheduled publication: .github/workflows/publish-catalogue.yml (daily + manual)
• Stable latest download URL: https://github.com/Plaintext-Lab/cdr-energy-research/releases/latest/download/catalogue.json.gz

Status

Private research repo. Not for public publication without legal review (AER/CDR data is public but operational details should be reviewed).

License

No license file is currently present in this repository.