Skip to content

Make combine_categories = false plot per-component series (#111) - #5

Open
PabloBotin wants to merge 1 commit into
mainfrom
issue-111-combine-categories
Open

Make combine_categories = false plot per-component series (#111)#5
PabloBotin wants to merge 1 commit into
mainfrom
issue-111-combine-categories

Conversation

@PabloBotin

Copy link
Copy Markdown
Owner

Closes Sienna-Platform#111.

The bug

_plot_powerdata!'s combine_categories = false branch assigned data = powerdata.data — a Dict{Symbol, DataFrame} — and handed it to _plot_dataframe!, which takes a DataFrame. Confirmed against real data:

MethodError: no method matching _plot_dataframe!(::CairoMakiePlot, ::Dict{Symbol, DataFrame},
             ::StepRange{DateTime, Millisecond}, ::CairoMakieBackend; ...)

A documented public keyword was dead on arrival for plot_powerdata, plot_powerdata!, plot_results, plot_results! and all _plotly variants.

Alongside it, all four docstrings claimed combine_categories::Bool = false while the code defaults to true — the documented default was the one that crashed.

What false now does

Concatenates the per-category frames into the single wide frame the backends expect, dropping the duplicate DateTime columns (the time axis is supplied separately as powerdata.time).

Column naming follows the PowerAnalytics convention. This is not cosmetic — component names genuinely collide across categories. Measured on get_generation_data(results_uc):

ActivePowerInVariable__EnergyReservoirStorage     ["DateTime","test_batt"]
ActivePowerOutVariable__EnergyReservoirStorage    ["DateTime","test_batt"]      <- same name
ActivePowerVariable__RenewableDispatch            ["DateTime","SolarBusC","WindBusA"]
ActivePowerVariable__RenewableDispatch__Curtailment ["DateTime","SolarBusC","WindBusA"]  <- same names

So each column is qualified as Category__Component using the exported PowerAnalytics.COMPONENT_NAME_DELIMITER ("__", defined in InfrastructureSystems/src/common.jl:45). That mirrors PA.metric_selector_to_string (PowerAnalytics/src/metrics.jl:199-200) — how compute_all names columns when it combines frames — rather than inventing a PowerGraphics-local scheme.

It also makes the documented label_fn behavior actually work. The docstrings already said combine_categories = false is how you "see the effect of label_fn on the raw labels", but the underlying columns are bare component names with no __ to split on. Qualified, they parse correctly: label_component"Solitude", label_variable"ActivePowerVariable".

Categories are visited in sorted order — Dict iteration order is not stable, and without this the series and legend would shuffle between identical calls.

Scope

plot_powerdata and plot_results families (both route through _plot_powerdata!). _plot_fuel! untouched. ext/ untouched — both backends already consume a single wide frame, so the fix is entirely in the backend-agnostic core. The combine_categories = true path and the code default are unchanged.

Verification

Full suite 48/48, zero Error log events; formatter clean.

Directly exercised against real generation data — 16/16, and 9 adversarial probes:

check result
combine_categories = false on both backends 15 series, matching the 15 source non-DateTime columns
colliding test_batt 2 distinct series (storage In / Out)
label_component / label_variable on a qualified name "Solitude" / "ActivePowerVariable"
series order across repeated calls deterministic
source PowerData after flattening not mutated (incl. the no-DateTime edge case; repeat call does not double-prefix)
combine_categories = true unchanged, 8 series; default still true
empty category frame skipped, matching PA.combine_categories
all frames empty empty DataFrame, no throw
mismatched row counts raises ArgumentError

Known limitations

  • A theoretically ambiguous key/column pair raises rather than being prevented. Key :A with column B__C and key :A__B with column C both qualify to A__B__C; hcat then raises ArgumentError: Duplicate variable names: :A__B__C. This is a loud failure, not a silent merge, and makeunique was deliberately not used so a genuine ambiguity cannot be papered over. It is unreachable with real PowerAnalytics data, where component names contain no __ — verified that the fixture's adjacent RenewableDispatch / RenewableDispatch__Curtailment keys stay distinct.
  • The row-count mismatch error comes from DataFrames (New columns must have the same length as old columns) rather than a PowerGraphics-level guard. PA.hcat_timed_dfs would give a time-axis-specific message; not used here since all frames in a PowerData come from the same results and are structurally equal-length.
  • Series names are asserted against the internal helper rather than the plot object, because label_fn rewrites them before they reach the backend. The series count is asserted end to end on the real plot object for both backends.

…nna-Platform#111)

The `combine_categories = false` branch of `_plot_powerdata!` assigned the
`PowerData`'s raw `Dict{Symbol, DataFrame}` and handed it to
`_plot_dataframe!`, which takes a `DataFrame` — so a documented keyword
raised a `MethodError` on `plot_powerdata`, `plot_results`, and their `!`
and `_plotly` variants.

Concatenate the per-category frames into the single wide frame the backends
expect, dropping the duplicate `DateTime` columns. Component names recur
across categories — a battery appears under both the storage charge and
discharge categories, and renewable units under both dispatch and
curtailment — so qualify each column as `Category__Component` using
`PowerAnalytics.COMPONENT_NAME_DELIMITER`. That mirrors
`metric_selector_to_string`, which is how PowerAnalytics names columns when
`compute_all` combines frames, and it is what makes the documented
`label_fn` behavior work: `label_component` and `label_variable` split on
the same delimiter.

Categories are visited in sorted order, since `Dict` iteration order is not
stable and would otherwise shuffle series between calls.

The docstrings claimed this keyword defaults to `false` when the code
defaults to `true`; correct the docs rather than the long-standing behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

plot_powerdata! broken on combine_categories=false

1 participant