Repository navigation
Install, update and export Model Bundles on Special:DataModel - #1877
Draft
JeroenDeDauw wants to merge 37 commits into
Draft
JeroenDeDauw wants to merge 37 commits into
JeroenDeDauw wants to merge 37 commits into
Conversation
JeroenDeDauw
force-pushed
the
feature/model-export-import
branch
from
November 16, 2026 01:11
38b512d to
69dce87
Compare
JeroenDeDauw
force-pushed
the
feature/model-export-import
branch
from
November 17, 2026 23:37
4062613 to
fa9fe87
Compare
Modellers build a data model in one wiki: Schemas, their Layouts and Mappings, and controlled vocabularies kept as Subjects that the Schemas point at by ID. Other wikis need that model and each later version of it. This PR adds Model Bundles, one JSON file that carries such a model. The new Special:DataModel installs a bundle from a file or a URL, updates it with a preview that keeps what the wiki changed, and exports one. It lists the wiki's Schemas, Layouts and Mappings by the bundle that installed them. The Import button on Special:Schemas reads the same files. ## The change - **Special:DataModel** (`SpecialDataModel.php`, `components/DataModelPage/`): a group per installed bundle with its version, source, record link, changes made here and any other version at its URL, expanding to its items by type; a "This wiki" group; full-page steps to install or update (`BundleInstaller`) and to export (`BundleExporter`), which lists the Schemas and the Mappings with a checkbox each. Linked from Special:NeoWiki. - **Engine** (`application/modelBundle/`): the canonical form; planning, which compares a file with the wiki and with the installed version; the run, which writes in order and then the record; and export. It imports neither persistence nor stores, so `normalizeSchemaName` and `PageTitleTakenError` move into the domain. - **Persistence**: the file format (`bundleFile.ts`), `RestDataModelRepository` over the REST endpoints and the Action API, and `MediaWikiInstallRecordRepository` for the record pages. - **Presentation** (`presentation/bundleItems.ts`, `lineDiff.ts`): the preview's grouping, status texts and results headline, and the line diff behind "Show changes". - **Special:Schemas**: Import uses the same engine and preview. It installs a file with bundle metadata and imports any other as plain items; files exported before this PR still import. Export writes Schemas only, as before, now through the bundle export. - **Docs**: ADR 41, glossary entries for Data Model and Model Bundle, the reference page `docs/api/model-bundles.md`, and a correction to `docs/api/subject-format.md`, which said extra fields on a Subject `PUT` are ignored; they answer 400. ### Not in this PR - Catalogs: lists of available bundles. - Origin badges and filters on Special:Schemas, Special:Layouts, Special:Mappings and item pages. - Edit notices on bundled items. - Preselecting bundles by questions such as "I am a museum". - Controlled Subjects: limiting who may add terms to a controlled vocabulary, designed separately. - A command line, and a detach or uninstall UI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JeroenDeDauw
force-pushed
the
feature/model-export-import
branch
from
November 18, 2026 12:02
1b4b721 to
674e15a
Compare
Model Bundles added 68 KB of script and 97 messages to the module every page with NeoWiki's interface loads. They now form NeoWikiModelBundles, a second extension in this repository, which a wiki enables with wfLoadExtension( 'NeoWikiModelBundles', "$IP/extensions/NeoWiki/extension-model-bundles.json" ). Its frontend is a ResourceLoader module of its own, which a second Vite library build writes to dist/modelBundles.js and which only Special:DataModel loads. It reaches NeoWiki through require( 'ext.neowiki' ), whose public API gains the exports it needs, among them onUiStart, which runs an initializer once NeoWiki's UI has started. A lint rule keeps it off NeoWiki's internals. Special:Schemas goes back to importing and exporting Schemas only. When Model Bundles is loaded, it refers a file that holds more, or a bundle's metadata, to Special:DataModel; otherwise it imports the file's Schemas and says it leaves the rest. Special:NeoWiki shows the pages extensions list under NeoWikiOverviewLinks in their extension.json, which is how the Data model card comes back. The dev stacks, the PHP CI and the release bundle carry the extension. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the Special:Schemas import dialog imported a file without writing an install record, and Special:Schemas imports Schemas only again. Special:DataModel installs every file as a Model Bundle, under the name given to one without, so the mode, its status texts, the "Belongs to model bundle" note and the import edit summary go. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deployments of the published image, such as a Helm release, turn Model Bundles on with NEOWIKI_MODEL_BUNDLES=1. The image leaves them off by default, and dev stacks load them as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hors Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new item the admin left unticked was not recorded, so every later update offered it again as New, ticked. The install record now lists such items as declined, and later updates offer them as Not installed, unticked, however the bundle changed them. Ticking one installs it as usual. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A catalog is a JSON file at a URL that lists bundles by the URLs of their files, each a model or a destination, with the entries it requires. $wgNeoWikiModelBundleCatalogs names a wiki's catalogs, which the browser reads, and $wgNeoWikiModelBundlePreselection the entries the coming setup page ticks. "Install model bundle" lists the catalogs' entries by kind, saying which the wiki has installed, and loads the one chosen from its URL, refusing a file that holds another bundle. An entry that requires bundles the wiki lacks installs them too: the bundles are planned together, each after those it requires, so that what an earlier one provides counts as present for a later one and belongs to the earlier one. One preview shows them all, and the install writes each in turn with its own record, so that running it again finds what was written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Special:DataModel/setup takes a wiki to its first data model in steps. It asks how the wiki works first, page-first or subject-first and the data languages, saving only what is changed to MediaWiki:NeoWiki, as every first install on a wiki without Subjects now does, since the pages of installed Subjects depend on it. Then it offers the models of the catalogs, or starting empty, then the destinations that the chosen model and the installed bundles serve, ticking the entries that $wgNeoWikiModelBundlePreselection names, and installs them in one preview with what they require. It ends with a link to create a Subject. Special:NeoWiki offers the setup to users who may install bundles while the wiki has no Schemas, in place of the note on creating a first Subject, and counts the installed bundles with another version at their source. Special:DataModel offers it while the wiki has no data model. Extensions show such notices on Special:NeoWiki through registrar.registerOverviewNotice(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… steps On a wiki without Subjects, Special:NeoWiki leaves the recent Subjects out for a user who gets the note on creating a first Subject. A notice in the note's place, such as the setup of the data model, now leaves them out too, where an empty table showed under it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The preview of an install or update listed every item of every bundle, so the setup's review of a model with its destinations ran to several screens before its Install button. Each bundle now gets one line, such as "NeoWiki demo model version 1.0: 8 schemas, 30 subjects, all new", with a button that shows all its items. What needs a decision stays in view without it: items this wiki has otherwise or changed too, items that cannot be installed or were not installed before, and those the bundle no longer has. After an install, the items are counted under their results, and only the failures are listed until the bundle is expanded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bar above the setup names its steps, as far as the choices made lead through them: How this wiki works, Model, Destinations and Review. It marks the current one for assistive technology, and the completed ones are buttons that return to them. Each step is a history entry, named in the URL as ?step=..., so the browser's Back and Forward move between steps with the choices kept, and keep the review as it was left while they do not change. The setup's own Back goes back through the history where it can, so that Forward returns. Session storage keeps the choices through a reload; a URL that names no step starts afresh. An install's end takes the review's place in the history, and going back from it reloads the page, as the plans compare the wiki as it was before the install. The setup lives on Special:DataModel/setup only: its Cancel and the way back at its end link to Special:DataModel, and the empty data model links to the setup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The other cards on Special:NeoWiki describe what their page is for in a line; the Data model card's description ran to two. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collapsed, the preview's bundles are one line each, and the space kept between them for their lists of items set the lines far apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The space under each summary line was there for the items listed under it, and with the skin's paragraph margins it set the lines of collapsed bundles far apart. The lists and results now keep the space above themselves, and the line is no longer a paragraph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Message texts as the text reviews settled them: the setup's intro and card, the extension's description, "Left out earlier" for an item an earlier install left out, the dependence conflict, "another version" at a bundle's source, and a catalog file that names no bundle at all. The extension's word for a subject takes its own prefix, so that it cannot clash with a core message later, and message docs no longer mention Special:Schemas uses that are gone. The docs link to the installation page's load step instead of repeating it, say where the setup is offered and that the module loads on Special:NeoWiki too, give Catalog a glossary entry, and correct the order of extension cards and the onUiStart wording. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The settings step of the setup gets Cancel beside Continue, and the same column width as the steps after it. Before a first install from a file or URL, the same step is headed as the install, and Cancel goes back to the data model. Each wiki type says what it means for installed subjects: page-first for wikis built around articles, with pages titled by name; subject-first for wikis that are mostly data, with pages titled by ID. Where the settings cannot be changed on the wiki, the note says they are set in LocalSettings.php. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After an install or update, each bundle gets one line saying what it came to and how many items were created, replaced or removed, with the items that failed listed under it. The headings of what was written, empty unless expanded, are gone, and so is the summary line above. The setup ends with "Create a subject" as its primary button and "Open the data model" beside it. Starting empty says that nothing was installed, and leads to creating a schema instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ary gives In the preview of an install, a bundle installed because the chosen ones require it says "Required by" and names them, in the setup and on Special:DataModel alike. Once expanded, an item's row leaves out its status where the bundle's summary line already gives it of every item, such as "all new". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A catalog that could not be read is reported with its cause: an HTTP status from its server, a file that is no catalog, or, when the browser could not fetch it at all, that its server may not let other websites read the file, which is the likely cause. When no catalog could be read, the setup says that there is no model to offer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Special:DataModel on a wiki without a data model makes the setup its primary button, and leaves out Export, which would have nothing to export. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Special:Schemas gets the notice extension point Special:NeoWiki has: registerSchemasNotice shows a component above the Schemas. Model Bundles registers its notice there too, and loads it on Special:Schemas for users who may install bundles while the setup is offered, so that an empty wiki's Schemas page points to the setup. The setup is offered while the wiki has no Schemas and no installed bundle, on Special:NeoWiki as on Special:DataModel. The hook that loads the notice is named for it now that it serves both pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured on the current build: the extension's module is 100 KB of script, 24 KB gzipped, some 17% of NeoWiki's module, with 155 messages NeoWiki's module does not already load. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The settings step started from the values and revision the page was loaded with. Shown again before a reload, as when the install is left and opened again, it showed the wiki type from before the save, and Continue then told the later steps that type, so a subject-first choice reverted to page-first for the install. A second change would also have been saved over the old revision. A successful save now writes the saved values and the new revision back into the setup's configuration, which the step starts from. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Installing a Subject again after an update removed it puts it back on the page the removal left empty. The install looked that page up by the bare Subject ID, in the main namespace, but a subject-first wiki keeps the page in the namespace of Subject pages, as Subject:<ID>, so the install failed with the title taken. The fallback now takes the title of the page that refused the create. The test wiki titles subject-first pages as the wiki does, so that the test fails without the fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tests pin that creating and replacing a Subject send its Dependent Subjects, that saving over a Schema, Layout or Mapping the wiki has goes over the revision it was read from, and that a record of a Subject without a label reads back as it was saved, the label left out rather than written as null. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The canonical form of a Schema, Layout or Mapping was written out where the record hashes its version, where the wiki's version is read, where the file's version is planned, where the run records it, and where an export writes it. Any copy drifting from the others would show items as changed. They now go through one rule, and the plan builds what the record keeps of the file's version, which the run writes as it is. How NeoWiki saves a Schema or Layout, which reading a page, a file and a record all apply, is one function too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bundle installed from a URL keeps that URL as its source when an update reads it from a file. Until now the record took "file" instead, which silently ended the checks for another version at the URL. To decide that where the record is written, each plan carries where its bundle was read from, rather than the run being told it apart. With that, the preview's default ticks follow from the status alone, and the installer reads a file, a URL and a catalog entry through one function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Model Bundles build takes NeoWiki's Vite config and overrides only what differs: its entry, its output names and ext.neowiki as an external. Widths and heights use Codex size tokens instead of rem values. The line diff keeps the lines both versions start and end with as they are, so that the table it compares the rest by grows with the change rather than the item. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bundle a record names was cut out of the page title by the length of "MediaWiki:", but the API gives titles with the wiki's own name of the namespace, so on a Korean, Persian, Hindi, Bengali or Thai wiki the name came out cut wrong. The name is now read after the namespace, whatever it is called. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check for another version at the URL a bundle was installed from parsed the whole file, so a version that this wiki cannot read, such as one made for a later NeoWiki, was not told of at all. It now reads only the file's name and version; installing it says what is wrong. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Special:DataModel gave every reader, anonymous ones included, the URLs of the wiki's catalogs and the preselection in its JavaScript config. Only users who may install bundles use them, so only they get them now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Docker README says where to set the catalogs and the preselection for an image: in a LocalSettings.local.php mounted next to the image's LocalSettings.php, which loads it last. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A notice on Special:NeoWiki told the page by an event that it took the place of the first steps, which the page tracked in a set of positions. It is registered with that now: registerOverviewNotice( component, replacesFirstSteps ). Special:Schemas gets the same: registerSchemasNotice( component, replacesEmptyNote ) lets a notice take the place of the note that the wiki has no Schemas, as the setup card of Model Bundles now does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tests now pin that removal is offered for Subjects the bundle no longer has that point only at each other, and refused for one whose list of referencing Subjects the wiki cut short; that a Subject whose Dependent Subject it holds is of another Schema on the wiki is refused; that an export writes each section's items by key; that a Schema saved by hand reads as NeoWiki saves it; that Special:NeoWiki's cards are in the reader's language; that the setup is handed the revision of MediaWiki:NeoWiki to save over once that page exists; and that no destination is offered whose requirements the model meets only in part. Each fails on the change it guards against; none found a bug. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The REST repository's test client matched requests by the bytes of their URL, so reordering a query or encoding a path differently broke tests that still sent the same request; it now matches the decoded path and the query parameters. The catalog entries are read by the role of each part rather than by its tag. Two PHP tests compare associative arrays with assertEquals instead of assertSame, whose key order does not matter here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the plain import gone, the extension's code installs bundles, yet its plans, runs, outcomes, statuses, composable, results component and CSS classes were still named for importing. They are renamed mechanically: InstallPlan, installRun, useBundleInstall, InstallResults and so on. Its public API no longer exports sameJson, which no extension uses, from NeoWiki's config editing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Modellers build a data model in one wiki: Schemas, their Layouts and Mappings, and controlled vocabularies kept as
Subjects that the Schemas point at by ID. Other wikis need that model and each later version of it. This PR adds Model
Bundles, one JSON file that carries such a model. The new Special:DataModel installs a bundle from a file or a URL,
updates it with a preview that keeps what the wiki changed, and exports one. It lists the wiki's Schemas, Layouts and
Mappings by the bundle that installed them. The Import button on Special:Schemas reads the same files.
Human input
Goal and shape: goal set by @JeroenDeDauw; Claude proposed the shape and he redirected it in several places; Claude
made the technical design within it.
handle name collisions as that dialog does; design the update side now, as "doing just the first steps and ending
with a shape that doesnt work or is poor for the remaining ones isnt nice"; a new concept, "Bundle" with a prefix
(Claude chose "Model"); one PR; Fable subagent passes for simplification, product
shape and text.
place for the bundles", so the bundles got a page of their own, a "data model overview" showing installed bundles
and where each item came from. On that page a file is always installed as a bundle, "even if it only contains
schemas". After trying it: the install's result line was "a poor success message"; it now reads "Installed model bundle
Foo", in the wiki's success style.
deleting the record page detaches, the detach UI was dropped. He weighed retiring Special:Schemas' Import and Export
for a single mechanism against people who "just want to copy some schemas between their wikis"; they stay, on the
same engine. After trying it, he asked whether the export took "all or no Mappings"; it now lists them one by one.
the installed Schemas, Layouts and Mappings, not just hashes, so a per-property merge can follow when a wiki needs
it, without migrating records.
Outcome: the shape was agreed in conversation before the build. @JeroenDeDauw tried the export form and an install
on a dev wiki; what followed his try-out — per-Mapping export, the result headlines and the fixes from the AI shape
review he asked for — no human has seen. Architecture and code not reviewed by a human. Left for you: export a bundle
from a dev wiki, edit the file, then install and update it on Special:DataModel; Import and Export on Special:Schemas;
the architecture (ADR 41); the calls below.
Claude's calls
drops the hashes but cannot tell a change made in the bundle from one made on the wiki. Claude kept three-way.
permission checks, validation and history.
MediaWiki:NeoWiki/Model bundles/<name>.json, holds version, source, each installed Schema,Layout and Mapping in canonical form, and per Subject its Schema, label and a cyrb53 hash. Stored items are
re-canonicalized before comparing. cyrb53 is written out because SubtleCrypto exists only on HTTPS pages.
the file's title, else its label, else its ID; a Subject restored after removal goes on the empty page titled by
its ID, never on one titled by its label.
the wiki lacks a Schema its Dependent Subjects use.
its Subjects go too; one kept back says why. Tick-all never ticks a removal.
to save is reported, with installing the file again as the remedy.
show as changed here. A bundle install refuses them as conflicts.
a bundle's http(s) source URL for another version.
wikis with read-restricted content, where the Subject listing can end early; an install into a second wiki, as every
browser run used one dev wiki; Layouts in the browser runs, which the run, export and repository specs cover.
changed in both on every update.
editinterfaceandeditsitejson, so by default only administrators install and updatebundles.
baseRevisionId, an updatecan overwrite a Subject someone edited after the preview, as every Subject write can today.
The change
SpecialDataModel.php,components/DataModelPage/): a group per installed bundle with itsversion, source, record link, changes made here and any other version at its URL, expanding to its items by type; a
"This wiki" group; full-page steps to install or update (
BundleInstaller) and to export (BundleExporter), whichlists the Schemas and the Mappings with a checkbox each. Linked from Special:NeoWiki.
application/modelBundle/): the canonical form; planning, which compares a file with the wiki and withthe installed version; the run, which writes in order and then the record; and export. It imports neither
persistence nor stores, so
normalizeSchemaNameandPageTitleTakenErrormove into the domain.bundleFile.ts),RestDataModelRepositoryover the REST endpoints and the ActionAPI, and
MediaWikiInstallRecordRepositoryfor the record pages.presentation/bundleItems.ts,lineDiff.ts): the preview's grouping, status texts and resultsheadline, and the line diff behind "Show changes".
other as plain items; files exported before this PR still import. Export writes Schemas only, as before, now through
the bundle export.
docs/api/model-bundles.md,and a correction to
docs/api/subject-format.md, which said extra fields on a SubjectPUTare ignored; theyanswer 400.
Not in this PR
Verification evidence
DateTimeDisplay'shost-timezone test, which depends on this host's 12-hour locale and covers code this PR does not touch;
vue-tsc -b,ESLint and Stylelint clean.
make csclean and PHPUnit passing; no PHP changed after those runs.each mutant caught by a test.
same wiki found every item unchanged and recorded it, which checks the hash round trip.
changed here, new, removed, removal blocked, deleted here with re-create.
allows CORS, and showed the download-the-file message for one that does not.
a Dependent Subject installed. Version 2 replaced a Schema over its read revision and removed the label-less
Subject; version 3 put it back on its old ID page.
pr-reviewandtext-review. The later commits fix what they found, apartfrom base revisions on Subject writes and showing write warnings.
review and a text pass over the docs and UI messages, each by a subagent; the last commits apply their findings.
Text pass and this description's update:
Fable 5.1.POST /subjectswith an unknown Schema, a missing relation target or a relationdefault to a missing Subject is accepted with warnings; a
PUT /subject/{id}withid,schemaor page fieldsanswers 400; a Dependent Subject ID held by another Subject answers 409.