Embed classic MDS protocol into Lupo

[kaysiz]

Summary

• Embeds the classic DataCite MDS protocol (formerly Poodle) in-process on dedicated hosts (mds.datacite.org / test/stage/local), so MDS URLs keep working while Lupo remains the system of record.
• Host-constrained routes serve plain-text /doi, /metadata, /media, and /heartbeat when MDS_ENABLED is on and the request host is in MDS_HOSTS; production stays off unless explicitly enabled; test/dev use mds.local only so REST specs are not captured.
• Thin protocol controllers over existing Lupo domain (DataciteDoi, CanCan, ParamsSanitizer, Bolognese format detection, shared RequestCredentials, domain landing-URL helpers). Single DOI upsert path for PUT /doi and PUT /metadata; no parallel Operations/Result stack.

Configuration

• MDS_ENABLED (default false)
• MDS_HOSTS (comma-separated; falls back to production-like defaults when blank and enabled)
• MDS_URL, MDS_REALM
• Nginx template gains a dual server_name for MDS hosts

Test plan

• bundle exec rspec spec/lib/mds_spec.rb spec/routing/mds_routing_spec.rb spec/requests/mds/
• Confirm non-MDS host still serves REST /dois and does not return MDS plain-text bodies for /doi
• Smoke: Basic auth success/failure on mds.local for GET/PUT /doi, PUT/GET/DELETE /metadata, POST/GET /media
• Smoke: draft delete allowed; findable delete returns 405
• Verify production image does not enable MDS unless MDS_ENABLED=true is set
• Optional: REST GET .../dois/:id/get-url still resolves landing URLs (shared domain policy)

[kaysiz]

CI flake fix (shard 14)

Latest push (a2665603) addresses the intermittent parallel-test (14, 16) failure:

NoMethodError: undefined method 'uid' for nil
  app/models/client.rb (Client#assign_prefix)

Not MDS-related — it showed up in repository_type_spec / reference_repositories_spec setup (create(:client) after a fixed 10.17616 prefix).

Root cause: cached_prefix_response could return a stale Prefix primary key after destroy/recreate of the same uid (memcached in test + before/after :all). assign_prefix then built a ProviderPrefix with an orphan FK and crashed on provider_prefix.prefix.uid.

Fix:

1. Cacheable#cached_prefix_response — cache only the numeric id, reload and verify uid, invalidate on mismatch
2. Client#assign_prefix — use create!, prefer the known Prefix#uid, nil-guard instead of raising

This is shared infrastructure (same code path on master); this long-lived PR hit the flake more often via shard packing / random order.

[Copilot]

Pull request overview

This PR embeds the classic DataCite MDS (legacy Metadata Store) protocol surface into Lupo, enabling plain-text MDS endpoints (/doi, /metadata, /media, /heartbeat) on host-constrained routes while keeping Lupo as the system of record.

Changes:

• Adds an MDS routing constraint (Mds.host_match?) and introduces dedicated MDS controllers, shared helpers, and protocol error handling.
• Refactors/shared logic for DOI minting and landing-URL resolution; improves API-key handling and observability tagging via a shared RequestCredentials concern.
• Adds MDS configuration defaults for dev/test, updates nginx template, and adds extensive routing/request/service specs plus documentation.

Reviewed changes

Copilot reviewed 37 out of 39 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
vendor/docker/webapp.conf.template	Adds an nginx server block for MDS hostnames and keeps the default REST server block.
README.md	Documents the embedded MDS protocol surface and configuration flags.
.env.example	Adds environment variables for enabling and configuring MDS hosts/realm/url.
Gemfile	Bumps graphql minimum patch version.
Gemfile.lock	Updates locked graphql version accordingly.
config/routes.rb	Adds host-constrained MDS routes and a catch-all for MDS-style 404s.
config/environments/test.rb	Enables MDS by default in tests for mds.local only.
config/environments/development.rb	Enables MDS by default in development for mds.local only.
config/application.rb	Loads MDS helper module and provides default MDS URL/realm env fallbacks.
lib/mds/error.rb	Introduces an MDS protocol-level error type carrying HTTP status.
lib/mds.rb	Adds MDS enablement/host matching helpers and shared protocol constants.
app/services/mds/doi_minter.rb	Adds metadata-body DOI extraction + minting logic for MDS metadata registration.
app/models/landing_url_resolution.rb	Introduces a domain object representing landing URL resolution outcomes.
app/models/concerns/helpable.rb	Refactors DOI minting + landing URL resolution into reusable methods and domain outcomes.
app/models/concerns/doi_minting.rb	Extracts random DOI generation into a shared concern.
app/models/concerns/cacheable.rb	Hardens prefix caching by caching/verifying IDs to avoid stale-orphan behavior.
app/models/concerns/authenticable.rb	Centralizes DC.* API-key detection and tightens API-key Basic auth behavior.
app/models/user.rb	Prefers API-key parsing ahead of JWT parsing and avoids JWT blacklist interaction for keys.
app/models/client.rb	Makes prefix assignment logic clearer and uses bang-creates for invariant records.
app/models/ability.rb	Grants get_urls ability for appropriate client roles (used by MDS GET /doi).
app/controllers/concerns/request_credentials.rb	Adds shared auth header parsing, blacklist handling (JWT-only), and Sentry tagging.
app/controllers/application_controller.rb	Switches to the shared RequestCredentials authentication flow.
app/controllers/datacite_dois_controller.rb	Uses shared landing-URL resolution and hardens get_dois nil handling.
app/controllers/concerns/mds/doi_lookup.rb	Adds shared DOI lookup + visibility behavior for MDS controllers.
app/controllers/concerns/mds/doi_writer.rb	Adds a shared MDS upsert path for DOI writes with CanCan checks.
app/controllers/mds/application_controller.rb	Adds MDS base controller with auth, plain-text rendering, and protocol error mapping.
app/controllers/mds/dois_controller.rb	Implements classic MDS /doi endpoints with plain-text semantics.
app/controllers/mds/metadata_controller.rb	Implements classic MDS /metadata endpoints and metadata registration flow.
app/controllers/mds/media_controller.rb	Implements classic MDS /media endpoints with plain-text semantics.
spec/lib/mds_spec.rb	Adds unit specs for MDS enablement/hosts and path-vs-body DOI matching.
spec/services/mds/doi_minter_spec.rb	Adds service specs for resolving DOI IDs from path/body formats.
spec/routing/mds_routing_spec.rb	Adds routing specs validating MDS host isolation and path routing.
spec/requests/mds/misc_spec.rb	Adds request specs for MDS heartbeat and unknown-path behavior.
spec/requests/mds/dois_spec.rb	Adds request specs for MDS /doi behavior, auth, host isolation, and Handle errors.
spec/requests/mds/metadata_spec.rb	Adds request specs for MDS /metadata behavior, validation, and visibility rules.
spec/requests/mds/media_spec.rb	Adds request specs for MDS /media behavior and visibility rules.
spec/requests/mds/api_keys_spec.rb	Adds request specs for Bearer and Basic API-key auth variants on MDS endpoints.
spec/models/ability_spec.rb	Updates ability expectations to include get_urls access.
spec/concerns/authenticable_spec.rb	Updates/extends auth decoding specs for invalid API keys and clearer error payloads.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[kaysiz]

Good catch on the nested param name.

Poodle uses the same nesting (resources :dois + nested resources :media) with only the parent :id constrained. Top-level /media/:doi_id already had doi_id: /.+/.

Fixed in 9437d471: nested media now has constraints: { doi_id: /.+/ } so slashy DOIs match (routing spec covers GET/POST /doi/10.14454/abc/media). Protocol paths unchanged — routing hardening only.

[kaysiz]

Agreed that show can move registered → findable in Lupo AASM — and that is intentional Poodle parity, not an oversight.

Classic Poodle Metadatable#create_metadata sends exactly this to REST:

"event" => "show",
"source" => "mds",
"should_validate" => "true",

(and delete metadata uses "event" => "hide", PUT /doi uses "event" => "publish" — same as this embed).

We are moving Poodle in-process, not redesigning MDS state transitions. Changing to register would diverge from production MDS/Poodle behavior (including the historical “metadata resubmit can unhide” path). Left as show with an in-code comment pointing at Poodle in 9437d471.