diff --git a/README.md b/README.md index 63c96e0..c90adf5 100644 --- a/README.md +++ b/README.md @@ -288,6 +288,32 @@ connections cannot be told apart, so an agent may hold only one of them. Declaring a second one in `gete.yaml` is fine; naming both under one agent's `connections` is what `gete validate` refuses. +Some services announce themselves without a prefix: their access tokens are +JWTs whose `iss` claim names the service's own host. `tokens.format: jwt` says +so, and gete holds the connection to it — a token that is not such a JWT is +refused, however little else claims it. In return the connection is no longer +accepted by elimination, and an agent may hold it next to one that is: + +```yaml +connections: + zendesk: + base_url: https://acme.zendesk.com + tokens: + format: jwt # tokens are JWTs issued by acme.zendesk.com, nothing else +``` + +An agent may then read Zendesk and write to `internal-api` above, which keeps +the one place a token is taken by elimination. What the two may not share is +an issuer: one authorization server in front of both services puts its host +in either connection's tokens, and `gete validate` refuses that pairing for +the same reason it refuses two anonymous connections. + +Whether a provider issues such tokens can be a setting rather than a promise — +Zendesk does so only with token expiry turned on — so the declaration belongs +in `gete.yaml`, not in a catalog entry that would promise it for every +installation. It cuts both ways: turn that setting off and every token the +connection is handed is refused until the declaration goes with it. + A service whose root moves with the installation — the tenant in a subdomain, or a deployment you host — writes its URLs around `{base_url}`, and the installation fills it in: diff --git a/src/gete/catalog/connections/zendesk.yaml b/src/gete/catalog/connections/zendesk.yaml index f0a96f8..9e14064 100644 --- a/src/gete/catalog/connections/zendesk.yaml +++ b/src/gete/catalog/connections/zendesk.yaml @@ -6,10 +6,15 @@ docs: https://developer.zendesk.com/api-reference/ # authorization and token endpoints move with it. Nothing here is an address # until the installation sets connections.zendesk.base_url in its gete.yaml; # hosts stays empty because the only host comes from that root. -# Zendesk issues JWT-shaped access tokens whose claims name no issuer, and -# the service announces no prefix. A JWT like that says as little about its -# origin as an opaque token, so it is accepted by elimination, and an agent -# may hold only one connection of this kind. +# Zendesk announces no prefix, and what its access tokens look like is the +# installation's own setting rather than the service's promise: with token +# expiry turned on they are JWTs issued by the subdomain, and without it they +# are JWTs whose claims name no issuer, which says as little about their +# origin as an opaque token. The catalog cannot promise either, so tokens are +# accepted by elimination here - and an agent may hold only one connection of +# that kind. An installation that has expiry on says so with +# connections.zendesk.tokens.format: jwt in its gete.yaml, and then this +# connection no longer takes that place. token_prefixes: [] oauth: @@ -40,13 +45,20 @@ setup: | here. When they do not, the consent screen still appears and the exchange fails afterwards. -# The accepted example records the observed shape - a JWT whose claims name -# no issuer - not a contract: the service promises nothing about its tokens. -# verified is absent because no authorization has been taken through Gemini -# Enterprise yet. + Zendesk can be set to issue access tokens that expire. Those tokens are + JWTs issued by the subdomain, and an installation that has the setting on + may declare connections.zendesk.tokens.format: jwt in its gete.yaml, which + frees an agent to hold Zendesk beside another connection whose tokens + announce themselves no way at all. The declaration is held to: turn the + setting off again and every Zendesk token is refused until it is removed. + +# No accepted example: which of the two shapes arrives is the installation's +# setting, and an entry fixing one of them would contradict an installation +# that has the other - the declared one most of all, whose accepted tokens +# name a subdomain the catalog cannot know. What is refused is refused either +# way. verified is absent because no authorization has been taken through +# Gemini Enterprise yet. examples: - accepts: - - "eyJhbGciOiJFZERTQSJ9.eyJleHAiOjB9.sig" # claims: {"exp": 0}, no issuer rejects: - "ya29.a0AfH6SMB" # Google access token # Claims: {"iss": "https://accounts.google.com"}, as in an ID token. diff --git a/src/gete/connection/checks.py b/src/gete/connection/checks.py index 79d77ba..e74b513 100644 --- a/src/gete/connection/checks.py +++ b/src/gete/connection/checks.py @@ -44,26 +44,62 @@ def elimination_problems( ) -> list[str]: """Describe connections that cannot be held together, or return an empty list. - A connection without token prefixes accepts whatever no other connection - claims. Two of them are indistinguishable: a token issued by either - authorization passes as the other's. Only the connections handed to the - same agent can be confused that way, so the pairing is what is refused, - not the second prefixless connection an installation declares. The - registry holds every connection gete ships as well, and declaring a - service of your own must not depend on which of those announce themselves. + A connection that announces itself neither by a token prefix nor by a + declared token format accepts whatever no other connection claims. Two of + them are indistinguishable: a token issued by either authorization passes + as the other's. Only the connections handed to the same agent can be + confused that way, so the pairing is what is refused, not the second such + connection an installation declares. The registry holds every connection + gete ships as well, and declaring a service of your own must not depend on + which of those announce themselves. + + A declared format announces the service in every token, so it does not + take that one place - unless one issuer stands behind two of the + connections judging tokens by issuer, which is the same confusion by + another route. An anonymous connection is one of those: it takes a JWT + naming its own authorization server as readily as a declaring one does, + so a shared issuer confuses the two whichever of them declared. """ - prefixless = sorted( - connection_id - for connection_id in set(connection_ids) - if not registry.get(connection_id, include_retired=True).token_prefixes - ) - if len(prefixless) < 2: - return [] - return [ - f"{', '.join(prefixless)} declare no token_prefixes; only one of an " - "agent's connections may accept tokens by elimination, or a token from " - "one of them would be accepted as another's" + connections = [ + registry.get(connection_id, include_retired=True) + for connection_id in sorted(set(connection_ids)) + ] + problems: list[str] = [] + anonymous = [ + connection + for connection in connections + if not connection.token_prefixes and connection.token_format is None + ] + if len(anonymous) >= 2: + problems.append( + f"{', '.join(connection.id for connection in anonymous)} declare " + "neither token_prefixes nor tokens.format; only one of an agent's " + "connections may accept tokens by elimination, or a token from one " + "of them would be accepted as another's" + ) + # A prefix decides on its own and before any issuer does, so a connection + # held to one judges no token by who issued it, and shares an issuer with + # nothing. + by_issuer = [ + connection + for connection in connections + if connection.token_format is not None or not connection.token_prefixes ] + for index, connection in enumerate(by_issuer): + for other in by_issuer[index + 1 :]: + if connection.token_format is None and other.token_format is None: + # Already refused above, by everything the pair fails to say + # about itself rather than by the one issuer they happen to + # share; naming it here would say it twice. + continue + shared = connection.issuer_hosts & other.issuer_hosts + if shared: + problems.append( + f"{connection.id}, {other.id} both accept tokens issued by " + f"{', '.join(sorted(shared))}; a token from one of them " + "would be accepted as the other's" + ) + return problems def connection_problems(connection: Connection, registry: Registry) -> list[str]: @@ -101,6 +137,14 @@ def connection_problems(connection: Connection, registry: Registry) -> list[str] problems.append( f"hosts: {entry} never applies; the bare {host} entry admits every path" ) + if connection.token_format is not None and connection.token_prefixes: + # The format decides on its own, so the prefixes beside it are never + # read - and a reader would have to know that to see which of the two + # rules the connection is actually held to. + problems.append( + f"tokens: format {connection.token_format} decides on its own; the " + "token_prefixes declared beside it are never read" + ) for other in registry.all(include_retired=True): if other.id == connection.id: continue diff --git a/src/gete/connection/registry.py b/src/gete/connection/registry.py index 8d82455..148b529 100644 --- a/src/gete/connection/registry.py +++ b/src/gete/connection/registry.py @@ -30,6 +30,10 @@ # otherwise a prefixless service accepts Google tokens by elimination. GOOGLE_ACCESS_TOKEN_PREFIX = "ya29." +# The one token format a connection can commit to. A service that issues it +# names itself in every token, which is what a prefix would otherwise do. +JWT_FORMAT = "jwt" + # Stands for the root of a service that differs per installation, wherever the # connection's URLs are written around it. A definition meant to be shared # cannot spell a tenant's host, and a stand-in host would be a name a stranger @@ -175,6 +179,11 @@ class Connection: # never travels to them. Empty means downloads stay on hosts. redirect_hosts: frozenset[str] = frozenset() token_prefixes: tuple[str, ...] = () + # The format the service commits to issuing, JWT_FORMAT or None. A + # connection that declares one takes nothing else, so it announces itself + # as surely as a prefix does and is not accepted by elimination; one this + # gete cannot judge takes nothing at all. + token_format: str | None = None # Prefixes declared by every other connection in the registry, filled in by # Registry. A connection without prefixes of its own accepts a token only # if none of these match, so a bare from_mapping() connection judges more @@ -213,6 +222,7 @@ def from_mapping(cls, data: Mapping[str, Any]) -> "Connection": hosts=frozenset(hosts), redirect_hosts=frozenset(data.get("redirect_hosts", ())), token_prefixes=tuple(data.get("token_prefixes", ())), + token_format=data.get("tokens", {}).get("format"), base_url=base_url, docs=data.get("docs"), oauth_client=data.get("oauth_client"), @@ -262,16 +272,23 @@ def client_secret_secret(self) -> str: def accepts_token(self, token: str) -> bool: """Whether the token may be treated as this connection's. - A declared prefix decides on its own: Google issues ya29.c. - access tokens that carry two dots, and a shape heuristic must not - overrule a prefix the catalog vouches for. Without prefixes, a JWT - is judged by the issuer its claims name - Google ID tokens and - service account tokens always name Google's - and a JWT that names - none says as little as an opaque token, so it is judged like one. - One that cannot be read is refused rather than guessed about. + A declared token format decides first and alone: the connection has + promised what its tokens look like, and anything else is refused + however little else claims it. A declared prefix decides next: Google + issues ya29.c. access tokens that carry two dots, and a shape + heuristic must not overrule a prefix the catalog vouches for. With + neither, a JWT is judged by the issuer its claims name - Google ID + tokens and service account tokens always name Google's - and a JWT + that names none says as little as an opaque token, so it is judged + like one. One that cannot be read is refused rather than guessed about. """ if not token: return False + if self.token_format is not None: + # A resolved declaration outlives the gete that wrote it, and a + # format this one cannot judge is not one it may fall back from: + # elimination would accept what the declaration meant to narrow. + return self.token_format == JWT_FORMAT and self._issued_here_jwt(token) if self.token_prefixes: return token.startswith(self.token_prefixes) if token.startswith(GOOGLE_ACCESS_TOKEN_PREFIX): @@ -287,29 +304,54 @@ def accepts_token(self, token: str) -> bool: # it is not some other service's. return not any(token.startswith(prefix) for prefix in self.foreign_prefixes) - def issued_here(self, issuer: str) -> bool: - """Whether the issuer names this connection's own service. + def _issued_here_jwt(self, token: str) -> bool: + """Whether the token is a JWT this connection's own service issued. - The hosts the token may travel to and the OAuth endpoints all count - as its service: the issuer of a service's tokens is its authorization - server, which may live beside the API rather than on it. Matched by - host, exactly, whether the issuer is written as a URL or bare. + What a connection declaring the format holds every token to. A token + of any other shape, one whose claims cannot be read, and one that + names no issuer are all refused alike: none of them is what was + promised, and the promise is the only reason the connection may sit + beside another that accepts tokens by elimination. """ - try: - # The issuer arrives inside an unvetted token; one that urlsplit - # cannot read names no host. - named = urlsplit(issuer if "://" in issuer else f"//{issuer}").hostname - except ValueError: + if not looks_like_jwt(token): return False - if named is None: + claims = jwt_claims(token) + if claims is None: return False + issuer = claims.get("iss") + return isinstance(issuer, str) and self.issued_here(issuer) + + @property + def issuer_hosts(self) -> frozenset[str]: + """Hosts a token of this connection's may name as its issuer. + + The hosts the token may travel to and the OAuth endpoints all count + as its service: the issuer of a service's tokens is its authorization + server, which may live beside the API rather than on it. A URL still + written around {base_url} contributes nothing, because it is not a + name until the installation sets the root. + """ own = {entry.partition("/")[0] for entry in self.hosts} own.update( host for url in (self.oauth.authorization_url, self.oauth.token_url) if (host := urlsplit(url).hostname) is not None ) - return named in own + return frozenset(own) + + def issued_here(self, issuer: str) -> bool: + """Whether the issuer names this connection's own service. + + Matched by host, exactly, whether the issuer is written as a URL or + bare. + """ + try: + # The issuer arrives inside an unvetted token; one that urlsplit + # cannot read names no host. + named = urlsplit(issuer if "://" in issuer else f"//{issuer}").hostname + except ValueError: + return False + return named is not None and named in self.issuer_hosts def allows(self, url: str) -> bool: """Whether a token may be attached to a request for this URL. diff --git a/src/gete/connection/runtime.py b/src/gete/connection/runtime.py index 7640b33..f89722a 100644 --- a/src/gete/connection/runtime.py +++ b/src/gete/connection/runtime.py @@ -11,7 +11,13 @@ from functools import cache from typing import Any -from gete.connection.registry import Connection, Registry, jwt_claims, looks_like_jwt +from gete.connection.registry import ( + JWT_FORMAT, + Connection, + Registry, + jwt_claims, + looks_like_jwt, +) from gete.request_context import current_tool_call logger = logging.getLogger(__name__) @@ -88,24 +94,58 @@ def describe_state(state: Any) -> list[dict[str, Any]]: ] +NO_DECLARED_SHAPE = "matches none of the declared shapes" + + def describe_token(connection: Connection, token: str) -> str: """Which declared shape the token has, without repeating any of it. A connection without prefixes declares no shapes, so a refused JWT is described by what refused it - claims that cannot be read, or an issuer that does not name this service - or the shape message would leave the - refusal looking like a mystery. + refusal looking like a mystery. A connection that declares a token format + is refusing against a promise, and the promise is named: an operator who + turned the provider's expiring-token setting off has every token refused + at once, and nothing else in the log would say why. Read in the order + accepts_token decides in, so the message names the rule that actually + refused the token. """ + if connection.token_format is not None: + return _unmet_format(connection.token_format, token) for prefix in connection.token_prefixes: if token.startswith(prefix): return f"starts with {prefix}" - if not connection.token_prefixes and looks_like_jwt(token): - claims = jwt_claims(token) - if claims is None: - return "a JWT whose claims cannot be read" - if "iss" in claims: - return "a JWT whose issuer does not name this service" - return "matches none of the declared shapes" + if connection.token_prefixes: + return NO_DECLARED_SHAPE + return _refused_jwt(token) or NO_DECLARED_SHAPE + + +def _unmet_format(token_format: str, token: str) -> str: + """Which part of the declared format the token missed. + + A format this gete cannot judge refuses every token alike, so it is the + format that has to be named and not the token: an operator told the + issuer was wrong would go looking at a token that may be exactly right, + when what refused it is a declaration newer than the gete reading it. + """ + if token_format != JWT_FORMAT: + return f"declared as {token_format}, which this gete cannot judge" + if not looks_like_jwt(token): + return f"not the {token_format} this connection declares" + return _refused_jwt(token) or "a JWT that names no issuer" + + +def _refused_jwt(token: str) -> str | None: + """Why a JWT-shaped token is not taken for this service's, or None when + nothing about the JWT itself says so.""" + if not looks_like_jwt(token): + return None + claims = jwt_claims(token) + if claims is None: + return "a JWT whose claims cannot be read" + if "iss" in claims: + return "a JWT whose issuer does not name this service" + return None def usable_token(connection: Connection, key: str, state: Any) -> str | None: diff --git a/src/gete/connections_listing.py b/src/gete/connections_listing.py index 2142e47..fcc9f9b 100644 --- a/src/gete/connections_listing.py +++ b/src/gete/connections_listing.py @@ -11,6 +11,20 @@ BY_ELIMINATION = "(none: by elimination)" +def token_shapes(connection: Connection) -> str: + """What the connection accepts a token by, for the person preparing it. + + Which of the three it is decides whether an agent may hold the connection + beside another that announces itself no other way, so a declared format + must not read as the elimination it replaces. + """ + if connection.token_prefixes: + return ", ".join(connection.token_prefixes) + if connection.token_format is not None: + return f"(none: {connection.token_format} issued by this service)" + return BY_ELIMINATION + + def connections_table(registry: Registry) -> list[dict[str, Any]]: """One row per connection, retired ones included, in id order.""" catalog = set(catalog_connections()) @@ -22,8 +36,7 @@ def connections_table(registry: Registry) -> list[dict[str, Any]]: "display_name": connection.display_name, "status": "retired" if connection.retired else "available", "hosts": ", ".join(sorted(connection.hosts)), - "token_prefixes": ", ".join(connection.token_prefixes) - or BY_ELIMINATION, + "token_prefixes": token_shapes(connection), "verified": connection.verified.get("gemini_enterprise", NOT_VERIFIED), "source": "catalog" if connection.id in catalog else "gete.yaml", "retired": connection.retired or "", @@ -59,7 +72,7 @@ def format_connection(connection: Connection) -> str: "redirect hosts", [", ".join(sorted(connection.redirect_hosts)) or NONE_DECLARED], ), - ("token prefixes", [", ".join(connection.token_prefixes) or BY_ELIMINATION]), + ("token prefixes", [token_shapes(connection)]), ("mcp url", [connection.mcp_url or NONE_DECLARED]), ("authorization", [oauth.authorization_url]), ("token url", [oauth.token_url]), diff --git a/src/gete/schema/connection.json b/src/gete/schema/connection.json index e20d3f6..11d34a9 100644 --- a/src/gete/schema/connection.json +++ b/src/gete/schema/connection.json @@ -50,7 +50,7 @@ } }, "token_prefixes": { - "description": "Prefixes of issued tokens. Empty means the service does not announce itself and tokens are accepted by elimination; an agent may declare only one such connection.", + "description": "Prefixes of issued tokens. Empty means the service does not announce itself and tokens are accepted by elimination, unless tokens.format says what they look like; an agent may declare only one connection that announces itself neither way.", "type": "array", "uniqueItems": true, "items": { @@ -58,6 +58,22 @@ "minLength": 1 } }, + "tokens": { + "description": "What issued tokens look like, for a service whose tokens announce themselves by their shape rather than by a prefix. Declaring it is a promise held at runtime: a token that does not have the declared shape is refused. Whether a provider issues such tokens can be a per-installation setting, so this belongs in gete.yaml rather than in a catalog entry that would promise it everywhere.", + "type": "object", + "additionalProperties": false, + "required": [ + "format" + ], + "properties": { + "format": { + "description": "jwt: the tokens are JWTs whose iss claim names the connection's own service. Nothing else is a format that says whose token it is.", + "enum": [ + "jwt" + ] + } + } + }, "oauth": { "type": "object", "additionalProperties": false, diff --git a/tests/conformance/test_catalog.py b/tests/conformance/test_catalog.py index f27bea9..f57c57b 100644 --- a/tests/conformance/test_catalog.py +++ b/tests/conformance/test_catalog.py @@ -229,3 +229,33 @@ def test_zendesk_says_what_a_person_has_to_do_before_authorizing() -> None: setup = CATALOG["zendesk"]["setup"] assert "OAuth" in setup assert "redirect URI" in setup + + +def test_zendesk_promises_no_token_format_for_every_installation() -> None: + """Zendesk issues JWTs of its own only with token expiry turned on, which + is the installation's setting; the catalog must not promise it.""" + assert "tokens" not in CATALOG["zendesk"] + assert "tokens.format" in CATALOG["zendesk"]["setup"] + + +def test_zendesk_records_no_accepted_shape_because_it_has_two() -> None: + """An accepted example would fix one of the shapes, and contradict the + installation that has the other. What is refused is refused either way.""" + assert "accepts" not in CATALOG["zendesk"]["examples"] + assert CATALOG["zendesk"]["examples"]["rejects"] + + +def test_an_installation_may_declare_that_zendesk_issues_jwts() -> None: + """With expiry on the tokens name the subdomain, and Zendesk stops being + the one connection an agent may hold by elimination.""" + registry = Registry.from_catalog( + { + "zendesk": { + "base_url": "https://acme.zendesk.com", + "tokens": {"format": "jwt"}, + } + } + ) + zendesk = registry.get("zendesk") + assert connection_problems(zendesk, registry) == [] + assert elimination_problems(["zendesk", "freee"], registry) == [] diff --git a/tests/test_caller_token.py b/tests/test_caller_token.py index 90124d1..b9073d0 100644 --- a/tests/test_caller_token.py +++ b/tests/test_caller_token.py @@ -5,7 +5,7 @@ import pytest -from gete.connection import Registry +from gete.connection import Connection, Registry from gete.connection.runtime import ( authorization_id, caller_token, @@ -27,6 +27,48 @@ ) +# Claims: {"exp": 0}. A JWT that names no issuer says as little about its +# origin as an opaque token. +JWT_WITHOUT_ISSUER = "eyJhbGciOiJFZERTQSJ9.eyJleHAiOjB9.sig" +# A connection whose tokens are JWTs of its own making, as an installation +# declares one whose provider issues them. +DECLARED = Registry( + [ + Connection.from_mapping( + { + "id": "declared", + "display_name": "Declared", + "hosts": ["api.example.com"], + "tokens": {"format": "jwt"}, + "oauth": { + "authorization_url": "https://api.example.com/authorize", + "token_url": "https://api.example.com/token", + "scopes": {}, + }, + } + ) + ] +).get("declared") +# Claims: {"iss": "https://api.example.com"}: a token DECLARED's own service +# issued, and the only kind it takes. +OWN_ISSUER_JWT = "eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJodHRwczovL2FwaS5leGFtcGxlLmNvbSJ9.sig" +# A format no gete of this version knows, as a resolved declaration written +# by a later one carries it. +UNJUDGEABLE = Connection.from_mapping( + { + "id": "later", + "display_name": "Later", + "hosts": ["api.example.com"], + "tokens": {"format": "paseto"}, + "oauth": { + "authorization_url": "https://api.example.com/authorize", + "token_url": "https://api.example.com/token", + "scopes": {}, + }, + } +) + + def teardown_function() -> None: clear_tool_call() @@ -121,6 +163,43 @@ def test_describe_token_says_why_a_jwt_was_refused_without_repeating_it() -> Non ) +def test_describe_token_says_a_declared_format_was_not_met() -> None: + """The connection promised JWTs of its own making. An operator who turned + the provider's expiring-token setting off sees every token refused, and + the message has to say which promise the token missed.""" + assert ( + describe_token(DECLARED, FREEE_TOKEN) == "not the jwt this connection declares" + ) + assert describe_token(DECLARED, JWT_WITHOUT_ISSUER) == "a JWT that names no issuer" + assert ( + describe_token(DECLARED, GOOGLE_ISSUED_JWT) + == "a JWT whose issuer does not name this service" + ) + assert FREEE_TOKEN not in describe_token(DECLARED, FREEE_TOKEN) + + +def test_describe_token_names_a_format_this_gete_cannot_judge() -> None: + """Nothing is accepted under a format this gete cannot judge, so the + reason is the format and not the token: a JWT issued by this very service + is refused along with everything else, and saying its issuer is wrong + would send the operator after the wrong thing.""" + unjudgeable = "declared as paseto, which this gete cannot judge" + assert not UNJUDGEABLE.accepts_token(OWN_ISSUER_JWT) + assert describe_token(UNJUDGEABLE, OWN_ISSUER_JWT) == unjudgeable + assert describe_token(UNJUDGEABLE, FREEE_TOKEN) == unjudgeable + assert describe_token(UNJUDGEABLE, GOOGLE_ISSUED_JWT) == unjudgeable + + +def test_usable_token_refuses_what_a_declared_format_rules_out( + caplog: pytest.LogCaptureFixture, +) -> None: + """Elimination would have taken the opaque token; the declaration is what + keeps it from travelling.""" + with caplog.at_level(logging.WARNING): + assert usable_token(DECLARED, "declared", {"declared": FREEE_TOKEN}) is None + assert FREEE_TOKEN not in caplog.text + + def test_caller_token_takes_the_state_from_the_current_call() -> None: context = SimpleNamespace(state={"mail-triage-github": GITHUB_TOKEN}) set_tool_call(ToolCall(context, {"github": "mail-triage-github"}, registry=CATALOG)) diff --git a/tests/test_connection_registry.py b/tests/test_connection_registry.py index 0209d1c..39f93b6 100644 --- a/tests/test_connection_registry.py +++ b/tests/test_connection_registry.py @@ -147,6 +147,79 @@ def test_prefixless_connection_alone_still_refuses_google_access_tokens() -> Non assert alone.accepts_token("a1b2c3d4e5f60718293a4b5c6d7e8f90") +JWT_TOKENS: dict[str, Any] = {"tokens": {"format": "jwt"}} + + +def test_a_declared_jwt_format_accepts_a_token_the_service_issued() -> None: + """The declaration says the tokens are JWTs of this service's own making, + and the issuer is what says so.""" + entry = Registry([connection(**JWT_TOKENS)]).get("example") + assert entry.accepts_token(jwt_with({"iss": "https://auth.example.com"})) + assert entry.accepts_token(jwt_with({"iss": "api.example.com"})) + + +def test_a_declared_jwt_format_refuses_a_token_that_is_not_a_jwt() -> None: + """Without the declaration elimination would take it; with it, the + connection promises a shape and holds itself to it.""" + declared = Registry([connection(**JWT_TOKENS)]).get("example") + anonymous = Registry([connection()]).get("example") + opaque = "a1b2c3d4e5f60718293a4b5c6d7e8f90" + assert anonymous.accepts_token(opaque) + assert not declared.accepts_token(opaque) + + +@pytest.mark.parametrize( + "claims", + [ + {"exp": 0}, # a JWT that names no issuer says nothing about its origin + {"iss": "https://idp.example.org"}, + {"iss": "https://accounts.google.com"}, + {"iss": None}, + ], +) +def test_a_declared_jwt_format_refuses_a_jwt_from_anywhere_else(claims: Any) -> None: + entry = Registry([connection(**JWT_TOKENS)]).get("example") + assert not entry.accepts_token(jwt_with(claims)) + + +@pytest.mark.parametrize( + "token", + ["", "a.b.c", "eyJhbGciOiJSUzI1NiJ9.x.sig", "ya29.a0AfH6SMB", "gho_16C7e42F29"], +) +def test_a_declared_jwt_format_refuses_what_it_cannot_read(token: str) -> None: + entry = Registry([connection(**JWT_TOKENS)]).get("example") + assert not entry.accepts_token(token) + + +def test_a_declared_jwt_format_is_judged_before_any_foreign_prefix() -> None: + """Elimination against the others' prefixes is what the declaration replaces.""" + declared = connection(**JWT_TOKENS) + other = connection(id="other", token_prefixes=["ex_"]) + entry = Registry([declared, other]).get("example") + assert entry.foreign_prefixes == ("ex_",) + assert entry.accepts_token(jwt_with({"iss": "https://api.example.com"})) + assert not entry.accepts_token("something_no_prefix_matches") + + +def test_a_token_format_this_gete_cannot_judge_accepts_nothing() -> None: + """A resolved declaration outlives the gete that wrote it. The schema + refuses an unknown format where it is written; where it is only read, + falling back to elimination would widen what the declaration narrowed.""" + entry = Registry([connection(tokens={"format": "paseto"})]).get("example") + assert not entry.accepts_token("a1b2c3d4e5f60718293a4b5c6d7e8f90") + assert not entry.accepts_token(jwt_with({"iss": "https://api.example.com"})) + + +def test_a_declared_jwt_format_accepts_the_installation_root_as_the_issuer() -> None: + """The root is where the service lives for a rooted connection, and its + tokens are issued there.""" + entry = Connection.from_mapping( + {**ROOTED, **JWT_TOKENS, "base_url": "https://acme.example.com"} + ) + assert entry.accepts_token(jwt_with({"iss": "https://acme.example.com"})) + assert not entry.accepts_token(jwt_with({"iss": "https://other.example.com"})) + + @pytest.mark.parametrize( ("url", "allowed"), [ @@ -448,6 +521,82 @@ def test_connections_that_announce_themselves_never_collide(catalog: Registry) - assert elimination_problems(["freee", "github", "google"], catalog) == [] +def test_a_declared_token_format_is_not_an_acceptance_by_elimination() -> None: + """It takes only tokens its own service issued, so the connection beside + it keeps every token that announces itself no other way.""" + zendesk = connection(id="zendesk", token_prefixes=[], **JWT_TOKENS) + internal = connection( + id="internal", + token_prefixes=[], + hosts=["api.internal.example.com"], + oauth={ + "authorization_url": "https://auth.internal.example.com/authorize", + "token_url": "https://auth.internal.example.com/token", + "scopes": {"read": "Read internal data"}, + }, + ) + registry = Registry([zendesk, internal]) + assert elimination_problems(["zendesk", "internal"], registry) == [] + + +def test_a_shared_issuer_confuses_a_declared_format_and_an_anonymous_one() -> None: + """The anonymous connection takes a JWT naming its own authorization + server too, so one issuer serving both is the same confusion as two + declaring connections sharing one - the declaration says nothing the + other's token does not say as well.""" + declared = connection(id="declared", **JWT_TOKENS) + anonymous = connection(id="anonymous", hosts=["api.other.example.com"]) + registry = Registry([declared, anonymous]) + token = jwt_with({"iss": "https://auth.example.com"}) + assert registry.get("declared").accepts_token(token) + assert registry.get("anonymous").accepts_token(token) + found = elimination_problems(["declared", "anonymous"], registry) + assert len(found) == 1, found + assert "auth.example.com" in found[0], found + + +def test_an_anonymous_pair_sharing_an_issuer_is_reported_once() -> None: + """Neither can be told from the other by anything at all; naming the + issuer they share on top of that would say it twice.""" + one = connection(id="one") + two = connection(id="two", hosts=["api.two.example.com"]) + found = elimination_problems(["one", "two"], Registry([one, two])) + assert len(found) == 1, found + + +def test_two_declared_token_formats_from_different_services_never_collide() -> None: + """Each takes only what its own issuer named; neither reaches the other's.""" + one = connection(id="one", **JWT_TOKENS) + two = connection( + id="two", + hosts=["api.two.example.com"], + oauth={ + "authorization_url": "https://auth.two.example.com/authorize", + "token_url": "https://auth.two.example.com/token", + "scopes": {"read": "Read data"}, + }, + **JWT_TOKENS, + ) + assert elimination_problems(["one", "two"], Registry([one, two])) == [] + + +def test_two_declared_token_formats_issued_by_one_host_are_reported() -> None: + """Both would accept a JWT that names the shared issuer, so a token from + either authorization passes as the other's.""" + api = connection(id="api", **JWT_TOKENS) + mcp = connection(id="mcp", hosts=["mcp.example.com"], **JWT_TOKENS) + found = elimination_problems(["api", "mcp"], Registry([api, mcp])) + assert [p for p in found if "api" in p and "mcp" in p] == found + assert len(found) == 1 + + +def test_prefixes_next_to_a_declared_token_format_are_reported() -> None: + """The format decides on its own, so the prefixes would never be read.""" + entry = connection(token_prefixes=["ex_"], **JWT_TOKENS) + problems = connection_problems(entry, Registry([entry])) + assert any("tokens" in problem for problem in problems), problems + + def test_a_connection_that_accepts_google_issued_jwts_is_reported() -> None: """Naming Google's authorization server as one's own host would let ID tokens through the issuer match; the checks must catch the declaration.""" diff --git a/tests/test_connections_command.py b/tests/test_connections_command.py index c0e66e7..636bfca 100644 --- a/tests/test_connections_command.py +++ b/tests/test_connections_command.py @@ -166,3 +166,36 @@ def test_a_retired_connection_reads_retired_with_the_reason_alongside() -> None: assert result.exit_code == 0, result.output assert "status" in result.output and "retired" in result.output assert "native connector" in result.output + + +def prefixless(**patch: Any) -> Connection: + return Connection.from_mapping( + { + "id": "example", + "display_name": "Example", + "hosts": ["api.example.com"], + "token_prefixes": [], + "oauth": { + "authorization_url": "https://auth.example.com/authorize", + "token_url": "https://auth.example.com/token", + "scopes": {"read": "Read data"}, + }, + **patch, + } + ) + + +def test_a_connection_without_prefixes_reads_as_accepted_by_elimination() -> None: + rows = connections_table(Registry([prefixless()])) + assert rows[0]["token_prefixes"] == "(none: by elimination)" + assert "(none: by elimination)" in format_connection(prefixless()) + + +def test_a_declared_token_format_reads_as_the_format_it_declares() -> None: + """It is not accepted by elimination, and the person preparing the + connection has to see which of the two it is.""" + entry = prefixless(tokens={"format": "jwt"}) + rows = connections_table(Registry([entry])) + assert "jwt" in rows[0]["token_prefixes"] + assert "elimination" not in rows[0]["token_prefixes"] + assert "jwt" in format_connection(entry) diff --git a/tests/test_resolve.py b/tests/test_resolve.py index d116263..521afd5 100644 --- a/tests/test_resolve.py +++ b/tests/test_resolve.py @@ -1,14 +1,27 @@ """The resolved declaration: one file that carries everything the runtime needs.""" +import base64 +import json from pathlib import Path import yaml from conftest import ProjectBuilder import gete +from gete.connection import Registry from gete.declaration import load_project, load_resolved, resolve +def jwt_issued_by(issuer: str) -> str: + """A JWT-shaped token naming this issuer. The signature is never checked.""" + payload = ( + base64.urlsafe_b64encode(json.dumps({"iss": issuer}).encode()) + .rstrip(b"=") + .decode() + ) + return f"eyJhbGciOiJSUzI1NiJ9.{payload}.sig" + + def write_project_with_policy(project: ProjectBuilder) -> None: project.write_policies( "finance", @@ -67,6 +80,32 @@ def test_resolved_connections_carry_the_overrides_and_every_known_prefix( assert connections["slack"]["retired"] +def test_a_declared_token_format_travels_to_the_runtime( + project: ProjectBuilder, +) -> None: + """It is the deployment that has to refuse a token of any other shape; + validate only let the connection be held beside an anonymous one.""" + project.write_project( + { + "version": 1, + "project": "example-project", + "location": "us-central1", + "connections": { + "zendesk": { + "base_url": "https://acme.zendesk.com", + "tokens": {"format": "jwt"}, + } + }, + } + ) + project.write_agent("mail-triage", {"connections": ["zendesk"]}) + loaded = load_project(project.root / "gete.yaml") + connections = resolve(loaded, loaded.agents[0])["resolved"]["connections"] + zendesk = Registry.from_documents(connections).get("zendesk") + assert zendesk.accepts_token(jwt_issued_by("https://acme.zendesk.com")) + assert not zendesk.accepts_token("a1b2c3d4e5f60718293a4b5c6d7e8f90") + + def test_resolved_file_reads_back_without_gete_yaml( project: ProjectBuilder, tmp_path: Path ) -> None: diff --git a/tests/test_schema.py b/tests/test_schema.py index c10654c..3ef5bff 100644 --- a/tests/test_schema.py +++ b/tests/test_schema.py @@ -592,3 +592,27 @@ def test_connection_may_ask_for_pkce() -> None: {**CONNECTION, "oauth": {**CONNECTION["oauth"], "pkce": "yes"}}, source="c.yaml", ) + + +def test_connection_may_declare_that_its_tokens_are_jwts() -> None: + """A service whose access tokens are JWTs issued by its own host is not + anonymous, however little its prefixes say.""" + validate_document( + "connection", {**CONNECTION, "tokens": {"format": "jwt"}}, source="c.yaml" + ) + + +@pytest.mark.parametrize( + "tokens", + [ + {"format": "opaque"}, # the only format that promises anything is jwt + {"format": "JWT"}, + {}, # a tokens block that declares nothing + {"format": "jwt", "issuer": "https://auth.example.com"}, + ], +) +def test_connection_token_format_is_one_declared_shape(tokens: dict[str, Any]) -> None: + with pytest.raises(DeclarationError, match="tokens"): + validate_document( + "connection", {**CONNECTION, "tokens": tokens}, source="c.yaml" + ) diff --git a/tests/test_validate.py b/tests/test_validate.py index 2bb0556..80e3f3b 100644 --- a/tests/test_validate.py +++ b/tests/test_validate.py @@ -372,6 +372,78 @@ def test_two_prefixless_connections_on_one_agent_are_reported( ), found +def test_a_declared_token_format_may_be_held_beside_a_prefixless_connection( + project: ProjectBuilder, +) -> None: + """An agent that reads one service and writes to another needs both, and + a service whose tokens name their issuer is not the anonymous one.""" + project.write_project( + { + "version": 1, + "project": "example-project", + "location": "us-central1", + "connections": { + "internal-api": INTERNAL_API, + "zendesk": { + "base_url": "https://acme.zendesk.com", + "tokens": {"format": "jwt"}, + }, + }, + } + ) + project.write_agent("mail-triage", {"connections": ["zendesk", "internal-api"]}) + assert problems(project) == [] + + +def test_a_declared_format_beside_a_connection_of_the_same_issuer_is_refused( + project: ProjectBuilder, +) -> None: + """One authorization server standing in front of both services puts its + host in either connection's tokens, and the declaration then says nothing + the other's token does not say as well.""" + project.write_project( + { + "version": 1, + "project": "example-project", + "location": "us-central1", + "connections": { + "internal-api": INTERNAL_API, + "internal-billing": { + "display_name": "Internal Billing", + "hosts": ["billing.internal.example.com"], + "tokens": {"format": "jwt"}, + "oauth": INTERNAL_API["oauth"], + }, + }, + } + ) + project.write_agent( + "mail-triage", {"connections": ["internal-api", "internal-billing"]} + ) + found = problems(project) + assert any("auth.internal.example.com" in problem for problem in found), found + + +def test_the_catalog_alone_still_leaves_zendesk_accepted_by_elimination( + project: ProjectBuilder, +) -> None: + """Zendesk issues JWTs only with token expiry turned on, which is the + installation's setting; the catalog cannot promise it for everyone.""" + project.write_project( + { + "version": 1, + "project": "example-project", + "location": "us-central1", + "connections": { + "internal-api": INTERNAL_API, + "zendesk": {"base_url": "https://acme.zendesk.com"}, + }, + } + ) + project.write_agent("mail-triage", {"connections": ["zendesk", "internal-api"]}) + assert any("elimination" in p for p in problems(project)) + + def test_prefixless_connections_on_separate_agents_are_accepted( project: ProjectBuilder, ) -> None: