diff --git a/docs/cli/mcp.md b/docs/cli/mcp.md new file mode 100644 index 0000000..b9beec4 --- /dev/null +++ b/docs/cli/mcp.md @@ -0,0 +1,63 @@ +--- +title: stardust mcp +description: Serve a built site's docs to AI clients over the Model Context Protocol (stdio). +--- + +# stardust mcp + +Serve a built Stardust site to AI clients (Claude Desktop, Cursor, and other [MCP](https://modelcontextprotocol.io/) clients) over the stdio transport. See [MCP Server](/features/mcp) for the full guide. + +## Usage + +```bash +stardust mcp [dir] [options] +``` + +`[dir]` is the built site directory. When omitted, it resolves from `build.outDir` in `stardust.yaml`, falling back to `dist`. + +The site must be built first — the server reads the `llms.json` manifest and per-page `.md` files that `stardust build` emits (with `build.llms` enabled, the default). + +## Examples + +```bash +# Build, then serve dist/ over MCP +stardust build +stardust mcp dist + +# Inside a project, resolve the output dir from stardust.yaml +stardust mcp +``` + +## What it exposes + +- **`list_pages`** tool — the table of contents (path, title, description), with an optional path-prefix filter. +- **`search_docs`** tool — full-text search across the documentation. +- **`read_page`** tool — the full markdown of a page by its path. +- **Resources** — one `text/markdown` resource per page. + +## Serve over HTTP + +By default the server speaks the **stdio** transport (clients launch it as a subprocess). Pass `--http` to serve a live **Streamable HTTP** `/mcp` endpoint instead — for self-hosters who run the binary as a service: + +```bash +stardust mcp dist --http --port 8080 +# → Endpoint: http://localhost:8080/mcp +``` + +It binds to `localhost` and validates the `Origin` header (DNS-rebinding guard); loopback origins and non-browser clients are always allowed, other browser origins need `--allow-origin`. See [MCP Server](/features/mcp) for details. + +## Options + +| Option | Default | Description | +|--------|---------|-------------| +| `[dir]` | `build.outDir` (else `dist`) | Built site directory to serve | +| `-c`, `--config` | `stardust.yaml` | Config file, used only to resolve the output dir when `[dir]` is omitted | +| `--http` | off | Serve Streamable HTTP (a live `/mcp` endpoint) instead of stdio | +| `-p`, `--port` | `8080` | Port for `--http` | +| `--host` | `localhost` | Bind host for `--http` | +| `--allow-origin` | — | Allowed browser `Origin` for `--http` (repeatable; `*` for any) | + +## Notes + +- In stdio mode, communication is JSON-RPC on **stdin/stdout** and all logs go to **stderr**, keeping stdout a clean protocol channel. In `--http` mode, requests come over `POST /mcp`. +- The server is **read-only** and holds no state between runs. diff --git a/docs/features/llm-output.md b/docs/features/llm-output.md index 3a69fd6..e43cefc 100644 --- a/docs/features/llm-output.md +++ b/docs/features/llm-output.md @@ -70,6 +70,34 @@ https://example.com/guide.md → its markdown source Readers get the same thing through the **Copy page as Markdown** button at the top of each page — one click to paste a page into an AI chat. +## Machine-Readable Manifest (llms.json) + +Alongside `llms.txt`, Stardust writes `llms.json` — a structured index of every +page with its `path`, `title`, `description`, absolute `url`, and the `md` path +to its raw markdown: + +```json +{ + "name": "My Project", + "description": "…", + "url": "https://example.com", + "generator": "stardust", + "pages": [ + { "path": "/guide", "title": "Guide", "description": "…", + "url": "https://example.com/guide", "md": "/guide.md" } + ] +} +``` + +Where `llms.txt` is written for a model to read, `llms.json` is written for a +program to parse: a remote agent can fetch it, enumerate pages, and pull exactly +the `.md` files it needs — consuming the whole site with no server to run. It is +also what powers the [MCP Server](/features/mcp): `stardust mcp` serves a built +site to Claude, Cursor, and other AI clients straight from this manifest. + +Pages with `llm: false` are excluded here too. The manifest is written whenever +`build.llms` is enabled (the default). + ## Use Cases ### AI Chatbots diff --git a/docs/features/mcp.md b/docs/features/mcp.md new file mode 100644 index 0000000..0b3cb48 --- /dev/null +++ b/docs/features/mcp.md @@ -0,0 +1,89 @@ +--- +title: MCP Server +description: Serve any built site's docs to Claude, Cursor, and other AI clients over the Model Context Protocol — locally, air-gapped, no service to run. +--- + +# MCP Server + +Every hosted docs platform now auto-generates an [MCP](https://modelcontextprotocol.io/) server so AI assistants can search and read the docs. Stardust brings that to **static, self-hosted sites**: `stardust mcp` turns the binary you already have into an MCP server over any built site — no service to run, works air-gapped. + +A Claude or Cursor user points their client at `stardust mcp ` and gets: + +- **`list_pages`** — the table of contents (path, title, description), optionally filtered by a path prefix. +- **`search_docs`** — full-text search across your documentation, returning matching pages with snippets. +- **`read_page`** — the full markdown of any page by its path. +- **Resources** — every page exposed as a `text/markdown` resource, so clients can browse and attach pages directly. + +The server reads a **built** site — its `llms.json` manifest and the per-page `.md` files that `stardust build` already emits. It never re-parses your markdown and needs no `stardust.yaml` at serve time, so it can serve any built site, even one you didn't build. + +## Usage + +Build your site, then serve it: + +```bash +stardust build # emits dist/, including llms.json + per-page .md +stardust mcp dist # serves dist/ over MCP on stdio +``` + +With no directory argument, `stardust mcp` resolves the output directory from `stardust.yaml` (falling back to `dist`), so inside a project you can just run: + +```bash +stardust mcp +``` + +The server speaks the [stdio transport](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports): it reads JSON-RPC requests on stdin and writes responses on stdout. All logging goes to stderr, so stdout stays a clean protocol channel. + +## Connect a client + +MCP clients launch the server as a subprocess. Point yours at the `stardust` binary with `mcp` and your built directory. + +**Claude Desktop / Cursor** (`claude_desktop_config.json` or the client's MCP settings): + +```json +{ + "mcpServers": { + "my-docs": { + "command": "stardust", + "args": ["mcp", "/absolute/path/to/dist"] + } + } +} +``` + +Restart the client and your docs appear as a connected server — search and read your guides without leaving the chat. + +## Serve over HTTP (self-hosting) + +Hosted docs platforms expose a live `https://yoursite/mcp` endpoint because they run a server for you. If **you** run a server — a VPS, on-prem box, or air-gapped host — `--http` gives you the same thing: + +```bash +stardust mcp dist --http --port 8080 +# → Endpoint: http://localhost:8080/mcp +``` + +This is the MCP [Streamable HTTP transport](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports): a single `/mcp` endpoint answering JSON-RPC over `POST`. Point any MCP client that accepts a URL at `http://your-host:8080/mcp`. + + +A **pure static host** (GitHub Pages, Netlify, a plain CDN) can't serve `--http` — a live endpoint needs a running process. Those deployments use the static [`llms.json` + `.md` files](/features/llm-output) instead (see below). `--http` is for when you actually run the binary as a service. + + +**Security.** The server binds to `localhost` by default and validates the `Origin` header to block [DNS-rebinding](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports#security-warning) attacks: loopback origins and non-browser clients (which send no `Origin`) are always allowed; other browser origins must be listed with `--allow-origin`. To expose it beyond localhost, set `--host 0.0.0.0` and add each browser origin (`--allow-origin https://app.example.com`, or `--allow-origin '*'` to accept any), ideally behind your own TLS/auth proxy. The server is read-only and stateless. + +## Options + +| Option | Default | Description | +|--------|---------|-------------| +| `[dir]` | `build.outDir` (else `dist`) | The built site directory to serve | +| `-c`, `--config` | `stardust.yaml` | Config file, used only to resolve the output dir when `[dir]` is omitted | +| `--http` | off | Serve Streamable HTTP (a live `/mcp` endpoint) instead of stdio | +| `-p`, `--port` | `8080` | Port for `--http` | +| `--host` | `localhost` | Bind host for `--http` | +| `--allow-origin` | — | Allowed browser `Origin` for `--http` (repeatable; `*` for any) | + +## Consume without a server + +The same build also produces a static, machine-readable manifest — `llms.json` — plus per-page `.md` files and `llms.txt`. Remote agents that can't launch a subprocess can fetch these directly over HTTP, no MCP server required. See [LLM-Friendly Output](/features/llm-output). + + +The MCP server is **read-only** and requires a built site. Run `stardust build` (with `build.llms` enabled, the default) first — the manifest and per-page markdown it emits are what the server serves. + diff --git a/lib/src/cli/cli_runner.dart b/lib/src/cli/cli_runner.dart index ade1945..73f8abd 100644 --- a/lib/src/cli/cli_runner.dart +++ b/lib/src/cli/cli_runner.dart @@ -10,6 +10,7 @@ import 'commands/clean_command.dart'; import 'commands/dartdoc_command.dart'; import 'commands/dev_command.dart'; import 'commands/init_command.dart'; +import 'commands/mcp_command.dart'; import 'commands/new_command.dart'; import 'commands/openapi_command.dart'; import 'commands/serve_command.dart'; @@ -30,6 +31,7 @@ class StardustCliRunner extends CommandRunner { addCommand(CleanCommand()); addCommand(OpenApiCommand()); addCommand(DartdocCommand()); + addCommand(McpCommand()); argParser.addFlag( 'version', diff --git a/lib/src/cli/commands/mcp_command.dart b/lib/src/cli/commands/mcp_command.dart new file mode 100644 index 0000000..4109b7a --- /dev/null +++ b/lib/src/cli/commands/mcp_command.dart @@ -0,0 +1,116 @@ +import 'dart:async'; +import 'dart:convert'; +import 'dart:io'; + +import 'package:args/command_runner.dart'; +import 'package:path/path.dart' as p; +import 'package:shelf/shelf.dart' as shelf; +import 'package:shelf/shelf_io.dart' as shelf_io; + +import '../../config/config_loader.dart'; +import '../../core/file_system.dart'; +import '../../mcp/docs_source.dart'; +import '../../mcp/mcp_http_server.dart'; +import '../../mcp/mcp_server.dart'; +import '../../utils/exceptions.dart'; +import '../../utils/logger.dart'; +import '../../version.dart'; + +/// Serve a built Stardust site's docs to AI clients (Claude Desktop, Cursor) +/// over MCP — stdio by default, or Streamable HTTP with `--http`. +class McpCommand extends Command { + final FileSystem fileSystem; + + @override + final name = 'mcp'; + + @override + final description = 'Serve a built site to AI clients over MCP (stdio, or --http)'; + + @override + String get invocation => 'stardust mcp [dir]'; + + McpCommand({FileSystem? fileSystem}) : fileSystem = fileSystem ?? const LocalFileSystem() { + argParser + ..addOption('config', + abbr: 'c', help: 'Path to stardust.yaml (used only to resolve the output dir)', defaultsTo: 'stardust.yaml') + ..addFlag('http', help: 'Serve over Streamable HTTP (a live /mcp endpoint) instead of stdio', negatable: false) + ..addOption('port', abbr: 'p', help: 'Port for --http', defaultsTo: '8080') + ..addOption('host', help: 'Host to bind for --http', defaultsTo: 'localhost') + ..addMultiOption('allow-origin', + help: 'Allowed browser Origin for --http (repeatable; "*" for any). ' + 'Loopback origins and non-browser clients are always allowed.') + ..addFlag('verbose', abbr: 'v', help: 'Verbose logging (to stderr)', negatable: false); + } + + @override + Future run() async { + final args = argResults; + if (args == null) return 1; + + final http = args['http'] == true; + // In stdio mode, stdout is the protocol channel, so logs go to stderr only. + // In HTTP mode, stdout is free — log there like `stardust serve`. + final logger = http + ? Logger(onLog: stdout.writeln, onError: stderr.writeln) + : Logger(onLog: stderr.writeln, onError: stderr.writeln); + final configPath = args['config'] as String; + + final dir = switch (args.rest) { + [final dir, ...] => dir, + [] when await fileSystem.fileExists(configPath) => + p.normalize((await ConfigLoader.load(configPath, logger: logger)).build.outDir), + [] => 'dist', + }; + + try { + final source = await DocsSource.load(dir, fileSystem: fileSystem, logger: logger); + final server = McpServer(source, name: source.siteName, version: version, logger: logger); + + if (http) { + final port = int.tryParse(args['port'] as String); + if (port == null || port < 0 || port > 65535) { + stderr.writeln('❌ Invalid port: ${args['port']}'); + return 1; + } + return _serveHttp(source, server, logger, + host: args['host'] as String, port: port, allowedOrigins: (args['allow-origin'] as List).toSet()); + } + + logger.log('📡 stardust mcp — serving ${source.pages.length} pages from $dir/ over stdio'); + final lines = stdin.transform(utf8.decoder).transform(const LineSplitter()); + await server.serve(lines, stdout.writeln); + return 0; + } on ContentException catch (e) { + stderr.writeln('❌ ${e.message}'); + return 1; + } + } + + Future _serveHttp( + DocsSource source, + McpServer server, + Logger logger, { + required String host, + required int port, + required Set allowedOrigins, + }) async { + final mcp = McpHttpServer(server, allowedOrigins: allowedOrigins, logger: logger); + final handler = const shelf.Pipeline().addMiddleware(shelf.logRequests()).addHandler(mcp.handle); + final httpServer = await shelf_io.serve(handler, host, port); + + logger.log(''); + logger.log('📡 stardust mcp — serving ${source.pages.length} pages over MCP (Streamable HTTP)'); + logger.log(' ➜ Endpoint: http://$host:$port/${McpHttpServer.endpoint}'); + logger.log(''); + + ProcessSignal.sigint.watch().listen((_) async { + logger.log('\n👋 Shutting down...'); + await httpServer.close(); + exit(0); + }); + + await Completer().future; + return 0; + } +} diff --git a/lib/src/generator/site_generator.dart b/lib/src/generator/site_generator.dart index d1d92bb..92a42a8 100644 --- a/lib/src/generator/site_generator.dart +++ b/lib/src/generator/site_generator.dart @@ -90,6 +90,7 @@ class SiteGenerator { if (config.build.llms.enabled) { await _generateLlms(pagesWithNav); await _generateLlmsFull(pagesWithNav); + await _generateManifest(pagesWithNav); } if (config.seo.ogImage == null && !config.devMode) { @@ -511,6 +512,33 @@ class SiteGenerator { logger.log('🤖 Generated llms-full.txt'); } + /// A machine-readable page index (`llms.json`) — the static manifest remote + /// agents can consume without a server, and the input `stardust mcp` reads to + /// serve a built site. Each `md` points at the verbatim source `_writePageMarkdown` + /// emits, so tools can fetch clean markdown per page. + Future _generateManifest(List pages) async { + final urls = UrlResolver(config); + final manifest = { + 'name': config.name, + if (config.description case final desc?) 'description': desc, + if (config.url case final url?) 'url': url, + 'generator': 'stardust', + 'pages': [ + for (final page in pages.where((page) => page.frontmatter['llm'] != false)) + { + 'path': page.path, + 'title': page.title, + if (page.description case final desc?) 'description': desc, + if (urls.absolute(page.path) case final abs?) 'url': abs, + 'md': page.path == '/' ? '/index.md' : '${page.path}.md', + }, + ], + }; + await fileSystem.writeFile( + p.join(outputDir, 'llms.json'), '${const JsonEncoder.withIndent(' ').convert(manifest)}\n'); + logger.log('🤖 Generated llms.json'); + } + Future _generateRedirects(List pages) async { final hasConfigRedirects = config.build.redirects.isNotEmpty; final hasFrontmatterRedirects = pages.any((p) => p.redirectFrom.isNotEmpty); diff --git a/lib/src/mcp/docs_source.dart b/lib/src/mcp/docs_source.dart new file mode 100644 index 0000000..fb29fab --- /dev/null +++ b/lib/src/mcp/docs_source.dart @@ -0,0 +1,178 @@ +import 'dart:convert'; + +import 'package:path/path.dart' as p; + +import '../core/file_system.dart'; +import '../utils/exceptions.dart'; +import '../utils/logger.dart'; + +/// One documentation page discovered from a built site's `llms.json` manifest. +class DocPage { + final String path; + final String title; + final String? description; + final String? url; + + /// Site-root-relative path to the verbatim markdown, e.g. `/introduction.md`. + final String mdFile; + + const DocPage({ + required this.path, + required this.title, + this.description, + this.url, + required this.mdFile, + }); +} + +/// A search hit: the matched page plus a short snippet of surrounding text. +class DocHit { + final DocPage page; + final String snippet; + + const DocHit(this.page, this.snippet); +} + +/// Reads a **built** Stardust site (its `llms.json` manifest + per-page `.md` +/// files) and answers docs queries — the read-only source `stardust mcp` serves. +/// It never re-parses markdown or needs `stardust.yaml`: everything comes from +/// the build output, so it serves any built site, air-gapped. +class DocsSource { + final String dir; + final FileSystem fileSystem; + final Logger logger; + final String siteName; + final String? siteDescription; + final String? siteUrl; + final List pages; + + final Map _byPath; + final Map _bodies = {}; + + DocsSource._({ + required this.dir, + required this.fileSystem, + required this.logger, + required this.siteName, + this.siteDescription, + this.siteUrl, + required this.pages, + }) : _byPath = {for (final page in pages) page.path: page}; + + /// Loads the manifest at `/llms.json`. Throws [ContentException] with an + /// actionable message when the directory hasn't been built. + static Future load(String dir, {FileSystem? fileSystem, Logger logger = const Logger()}) async { + final fs = fileSystem ?? const LocalFileSystem(); + final manifestPath = p.join(dir, 'llms.json'); + if (!await fs.fileExists(manifestPath)) { + throw ContentException( + 'No llms.json in "$dir" — run `stardust build` (with build.llms enabled) first, or pass the built site directory.', + ); + } + + final decoded = switch (jsonDecode(await fs.readFile(manifestPath))) { + final Map decoded => decoded, + _ => throw ContentException('Malformed llms.json in "$dir".'), + }; + + final pages = []; + if (decoded['pages'] case final List rawPages) { + for (final raw in rawPages) { + if (raw case {'path': final String path, 'title': final String title, 'md': final String md}) { + pages.add(DocPage( + path: path, + title: title, + description: _asString(raw['description']), + url: _asString(raw['url']), + mdFile: md, + )); + } + } + } + + return DocsSource._( + dir: dir, + fileSystem: fs, + logger: logger, + siteName: _asString(decoded['name']) ?? 'Documentation', + siteDescription: _asString(decoded['description']), + siteUrl: _asString(decoded['url']), + pages: pages, + ); + } + + /// The verbatim markdown for [path], or null when no such page exists (or its + /// `.md` file is missing from the build). + Future readPage(String path) async { + if (_byPath[path] case final page?) return _body(page); + return null; + } + + /// Full-text search over titles, descriptions, and page bodies. Deterministic: + /// ranked by weighted term frequency, ties broken by manifest order. + Future> search(String query, {int limit = 10}) async { + final terms = _terms(query); + if (terms.isEmpty) return const []; + + final scored = <({DocPage page, int score, int index, String snippet})>[]; + for (var i = 0; i < pages.length; i++) { + final page = pages[i]; + final body = await _body(page) ?? ''; + final title = page.title.toLowerCase(); + final lowerBody = body.toLowerCase(); + final desc = page.description?.toLowerCase() ?? ''; + + var score = 0; + for (final term in terms) { + score += _count(title, term) * 5; + score += _count(desc, term) * 2; + score += _count(lowerBody, term); + } + if (score > 0) { + scored.add((page: page, score: score, index: i, snippet: _snippet(body, terms, page.description))); + } + } + + scored.sort((a, b) { + final byScore = b.score.compareTo(a.score); + return byScore != 0 ? byScore : a.index.compareTo(b.index); + }); + return [for (final s in scored.take(limit)) DocHit(s.page, s.snippet)]; + } + + Future _body(DocPage page) async { + if (_bodies.containsKey(page.path)) return _bodies[page.path]; + final file = p.join(dir, page.mdFile.startsWith('/') ? page.mdFile.substring(1) : page.mdFile); + final content = await fileSystem.fileExists(file) ? await fileSystem.readFile(file) : null; + _bodies[page.path] = content; + return content; + } + + static List _terms(String query) => + query.toLowerCase().split(RegExp(r'[^a-z0-9]+')).where((t) => t.isNotEmpty).toSet().toList(); + + static int _count(String haystack, String term) => term.isEmpty ? 0 : term.allMatches(haystack).length; + + String _snippet(String body, List terms, String? description) { + final lower = body.toLowerCase(); + var idx = -1; + for (final term in terms) { + final at = lower.indexOf(term); + if (at >= 0 && (idx < 0 || at < idx)) idx = at; + } + + final raw = switch (idx) { + >= 0 => _window(body, idx), + _ => description ?? (body.length > 180 ? '${body.substring(0, 180)}…' : body), + }; + return raw.replaceAll(RegExp(r'\s+'), ' ').trim(); + } + + String _window(String body, int idx) { + final start = idx - 40 < 0 ? 0 : idx - 40; + final end = start + 180 > body.length ? body.length : start + 180; + return '${start > 0 ? '…' : ''}${body.substring(start, end)}${end < body.length ? '…' : ''}'; + } + + static String? _asString(Object? value) => value is String ? value : null; +} diff --git a/lib/src/mcp/mcp_http_server.dart b/lib/src/mcp/mcp_http_server.dart new file mode 100644 index 0000000..84cc256 --- /dev/null +++ b/lib/src/mcp/mcp_http_server.dart @@ -0,0 +1,56 @@ +import 'package:shelf/shelf.dart' as shelf; + +import '../utils/logger.dart'; +import 'mcp_server.dart'; + +/// The MCP [Streamable HTTP transport](https://modelcontextprotocol.io/specification/2025-06-18/basic/transports) +/// over an [McpServer]: a single `/mcp` endpoint answering JSON-RPC POSTs. This +/// is the "run it as a service" path for self-hosters (VPS, on-prem, air-gapped) +/// who want a live endpoint; a pure static host uses the static `llms.json`/`.md` +/// files instead. +/// +/// The server is stateless (no `Mcp-Session-Id`) and has no server-initiated +/// messages, so `GET` returns 405. It validates the `Origin` header to prevent +/// DNS-rebinding attacks: loopback origins and non-browser clients (no `Origin`) +/// are always allowed; other browser origins must be listed in [allowedOrigins]. +class McpHttpServer { + /// The single endpoint path clients POST to. + static const endpoint = 'mcp'; + + final McpServer server; + + /// Extra browser origins to accept (exact match, or `*` for any). + final Set allowedOrigins; + final Logger logger; + + McpHttpServer(this.server, {Set? allowedOrigins, this.logger = const Logger()}) + : allowedOrigins = allowedOrigins ?? const {}; + + Future handle(shelf.Request request) async { + if (request.url.path != endpoint) return shelf.Response.notFound('Not found — POST to /$endpoint.'); + if (!_originAllowed(request.headers['origin'])) return shelf.Response.forbidden('Origin not allowed.'); + + return switch (request.method) { + 'POST' => await _post(request), + // No server-initiated stream, and no sessions to terminate. + 'GET' || 'DELETE' => shelf.Response(405, headers: const {'allow': 'POST'}), + _ => shelf.Response(405, headers: const {'allow': 'POST'}), + }; + } + + Future _post(shelf.Request request) async { + final response = await server.handle(await request.readAsString()); + // A JSON-RPC notification/response (no id) yields no reply → 202 Accepted. + if (response == null) return shelf.Response(202); + return shelf.Response.ok(response, headers: const {'content-type': 'application/json'}); + } + + bool _originAllowed(String? origin) { + if (origin == null) return true; + if (allowedOrigins.contains('*') || allowedOrigins.contains(origin)) return true; + return switch (Uri.tryParse(origin)?.host) { + 'localhost' || '127.0.0.1' || '::1' => true, + _ => false, + }; + } +} diff --git a/lib/src/mcp/mcp_server.dart b/lib/src/mcp/mcp_server.dart new file mode 100644 index 0000000..1cf7064 --- /dev/null +++ b/lib/src/mcp/mcp_server.dart @@ -0,0 +1,221 @@ +import 'dart:convert'; + +import '../utils/logger.dart'; +import 'docs_source.dart'; + +/// A minimal, dependency-free MCP server over the stdio transport: newline- +/// delimited JSON-RPC 2.0 on stdin/stdout, per the MCP spec. It exposes a +/// [DocsSource] through three tools (`list_pages`, `search_docs`, `read_page`) +/// and one resource per page, so a Claude/Cursor user can browse and search a +/// built site's docs. +/// +/// The protocol core is [handle] (pure — one request string in, one response +/// string or null out), kept separate from the [serve] IO loop for testing. +class McpServer { + /// The protocol version advertised when a client doesn't request one. + static const defaultProtocolVersion = '2025-06-18'; + + final DocsSource source; + final String name; + final String version; + final Logger logger; + + McpServer(this.source, {required this.name, this.version = 'dev', this.logger = const Logger()}); + + /// Reads one JSON-RPC message per line and writes each response line. Ends + /// when [input] closes (e.g. the client terminates the subprocess). + Future serve(Stream input, void Function(String) output) async { + await for (final line in input) { + if (line.trim().isEmpty) continue; + if (await handle(line) case final response?) output(response); + } + } + + /// Handles one raw request and returns the response JSON, or null for + /// notifications (JSON-RPC messages without an `id`). + Future handle(String requestJson) async { + final Object? decoded; + try { + decoded = jsonDecode(requestJson); + } catch (_) { + return jsonEncode(_error(null, -32700, 'Parse error')); + } + if (decoded is! Map) return jsonEncode(_error(null, -32600, 'Invalid Request')); + + final id = decoded['id']; + final isNotification = !decoded.containsKey('id'); + final params = switch (decoded['params']) { + final Map params => params, + _ => const {}, + }; + + try { + final result = await _dispatch(decoded['method'], params); + if (identical(result, _noResponse) || isNotification) return null; + return jsonEncode({'jsonrpc': '2.0', 'id': id, 'result': result}); + } on _McpError catch (e) { + return isNotification ? null : jsonEncode(_error(id, e.code, e.message)); + } catch (e) { + logger.error('mcp: ${decoded['method']} failed: $e'); + return isNotification ? null : jsonEncode(_error(id, -32603, 'Internal error')); + } + } + + Future _dispatch(Object? method, Map params) async => switch (method) { + 'initialize' => { + 'protocolVersion': switch (params['protocolVersion']) { + final String requested when requested.isNotEmpty => requested, + _ => defaultProtocolVersion, + }, + 'capabilities': {'tools': {}, 'resources': {}}, + 'serverInfo': {'name': name, 'version': version}, + }, + 'notifications/initialized' || 'notifications/cancelled' => _noResponse, + 'ping' => const {}, + 'tools/list' => {'tools': _toolDefs}, + 'tools/call' => await _callTool(params), + 'resources/list' => {'resources': _resourceDefs()}, + 'resources/read' => await _readResource(params), + _ => throw const _McpError(-32601, 'Method not found'), + }; + + Future _callTool(Map params) async { + final args = switch (params['arguments']) { + final Map args => args, + _ => const {}, + }; + switch (params['name']) { + case 'list_pages': + final prefix = switch (args['prefix']) { + final String p when p.isNotEmpty => p, + _ => '', + }; + final pages = source.pages.where((page) => page.path.startsWith(prefix)).toList(); + if (pages.isEmpty) return _text(prefix.isEmpty ? 'No pages.' : 'No pages under "$prefix".'); + final buffer = StringBuffer(); + for (final page in pages) { + buffer.write('- ${page.path} — ${page.title}'); + if (page.description case final desc?) buffer.write(': $desc'); + buffer.writeln(); + } + return _text(buffer.toString().trimRight()); + case 'search_docs': + if (args['query'] case final String query when query.trim().isNotEmpty) { + final limit = switch (args['limit']) { + final int n when n > 0 => n, + _ => 10, + }; + final hits = await source.search(query, limit: limit); + if (hits.isEmpty) return _text('No results for "$query".'); + final buffer = StringBuffer(); + for (final hit in hits) { + buffer.writeln('## ${hit.page.title} (${hit.page.path})'); + if (hit.snippet.isNotEmpty) buffer.writeln(hit.snippet); + buffer.writeln(); + } + return _text(buffer.toString().trimRight()); + } + return _text('Provide a non-empty "query".', isError: true); + case 'read_page': + if (args['path'] case final String path when path.isNotEmpty) { + return switch (await source.readPage(path)) { + final String md => _text(md), + null => _text('No page at "$path".', isError: true), + }; + } + return _text('Provide a "path", e.g. /introduction.', isError: true); + default: + throw const _McpError(-32602, 'Unknown tool'); + } + } + + Future _readResource(Map params) async { + if (params['uri'] case final String uri) { + final path = uri.startsWith(_uriScheme) ? uri.substring(_uriScheme.length) : uri; + return switch (await source.readPage(path)) { + final String md => { + 'contents': [ + {'uri': uri, 'mimeType': 'text/markdown', 'text': md}, + ], + }, + null => throw _McpError(-32602, 'Resource not found: $uri'), + }; + } + throw const _McpError(-32602, 'Missing uri'); + } + + List> _resourceDefs() => [ + for (final page in source.pages) + { + 'uri': '$_uriScheme${page.path}', + 'name': page.title, + if (page.description case final desc?) 'description': desc, + 'mimeType': 'text/markdown', + }, + ]; + + static Map _text(String text, {bool isError = false}) => { + 'content': [ + {'type': 'text', 'text': text}, + ], + if (isError) 'isError': true, + }; + + static Map _error(Object? id, int code, String message) => { + 'jsonrpc': '2.0', + 'id': id, + 'error': {'code': code, 'message': message} + }; + + static const _uriScheme = 'stardust://'; + + static final Object _noResponse = Object(); + + static const _toolDefs = [ + { + 'name': 'list_pages', + 'description': 'List the documentation pages (path, title, description) — the table of contents. ' + 'Optionally filter by a path prefix. Use read_page to fetch a page by its path.', + 'inputSchema': { + 'type': 'object', + 'properties': { + 'prefix': { + 'type': 'string', + 'description': 'Only list pages whose path starts with this prefix, e.g. /features' + }, + }, + }, + }, + { + 'name': 'search_docs', + 'description': 'Full-text search across the documentation. Returns matching pages with a snippet and ' + 'their path; use read_page to fetch a page\'s full markdown.', + 'inputSchema': { + 'type': 'object', + 'properties': { + 'query': {'type': 'string', 'description': 'Search terms'}, + 'limit': {'type': 'integer', 'description': 'Maximum number of results (default 10)'}, + }, + 'required': ['query'], + }, + }, + { + 'name': 'read_page', + 'description': 'Return the full markdown source of a documentation page by its path (e.g. "/introduction").', + 'inputSchema': { + 'type': 'object', + 'properties': { + 'path': {'type': 'string', 'description': 'Page path, e.g. /introduction'}, + }, + 'required': ['path'], + }, + }, + ]; +} + +class _McpError implements Exception { + final int code; + final String message; + + const _McpError(this.code, this.message); +} diff --git a/rnd/ROADMAP.md b/rnd/ROADMAP.md index 5412178..adf63c8 100644 --- a/rnd/ROADMAP.md +++ b/rnd/ROADMAP.md @@ -36,6 +36,7 @@ | Official GitHub Action | Composite action: checksum-verified install, `stardust check` + build, outputs the built dir; PR-preview recipes for Netlify/Vercel/Cloudflare Pages. Ships with the `v0.7.0` release | | DartPad embeds | `` runnable Dart/Flutter snippets from a gist — lazy-loaded `dartpad.dev/?id=…&theme=…` iframe (v0.8 item 3) | | dartdoc integration | `stardust dartdoc` runs `dart doc` and co-hosts the API reference under `/api/` with a **theme-aware** Stardust top-bar (logo + primary-color badge, light/dark synced to the site both ways); the content column is tagged `data-pagefind-body` so the site search returns API symbols (unified search). Bodies stay dartdoc-styled (v0.8 item 2, phase 1) | +| Static-friendly MCP | `stardust mcp` serves any **built** site to Claude/Cursor over MCP (hand-rolled JSON-RPC, zero new deps): `list_pages` + `search_docs` + `read_page` tools and per-page markdown resources. **stdio** by default; `--http` adds a live Streamable HTTP `/mcp` endpoint (Origin-validated, localhost-bound) for self-hosters. Fed by a new build-time `llms.json` manifest — which also lets remote agents consume the site with no server at all (v0.8 item 1) | ### ⚠️ Have on paper — partial, broken, or documented-but-unimplemented | Feature | Reality | @@ -177,7 +178,9 @@ > > Then item 2 (**dartdoc integration**, phase 1) — `stardust dartdoc` runs `dart doc`, injects a self-contained Stardust top-bar and `data-pagefind-body` into each page via a single HTML post-pass (no scraping, no `dartdoc_options.yaml` intrusion), and writes to `public/api/`. The normal build copies it to `/api/` and Pagefind indexes it, so the site's search returns API symbols — **unified search verified end-to-end** (9 fragments incl. API pages on a fixture). Bodies stay dartdoc-styled; full native styling + auto-nav are deferred follow-ups. **Approach chosen because dartdoc has no clean machine-readable content export or stable library API** (verified against the official guide) — co-hosting dartdoc's real output is the robust path. > -> A "is it actually correct/UI-good?" review then hardened phase 1: (1) fixed a search-noise bug — tag `#dartdoc-main-content`, not dartdoc's `
` (which wraps the nav sidebars); (2) the top-bar is now **theme-aware** (logo + primary-color badge, static so it never collides with dartdoc's `z-index:2000` fixed-on-scroll header) with a **two-way theme sync** (site `localStorage['theme']` ↔ dartdoc's `colorTheme`); (3) subpath-aware home link (`basePath`, not the absolute `url`); (4) the output dir is cleared before writing. Demo'd end-to-end on a splash-landing + sidebar-guides + `/api/` site with unified search. Items 1, 4–7 remain. +> A "is it actually correct/UI-good?" review then hardened phase 1: (1) fixed a search-noise bug — tag `#dartdoc-main-content`, not dartdoc's `
` (which wraps the nav sidebars); (2) the top-bar is now **theme-aware** (logo + primary-color badge, static so it never collides with dartdoc's `z-index:2000` fixed-on-scroll header) with a **two-way theme sync** (site `localStorage['theme']` ↔ dartdoc's `colorTheme`); (3) subpath-aware home link (`basePath`, not the absolute `url`); (4) the output dir is cleared before writing. Demo'd end-to-end on a splash-landing + sidebar-guides + `/api/` site with unified search. + +> Then item 1 (**Static-friendly MCP**) — the second exit-criteria driver. Verified the MCP spec online first (stdio transport, protocol `2025-11-25`/`2025-06-18`; the "no-server" convention is llms.txt + per-page `.md`). Two halves: (a) `stardust mcp ` runs a **hand-rolled** JSON-RPC server (zero new deps, DI-testable via `MockFileSystem` + in-memory streams — chosen over the experimental `dart_mcp` package) exposing `list_pages` + `search_docs` + `read_page` tools and per-page markdown resources; (b) a new build-time `llms.json` manifest (gated by the existing `build.llms`, so no config/schema change) that both the local server and remote agents consume — the manifest is the unification point. The server reads a **built** dir (its `llms.json` + the `.md` files the build already emits), so it serves any built site, air-gapped, with no `stardust.yaml` at serve time. Two transports: **stdio** by default (logs to stderr so stdout stays a clean protocol channel), plus `--http` for a live Streamable HTTP `/mcp` endpoint (single POST endpoint over `shelf`, mandatory `Origin`-header validation for DNS-rebinding, localhost-bound, stateless) — the "run it as a service" path for self-hosters, since a pure static host can't answer a live endpoint. Verified end-to-end both ways: `stardust build` → `dist/llms.json` (45 pages), a piped stdio `initialize`→`tools/list`→`tools/call`→`read_page` handshake, and a real-socket `--http` run (`initialize`/`search`/202-notification/405-GET/403-foreign-origin all correct). **Both v0.8 exit criteria are now met** — one command replaces a hosted docs+dartdoc setup, and a Claude/Cursor user connects to any Stardust site in two clicks without the owner running a service. Items 4–7 remain. ### v1.0 — "Platform" (commit to stability) · when the above is real - **Compatibility promise**: stable config schema (migrations for breaking changes), stable CSS tokens, stable component syntax. diff --git a/stardust.yaml b/stardust.yaml index 2509de8..318d3f7 100644 --- a/stardust.yaml +++ b/stardust.yaml @@ -86,6 +86,8 @@ sidebar: label: OpenAPI Import - slug: features/dartdoc label: Dart API Docs + - slug: features/mcp + label: MCP Server - slug: features/llm-output label: LLM-Friendly Output - slug: features/redirects @@ -118,6 +120,8 @@ sidebar: label: stardust clean - slug: cli/openapi label: stardust openapi + - slug: cli/mcp + label: stardust mcp - group: Deployment icon: upload diff --git a/test/generator/site_generator_test.dart b/test/generator/site_generator_test.dart index 52aa28e..affc44f 100644 --- a/test/generator/site_generator_test.dart +++ b/test/generator/site_generator_test.dart @@ -1,3 +1,4 @@ +import 'dart:convert'; import 'dart:io'; import 'package:path/path.dart' as p; @@ -886,6 +887,56 @@ title: Home await tempDir.delete(recursive: true); } }); + + test('writes llms.json manifest with per-page md pointers, excluding llm: false', () async { + final tempDir = await Directory.systemTemp.createTemp('stardust_manifest'); + try { + final contentDir = p.join(tempDir.path, 'content'); + await Directory(contentDir).create(); + await File(p.join(contentDir, 'index.md')).writeAsString('---\ntitle: Home\ndescription: Landing.\n---\n\nHi.'); + await File(p.join(contentDir, 'guide.md')).writeAsString('---\ntitle: Guide\n---\n\nBody.'); + await File(p.join(contentDir, 'secret.md')).writeAsString('---\ntitle: Secret\nllm: false\n---\n\nHidden.'); + + final outputDir = p.join(tempDir.path, 'out'); + final config = StardustConfig(name: 'T', url: 'https://x.dev', content: ContentConfig(dir: contentDir)); + await SiteGenerator(config: config, outputDir: outputDir, logger: const Logger()).generate(); + + final manifest = jsonDecode(File(p.join(outputDir, 'llms.json')).readAsStringSync()) as Map; + expect(manifest['name'], 'T'); + expect(manifest['generator'], 'stardust'); + final pages = (manifest['pages'] as List).cast(); + final byPath = {for (final page in pages) page['path']: page}; + expect(byPath.keys, containsAll(['/', '/guide'])); + expect(byPath.containsKey('/secret'), isFalse, reason: 'llm: false page excluded'); + expect(byPath['/']?['md'], '/index.md'); + expect(byPath['/guide']?['md'], '/guide.md'); + expect(byPath['/']?['url'], 'https://x.dev/'); + expect(byPath['/']?['description'], 'Landing.'); + } finally { + await tempDir.delete(recursive: true); + } + }); + + test('omits llms.json when build.llms is disabled', () async { + final tempDir = await Directory.systemTemp.createTemp('stardust_manifest_off'); + try { + final contentDir = p.join(tempDir.path, 'content'); + await Directory(contentDir).create(); + await File(p.join(contentDir, 'index.md')).writeAsString('# Home'); + + final outputDir = p.join(tempDir.path, 'out'); + final config = StardustConfig( + name: 'T', + content: ContentConfig(dir: contentDir), + build: const BuildConfig(llms: LlmsConfig(enabled: false)), + ); + await SiteGenerator(config: config, outputDir: outputDir, logger: const Logger()).generate(); + + expect(File(p.join(outputDir, 'llms.json')).existsSync(), isFalse); + } finally { + await tempDir.delete(recursive: true); + } + }); }); group('incremental dev rebuilds', () { diff --git a/test/mcp/docs_source_test.dart b/test/mcp/docs_source_test.dart new file mode 100644 index 0000000..60d34c6 --- /dev/null +++ b/test/mcp/docs_source_test.dart @@ -0,0 +1,119 @@ +import 'package:stardust/src/mcp/docs_source.dart'; +import 'package:stardust/src/utils/exceptions.dart'; +import 'package:stardust/src/utils/logger.dart'; +import 'package:test/test.dart'; + +import '../mocks/mock_file_system.dart'; + +const _manifest = ''' +{ + "name": "Acme Docs", + "description": "The Acme SDK.", + "url": "https://acme.dev", + "generator": "stardust", + "pages": [ + { "path": "/", "title": "Home", "description": "Landing.", "url": "https://acme.dev/", "md": "/index.md" }, + { "path": "/guide", "title": "User Guide", "url": "https://acme.dev/guide", "md": "/guide.md" } + ] +} +'''; + +MockFileSystem _seed() { + final fs = MockFileSystem(); + fs.addFile('site/llms.json', _manifest); + fs.addFile('site/index.md', 'Welcome to the home page.'); + fs.addFile('site/guide.md', 'This guide explains widgets and gadgets. widgets widgets are everywhere.'); + return fs; +} + +void main() { + group('DocsSource.load', () { + test('parses the manifest into site metadata and pages', () async { + final source = await DocsSource.load('site', fileSystem: _seed(), logger: const Logger()); + + expect(source.siteName, 'Acme Docs'); + expect(source.siteDescription, 'The Acme SDK.'); + expect(source.siteUrl, 'https://acme.dev'); + expect(source.pages.map((p) => p.path), ['/', '/guide']); + expect(source.pages.last.title, 'User Guide'); + expect(source.pages.last.mdFile, '/guide.md'); + }); + + test('throws an actionable ContentException when llms.json is absent', () async { + final fs = MockFileSystem()..addFile('site/index.md', 'x'); + expect( + () => DocsSource.load('site', fileSystem: fs), + throwsA(isA().having((e) => e.message, 'message', contains('stardust build'))), + ); + }); + + test('throws on malformed manifest json', () async { + final fs = MockFileSystem()..addFile('site/llms.json', '"not an object"'); + expect(() => DocsSource.load('site', fileSystem: fs), throwsA(isA())); + }); + }); + + group('DocsSource.readPage', () { + test('returns the verbatim markdown for a known path', () async { + final source = await DocsSource.load('site', fileSystem: _seed()); + expect(await source.readPage('/guide'), contains('widgets and gadgets')); + }); + + test('returns null for an unknown path', () async { + final source = await DocsSource.load('site', fileSystem: _seed()); + expect(await source.readPage('/missing'), isNull); + }); + + test('returns null when the manifest lists a page whose md file is missing', () async { + final fs = MockFileSystem() + ..addFile('site/llms.json', '{"name":"X","pages":[{"path":"/gone","title":"Gone","md":"/gone.md"}]}'); + final source = await DocsSource.load('site', fileSystem: fs); + expect(await source.readPage('/gone'), isNull); + }); + }); + + group('DocsSource.search', () { + test('finds pages by body text and returns a snippet', () async { + final source = await DocsSource.load('site', fileSystem: _seed()); + final hits = await source.search('widgets'); + + expect(hits, hasLength(1)); + expect(hits.single.page.path, '/guide'); + expect(hits.single.snippet.toLowerCase(), contains('widgets')); + }); + + test('empty/whitespace query returns no hits', () async { + final source = await DocsSource.load('site', fileSystem: _seed()); + expect(await source.search(' '), isEmpty); + }); + + test('ranks a title match above a body-only match', () async { + final fs = MockFileSystem() + ..addFile( + 'site/llms.json', + '{"name":"X","pages":[' + '{"path":"/a","title":"Widgets","md":"/a.md"},' + '{"path":"/b","title":"Other","md":"/b.md"}]}') + ..addFile('site/a.md', 'nothing relevant here') + ..addFile('site/b.md', 'widgets widgets widgets'); + final source = await DocsSource.load('site', fileSystem: fs); + + final hits = await source.search('widgets'); + expect(hits.map((h) => h.page.path), ['/a', '/b']); + }); + + test('respects the result limit', () async { + final fs = MockFileSystem() + ..addFile( + 'site/llms.json', + '{"name":"X","pages":[' + '{"path":"/a","title":"Alpha","md":"/a.md"},' + '{"path":"/b","title":"Beta","md":"/b.md"}]}') + ..addFile('site/a.md', 'topic topic') + ..addFile('site/b.md', 'topic'); + final source = await DocsSource.load('site', fileSystem: fs); + + expect(await source.search('topic', limit: 1), hasLength(1)); + }); + }); +} diff --git a/test/mcp/mcp_http_server_test.dart b/test/mcp/mcp_http_server_test.dart new file mode 100644 index 0000000..94cafcd --- /dev/null +++ b/test/mcp/mcp_http_server_test.dart @@ -0,0 +1,111 @@ +import 'dart:convert'; + +import 'package:shelf/shelf.dart' as shelf; +import 'package:stardust/src/mcp/docs_source.dart'; +import 'package:stardust/src/mcp/mcp_http_server.dart'; +import 'package:stardust/src/mcp/mcp_server.dart'; +import 'package:test/test.dart'; + +import '../mocks/mock_file_system.dart'; + +Future _http({Set? allowedOrigins}) async { + final fs = MockFileSystem() + ..addFile('site/llms.json', '{"name":"Acme","pages":[{"path":"/","title":"Home","md":"/index.md"}]}') + ..addFile('site/index.md', 'Welcome home.'); + final source = await DocsSource.load('site', fileSystem: fs); + return McpHttpServer(McpServer(source, name: source.siteName), allowedOrigins: allowedOrigins); +} + +shelf.Request _req(String method, {String path = '/mcp', Map? headers, Object? body}) => shelf.Request( + method, + Uri.parse('http://localhost:8080$path'), + headers: headers, + body: body is Map ? jsonEncode(body) : body, + ); + +void main() { + group('POST /mcp', () { + test('a JSON-RPC request gets a 200 application/json response', () async { + final server = await _http(); + final res = await server.handle(_req('POST', body: {'jsonrpc': '2.0', 'id': 1, 'method': 'ping'})); + + expect(res.statusCode, 200); + expect(res.headers['content-type'], contains('application/json')); + expect((jsonDecode(await res.readAsString()) as Map)['id'], 1); + }); + + test('a notification (no id) gets 202 Accepted with no body', () async { + final server = await _http(); + final res = await server.handle(_req('POST', body: {'jsonrpc': '2.0', 'method': 'notifications/initialized'})); + + expect(res.statusCode, 202); + expect(await res.readAsString(), isEmpty); + }); + + test('initialize round-trips over HTTP', () async { + final server = await _http(); + final res = + await server.handle(_req('POST', body: {'jsonrpc': '2.0', 'id': 1, 'method': 'initialize', 'params': {}})); + final result = (jsonDecode(await res.readAsString()) as Map)['result'] as Map; + expect(result['serverInfo'], {'name': 'Acme', 'version': 'dev'}); + }); + }); + + group('method and path handling', () { + test('GET /mcp is 405 (no server-initiated stream)', () async { + final server = await _http(); + final res = await server.handle(_req('GET')); + expect(res.statusCode, 405); + expect(res.headers['allow'], 'POST'); + }); + + test('DELETE /mcp is 405 (stateless, no sessions)', () async { + final server = await _http(); + expect((await server.handle(_req('DELETE'))).statusCode, 405); + }); + + test('any other method (e.g. PUT) is 405', () async { + final server = await _http(); + final res = await server.handle(_req('PUT')); + expect(res.statusCode, 405); + expect(res.headers['allow'], 'POST'); + }); + + test('a wrong path is 404', () async { + final server = await _http(); + expect((await server.handle(_req('POST', path: '/'))).statusCode, 404); + }); + }); + + group('Origin validation (DNS-rebinding guard)', () { + Future status(McpHttpServer server, String? origin) async { + final res = await server.handle(_req('POST', + headers: origin == null ? null : {'origin': origin}, body: {'jsonrpc': '2.0', 'id': 1, 'method': 'ping'})); + return res.statusCode; + } + + test('requests with no Origin (native clients) are allowed', () async { + expect(await status(await _http(), null), 200); + }); + + test('loopback origins are allowed', () async { + final server = await _http(); + expect(await status(server, 'http://localhost:3000'), 200); + expect(await status(server, 'http://127.0.0.1:9999'), 200); + }); + + test('a foreign browser origin is rejected with 403', () async { + expect(await status(await _http(), 'https://evil.example.com'), 403); + }); + + test('an explicitly allowed origin passes', () async { + final server = await _http(allowedOrigins: {'https://docs.acme.dev'}); + expect(await status(server, 'https://docs.acme.dev'), 200); + }); + + test('a wildcard allows any origin', () async { + final server = await _http(allowedOrigins: {'*'}); + expect(await status(server, 'https://anything.example.com'), 200); + }); + }); +} diff --git a/test/mcp/mcp_server_test.dart b/test/mcp/mcp_server_test.dart new file mode 100644 index 0000000..cf9d8e0 --- /dev/null +++ b/test/mcp/mcp_server_test.dart @@ -0,0 +1,330 @@ +import 'dart:async'; +import 'dart:convert'; +import 'dart:io'; +import 'dart:typed_data'; + +import 'package:stardust/src/core/file_system.dart'; +import 'package:stardust/src/mcp/docs_source.dart'; +import 'package:stardust/src/mcp/mcp_server.dart'; +import 'package:test/test.dart'; + +import '../mocks/mock_file_system.dart'; + +/// Wraps a [MockFileSystem] but throws on `readFile` for paths ending in +/// [throwFor] — so a manifest can load yet a page body read blows up. +class _ReadThrowsFileSystem implements FileSystem { + final MockFileSystem inner; + final String throwFor; + + _ReadThrowsFileSystem(this.inner, this.throwFor); + + @override + Future readFile(String path) async { + if (path.replaceAll('\\', '/').endsWith(throwFor)) throw const FileSystemException('boom'); + return inner.readFile(path); + } + + @override + Future fileExists(String path) => inner.fileExists(path); + @override + Future directoryExists(String path) => inner.directoryExists(path); + @override + Future readFileBytes(String path) => inner.readFileBytes(path); + @override + Future writeFile(String path, String content) => inner.writeFile(path, content); + @override + Future writeFileBytes(String path, Uint8List bytes) => inner.writeFileBytes(path, bytes); + @override + Future copyFile(String source, String destination) => inner.copyFile(source, destination); + @override + Stream listDirectory(String path, {bool recursive = false}) => + inner.listDirectory(path, recursive: recursive); + @override + Future lastModified(String path) => inner.lastModified(path); + @override + Future createDirectory(String path, {bool recursive = false}) => + inner.createDirectory(path, recursive: recursive); + @override + Future deleteDirectory(String path, {bool recursive = false}) => + inner.deleteDirectory(path, recursive: recursive); +} + +Future _server() async { + final fs = MockFileSystem() + ..addFile( + 'site/llms.json', + '{"name":"Acme Docs","pages":[' + '{"path":"/","title":"Home","description":"Landing.","md":"/index.md"},' + '{"path":"/guide","title":"User Guide","md":"/guide.md"}]}') + ..addFile('site/index.md', 'Welcome home.') + ..addFile('site/guide.md', 'A guide about widgets.'); + final source = await DocsSource.load('site', fileSystem: fs); + return McpServer(source, name: source.siteName, version: '1.2.3'); +} + +/// Sends one request and decodes the JSON-RPC response (fails if none). +Future> _call(McpServer server, Map request) async { + final raw = await server.handle(jsonEncode(request)); + expect(raw, isNotNull, reason: 'expected a response for ${request['method']}'); + return jsonDecode(raw as String) as Map; +} + +void main() { + group('initialize', () { + test('echoes the requested protocol version and advertises capabilities', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 1, + 'method': 'initialize', + 'params': {'protocolVersion': '2025-11-25'}, + }); + + final result = res['result'] as Map; + expect(result['protocolVersion'], '2025-11-25'); + expect((result['capabilities'] as Map).keys, containsAll(['tools', 'resources'])); + expect(result['serverInfo'], {'name': 'Acme Docs', 'version': '1.2.3'}); + }); + + test('falls back to the default protocol version when none is requested', () async { + final server = await _server(); + final res = await _call(server, {'jsonrpc': '2.0', 'id': 1, 'method': 'initialize', 'params': {}}); + expect((res['result'] as Map)['protocolVersion'], McpServer.defaultProtocolVersion); + }); + }); + + group('notifications', () { + test('notifications/initialized produces no response', () async { + final server = await _server(); + expect(await server.handle(jsonEncode({'jsonrpc': '2.0', 'method': 'notifications/initialized'})), isNull); + }); + + test('an unknown notification (no id) produces no response', () async { + final server = await _server(); + expect(await server.handle(jsonEncode({'jsonrpc': '2.0', 'method': 'x/unknown'})), isNull); + }); + }); + + group('tools', () { + test('tools/list advertises list_pages, search_docs, and read_page with input schemas', () async { + final server = await _server(); + final res = await _call(server, {'jsonrpc': '2.0', 'id': 2, 'method': 'tools/list'}); + + final tools = ((res['result'] as Map)['tools'] as List).cast(); + expect(tools.map((t) => t['name']), containsAll(['list_pages', 'search_docs', 'read_page'])); + expect(tools.every((t) => t['inputSchema'] is Map), isTrue); + }); + + test('tools/call list_pages returns the full catalog, and a prefix filters it', () async { + final server = await _server(); + + final all = await _call(server, { + 'jsonrpc': '2.0', + 'id': 2, + 'method': 'tools/call', + 'params': {'name': 'list_pages', 'arguments': {}}, + }); + final allText = ((all['result'] as Map)['content'] as List).first['text'] as String; + expect(allText, contains('/ — Home')); + expect(allText, contains('/guide — User Guide')); + + final filtered = await _call(server, { + 'jsonrpc': '2.0', + 'id': 3, + 'method': 'tools/call', + 'params': { + 'name': 'list_pages', + 'arguments': {'prefix': '/guide'} + }, + }); + final filteredText = ((filtered['result'] as Map)['content'] as List).first['text'] as String; + expect(filteredText, contains('/guide — User Guide')); + expect(filteredText, isNot(contains('/ — Home'))); + }); + + test('tools/call search_docs returns a text hit', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 3, + 'method': 'tools/call', + 'params': { + 'name': 'search_docs', + 'arguments': {'query': 'widgets'} + }, + }); + + final content = ((res['result'] as Map)['content'] as List).cast(); + expect(content.single['type'], 'text'); + expect(content.single['text'], contains('/guide')); + expect((res['result'] as Map).containsKey('isError'), isFalse); + }); + + test('tools/call search_docs with no results reports so (not an error)', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 3, + 'method': 'tools/call', + 'params': { + 'name': 'search_docs', + 'arguments': {'query': 'zzzznotfound'} + }, + }); + final result = res['result'] as Map; + expect((result['content'] as List).first['text'], contains('No results')); + expect(result.containsKey('isError'), isFalse); + }); + + test('tools/call search_docs with empty query is a tool error', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 3, + 'method': 'tools/call', + 'params': { + 'name': 'search_docs', + 'arguments': {'query': ' '} + }, + }); + expect((res['result'] as Map)['isError'], isTrue); + }); + + test('tools/call read_page returns the page markdown', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 4, + 'method': 'tools/call', + 'params': { + 'name': 'read_page', + 'arguments': {'path': '/guide'} + }, + }); + expect(((res['result'] as Map)['content'] as List).first['text'], 'A guide about widgets.'); + }); + + test('tools/call read_page for a missing page is a tool error', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 4, + 'method': 'tools/call', + 'params': { + 'name': 'read_page', + 'arguments': {'path': '/nope'} + }, + }); + expect((res['result'] as Map)['isError'], isTrue); + }); + + test('tools/call read_page with an empty path is a tool error', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 4, + 'method': 'tools/call', + 'params': { + 'name': 'read_page', + 'arguments': {'path': ''} + }, + }); + expect((res['result'] as Map)['isError'], isTrue); + }); + + test('an unexpected error while serving a tool surfaces as JSON-RPC -32603', () async { + // Manifest loads fine, but reading the page body throws — exercising the + // internal-error fallback. + final fs = MockFileSystem() + ..addFile('site/llms.json', '{"name":"X","pages":[{"path":"/boom","title":"Boom","md":"/boom.md"}]}') + ..addFile('site/boom.md', 'body'); + final source = await DocsSource.load('site', fileSystem: _ReadThrowsFileSystem(fs, 'boom.md')); + final server = McpServer(source, name: 'X'); + + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 4, + 'method': 'tools/call', + 'params': { + 'name': 'read_page', + 'arguments': {'path': '/boom'} + }, + }); + expect((res['error'] as Map)['code'], -32603); + }); + }); + + group('resources', () { + test('resources/list exposes one markdown resource per page', () async { + final server = await _server(); + final res = await _call(server, {'jsonrpc': '2.0', 'id': 5, 'method': 'resources/list'}); + + final resources = ((res['result'] as Map)['resources'] as List).cast(); + expect(resources.map((r) => r['uri']), ['stardust:///', 'stardust:///guide']); + expect(resources.every((r) => r['mimeType'] == 'text/markdown'), isTrue); + }); + + test('resources/read returns the page contents by uri', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 6, + 'method': 'resources/read', + 'params': {'uri': 'stardust:///guide'}, + }); + final contents = ((res['result'] as Map)['contents'] as List).cast(); + expect(contents.single['uri'], 'stardust:///guide'); + expect(contents.single['text'], 'A guide about widgets.'); + }); + + test('resources/read for an unknown uri returns a JSON-RPC error', () async { + final server = await _server(); + final res = await _call(server, { + 'jsonrpc': '2.0', + 'id': 6, + 'method': 'resources/read', + 'params': {'uri': 'stardust:///nope'}, + }); + expect((res['error'] as Map)['code'], -32602); + }); + }); + + group('protocol basics', () { + test('ping returns an empty result', () async { + final server = await _server(); + final res = await _call(server, {'jsonrpc': '2.0', 'id': 7, 'method': 'ping'}); + expect(res['result'], isEmpty); + }); + + test('an unknown method (with id) returns method-not-found', () async { + final server = await _server(); + final res = await _call(server, {'jsonrpc': '2.0', 'id': 8, 'method': 'does/notexist'}); + expect((res['error'] as Map)['code'], -32601); + }); + + test('invalid json returns a parse error with null id', () async { + final server = await _server(); + final res = jsonDecode((await server.handle('{ not json')) as String) as Map; + expect((res['error'] as Map)['code'], -32700); + expect(res['id'], isNull); + }); + }); + + group('serve loop', () { + test('reads newline-delimited requests and writes each response, skipping notifications', () async { + final server = await _server(); + final input = Stream.fromIterable([ + jsonEncode({'jsonrpc': '2.0', 'id': 1, 'method': 'ping'}), + '', // blank line ignored + jsonEncode({'jsonrpc': '2.0', 'method': 'notifications/initialized'}), // no response + jsonEncode({'jsonrpc': '2.0', 'id': 2, 'method': 'tools/list'}), + ]); + final out = []; + await server.serve(input, out.add); + + expect(out, hasLength(2)); + expect((jsonDecode(out[0]) as Map)['id'], 1); + expect((jsonDecode(out[1]) as Map)['id'], 2); + }); + }); +}