feat: add Query.Builder, aggregates, joins, and nested ON - #1817
feat: add Query.Builder, aggregates, joins, and nested ON#1817abnegate wants to merge 30 commits into
Conversation
Greptile SummaryThe PR expands generated Query APIs across supported SDKs with aggregate, grouping, spatial, JOIN, nested ON, and fluent builder support, while adding structural request-value flattening and cross-language tests.
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope. No blocking failure remains. Important Files Changed
Reviews (28): Last reviewed commit: "fix: type QueryTest fixtures as array an..." | Re-trigger Greptile |
|
@greptileai review |
1 similar comment
|
@greptileai review |
Query helpers now cover the query-lib methods Appwrite allows (aggregates, groupBy, having, distinct, joins, extra spatial). Query.builder()/build() collects filters as a flat string list, and Client.normalize unwraps the common extra-array wrap of a builder or build()/page() result so list and POST /query calls stay valid.
`any` does not exist on Go 1.13, which still compiles the SDK in CI. Realtime subscription updates now go through Query.normalize instead of iterating QueryInput, and Client.subscribe no longer writes a queries set that is not on the Realtime connection type.
Query.normalize was only flattening a builder or nested build()/page() array before send. Call sites already do that on the queries param, so it does not belong on the public Query helper surface.
Keep the simple join(table, left, right, op, alias) triple. Array overloads encode Query.on() plus filter queries into nested join values so extra ON predicates stay on the JOIN.
Add e2e prints and unit tests for aggregates, joins, ON, spatial, and page so each SDK exercises every new query type.
Clients now map every param through a type-based flatten so builders and extra nested lists unwrap without special-casing the queries key. Document the new public Query helpers.
Live specs encode closed enums as type:string plus oneOf of single-value string schemas. Parsing that as an object made path params Any, JSON keys the enum title, and Ruby docs crash on non-JSON examples. Keep model JSON keys as the property name, type signatures from the collapsed enum, and flatten Dart request maps as Map<String, dynamic>.
420fd0f to
844e3e1
Compare
Collapsed oneOf string enums with no title produced empty type names (`val status: ,`, `resourceType: ;`) and invalid PHP properties.
Move QueryBuilder into Query/Builder files and nested types where the language allows it. Alias QueryBuilder at import for PHP and the TypeScript public export.
Dart part-of is resolved relative to lib/query, so the builder library looked for packageName.dart in the wrong directory, and a top-level Builder class collides with Flutter's widget. Go treats query.Builder() as a type conversion. PHP mock payloads and from() treated array-of-enum fields as a single string, which is how live-spec OAuth prompt examples are encoded. CLI Validation now compiles against a Go SDK generated from the same spec instead of the last published module.
A replace in go.mod is the local pin; Validation no longer needs an env override or go mod edit. example.php writes examples/go from the same spec so ../go matches the CLI module path.
Request flattening is a transport concern. Keep Query as the helper surface and run flatten from Client in Kotlin, Android, Swift, and Apple.
Keep annotated enumerations from main and the query builder work. CLI go.mod still replace-pins the sibling generated Go SDK at v7.2.0-rc.3.
There was a problem hiding this comment.
Pull request overview
Adds cross-language query aggregation, grouping, joins, spatial operators, fluent builders, request flattening, and enum-handling improvements to generated SDKs.
Changes:
- Expands Query APIs and adds fluent builders across supported SDK languages.
- Normalizes nested query inputs in clients and adds broad E2E/unit coverage.
- Improves closed
oneOfenum detection and scalar array-enum hydration.
Reviewed changes
Copilot reviewed 110 out of 110 changed files in this pull request and generated 29 comments.
Show a summary per file
| File(s) | Description |
|---|---|
tests/resources/spec-openapi3.json |
Adds closed oneOf enum fixtures. |
tests/e2e/Base.php |
Adds helper-output and enum hydration assertions. |
tests/e2e/languages/{web,node,react-native,deno}/* |
Exercises TypeScript query helpers and builders. |
tests/e2e/languages/{swift,apple}/Tests.swift |
Exercises Swift query helpers. |
tests/e2e/languages/{kotlin,android}/Tests.kt |
Exercises Kotlin query helpers. |
tests/e2e/languages/{go,go-v2}/tests.go |
Exercises Go query helpers. |
tests/e2e/languages/{dart,flutter}/tests.dart |
Exercises Dart query helpers. |
tests/e2e/languages/{dotnet,unity}/Tests.cs |
Exercises C# query helpers. |
tests/e2e/languages/{ruby,python,php}/* |
Exercises dynamic-language query helpers. |
templates/web/src/{query,client,index}.ts.twig |
Adds Query builder/input normalization and exports. |
templates/web/src/services/realtime.ts.twig |
Normalizes realtime query inputs. |
templates/node/src/{client,index}.ts.twig |
Adds Node request flattening and exports. |
templates/node/test/query.test.js.twig |
Tests Node query behavior. |
templates/react-native/src/{query,client,index}.ts.twig |
Adds React Native query building and flattening. |
templates/deno/{mod.ts,src/query.ts,src/client.ts}.twig |
Adds Deno query building and flattening. |
templates/deno/test/query.test.ts.twig |
Tests Deno query behavior. |
templates/swift/Sources/Query*.twig |
Adds Swift helpers and nested builder. |
templates/swift/Sources/Client.swift.twig |
Flattens Swift request parameters. |
templates/swift/{Tests/Tests.swift,Package.swift}.twig |
Adds tests and conditional package targets. |
templates/apple/{Sources/Client.swift,Package.swift}.twig |
Applies inherited Swift client/package updates. |
templates/kotlin/src/main/kotlin/io/appwrite/{Query,Client}.kt.twig |
Adds Kotlin query APIs and flattening. |
templates/kotlin/src/test/kotlin/io/appwrite/QueryTest.kt.twig |
Adds Kotlin Query tests. |
templates/android/library/src/main/java/io/package/{Query,Client}.kt.twig |
Adds Android query APIs and flattening. |
templates/go/query*.twig |
Adds Go helpers, builder, and tests. |
templates/go/client.go.twig |
Flattens Go request values. |
templates/rust/src/query/{mod,builder}.rs.twig |
Adds Rust Query APIs and builder. |
templates/rust/src/client.rs.twig |
Normalizes Rust request parameters. |
templates/rust/tests/tests.rs |
Exercises Rust query helpers. |
templates/dotnet/Package/Query*.twig |
Adds .NET helpers and nested builder. |
templates/dotnet/Package/Client.cs.twig |
Flattens .NET request parameters. |
templates/php/src/{Query,Client}.php.twig |
Adds PHP helpers, builder integration, and flattening. |
templates/php/src/Query/Builder.php.twig |
Implements the PHP fluent builder. |
templates/php/src/Models/{Model,RequestModel}.php.twig |
Wraps scalar array-enum values during hydration. |
templates/php/tests/QueryTest.php.twig |
Tests PHP Query behavior. |
templates/python/package/{query,client}.py.twig |
Adds Python builder APIs and flattening. |
templates/python/test/test_query.py.twig |
Tests Python Query behavior. |
templates/ruby/lib/container/{query,client}.rb.twig |
Adds Ruby helpers and request flattening. |
templates/ruby/lib/container/query/builder.rb.twig |
Implements the Ruby fluent builder. |
templates/dart/lib/{query,package}.dart.twig |
Adds Dart Query APIs and builder registration. |
templates/dart/lib/query/builder.dart.twig |
Implements QueryBuilder. |
templates/dart/lib/src/client_mixin.dart.twig |
Flattens Dart request parameters. |
templates/dart/lib/src/models/{model,request_model}.dart.twig |
Handles scalar array-enums and enum serialization. |
templates/dart/test/{query_test,src/models/model_test}.dart.twig |
Extends Dart Query/model tests. |
templates/flutter/lib/{package,src/client_mixin}.dart.twig |
Registers builders and flattens Flutter parameters. |
templates/skills/*.md.twig |
Documents query aggregation, joins, and builders. |
templates/cli/internal/typegen/templates/{types,databases}.ts.hbs |
Extends typed CLI Query helpers. |
templates/cli/internal/generator/typescript.go |
Reuses the generated QueryBuilder type. |
templates/cli/internal/generator/testdata/* |
Updates expected TypeScript output. |
templates/cli/go.mod.twig |
Redirects CLI builds to a local generated Go SDK. |
src/SDK/SDK.php |
Improves enum-name fallback resolution. |
src/SDK/Language.php |
Resolves untitled enum names from parent properties. |
src/SDK/Language/{Swift,Apple}.php |
Registers Swift builder output. |
src/SDK/Language/{Kotlin,Android}.php |
Registers Query tests. |
src/SDK/Language/{Dart,Flutter}.php |
Registers Dart builder output. |
src/SDK/Language/{Go,Rust,PHP,Ruby,DotNet}.php |
Registers new builder/module templates. |
example.php |
Generates Go alongside CLI and updates the Go SDK version. |
Suppressed comments (2)
templates/cli/internal/typegen/templates/types.ts.hbs:76
- These fixed scalar signatures cannot express the new nested JOIN ON form, and the facade exposes no
onhelper. Addonplus overloads/unions for(table, on)and(table, alias, on)so the typed callback can construct the nested wire shape supported byQuery.
join: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
leftJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
rightJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
fullOuterJoin: (table: string, left: string, right: string, operator?: string, alias?: string) => string;
templates/cli/internal/typegen/templates/databases.ts.hbs:44
- The facade delegates only the scalar join form and does not expose
Query.on, so typed CLI callbacks cannot use the nested JOIN ON feature added by this PR. Mirror the nested overloads andonhelper exposed byQuery.
join: (table, left, right, operator = '=', alias = '') => Query.join(table, left, right, operator, alias),
leftJoin: (table, left, right, operator = '=', alias = '') => Query.leftJoin(table, left, right, operator, alias),
rightJoin: (table, left, right, operator = '=', alias = '') => Query.rightJoin(table, left, right, operator, alias),
fullOuterJoin: (table, left, right, operator = '=', alias = '') => Query.fullOuterJoin(table, left, right, operator, alias),
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Rust was treating every nested array as query input, Go compared uncomparable slices, and Dart/Flutter left wrapped builders nested. Skills now show build() only; typed CLI query helpers include the new aggregates. Regenerating Rust deletes leftover src/query.rs.
|
Went through Copilot's 29 threads. Fixed
Left as-is
|
Language e2e tests now print flatten-builder/geometry/list so a wrapped builder that stays nested or a numeric array that gets spread fails CI. Query flatten is callable from those tests; PHP form flatten is flattenForm so it no longer collides with Client::flatten.
Typed QueryBuilder gained stddev/variance/bitwise; baseline fixtures still expected groupBy at that offset and failed TestGenerateMatchesBaseline.
|
Pushed two commits on top of the flatten-bugfix head:
CLIGo126 / Validation |
|
@greptileai review |
Flutter query_test constructing Client() hits path_provider before the widget binding exists. Apple flatten returns [Any], so as? [String] failed even when the query strings were correct.
|
@greptileai review |
Linux Swift does not cast [Query.Builder] or [String] to [Any], so flatten left wrapped builders nested. Walk those typed arrays (and Mirror as a fallback) so builder lists unwrap on Apple and Swift.
|
@greptileai review |
Foundation can surface [String] as NSArray, so casting to [Any] first walked NSString as a character collection and broke flatten-list on Apple.
|
@greptileai review |
Keep query flatten tests and enum hydration checks. Take main's inferred upload ID fixture assertion and Black-formatted Python SDK templates.
|
@greptileai review |
Validation runs black --check on generated Python tests. Swift flatten now returns [Query.Builder].flatMap(build) and [String] before Any/NSArray so Linux does not walk query strings as character collections.
|
@greptileai review |
Homogeneous [Query.Builder] does not round-trip through Any on Linux. Request params are Any values, so the language tests now pass [Any].
|
@greptileai review |
Keep query flatten helpers and enum fixtures. Take PHP Pint/PHPStan quality templates and localized enum keys from main. Swift flatten spreads any nested string list returned from a builder.
|
@greptileai review |
Generated PHP tests now use PSR-12 braces. Swift/Apple language tests no longer assert Client.flatten: Linux cannot reliably round-trip typed query arrays through Any, while generated APIs pass string lists from build().
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
|
@greptileai review |
Why
Appwrite query-lib added aggregates, grouping, joins, extra spatial operators, and a POST
/querybody so largequeriesarrays are not capped by URL length. SDKs that already exposeQueryhelpers need those methods, a fluent collector, and a way to nest JOIN ON conditions without leaking SQL.What changed
count,sum,avg, …),groupBy/having/distinct, joins (join/leftJoin/rightJoin/fullOuterJoin/crossJoin), extra spatial (covers, …).Query.Builderwhere the language allows it (C#/Swift/Kotlin/Ruby/Python/TS namespace). PHP isAppwrite\Query\Builderimported asQueryBuilder. Go/Rust use packageBuilderwithNewBuilder()/query::Builder. Dart cannot nest classes and must not export a top-levelBuilder(Flutter widget clash), so the type isQueryBuilderinlib/query/builder.dart.Query.on(...)plus join overloads that take an alias, an ON query, and extra filters. The wire shape is[alias?, Query.on, filters…].Query.normalize. Flattening is structural (builder / nested query-string lists) and does not special-case thequeriesparameter name.type: stringplusoneOfof single-value schemas are treated as wire enums. Untitled oneOf members take their name from the parent property.from()and generated mocks wrap a scalar example for an array-of-enum field into a one-element list (OAuthpromptin the live spec is a string example on an array field).QUERY_HELPER_RESPONSEScovers each new helper in every SDK language test.Raw/union/json/naturalJoin stay unexposed — those are not on the Documents validator allow-list.
Vector helpers were already on
main; duplicate copies introduced during the rebase were removed so each language has a single definition.Why this approach
A client-side JSON collector (not SQL) matches the existing Query string format
{method, attribute?, values?}. Nesting the builder underQuerykeeps the public surface one type. Flattening inside the Client, without aquerieskey check, means any param that already carries query strings (join ON lists,having,or/and) is handled the same way.POST
/queryitself lives in Appwrite (appwrite/appwrite#13300, stacked on#11649). SDK methods will be generated ascreateQueryonce that spec is published.Verified
composer lint(phpcs PSR-12)composer lint-twig(djLint, 0 errors)composer refactor:check(Rector dry-run)composer test tests/e2e/PHP83Test.php— OK, 1511 assertions (includes array-enum scalar hydration)lib/+tests/tests.dartanalyze clean for the Builder/part-of/MockKind failuresquery.NewBuilder().Limit(1).Build()compiles and prints the expected limit queryNot verified
/queryround-trip from a generated SDK (depends on appwrite#13300 + spec publish).../govia areplaceingo.mod.twig(generated from the same spec as the CLI). Compiling the published CLI package against the last published Go module is not verified here.