Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a “Similarity” feature across the stack: a self-hosted CLIP embedder service generates vectors for assets, the Databox API stores/indexes them and serves a “similar assets” endpoint, and the Databox UI surfaces similar results in the asset view and via an action dialog.
Changes:
- Add a Python/FastAPI
similarity-embedderservice (Dockerized) to compute CLIP embeddings. - Add backend support for storing embeddings, indexing them into Elasticsearch as a
dense_vector, and servingGET /assets/{id}/similarusing kNN + ACL pre-filtering. - Add frontend UI (panel + dialog + action) to fetch and display similar assets and their scores.
Reviewed changes
Copilot reviewed 36 out of 37 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| similarity/embedder/requirements.txt | Pins Python dependencies for the embedder service. |
| similarity/embedder/main.py | Implements /embed, /embed-text, and /healthz endpoints for embedding generation. |
| similarity/embedder/Dockerfile | Builds the embedder image and pre-downloads model weights at build time. |
| docker-compose.yml | Adds the similarity-embedder service/profile and related env; adjusts Elasticsearch volume/heap; removes ElasticHQ. |
| docker-compose.dev.yml | Exposes the embedder port in dev compose override. |
| doc/tech/Databox/integration/01_integrations.md | Documents the new similarity integration and adds it to the integration list. |
| databox/client/src/components/Media/Asset/View/AssetView.tsx | Adds the Similar Assets panel into the asset view layout. |
| databox/client/src/components/Media/Asset/SimilarAssets.tsx | New accordion panel that loads and displays similar assets. |
| databox/client/src/components/Media/Asset/Actions/SimilarAssetsDialog.tsx | New modal dialog to browse more similar assets. |
| databox/client/src/components/Media/Asset/Actions/SaveFileAsNewAssetDialog.tsx | Refactors destination handling using tree node data + IRI helper. |
| databox/client/src/components/Media/Asset/Actions/AssetViewActions.tsx | Adds “Find similar” action opening the similar-assets dialog. |
| databox/client/src/api/asset.ts | Adds getSimilarAssets() API helper and result typing. |
| databox/api/tests/ElasticSearch/KnnQueryTest.php | Adds unit tests for the new ES kNN query serialization/filter embedding. |
| databox/api/src/Service/Vector/EmbedderClient.php | Adds an HTTP client wrapper to call the similarity embedder service. |
| databox/api/src/Service/Vector/AssetEmbeddingManager.php | Adds embedding generation + persistence logic for assets. |
| databox/api/src/Integration/Core/Similarity/SimilarityIntegration.php | Adds a workflow integration to compute embeddings on ingest and depend on rendition generation. |
| databox/api/src/Integration/Core/Similarity/SimilarityEmbedAction.php | Workflow action that triggers embedding generation. |
| databox/api/src/Entity/Core/AssetEmbedding.php | New entity/table for persisted embeddings and metadata. |
| databox/api/src/Entity/Core/Asset.php | Exposes /assets/{id}/similar collection operation via API Platform. |
| databox/api/src/Elasticsearch/SimilarAssetSearch.php | Implements similar-asset search with ES kNN + ACL pre-filtering. |
| databox/api/src/Elasticsearch/Query/Knn.php | Adds Elastica query wrapper for ES kNN query (ES >= 8.12). |
| databox/api/src/Elasticsearch/Listener/AssetPostTransformListener.php | Injects embedding vectors into indexed asset documents. |
| databox/api/src/Elasticsearch/AppIndexableDependencyResolver.php | Ensures Asset reindexing happens when an embedding changes. |
| databox/api/src/Controller/Admin/DashboardController.php | Adds admin menu entry for the Asset Embedding CRUD. |
| databox/api/src/Controller/Admin/AssetEmbeddingCrudController.php | Adds read-only admin UI for viewing embeddings. |
| databox/api/src/Consumer/Handler/Similarity/SimilarityEmbedHandler.php | Adds async handler to compute embeddings via messenger. |
| databox/api/src/Consumer/Handler/Similarity/SimilarityEmbed.php | Adds messenger message for embedding computation jobs. |
| databox/api/src/Command/SimilarityIndexCommand.php | Adds CLI command to backfill embeddings (sync or queued). |
| databox/api/src/Api/Provider/SimilarAssetCollectionProvider.php | API provider for the “similar assets” collection, including scores meta. |
| databox/api/migrations/Version20260802120000.php | Migration creating asset_embedding table and constraints. |
| databox/api/fixtures/Newspaper.yaml | Enables the similarity integration in fixtures for the Newspaper workspace. |
| databox/api/config/packages/framework.yaml | Configures the named HTTP client for the embedder base URI. |
| databox/api/config/packages/fos_elastica.yaml | Adds embedding as an indexed dense_vector (cosine similarity). |
| databox/api/.env | Adds default SIMILARITY_EMBEDDER_URL. |
| dashboard/client/src/global.d.ts | Removes ELASTICHQ_URL from typed globals. |
| dashboard/client/src/App.tsx | Removes ElasticHQ link from the dashboard UI. |
| .env | Enables the similarity compose profile and updates Elasticsearch image/version + embedder dev port. |
Suppressed comments (1)
doc/tech/Databox/integration/01_integrations.md:161
- The integration list entry uses
similarity, but the actual integration key added by this PR iscore.similarity. The table should reflect the real key so it can be copy/pasted for configuration.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+28
to
+41
| $rendition = $this->renditionManager->getAssetRenditionByName($asset->getId(), $renditionName); | ||
| $file = $rendition?->getFile(); | ||
| if (null === $file || !FileUtil::isImageType($file->getType())) { | ||
| return false; | ||
| } | ||
|
|
||
| $path = $this->fileFetcher->getFile($file); | ||
| $result = $this->embedderClient->embedImageFile($path); | ||
|
|
||
| $existingEmbedding = $this->em->getRepository(AssetEmbedding::class) | ||
| ->findOneBy(['asset' => $asset->getId()]); | ||
| if (null !== $existingEmbedding && !$force) { | ||
| return false; | ||
| } |
Comment on lines
+45
to
+52
| @app.post("/embed") | ||
| async def embed(file: UploadFile) -> dict: | ||
| data = await file.read() | ||
| try: | ||
| image = Image.open(io.BytesIO(data)).convert("RGB") | ||
| except Exception: | ||
| raise HTTPException(status_code=422, detail="Unsupported or corrupted image") | ||
|
|
…er-compose for elasticsearch volume
The service now lives in alchemy-fr/phrasea-similarity-embedder with its own CI publishing to the private AWS ECR registry. Phrasea pulls the published image (SIMILARITY_EMBEDDER_IMAGE, defaults to 122649456891.dkr.ecr.eu-west-3.amazonaws.com/ps-similarity-embedder:latest) instead of building it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…atusChangedHandler
… rendition handler
The report API now lives in alchemy-fr/phrasea-report with its own CI publishing to ECR Public. Phrasea pulls the published image (REPORT_API_IMAGE, defaults to public.ecr.aws/b2s9z7l1/ps-report-api:latest) instead of building it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The custom Keycloak image (SPI mappers + phrasea theme) now lives in alchemy-fr/phrasea-keycloak with its own CI publishing to ECR Public. Phrasea pulls the published image (KEYCLOAK_IMAGE, defaults to public.ecr.aws/b2s9z7l1/ps-keycloak:latest) instead of building it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.