ORION

Operational Routine for the Ingest and Output of Networks

ORION ingests data from over 40 knowledge sources and converts them into interoperable Biolink Model knowledge graphs in KGX format.

Knowledge graphs can be composed of any number of data sources. ORION operates like a dependency manager; to build a graph ORION resolves data source dependencies as needed, by either downloading preprocessed artifacts from a centralized repository or by running an ingest pipeline to build graphs from scratch, before merging them together into unified graphs. Sub-graphs downloaded or generated by the ingest pipeline are stored in versioned directories with accompanying metadata and can be reused in subsequent graph builds.

ORION tracks versions of data sources and is designed to automatically update source data dependencies as they become available. It also tracks versions of the software, services, and data models used so that it can build new graphs as appropriate when underlying dependencies change.

ORION generates thorough metadata about its data, processes, and pipelines, for troubleshooting and understanding, and for distribution with graph outputs. These files align with FAIR principles aimed to increase transparency, provenance, and (re)usability of knowledge graphs for humans and machines.

Installation

ORION requires Python 3.12+ and uv for dependency management (uv will install a suitable Python for you if needed).

Alternatively, using Docker is the best way to ensure the environment is configured correctly, and is highly recommended for any usage of ORION involving neo4j.

git clone https://github.com/RobokopU24/ORION.git
cd ORION
uv sync --extra robokop

uv sync installs the orion-* commands (listed under CLI Commands) into the project's .venv. Run them with uv run, e.g. uv run orion-build ..., or activate the environment first with source .venv/bin/activate and call them directly.

The core library is also available on PyPI (pip install robokop-orion), but the full repository is needed to utilize ingest modules from the ROBOKOP project.

Quickstart

Build a customized graph from any of the available data sources:

uv run orion-build My_Graph --sources DrugCentral,HGNC

Outputs are written to ~/ORION-workspace/graphs/<graph_id>/<version>/ by default. See Configuration to select a different location.

CLI Commands

The following commands are available (prefix with uv run if not using a uv-managed shell):

Command	Description
orion-build	Build complete knowledge graphs from a Graph Spec, by specifying sources on the command line, or by naming a single data source to materialize it directly as a graph
orion-ingest	Run the ingest pipeline to download and process individual data sources from scratch
orion-merge	Merge pairs of KGX node/edge files into one knowledge graph
orion-meta-kg	Generate MetaKG and test data files
orion-redundant-kg	Generate edge files with redundant biolink predicates
orion-ac	Generate AnswerCoalesce files
orion-neo4j-dump	Generate Neo4j database dumps
orion-memgraph-dump	Generate Memgraph database dumps

Graph Specs

A Graph Spec yaml file defines which sources to include in a knowledge graph. Every *.yaml file at the top level of graph_specs/ is loaded automatically when you run orion-build, so any graph_id defined in those files can be built directly by name. Project-specific specs that aren't part of the default build live under graph_specs/optional/.

To build a graph outside the default available set, you have several options:

# 1. Provide any local file or http(s) URL for a Graph Spec outside of ORION.
orion-build My_Graph --graph_spec /path/to/my-graph-spec.yaml
orion-build My_Graph --graph_spec https://example.org/my-graph-spec.yaml

# 2. Build a simple graph from a list of sources without writing a yaml file.
orion-build My_Graph --sources DrugCentral,HGNC
orion-build My_Graph --sources DrugCentral,HGNC --output_format neo4j

# 3. Build one of the optional specs in ORION by providing its file path.
orion-build LitCoin_ORION --graph_spec graph_specs/optional/litcoin-graph-spec.yaml

# 4. Materialize a single data source directly as a graph (using its data source id).
orion-build CTD

You can build several graphs (or sources) in one invocation by separating their ids with commas, e.g. orion-build HGNC,CTD or orion-build My_Robokop_Graph,LitCoin_ORION. Note that a graph_id cannot be the same as a data source id — the two share one namespace so every id maps to exactly one thing to build.

Add --conflation (-c) to any orion-build command to force conflation on for every source, overriding the spec.

Here is a simple Graph Spec example:

graphs:
  - graph_id: Example_Graph
    graph_name: Example Graph
    graph_description: A free text description of what is in the graph.
    output_format: neo4j
    sources:
      - id: DrugCentral
      - id: HGNC

See the full list of data sources and their identifiers in the data sources file.

Graph Spec Parameters

Graph Specs allow a number of options for customization. The following parameters can be set per data source:

• merge_strategy - alternative merge strategies
• strict_normalization - discard nodes that fail to normalize (true/false)
• conflation - conflate genes with proteins and chemicals with drugs during normalization (true/false) see Babel's Conflation docs.

The following can be set at the graph level:

• output_format - which database/file formats to generate. Valid values: jsonl (the default KGX node/edge files, always produced), neo4j, memgraph, redundant_jsonl, redundant_neo4j, collapsed_qualifiers_jsonl, collapsed_qualifiers_neo4j, answercoalesce. Combine multiple with +, e.g. neo4j+jsonl. The neo4j/memgraph formats (and the Reactome ingest) require a Neo4j installation; the provided Docker image includes it. If Neo4j is not installed, the dump is logged as an error and skipped, but the rest of the graph still builds.
• add_edge_id - whether to add unique identifiers to edges (true/false)
• edge_id_type - if add_edge_id is true, the type of identifier can be specified (uuid or orion)
• base_release_version - release version floor for the graph (e.g. "2.0" or "2.1.0")

Graph Versions

Every built graph has two versions:

• A build version — a deterministic hash of the graph's inputs (source versions, normalization scheme, merge strategy, graph dependency versions). Identical inputs always produce the same build version.
• A release version — a human-facing semantic version (MAJOR.MINOR.PATCH) used in output directory names, URLs, and metadata.

ORION picks the release version automatically. On a new build it checks existing releases of the same graph (from the graph registry, then local storage). If a previous release has the same build version, that release is reused. Otherwise the highest existing release version's PATCH component is incremented. To start at or jump to a different MAJOR/MINOR, set base_release_version: in the graph spec — it acts as a floor that the auto-bump never goes below.

See the graph_specs/ directory for more examples.

Configuration

ORION can be configured via environment variables, which can be set directly or through an .env file.

All variables are optional; the table below lists some that users typically override.

Variable	Purpose	Default
ORION_STORAGE	Path to a directory for data ingest pipeline storage	~/ORION-workspace/storage
ORION_GRAPHS	Path to a directory for Knowledge Graph outputs	~/ORION-workspace/graphs
ORION_LOGS	Path to a Log file directory (if unset, logs go to stdout)	None
ORION_OUTPUT_URL	Base URL recorded in graph metadata and distribution links	https://localhost
ORION_USE_GRAPH_REGISTRY	Consult the remote graph registry for prebuilt dependencies and version discovery. Set false for fully offline/air-gapped builds.	true
ORION_GRAPH_REGISTRY_URL	Base URL of the graph registry.	https://robokop-graph-registry.apps.renci.org

ORION will create ~/ORION-workspace/storage/ and ~/ORION-workspace/graphs/ on first use if the corresponding env vars aren't set.

When the graph registry is enabled (the default), ORION checks it for prebuilt graph dependencies before building them from scratch. If the registry is unreachable or misconfigured, ORION logs a warning and falls back to local storage and building — it never fails a build. For offline use, set ORION_USE_GRAPH_REGISTRY=false to skip the registry entirely.

For more customization and settings, copy or rename .env.example to .env and uncomment the variables you want to override.

If you'd prefer repo-adjacent output directories during development, as opposed to your home directory, this optional helper script will create them and set the corresponding environment variables for you:

source ./set_up_dev_env.sh

Configuration is managed by pydantic-settings — environment variables override .env file values, and sensible defaults are provided where possible. See orion/config.py for the full list of settings.

Running with Docker

ORION's Docker compose does require ORION_STORAGE and ORION_GRAPHS to be set on the host - docker-compose mounts those host paths into the container as volumes, so outputs persist on disc after the container exits. Either export them in your shell or put them in .env.

Build the image:

docker compose build

Build a graph using docker compose:

docker compose run orion orion-build Example_Graph

Run the ingest pipeline for a single data source:

docker compose run orion orion-ingest DrugCentral

See available data sources and options:

docker compose run orion orion-ingest -h

Running with Helm

Helm charts are available for running ORION on Kubernetes. See helm/orion/values.yaml for the full set of values, including graphSpecUrl, graphSpecYaml, and optionalGraphSpec for a variety of options for providing graph specs.

Development

Install dev dependencies with uv:

uv sync --extra robokop --group dev

Run tests:

uv run pytest tests/

Contributing

Contributions are welcome, see the Contributor README.