manage_GRAPH_global_contexts loads the whole graph per backend, unbounded and label-unfiltered

[NotHimmel]

Describe the bug

manage_GRAPH_global_contexts() (age_global_graph.c:1024) builds a full in-memory copy
of every vertex and edge of a graph in TopMemoryContext (:1033), once per backend.
There is no memory ceiling and no GUC, and the load ignores the label scope of the calling
query. A 3-hop shortest_path() on a 500k-vertex graph takes ~3.3 s the first time, of
which ~1 ms is the search and the rest is the load. On a 3M-vertex / 9M-edge graph the
cache is ~2.3 GB per backend, so a few concurrent graph queries OOM a modest server.

Three separate observations:

1. The load is not shared between backends. A second session pays the full cost again.
2. The load ignores the query's label scope. load_vertex_hashtable() (:715) and
   load_edge_hashtable() (:817) call table_beginscan(..., snapshot, 0, NULL) at
   :748 and :850 — zero scan keys. Passing edge_types does not reduce the load;
   the filter is applied inside the traversal, after the load.
3. There is no upper bound. The context grows to whatever the graph size demands and is
   released only when the graph version changes (is_ggctx_invalid(), :235) or on an
   explicit age_delete_global_graphs(). A read-only workload holds it for the life of
   the session, and nothing stops the OOM killer.

This is not a regression, and not a defect in the new shortest-path code: sp_run_bfs()
(age_vle.c:2983) and the SP_MAX_RESULT_PATHS cap (:3215) are fine. But
age_shortest_path calls manage_GRAPH_global_contexts() at age_vle.c:3632, so it
inherits this from VLE and makes it much easier to hit — one shortest_path() call now
pays the whole load.

How are you accessing AGE (Command line, driver, etc.)?

• psql (command line)

What data setup do we need to do?

Two graphs with identical vertices and an identical E1 ring. sp3 additionally has
E2/E3, which every query below explicitly excludes. Takes about a minute to build.

LOAD 'age'; SET search_path = ag_catalog, public;
SET maintenance_work_mem = '1GB';

-- sp3: 500k vertices, THREE edge labels x 500k edges
SELECT create_graph('sp3');
SELECT create_vlabel('sp3','V');
SELECT create_elabel('sp3','E1');
SELECT create_elabel('sp3','E2');
SELECT create_elabel('sp3','E3');
INSERT INTO sp3."V"(id, properties)
SELECT _graphid(3,i::bigint), ('{"k":'||i||'}')::agtype FROM generate_series(1,500000) i;
-- E1 is a ring i -> i+1, so vertex 1 reaches vertex 4 in exactly 3 hops
INSERT INTO sp3."E1"(id, start_id, end_id, properties)
SELECT _graphid(4,i::bigint), _graphid(3,i::bigint), _graphid(3,(i%500000)+1), '{}'::agtype
FROM generate_series(1,500000) i;
-- E2/E3 are chords, irrelevant to any E1-only query
INSERT INTO sp3."E2"(id, start_id, end_id, properties)
SELECT _graphid(5,i::bigint), _graphid(3,i::bigint), _graphid(3,((i::bigint*104729)%500000)+1), '{}'::agtype
FROM generate_series(1,500000) i;
INSERT INTO sp3."E3"(id, start_id, end_id, properties)
SELECT _graphid(6,i::bigint), _graphid(3,i::bigint), _graphid(3,((i::bigint*15485863)%500000)+1), '{}'::agtype
FROM generate_series(1,500000) i;

-- sp1: same vertices, ONLY the E1 ring
SELECT create_graph('sp1');
SELECT create_vlabel('sp1','V');
SELECT create_elabel('sp1','E1');
INSERT INTO sp1."V"(id, properties)
SELECT _graphid(3,i::bigint), ('{"k":'||i||'}')::agtype FROM generate_series(1,500000) i;
INSERT INTO sp1."E1"(id, start_id, end_id, properties)
SELECT _graphid(4,i::bigint), _graphid(3,i::bigint), _graphid(3,(i%500000)+1), '{}'::agtype
FROM generate_series(1,500000) i;

ANALYZE sp3."V"; ANALYZE sp3."E1"; ANALYZE sp3."E2"; ANALYZE sp3."E3";
ANALYZE sp1."V"; ANALYZE sp1."E1";

What is the necessary configuration info needed?

• Nothing beyond a default install. shared_buffers = 1GB here; no concurrent writers,
  so the cache stays valid across the measurements.
• On PG 18, detect_version_mode() selects VERSION_MODE_DSM, so snapshot-based
  invalidation is not involved.

What is the command that caused the error?

There is no error message — the symptom is time and memory. Run this in a fresh
session each time, changing the graph name for step 5:

LOAD 'age'; SET search_path = ag_catalog, public;
SELECT pg_backend_pid() AS pid \gset
\setenv PID :pid
\! grep VmRSS /proc/$PID/status
\timing on
SELECT count(*) FROM age_shortest_path('"sp3"'::agtype,
       _graphid(3,1)::agtype, _graphid(3,4)::agtype,
       '"E1"'::agtype, NULL, NULL, 3::agtype);
\timing off
\! grep VmRSS /proc/$PID/status

Every step returns the one correct 3-hop path. VmRSS is the serving backend's.

step	session	graph	time	backend RSS
1	A	-	baseline after LOAD 'age'	15.7 MB
2	A	sp3	3296 ms	15.7 MB -> 439 MB
3	A	sp3	1.15 ms (identical call again)	439 MB
4	B (new)	sp3	3751 ms	19.9 MB -> 439 MB
5	C (new)	sp1	1252 ms	15.6 MB -> 307 MB

Step 4 repeated in three further fresh sessions: 3162 / 3383 / 3437 ms, 438.6 MB each
time.

• 2 vs 3 — the search costs 1.15 ms; the other ~3295 ms is the cache build.
• 2 vs 4 — a second backend pays it again. The cache is backend-private.
• 4 vs 5 — same query, same answer, edge_types = '"E1"' in both. sp3 merely also
  contains E2/E3; those 1M excluded edges still cost +132 MB and +2.50 s.

Same build at 3M vertices + 3 x 3M edges, max_hops = 4: first call 22.4 s / 2.27 GB,
identical call in the same session 3.0 ms, a new session 24.8 s / 2.27 GB again, and
edge_types = '"E1"' in a fresh session still 23.4 s / 2.27 GB. A user reported a backend
killed with anon-rss 3.3 GB + signal 9 on a graph of this shape (their report, not my
measurement).

Expected behavior

Two things, in order of how badly they are needed:

1. A graph query should not be able to drive the backend into the OOM killer with no
   configurable limit. Exceeding a bound should fail the query with a clear message.
2. A 3-hop query on a graph with a start vertex should not require the entire graph to be
   resident, should not re-pay the load in every backend, and should not load edge labels
   the query has explicitly excluded.

Environment (please complete the following information):

• Version: AGE 1.8.0 (PG18/v1.8.0-rc0, commit e43dc1a1)
• PostgreSQL 18.4