Inference Strategies

We benchmarked four different strategies for running inference with a trained GNN model, ranging from loading the entire graph into memory to running inference entirely inside the database.

1. Full Graph (Baseline)

The simplest approach: load the entire graph into RAM and run the forward pass in Python using PyTorch Geometric.

How it works:

• Fetch all nodes, edges, and features from Neo4j at startup
• Store in PyG’s in-memory Data object
• Run standard PyG inference

Pros:

• Fastest inference (no database queries during inference)
• Simple implementation

Cons:

• Requires enough RAM to hold the entire graph
• Not feasible for very large graphs (e.g., ogbn-papers100M)

This serves as our baseline for accuracy and speed.

2. Neighborhood Sampling

Instead of loading the full graph, fetch only the k-hop neighborhood for each node being inferred.

How it works:

• For each target node, query Neo4j for its k-hop subgraph
• Build a mini-graph and run the forward pass
• Aggregate results

Pros:

• Much lower memory usage
• Scales to larger graphs

Cons:

• Slower due to database queries
• Approximate (depends on sampling strategy)

3. In-Database Cypher Inference

Run the entire inference pipeline inside Neo4j using Cypher queries.

How it works:

• Express the GNN forward pass as a series of Cypher queries
• Neo4j computes node representations layer by layer
• Return final predictions

Pros:

• No data transfer between database and application
• Can leverage Neo4j’s query optimization

Cons:

• Requires float features (embedding_bytes_floats), which are slower for training
• Limited by Cypher’s expressiveness

4. In-Database Java Inference

Use the custom Java UDP plugin to run inference inside Neo4j.

How it works:

• Deploy the trained model to Neo4j as a Java procedure
• Call the procedure from Cypher to run inference
• Model runs in the JVM inside Neo4j

Pros:

• Minimal data transfer
• Faster than Cypher-based inference
• Can use optimized byte-array features

Cons:

• Requires building and deploying the Java plugin
• More complex setup

Comparison

All four methods produced statistically indistinguishable accuracy on our benchmark datasets. The key differentiator was speed and resource usage:

• Full graph: Fastest but highest memory
• Neighborhood sampling: Lower memory but slower
• In-database Cypher: Moderate speed, requires specific feature format
• In-database Java: Best balance of speed and memory for large graphs