Neo4j MPGNN Thesis/ Page 5 of 5
Results & Findings
Key findings: no accuracy loss with database-backed pipelines, cost-efficient training on smaller machines, and competitive in-database inference performance.
Main Findings
Our thesis produced three key findings:
1. No Accuracy Loss with Database-Backed Pipelines
Our custom sampling methods for Neo4j were statistically indistinguishable from standard implementations in PyTorch Geometric.
We ran extensive experiments across five datasets (Cora, ogbn-arxiv, ogbn-products, ogbn-papers100M, Coauthor Physics) and found no significant difference in accuracy between:
- PyG’s in-memory neighbor sampling
- Neo4j Cypher-based sampling
- Neo4j Java UDP sampling
This is a critical result: it means we can shift to a database-backed approach without sacrificing model quality.
2. Cost-Efficient Training on Smaller Machines
Moving from a pure in-memory data source to a RAM/disk hybrid via a graph database allows smaller, cheaper machines to be used for training.
The tradeoff is that the pipeline can become slower due to database query latency. However, our custom Java UDP plugin addressed this by:
- Minimizing data transfer (byte-array features)
- Moving computation into the database
- Optimizing sampling procedures
On large graphs like ogbn-papers100M, the database-backed approach made training feasible on machines with limited RAM, whereas the in-memory approach would require expensive high-memory instances.
3. In-Database Inference is Competitive
In-database inference was at least as accurate as PyTorch Geometric, and one of our methods was actually faster than sending data out for computation externally.
Specifically:
- In-database Cypher inference: Same accuracy, moderate speed
- In-database Java inference: Same accuracy, faster than Cypher, competitive with external inference
This opens up possibilities for running GNN inference entirely within the database, reducing architectural complexity.
Performance Benchmarks
We benchmarked across multiple dimensions:
- Accuracy: Node classification accuracy on test sets
- Training time: Wall-clock time per epoch
- Memory usage: Peak RAM consumption
- Data transfer: Bytes transferred between Neo4j and Python
The Java UDP approach consistently outperformed Cypher-based methods on speed and data transfer, while maintaining identical accuracy.
Practical Application
As a practical application, we studied categorization of scientific articles in citation networks. The model was trained on keywords in articles from fields like computer science and economics, but also drew conclusions based on word choice in citing articles. After training, the model could extrapolate and categorize large volumes of data not seen during training.
This demonstrates the real-world applicability of our approach: building cost-efficient ML pipelines for graph-structured data that scale to large, real-world graphs.
Read More
The full thesis is available as open access: Cost-Efficient Graph Database-backed Graph Neural Network Pipelines
A popular scientific summary in Swedish is also available here.