Benchmarking the Combinatorial Generalizability of Complex Query Answering on Knowledge Graphs

by   Zihao Wang, et al.
Tsinghua University
The Hong Kong University of Science and Technology

Complex Query Answering (CQA) is an important reasoning task on knowledge graphs. Current CQA learning models have been shown to be able to generalize from atomic operators to more complex formulas, which can be regarded as the combinatorial generalizability. In this paper, we present EFO-1-QA, a new dataset to benchmark the combinatorial generalizability of CQA models by including 301 different queries types, which is 20 times larger than existing datasets. Besides, our work, for the first time, provides a benchmark to evaluate and analyze the impact of different operators and normal forms by using (a) 7 choices of the operator systems and (b) 9 forms of complex queries. Specifically, we provide the detailed study of the combinatorial generalizability of two commonly used operators, i.e., projection and intersection, and justify the impact of the forms of queries given the canonical choice of operators. Our code and data can provide an effective pipeline to benchmark CQA models.


Logical Message Passing Networks with One-hop Inference on Atomic Formulas

Complex Query Answering (CQA) over Knowledge Graphs (KGs) has attracted ...

EFO_k-CQA: Towards Knowledge Graph Complex Query Answering beyond Set Operation

To answer complex queries on knowledge graphs, logical reasoning over in...

Neural-Symbolic Entangled Framework for Complex Query Answering

Answering complex queries over knowledge graphs (KG) is an important yet...

Neural Methods for Logical Reasoning Over Knowledge Graphs

Reasoning is a fundamental problem for computers and deeply studied in A...

Rethinking Existential First Order Queries and their Inference on Knowledge Graphs

Reasoning on knowledge graphs is a challenging task because it utilizes ...

Complex Query Answering with Neural Link Predictors

Neural link predictors are immensely useful for identifying missing edge...

Wasserstein-Fisher-Rao Embedding: Logical Query Embeddings with Local Comparison and Global Transport

Answering complex queries on knowledge graphs is important but particula...

Code Repositories


Benchmark for Answering Existential First Order Queries with Single Free Variable

view repo

Please sign up or login with your details

Forgot password? Click here to reset