GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking

by   Jiayan Guo, et al.

Large language models (LLM) like ChatGPT have become indispensable to artificial general intelligence (AGI), demonstrating excellent performance in various natural language processing tasks. In the real world, graph data is ubiquitous and an essential part of AGI and prevails in domains like social network analysis, bioinformatics and recommender systems. The training corpus of large language models often includes some algorithmic components, which allows them to achieve certain effects on some graph data-related problems. However, there is still little research on their performance on a broader range of graph-structured data. In this study, we conduct an extensive investigation to assess the proficiency of LLMs in comprehending graph data, employing a diverse range of structural and semantic-related tasks. Our analysis encompasses 10 distinct tasks that evaluate the LLMs' capabilities in graph understanding. Through our study, we not only uncover the current limitations of language models in comprehending graph structures and performing associated reasoning tasks but also emphasize the necessity for further advancements and novel approaches to enhance their graph processing capabilities. Our findings contribute valuable insights towards bridging the gap between language models and graph understanding, paving the way for more effective graph mining and knowledge extraction.


page 1

page 2

page 3

page 4


Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations

Large Language Models (LLMs) have revolutionized natural language proces...

Do Large Language Models Know What They Don't Know?

Large language models (LLMs) have a wealth of knowledge that allows them...

RoCar: A Relationship Network-based Evaluation Method to Large Language Models

Large language models (LLMs) have received increasing attention. However...

TaskLAMA: Probing the Complex Task Understanding of Language Models

Structured Complex Task Decomposition (SCTD) is the problem of breaking ...

Large Language Models for Semantic Monitoring of Corporate Disclosures: A Case Study on Korea's Top 50 KOSPI Companies

In the rapidly advancing domain of artificial intelligence, state-of-the...

LLM4Jobs: Unsupervised occupation extraction and standardization leveraging Large Language Models

Automated occupation extraction and standardization from free-text job p...

Large Language Models as Data Preprocessors

Large Language Models (LLMs), typified by OpenAI's GPT series and Meta's...

Please sign up or login with your details

Forgot password? Click here to reset