' which treats the LLM as an agent to interactively explore related entities and relations on KGs and perform reasoning based on the retrieved knowledge. We further implement this paradigm by introducing a new approach called Think-on-Graph (ToG), in which the LLM agent iteratively executes beam search on KG, discovers the most promising reasoning paths, and returns the most likely reasoning results. We use a number of well-designed experiments to examine and illustrate the following advantages of ToG: 1) compared with LLMs, ToG has better deep reasoning power; 2) ToG has the ability of knowledge traceability and knowledge correctability by leveraging LLMs reasoning and expert feedback; 3) ToG provides a flexible plug-and-play framework for different LLMs, KGs and prompting strategies without any additional training cost; 4) the performance of ToG with small LLM models could exceed large LLM such as GPT-4 in certain scenarios and this reduces the cost of LLM deployment and application. As a training-free method with lower computational cost and better generality, ToG achieves overall SOTA in 6 out of 9 datasets where most previous SOTAs rely on additional training.","authors":[{"id":"65d45da0c136ef133167f4f8","name":"Jiashuo Sun","org":"Xiamen University","orgid":"5f71b2bd1c455f439fe3deb7"},{"id":"5621bfd845cedb3398351800","name":"Chengjin Xu","org":"International Digital Economy Academy","orgid":"5f71b4021c455f439fe46f0d"},{"id":"6526776055b3f8ac46609063","name":"Lumingyuan Tang","org":"University of Southern California","orgid":"5f71b4161c455f439fe47839"},{"id":"64484aeee3c28ee84c2cc0cd","name":"Saizhuo Wang","org":"The Hong Kong University of Science and Technology","orgid":"62331e370a6eb147dca8abfb"},{"id":"542b7405dabfae2b4e16abd0","name":"Chen Lin","org":"Xiamen University","orgid":"5f71b2bd1c455f439fe3deb7"},{"id":"53f4479fdabfaeecd69b16e0","name":"Yeyun Gong","org":"Microsoft","orgid":"5f71b2831c455f439fe3c634"},{"id":"63156db0cd729caec636deee","name":"Lionel Ni","org":"The Hong Kong University of Science and Technology (Guangzhou))","orgid":"5f71b2961c455f439fe3ce4f"},{"id":"53f4cdfcdabfaeedd377b4ed","name":"Heung-Yeung Shum","org":"Microsoft","orgid":"5f71b2831c455f439fe3c634"},{"id":"6597e4c1e577a45cc2f0213c","name":"Jian Guo","org":"Hong Kong University of Science and Technology","orgid":"62331e370a6eb147dca8abfb"}],"create_time":"2023-07-18T04:54:09.925Z","doi":"10.48550\u002Farxiv.2307.07697","id":"64b60eaa3fda6d7f06eae92c","issn":"2331-8422","keywords":["Knowledge Graph","Chain-of-Thought","Large Language Models"],"lang":"en","num_citation":131,"pdf":"https:\u002F\u002Fcz5waila03cyo0tux1owpyofgoryroob.aminer.cn\u002F77\u002F2F\u002F53\u002F772F53026680C3728976F44F51E0F4D2.pdf","title":"Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph","update_times":{"u_a_t":"2024-09-29T19:08:40Z","u_c_t":"2024-11-17T05:52:47.658Z","u_v_t":"2024-12-16T14:54:15.103Z"},"urls":["http:\u002F\u002Farxiv.org\u002Fabs\u002F2307.07697"],"venue":{"info":{"name":"arXiv (Cornell University)","publisher":"Cornell University"},"volume":"abs\u002F2307.07697"},"venue_hhb_id":"5ea18efeedb6e7d53c00a01c","versions":[{"id":"64b60eaa3fda6d7f06eae92c","sid":"2307.07697","src":"arxiv","vsid":"S4306400194","year":2023},{"id":"64c78b983fda6d7f06db49b8","sid":"journals\u002Fcorr\u002Fabs-2307-07697","src":"dblp","vsid":"journals\u002Fcorr","year":2023},{"id":"65ea8ca813fb2c6cf6314f10","sid":"nnVO1PvbTv","src":"conf_iclr","vsid":"ICLR.cc\u002F2024\u002FConference","year":2024},{"id":"667b00169255e7a318a59986","sid":"119601712acbe6ee133a1744f0970190c4195519","src":"semanticscholar","vsid":"1901e811-ee72-4b20-8f7e-de08cd395a10","year":2023},{"id":"66c60bff6c88b2fc28cb717d","sid":"fbcda3e1f19b1ed86a30b7c34f792e8e738f32ae","src":"semanticscholar","year":2023},{"id":"66e0233701d2a3fbfc29e4df","sid":"conf\u002Ficlr\u002FSunXTW0GNSG24","src":"dblp","vsid":"conf\u002Ficlr","year":2024},{"id":"6578d6f2939a5f40826c4e74","sid":"W4384643740","src":"openalex","vsid":"S4306400194","year":2023}],"year":2023},{"abstract":"While the celebrated graph neural networks (GNNs) yield effective representations for individual nodes of a graph, there has been relatively less success in extending to the task of graph similarity learning. Recent work on graph similarity learning has considered either global-level graph–graph interactions or low-level node–node interactions, however, ignoring the rich cross-level interactions (e.g., between each node of one graph and the other whole graph). In this article, we propose a multilevel graph matching network (MGMN) framework for computing the graph similarity between any pair of graph-structured objects in an end-to-end fashion. In particular, the proposed MGMN consists of a node–graph matching network (NGMN) for effectively learning cross-level interactions between each node of one graph and the other whole graph, and a siamese GNN to learn global-level interactions between two input graphs. Furthermore, to compensate for the lack of standard benchmark datasets, we have created and collected a set of datasets for both the graph–graph classification and graph–graph regression tasks with different sizes in order to evaluate the effectiveness and robustness of our models. Comprehensive experiments demonstrate that MGMN consistently outperforms state-of-the-art baseline models on both the graph–graph classification and graph–graph regression tasks. Compared with previous work, multilevel graph matching network (MGMN) also exhibits stronger robustness as the sizes of the two input graphs increase.","authors":[{"email":"lingxiang@zju.edu.cn","id":"64cc9dbe70d44317c2e37ce3","name":"Xiang Ling","org":"Zhejiang Univ, Coll Comp Sci & Technol, Hangzhou 310027, Peoples R China","orgid":"5f71b29c1c455f439fe3d0e1"},{"email":"lwu@email.wm.edu","id":"562d97d045cedb3398e741ce","name":"Lingfei Wu","org":"JD COM Silicon Valley Res Ctr, Mountain View, CA 94043 USA"},{"email":"szwang@zju.edu.cn","id":"64484aeee3c28ee84c2cc0cd","name":"Saizhuo Wang","org":"Zhejiang Univ, Coll Comp Sci & Technol, Hangzhou 310027, Peoples R China","orgid":"5f71b29c1c455f439fe3d0e1"},{"email":"tengfei.ma1@ibm.com","id":"5d50c3a07390bff0db2a732c","name":"Tengfei Ma","org":"IBM TJ Watson Res Ctr, Ossining, NY 10598 USA","orgid":"5f71b5c41c455f439fe532ea"},{"id":"53f42c95dabfaee1c0a27aa1","name":"Fangli Xu","org":"Yixue Educ Inc, Squirrel AI Learning, Highland Pk, NJ 08904 USA"},{"email":"alexliu@antfin.com","id":"54489d12dabfae87b7e4fe6d","name":"Alex X. Liu","org":"Ant Financial Serv Grp, Hangzhou 310013, Peoples R China"},{"email":"wuchunming@zju.edu.cn","id":"56cb18c6c35f4f3c65662081","name":"Chunming Wu","org":"Zhejiang Univ, Coll Comp Sci & Technol, Hangzhou 310027, Peoples R China","orgid":"5f71b29c1c455f439fe3d0e1"},{"email":"sji@zju.edu.cn","id":"53f42cd6dabfaeb1a7b83432","name":"Shouling Ji","org":"Zhejiang Univ, Coll Comp Sci & Technol, Hangzhou 310027, Peoples R China","orgid":"5f71b29c1c455f439fe3d0e1"}],"create_time":"2023-06-08T06:23:48.476Z","doi":"10.1109\u002Ftnnls.2021.3102234","id":"5f08318791e01137f866759c","issn":"2162-237X","keywords":["Task analysis","Computational modeling","Robustness","Training","Benchmark testing","Standards","Research and development","Code similarity","deep learning","graph neural network (GNN)","graph similarity"],"lang":"en","num_citation":21,"pages":{"end":"813","start":"799"},"pdf":"https:\u002F\u002Fstatic.aminer.cn\u002Fstorage\u002Fpdf\u002Farxiv\u002F20\u002F2007\u002F2007.04395.pdf","title":"Multilevel Graph Matching Networks for Deep Graph Similarity Learning","update_times":{"u_a_t":"2024-12-25T19:10:57Z","u_c_t":"2023-11-06T03:29:04.19Z","u_v_t":"2024-12-25T19:10:57Z"},"urls":["https:\u002F\u002Farxiv.org\u002Fabs\u002F2007.04395"],"venue":{"info":{"name":"IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS"},"issue":"2","volume":"34"},"venue_hhb_id":"5ea19ab7edb6e7d53c00ac14","versions":[{"id":"6456478ed68f896efae29654","sid":"journals\u002Fcorr\u002Fabs-2007-04395","src":"dblp","vsid":"IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS","year":2023},{"id":"64b510743fda6d7f06ea16b1","sid":"W3131761525","src":"openalex","year":2021},{"id":"65793adc939a5f408212fa76","sid":"W3139081114","src":"openalex","vsid":"S4210175523","year":2021},{"id":"66762d3c93c353b323d52e24","sid":"d3f4749bb7955c9f50c76a27809b17be196c5326","src":"semanticscholar","vsid":"79c5a18d-0295-432c-aaa5-961d73de6d88","year":2020},{"id":"66c5633e6c88b2fc283dc97b","sid":"2fa0a90ed9d1f44999767b4d4e176ac6165c86ca","src":"semanticscholar","vsid":"79c5a18d-0295-432c-aaa5-961d73de6d88","year":2020},{"id":"5f08318791e01137f866759c","sid":"2007.04395","src":"arxiv","year":2020},{"id":"6124cf945244ab9dcb975f6f","sid":"34406948","src":"pubmed","vsid":"101616214","year":2023},{"id":"63dee8f090e50fcafde094d4","sid":"9516695","src":"ieee","vsid":"5962385","year":2023},{"id":"6419634a90e50fcafd5bd133","sid":"journals\u002Ftnn\u002FLingWWMXLWJ23","src":"dblp","vsid":"journals\u002Ftnn","year":2023},{"id":"64a355c3d68f896efac334d7","sid":"10.1109\u002Ftnnls.2021.3102234","src":"crossref","year":2023},{"id":"645bc00cf11ef10fc633888d","sid":"WOS:000732079600001","src":"wos","vsid":"IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS","year":2023}],"year":2023},{"authors":[{"id":"663f57cca05b27f854d0c047","name":"Xuhui Jiang"},{"id":"5621bfd845cedb3398351800","name":"Chengjin Xu"},{"id":"53f7997ddabfae938c6c5242","name":"Yinghan Shen"},{"name":"Xun Sun"},{"id":"6526776055b3f8ac46609063","name":"Lumingyuan Tang"},{"id":"64484aeee3c28ee84c2cc0cd","name":"Saizhuo Wang"},{"id":"645afe25c81747b42a805d8a","name":"Zhongwu Chen"},{"id":"542d7637dabfae11fc4732a3","name":"Yuanzhuo Wang"},{"id":"6597e4c1e577a45cc2f0213c","name":"Jian Guo"}],"create_time":"2023-10-10T11:29:39.156Z","id":"654a244a939a5f40827d56cd","lang":"en","num_citation":4,"pdf":"https:\u002F\u002Fcz5waila03cyo0tux1owpyofgoryroob.aminer.cn\u002F55\u002F2A\u002F04\u002F552A049AAD3EC605A65236D0953CAF1A.pdf","title":"On the Evolution of Knowledge Graphs: A Survey and Perspective.","update_times":{"u_a_t":"2024-09-14T02:46:31Z","u_c_t":"2024-08-18T09:36:22.496Z","u_v_t":"2024-09-14T02:46:31Z"},"urls":["https:\u002F\u002Fdoi.org\u002F10.48550\u002FarXiv.2310.04835"],"venue":{"info":{"name":"CoRR"},"volume":"abs\u002F2310.04835"},"versions":[{"id":"654a244a939a5f40827d56cd","sid":"journals\u002Fcorr\u002Fabs-2310-04835","src":"dblp","vsid":"journals\u002Fcorr","year":2023}],"year":2023}],"profilePubsTotal":13,"profilePatentsPage":0,"profilePatents":null,"profilePatentsTotal":null,"profilePatentsEnd":false,"profileProjectsPage":0,"profileProjects":null,"profileProjectsTotal":null,"newInfo":null,"checkDelPubs":[]}};