' (e.g., *Tom is the parent of John*), LLM fails to directly conclude ''$B \\gets A ' (e.g., *John is the child of Tom*) during inference even if the two sentences are semantically identical, which is known as the ''reversal curse''. In this paper, we theoretically analyze the reversal curse via the training dynamics of (stochastic) gradient descent for two auto-regressive models: (1) a bilinear model that can be viewed as a simplification of a one-layer transformer; (2) one-layer transformers under certain assumptions. Our analysis reveals that for both models, the reversal curse is a consequence of the (effective) model weights *asymmetry*, i.e., the increase of weights from a token $A$ to token $B$ during training does not necessarily cause the increase of the weights from $B$ to $A$, which is caused by the training dynamics under certain choice of loss function and the optimization space of model parameters. Moreover, our analysis can be naturally applied to other logical reasoning tasks such as chain-of-thought (COT), which provides a new perspective different from previous work that focuses on expressivity. Finally, we conduct experiments to validate our theory on multi-layer transformers under different settings. Our code is available at [https:\u002F\u002Fgithub.com\u002Fmarlo-z\u002Freversal_curse_analysis\u002F](https:\u002F\u002Fgithub.com\u002Fmarlo-z\u002Freversal_curse_analysis\u002F).","authors":[{"id":"6551c204c2bd4f93db2191d9","name":"Hanlin Zhu","org":"PhD student, Electrical Engineering & Computer Science Department, University of California Berkeley"},{"id":"5434d49ddabfaebb1c198bec","name":"Baihe Huang","org":"PhD student, University of California, Berkeley"},{"name":"Shaolun Zhang","org":"Undergrad student, University of California, Berkeley"},{"id":"53f48cfedabfaea7cd1d0d8e","name":"Michael I. Jordan","org":"Full Professor, University of California, Berkeley"},{"id":"53f43b2edabfaee2a1d14f03","name":"Jiantao Jiao","org":"Assistant Professor, University of California Berkeley"},{"id":"5630a1b745cedb3399aef865","name":"Yuandong Tian","org":"Research Scientist, Meta AI (FAIR)"},{"id":"53f46e42dabfaedf43664803","name":"Stuart Russell","org":"Full Professor, University of California Berkeley"}],"create_time":"2024-07-05T03:25:03.456Z","id":"663c2e4901d2a3fbfce79e6a","keywords":["logical reasoning","reversal curse","training dynamics","autoregressive transformer","chain of thought"],"lang":"en","num_citation":0,"pdf":"https:\u002F\u002Fcz5waila03cyo0tux1owpyofgoryroob.aminer.cn\u002F6C\u002F13\u002F0E\u002F6C130E2B99E4DE193AD1166CFBD001C0.pdf","title":"Towards a Theoretical Understanding of the 'reversal Curse' Via Training Dynamics","update_times":{"u_a_t":"2024-12-10T10:42:34Z","u_v_t":"2024-12-10T10:42:34Z"},"urls":["http:\u002F\u002Farxiv.org\u002Fabs\u002F2405.04669"],"venue":{"info":{"name":"NeurIPS 2024"}},"versions":[{"id":"663c2e4901d2a3fbfce79e6a","sid":"2405.04669","src":"arxiv","vsid":"NeurIPS.cc\u002F2024\u002FConference","year":2024},{"id":"6683b81e01d2a3fbfc8ed2a8","sid":"journals\u002Fcorr\u002Fabs-2405-04669","src":"dblp","vsid":"journals\u002Fcorr","year":2024},{"id":"6756aee0ae8580e7ffbf9f18","sid":"QoWf3lo6m7","src":"conf_neurips","vsid":"NeurIPS.cc\u002F2024\u002FConference","year":2024}],"year":2024}],"profilePubsTotal":185,"profilePatentsPage":0,"profilePatents":null,"profilePatentsTotal":null,"profilePatentsEnd":false,"profileProjectsPage":0,"profileProjects":null,"profileProjectsTotal":null,"newInfo":null,"checkDelPubs":[]}};