Take a Step Back: Evoking Reasoning Via Abstraction in Large Language Models

We present Step-Back Prompting, a simple prompting technique that enablesLLMs to do abstractions to derive high-level concepts and first principles frominstances containing specific details. Using the concepts and principles toguide reasoning, LLMs significantly improve their abilities in following acorrect reasoning path towards the solution. We conduct experiments ofStep-Back Prompting with PaLM-2L, GPT-4 and Llama2-70B models, and observesubstantial performance gains on various challenging reasoning-intensive tasksincluding STEM, Knowledge QA, and Multi-Hop Reasoning. For instance, Step-BackPrompting improves PaLM-2L performance on MMLU (Physics and Chemistry) by 7and 11