AbsInstruct: Eliciting Abstraction Ability from LLMs Through Explanation Tuning with Plausibility Estimation

Abstraction ability is crucial in human intelligence, which can also benefitvarious tasks in NLP study. Existing work shows that LLMs are deficient inabstract ability, and how to improve it remains unexplored. In this work, wedesign the framework AbsInstruct to enhance LLMs' abstraction ability throughinstruction tuning. The framework builds instructions with in-depthexplanations to assist LLMs in capturing the underlying rationale ofabstraction. Meanwhile, we introduce a plausibility estimator to selectinstructions that are more consistent with the abstraction knowledge of LLMs tobe aligned. Then, our framework combines abstraction instructions withgeneral-purpose ones to build a hybrid dataset. Extensive experiments andanalyses demonstrate that our framework can considerably enhance LLMs'abstraction ability with strong generalization performance while maintainingtheir general instruction-following abilities.