BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses

Pre-trained language models have been successful in many scenarios. However,their usefulness in task-oriented dialogues is limited due to the intrinsiclinguistic differences between general text and task-oriented dialogues.Current task-oriented dialogue pre-training methods rely on a contrastiveframework, which faces challenges such as selecting true positives and hardnegatives, as well as lacking diversity. In this paper, we propose a noveldialogue pre-training model called BootTOD. It learns task-oriented dialoguerepresentations via a self-bootstrapping framework. Unlike contrastivecounterparts, BootTOD aligns context and context+response representations anddismisses the requirements of contrastive pairs. BootTOD also uses multipleappropriate response targets to model the intrinsic one-to-many diversity ofhuman conversations. Experimental results show that BootTOD outperforms strongTOD baselines on diverse downstream dialogue tasks.