DiffSLU: Knowledge Distillation Based Diffusion Model for Cross-Lingual Spoken Language Understanding

Spoken language understanding (SLU) has achieved great success in high-resource languages, but it still remains challenging in the low-resource languages due to the scarcity of labeled training data. Hence, there is an increasing interest in zero-shot cross-lingual SLU. SLU typically has two subtasks, including intent detection and slot filling. Slots and intent in the same utterance are correlated, thus it is beneficial to achieve mutual guidance between them. In this paper, we propose a novel cross-lingual SLU framework termed DiffSLU, which leverages powerful diffusion model to enhance the mutual guidance. In addition, we also utilize knowledge distillation to facilitate knowledge transfer. Experimental results demonstrate that our DiffSLU can improve the performance compared with the strong baselines and achieves the new state-of-the-art performance on MultiATIS++ dataset, obtaining a relative improvement of 3.1% over the previous best model in overall accuracy.