Understanding the Challenges of Data Management in the AI Application Development

With the development of AI model ecology pioneered by the large language models, data plays a more important and diversified role than the traditional one. In recent years, data problems have been increasingly noticed and voiced by developers, and it is crucial to understand the existing challenges that they are facing in reality, as well as the potential features of these problems. Unfortunately, we currently have relatively little knowledge in this area. To fill this gap, we conduct an empirical study based on 2,154 posts and 785 issues by collecting model data problems from three online communities (i.e., GitHub, Stack Overflow, and Hugging Face). We present the first taxonomy of data problems around model production encountered by developers, covering 11 topics in 4 categories. Then, we analyze the popularity and the difficulty of the posts and issues in the derived topics. We distill many findings from the study and present the practical implications from the perspectives of different audiences.