MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation

In this paper, we present MoMA: an open-vocabulary, training-freepersonalized image model that boasts flexible zero-shot capabilities. Asfoundational text-to-image models rapidly evolve, the demand for robustimage-to-image translation grows. Addressing this need, MoMA specializes insubject-driven personalized image generation. Utilizing an open-source,Multimodal Large Language Model (MLLM), we train MoMA to serve a dual role asboth a feature extractor and a generator. This approach effectively synergizesreference image and text prompt information to produce valuable image features,facilitating an image diffusion model. To better leverage the generatedfeatures, we further introduce a novel self-attention shortcut method thatefficiently transfers image features to an image diffusion model, improving theresemblance of the target object in generated images. Remarkably, as atuning-free plug-and-play module, our model requires only a single referenceimage and outperforms existing methods in generating images with high detailfidelity, enhanced identity-preservation and prompt faithfulness. Our work isopen-source, thereby providing universal access to these advancements.