Limitation

Table of Links

Abstract and 1. Introduction

2. Related Works

3. Methodology and 3.1 Preliminary

   3.2 Query-specific Visual Role-play

   3.3 Universal Visual Role-play

4. Experiments and 4.1 Experimental setups

   4.2 Main Results

   4.3 Ablation Study

   4.4 Defense Analysis

   4.5 Integrating VRP with Baseline Techniques

5. Conclusion

6. Limitation

7. Future work and References

   
   

A. Character Generation Detail

B. Ethics and Broader Impact

C. Effect of Text Moderator on Text-based Jailbreak Attack

D. Examples

E. Evaluation Detail

6 Limitation

One potential limitation of our work, despite its strong performance on state-of-the-art MLLMs, lies in its effectiveness against poorly performing MLLMs. These models may lack adequate instructionfollowing and image understanding capabilities, rendering them ineffective in role-playing tasks. Another limitation is our approach for generating character prompts for the diffusion model, which relies on direct generation by a LLM. This method, while effective and straightforward, may be constrained by the LLM’s ability to produce effective diffusion model prompts. Additionally, the diffusion model’s capability to generate character images from these may further limit the efficacy of our approach.