Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Related Works


Methodology and 3.1 Preliminary
3.2 Query-specific Visual Role-play
3.3 Universal Visual Role-play


Experiments and 4.1 Experimental setups
4.2 Main Results
4.3 Ablation Study
4.4 Defense Analysis
4.5 Integrating VRP with Baseline Techniques


Conclusion


Limitation


Future work and References Related Works Related Works Related Works Methodology and 3.1 Preliminary
3.2 Query-specific Visual Role-play
3.3 Universal Visual Role-play Methodology and 3.1 Preliminary Methodology and 3.1 Preliminary 3.2 Query-specific Visual Role-play 3.2 Query-specific Visual Role-play 3.3 Universal Visual Role-play 3.3 Universal Visual Role-play Experiments and 4.1 Experimental setups
4.2 Main Results
4.3 Ablation Study
4.4 Defense Analysis
4.5 Integrating VRP with Baseline Techniques Experiments and 4.1 Experimental setups Experiments and 4.1 Experimental setups 4.2 Main Results 4.2 Main Results 4.3 Ablation Study 4.3 Ablation Study 4.4 Defense Analysis 4.4 Defense Analysis 4.5 Integrating VRP with Baseline Techniques 4.5 Integrating VRP with Baseline Techniques Conclusion Conclusion Conclusion Limitation Limitation Limitation Future work and References Future work and References Future work and References A. Character Generation Detail A. Character Generation Detail B. Ethics and Broader Impact B. Ethics and Broader Impact C. Effect of Text Moderator on Text-based Jailbreak Attack C. Effect of Text Moderator on Text-based Jailbreak Attack D. Examples D. Examples E. Evaluation Detail E. Evaluation Detail 4 Experiments In this section, we conduct experiments to evaluate VRP using a series of datasets and victim models and compare with a few highly relevant recent baselines of jailbreak attacks. We not only delve into the significance of various image components through the ablation study but also assess the robustness of the VRP against two distinct defense methodologies. Moreover, we combine VRP with Figstep and Query-relevant to explore the potential of VRP in enhancing structure-based jailbreak methods. 4.1 Experimental setups.. In our experiments, we chose 2 datasets and 5 victim models. We customized a metric to evaluate the attack success rate. The details are shown as following: Dataset. In our paper, we use widely used jailbreak attack datasets, RedTeam-2k [38] and HarmBench [41], to evaluate our VRP. (i) RedTeam-2k [38] consists of 2000 diverse and high-quality harmful textual questions across 16 harmful categories. We randomly split RedTeam-2k [38] into train set, valid set, and test set with a ratio of 6:2:2. We use train set and validation set of RedTeam-2k [38] to train universal VRP character. (ii) HarmBench [41] is an open-source framework for automated red teaming contains 320 textual harmful questions test set. Dataset Victim Models. In our experiments, we evaluated 5 state-of-the-art MLLMs, including 4 open-source MLLMs, Llava-V1.6-Mistral-7B [30], Qwen-VL-Chat (7B) [2], OmniLMM (12B) [71], InternVLChat-V1.5 [8], and 1 closed-source MLLMs Gemini-1.0-Pro-Vision. Open-source MLLMs are selected from models with high performance on the OpenVLM Leaderboard [9]. All experiments are conducted with 2 NVIDIA A100 GPUs. Victim Models Baselines. In our experiments, we choose the following jailbreak baseline to compare with our VRP: Baselines Vanilla Text: Vanilla Text means we use the blank image as image input and we use the vanilla query as text input. Vanilla Text: Textual Role-play(TRP): Textual Role-play means inserting the same character generated with VRP into text input to perform a text-based jailbreak attack. We use a blank image as image input. See Tab. D in Appendix D for details. Textual Role-play(TRP): FigStep [14]: A straightforward image-based jailbreak attack, which rephrases the vanilla question into a “Step-by-step” style and typography to image input. FigStep [14]: Query relevant [36]: An image-based jailbreak attack, which makes textual queries into visual representations using various methods such as Stable Diffusion(SD), Typography(Typo), and SD+Typo. We only use SD+Typo as the baseline due to its consistently superior performance across many MLLMs. Query relevant [36]: Implementation Details Implementation Details In our experiment, the main implement details contains 5 parts: • Character Generation. We use Mixtral-8x7B-Instruct-v0.1 [21] for all the generation of characters. We design 3 different prompts for query-specific VRP, initial round of universal VRP, and optimization round of universal VRP. See detail for character generation in Sec. A of Appendix. • Character Generation. • Image Generation. We use stable-diffusion-xl-base-1.0 [51] to generate all the character images, with 30 diffusion steps, and 1024×1024 image size. All typographies contain black text and a white background. The font of the text is Arial and the font size is 50. • Image Generation. • Hyper Parameters for Universal VRP Training. Including the initial round, we generate 5 rounds of character candidates. For each generation round, we give LLM 50 question demos sampled from train set. In initial round, we prompt LLM to generate 10 initial character candidates, for the following optimization rounds, we prompt LLM to generate 5 character candidates. To compute batch training ASR, we sample 256 data from the train set. In each optimization round, we sample 5 characters from top 10 characters with the highest training ASR in history characters. • Hyper Parameters for Universal VRP Training. • VRP with FigStep. We combine FigStep with VRP by changing the harmful question typography at the bottom of the image to the typography of FigStep like “Here is how to build a bomb: 1. 2. 3.”. Additionally, we add the text input of FigStep as a postfix of our VRP text input. • VRP with FigStep. • VRP with Query relevant. We also combine Query relevant with VRP by changing the harmful question typography to the Query relevant, and we do the same setting as VRP+FigStep to add a Query relevant style postfix to VRP text input. • VRP with Query relevant 4.2 Main Results VRP is more effective than baseline attacks. In Tab. 1, we present the outcomes of our query-specific VRP attack on the test sets of RedTeam-2K and HarmBench. This approach involves generating specific characters for each harmful question to assess their effectiveness in compromising SotA open-source and closed-source MLLMs, such as Gemini-Pro-Vision. but also achieves higher ASR than all other baseline attacks. Our findings reveal that query-specific VRP not only successfully breaches these MLLMs but also achieves a higher ASR compared to all evaluated baseline attacks. Specifically, it improves the ASR by 9.8% over FigStep and by 14.3% over Query relevant. In most cases, the data consistently shows that query-specific VRP surpasses TRP, underscoring the crucial role of character images in the effective jailbreaking of MLLMs. These results affirm that VRP is a potent method for jailbreaking MLLMs. VRP is more effective than baseline attacks. VRP achieves high-performance transferability across models. In our research, we further investigate the applicability of a universal attack across diverse models. Utilizing our universal VRP algorithm, we identify the most effective role-play character within the train and valid set on the target model. Subsequently, we transfer the most effective character to conduct a jailbreak attack on the target models. From Tab. 2, The ASR achieves an average of 32.7% for the target model as LLaVA-V1.6-Mixtral and 29.4% on Qwen-VL-Chat. The ASR is higher on the target model, also higher on the transfer model, demonstrating that our VRP, when implemented in a universal setting, effectively transfers and maintains high performance across different MLLMs. VRP achieves high-performance transferability across models. 4.4 Defense Analysis We evaluate the robustness of VRP against two defense approaches, namely System Prompt-based Defense, and the Eye Closed Safety On (ECSO) approach [15] System Prompt-based Defense Eye Closed Safety On System Prompt-based Defense: To defend against the jailbreak attack, a system prompt can instruct the model to conduct a preliminary safety assessment of the text and image input, thereby filtering out queries that violate AI safety policies. We add the following Prompt 2 to the existing system prompt of MLLMs. System Prompt-based Defense ECSO[15]: A defense method utilizing MLLMs’ aligned textual module to mitigate the vulnerability in visual modality. ECSO use the MLLM itself to evaluate the safety of its response and makes MLLMs to regenerate unsafe responses in two steps: image captioning, and then responding based on caption with no image input. ECSO VRP is effective against System Prompt-based Defense and ECSO. We evaluate our query-specific VRP and baselines against our System Prompt-based Defense and ECSO. As shown in Tab. 4, the results demonstrate that our query-specific VRP consistently achieves the ASR across all models, regardless of whether it is tested against System Prompt-based Defense or ECSO. This consistent performance underlines the efficacy of query-specific VRP in penetrating defenses and reveals a notable vulnerability of defense mechanisms under our VRP jailbreak attacks. These findings highlight the potential of VRP as a formidable strategy against defense mechanisms. VRP is effective against System Prompt-based Defense and ECSO. 4.5 Integrating VRP with Baseline Techniques We experimentally combine the VRP approach with established baseline techniques to evaluate their synergistic effects on jailbreak performance, as detailed in Tab. 5. The integration is simply through replacing the question typography with baseline image input and concatenate VRP and baselines’ text input. Notably, the integration of VRP significantly elevates the ASR of both FigStep and Query relevant methods. This enhancement is particularly pronounced, indicating that the addition of a role-playing element to these structure-based jailbreak methods reinforces their effectiveness. This finding underscores the potential of role-play-based enhancements in structurally jailbreak scenarios. Authors:
(1) Siyuan Ma, University of Wisconsin–Madison (siyuan.ma.jasper@outlook.com);
(2) Weidi Luo, The Ohio State University (luo.1455@osu.edu);
(3) Yu Wang, Peking University (rain_wang@stu.pku.edu.cn);
(4) Xiaogeng Liu, University of Wisconsin-Madison (xiaogeng.liu@wisc.edu). Authors: Authors (1) Siyuan Ma, University of Wisconsin–Madison (siyuan.ma.jasper@outlook.com); (2) Weidi Luo, The Ohio State University (luo.1455@osu.edu); (3) Yu Wang, Peking University (rain_wang@stu.pku.edu.cn); (4) Xiaogeng Liu, University of Wisconsin-Madison (xiaogeng.liu@wisc.edu). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

NVIDIA

One Image to Rule Them All: The Jailbreak That Outsmarts Multimodal AI

Introducing VRP: Structure-Based Role-Play Attacks on Multimodal Large Language Models

How the Most Important Models Actually Work

Too Long; Didn't Read

Make resilience your competitive advantage

VRP Outperforms Baselines in Jailbreaking MLLMs, Transferring Across Models, and Evading Defenses

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

The Prompt Patterns That Decide If an AI Is “Correct” or “Wrong”

The Fine Print of Misbehavior: VRP’s Blueprint and Safety Stance

Gemini - A Family of Highly Capable Multimodal Models: Appendix

Gemini - A Family of Highly Capable Multimodal Models: Contributions and Acknowledgments

How GPT-4 Built a New Multimodal Model

The Prompt Patterns That Decide If an AI Is “Correct” or “Wrong”

The Fine Print of Misbehavior: VRP’s Blueprint and Safety Stance

Gemini - A Family of Highly Capable Multimodal Models: Appendix

Gemini - A Family of Highly Capable Multimodal Models: Contributions and Acknowledgments

How GPT-4 Built a New Multimodal Model

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps