Effect of Text Moderator on Text-based Jailbreak Attack

Table of Links

Abstract and 1. Introduction

2. Related Works

3. Methodology and 3.1 Preliminary

   3.2 Query-specific Visual Role-play

   3.3 Universal Visual Role-play

4. Experiments and 4.1 Experimental setups

   4.2 Main Results

   4.3 Ablation Study

   4.4 Defense Analysis

   4.5 Integrating VRP with Baseline Techniques

5. Conclusion

6. Limitation

7. Future work and References

   
   

A. Character Generation Detail

B. Ethics and Broader Impact

C. Effect of Text Moderator on Text-based Jailbreak Attack

D. Examples

E. Evaluation Detail

C Effect of Text Moderator on Text-based Jailbreak Attack

Moderator models such as Llamaguard [19] classify textual inputs as “safe” and “unsafe”. A jailbreak attack’s textual input can be detected by such a moderator model, and the attack can be directly blocked and fail. To demonstrate the effect of a text moderator on text-based jailbreak attacks, we use Llamaguard to classify the text input of JailbreakV28k [38]. We report the ASR after applying the moderator in Table 6. The ASR for all models dropped drastically to lower than 7%.

We also use Llamaguard [19] to detect textual input of VRP, which is a fixed harmless instruction. Llamaguard classify VRP as “safe“.