Table of Links Abstract and 1. Introduction Background and Related Work


Threat Model


Robust Style Mimicry


Experimental Setup


Results
6.1 Main Findings: All Protections are Easily Circumvented
6.2 Analysis


Discussion and Broader Impact, Acknowledgements, and References A. Detailed Art Examples B. Robust Mimicry Generations C. Detailed Results D. Differences with Glaze Finetuning E. Findings on Glaze 2.0 F. Findings on Mist v2 G. Methods for Style Mimicry H. Existing Style Mimicry Protections I. Robust Mimicry Methods J. Experimental Setup K. User Study L. Compute Resources I Robust Mimicry Methods This section details the robust mimicry methods we use in our work. These methods are not aimed at maximizing performance. Instead, they demonstrate how various ”off-the-shelf” and low-effort techniques can significantly weaken style mimicry protections. I.1 DiffPure If the text-to-image model M supports unconditional image generation, then we can use model M for the reverse diffusion process. For example, Stable Diffusion (Rombach et al., 2022) generates images unconditionally when the prompt P equals the empty string. Under these conditions, Img2Img is equivalent to DiffPure. Therefore, in the context of defenses for style mimicry, we refer to Img2Img applied with an empty prompt P as unconditional DiffPure, and to Img2Img applied with a non-empty prompt P as conditional DiffPure. I.2 Noisy Upscaling Upscaling increases the resolution of an image by predicting new pixels that enhance the level of detail. Upscaling images can purify adversarially perturbed images (Mustafa et al., 2019). However, we discover that applying upscaling directly on protected images fails to remove the perturbations. Second, we note that upscaling has shown success against adversarial perturbations for classifiers (Mustafa et al., 2019), but not against adversarial perturbations for generative models (Liang et al., 2023; Shan et al., 2023a). I.3 IMPRESS++ We enhance the IMPRESS algorithm (Cao et al., 2024). We change the loss of the reverse encoding optimization from patch similarity to l∞ and include two additional steps: negative prompting and post-processing. All in all, IMPRESS++ first preprocesses protected images with Gaussian noise and reverse encoder optimization, then samples using negative prompting and finally post-processes the generated images with DiffPure to remove noise. Figure 27 illustrates the improvements introduced by each additional step. Authors:
(1) Robert Honig, ETH Zurich (robert.hoenig@inf.ethz.ch);
(2) Javier Rando, ETH Zurich (javier.rando@inf.ethz.ch);
(3) Nicholas Carlini, Google DeepMind;
(4) Florian Tramer, ETH Zurich (florian.tramer@inf.ethz.ch). This paper is available on arxiv under CC BY 4.0 license. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Background and Related Work Threat Model Robust Style Mimicry Experimental Setup Results
6.1 Main Findings: All Protections are Easily Circumvented
6.2 Analysis Discussion and Broader Impact, Acknowledgements, and References Background and Related Work Background and Related Work Background and Related Work Threat Model Threat Model Threat Model Robust Style Mimicry Robust Style Mimicry Robust Style Mimicry Experimental Setup Experimental Setup Experimental Setup Results 6.1 Main Findings: All Protections are Easily Circumvented 6.2 Analysis Results Results 6.1 Main Findings: All Protections are Easily Circumvented 6.1 Main Findings: All Protections are Easily Circumvented 6.2 Analysis 6.2 Analysis Discussion and Broader Impact, Acknowledgements, and References Discussion and Broader Impact, Acknowledgements, and References Discussion and Broader Impact, Acknowledgements, and References A. Detailed Art Examples A. Detailed Art Examples B. Robust Mimicry Generations B. Robust Mimicry Generations C. Detailed Results C. Detailed Results D. Differences with Glaze Finetuning D. Differences with Glaze Finetuning E. Findings on Glaze 2.0 E. Findings on Glaze 2.0 F. Findings on Mist v2 F. Findings on Mist v2 G. Methods for Style Mimicry G. Methods for Style Mimicry H. Existing Style Mimicry Protections H. Existing Style Mimicry Protections I. Robust Mimicry Methods I. Robust Mimicry Methods J. Experimental Setup J. Experimental Setup K. User Study K. User Study L. Compute Resources L. Compute Resources I Robust Mimicry Methods This section details the robust mimicry methods we use in our work. These methods are not aimed at maximizing performance. Instead, they demonstrate how various ”off-the-shelf” and low-effort techniques can significantly weaken style mimicry protections. I.1 DiffPure If the text-to-image model M supports unconditional image generation, then we can use model M for the reverse diffusion process. For example, Stable Diffusion (Rombach et al., 2022) generates images unconditionally when the prompt P equals the empty string. Under these conditions, Img2Img is equivalent to DiffPure. Therefore, in the context of defenses for style mimicry, we refer to Img2Img applied with an empty prompt P as unconditional DiffPure, and to Img2Img applied with a non-empty prompt P as conditional DiffPure. I.2 Noisy Upscaling Upscaling increases the resolution of an image by predicting new pixels that enhance the level of detail. Upscaling images can purify adversarially perturbed images (Mustafa et al., 2019). However, we discover that applying upscaling directly on protected images fails to remove the perturbations. Second, we note that upscaling has shown success against adversarial perturbations for classifiers (Mustafa et al., 2019), but not against adversarial perturbations for generative models (Liang et al., 2023; Shan et al., 2023a). I.3 IMPRESS++ We enhance the IMPRESS algorithm (Cao et al., 2024). We change the loss of the reverse encoding optimization from patch similarity to l∞ and include two additional steps: negative prompting and post-processing. All in all, IMPRESS++ first preprocesses protected images with Gaussian noise and reverse encoder optimization, then samples using negative prompting and finally post-processes the generated images with DiffPure to remove noise. Figure 27 illustrates the improvements introduced by each additional step. Authors: (1) Robert Honig, ETH Zurich (robert.hoenig@inf.ethz.ch); (2) Javier Rando, ETH Zurich (javier.rando@inf.ethz.ch); (3) Nicholas Carlini, Google DeepMind; (4) Florian Tramer, ETH Zurich (florian.tramer@inf.ethz.ch). Authors: Authors: (1) Robert Honig, ETH Zurich (robert.hoenig@inf.ethz.ch); (2) Javier Rando, ETH Zurich (javier.rando@inf.ethz.ch); (3) Nicholas Carlini, Google DeepMind; (4) Florian Tramer, ETH Zurich (florian.tramer@inf.ethz.ch). This paper is available on arxiv under CC BY 4.0 license. This paper is available on arxiv under CC BY 4.0 license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

How We Evaluated Different AI Techniques for Mimicking Artistic Styles

How Are Artists Protecting Their Unique Styles from Imitation in AI Art?

Read My Stories

Too Long; Didn't Read

Get Free Unlimited IP Geo Data With IPinfo Lite

Why AI Art Protections Aren’t as Strong as They Seem

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Art Protection Tools Fail Against Advanced AI Mimicry Methods

New Research Reveals Vulnerabilities in Popular Art Protection Tools Against AI Theft

Why AI Style Protections Fall Short Against Advanced Mimicry Techniques

How AI Forgers Bypass Style Protections to Mimic Artists' Work

New Study Shows AI Can Now Mimic Art Styles More Accurately Than Ever

Can AI Mimic Famous Art Styles Despite Protective Measures?

Art Protection Tools Fail Against Advanced AI Mimicry Methods

New Research Reveals Vulnerabilities in Popular Art Protection Tools Against AI Theft

Why AI Style Protections Fall Short Against Advanced Mimicry Techniques

How AI Forgers Bypass Style Protections to Mimic Artists' Work

New Study Shows AI Can Now Mimic Art Styles More Accurately Than Ever

Can AI Mimic Famous Art Styles Despite Protective Measures?

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps