Table of Links Abstract and 1. Introduction Background and Related Work


Threat Model


Robust Style Mimicry


Experimental Setup


Results
6.1 Main Findings: All Protections are Easily Circumvented
6.2 Analysis


Discussion and Broader Impact, Acknowledgements, and References A. Detailed Art Examples B. Robust Mimicry Generations C. Detailed Results D. Differences with Glaze Finetuning E. Findings on Glaze 2.0 F. Findings on Mist v2 G. Methods for Style Mimicry H. Existing Style Mimicry Protections I. Robust Mimicry Methods J. Experimental Setup K. User Study L. Compute Resources 3 Threat Model The goal of style mimicry is to produce images, of some chosen content, that mimic the style of a targeted artist. Since artistic style is challenging to formalize or quantify, we refrain from doing so and define a mimicry attempt as successful if it generates new images that a human observer would qualify as possessing the artist’s style. We assume two parties, the artist who places art online (e.g., in their portfolio), and a forger who performs style mimicry using these images. The challenge for the forger is that the artist first protects their original art collection before releasing it online, using a state-of-the-art protection tool such as Glaze, Mist or Anti-DreamBooth. We make the conservative assumption that all the artist’s images available online are protected. If a mimicry method succeeds in this setting, we call it robust. In this work, we consider style forgers who finetune a text-to-image model on an artist’s images—the most successful style mimicry method to date (Shan et al., 2023a). Specifically, the forger finetunes a pretrained model f on protected images X from the artist to obtain a finetuned model ˆf. The forger has full control over the protected images and finetuning process, and can arbitrarily modify to maximize the mimicry success. Our robust mimicry methods combine a number of “off-the-shelf” manipulations that allow even low-skilled parties to bypass existing style mimicry protections. In fact, our most successful methods require only black-box access to a finetuning API for the model f, and could thus also be applied to proprietary text-to-image models that expose such an interface. Authors:
(1) Robert Honig, ETH Zurich (robert.hoenig@inf.ethz.ch);
(2) Javier Rando, ETH Zurich (javier.rando@inf.ethz.ch);
(3) Nicholas Carlini, Google DeepMind;
(4) Florian Tramer, ETH Zurich (florian.tramer@inf.ethz.ch). This paper is available on arxiv under CC BY 4.0 license. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction Background and Related Work Threat Model Robust Style Mimicry Experimental Setup Results
6.1 Main Findings: All Protections are Easily Circumvented
6.2 Analysis Discussion and Broader Impact, Acknowledgements, and References Background and Related Work Background and Related Work Background and Related Work Threat Model Threat Model Threat Model Robust Style Mimicry Robust Style Mimicry Robust Style Mimicry Experimental Setup Experimental Setup Experimental Setup Results 6.1 Main Findings: All Protections are Easily Circumvented 6.2 Analysis Results Results 6.1 Main Findings: All Protections are Easily Circumvented 6.1 Main Findings: All Protections are Easily Circumvented 6.2 Analysis 6.2 Analysis Discussion and Broader Impact, Acknowledgements, and References Discussion and Broader Impact, Acknowledgements, and References Discussion and Broader Impact, Acknowledgements, and References A. Detailed Art Examples A. Detailed Art Examples B. Robust Mimicry Generations B. Robust Mimicry Generations C. Detailed Results C. Detailed Results D. Differences with Glaze Finetuning D. Differences with Glaze Finetuning E. Findings on Glaze 2.0 E. Findings on Glaze 2.0 F. Findings on Mist v2 F. Findings on Mist v2 G. Methods for Style Mimicry G. Methods for Style Mimicry H. Existing Style Mimicry Protections H. Existing Style Mimicry Protections I. Robust Mimicry Methods I. Robust Mimicry Methods J. Experimental Setup J. Experimental Setup K. User Study K. User Study L. Compute Resources L. Compute Resources 3 Threat Model The goal of style mimicry is to produce images, of some chosen content, that mimic the style of a targeted artist. Since artistic style is challenging to formalize or quantify, we refrain from doing so and define a mimicry attempt as successful if it generates new images that a human observer would qualify as possessing the artist’s style. We assume two parties, the artist who places art online (e.g., in their portfolio), and a forger who performs style mimicry using these images. The challenge for the forger is that the artist first protects their original art collection before releasing it online, using a state-of-the-art protection tool such as Glaze, Mist or Anti-DreamBooth. We make the conservative assumption that all the artist’s images available online are protected. If a mimicry method succeeds in this setting, we call it robust. In this work, we consider style forgers who finetune a text-to-image model on an artist’s images—the most successful style mimicry method to date (Shan et al., 2023a). Specifically, the forger finetunes a pretrained model f on protected images X from the artist to obtain a finetuned model ˆf. The forger has full control over the protected images and finetuning process, and can arbitrarily modify to maximize the mimicry success. Our robust mimicry methods combine a number of “off-the-shelf” manipulations that allow even low-skilled parties to bypass existing style mimicry protections. In fact, our most successful methods require only black-box access to a finetuning API for the model f, and could thus also be applied to proprietary text-to-image models that expose such an interface. Authors: (1) Robert Honig, ETH Zurich (robert.hoenig@inf.ethz.ch); (2) Javier Rando, ETH Zurich (javier.rando@inf.ethz.ch); (3) Nicholas Carlini, Google DeepMind; (4) Florian Tramer, ETH Zurich (florian.tramer@inf.ethz.ch). Authors: Authors: (1) Robert Honig, ETH Zurich (robert.hoenig@inf.ethz.ch); (2) Javier Rando, ETH Zurich (javier.rando@inf.ethz.ch); (3) Nicholas Carlini, Google DeepMind; (4) Florian Tramer, ETH Zurich (florian.tramer@inf.ethz.ch). This paper is available on arxiv under CC BY 4.0 license. This paper is available on arxiv under CC BY 4.0 license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

How AI Forgers Bypass Style Protections to Mimic Artists' Work

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Art Protection Tools Fail Against Advanced AI Mimicry Methods

New Research Reveals Vulnerabilities in Popular Art Protection Tools Against AI Theft

Why AI Style Protections Fall Short Against Advanced Mimicry Techniques

New Study Shows AI Can Now Mimic Art Styles More Accurately Than Ever

Can AI Mimic Famous Art Styles Despite Protective Measures?

How Easy Is It for AI To Mimic a Human Artist’s Style?

Art Protection Tools Fail Against Advanced AI Mimicry Methods

New Research Reveals Vulnerabilities in Popular Art Protection Tools Against AI Theft

Why AI Style Protections Fall Short Against Advanced Mimicry Techniques

New Study Shows AI Can Now Mimic Art Styles More Accurately Than Ever

Can AI Mimic Famous Art Styles Despite Protective Measures?

How Easy Is It for AI To Mimic a Human Artist’s Style?

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps