Authors:
(1) Xueying Mao, School of Computer Science, Fudan University, China (xymao22@m.@fudan.edu.cn);
(2) Xiaoxiao Hu, School of Computer Science, Fudan University, China (xxhu23@m.fudan.edu.cn);
(3) Wanli Peng, School of Computer Science, Fudan University, China (pengwanli@fudan.edu.cn);
(4) Zhenliang Gan, School of Computer Science, Fudan University, China (zlgan23@m.@fudan.edu.cn);
(5) Qichao Ying, School of Computer Science, Fudan University, China (qcying20@fudan.edu.cn);
(6) Zhenxing Qian, School of Computer Science, Fudan University, China and a Corresponding Author (zxqian@fudan.edu.cn);
(7) Sheng Li, School of Computer Science, Fudan University, China (lisheng@fudan.edu.cn);
(8) Xinpeng Zhang, School of Computer Science, Fudan University, China (zhangxinpeng@fudan.edu.cn). Editor's note: This is Part 3 of 7 of a study describing the development of a new method to hide secret messages in semantic features of videos, making it more secure and resistant to distortion during online sharing. Read the rest below. Table of Links Abstract and 1. Introduction
2. Related Work
3. Proposed Approach


Experiments


4.1. Experimental Setups
4.2. Performance Analysis
4.3. Ablation Study
Conclusions and References 3. PROPOSED APPROACH 3.1. Information Encoding Module 3.2. Secret Message Embedding and Extraction Module This module aims to embed the secret message during face swapping. The key problem is how to implement face swapping under the guidance of secret message. To our understanding, the latent features of the cover video encompass both identity and attribute feature. Face swapping essentially involves replacing the cover video’s identity with that of the reference image. Consequently, we embed the secret message into the identity feature of the reference image, formulated as follows: where λ is a hyper-parameter adjusting the influence of secret message on identity feature. 3.3. Attacking Layer To bolster the robustness of our method for face-swapping videos in real-world scenarios, we design a attacking layer. This module simulates prevalent distortions encountered across social network platforms. JPEG Compression. JPEG compression involves a nondifferentiable quantization step due to rounding. To mitigate this, we apply Shin et al.’s method [53] to approximate the near-zero quantization step using function Eq. (6): where x denotes pixels of the input image. We uniformly sample the JPEG quality from within the range of [50, 100]. Color Distortions. We consider two general color distortions: brightness and contrast. We perform a linear transformation on the pixels of each channel as the formula Eq. (7): where p(x) and f(x) refers to the distorted and the original image. The parameters a and c regulate contrast and brightness, respectively. Color Saturation. We perform random linear interpolation between RGB and gray images equivalent to simulate the distortion. Additive Noise. We use Gaussian noise to simulate any other distortions that are not considered in the attacking layer. We employ a Gaussian noise model (sampling the standard deviation δ ∼ U[0, 0.2]) to simulate imaging noise. 3.4. Loss Function The proposed method ensures both high stego video quality and precise extraction of secret message. We achieve this by training the modules using the following losses. Attribute Loss. We use the weak feature matching loss [26] to constrain attribute difference before and after embedding secret message. The loss function is defined as follows: where Dj refers to the feature extractor of Discriminator D for the j-th layer, Nj is the number of elements in the j-th layer, and H is the total number of layers. Additionally, h represents the starting layer for computing the weak feature matching loss. Adversarial Loss. To enhance performance, we use multiscale Discriminator with gradient penalty. We adopt the Hinge version of adversarial loss defined as follows: Secret Loss. To address this, we use the Binary Cross-Entropy loss (BCE) as defined in Eq. (11). Total loss. The total loss is defined as follows: This paper is available on arxiv under CC 4.0 license. Authors: (1) Xueying Mao, School of Computer Science, Fudan University, China (xymao22@m.@fudan.edu.cn); (2) Xiaoxiao Hu, School of Computer Science, Fudan University, China (xxhu23@m.fudan.edu.cn); (3) Wanli Peng, School of Computer Science, Fudan University, China (pengwanli@fudan.edu.cn); (4) Zhenliang Gan, School of Computer Science, Fudan University, China (zlgan23@m.@fudan.edu.cn); (5) Qichao Ying, School of Computer Science, Fudan University, China (qcying20@fudan.edu.cn); (6) Zhenxing Qian, School of Computer Science, Fudan University, China and a Corresponding Author (zxqian@fudan.edu.cn); (7) Sheng Li, School of Computer Science, Fudan University, China (lisheng@fudan.edu.cn); (8) Xinpeng Zhang, School of Computer Science, Fudan University, China (zhangxinpeng@fudan.edu.cn). Authors: Authors: (1) Xueying Mao, School of Computer Science, Fudan University, China (xymao22@m.@fudan.edu.cn); (2) Xiaoxiao Hu, School of Computer Science, Fudan University, China (xxhu23@m.fudan.edu.cn); (3) Wanli Peng, School of Computer Science, Fudan University, China (pengwanli@fudan.edu.cn); (4) Zhenliang Gan, School of Computer Science, Fudan University, China (zlgan23@m.@fudan.edu.cn); (5) Qichao Ying, School of Computer Science, Fudan University, China (qcying20@fudan.edu.cn); (6) Zhenxing Qian, School of Computer Science, Fudan University, China and a Corresponding Author (zxqian@fudan.edu.cn); (7) Sheng Li, School of Computer Science, Fudan University, China (lisheng@fudan.edu.cn); (8) Xinpeng Zhang, School of Computer Science, Fudan University, China (zhangxinpeng@fudan.edu.cn). Editor's note: This is Part 3 of 7 of a study describing the development of a new method to hide secret messages in semantic features of videos, making it more secure and resistant to distortion during online sharing. Read the rest below. Editor's note: This is Part 3 of 7 of a study describing the development of a new method to hide secret messages in semantic features of videos, making it more secure and resistant to distortion during online sharing. Read the rest below. Editor's note: This is Part 3 of 7 of a study describing the development of a new method to hide secret messages in semantic features of videos, making it more secure and resistant to distortion during online sharing. Read the rest below. Editor's note: This is Part 3 of 7 of a study describing the development of a new method to hide secret messages in semantic features of videos, making it more secure and resistant to distortion during online sharing. Read the rest below. Table of Links Abstract and 1. Introduction 2. Related Work 3. Proposed Approach Experiments 4.1. Experimental Setups 4.2. Performance Analysis 4.3. Ablation Study Conclusions and References Abstract and 1. Introduction Abstract and 1. Introduction 2. Related Work 2. Related Work 3. Proposed Approach 3. Proposed Approach Experiments Experiments Experiments 4.1. Experimental Setups 4.1. Experimental Setups 4.2. Performance Analysis 4.2. Performance Analysis 4.3. Ablation Study 4.3. Ablation Study Conclusions and References Conclusions and References 3. PROPOSED APPROACH 3.1. Information Encoding Module 3.2. Secret Message Embedding and Extraction Module This module aims to embed the secret message during face swapping. The key problem is how to implement face swapping under the guidance of secret message. To our understanding, the latent features of the cover video encompass both identity and attribute feature. Face swapping essentially involves replacing the cover video’s identity with that of the reference image. Consequently, we embed the secret message into the identity feature of the reference image, formulated as follows: where λ is a hyper-parameter adjusting the influence of secret message on identity feature. 3.3. Attacking Layer To bolster the robustness of our method for face-swapping videos in real-world scenarios, we design a attacking layer. This module simulates prevalent distortions encountered across social network platforms. JPEG Compression. JPEG compression involves a nondifferentiable quantization step due to rounding. To mitigate this, we apply Shin et al.’s method [53] to approximate the near-zero quantization step using function Eq. (6): JPEG Compression. where x denotes pixels of the input image. We uniformly sample the JPEG quality from within the range of [50, 100]. Color Distortions. We consider two general color distortions: brightness and contrast. We perform a linear transformation on the pixels of each channel as the formula Eq. (7): Color Distortions. where p(x) and f(x) refers to the distorted and the original image. The parameters a and c regulate contrast and brightness, respectively. Color Saturation. We perform random linear interpolation between RGB and gray images equivalent to simulate the distortion. Color Saturation. Additive Noise. We use Gaussian noise to simulate any other distortions that are not considered in the attacking layer. We employ a Gaussian noise model (sampling the standard deviation δ ∼ U[0, 0.2]) to simulate imaging noise. Additive Noise. 3.4. Loss Function The proposed method ensures both high stego video quality and precise extraction of secret message. We achieve this by training the modules using the following losses. Attribute Loss. We use the weak feature matching loss [26] to constrain attribute difference before and after embedding secret message. The loss function is defined as follows: Attribute Loss. where Dj refers to the feature extractor of Discriminator D for the j-th layer, Nj is the number of elements in the j-th layer, and H is the total number of layers. Additionally, h represents the starting layer for computing the weak feature matching loss. Dj Nj H h Adversarial Loss. To enhance performance, we use multiscale Discriminator with gradient penalty. We adopt the Hinge version of adversarial loss defined as follows: Adversarial Loss. Secret Loss. To address this, we use the Binary Cross-Entropy loss (BCE) as defined in Eq. (11). Secret Loss. Total loss. The total loss is defined as follows: Total loss. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Researchers Test Secret Message Embedding in Videos Using AI

Chinese Researchers Embed Secret Messages in Videos That Survive Social Media Distortions

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A Reference List to Learn More About Image Editing, Video Editing, and Diffusion Models

The Noonification: Use This 7-Step McKinsey Framework to Solve Any Problem (1/10/2023)

The Noonification: A Taxonomy of Inclusiveness (1/11/2024)

The Noonification: What is the InfiniteNature-Zero AI Model? (11/19/2022)

10 Ways AI Has Changed Our Lives

A Reference List to Learn More About Image Editing, Video Editing, and Diffusion Models

The Noonification: Use This 7-Step McKinsey Framework to Solve Any Problem (1/10/2023)

The Noonification: A Taxonomy of Inclusiveness (1/11/2024)

The Noonification: What is the InfiniteNature-Zero AI Model? (11/19/2022)

10 Ways AI Has Changed Our Lives

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps