Authors:
(1) Hoon Kim, Beeble AI, and contributed equally to this work;
(2) Minje Jang, Beeble AI, and contributed equally to this work;
(3) Wonjun Yoon, Beeble AI, and contributed equally to this work;
(4) Jisoo Lee, Beeble AI, and contributed equally to this work;
(5) Donghyun Na, Beeble AI, and contributed equally to this work;
(6) Sanghyun Woo, New York University, and contributed equally to this work.
Editor's Note: This is Part 10 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below.
Table of Links
- Abstract and 1. Introduction
- 2. Related Work
- 3. SwitchLight and 3.1. Preliminaries
- 3.2. Problem Formulation
- 3.3. Architecture
- 3.4. Objectives
- 4. Multi-Masked Autoencoder Pre-training
- 5. Data
- 6. Experiments
- 7. Conclusion
Appendix
- A. Implementation Details
- B. User Study Interface
- C. Video Demonstration
- D. Additional Qualitative Results & References
7. Conclusion
We introduce SwitchLight, an architecture based on Cook-Torrance rendering physics, enhanced with a selfsupervised pre-training framework. This co-designed approach significantly outperforms previous models. Our future plans include scaling the current model beyond images to encompass video and 3D data. We hope our proposal serve as a new foundational model for relighting tasks.
This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license.