Researchers Build Massive AI Training Dataset to Perfect Lighting on Faces

Table of Links

• Abstract and 1. Introduction
• 2. Related Work
• 3. SwitchLight and 3.1. Preliminaries
• 3.2. Problem Formulation
• 3.3. Architecture
• 3.4. Objectives
• 4. Multi-Masked Autoencoder Pre-training
• 5. Data
• 6. Experiments
• 7. Conclusion

Appendix

• A. Implementation Details
• B. User Study Interface
• C. Video Demonstration
• D. Additional Qualitative Results & References

5. Data

We constructed the OLAT (One Light at a Time) dataset using a light stage [10, 49] equipped with 137 programmable LED lights and 7 frontal-viewing cameras. Our dataset comprises images of 287 subjects, with each subject being captured in up to 15 different poses, resulting in a total of 29,705 OLAT sequences. We sourced HDRI dataset from several publicly available archives. Specifically, we acquired 559 HDRI maps from Polyhaven, 76 from Noah Witchell, 364 from HDRMAPS, 129 from iHDRI, and 34 from eisklotz. In addition, we incorporated synthetic HDRIs created using the method proposed in [31]. During training, HDRIs are randomly selected with equal probability from either real-world or synthetic collections.

We produced training pairs by projecting the sampled source and target lighting maps onto the reflectance fields of the OLAT images [10]. To derive the ground truth intrinsics, we applied the photometric stereo method [51] and obtained normal and albedo maps.