Irving’s new Cafe Hana sits in a charming niche at the intersection of two beloved culinary worlds in Dallas: cafes and Asian food. Opened in August, the Asian-inspired spot dabbles in just about everything in the name of Asian cafes: curries,

Cafe Hana: Asian Flavors Meet Cafe Charm in Irving

NEW YORK, October 31, 2023 (Newswire.com) - JFOODO, the Japan Food Product Overseas Promotion Center established by the Japanese government, is launching its 4th annual Japanese Wagyu campaign in the United States on Nov. 1, 2023. In partnership with ...

Japanese Wagyu Campaign Redefining Everyday Dining Experiences Launches November 1st in the U.S.

Currently, Google only offers a limited version of a search product powered by generative AI – called the Search Generative Experience – in the US, India and Japan, Raghavan said, with a disclaimer that warns about the product’s limitations.

Google Walks Tightrope on AI in Search Antitrust Trial

Pioneer Consulting, the full-service submarine fiber optic telecommunications consulting and project management company, announces that it has completed deployment of the Togolese branch of Google's Equiano Cable on behalf of CSquared Woezon.

Pioneer Consulting Deploys Togolese Branch of Google's Equiano Subsea Cable for CSquared Woezon

Plus: Major vulnerability fixes are now available for a number of enterprise giants, including Cisco, VMWare, Citrix, and SAP.

Apple, Google, and Microsoft Just Patched Some Spooky Security Flaws

In a major antitrust case, Google CEO Sundar Pichai defended his companys practice of paying tech giants, including Apple, as the default search engine on their devices. Pichai stated that the intention was to enhance

Google CEO Defends $26B Payments Amid Antitrust Scrutiny, Cites User Experience Boost

Singapore, Japan, U.K., and Swiss regulators have recently announced ... ‘Buy Bitcoin’ Search Queries On Google Surge 826% In the UK In a surprising trend, ‘Buy Bitcoin’ search queries on Google have soared by a staggering 826% in the UK.

Switzerland, U.K., Japan, And Singapore Collaborating on Tokenization Of Assets.

Awkwafina starring as a game-show-obsessed woman in “Quiz Lady” and the animated historical drama “Blue Eye Samurai” about a mixed-race, revenge-seeking female samurai in Japan are some of the

What to stream this week: Annette Bening, Jason Aldean, Awkwafina, NKOTB and ‘Blue Eye Samurai’

Google has agreed to invest up to $2 billion into Anthropic, a San Francisco-based generative AI startup that recently secured $4 billion from Amazon.Why it matters: Not only does this escalate Big Tech's proxy fight over the industry's next big platform shift,

Google is investing $2 billion into Anthropic, a rival to OpenAI

Panasonic announced that it is slowing down its battery production amid weaker demand for electric vehicles, but it hints

Panasonic slows battery production amid weaker EV business, but Tesla looks good

Google Japan

google-japan

Author:
(1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com);
(2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com);
(3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com);
(4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction
Background and Related Work
Method
Results and Failure Cases
Limitations and Conclusion, Acknowledgements and References 5 LIMITATIONS AND CONCLUSION We have introduced a matte extraction approach using the deep image prior. The algorithm is simple, requiring only a few tens of lines of code modification to an existing U-net. Our approach is training-free and is thus particularly suitable for the diverse, few-ofa-kind subjects in entertainment video production. It also may be of intrinsic theoretical interest in terms of the nature and solution of the matte extraction problem. A further potential use would be to produce ground-truth mattes to be used for DL training. As is the case with many matting algorithms, it assumes coarse guidance in the form of a trimap or similar constraints. This can be created by the artist using readily available semi-automatic tools. Computational cost is the major limitation of the method, in common with classic methods [Levin et al. 2008]. Compute times for the examples shown in the paper are measured in minutes (but not hours) on a single previous generation Nvidia Volta GPU. This restricts the use of our algorithm to high-quality offline applications where extensive non-real-time computation is the norm, primarily movies and videos. On the other hand, the computation can take advantage of support for multiple GPUs provided in deep learning frameworks, and intermediate results can be visualized. Our method can produce temporally consistent matte extractions from video by warm-starting the optimization from the previous frame (see accompanying video), however in our experience this requires that the trimaps have smooth motion from frame-to-frame. A topic for future work is to consider recurrent or other network architectures that might make the trimap choice more forgiving. This paper has focused on introducing the DIP matting algorithm. There was relatively little architecture and parameter exploration, and further improvements may be possible. ACKNOWLEDGMENTS G.G. Heitmann, Peter Hillman, and Kathleen Beeler gave helpful insights and feedback. REFERENCES Adobe 2018. How to use Select Subject in Photoshop for One-Click Selections. https: //www.photoshopessentials.com Yagiz Aksoy, Tae-Hyun Oh, Sylvain Paris, Marc Pollefeys, and Wojciech Matusik. 2018. Semantic soft segmentation. ACM Trans. Graph. 37, 4 (2018). Godzilla vs. Cat 2021. https://www.youtube.com/watch?v=nf7GsKFepDg. Mikhail Erofeev, Yury Gitman, Dmitriy Vatolin, Alexey Fedorov, and Jue Wang. 2015. Perceptually Motivated Benchmark for Video Matting. In British Machine Vision Conference. Yossi Gandelsman, Assaf Shocher, and Michal Irani. 2019. "Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priors. In Comp. Vision and Pattern Recognition Kaiming He, Christoph Rhemann, Carsten Rother, Xiaoou Tang, and Jian Sun. 2011. A global sampling method for alpha matting. Comp. Vision and Pattern Recognition Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Guided Image Filtering. In European Conf. Comp. Vision. G.G. Heitman. 2020. Communication from technical artist, Weta Digital Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Int. Conf. Learning Representations. Anat Levin, Dani Lischinski, and Yair Weiss. 2008. A Closed-Form Solution to Natural Image Matting. IEEE Trans. PAMI 30, 2 (2008). Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian Curless, Steve Seitz, and Ira Kemelmacher-Shlizerman. 2020. Real-Time High-Resolution Background Matting. arXiv (2020). Y. A Mishima. 1992. Software Chromakeyer Using Polyhedric Slice. In NICOGRAPH. Christoph Rhemann, Carsten Rother, Jue Wang, Margrit Gelautz, Pushmeet Kohli, and Pamela Rott. 2009. A Perceptually Motivated Online Benchmark for Image Matting. In Comp. Vision and Pattern Recognition. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI (2015). Mike Seymour. 2020. Art of LED Wall Virtual Production. fxguide.com. Yanan Sun, Chi-Keung Tang, and Yu-Wing Tai. 2021. Semantic Image Matting. In Comp. Vision and Pattern Recognition. Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2018. Deep Image Prior. In Comp. Vision and Pattern Recognition. Jue Wang and Michael F. Cohen. 2007. Image and Video Matting: A Survey. Found. Trends Comput. Graph. Vis. 3, 2 (2007). Yong Xu, Baoling Liu, Yuhui Quan, and Hui Ji. 2022. Unsupervised Deep Background Matting Using Deep Matte Prior. IEEE Trans. Circuits Syst. Video Technol. (2022). This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Author: Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction Background and Related Work Method Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Abstract and Introduction Abstract and Introduction Background and Related Work Background and Related Work Method Method Results and Failure Cases Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Limitations and Conclusion, Acknowledgements and References 5 LIMITATIONS AND CONCLUSION We have introduced a matte extraction approach using the deep image prior. The algorithm is simple, requiring only a few tens of lines of code modification to an existing U-net. Our approach is training-free and is thus particularly suitable for the diverse, few-ofa-kind subjects in entertainment video production. It also may be of intrinsic theoretical interest in terms of the nature and solution of the matte extraction problem. A further potential use would be to produce ground-truth mattes to be used for DL training. As is the case with many matting algorithms, it assumes coarse guidance in the form of a trimap or similar constraints. This can be created by the artist using readily available semi-automatic tools. Computational cost is the major limitation of the method, in common with classic methods [Levin et al. 2008]. Compute times for the examples shown in the paper are measured in minutes (but not hours) on a single previous generation Nvidia Volta GPU. This restricts the use of our algorithm to high-quality offline applications where extensive non-real-time computation is the norm, primarily movies and videos. On the other hand, the computation can take advantage of support for multiple GPUs provided in deep learning frameworks, and intermediate results can be visualized. Our method can produce temporally consistent matte extractions from video by warm-starting the optimization from the previous frame (see accompanying video), however in our experience this requires that the trimaps have smooth motion from frame-to-frame. A topic for future work is to consider recurrent or other network architectures that might make the trimap choice more forgiving. This paper has focused on introducing the DIP matting algorithm. There was relatively little architecture and parameter exploration, and further improvements may be possible. ACKNOWLEDGMENTS G.G. Heitmann, Peter Hillman, and Kathleen Beeler gave helpful insights and feedback. REFERENCES Adobe 2018. How to use Select Subject in Photoshop for One-Click Selections. https: //www.photoshopessentials.com Yagiz Aksoy, Tae-Hyun Oh, Sylvain Paris, Marc Pollefeys, and Wojciech Matusik. 2018. Semantic soft segmentation. ACM Trans. Graph. 37, 4 (2018). Godzilla vs. Cat 2021. https://www.youtube.com/watch?v=nf7GsKFepDg. Mikhail Erofeev, Yury Gitman, Dmitriy Vatolin, Alexey Fedorov, and Jue Wang. 2015. Perceptually Motivated Benchmark for Video Matting. In British Machine Vision Conference. Yossi Gandelsman, Assaf Shocher, and Michal Irani. 2019. "Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priors. In Comp. Vision and Pattern Recognition Kaiming He, Christoph Rhemann, Carsten Rother, Xiaoou Tang, and Jian Sun. 2011. A global sampling method for alpha matting. Comp. Vision and Pattern Recognition Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Guided Image Filtering. In European Conf. Comp. Vision. G.G. Heitman. 2020. Communication from technical artist, Weta Digital Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Int. Conf. Learning Representations. Anat Levin, Dani Lischinski, and Yair Weiss. 2008. A Closed-Form Solution to Natural Image Matting. IEEE Trans. PAMI 30, 2 (2008). Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian Curless, Steve Seitz, and Ira Kemelmacher-Shlizerman. 2020. Real-Time High-Resolution Background Matting. arXiv (2020). Y. A Mishima. 1992. Software Chromakeyer Using Polyhedric Slice. In NICOGRAPH. Christoph Rhemann, Carsten Rother, Jue Wang, Margrit Gelautz, Pushmeet Kohli, and Pamela Rott. 2009. A Perceptually Motivated Online Benchmark for Image Matting. In Comp. Vision and Pattern Recognition. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI (2015). Mike Seymour. 2020. Art of LED Wall Virtual Production. fxguide.com. Yanan Sun, Chi-Keung Tang, and Yu-Wing Tai. 2021. Semantic Image Matting. In Comp. Vision and Pattern Recognition. Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2018. Deep Image Prior. In Comp. Vision and Pattern Recognition. Jue Wang and Michael F. Cohen. 2007. Image and Video Matting: A Survey. Found. Trends Comput. Graph. Vis. 3, 2 (2007). Yong Xu, Baoling Liu, Yuhui Quan, and Hui Ji. 2022. Unsupervised Deep Background Matting Using Deep Matte Prior. IEEE Trans. Circuits Syst. Video Technol. (2022). This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv

Training-Free Neural Matte Extraction for Visual Effects: Limitations and Conclusion

Author:
(1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com);
(2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com);
(3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com);
(4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction
Background and Related Work
Method
Results and Failure Cases
Limitations and Conclusion, Acknowledgements and References ABSTRACT Alpha matting is widely used in video conferencing as well as in movies, television, and social media sites. Deep learning approaches to the matte extraction problem are well suited to video conferencing due to the consistent subject matter (front-facing humans), however training-based approaches are somewhat pointless for entertainment videos where varied subjects (spaceships, monsters, etc.) may appear only a few times in a single movie – if a method of creating ground truth for training exists, just use that method to produce the desired mattes. We introduce a training-free high quality neural matte extraction approach that specifically targets the assumptions of visual effects production. Our approach is based on the deep image prior, which optimizes a deep neural network to fit a single image, thereby providing a deep encoding of the particular image. We make use of the representations in the penultimate layer to interpolate coarse and incomplete "trimap" constraints. Videos processed with this approach are temporally consistent. The algorithm is both very simple and surprisingly effective. CCS CONCEPTS • Computing methodologies → Image processing; • Applied computing → Media arts. KEYWORDS Alpha matting, deep learning, visual effects. ACM Reference Format: Sharif Elcott, J.P. Lewis, Nori Kanazawa, and Christoph Bregler. 2022. TrainingFree Neural Matte Extraction for Visual Effects. In SIGGRAPH Asia 2022 Technical Communications (SA ’22 Technical Communications), December 6–9, 2022, Daegu, Republic of Korea. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3550340.3564230 1 INTRODUCTION Alpha matte extraction refers to the underconstrained inverse problem of finding the unknown translucency or coverage 𝛼 of a foreground object [Wang and Cohen 2007]. Matte extraction is widely used to provide alternate backgrounds for video meetings, as well as to produce visual effects (VFX) for movies, television, and social media. However, it is not always recognized in the research literature that these two applications have significantly different requirements. This paper introduces a neural matte extraction method specifically addressed to the assumptions and requirements of VFX. pecifically addressed to the assumptions and requirements of VFX. Matting for video calls requires real-time performance and assumes a single class of subject matter, front-facing humans. This is a "train once and use many times" situation for which it is possible and advantageous to obtain training data. Video call matting often assumes a fixed camera, and may require a "clean plate" image of the room without the participant. On the contrary, VFX production has the following assumptions and requirements: • Diverse and often rare ("one-off") subject matter. For example the youtube video [Godzilla vs. Cat 2021] (30M views) involves mattes of a cat, ships, and wreckage. A physical prop such as a crashed alien spaceship might only be used in a few seconds in a single movie, never to be seen again. Thus, gathering a ground-truth training dataset for deep learning is often pointless: if there is a method to generate the ground-truth mattes for training, just use this method to produce the desired mattes – no need to train a model! • Visual effects frequently involves moving cameras as well as extreme diversity of moving backgrounds. For example, an actor might be filmed as they run or travel in a vehicle any place on earth, or on set with extraterrestrial props in the near background. Thus, even for the major case of repeated subject matter (humans), gathering a representative training dataset is more challenging than in the case of video calls. • Real-time performance is not required. Instead, the on-set filming time is to be minimized, due to the combined cost of the actors (sometimes with 7-figure salaries) and movie crew. It is often cheaper to employ an artist to work days to "fix it in post" rather than spend a few minutes on-set (with actors and crew waiting) to address an issue. • Clean-plates are not a desirable approach for matting, and are often not possible. With moving cameras, a motion control rig is required to obtain a clean plate. The indoor use of motion control for clean plates is expensive due to the previous principle (minimizing on-set time). Moreover, this approach is generally not feasible outdoors, for reasons including background movement (e.g. plants moved by wind) and changing lighting (moving clouds blocking the sun). Our solution, a matte extraction approach employing the deep image prior [Ulyanov et al. 2018], addresses these requirements. It has the following characteristics and contributions: • It is a deep neural network matte extraction method targeted towards the requirements of VFX rather than video meetings. To our knowledge, this is the first deep neural matte extraction method that requires neither training data nor expensive on-set image capture to support the matting process (i.e. clean plates or greenscreens). • It relies exclusively on the given image or video along with coarse "trimaps" (Fig. 1) that are easily created during post production using existing semi-automatic tools. Professional software such as Nuke or After Effects is typically used, however a familiar example is the "Select Subject" tool in Photoshop [Adobe 2018], followed by dilation/erosion and automated with "Actions". • It does not require a clean plate, thus allowing extractions from outdoor footage. • It does not require greenscreens, yet can produce detailed high quality mattes even when the foreground and background subject have similar colors (green hair in Fig. 1). This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Author: Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction Background and Related Work Method Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Abstract and Introduction Abstract and Introduction Background and Related Work Background and Related Work Method Method Results and Failure Cases Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Limitations and Conclusion, Acknowledgements and References ABSTRACT Alpha matting is widely used in video conferencing as well as in movies, television, and social media sites. Deep learning approaches to the matte extraction problem are well suited to video conferencing due to the consistent subject matter (front-facing humans), however training-based approaches are somewhat pointless for entertainment videos where varied subjects (spaceships, monsters, etc.) may appear only a few times in a single movie – if a method of creating ground truth for training exists, just use that method to produce the desired mattes. We introduce a training-free high quality neural matte extraction approach that specifically targets the assumptions of visual effects production. Our approach is based on the deep image prior, which optimizes a deep neural network to fit a single image, thereby providing a deep encoding of the particular image. We make use of the representations in the penultimate layer to interpolate coarse and incomplete "trimap" constraints. Videos processed with this approach are temporally consistent. The algorithm is both very simple and surprisingly effective. CCS CONCEPTS • Computing methodologies → Image processing; • Applied computing → Media arts. KEYWORDS Alpha matting, deep learning, visual effects. ACM Reference Format: Sharif Elcott, J.P. Lewis, Nori Kanazawa, and Christoph Bregler. 2022. TrainingFree Neural Matte Extraction for Visual Effects. In SIGGRAPH Asia 2022 Technical Communications (SA ’22 Technical Communications), December 6–9, 2022, Daegu, Republic of Korea. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3550340.3564230 1 INTRODUCTION Alpha matte extraction refers to the underconstrained inverse problem of finding the unknown translucency or coverage 𝛼 of a foreground object [Wang and Cohen 2007]. Matte extraction is widely used to provide alternate backgrounds for video meetings, as well as to produce visual effects (VFX) for movies, television, and social media. However, it is not always recognized in the research literature that these two applications have significantly different requirements. This paper introduces a neural matte extraction method specifically addressed to the assumptions and requirements of VFX. pecifically addressed to the assumptions and requirements of VFX. Matting for video calls requires real-time performance and assumes a single class of subject matter, front-facing humans. This is a "train once and use many times" situation for which it is possible and advantageous to obtain training data. Video call matting often assumes a fixed camera, and may require a "clean plate" image of the room without the participant. On the contrary, VFX production has the following assumptions and requirements: • Diverse and often rare ("one-off") subject matter. For example the youtube video [Godzilla vs. Cat 2021] (30M views) involves mattes of a cat, ships, and wreckage. A physical prop such as a crashed alien spaceship might only be used in a few seconds in a single movie, never to be seen again. Thus, gathering a ground-truth training dataset for deep learning is often pointless: if there is a method to generate the ground-truth mattes for training, just use this method to produce the desired mattes – no need to train a model! • Visual effects frequently involves moving cameras as well as extreme diversity of moving backgrounds. For example, an actor might be filmed as they run or travel in a vehicle any place on earth, or on set with extraterrestrial props in the near background. Thus, even for the major case of repeated subject matter (humans), gathering a representative training dataset is more challenging than in the case of video calls. • Real-time performance is not required. Instead, the on-set filming time is to be minimized, due to the combined cost of the actors (sometimes with 7-figure salaries) and movie crew. It is often cheaper to employ an artist to work days to "fix it in post" rather than spend a few minutes on-set (with actors and crew waiting) to address an issue. • Clean-plates are not a desirable approach for matting, and are often not possible. With moving cameras, a motion control rig is required to obtain a clean plate. The indoor use of motion control for clean plates is expensive due to the previous principle (minimizing on-set time). Moreover, this approach is generally not feasible outdoors, for reasons including background movement (e.g. plants moved by wind) and changing lighting (moving clouds blocking the sun). Our solution, a matte extraction approach employing the deep image prior [Ulyanov et al. 2018], addresses these requirements. It has the following characteristics and contributions: • It is a deep neural network matte extraction method targeted towards the requirements of VFX rather than video meetings. To our knowledge, this is the first deep neural matte extraction method that requires neither training data nor expensive on-set image capture to support the matting process (i.e. clean plates or greenscreens). • It relies exclusively on the given image or video along with coarse "trimaps" (Fig. 1) that are easily created during post production using existing semi-automatic tools. Professional software such as Nuke or After Effects is typically used, however a familiar example is the "Select Subject" tool in Photoshop [Adobe 2018], followed by dilation/erosion and automated with "Actions". • It does not require a clean plate, thus allowing extractions from outdoor footage. • It does not require greenscreens, yet can produce detailed high quality mattes even when the foreground and background subject have similar colors (green hair in Fig. 1). This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv

Training-Free Neural Matte Extraction for Visual Effects: Abstract and Introduction

Author:
(1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com);
(2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com);
(3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com);
(4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction
Background and Related Work
Method
Results and Failure Cases
Limitations and Conclusion, Acknowledgements and References 3 METHOD Our method starts with a DIP network to reconstruct the target image. A second output head is added and tasked with inpainting the desired alpha in the trimap unknown region, constrained by the values in known regions. The main idea is that the first output head forces the network representation preceding it to "understand the structure of the image," while the second output head makes use of that information in estimating the matte. In addition to the alpha output we add two additional networks which simultaneously reconstruct the foreground and background. Similarly to the first output of the first network, these networks’ outputs are constrained to match the target image but, unlike the first network, they are constrained only in their respective regions of the trimap. Similarly to the alpha output, they extrapolate those constraints to inpaint the unconstrained region (see Fig. 2). The latter three outputs – 𝛼,ˆ 𝐹,ˆ 𝐵ˆ – are coupled via an additional constraint that they together satisfy (1) (see Detailed Loss below). We put 𝐹ˆ and 𝐵ˆ in separate networks (as in [Gandelsman et al. 2019]) rather than as additional outputs heads of the first network (as in [Sun et al. 2021]) in order to allow the inpainting of the foreground to be independent of the background. All three networks share the same generic U-net structure [Ulyanov et al. 2018], except for an additional output head in the first network. Our experiments use Adam [Kingma and Ba 2015] with a learning rate of 0.001. 3.1 Detailed Loss The first term of our loss function is the reconstruction loss between the first network output and the target image: The second loss term constrains 𝛼ˆ, where 𝑇 is the trimap and 𝐶 = 𝐹 ∪ 𝐵 is the constrained region of the trimap. The reconstruction losses for the foreground and background outputs are defined similarly to 𝐿𝐼 , but constrained only in their respective regions of the trimap: The three networks’ outputs are coupled via the alpha-compositing equation as follows: where 𝑈 = 𝐼 − 𝐶 is the unconstrained region of the trimap. Finally, we include an exclusion loss similar to [Sun et al. 2021] to prevent the structure of the foreground from leaking into the background and vice-versa: The total loss is the sum of the above six components: Unlike other DIP-based techniques, our algorithm does not require early stopping since the goal is to exactly fit both the image and the trimap constraints. 3.2 Temporal continuity In our experiments temporal continuity was obtained by warmstarting the optimization for frames other than the first with the final weight values of the previous frame, and stopping with a loss threshold rather than a fixed number of iterations (see video). This simple strategy produces reasonable results even on the relatively difficult case of hair. It also reduces the compute time by roughly an order of magnitude. 3.3 Why does it work? The deep image prior empirically demonstrates that standard convolutional networks can act as a good low-level prior for image reconstruction tasks, but why can it be adapted to invent plausible alpha mattes? Our intuition is as follows: The DIP provides a somewhat deep and hierarchical encoding of the particular image. The features in this encoding will primarily span valid image structures, with noise lying outside this space. In our work these features are re-combined to produce the matte. This works because the alpha matte contains structures that relate to those in the foreground image. In this respect our approach resembles a "deep" version of the guided image filter [He et al. 2010]. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Author: Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction Background and Related Work Method Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Abstract and Introduction Abstract and Introduction Background and Related Work Background and Related Work Method Method Results and Failure Cases Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Limitations and Conclusion, Acknowledgements and References 3 METHOD Our method starts with a DIP network to reconstruct the target image. A second output head is added and tasked with inpainting the desired alpha in the trimap unknown region, constrained by the values in known regions. The main idea is that the first output head forces the network representation preceding it to "understand the structure of the image," while the second output head makes use of that information in estimating the matte. In addition to the alpha output we add two additional networks which simultaneously reconstruct the foreground and background. Similarly to the first output of the first network, these networks’ outputs are constrained to match the target image but, unlike the first network, they are constrained only in their respective regions of the trimap. Similarly to the alpha output, they extrapolate those constraints to inpaint the unconstrained region (see Fig. 2). The latter three outputs – 𝛼,ˆ 𝐹,ˆ 𝐵ˆ – are coupled via an additional constraint that they together satisfy (1) (see Detailed Loss below). We put 𝐹ˆ and 𝐵ˆ in separate networks (as in [Gandelsman et al. 2019]) rather than as additional outputs heads of the first network (as in [Sun et al. 2021]) in order to allow the inpainting of the foreground to be independent of the background. All three networks share the same generic U-net structure [Ulyanov et al. 2018], except for an additional output head in the first network. Our experiments use Adam [Kingma and Ba 2015] with a learning rate of 0.001. 3.1 Detailed Loss The first term of our loss function is the reconstruction loss between the first network output and the target image: The second loss term constrains 𝛼ˆ, where 𝑇 is the trimap and 𝐶 = 𝐹 ∪ 𝐵 is the constrained region of the trimap. The reconstruction losses for the foreground and background outputs are defined similarly to 𝐿𝐼 , but constrained only in their respective regions of the trimap: The three networks’ outputs are coupled via the alpha-compositing equation as follows: where 𝑈 = 𝐼 − 𝐶 is the unconstrained region of the trimap. Finally, we include an exclusion loss similar to [Sun et al. 2021] to prevent the structure of the foreground from leaking into the background and vice-versa: The total loss is the sum of the above six components: Unlike other DIP-based techniques, our algorithm does not require early stopping since the goal is to exactly fit both the image and the trimap constraints. 3.2 Temporal continuity In our experiments temporal continuity was obtained by warmstarting the optimization for frames other than the first with the final weight values of the previous frame, and stopping with a loss threshold rather than a fixed number of iterations (see video). This simple strategy produces reasonable results even on the relatively difficult case of hair. It also reduces the compute time by roughly an order of magnitude. 3.3 Why does it work? The deep image prior empirically demonstrates that standard convolutional networks can act as a good low-level prior for image reconstruction tasks, but why can it be adapted to invent plausible alpha mattes? Our intuition is as follows: The DIP provides a somewhat deep and hierarchical encoding of the particular image. The features in this encoding will primarily span valid image structures, with noise lying outside this space. In our work these features are re-combined to produce the matte. This works because the alpha matte contains structures that relate to those in the foreground image. In this respect our approach resembles a "deep" version of the guided image filter [He et al. 2010]. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv

Training-Free Neural Matte Extraction for Visual Effects: Method

Author:
(1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com);
(2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com);
(3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com);
(4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction
Background and Related Work
Method
Results and Failure Cases
Limitations and Conclusion, Acknowledgements and References 4 RESULTS AND FAILURE CASES We use images, trimaps, and GT alpha from the dataset [Rhemann et al. 2009] to allow comparison to the ground truth. Fig. 1 shows a moderately challenging case with extensive hair, some of which is similar to the background color. Fig. 3 shows several additional examples. While the ground truth and estimated alpha maps appear identical at first glance, small differences can be seen upon enlargement. These may be due to imperfections in our algorithm, but also may reflect differences in color spaces between our algorithm and the ground-truth estimation process. Fig. 2 shows the extrapolated foreground 𝐹ˆ and background 𝐵ˆ for a particular image. Note that the inpainted colors need to be correct only in the unconstrained region surrounding the object border in order to allow compositing over a new background, as is the case in this figure. The unrealistic extrapolated 𝐹ˆ over the pure background regions is ignored. 4.1 Failure cases While objects with "holes" sometimes yield good mattes (e.g. the plant in Fig. 3), they are also a failure case (Fig. 4). In some cases the trimap can be adjusted to highlight the missing holes, though this would be laborious in cases such as the cup in Fig. 4. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Author: Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction Background and Related Work Method Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Abstract and Introduction Abstract and Introduction Background and Related Work Background and Related Work Method Method Results and Failure Cases Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Limitations and Conclusion, Acknowledgements and References 4 RESULTS AND FAILURE CASES We use images, trimaps, and GT alpha from the dataset [Rhemann et al. 2009] to allow comparison to the ground truth. Fig. 1 shows a moderately challenging case with extensive hair, some of which is similar to the background color. Fig. 3 shows several additional examples. While the ground truth and estimated alpha maps appear identical at first glance, small differences can be seen upon enlargement. These may be due to imperfections in our algorithm, but also may reflect differences in color spaces between our algorithm and the ground-truth estimation process. Fig. 2 shows the extrapolated foreground 𝐹ˆ and background 𝐵ˆ for a particular image. Note that the inpainted colors need to be correct only in the unconstrained region surrounding the object border in order to allow compositing over a new background, as is the case in this figure. The unrealistic extrapolated 𝐹ˆ over the pure background regions is ignored. 4.1 Failure cases While objects with "holes" sometimes yield good mattes (e.g. the plant in Fig. 3), they are also a failure case (Fig. 4). In some cases the trimap can be adjusted to highlight the missing holes, though this would be laborious in cases such as the cup in Fig. 4. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv

Training-Free Neural Matte Extraction for Visual Effects: Results and Failure Cases

Author:
(1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com);
(2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com);
(3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com);
(4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction
Background and Related Work
Method
Results and Failure Cases
Limitations and Conclusion, Acknowledgements and References 2 BACKGROUND AND RELATED WORK The alpha-compositing equation is where 𝑖 ∈ {𝑟, 𝑔, 𝑏}, 𝐼𝑖 are red, green, and blue values at a pixel of the given image (given), 𝐹𝑖 are the foreground object color (unknown), 𝐵𝑖 are the background object color (unknown), and 𝛼 ∈ [0, 1] is the (partially unknown) "alpha matte", representing the translucency or partial coverage of the foreground object at the pixel. Typical images contain large areas that are relatively easily estimated to be solid "foreground" (𝛼 = 1) or "background" (𝛼 = 0), as well as smaller regions where the alpha value is fractional. The matte extraction problem requires finding the these fractional alpha values. The problem is underconstrained because there are only three known values 𝐼𝑟, 𝐼𝑔, 𝐼𝑏 but seven unknown values. Hair is the prototypical challenge for matte extraction due to its irregular coverage and translucency, though fractional alpha also generally appears along all object edges due to the partial coverage of a pixel by the foreground object. It is also necessary to estimate the unknown foreground color in the fractional alpha regions, since without this the foreground cannot be composited over a different background. Many methods assume that approximate demarcations of the solid foreground and background are provided, e.g. in the form of artist-provided "trimaps" [Rhemann et al. 2009] or scribbles [Levin et al. 2008]. While the research literature generally considers the problem of matte extraction from arbitrary natural backgrounds, in industry practice matte extraction from a greenscreen background is far from a solved problem and often requires an artist-curated combination of techniques to obtain the required quality [Erofeev et al. 2015; Heitman 2020]. LED walls remove the need for greenscreens in some cases, and are well suited for fast-paced television production, however they also have drawbacks: computationally expensive physical or character simulations are not possible due to the need for real-time playback in a game engine, bright backgrounds may introduce challenging light spill on the foreground physical objects [Seymour 2020], their cost is prohibitive for smaller studios, and the effects must be finished at the time of the principle shoot thus excluding the creative control and iterative improvement available in traditional post production. Progress on matte extraction algorithms has been greatly facilitated by datasets that include ground truth (GT) mattes [Erofeev et al. 2015; Rhemann et al. 2009]. The GT mattes have been obtained by different means including chroma keying from a greenscreen and photographing a representative toy object in front of a background image on a monitor (Fig. 1 (a)). Failure to closely approximate these GT mattes indicates a poor algorithm, but an exact match may be unobtainable for several reasons: 1) the chroma key itself involves an imperfect algorithm [Erofeev et al. 2015], 2) light from multiple locations can physically scatter though (e.g.) translucent hair to arrive at a single pixel; this cannot be simulated in a purely 2D matte extraction process, 3) the image gamma or color space used in the benchmarks is not always evident and using a different gamma will produce small differences. The underconstrained nature of matte extraction has been approached with a variety of methods [Wang and Cohen 2007]. One classic approach estimates the unknown alpha at a pixel based on similarity to the distributions of known foreground and background colors in solid regions [He et al. 2011; Mishima 1992]. Another prominent principle finds the unknown alpha value by propagating from surrounding known values [Aksoy et al. 2018; Levin et al. 2008]. These approaches often require solving a system involving a generalized Laplacian formed from the pixel affinities in the unknown region, which prevents real-time or interactive use. Deep learning (DL) approaches are used in recent research e.g. [Lin et al. 2020; Sun et al. 2021]. Much of this work focuses on providing alternate backgrounds for video meetings, and training databases consist of mostly forward-facing human heads. A state of the art method [Lin et al. 2020] demonstrates high-quality matte extraction on HD-resolution images at 60fps. Many DL methods adopt a combination of techniques, e.g. separate networks for overall segmentation and for fractional alpha regions, or other hybrid approaches. 2.1 Deep Image Prior The Deep Image Prior (DIP) [Ulyanov et al. 2018] demonstrates that the architecture of an untrained convolutional network provides a surprisingly good prior for tasks such as image inpainting and denoising. The key observation is that while a powerful DNN can fit arbitrary image structures such as noise, it is "easier" to fit natural image structures, as reflected in faster loss curve decay. Most experiments [Ulyanov et al. 2018] optimize the weights of a U-net [Ronneberger et al. 2015] that maps fixed random noise to a single output image. Uncorrupted features of the image are fit earlier in the optimization process, so early stopping results in a corrected (denoised or inpainted) image. The DIP has been successfully applied to other problems in image processing. Unsupervised coarse binary segmentation is demonstrated in [Gandelsman et al. 2019], based on a principle that it is easier (in terms of loss decay) to fit each component of a mixture of images with a separate DIP rather than using a single model. Concurrent with our work, [Xu et al. 2022] formulate a DIP approach to background matting. This problem scenario differs from our work in that it requires a clean plate, and hence is unsuitable for many VFX applications such as those that are filmed outdoors with a moving camera. Our work also uses the DIP. We focus on high-quality estimation of fractional alpha mattes and, in contrast to previous work, undertake the challenging case where no clean plate is available, instead relying only on trimaps that can be easily created during post-production. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Author: Author: (1) Sharif Elcott, equal contribution of Google Japan (Email: selcott@google.com); (2) J.P. Lewis, equal contribution of Google Research USA (Email: jplewis@google.com); (3) Nori Kanazawa, Google Research USA (Email: kanazawa@google.com); (4) Christoph Bregler, Google Research USA (Email: bregler@google.com). Table of Links Abstract and Introduction Background and Related Work Method Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Abstract and Introduction Abstract and Introduction Background and Related Work Background and Related Work Method Method Results and Failure Cases Results and Failure Cases Limitations and Conclusion, Acknowledgements and References Limitations and Conclusion, Acknowledgements and References 2 BACKGROUND AND RELATED WORK The alpha-compositing equation is where 𝑖 ∈ {𝑟, 𝑔, 𝑏}, 𝐼𝑖 are red, green, and blue values at a pixel of the given image (given), 𝐹𝑖 are the foreground object color (unknown), 𝐵𝑖 are the background object color (unknown), and 𝛼 ∈ [0, 1] is the (partially unknown) "alpha matte", representing the translucency or partial coverage of the foreground object at the pixel. Typical images contain large areas that are relatively easily estimated to be solid "foreground" (𝛼 = 1) or "background" (𝛼 = 0), as well as smaller regions where the alpha value is fractional. The matte extraction problem requires finding the these fractional alpha values. The problem is underconstrained because there are only three known values 𝐼𝑟, 𝐼𝑔, 𝐼𝑏 but seven unknown values. Hair is the prototypical challenge for matte extraction due to its irregular coverage and translucency, though fractional alpha also generally appears along all object edges due to the partial coverage of a pixel by the foreground object. It is also necessary to estimate the unknown foreground color in the fractional alpha regions, since without this the foreground cannot be composited over a different background. Many methods assume that approximate demarcations of the solid foreground and background are provided, e.g. in the form of artist-provided "trimaps" [Rhemann et al. 2009] or scribbles [Levin et al. 2008]. While the research literature generally considers the problem of matte extraction from arbitrary natural backgrounds, in industry practice matte extraction from a greenscreen background is far from a solved problem and often requires an artist-curated combination of techniques to obtain the required quality [Erofeev et al. 2015; Heitman 2020]. LED walls remove the need for greenscreens in some cases, and are well suited for fast-paced television production, however they also have drawbacks: computationally expensive physical or character simulations are not possible due to the need for real-time playback in a game engine, bright backgrounds may introduce challenging light spill on the foreground physical objects [Seymour 2020], their cost is prohibitive for smaller studios, and the effects must be finished at the time of the principle shoot thus excluding the creative control and iterative improvement available in traditional post production. Progress on matte extraction algorithms has been greatly facilitated by datasets that include ground truth (GT) mattes [Erofeev et al. 2015; Rhemann et al. 2009]. The GT mattes have been obtained by different means including chroma keying from a greenscreen and photographing a representative toy object in front of a background image on a monitor (Fig. 1 (a)). Failure to closely approximate these GT mattes indicates a poor algorithm, but an exact match may be unobtainable for several reasons: 1) the chroma key itself involves an imperfect algorithm [Erofeev et al. 2015], 2) light from multiple locations can physically scatter though (e.g.) translucent hair to arrive at a single pixel; this cannot be simulated in a purely 2D matte extraction process, 3) the image gamma or color space used in the benchmarks is not always evident and using a different gamma will produce small differences. The underconstrained nature of matte extraction has been approached with a variety of methods [Wang and Cohen 2007]. One classic approach estimates the unknown alpha at a pixel based on similarity to the distributions of known foreground and background colors in solid regions [He et al. 2011; Mishima 1992]. Another prominent principle finds the unknown alpha value by propagating from surrounding known values [Aksoy et al. 2018; Levin et al. 2008]. These approaches often require solving a system involving a generalized Laplacian formed from the pixel affinities in the unknown region, which prevents real-time or interactive use. Deep learning (DL) approaches are used in recent research e.g. [Lin et al. 2020; Sun et al. 2021]. Much of this work focuses on providing alternate backgrounds for video meetings, and training databases consist of mostly forward-facing human heads. A state of the art method [Lin et al. 2020] demonstrates high-quality matte extraction on HD-resolution images at 60fps. Many DL methods adopt a combination of techniques, e.g. separate networks for overall segmentation and for fractional alpha regions, or other hybrid approaches. 2.1 Deep Image Prior The Deep Image Prior (DIP) [Ulyanov et al. 2018] demonstrates that the architecture of an untrained convolutional network provides a surprisingly good prior for tasks such as image inpainting and denoising. The key observation is that while a powerful DNN can fit arbitrary image structures such as noise, it is "easier" to fit natural image structures, as reflected in faster loss curve decay. Most experiments [Ulyanov et al. 2018] optimize the weights of a U-net [Ronneberger et al. 2015] that maps fixed random noise to a single output image. Uncorrupted features of the image are fit earlier in the optimization process, so early stopping results in a corrected (denoised or inpainted) image. The DIP has been successfully applied to other problems in image processing. Unsupervised coarse binary segmentation is demonstrated in [Gandelsman et al. 2019], based on a principle that it is easier (in terms of loss decay) to fit each component of a mixture of images with a separate DIP rather than using a single model. Concurrent with our work, [Xu et al. 2022] formulate a DIP approach to background matting. This problem scenario differs from our work in that it requires a clean plate, and hence is unsuitable for many VFX applications such as those that are filmed outdoors with a moving camera. Our work also uses the DIP. We focus on high-quality estimation of fractional alpha mattes and, in contrast to previous work, undertake the challenging case where no clean plate is available, instead relying only on trimaps that can be easily created during post-production. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv

Training-Free Neural Matte Extraction for Visual Effects: Background and Related Work

Emojipedia describes it as “a pile of poo that is shaped like a soft-serve ice cream; brown in color with a friendly smile in most versions of this emoji.” The emoji is inspired by Dr. Slump, a Japanese animation character from the 80s. “However, it wasn’t always so welcomed amongst emoji-fans,” writes Amanda Vanschaick in a mentioning that “people were not impressed.” 2017 blog post “Gmail held the strongest path of resistance against the emoji, stating that they felt people would be offended by the vulgarity of the emoji,” she says. “It took the persuasion of the creators of the emoji, Darick Tong, Darren Lewis, Takeshi Kishimoto, and Katsuhiko “Kat” Momoi, to get the poo-moji on to Gmail.” At the time, the creators were all from Google. Tong and Lewis were Google software engineers — Tong led Google’s emoji project; Kishimoto was Google’s Japanese product manager; and Momoi was Google test engineer and internationalization expert. Why Google? While the emoji project began in Japan in 1999 when Japan’s three major telecom carriers — KDDI AU, SoftBank, and NTT-DoCoMo — created their first series of small rudimentary graphics for text messages and emails, it wasn’t until 2007 until 2007, when Google partnered with KDDI AU to adopt emojis for Gmail, that the emoji code was fixed to make it simple for everybody to use them. And with Google on board emojis became international. In 2014, about the beginning of the emoji project: “The internal project name was ,” they write. “The motivation was to expand Google’s presence in Japan and Asia. The one thing that was missing from the Gmail experience was a good handling of emoji, which is very engrained in the culture of Japan. Fast Company quoted Tong Mojo The article described the history of the emoji as “an international tale of drama, cultural differences, and near-disasters.” The team of creators thought that the poop emoji “was absolutely one of the necessary emoji” to have in the first release for Gmail, but many at Google headquarters in California felt it was too offensive and some even were against even implementing emojis for Gmail as they would ruin the user experience. “We pushed hard for the poop emoji,” Lewis told Fast Company. It passed and in 2008, Google posted about emojis in a blog post aptly titled: “ .” The poop emoji was included. A picture is worth a thousand words (Credits: Google) “The black and white days of text-based emails have had their day,” the post reads. “Following the evolutionary path blazed by colored labels, we present, in all their technicolor glory, emoticons in your mail.” A month later, Apple began to embrace the idea and finally two years later, in 2010, emoji was officially approved by the Unicode Consortium, which meant that it was accepted industry-wide as a real language. The Unicode Consortium is a non-profit corporation devoted to developing, maintaining, and promoting software internationalization standards and data, particularly the Unicode Standard, which specifies the representation of text in all modern software products and standards. It actively develops standards in the area of internationalization including defining the behavior and relationships between Unicode characters. Today, emojis are part of pop culture and used as part of our language. But “it’s easy to dismiss emoji,” wrote Adam Sternbergh in a . “They are, at first glance, ridiculous.” New York Magazine article in 2014 And the poop emoji maybe even more ridiculous than others. Sternbergh described emojis as “a small invasive cartoon army of faces and vehicles and flags and food and symbols trying to topple the millennia-long reign of words.” And yet, emojis have become a daily occurrence for many of us. Consider for instance the use of reaction emojis on Facebook — within the Gear VR’s video app — and how they have transformed the way we interact with posts, beyond the simple thumbs up emoji. even on 360 videos In 2015, for the first time ever, the Oxford Dictionaries was an emoji: Word of the Year Officially called the emoji, 😂 “was chosen as the ‘word’ that best reflected the ethos, mood, and preoccupations of 2015.” The emoji won over other shortlisted words like ‘sharing economy’, ‘dark web’, ‘ad blocker’, and even ‘refugee’ and Brexit’. Face with Tears of Joy According to , an experiment in real-time visualization of all emoji symbols used on Twitter started in 2013 by , an independent artist and hacker from Brooklyn, NY, the poop emoji is the 116th most used emoji on Twitter. emojitracker Matthew Rothenberg I analyzed the use of emojis in politics and diplomacy in a 2017 blog post here on . Medium _Storytelling via round smiley faces, thumbs up, and flags._medium.com 😂🌍🇺🇸 Emojis for diplomats The poop emoji was also cited in my post, together with the toilet 🚽 emoji. Both were featured quite successfully in the November 2016 #WorldToiletDay campaign by the (WHO) of the . World Health Organization United Nations I’m glad we all look at the poop emoji as a way to engage audiences around the globe! It’s fun, universal, and easy to implement as part of any campaign. Happy birthday 💩!

10th anniversary of the 💩 poop emoji

In 2012, I moved back to the US after spending most my adult life in Taiwan and then Beijing. It was a brutal adjustment. San Francisco I earned well less than $10k that year. Living in San Francisco’s Chinatown, I was able to rent for just $500 per month, but I was still seeing my savings dwindle as I tried to turn myself into more of a coder than running a WP blog for years had made me. I learned what I could online with , and looked for offline meetups that taught coding. Unfortunately, most were aimed at people who were far more advanced than I was (and probably funded by companies who wanted to recruit the attendees). The meetups I found that were aimed at beginners were generally limited to specific demographics that I wasn’t a part of. Still, I was able to meet some very friendly people at some meetups and while it didn’t to a great deal of learning, it was motivating. Code School I looked everywhere for entry level jobs. Little did I realize that “entry level” in San Francisco was more like solidly mid-level in Beijing, especially for people who weren’t fresh grads on an internship program. So I went out and offered to improve websites of brick and mortar businesses. The rate was very low and sometimes it was hard to get clients to pay, but since I was living in that cheap room in Chinatown, it was just enough. Learning about bootcamps When I first heard of bootcamps, they sounded amazing! There was this “Starter League” out in Chicago and DHH, the creator of Ruby on Rails, was helping teach people! There was no way I wanted to spend years getting another college degree, but this path sounded like a viable alternative for motivated life-long learners. Then I heard there was a school “Dev Bootcamp” in San Francisco! It was fully booked for longer than I could afford to wait, but there was another school called App Academy that had just started teaching its first class of students. They were kind enough to let me watch some of the instruction. It was top notch. The students were amazing too. I had never seen such a group I’d felt so much in common with that was doing exactly what I wanted to do. Trying to get in I put everything I had into trying to get into their second class. At the same time, I learned of another school called “Catalyst Class” on Hacker News and applied to it, too. Unlike App Academy, Catalyst Class didn’t have a “pay only if you’re hired and then pay a percentage of your salary” model. They would only accept cash, and at $18,000 there was no way I could afford it. Still, their interview process was amazing. On of their instructors, Shawn, actually taught me what JavaScript prototypes were . Even if I were rejected or accepted but couldn’t afford it, it was still a great use of my time. as part of the interview I made it to App Academy’s final interview! Then they rejected me. Strangely, their question was literally just fizzbuzz, which I knew well. It’s possible I was rejected due to the fact that I couldn’t pay a deposit they charged students to make sure they stick with the program, but I’m truly not sure what the real reason was. They were kind enough to tell me that they knew the owners of Catalyst Class and that whatever program they were making would be top notch, though. Soon after that, Catalyst Class called up to tell me I was accepted into their second class and that they could defer of my tuition! some I was still thousands short, but I knew this was an amazing opportunity so I looked far and wide for loans. I went into bank after bank asking both for educational and personal loans. My current bank, cut me off from the one of my two credit cards that had no balance on it. I got some help A/B testing and optimizing every part of my pitch from some friends who were far more into the arts of sales and persuasion than I ever have been. It took a lot of uncomfortable asks, but in the end I was barely able to scrape enough money to join the class. Hack Reactor Nobody had graduated from the school yet. I literally spent thousands of dollars I didn’t yet have to get in. The school even changed its name from Catalyst Class to Hack Reactor in the first few weeks of my time there. Some evenings when I got back up the hills to my room at Stockton and Clay, I felt like I’d taken a ridiculous leap to try to move to San Francisco to become a “real” programmer. Usually, I just fell asleep. Classes were intense and I spent at least 11 and a half hours a day at the school. Sometimes it was more like 17 hours. But the quality was top notch. I’m sure the curriculum has changed considerably since that time given that JavaScript is barely the same language it was then, but I recorded it as I was going through it. in detail When I finished the program, Hack Reactor had no brand value and its network of alumni was just the class that overlapped mine and finished 6 weeks earlier. But that didn’t matter. There was a point where suddenly I went from companies all telling me no as they had been for a year already to where suddenly I was getting interest from Google, Japanese game companies, start-ups and consultancies. I was completely broke not having earned anything for the past four months and getting stiffed by my final website client from before starting the , so I had to go with a company that was willing to move quickly. bootcamp Groupon That company ended up being Groupon. They had seen me at a meetup showing off a project I did during Hack Reactor that I’d built with CoffeeScript and Backbone.js. They were also using CoffeeScript and Backbone.js and after doing well on a phone screen I got the opportunity to go to a brutal 5 hour on-site interview. Each hour involved a different person coming into the room and asking me to explain how I’d solve technical problems and then write their answers on the whiteboard. I felt like I did well with all but one of the interviewers but had bombed on the other, so I was overjoyed when I received an email from them making me an offer. Months later, I learned they had a practice borrowed from Amazon that they called “bar raiser” interviewers. I.e., one of the hours was to be brutal. supposed https://youtu.be/fG716T3wIUw Working life I learned a lot at Groupon and worked with great people. My team was full of Chileans and Argentinians from a start-up Groupon had bought. While I was easily the most junior dev on my team, I was able to start contributing very quickly and I really liked the inclusive, quirky company culture. I had little passion for the company’s mission, though I guess I do like bargains as much as the next person, but it was a great place for learning and growing. The learning wasn’t as rapid as while I was at Hack Reactor but it was far better than any job I’d had before. I also earned more in my signing bonus than I had in the entire previous year. I could finally afford to live in Silicon Valley. Thoughts about bootcamps I don’t think that bootcamps are the ideal solution for everyone. This may be part of why their grads have done less well as they’ve grown over the years. However, I also don’t agree with much of the hatred I’ve seen directed at bootcamps on HN, Reddit and other forums. Years of seat time is an awful metric of how much someone learns. Four year degrees involve a lot of time studying things that aren’t related to . Yes it’s true that not all of what’s covered in a CS degree can be covered in a 12 week program, even if over 900 hours of work are packed into those 12 weeks. However, there’s also a lot that bootcamps cover that aren’t in most universities. They tend to focus on more cutting edge things and on things that the market currently wants. programming It’s not a good thing if everyone is studying exactly the same things and picking up the same set of skills. In any functioning economy, a huge supply of people with identical skillsets will simply drive down the value of those skills. This is part of why bootcamp grads have often had better pricing power in the market than fresh university grads even though they don’t have same coverage of CS fundamentals. In the long term, fundamentals are valuable. Anyone who does a bootcamp, learns a lot for a few months, graduates and immediately quits learning is going to be in for a world of pain. This knowledge drove me to spend hundreds of hours going through , and then later branch out of engineering entirely to learn more about online marketing and even go through some business school reading lists. MOOCs Not everything was perfect at Hack Reactor, of course. There were hiccups, especially being such a new school. I also could have learned everything they taught on my own. But it would have taken me years longer without them. I consider that $18k and 13 weeks to be one of the best life investments I’ve ever made.

Stories, News & Mentions about GOOGLE-JAPAN

Google-japan’s Stories on HackerNoon

Google-japan's News & Mentions