We’re building Verse™: a visual programming tool made for developers by developers

Factorize

factorize

Efficient Neural Network Approaches for Conditional Optimal Transport: Numerical Experiments

In today's edition: What does research say about how far we are from AGI? News from the usual subjects: Anthropic, Cohere, OpenAI, etc. The freshest AI&ML research papers from the week of Mar 4 — Mar 10 Though the question, “ ” predates the term “artificial intelligence” itself, it saw a significant resurgence on Twitter last week, prompted by the Musk vs. OpenAI lawsuit (Musk accuses OpenAI of abandoning open-source principles and prioritizing profit over safety, hindering the safe development of AGI.) . Today, we will discuss “How Far Are We from Intelligent Visual Deductive Reasoning?” “How Far Are We From Automating Front-End Engineering?” and Stephen Wolfram’s article “Can AI Solve Science?” These papers offer fascinating explorations of the differences between human and artificial intelligence. How far are we from achieving human-level intelligence in machines (or AGI, or ASI)? But far more interesting were the papers and an article that came out last week tackling this question Intelligent Visual Deductive Reasoning In “ ?”, researchers from Apple explore Vision-Language Models (VLMs), like GPT-4V, in visual-based deductive reasoning, a complex yet less studied area, using Raven’s Progressive Matrices (RPMs)*. How Far Are We from Intelligent Visual Deductive Reasoning *Raven’s Progressive Matrices is a nonverbal intelligence test measuring abstract reasoning, using patterns to assess cognitive functioning without language. What caught my attention was the finding that . The paper notes, “ ” This inability to deal with abstract concepts marks a fundamental difference between computational processing and human cognitive abilities. AI systems like VLMs struggle with tasks requiring abstract pattern recognition and deduction VLMs struggle to solve these tasks mainly because they are unable to perceive and comprehend multiple, confounding abstract patterns in RPM examples. Being a sophisticated pattern recognizer doesn’t equate to sentience. The observation that “ ” highlights , suggesting a nuanced aspect of intelligence that current AI lacks. Another intriguing point was the models’ overconfidence. all the tested models never express any level of uncertainty the importance of doubt and uncertainty in human cognition Automating Front-End Engineers In “ ”, researchers from Stanford University, Georgia Tech, Microsoft, and Google DeepMind have developed a benchmark for Design2Code, aiming to evaluate how well multimodal LLMs convert visual designs into code. Here, the replacement of humans came closer. Despite some limitations, there were considerable advancements in using generative AI to convert designs into front-end code. It’s remarkable that “ .” This finding challenges traditional notions of artistic and creative value, Design2Code: How Far Are We From Automating Front-End Engineering? annotators think GPT-4V generated webpages can replace the original reference webpages in 49% of cases in terms of visual appearance and content; and in 64% of cases, GPT-4V generated webpages are considered better questioning whether creativity is uniquely human or can be algorithmically reproduced — or even surpassed. However, . VLMs struggle with “ ” posing questions about understanding and interpretation. significant limitations persist recalling visual elements from the input webpages and generating correct layout designs. So, the important question is actually not how far we are from AGI (whatever it is), but how we embrace human-AI collaboration most effectively. AI Solving Science In that sense, Stephen Wolfram’s blog post “ ” serves as an excellent example. In the very beginning, he plainly states that AI cannot solve all scientific questions. However, there is significant value in AI assisting scientific progress. He discusses how LLMs can serve as a new kind of linguistic interface to computational capabilities, providing high-level “autocomplete” for scientific work. As he usually does, he emphasizes the transformative potential of representing the world computationally and suggests that pockets of computational reducibility* can be found by AI as well. Can AI Solve Science? *A pocket of computational reducibility — a fascinating concept introduced by Wolfram — is a situation or problem within a complex system where, despite the system’s overall unpredictability, predictable patterns or simplified behaviors emerge, allowing for easier understanding or calculation. Wolfram argues that AI can significantly aid scientific discovery by providing new tools for analysis and exploration, but its ability to completely “solve” science is limited by fundamental principles such as computational irreducibility. The future of AI in science lies in its integration with human creativity and understanding, leveraging its strengths to uncover new knowledge within the constraints of what is computationally possible. We might be able to survive without front-end developers (no offense intended), but scientists remain indispensable! To summarize: https://x.com/pmddomingos/status/1766945083314827455?s=20&embedable=true News from The Usual Suspects © Cohere and its commitment to the research community https://x.com/cohere/status/1767275128813928611?s=20&embedable=true Hugging Face Starting an ambitious open robotics project and hiring , a former Tesla scientist, to expand beyond software into robotics, including work on humanoid robots like Tesla’s Optimus. Remi Cadene Offers an amazing tool: https://x.com/maximelabonne/status/1767124527551549860?s=20&embedable=true Russia’s talent is invisible According to “The Global AI Talent Tracker 2.0”, Russia has no chance in the AI race: this analysis Inflection enhances its Pi Inflection , enhancing its personal AI, Pi, with IQ capabilities alongside its empathetic EQ. They claim to compete with GPT-4, achieving significant efficiency and performance improvements, especially in STEM areas, with less computational resource usage. launches Inflection-2.5 Chips TSMC (based in Taiwan), the world’s largest contract chipmaker, is for an Arizona chip plant. This funding, part of the CHIPS and Science Act of 2022, aims to boost domestic semiconductor production. TSMC’s $40 billion investment in the plant marks one of the largest foreign investments in U.S. history. expected to receive over $5 billion in U.S. federal grants OpenAI: new members on the board Sam Altman is reinstated as a member of the OpenAI board. , with a rich history as the CEO of the Bill and Melinda Gates Foundation, brings extensive experience in healthcare and philanthropy. , a seasoned executive with roles at Sony Entertainment and as a lawyer, offers legal and entertainment industry insights. , leading Instacart and with a background at Meta Platforms, including as head of Facebook, contributes expertise in technology and e-commerce. This diverse trio enhances OpenAI’s governance with their wide-ranging expertise and perspectives. He is joined by three women: Sue Desmond-Hellmann Nicole Seligman Fidji Simo Elon’s Grok https://x.com/elonmusk/status/1767108624038449405?s=20&embedable=true Anthropic Offers a great collection of prompts in its Prompt library Enjoyed This Story? I write a weekly analysis of the AI world in the newsletter. We aim to equip you with comprehensive knowledge and historical insights, so you can make informed decisions about AI and ML. Turing Post 🎁 Bonus: The freshest research papers, categorized for your convenience Enhancements in Language Models and Multimodal Understanding : Showcases an LLM’s intrinsic ability to understand and generate music, expanding LLMs’ applications beyond text. ChatMusician Read the paper : Demonstrates advanced multimodal understanding by processing extensive contexts of text, video, and audio. Gemini 1.5 Read the paper : Enhances TTS synthesis with a factorized diffusion model for zero-shot natural speech generation. NaturalSpeech 3 Read the paper : Utilizes fine-tuned MLMs for superior image-text data filtering, improving dataset quality for MLM training. Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filters Read the paper : Introduces the first LLM designed for the legal domain, demonstrating proficiency in legal text understanding and generation. SaulLM-7B Read the paper Novel Training and Evaluation Techniques : Proposes a method for scaling instruction tuning in mathematical reasoning, significantly enhancing LLMs’ problem-solving abilities. MathScale Read the paper : Reveals redundancy in LLM layers and proposes a pruning strategy, maintaining performance while reducing model size. ShortGPT Read the paper : Enhances memory efficiency in LLM training without sacrificing performance by applying gradient low-rank projection. GaLore Read the paper : Develops a method for collaborative decoding among multiple LLMs, improving performance across various tasks. Learning to Decode Collaboratively with Multiple Language Models Read the paper : Advocates for training RL value functions through classification instead of regression, boosting performance and scalability. Stop Regressing Read the paper Advances in Generative Models and Data Synthesis : Introduces a framework for generating synthetic multimodal datasets, overcoming limitations in data privacy and diversity. MAGID Read the paper : Trains a generative model to create interactive virtual worlds from text, images, or sketches, advancing interactive environment simulation. Read the paper Genie Scalability and Efficiency in AI Systems : Details a system for training LLMs on over 10,000 GPUs, tackling challenges in training efficiency and stability at large scales. MegaScale Read the paper : Improves State Space Models with dense hidden connections for efficient large language models, enhancing performance with minimal parameter increase. DenseMamba Read the paper Exploring New Frontiers in AI and Machine Learning : Illustrates that planning and prediction in time series can be simplified through learned contrastive representations. Inference via Interpolation Read the paper : Employs simulated trial and error for tool learning, significantly enhancing LLMs’ practical application capabilities. LLMs in the Imaginarium Read the paper : Aims to improve context length generalization in LLMs by refining interpolation techniques for out-of-distribution token positions. Resonance RoPE Read the paper : Enhances the localization of behaviors in LLMs to specific components, improving diagnostic capabilities and model understanding. AtP Read the paper : Investigates using world models for enhancing visual representation learning, beyond reinforcement learning applications. Learning and Leveraging World Models in Visual Representation Learning Read the paper Platforms and Tools for Model Evaluation and Interaction : Provides a platform for evaluating LLMs based on human preferences, offering insights into model rankings through pairwise comparisons. Chatbot Arena Read the paper : Explores enhancing LLMs’ reasoning capabilities using RLHF, comparing different algorithms and reward strategies. Teaching Large Language Models to Reason with Reinforcement Learning Read the paper Also published . here

How Far Are We From Human-level Intelligence in AI?

A program that maintain its correctness in a multi-threaded environment is said to be a program. Thread-Safe Fundamentally, thread safety revolves around proper management of the shared state. In the absence of such proper management, the program could potentially spiral into an invalid state when executed in a multi-threaded environment. Defining Application’s State An application’s state is formed by the collective states of the individual objects that constitute it. An object’s state encompasses the data stored in its static or instance fields, which can influence its externally visible behavior. Furthermore, an object’s state may encompass fields from other dependent objects, creating a complex web of interconnected states within the application. Whether an object’s state need to be managed for concurrent access depends on its usage in your program. If an object is only ever accessed from a single thread, state management might not be needed! If multiple threads access the same mutable state of an object without proper synchronization, your application is technically broken. There are three ways to fix it — Don’t share the object across threads Make the object’s state immutable Implement proper synchronization for accessing the object’s shared state Using good Object Oriented Principles can help us in designing thread-safe classes. Thoughtful implementation of encapsulation, immutability, and precise specification of invariants can significantly reduce the effort required to create thread-safe classes, ultimately streamlining the development process. An important thing to note is that a program consisting of only thread-safe classes may not be thread-safe. Additionally, a program may be thread-safe even when some of its classes are not thread-safe. It all depends on how your objects interact with each other in your program! Defining Thread Safety A thread-safe class behaves correctly even when accessed from multiple threads, regardless of the scheduling or interleaving of the execution. The calling code needs no synchronization on its part when working with a thread-safe class. Thread-safe classes encapsulate the necessary synchronisation, relieving the client from the burden of providing their own. The need for synchronization arises when multiple threads access the same mutable state. However, stateless objects, by virtue of having no state to share, inherently possess thread safety. As a result, . stateless objects are always thread-safe Example of a thread-unsafe class Consider the following class that is used to find a certain number of primes, Java public class PrimeFinder {
    // Variable to count the number of invocations
    private int hitCount = 0;
    public int getHitCount() {
        return hitCount;
    }
    public List<Integer> findPrimes(int count) {
        hitCount++;
        // Find 'count' number of primes
        final List<Integer> primes = new ArrayList<>();
        for(int i = 2; i < Integer.MAX_VALUE; i++) {
            if(isPrime(i)) {
                primes.add(i);
                if(primes.size() == count) {
                    break;
                }
            }
        }
        return primes;
    }

    private boolean isPrime(int num) {
        for(int i = 2; i < num; i++) {
            if(num % i == 0) {
                return false;
            }
        }
        return true;
    }
} Note that the instance variable hitCount constitutes the state of any PrimeFinder object. Unfortunately, is not thread-safe, even though it would work fine in a single-threaded environment. This class is vulnerable to , as the operation to increase the , although seemingly atomic, is actually broken down into three distinct steps. Here is an example of an unlucky sequence of events causing to enter an invalid state. PrimeFinder lost updates hitCount hitCount Unlucky interleaving of execution causes hitCount to be 10 when it should be 11 Computation of incorrect results in case of some unlucky timings in a concurrent program is called a . The particular type of race condition that plagues our is called or . To eliminate this race condition, we must increment the hitCount in a single atomic operation. race condition PrimeFinder check-then-act read-modify-write There are several ways to fix the class. One such way is to use a thread-safe class inside . Here is the updated code, PrimeFinder PrimeFinder public class PrimeFinder {
    private final AtomicInteger hitCount = new AtomicInteger(0);
    public int getHitCount() {
        return hitCount.get();
    }
    public List<Integer> findPrimes(int count) {
        hitCount.getAndIncrement();
        // Find 'count' number of primes
        final List<Integer> primes = new ArrayList<>();
        for(int i = 2; i < Integer.MAX_VALUE; i++) {
            if(isPrime(i)) {
                primes.add(i);
                if(primes.size() == count) {
                    break;
                }
            }
        }
        return primes;
    }

    private boolean isPrime(int num) {
        for(int i = 2; i < num; i++) {
            if(num % i == 0) {
                return false;
            }
        }
        return true;
    }
} Using the atomic variable, we can ensure atomic operations on numbers and object references. Since the state of consists only of , which is thread-safe now, the class is thread-safe now! PrimeFinder hitCount PrimeFinder When a single element of state is added to a stateless class, the resulting class will be thread-safe if the state is entirely managed by a thread-safe object. Is it possible to add more state variables to PrimeFinder in a similar manner and still maintain its thread safety? Let us see… Consider the following (updated) class, PrimeFinder public class PrimeFinder {
    private final AtomicInteger numCache = new AtomicInteger(1);
    private final AtomicReference<List<Integer>> primeFactorCache = new AtomicReference<>(new ArrayList<>());
    
    // ...

    public List<Integer> findPrimeFactors(int num) {
        if(numCache.get() == num) {
            return new ArrayList<>(primeFactorCache.get());
        }
        // Find prime factors of 'num'
        final List<Integer> primeFactors = new ArrayList<>();
        for(int i = 2; i < num; i++) {
            if(isPrime(i) && num % i == 0) {
                primeFactors.add(i);
            }
        }
        numCache.set(num);
        primeFactorCache.set(new ArrayList<>(primeFactors));
        return primeFactors;
    }

    // ...

} The method is supposed to return prime factors of a number. It also stores a cache of the last number factorized and its factors. We have used atomic classes to store both the and the . findPrimeFactors numCache primeFactorCache Still, our class is not thread-safe! Can you spot the race condition here? Hint: There is a check-then-act race condition here! One invariant of our class is that the list will hold all prime factors of . Since two different variables participate in a single variant, these variables are not independent! In such cases, all the dependent variables must be updated in the same atomic operation! PrimeFinder primeFactorCache numCache To maintain state consistency, ensure that dependent state variables are updated in a single atomic operation. Intrinsic Locking in Java Every Java object can implicitly act as a lock for the purpose of synchronization. These built-in locks are what we call or . intrinsic locks monitor locks There is a built-in mechanism in Java to provide synchronization called a synchronized block. A synchronized block is a set of statements guarded by an intrinsic lock (object reference). Here is its syntax — synchronized (lock) {
  // Statements working with shared mutable state
} The lock is automatically acquired by an executing thread when it enters a synchronized block and is released when the block is exited. Since these intrinsic locks can be held by at most one executing thread at once, they allow exclusive access to the guarded code. A synchronized method is a synchronized block with the entire method as body and object as its lock. this Let us fix our class using intrinsic locks. PrimeFinder public class PrimeFinder {
    private final AtomicInteger numCache = new AtomicInteger(1);
    private final AtomicReference<List<Integer>> primeFactorCache = new AtomicReference<>(new ArrayList<>());
    
    // ...

    public List<Integer> findPrimeFactors(int num) {
        synchronized (this) {
            if (numCache.get() == num) {
                return new ArrayList<>(primeFactorCache.get());
            }
        }
        // Find prime factors of 'num'
        final List<Integer> primeFactors = new ArrayList<>();
        for(int i = 2; i < num; i++) {
            if(isPrime(i) && num % i == 0) {
                primeFactors.add(i);
            }
        }
        synchronized (this) {
            numCache.set(num);
            primeFactorCache.set(new ArrayList<>(primeFactors));
        }
        return primeFactors;
    }

    // ...

} With this, our class is thread-safe again! PrimeFinder With this, we reach the end of this blog. We discussed the fundamentals of thread safety in terms of program state and explored some basic synchronization techniques in Java. If you enjoy reading this blog, consider throwing in a like and subscribe to not miss future articles on Java concurrency! Also published . here

Safeguarding Concurrent Programs in Java: Thread Safety from an Object Oriented Perspective

Introduction Would you like to see Mona Lisa smile like a witch? Or would you like the girl with the pearl earring to wink and smile? Google has just dropped a video generation model named Lumiere [1] that's capable of doing all of it for you. Though it's primarily a text-to-video model, it's able to do much more than that. Given a reference image with a prompt, it can stylize your videos by copying the style of the reference image to your videos. You can even edit your videos with just a single prompt. The model is Lumiaire. It's even able to animate objects within a user-specified region in an image, a technique called Cinemagraphs. When it comes to inpainting, Lumiere is even able to reason about a completely missing object such as a cake in this example. It all boils down to a diffusion model with a novel space-time U-Net architecture [3]. It's a customization of the U-Net architecture to solve the problem of temporal consistency which is quite prevalent in video generation models. Visual Explanation A visual explanation of the Lumiere paper, the model architecture, and the results are available. https://youtu.be/QcYa6xxsMTU?si=IBpUqjPtj8lJDHo0&embedable=true So, What Is Temporal Consistency? We all know that videos are a sequence of images. So, let's take a sequence of images shown in the top row in the above figure. If we narrow down just one row in the image, which is indicated by the green line going from left to right, we need to see a smooth transition in the pixel values between images in the sequence. If the transition is smooth, then we won’t see a jumping effect while we are watching the video. For example, if we take Stable Video Diffusion [2], and see the video of an astronaut walking on the moon (above), we can see that his hands just disappear between frames. In other words, the temporal consistency between frames is lacking. This kind of temporal inconsistency between time and the intensity in the x direction can be plotted as the X-T Slice as highlighted in the figure above. And if there is temporal inconsistency, it is highlighted in the plot of the X-T slice. Lumiere addresses this problem by introducing a space-time diffusion model and a modified U-Net architecture present in the diffusion model. Pipeline of a Text to Video Before looking into the details, let's start with the typical pipeline of a Text-to-video generation model. These pipelines sample every 5th frame as the keyframes from the input video sequence and train a base model that can generate these keyframes at a resolution as low as 128 by 128 at just 3 frames per second. Then temporal Super Resolution is used to increase the frame rate by predicting the intermediate frames. So the frame rate now becomes 16 frames per second. The spatial resolution of these frames is then increased to say 1024 by 1024 by a spatial superresolution network (SSR) which finally leads to our generated video. The base model used in this framework is usually a diffusion model which in turn has a U-Net inside it. The Proposed Pipeline of Lumiere The proposed pipeline of Lumiere on the other hand processes all the frames at once without dropping frames. To cope with the computational cost of processing all the frames, the architecture of the base diffusion model is modified to come up with the space-time UNet architecture or STUNet. As STUNet deals with all the input frames, the need for Temporal Super Resolution or TSR is eliminated. So, the pipeline still has the spatial super-resolution or SSR. But the novelty is the introduction of MultiDiffusion. From U-Net to STUNet Let's quickly review the U-Net before looking into the space-time U-Net. The input to the U-Net is a 3 3-dimensional image with Width W, Height H, and channels RGB. After every double convolution stage of the U-Net, we apply max pooling to downsample or reduce the spatial dimension of the features. This spatial dimension reduction step is indicated by the red arrows. Similarly, during the decoder stage, there are up convolutions to increase or upsample the resolution back to the size of the input. When it comes to videos, we have an additional dimension in the input which is time. so space-time U-Net proposes to downsample and up-sample the video not only in the spatial dimension but also in the dimension of time T. This is the main idea of temporal resizing and is the main contribution of this Lumiere paper. For resizing, they use 3D pooling instead of 2D pooling as the input now has an additional dimension. Like me, you may be surprised by the simplicity of the idea. The authors themselves have mentioned in the paper: Surprisingly, this design choice has been overlooked by previous T2V models, which follows the convention to include only spatial down and up-sampling operations in the architecture, and maintain a fixed temporal resolution across the network. Implementation Let's get to some of the nuances of the implementation. They use factorized convolution introduced in this paper called Video Diffusion Models. The idea is to change each 2D convolution into a space-only 3D convolution, for instance, by changing each 3x3 convolution into a 1x3x3 convolution. For the attention, after each spatial attention block, we insert a temporal attention block that performs attention over the first axis and treats the spatial axes as batch axes. With those two changes, the factorized convolution blocks are added to the pre-trained model, and only the additional layers are trained with the pre-trained layer weights fixed. The second novelty of the paper is the MultiDiffusion introduced during the spatial superresolution. If you take the video generation models before lumiere, the spatial superresolution model takes a sequence of frames. However, the sequences were not overlapping. For example, the first 8 frames and the next 8 frames taken as input by the SSR module are separate without any overlap. But when it comes to Lumiere, the first 8 frames and the second 8 frames have an overlap of two frames. By doing this, the spatial super-resolution model seems to achieve smooth transitions between the temporal segments. This is what is referred to as multiDiffusion in the paper. Applications Coupling the two proposed techniques along with the absence of a cascaded diffusion model architecture which is prevalent in prior architectures such as imagen video, leads to quite a few varied applications. For example: The model can convert text to videos with prompts such as “an astronaut walking on planet Mars making a detour around his base” or “a dog driving a car wearing funny sunglasses.” It can convert images to videos along with a text prompt such as “A girl winking and smiling.” It can stylize generation with a reference image and a text prompt such as “a bear dancing.” Talking of Cinemagraphs, it can animate regions selected by the user such as fire or steam. It can even edit dresses that people are wearing with just a single prompt. Evaluation To evaluate the model quantitatively, the model was run through a user study where the users compared the results of the proposed model against some state-of-the-art models such as Pika, ZeroScope, or stable video diffusion. The results indicate that the users preferred the Lumiere model both in the case of text to video and image to video. Conclusion So, to conclude, other than all the promotional video stunts, such as the smiling Monalisa, the contribution of this paper is fairly simple. Put in one line, the paper introduces a downsampling of the temporal channel. This coupled with MultiDiffusion, which is nothing but having overlapped frames fed into the superresolution model, generates high-fidelity videos that are temporally consistent. What I would have liked to see is some ablation studies in the paper showing the results with and without the multiDiffusion process. That brings us to the end of this article. The next time someone talks to you about Lumiere, you know what to say in one line. I hope that provided some insight into the Lumiere model. I will see you in my next, until then, take care… References [1] Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, Oliver Wang, Deqing Sun, Tali Dekel, Inbar Mosseri, (2024), arXiv preprint. A Space-Time Diffusion Model for Video Generation [2] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, Robin Rombach, (2023), arXiv preprint. Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets [3] Olaf Ronneberger, Philipp Fischer and Thomas Brox, (2015), International Conference on Medical Image Computing and Computer-Assisted Intervention. U-Net: Convolutional Networks for Biomedical Image Segmentation Also published here

Google Unveils Its Most Promising Text-to-Video Model Yet: Lumiere

গুগল এখনও পর্যন্ত তার সবচেয়ে প্রতিশ্রুতিশীল পাঠ্য-টু-ভিডিও মডেল উন্মোচন করেছে: লুমিয়ের

Google stellt sein bisher vielversprechendstes Text-to-Video-Modell vor: Lumiere

Google presenta su modelo de texto a video más prometedor hasta el momento: Lumiere

Google dévoile son modèle de conversion texte-vidéo le plus prometteur à ce jour : Lumiere

Google ने अपना अब तक का सबसे आशाजनक टेक्स्ट-टू-वीडियो मॉडल लॉन्च किया: लुमिएरे

Google、これまでで最も有望なテキストからビデオへのモデル「Lumiere」を発表

Google은 지금까지 가장 유망한 텍스트-비디오 모델인 Lumiere를 공개했습니다.

Google revela seu modelo de conversão de texto em vídeo mais promissor: Lumiere

Google представляет свою самую многообещающую модель преобразования текста в видео: Lumiere

Google Şimdiye Kadarki En Umut Verici Metinden Videoya Modelini Açıkladı: Lumiere

Google công bố mô hình chuyển văn bản thành video hứa hẹn nhất: Lumiere

谷歌推出迄今为止最有前途的文本转视频模型：Lumiere

In the dawn of time, long before the world unfurled its mysteries, the realm of mathematics concealed a plethora of treasures and secrets. Ancient, primordial laws seemed to govern the realm of numbers since time immemorial, revealing its most enchanting wonders through the dance of prime numbers. You are likely acquainted with the concept of primes—numbers divisible only by 1 and themselves. Among them, we encounter such distinguished figures as 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, and 31. (It's worth noting that 1 is not considered prime, and one might argue that 2 and 3 deserve unique categorization). Among the intriguing categories of prime numbers, we find the enigmatic "twin primes"—pairs of primes differing by a mere 2, such as 5-7, 11-13, 17-19, and so forth. For now, let's keep these notions in mind: prime numbers and their various categories, each showcasing specific similarities. Some may appear disorderly until their underlying patterns are unveiled. But until then... What is a prime? The concept of prime numbers serves as a method to delineate the number line without leaving any voids. Primes can be regarded as the fundamental building blocks of all numbers. Take, for instance, the number 2. Multiplying 2 by itself yields 4, then 6, 8, 10, 12, and so forth. Consequently, no even number can be prime, as it inevitably results from combining 2 with another number. We now understand that prime numbers, in their most foundational form, adopt the structure of odd numbers (2n+1). I opt for the 2n+1 format because, if we were to choose 2n-1, we would encounter -1 when considering 0. This may not necessarily violate any rules, as in the case of 2n-1, with n as 1, we obtain the result of 1, which is charming, but uncertainties persist. Now, we have illustrated that the essence of primes, in its most general and universal form, is 2n+1 (or -). Concerning twin primes, if we locate the midpoint between them, we discover that (2n+1)+(2n+3) equals 4n+4. (4n+4)/2 is 2n+2. This midpoint, as gleaned from numerous pattern-seeking endeavors, is consistently divisible by 6 and 9. Further division by 2, yielding n+1, reveals that n+1 is always divisible by 3, 6, and 9. The proportions among their divisors are remarkably balanced, with the difference between divisors of 3 being approximately ±5% more or less than 6 and 9, and the same for 6 and 9 in relation to others, respectively. But then, what do we find when seeking patterns? What exactly do we want to achieve? The patterns in our interactions I won't delve into the etymology of the word "pattern,” but I am willing to explain what it simply means to me. Patterns are self-repetitive structures or elements that define a whole structure. The emergence of patterns can be observed by an external observer focusing on the object or behavior itself—a means to predict outcomes. But why do we seek to predict prime numbers? What is so special about them? Why are they extensively used? Here, we step into the realm of cryptography and the way primes function as an unbreakable cipher meant to resist decryption. Their resilience is anchored in the difficulty of factoring these primes into smaller primes. This is a fundamental principle in the field of cryptography. The security of certain cryptographic algorithms, such as RSA (Rivest–Shamir–Adleman), relies on the difficulty of factoring the product of two large prime numbers back into their original primes. This process is known as integer factorization. The larger the prime numbers involved, the more computationally challenging it becomes to factorize their product, making it difficult for unauthorized parties to decipher encrypted messages. This property forms the basis for the security of many cryptographic systems that use prime numbers in their algorithms. Now, of course, a way of finding and playing with every single prime would indeed be an interesting avenue. But what could it mean in its very essence? What if we choose the envision this play differently? Exploring Primes as an Unbreakable Form of Numbering: Imagine thinking of prime numbers in a similar way to squares. For instance, take the number 7. You can represent it as a sum of 7 smaller squares. Now, the interesting thing is that these small squares cannot be divided into any smaller ones. Unlike other numbers, there's no way to break them down further. The analogy with Squares: Consider the idea of "folding" a number. Take the number 8 as an example. You can fold it into 2 x 4 squares, or even 4 x 2 squares, effectively changing its shape. This folding and reshaping are possible for certain numbers. The Concept of Folding: When it comes to prime numbers, this folding or reshaping is impossible. The shape of a prime number is always static, and there's no known way to alter it. This unique property suggests that primes could be used to create a new numbering system. Unalterable Nature of Primes: Now, you might wonder, what benefits could we derive from such a system based solely on primes? One interesting application could be in communication. For example, if we send signals in the form of primes from one point to another, the receiving end would immediately recognize that the message is based on a prime number. It could then intelligently select the most similar prime to interpret the message. This process could lead to more efficient noise reduction, filtering out irrelevant information. Potential Benefits: By considering primes as unalterable shapes in a numbering system, we open up possibilities for unique applications. The analogy with squares helps illustrate the distinct nature of prime numbers. The idea of using primes in communication suggests a potential avenue for creating a system where the unbreakable aspects of primes contribute to efficient information transmission and noise filtering. In Summary: I like how clearly ChatGPT explains some ideas. However, who is the one who arrives at those ideas? Well, you. The one who simply sits, reads, and learns something new every day. Not for the mere purpose of learning, but to use what you learn to learn something completely new. The end? As we delve into the profound realm of prime numbers, a tantalizing question arises: Can we devise a form of information that harnesses the intrinsic properties of primes, effectively mitigating the impact of noise interference? Such a revolutionary system might demand larger primes with wider intervals, considering the heightened differences between them. Remarkably, even for global communication, primes up to 10,000 could prove sufficient based on our existing knowledge. However, as complexity beckons, the necessity for larger intervals becomes apparent. Yet, this challenge invites us to redefine not only letters but entire concepts. Picture a scenario where, regardless of distance, we exchange ideas through a series of carefully crafted formulas—each recipient deciphering the concepts within their knowledge domain. The world of primes unveils not just mathematical wonders but an intriguing prospect for reshaping the very nature of information transfer.

Prime Numbers: The Unbreakable Building Blocks of Mathematics

Tranformer-based vision models are gradually evolving and are reported to be as good as Convolutional models on Classification, Segmentations and Action Recognition tasks. We have a whole array of Convolutional models for vision tasks and are more popular than Transformer-based ones. This blog delves into the SWin Transformer Vision model that was presented in the International Conference for Computer Vision (ICCV) 2021 by the Microsoft Research Team. It  benchmarks its performance against several SOTA Convolution-based models on the Dog Breed Image Classification task. Will transformer-based models become the next big thing in Computer Vision? With transformers being a successful solution for language tasks, will it unify the various AI subfields and present powerful solutions to more complex problems? So rolling up my sleeves to evaluate how good they are on the classification task to begin with. Chosen Data and its Problem Statement: The myriad of dog breeds with subtle changes in their physical appearances have been a challenge to veterinarians, dog owners, animal shelter staff and potential dog owners in identifying their right breed. They need to identify the breed in order to provide appropriate training, treatment and meet their nutritional needs.The data is sourced from the that contains ~20K images of 120 breeds of dogs across the world. This data has been split almost equally into train and test set for a Kaggle Competition . Stanford Dog Dataset Dog Breed Identification The objective of the solution is to build a dog breed identification system capable of correctly identifying dog breeds with minimal data and rightly identifying even similar looking dog breeds. This is  a multi-class classification task and for every image, the model has to predict probability for all the 120 breeds. The one with the highest probability is the most probable breed of the dog in the image. Exploratory Data Analysis For a breed, the training data has a maximum of 120+ images and a minimum of 66 images. Each class, on an average is expected to have 85 images. Looking at the bar plot, it is concluded that there are no class imbalances. Though there is no class imbalance, data may be insufficient to train the neural network. Image Augmentation using random image perturbations and pre-trained models will be able to circumvent this problem. Top 5 breeds with the most images are scottish_deerhound, maltese_dog, afghan_hound, entlebucher and bernese_mountain_dog. Bottom 5 breeds with the least images are golden_retriever, brabancon_griffon, komondor, eskimo_dog and briard. A quick analysis on spatial dimensions of the training images is done to understand the distribution of image height, width and their aspect ratios. Images with very low (<0.5) and very high (>2.3) aspect ratios are considered to be anomaly images. 1.5 is considered to be a good aspect ratio. In analysing various dog breeds, below pairs of breeds were generally found to look alike. 'boston_bull', 'french_bulldog' 'beagle', 'english_foxhound' 'bernese_mountain_dog', 'greater_swiss_mountain_dog' 'malamute', 'siberian_husky' 'basenji', 'ibizan_hound' 'vizsla', 'rhodesian_ridgeback' 'lhasa', 'shih-tzu' 'whippet', 'italian_greyhound' 'brittany_spaniel', 'welsh_springer_spaniel', 'blenheim_spaniel' 'malinois', 'german_shepherd' 'border_collie', 'collie' 'norfolk_terrier', 'norwich_terrier’ Evaluation Metrics LogLoss, which stringently evaluates the confidence of the model predictions for all 120 classes by comparing the prediction probability to the ground truth probability, is the key metric. N*N confusion matrix is used to study the outcome of the predictions to analyse if there are any pair of classes confused by the model and incorrectly predicted. SWIN - Transformer Based Model This architecture is based on developed by Microsoft Research Team. This paper discusses an improved ViT architecture that produces a hierarchical representation of feature maps reducing the computation complexity of self-attention mechanism from quadratic to linear. It is proven to give identical results as SOTA Convolution Networks like EfficientNet on Imagenet classification problem. “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows” The building blocks of this architecture is explained in the below notes: Image Patching: In NLP, the tokens which are the processing elements of a model, are the words in a sentence where the size of 1 token is 1 (just 1 word). ViT ( sion ransformers) treat “image patches” as tokens where each patch is a partition of an image consisting of a group of neighbouring pixels. Each image patch is a token. The size of 1 token in any ViT is patch_height * patch_width * number of channels. Based on the patch dimensions, we get a number of patches or tokens for a single image. If the image size (H*W*3) is 224 * 224 * 3 pixels and the patch size is 4 * 4, then we get 224/4 * 224/4 = 56 * 56 patches or 3136 tokens from the image. Each token or patch will be of size 4*4*3 = 48 dimensions or 48 pixels of data. So the input to the architecture for this image consists of 3136 tokens each of size 48 dimensions. Vi T The underlying mechanism of the SWIN transformer is analogous to any CNN based architecture where the spatial dimensions of the image is decreased and the number of channels is increased. SWIN transformer at every stage in the hierarchical architecture, also reduces the number of image patches or the number of tokens while increasing the token dimensions. With this mechanism in mind, it is easier to understand the SWIN architecture easily. At every stage of the architecture, we can see the number of tokens decreasing while the token size is increasing. The SWIN-T architecture, apart from the “ is also made up of 3 other building blocks - Linear Embedding, Swin Transformer Block, Patch Merging. These building blocks are repeated and it process feature maps in a hierarchical fashion. Patch Partitioner”, Linear Embedding: The 3136 tokens each of 48 dimension from the “Patch Partitioner” are fed to a feed forward layer to embed the token of 48 feature vector into a feature vector of size ‘C’. ‘C’ here acts as the capacity of the transformer and the SWIN architecture has 4 variants based on it. Tiny (Swin-T)    - C is ‘96’ Small (Swin-S) - C is ‘96’ Base(Swin-B)   -  C is ‘128’ Large(Swin-L)  - C is ‘192’ Image patching and linear embedding are jointly implemented in a single convolution  whose kernel-size and stride-length is same as the patch-size. Number of channels in the convolutional will be ‘C’. SWin Transformer Block: The SWin Transformer Block is different from the standard transformer block in the ViT architecture. In SWin Transformers, the layer is replaced by either the or the . Multi-head Self Attention (MSA) Window MSA (W-MSA) module Shifted Window MSA (SW-MSA) module Stage1 consists of 2 SWIN-T Transformer Blocks (refer image) where the first Transformer Block has Window MSA (W-MSA) and the second Transformer Block has Shifted Window MSA (SW-MSA) module. In the SWin Transformer Block, the inputs and the outputs of the W-MSA and SW-MSA layers are passed via Normalization Layers. It is then subjected to a 2 layered Feed Forward Network with Gaussian Error Linear Units (GELU) activation. There are residual connections within each block and between these 2 blocks. Window MSA (W-MSA) and Shifted Window MSA (SW-MSA) modules Why is the standard attention layer in ViT replaced with the Windowed MSA layer? The standard attention layer in ViT was a global one calculating the attention of a patch with all other patches in the image thus leading to a quadratic complexity proportional to the image dimensions. This doesn't scale very well for high resolution images. The self-attention mechanism in the W-MSA or SW-MSA module is a local one that calculates self-attention only between patches within the same window of the image and not outside the windows. Windows are like larger partitions of the image where each window comprises of M*M patches. Replacing global self-attention with local self-attention reduced the computational complexity from quadratic to linear. Why both W-MSA and SW-MSA for local self-attention? What is the  difference between them? The key difference between W-MSA and SW-MSA attention modules is in the way how the windows for the image are configured. In W-MSA module, a regular window partitioning strategy is followed. The image is evenly partitioned into non-overlapping windows starting from the top-left pixel of the image, and each window contains M*M or M2 patches . In SW-MSA module, the window configuration is shifted from that of the W-MSA layer, by displacing the windows by (M/2, M/2) patches from the regular partitioning strategy. Why a shifted window partitioning strategy in SW-MSA? Since the attention is restricted locally within a window in W-MSA, the shifted window enables cross-window attention to still yield the benefits of global attention. This is possible because the boundaries of window1 in W-MSA layer are shared with windows W2, W4 and W5 in SW-MSA layer. Hence global attention happens indirectly via “local attention on shifted windows”. Patch Merging Layer Patch Merging layer reduces the number of tokens as the network gets deeper and deeper. The first patch merging layer concatenates the features of each group of 2×2 neighbouring patches. SWIN Transformers on Dog Breed Classification The package in PyPI  has pretrained TF-Keras variants of the SWIN Transformers and is built based on the . Its code is available in . tfswin is used to train the dog breed images. tfswin official pytorch implementation github from tfswin import SwinTransformerBase224, preprocess_input
def build_model1(swintransformer):
  tf.keras.backend.clear_session()

  inputs = Input(shape=(resize_height, resize_width, 3))

  outputs = Lambda(preprocess_input)(inputs)
  outputs = swintransformer(outputs)
  outputs = Dense(num_classes, activation='softmax')(outputs)

  swin_model = Model(inputs=inputs, outputs=outputs)

  return swin_model

#build the model
swintransformer = SwinTransformerBase224(include_top=False,pooling='avg')
swin_model1     = build_model1(swintransformer)

#set the layers of the pretrained model as non-trainable
for layer in swin_model1.layers[2].layers:   
  layer.trainable = False

swin_model1.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),loss='categorical_crossentropy',metrics=['accuracy']) Convolutional Based Models To begin with, various ResNet variants were trained with different replacements for top layers, and partially to totally freezing the network. #Logloss of the test set using various ResNet variants
+------------+---------------+-------------------------+----------+
| Model Name |   Retrained   |  Top Layers Replacement | Log_Loss |
+------------+---------------+-------------------------+----------+
|  ResNet50  |      None     |   ConvBlock_FC_Output   | 0.96463  |
|  ResNet50  |      None     | GlobalAvgPooling_Output | 0.58147  |
|  ResNet50  | last 4 layers |   ConvBlock_FC_Output   | 2.10158  |
|  ResNet50  | last 4 layers | GlobalAvgPooling_Output | 0.57019  |
+------------+---------------+-------------------------+----------+ Code corresponding to the ResNet50 model with least log loss from tensorflow.keras.layers import Input,Conv2D,Dense,BatchNormalization,Flatten,Concatenate, Dropout,MaxPooling2D
from tensorflow.keras.models import Model
from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input

def build_model():

  tf.keras.backend.clear_session()

  inputs = Input(shape=(resize_height, resize_width, 3))

  #added preprocess_input method as a layer to convert input images to those expected by Resnet
  processed_inputs = preprocess_input(inputs)  

  #use the pretrained ResNet model (Parameter pooling = 'avg' will take care of the Gobal Average Pooling of the ResNet model features)
  base_model = ResNet50(weights="imagenet", include_top=False,pooling='avg')(processed_inputs) 
  
  #output layer
  output = Dense(units=num_classes,activation='softmax',name='Output')(base_model)

  resnet_model = Model(inputs=inputs, outputs=output)
  return resnet_model

#build the model
resnet_model = build_model()

#set the layers of the resnet pretrained model as non-trainable except for its last 4 layers which needs to be re-trained for this data
for layer in resnet_model.layers[3].layers[:-4]:   
  layer.trainable = False

#compile the model
resnet_model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),loss='categorical_crossentropy',metrics=['accuracy'])
print(resnet_model.summary())

history = resnet_model.fit(train_ds,
                    epochs=50,
                    validation_data=val_ds, callbacks=callbacks_list) More sophisticated pre-trained Convolutional models like EfficientNet, NASNet, InceptionResNet, Xception and InceptionV3 were also trained using the dog breed images both in standalone and also in an ensembled fashion. The log loss on the test data (consisting of 10K images) from these models were lower than any of the ResNet variants. #Logloss of the standalone model variants
+----------------------------+-------------+
|         Model Name         |   Log_Loss  |
+----------------------------+-------------+
|      EfficientNetV2M       |   0.28347   |
|      Inception ResNet      |   0.28623   |
|        NasNetLarge         |   0.33285   |
|          Xception          |   0.34187   |
|        Inception_V3        |   0.54297   |
| EfficientNetV2M_GlobalAveg |   0.50423   |
|   InceptionV3_GlobalAveg   |   0.46402   |
+----------------------------+-------------+ For standalone models, the layers of the pre-trained models were frozen and their top layers were replaced by either Convolutional Layer followed by MaxPooling Layer Global Average Pooling Layer EfficientNet architecture is a rightly scaled model in all 3 dimensions - depth, width and resolution. NASNet-Large is the outcome of model architecture search done using various hyperparameter search algorithms, optimization algorithms and reinforcement learning algorithms. InceptionV3 factorized larger convolutions into smaller asynchronous convolutions and also included auxiliary classifiers to improve model convergence. Xception architecture does depth-wise separable convolutions instead of the inception modules in the Inception architecture. The log loss from the ensemble models reduced by > 0.1 than the best performing standalone model. +--------------------------------------------------------------------------+-----------+
|                             Model Name                                   |  Log_Loss |
+--------------------------------------------------------------------------+-----------+  
| Ensemble1 - EfficientNEt,InceptionResNet,NasNet,Xception)                |  0.17363  |
| Ensemble2 - EfficientNEt,InceptionResNet,NasNet,Xception and InceptionV3 |  0.16914  |
| Ensemble3 - Ensemble2 with 50% dropout.                                  |  0.16678  |
| Ensemble4 - Ensemble of various EfficientNet Architecture                |  0.16519  |
+--------------------------------------------------------------------------+-----------+ Each of these models accepts varied input formats and in Keras they have their own preprocessing functions. Benchmarking Outcome +----------------------------------+------------+----------------------+----------+
|            Model Name            | Parameters | Train time (seconds) | Log_Loss |
+----------------------------------+------------+----------------------+----------+
|  EfficientNet_ConvBlock_Output   |   54.7M    |        ~260s         | 0.28347  |
| InceptionResNet_ConvBlock_Output |   56.1M    |        ~260s         | 0.28623  |
|   NASNetLarge_ConvBlock_Output   |   89.6M    |        ~330s         | 0.33285  |
|    XCeption_ConvBlock_Output     |   23.3M    |        ~240s         | 0.34187  |
|   InceptionV3_ConvBlock_Output   |   24.2M    |        ~225s         | 0.54297  |
|      EfficientNet_GlobalAvg      |   53.3M    |        ~260s         | 0.50423  |
|      InceptionV3_GlobalAvg       |    22M     |        ~215s         | 0.46402  |
|           swin_base224           |   86.8M    |        ~550s         | 0.47289  |
|           swin_base384           |    87M     |        ~600s         | 0.41902  |
|          swin_large384           |    195M    |        ~1000s        | 0.42207  |
+----------------------------------+------------+----------------------+----------+ SWIN Transformers have performed better than all of the ResNet50 variants and InceptionV3 model. Log-Loss of SWIN Transformer on this data is slightly higher compared to InceptionResNet, EfficientNet, Xception and NasNet Large models when their outputs are processed subsequently by convolutional layer followed by Maxpooling. SWIN however performs is as good as EfficientNet model when their average outputs are directly processed by the output layer. SWIN models are larger compared to any of the Convolutional models and hence will take a hit on the system throughput and latency. This study proved useful in understanding the application of transformer-based models for computer vision. You can find the code to the notebooks on my and the GUI for this usecase here. Github References: https://www.kaggle.com/competitions/dog-breed-identification https://arxiv.org/pdf/2103.14030.pdf https://www.youtube.com/watch?v=tFYxJZBAbE8&t=362s https://towardsdatascience.com/swin-vision-transformers-hacking-the-human-eye-4223ba9764c3 https://github.com/shkarupa-alex/tfswin

Image Classification in 2022

Clasificación de imágenes en 2022

Classification des images en 2022

2022 में छवि वर्गीकरण

2022 年の画像分類

Classificação de imagens em 2022

Phân loại hình ảnh năm 2022

2022年的图像分类

Can software project estimation be as realistic as possible? This article will walk you through how we, at UpsilonIT , approach the estimating process. We will also share some hands-on techniques and useful tips on making it more accurate and robust. Why Estimation Is an Essential Stage of Any Software Project? Many outsourcing software projects fail because their estimate is not reliable. Failure means the developer did not meet the deadline, overran the budget, or did not deliver what was expected. A study conducted by found that 66% of software projects have cost overruns. A third of them go beyond the estimated schedule, and almost 20% of them fall short of promised benefits. McKinsey Sometimes it seems that the process of estimation can significantly increase the cost of the pre-development phase. But it is crucial to realize: the resources spent at the initial stage of the project are a good investment in the outcome. The more accurate the estimation, the more customer-satisfying the result will be. That is why a correct estimate is a crucial part of project management procedures, and it is really worth it. What Is Project Estimation? Project estimation is a process of calculating and predicting the costs necessary for its successful realization. In software development, estimations imply making predictions regarding the most realistic scale of efforts that should be put into developing a software product. This information allows suggesting the amount of money that needs to be invested in the project, the number of specialists who would work on the latter, and, finally, the time required for all that. Accurate Estimating: Complicated but Crucial Task As estimates are based upon incomplete, imperfect knowledge and assumptions about the future, they all have uncertainty. There is no such thing as a precise, single-value estimate. A more traditional ‘waterfall’ approach implies that all the software project specifications must be provided upfront before the estimation process starts. But now, everyone is going agile. Stakeholders want to begin immediately after preparing brief requirements and then add new features literally on-the-go. MVPs (Minimum Viable Products) are released quickly, feedback is obtained, and improvements are made iteratively. In those conditions, planning and estimating can be rather challenging. So the question we want to answer is, This article will walk you through how we approach the software project estimation process at UpsilonIT and share some tips that help us make it more accurate and robust. ‘How can we cover all those uncertainties and make the estimation as realistic as possible?’ Estimation Process: How We Run It at UpsilonIT Let’s look at the whole process of project analysis and estimations at UpsilonIT — from the moment of figuring out the client’s expectations to giving the final project calculation. 1. Preparation We always start by digging into the client’s main challenges to paint a clear picture of their business and development goals. It’s important to gather as much information on the future project as possible. There are two main sources where we get it: Online video calls via Skype, Zoom, or Hangouts: we discuss the project in general by asking questions and getting answers Documents and data provided by the client: specifications, mockups, wireframes, use cases, user stories, software specifications, etc. All of this helps us estimate the cost of building the client’s product more precisely. The more information we get our hands on at this stage, the more accurate our estimation will be. Here is a list of things we usually try to clarify at the very beginning: What is the pain that you are solving for your customers? What are the main features of the product? What platforms will the product be based on (web, mobile, back-end)? What existing components or parts of the system do you already have? Can you describe the main use cases of the new application? Can you prioritize the functions of the future product? Do you have a specific technical stack that needs to be used for this project?What is your expected timeframe for the project? What is your expected budget for this project? Once we are sure that nothing is missing and that we’re in line with the client’s expectations, we go to the next phase: assembling the team. 2. Assembling the Right Time After collecting all the information, we gather professionals who have the most extensive competence in the respective domain. The team responsible for the project estimation is usually composed of senior-level specialists and includes: CTO Senior back-end developer Senior front-end developer Other specialists upon request: a project manager, UX/UI designer, QA specialist, technical writer, and others. In the vast majority of cases, specialists who make the estimation subsequently become a part of a project development team. The key criteria for selecting that team’s members are: Professional expertise: the relevant experience of working with specific technologies and similar projects. Ability to start the project as soon as possible Other factors: presence in the client’s timezone, full- or part-time employment, language proficiency, etc. Pro Tip: Assembling the right team is one of the most sensitive questions in software outsourcing. A customer wants to be ensured that the most skilled specialists with relevant expertise will be assigned to the project. That will guarantee the expected quality and hassle-free delivery of the final product. Also, customers want to see ‘real faces’ not just abstract ‘senior front-end developers’ in the estimations, including in order to understand further whom they are speaking to on the conference call. That is why every estimate made by UpsilonIT is accompanied by a document of the CV format with key information about each team member, including his or her: Expertise Skill matrix Project history Personal info 3. Research & Analysis During this phase, the team members set up a number of meetings to discuss project requirements, exchange their opinions, and brainstorm ideas. If some details need more clarification, the discussions may alternate with the research sessions. At the end of the Research & Analysis phase, our team provides potential clients with the following deliverables: High-level architecture & infrastructure design Recommended tech stack Mockups and wireframes (if not provided by a client) Desired breakdown of the team Rough timelines 4. Defining Scope & Size of the Project The next step of our estimation process is building the scope of work that covers software requirements. At UpsilonIT, we use the Work Breakdown Structure (WBS), which visualizes each software development stage with related tasks. To achieve the project’s goal, we split it into milestones, which are broken down into tasks, subtasks, etc. At UpsilonIT, we define a milestone as a marker that indicates an achievement or a deliverable, for example, the completion of a large task or reaching a phase in a project. For most of the projects, we figure out the following milestones: Discovery phase: documentation analysis, choice of 3rd party software tools, decisions on possible integrations, preparation of wireframes (if not provided by a client), etc. Design phase: collaborative work with a product owner that can include design theme workshops, brand identity design, UX and UI design, prototyping, etc. Project setup phase: setting up of infrastructure, deployment of environments, configuring of security, etc. Development phase: development of functionality based on user stories (and also tasks grouped into milestones). It can include user authentication, messaging, payment integration, mapping, video streaming, reporting, etc. Testing phase: functional testing, usability testing, user acceptance testing (UAT). Delivery phase: transfer of documentation and intellectual property, deployment, and implementation of the solution, including publishing on marketplaces, training, presentation of the product, if necessary, and more. How does it work in practice? Here you can see an example of how we broke down into milestones and user stories the structure of a health & wellness digital platform. Another significant question is to decide whether to build a turnkey solution or start from the Minimum Viable Product (MVP). About 80% of startups and SMEs we worked with had an MVP first. From our experience, we recommend to ignore non-primary functions by creating an MVP before the finished product, especially if: Project requirements are unspecified Product market fit is not validated yet Features must be added and tested quickly Budget is limited Pro Tip: When defining the scope of the project, it can be very helpful to visualize everything needed to be accomplished, organize tasks into multiple levels, and display these elements graphically. For that purpose, we use the Miro platform. With Miro’s help, we develop the project’s hierarchical structure that makes it easier to oversee a complex project and make sure every task gets done. When creating a WBS for a future project, we follow four fundamental principles: Hierarchy: every ‘child’ on the graph has a hierarchical relationship with its parent task. 100% rule: every level of the graph must make up 100% of the ‘parent’ level, and it must have at least two ‘child’ elements. Mutually exclusive elements: every element at each level has to be mutually exclusive — there can’t be any overlap between deliverables or work. Outcome-oriented: graphic depiction must focus on the outcomes rather than the activities required to produce them. 5. Project Estimate by PERT Method After conducting a detailed scope breakdown, we start to create a more accurate estimate using a PERT (Program Evaluation and Review Technique) method. We calculate a realistic cost estimation using an , , and a within this approach. There are different ways of interpreting the numbers depending on how we want to weight potential risks, but in most cases, we use the formula: optimistic case estimate a pessimistic case estimate most likely case estimate (optimistic + pessimistic + (4 x most likely))/6 = expected estimate Let us consider an example to see how these formulas work. Let us assume we have to calculate the expected travel time it takes to deliver a shipment from point ‘A’ to point ‘B’. There could be 3 different scenarios: roads would be free of traffic congestion, and there will be no stopping at traffic signals. Optimistic scenario: there would be severe traffic bottlenecks (may be due to a major accident), or there will be some unscheduled stops (may be due to vehicle breakdown). Pessimistic scenario: there would be normal traffic conditions. Most likely scenario: Let us assume that our 3 estimates (optimistic, pessimistic & most likely) were 450 minutes, 2250 minutes, and 900 minutes respectively. Putting these values in the formulas, we get: Expected travel time = (450 + 2250 + 4 x 900))/6 = 1050 minutes It essentially means that, in most cases, the expected travel time will take 1050 minutes. How can it look in a real project estimate? Let us explain how we estimated the task by PERT method for a social sports application. ‘Credits (Bank) & Prize claim’ 7 days (including 7 days for back-end and 6 days for front-end development. Both teams work simultaneously). Optimistic scenario: 14 days (including 14 days for back-end and 12 days for front-end development). Pessimistic scenario: 11 days (including 11 days for back-end and 10 days for front-end development). Most likely scenario: Expected estimate: (7 + 14 + (4 x 11))/6 = 10.83 days At the output, we get an estimate which takes into account some uncertainty and risk in tasks. The PERT method can be quite time-consuming, but it can provide a high degree of accuracy. 6. Building a Timeline Chart After determining the total time needed for each task and milestone, we build the project timeline. We line tasks end-to-end, adjust their lengths to reflect the amount of time allotted, and then add milestones. We turn that all into a solid timeline chart with expected delivery dates provided. It is especially helpful to have such visualization on projects where several teams work simultaneously on several milestones. Such a timeline chart shapes the entire project, indicating which tasks are to be completed at which stage and which resources need to be involved along the way. It also allows us to quickly see the proportion of time allocated to each project task or milestone and provides a visual interpretation of how the project will run phase by phase. 7. Calculating Costs After estimating the number of hours of work required to complete the defined scope, it’s time to price the project. We calculate the total project cost by multiplying the number of man-hours (estimated using the PERT method) by the applicable rates. For ease of calculation, let’s make the following assumptions: One man-hour = work completed in an hour of uninterrupted effort by a specialist. One day = a standard 8-hour working day. In IT projects, hourly rates range depending on the specialist’s professional category, which is defined by his or her job functionality, level of seniority, uniqueness of skills, etc. Of course, it will not cost the same for one hour of a junior programmer as a senior one. Therefore, when calculating the cost of the project, we take into account that the multiplication of the cost must be factorized by the hourly rate of each professional category: Total cost = (RH1 x H1) + … + (RHn x Hn) In this formula, RH1 is equal to the hourly rate of the professional category #1, and H1 is the total number of hours estimated by the professional category #1. How does it work in practice? For example, we estimated a software project and got the following numbers: The total cost of the project will be: 50x180 + 40x150 + 30x60 = $16800 8. What’s Next? Having put all things together and made our calculations, we share it with the client in person, explaining our estimation instead of just sending over a number without context. We schedule a meeting and go over the document with them, row by row. It allows us to discuss any questions that may have come up along the way. If it becomes clear that the project isn’t doable within the client’s limitations, we can also help to prioritize the functionality and start by developing a minimum viable product (MVP) or even a proof-of-concept (PoC). We try to include this kind of out-of-the-box thinking in our estimates whenever possible, to add value at every step of our collaboration. Conclusion Estimating is prone to error. The overall complexity becomes fully visible only at the point of execution. In the conditions of uncertainty and lack of information, there is a temptation to provide lower numbers to outbid the competition. But that doesn’t mean that the final price for your product will be lower. It may just mean more stress, more unforeseen costs along the way, and as a result, your dissatisfaction. And this is definitely not how great products are built. Our goal is to predict as closely as possible. Of course, some things can change, especially if a project evolves unexpectedly or if a client comes up with new ideas during development. But in most cases, when we give an estimate, we stick to it. The best way to find this out is to partner with us. You can follow our processes, share your thoughts with our developers, and enjoy our transparent communication and quality approach firsthand. We will be happy to do our best to make your ideas happen. Follow UpsilonIT on Hacker Noon for the latest tech know-hows, best software development practices, and insightful case studies from industry experts! Remember: your likes and comments are fuel for our future posts!

Accurate Project Estimations: How To Make Them?

Embeddings have pervaded the data scientist’s toolkit, and dramatically changed how NLP, computer vision, and recommender systems work. However, many data scientists find them archaic and confusing. Many more use them blindly without understanding what they are. In this article, we’ll deep dive into what embeddings are, how they work, and how they are often operationalized in real-world systems. What’s an embedding? To understand embeddings, we must first understand the basic requirements of a machine learning model. Specifically, most machine learning algorithms can only take low-dimensional numerical data as inputs. In the neural network below each of the input features must be numeric. That means that in domains such as recommender systems, we must transform non-numeric variables (ex. items and users) into numbers and vectors. We could try to represent items by a product ID; however, neural networks treat numerical inputs as continuous variables. That means higher numbers are “greater than” lower numbers. It also sees numbers that are similar as being similar items. This makes perfect sense for a field like “age” but is nonsensical when the numbers represent a categorical variable. Prior to embeddings, one of the most common methods used was one-hot encoding. One-Hot Encoding One-hot encoding was a common method for representing categorical variables. This unsupervised technique maps a single category to a vector and generates a binary representation. The actual process is simple. We create a vector with a size equal to the number of categories, with all the values set to 0. We then set the row or rows associated with the given ID or IDs to 1. An illustration of One Hot Encoding This technically works in turning a category into a set of continuous variables, but we literally end up with a huge vector of 0s with a single or a handful of 1’s. This simplicity comes with drawbacks. For variables with many unique categories, it creates an unmanageable number of dimensions. Since each item is technically equidistant in vector space, it omits context around similarity. In vector space, categories with little variance are not any closer together than those with high variance. This means that the terms “Hotdog” and “Hamburger” are no closer together than “Hotdog” and “Pepsi”. As a result, we have no way of evaluating the relationship between two entities. We could generate more one-to-one mappings, or attempt to group them and look for similarities. This requires extensive work and manual labeling that’s typically infeasible. Intuitively, we want to be able to create a denser representation of the categories and maintain some of the implicit relationship information between items. We need a way to reduce the number of categorical variables so we can place items of similar categories closer together. That’s exactly what an embedding is. Embeddings solve the encoding problem Embeddings are dense numerical representations of real-world objects and relationships, expressed as a vector. The vector space quantifies the semantic similarity between categories. Embedding vectors that are close to each other are considered similar. Sometimes, they are used directly for the “Similar items to this” section in an e-commerce store. Other times, embeddings are passed to other models. In those cases, the model can share learnings across similar items rather than treating them as two completely unique categories, as is the case with one-hot encodings. For this reason, embeddings can be used to accurately represent sparse data like clickstreams, text, and e-commerce purchases as features to downstream models. On the other hand, embeddings are much more expensive to compute than one-hot encodings and are far less interpretable. How are Embeddings created? A common way to create an embedding requires us to first set up a supervised machine learning problem. As a side-effect, training that model encodes categories into embedding vectors. For example, we can set up a model that predicts the next movie a user will watch based on what they are watching now. An embedding model will factorize the input into a vector and that vector will be used to predict the next movie. This means that similar vectors are movies that are commonly watched after similar movies. This makes for a great representation to be used for personalization. So even though we are solving a supervised problem, often called the surrogate problem, the actual creation of embeddings is an unsupervised process. Defining a surrogate problem is an art, and dramatically affects the behavior of the embeddings. For example, YouTube’s recommender team realized that using the “predict the next video a user is going to click on” resulted in clickbait becoming rampantly recommended. They moved to “predict the next video and how long they are going to watch it” as a surrogate problem and achieved far better results. Common Embedding Models Below are some common embedding models to learn more about. Principal Component Analysis (PCA) One method for generating embeddings is called Principal Component Analysis (PCA). PCA reduces the dimensionality of an entity by compressing variables into a smaller subset. This allows the model to behave more effectively but makes variables more difficult to interpret, and generally leads to a loss of information. A popular implementation of PCA is a technique called SVD. SVD Singular Value Decomposition, also known as SVD, is a dimensionality reduction technique. SVD reduces the quantity of data set features from N-dimensions to K-dimensions via matrix factorization. For example, let’s represent a user’s video ratings as a matrix of size (Number of users) x (Number of Items) where the value of each cell is the rating that a user gave that item. We first pick a number, k, which is our embedding vector size, and use SVD to turn it into two matrices. One will be (Number of users) x k and the other will be k x (Number of items). In the resulting matrices, if we multiply a user vector by an item vector, we should get our predicted user rating. If we were to multiply both matrices, we’d end up with the original matrix, but densely filled with all of our predicted ratings. It follows that two items that have similar vectors would result in a similar rating from the same user. In this way, we end up creating user and item embeddings. Word2Vec Word2vec generates embeddings from words. Words are encoded into one-hot vectors and fed into a hidden layer that generates hidden weights. Those hidden weights are then used to predict other nearby words. Although these hidden weights are used for training, word2vec will not use them for the task it was trained on. Instead, the hidden weights are returned as embeddings and the model is tossed out. Words that are found in similar contexts will have similar embeddings. Beyond that, embeddings can be used to form analogies. For example, the vector from king to man is very similar to the one from queen to woman. One problem with Word2Vec is that single words have one vector mapping. This means that all semantic uses for a word are combined into one representation. For example, the word “play” in “I’m going to see a play” and “I want to play” will have the same embedding, without the ability to distinguish context. BERT Bidirectional Encoder Representations of Transformers, also known as BERT, is a pre-trained model that solves Word2Vec’s context problems. BERT is trained in two steps. First, it is trained across a huge corpus of data like Wikipedia to generate similar embeddings as Word2Vec. The end-user performs the second training step. They train on a corpus of data that fits their context well, for example, medical literature. BERT will be fine-tuned for that specific use case. Also, to create a word embedding, BERT takes into account the context of the word. That means that the word “play” in “I’m going to see a play” and “I want to play” will correctly have different embeddings. BERT has become the go-to transformer model for generating text embeddings. Embeddings in the Real World Embedding usage started in research labs and quickly became state of the art. Since then, embeddings have found themselves in production machine learning systems across a variety of different fields including NLP, recommender systems, and computer vision. Recommender Systems A recommender system predicts the preferences and ratings of users for a variety of entities/products. The two most common approaches are collaborative filtering and content-based. Collaborative filtering uses actions to train and form recommendations. Modern collaborative filtering systems almost all use embeddings. As an example, we can use the SVD method defined above to build a recommender system. In that system, multiplying a user embedding by an item embedding generates a rating prediction. This provides a clear relationship between users and products. Similar items beget similar ratings from similar users. This attribute can also be used in downstream models. For example, Youtube’s recommender uses embeddings as inputs to a neural network that predicts watch time. Semantic Search Users expect search bars to be smarter than a regex. Whether it’s a customer support page, a blog, or Google, a search bar should understand the intent and context of a query, not just look at words. Search engines used to be built around TF-IDF, which also creates an embedding from text. This kind of semantic search worked by finding a document embedding that’s closest to the query embedding using the nearest neighbor. Today, semantic search utilizes more sophisticated embeddings like BERT and may use them in downstream models. In fact, even Google uses BERT on a large percentage of their queries: . https://searchengineland.com/google-bert-used-on-almost-every-english-query-342193/ Computer Vision In computer vision, embeddings are often used as a way to translate between different contexts. For example, if training a self-driving car, we can transform the image from the car into an embedding and then decide what to do based on that embedded context. By doing so, we can perform transfer learning. We can take a generated image from a game like Grand Theft Auto, turn it into an embedding in the same vector space, and train the driving model without having to feed it tons of expensive, real-world images. Tesla is doing this in practice today. Another interesting example is the AI Art Machine: . It will generate an image based on user input text. For example, if we type in “Nostalgia”, we get the following image. https://colab.research.google.com/drive/1n_xrgKDlGQcCF6O-eL3NOd_x4NSqAUjK#scrollTo=TnMw4FrN6JeB It works by transforming the user’s text and an image into an embedding in the same latent space. It’s composed of four transformers: Image -> Embedding, Text -> Embedding, Embedding -> Text, Image -> Text. With all these, transformations we can translate text to image and visa-versa using an embedding as an intermediate representation. Embedding Operations In the above examples, we see that there are a few common operations applied to embeddings. Any production system that uses embeddings should be able to implement some or all of the below. Averaging Using something like word2vec, we can end up with an embedding for each word, but we often need an embedding for a full sentence. Similarly, in recommender systems, we may know the items a user clicked on recently, but their user embedding may not have been retrained in days. In these situations, we can average embeddings to create higher-level embeddings. In the sentence example, we can create a sentence embedding by averaging each of the word embeddings. In the recommender system, we can create a user embedding by averaging the last N items they clicked. Subtraction/Addition We mentioned earlier how word embeddings also encode analogies via vector differences. Adding and subtracting vectors can be used for a variety of tasks. For example, we can find the average difference between a coat from a cheap brand and a luxury brand. We can store that delta and use it whenever we want to recommend a luxury item that’s similar to the current item that a user is looking at. We can find the difference between a coke and a diet coke and apply it to other drinks, even those that don’t have diet equivalents, to find the closest thing to a diet version. Nearest Neighbor Nearest neighbor (NN) is often the most useful embedding operation. It finds things that are similar to the current embedding. In recommender systems, we can create a user embedding and find items that are most relevant to them. In a search engine, we can find a document that’s most similar to a search query. Nearest neighbor is a computationally expensive operation however. Performed naively it is O(N*K), where N is the number of items and K is the size of each embedding. However, in most cases when we need nearest neighbors, an approximation would suffice. If we recommend five items to a user, and one is technically the sixth closest item, the user probably won’t care. Approximate nearest neighbor (ANN) algorithms typically drop the complexity of a lookup to O(log(n)). Implementations of ANN There are many different algorithms to efficiently find approximate nearest neighbors, and many implementations of each of them. We’ll list a few of the most common algorithms and implementations below and break down how they work at a high level. Spotify’s Annoy In Spotify’s ANN implementation (Annoy), the embeddings are turned into a forest of trees. Each tree is built using . At every intermediate node in the tree, a random hyperplane is chosen, which divides the space into two subspaces. This hyperplane is chosen by sampling two points from the subset and taking the hyperplane equidistant from them. This is performed k times to generate a forest. Lookups are done via in-order traversal of the nearest tree. Annoy’s approach allows the index to be split into multiple static files, the index can be mapped in memory, and the number of trees can be tweaked to change speed and accuracy. random projections Here’s how Annoy breaks down the embeddings into an index of multiple trees via random projections. Locality Sensitive Hashing (LSH) Locality Sensitive Hashing (LSH) is another common approach. LSH employs a hash table and stores nearby points in their respective buckets. To create the index, LSH runs many hashing functions which place similar points in the same bucket. In doing so, LSH keeps points with large distances in separate buckets. To retrieve the nearest neighbor, the point in question is hashed, a lookup is done and the closest query point is returned. Some pros include a sub-linear run time, zero reliance on data distribution, and the ability to fine-tune accuracy with the existing data structure. Facebook’s FAISS and Hierarchical Navigable Small World Graphs (HNSW) Facebook’s implementation, FAISS, uses Hierarchical Navigable Small World Graphs (HNSW). HNSW typically performs well in accuracy and recall. It utilizes a hierarchical graph to create an average path towards certain areas. This graph has a hierarchical, transitive structure with a small average distance between nodes. HNSW traverses the graph and finds the closest adjacent node in each iteration and keeps track of the “best” neighbors seen so far. HNSW has a polylogarithmic time complexity (O(logN)). HNSW works by creating a hierarchical index to allow for faster nearest neighbor lookup. How embeddings are operationalized today Moving embeddings out of labs and into real-world systems, has surfaced real gaps in the current data infrastructure capabilities. For example, traditional databases and caches don’t support operations like nearest neighbor lookups. Specialized approximate nearest neighbor indices lack durable storage and other features required for full production use. MLOps systems lack dedicated methods to manage versioning, access, and training for embeddings. Modern ML systems need an embedding store: a database built from the ground up around the machine learning workflow with embeddings. Getting embeddings into production isn’t easy. The most common ways we’ve seen embeddings operationalized today are via Redis, Postgres, and S3 + Annoy/FAISS. We’ll cover how each of those implementations typically looks and the challenges with each. Redis Redis is a super-fast in-memory object-store. It makes storing and getting embeddings by key very fast. However, it doesn’t have any native embedding operations. It can’t do nearest-neighbor lookups and it can’t add or average vectors. All of this must be done on the model service. It also doesn’t fit cleanly in a typical MLOps workflow. It doesn’t support versioning, rolling back, or maintaining immutability. When training, the Redis client doesn’t automatically cache embeddings and this can cause unnecessary pressure and cost. It also doesn’t support partitioning embeddings and creating sub-indices. Postgres Postgres is far more versatile but far slower than Redis. Via plugins, it can be made to perform some of the vector operations manually. However, it does not have a performant nearest neighbor index. Also, having a Postgres lookup in the critical path of a model may add too much latency. Finally, Postgres doesn’t have a great way to cache embeddings on the client when training, causing very, very slow training. S3 files + Annoy/FAISS Annoy or FAISS are often used to operationalize embeddings when nearest-neighbor lookups are required. Both of these systems build indices of embeddings for approximate nearest neighbor operations, but they do not handle durable storage or other vector operations. Alone, they only solve one problem. Typically, companies will store their embeddings in S3 or a similar object storage service to fill in the rest of the gaps. They will load the embeddings and create the ANN index directly on the model when required. Updating embeddings becomes tough, and the system typically ends up with a lot of ad-hoc rules to manage versioning and rollbacks. FAISS and Annoy are great, but they need a full embedding store built around them. The Embedding Hub Machine learning systems that use an embedding need a type of data infrastructure that: - Stores their embeddings durably and with high availability - Allow for approximate nearest neighbor operations - Enable other operations like partitioning, sub-indices, and averaging - Manage versioning, access control, and rollbacks painlessly Embedding Hub sits atop your existing infrastructure. There are a few examples of companies like Pinterest who have built their own in-house, but the gap is still yet to be filled. Some startups have raised a few millions to build proprietary systems, and some database companies have tried to add nearest neighbors as a feature on top. There is a clear gap for an open-source embedding store, and we’ve decided to try to fill it. We’ve made our initial release of Embedding Hub on . Feel free to try it out, leave your comments and issues, and hand us a star! Github Conclusion Embeddings are a critical part of the data science toolkit, and continue to gain in popularity. Embeddings have allowed teams to break the state of the art in multiple disciplines from NLP to recommender systems. As they grow in popularity, a lot more focus will go into operationalizing them in real-world systems. We think that an embedding store will become a critical piece of machine learning infrastructure and that’s why we’ve open-sourced ours. GIve it a try, and if this sounds like an awesome project to work on full-time, we’re hiring! This article was first published on the featureform blog

Embeddings in Machine Learning: Everything You Need to Know

Targeting and fake followers: how can we solve the most acute problems of influencer marketing?

The International Conference on Learning Representations ( ) took place last week, and I had a pleasure to participate in it. ICLR is an event dedicated to . This year the event was a bit different as it went virtual. However, the online format didn’t change the great atmosphere of the event. It was engaging and interactive and attracted 5600 attendees (twice as many as last year). If you’re interested in what organizers think about the unusual online arrangement of the conference, you can read about it . ICLR research on all aspects of representation learning, commonly known as deep learning here Over 1300 speakers presented many interesting papers, so I decided to create a series of blog posts summarizing the best of them in four main areas. You can catch up with the first , the second , and the third . post with the best deep learning papers here post with reinforcement learning papers here post with generative models papers here Here, I want to share more information about 10 best Natural Language Processing/Understanding contributions from the ICLR: ALBERT: A Lite BERT for Self-supervised Learning of Language Representations A Mutual Information Maximization Perspective of Language Representation Learning Mogrifier LSTM High Fidelity Speech Synthesis with Adversarial Networks Reformer: The Efficient Transformer DeFINE: Deep Factorized Input Token Embeddings for Neural Sequence Modeling Depth-Adaptive Transformer On Identifiability in Transformers Mirror-Generative Neural Machine Translation FreeLB: Enhanced Adversarial Training for Natural Language Understanding 1. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations A new pretraining method that establishes new state-of-the-art results on the GLUE, RACE, and SQuAD benchmarks while having fewer parameters compared to BERT-large. (TL;DR, from OpenReview.net ) Paper | Code The L2 distances and cosine similarity (in terms of degree) of the input and output embedding of each layer for BERT-large and ALBERT-large. (source: figure 1, from the paper ). First author: Zhenzhong Lan ( ) LinkedIn 2. A Mutual Information Maximization Perspective of Language Representation Learning Word representation is a common task in NLP. Here, authors formulate new frameworks that combine classical word embedding techniques (like Skip-gram) with more modern approaches based on contextual embedding (BERT, XLNet). Paper The left plot shows F1 scores of BERT-NCE and INFOWORD as we increase the percentage of training examples on SQuAD (dev). The right plot shows F1 scores of INFOWORD on SQuAD (dev) as a function of λDIM. (source: figure 1, from the paper ). First author: Lingpeng Kong | | Twitter GitHub Website 3. Mogrifier LSTM An LSTM extension with state-of-the-art language modelling results. (TL;DR, from OpenReview.net ) Paper Mogrifier with 5 rounds of updates. The previous state h0 = hprev is transformed linearly (dashed arrows), fed through a sigmoid and gates x −1 = x in an elementwise manner producing x1 . Conversely, the linearly transformed x1 gates h 0 and produces h2 . After a number of repetitions of this mutual gating cycle, the last values of h∗ and x∗ sequences are fed to an LSTM cell. The prev subscript of h is omitted to reduce clutter. (source: figure 1, from the paper ) First author: Gábor Melis | | | Twitter LinkedIn GitHub Website 4. High Fidelity Speech Synthesis with Adversarial Networks We introduce GAN-TTS, a Generative Adversarial Network for Text-to-Speech, which achieves Mean Opinion Score (MOS) 4.2. (TL;DR, from OpenReview.net ) Paper | Code Residual blocks used in the model. Convolutional layers have the same number of input and output channels and no dilation unless stated otherwise. h — hidden layer representation, l — linguistic features, z — noise vector, m — channel multiplier, m = 2 for downsampling blocks (i.e. if their downsample factor is greater than 1) and m = 1 otherwise, M- G’s input channels, M = 2N in blocks 3, 6, 7, and M = N otherwise; size refers to kernel size. (source: figure 1, from the paper ). First author: Mikołaj Bińkowski | LinkedIn GitHub 5. Reformer: The Efficient Transformer Efficient Transformer with locality-sensitive hashing and reversible layers (TL;DR, from OpenReview.net ) Paper | Code An angular locality sensitive hash uses random rotations of spherically projected points to establish buckets by an argmax over signed axes projections. In this highly simplified 2D depiction, two points x and y are unlikely to share the same hash buckets (above) for the three different angular hashes unless their spherical projections are close to one another (below). (source: figure 1, from the paper ). Main authors: Nikita Kitaev | | LinkedIn GitHub Website Łukasz Kaiser | | Twitter LinkedIn GitHub 6. DeFINE: Deep Factorized Input Token Embeddings for Neural Sequence Modeling DeFINE uses a deep, hierarchical, sparse network with new skip connections to learn better word embeddings efficiently. (TL;DR, from OpenReview.net ) Paper With DeFINE, Transformer-XL learns input (embedding) and output (classification) representations in low n-dimensional space rather than high m-dimensional space, thus reducing parameters significantly while having a minimal impact on the performance. (source: figure 1, from the paper ). First author: | | | Twitter LinkedIn GitHub Website 7. Depth-Adaptive Transformer Sequence model that dynamically adjusts the amount of computation for each input. (TL;DR, from OpenReview.net ) Paper Training regimes for decoder networks able to emit outputs at any layer. Aligned training optimizes all output classifiers Cn simultaneously assuming all previous hidden states for the current layer are available. Mixed training samples M paths of random exits at which the model is assumed to have exited; missing previous hidden states are copied from below. (source: figure 1, from the paper ). First author: Maha Elbayad | | | Twitter LinkedIn GitHub Website 8. On Identifiability in Transformers We investigate the identifiability and interpretability of attention distributions and tokens within contextual embeddings in the self-attention based BERT model. (TL;DR, from OpenReview.net ) Paper (a) Each point represents the Pearson correlation coefficient of effective attention and raw attention as a function of token length. (b) Raw attention vs. (c) effective attention, where each point represents the average (effective) attention of a given head to a token type. (source: figure 1, from the paper ). First author: Gino Brunner | | Twitter LinkedIn Website 9. Mirror-Generative Neural Machine Translation Translation approaches known as Neural Machine Translation models (NMT), depend on availability of large corpus, constructed as a language pair. Here, a new method is proposed for translations in both directions using generative neural machine translation. Paper The graphical model of MGNMT. (source: figure 1, from the paper ). First author: Zaixiang Zheng | Twitter Website 10. FreeLB: Enhanced Adversarial Training for Natural Language Understanding Here, the authors propose a new algorithm, called FreeLB that formulate a novel approach to the adversarial training of the language model is proposed. Paper | Code Algorithm's pseudo-code. (source: figure 1, from the paper ). First author: Chen Zhu | | LinkedIn GitHub Website Summary Depth and breadth of the ICLR publications is quite inspiring. This post focuses on the “Natural Language Processing” topic, which is one of the main areas discussed during the conference. According to , these areas include: this analysis Deep learning ( ) here Reinforcement learning ( ) here Generative models ( ) here Natural Language Processing/Understanding (covered here) In order to create a more complete overview of the top papers at ICLR, we have built a series of posts, each focused on one topic mentioned above. This is the last one, so you may want to for a more complete overview. check the others We would be happy to extend our list, so feel free to share other interesting NLP/NLU papers with us. In the meantime — happy reading! This article was originally written by Kamil and posted on the Neptune blog .

The Ten Must Read NLP/NLU Papers from the ICLR 2020 Conference

I've been writing code for over three years. I have always asked myself about the good practices and how I could make my code more readable, understandable, easy to maintain and to be handled by other developers. We spend more time reading our code rather than writing it because we always look for the element that we can modify, improve or just to remember ourself how this or that element works. So, it’s important to write a good code in others words a clean, consistent, an extensible and correct code. Writing a code is not as easy as it appears. Without some writing rules, one can find himself with a complicated code to read and difficult to debug. Therfore, difficult to maintain and to evolve with the harmful effects on the performance of the software hence the importance of optimizing our code in other words, a code permeating good practices. So what are good practices in CSS? Use stickler Stickler is a powerful tool which allows a developer to respect the standards of coding established before the realisation of a project by himself or by his work team. Before writing a line code, most often developers establish coding standards that they will follow in order to realise the project on which they want to work. This tool intervenes on several aspects especially the improvements of the code indentation, the correction of syntax errors and the identification of places where there is a duplication of the code. Stickler is used by a wide community and it’s quite flexible in his configuration. Segment the CSS in multiple files Cutting out his CSS code into many files each corresponding to a module, a view, or a page of your application is very recommended. The choice of cutting is free depending on what appears for you most pertinent. The cutting of the code into files enables to navigate easily when we have to extend, modify or create new declarations. Examples: homepage .css header .css footer .css archive .css The merits of splitting the CSS in multiple files and to increase appears mainly when someone uses a preprocessor CSS ( or ) Sass Less Understanding the specificity The specificity determines what CSS rule is to apply by the navigator. The is a hierarchy, not necessarily intuitive in the order of application CSS rules. It’s important to keep in your mind that all selector has to be declared with the less of specificity as possible. This facilitates reusability of the selector and reduces the dependency to another selector. Prefer : {...} .myclass rather than: > {...} body .page .main .sidebar article .myclass In most cases, this will prevent you from having to deal with CSS-specific conflicts. Be DRY Don't repeat yourself (DRY, or sometimes do not repeat yourself) is a principle of software development aimed at reducing repetition of codes or software patterns. The DRY principle is simple to understand "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system". Example : { : ; : ; : red; : red;
} { : ; : ; : ; : ; : red; : red;
} .texte-erreur font-size 15px background-color #ccc border-color color .bloc-erreur width 250px height 250px background-color #000 font-size 17px border-color color We observe that two classes share their properties border-color and color. We can consider they always find themself together for a certain number of classes. Then we can create a mixin and the follow classes : () { : red; : red;
} { : ; : ;
 @include erreur-base();
} { : ; : ; : ; : ;
 @include erreur-base();
} mixin erreur-base border-color color .texte-erreur font-size 15px background-color #ccc .bloc-erreur width 250px height 250px background-color #000 font-size 17px We just gained readability and refactored our code to avoid repetition. Naming convention Naming things is never easy and the nomenclature of classes and id attributes in CSS is no exception. The problem with bad naming is that it is then difficult to find yourself in its code, so it’s important to find a particular convention to organize yourself well. During my researches on this subject, I have identified two popular naming conventions notably the and . OOCSS BEM OOCSS (OOP) is a programming paradigm that focuses on creating reusable objects and establishing relationships between them, as opposed to that organizes the code into procedures (routines, subroutines, or functions). Object-oriented programming procedural programming Object oriented CSS was proposed by web developer Nicole Sullivan in 2008. It’s neither a preprocessor nor even a new language, but rather a code philosophy. OOCSS PRINCIPLES: The separation of “ ” from “ ” structure skin The separation of “ ” from “ ” container content CONTENT - SKIN : Define the repeating “ ” patterns as reusable “ “ visual skins Define the repeating “ ” patterns as reusable “ “ invisible structures { : inline-block; : ; : center; : nowrap; : middle; : none; : none; : none; : none; : solid transparent; : ; : ; : ; : ; : color ; 
} /* structure */ .btn display font-weight 400 text-align white-space vertical-align -webkit-user-select -moz-user-select -ms-user-select user-select border 1px padding 0.375rem 0.75rem font-size 1rem line-height 1.5 border-radius 0.25rem transition 0.15s { : ; : ; : ;
} { : ; : ; : ;
} { : ; : ; : ;
} /*SKIN*/ .btn-success color #fff background-color #28a745 border-color #28a745 .btn-danger color #fff background-color #dc3545 border-color #dc3545 .btn-warning color #212529 background-color #ffc107 border-color #ffc107 CONTAINER - CONTENT An Object should be not depend of the location where you put it: It’s often the main cause of code duplication. > { : red; } { : inline-block; } .announcecategory div :nth(3) color /*...*/ /*BAD! div:nth(3) styles are location dependent*/ .announcecategory display /*...*/ /*we can reuse this category style wherever*/ Limitations and disadvantages of OOCSS: Utility classes tie your markup to presentation. OOCSS doesn’t give rules of the clear and firm building, and the past time to factorize the appearance can very often exceed the gain of the reuse. That could increase considerably the time of realisation projects. I highly recommend and advise the reader, the talk by Nicole Sullivan: , published in 2011. It has allowed to several integrators to better understand OOCSS and to start using this naming concept. Our best practices are killing us The reader wishing to deepen will then read the . wiki of the framework OOCSS If you've ever worked with the bootstrap, you probably noticed by reading this section that the bootstrap team uses this approach heavily. it exists other good practices of naming. BEM is the acronym of Block, Element, Modifier and all the methodology of the naming hold on in these three words. The force of the concept? What makes up a page or a web application can always be stored in an arborescence of blocks, Elements and modifiers. BEM A is an independent entity which must be able to be moved without affecting its appearance or its functioning. block An is a part of block. The context of an element is that of the block. This can be a header, a footer, a container (articles, sections ...), a menu or even a button. As it appears clearly, a block can also contain other blocks. An interface can include several instances of the same block. element A is a property which is used to create variants, to make minimal changes like changing colours, size. There exists modifiers of blocks and the element modifiers. modifier The methodology BEM has three essential rules: The blocks and elements have to each other have a unique name, which will be used as a CSS class The CSS selectors don't have to use the HTML elements (not of footer div) Cascades in CSS selectors should be avoided ( not of .foo > .bar) Naming convention: Bloc: bloc-name Element: bloc-name__element-name Modifier : bloc-name--modifier-name bloc-name__element-name--modifier-name The official website takes care to specify that only the concepts count, the syntax remaining free. Advantages of BEM: Performance: The performance concern more particularly the web application. The nomenclature of BEM allows to have the majority of CSS classes at the first level. Browsers organize CSS classes in a table of the global hash, but it would be too expensive to create subtables for descendants at the level of each HTML element. Also, in CSS, only the first level of selection is high-performance.  CSS cascade, when there are many of them, this creates fluidity problems, especially on the animated pages of Web applications. Lisibility: Class names are logical and intuitive, and every member of the team knows what that element does on the website. BEM gives everyone on a project a declarative syntax they can share. So, they’re on the same page. Limits and disadvantages of BEM: Verbose: The nomenclature of BEM is a little bit verbose. Indeed, we remark that as soon as we have modifiers to implement, we find ourself very fastly with a long and complex class attribute. The naming convention is open, someone can mix the two conventions listed previously depending on needs of the project or even in creating if someone can do better than the conventions of existing name Important to know: Conclusion: The best practices are very important because they allow optimizing our program on all plans (readability of the code, maintainability of the software, scalability, rapidity of execution, ...).

Best Practice in CSS: Organisation and Naming Conventions

Mejores prácticas en CSS: organización y convenciones de nomenclatura

Eight Leaders Explain Their Model Training Libraries For The PyTorch Ecosystem

To understand what PyTorch is, how it works, and its ability to catalyze technological advancements. It’s important first to understand the answer to the question, “What is PyTorch?” An open-source machine learning library, Pytorch, is used to develop and train deep learning models based on neural networks. Pytorch supports both Python and C++ and is used for applications like Natural Language Processing. It was developed by Facebook’s AI research group and can be used to solve modern challenges in the field of robotics and autonomous vehicles, with regards to artificial intelligence. On 6th February 2020, Facebook AI Research (FAIR) released , which is a library that allows developers to apply deep learning to 3D objects. PyTorch 3D In this article, I’ll be focusing on how this will impact the future of Self-Autonomous Cars, the AR and VR industries, and consequently, the evolution of AI within them. What does that mean for the future of AI technology and its application in the real world? Let’s find out. PyTorch3D As A Technological Advancement In Deep Learning: For an AI system to function efficiently in the real world, the system’s 3D understanding plays a crucial role. This includes operations in the field of VR and navigation. So far, the advancements and research in 3D deep learning have been limited due to insufficient tools and resources that can help manage the complex nature of training neural networks with 3D data. Developers have been facing a limitation with graphic operators not being differentiable – but PyTorch3D looks to be the answer that everyone was waiting for. PyTorch3D is a modular library that is designed with capabilities that make 3D deep learning easier with PyTorch. It gives developers access to frequently used 3D operations and loss functions for 3D data.  These operations and loss functions are differentiable, making them unique from any other available resource for 3D deep learning. PyTorch3D also comes with a rendering API that is both modular and differentiable in nature. This means that developers can import these new functions into their current deep learning systems instantly. For engineers, leveraging PyTorch3D can open new paths in the field of 3D deep learning research. It can be used for 3D reconstruction and 3D reasoning, which also helps improve 2D recognition tasks. Ultimately, PyTorch3D is the step forward in deep learning and its implementation in 3D operations and applications. PyTorch3 and Its Impact On 3D Deep Learning Research and Application: 3D data inputs are incredibly complex in comparison to 2D images and data. Their process requires more memory and computational power, though 2D images can be represented by leveraging tensors, which works like a vector. This is one reason why there is significantly less exploration of 3D understanding and deep learning. For 3D operations, there’s an additional requirement of being differentiable to allow gradients to channel backward through a system – allowing them to move from the model output back to the input data. The limitation of traditional operations with regards to rendering block gradients is solved by PyTorch3D. Just like PyTorch provides you access to optimized libraries for 2D recognition tasks, PyTorch3D optimizes training and inference by giving developers and engineers batching capabilities and support for 3D loss functions and operations. PyTorch3D in the Real-world: With this understanding of PyTorch3D, let’s move forward and dive into what is possible with 3D deep learning. A self-driving car or other forms of autonomous vehicles are modern technology that is set to transform the world. Currently being tested for safety and feasibility, a has been successful in completing their longest and most complex self-driving car journey of 230 miles, completely self-navigated on public roads. UK government-backed AV research project As such, what PyTorch3D offers is to take this feat to its logical conclusion, which is to have self-driving cars become much safer on the roads. It’s the ability to comprehend and compute 3D data, allows it to be able to assess nearby objects and their position, giving the system the capability to make better driving decisions on the road. This is a remarkable progression that will reshape the global debate around self-driving vehicles. Self-driving systems can be significantly improved by eliminating the reliance on manual 3D annotations, making highly optimized systems with 3D deep learning. This can help improve the overall experience of a self-driving car, make them safer for both the person inside it and others who are arguably in danger by improving their risk assessment based on accurate 3D image rendering and object positioning. So how does this work? PyTorch3D is fused with the 2D recognition library Detectron2, extend object understanding to 3D. Combined with this, the creation of makes understanding the motion of non-rigid structures possible in three dimensions. C3DPO Thanks to the factorization network implementation, a deep network is used to learn how to factorize shape and viewpoint – helping perform test time monocular reconstruction. But one challenge that 3D meshes are complex and are made up of a collection of face indices and vertex coordinates, there is significant difficulty in batching different sizes of 3D meshes. For this, is leveraged, which is a data structure that helps in batching heterogeneous meshes in deep learning applications. Meshes This helps it easier for researchers to take the mesh data and transform it into different views and match operators with efficient data representation. Another such application of 3D deep learning is in the field of AR and VR. Within any virtual environment, there are complex issues and simple issues. Functions such as a 360-degree view might not require artificial intelligence but interaction with objects within the virtual environment does. Interaction with objects within VR games, for example, is a form of data that is collected from the movement and interaction and then processed and translated into actions within the environment. Similarly, in AR, any interaction with the real world objects will also demand a response, for which Artificial intelligence is important. What PyTorch3D will allow is the ability not just to increase the scope of Artificial Intelligence but also make it smarter within these virtual environments. It promises accuracy, the solution to complexities of three-dimensions in comparison to two-dimensions, and the growth of AI within a wide variety of industries as if it weren’t already growing at an incredible pace. Wrapping it Up: PyTorch3D is the way forward, optimizing 3D object and image processing, and making technology much smarter in its application could make this unveiling by Google a special moment within the future of many technologies that will leverage 3D image modeling, object positioning, and 3D object recognition to improve their numerous applications in the real world – We’ve only seen the tip of the iceberg yet and I think it won’t be long until we see the impact of PyTorch3D in the real world.

Facebook’s PyTorch3D : A Catalyst for Deep Learning and 3D Objects

Behavior-Driven Development (BDD) is a really powerful tool that helps us build robust, value-based software. You can sometimes hear detractors say that it brings a lot of complexity and leads to long tests difficult to maintain. Let's take stock of what BDD really is, by determining anti-patterns and best practices. "I will write/fix the BDDs" This sentence is the main symptom of a misunderstanding about Behavior-Driven Development. BDD is not a file, a test or a testing activity. It's a development process: . a methodology You cannot write a methodology, you apply it. Most of the people believe that having written tests in a style means that they do BDD. But we cannot blame them when we can find badly named frameworks such as or . And there are two main reasons why the keyword "BDD" is misused in these cases. Given When Then BDDMockito Chai BDD First, it is not because they are labeled BDD that they facilitate or encourage people to work in a BDD fashion. Even worse, they allow them to think they are doing it! Suppose we rename JUnit5 in TDD JUnit, will developers start writing their tests before coding only because of that? How can a framework ensure that a methodology is applied? Secondly, is a convention, a nomenclature in which we have to write scenarios in natural language, also called Gherkin, if you use . This is only a small part of what BDD is. Given When Then Cucumber Back to the basics: the theorical part Behavior-Driven Development, it's all in the name: focus on the behavior of the software and let it drive our development process. The main advantages: business value first and robustness. User Scenarios and Examples The starting point is always a scenario, defining the needs of the user. Without this, you can not create software that matters. This "specification" must be made with the main stakeholder: the user (or at least one person who represents and understands him). Taking the need at the source is always the best way. The user (his representative) is not only a stakeholder, he must participate in the design phase of the application. That's why it's essential to use a natural language to define your needs. As a general rule, an end user can not write or review JUnit tests. It also focuses on the definition and the usage of an ubiquitous language. If the user calls this a "dog", it must be named so in the scenarios. Everyone, from the user to the developer, must speak the same language, it will avoid confusion and facilitate communication, therefore the collaboration, and then speed up the development process. The scenarios are not just the description of business rules. These rules must be "instanciated" on concrete examples and you can then call them scenarios. This way, you will identify what are the acceptance criteria (Then) from some "Given" context and/or inputs "When" a user performs an action => e.g. how your application should behave in that situation. specific kind of : As a cinephile In order to not loose a single minute of a movie I want to know what are the next showings of the movie theater : a cinephile : Looking for the next showings before the start of the last one the last showing is not started yet the user asks for the next showings the returned showings are not started yet the returned showings are not starting in the next 15 minutes Feature Background Given Scenario Given When Then And Every scenario must define: is the main stakeholder of the feature, whose needs we must meet (As a). who , to qualify the need of this stakeholder (In order to). why are the expected results of the feature (I want to). what In the above example written in the Cucumber-Gherkin way, you can see that all the terms belong to the lexical fields of the cinema. Cinephile, Movie, Movie Theatre and Projection define the ubiquitous language. This business language, combined with Gherkin's natural language nomenclature, makes this specification perfectly understandable for our users. The context is clearly identified: before the start of the last showing. The are the expected results of the previous scenario. You may have also noticed that we have here several levels of abstraction. next showings The definition of the feature is really high level. Each scenario is a specialization of the functionality in a given context, and the results of the feature are refined in the "then" directives. As you can see, the concept of is refined in the "then" statement: showings that are not yet started and do not start in the next 15 minutes. next showings Sometimes this refinement does not come from the user himself. You can imagine that the movie theatre - which is also a stakeholder - does not want us to show the user the immediate for some reasons of crowd management. Or maybe, just to avoid the disturbance of someone entering late a room where a movie has already started. Every stakeholders count, this is one of the principles of the methodology. outside-in When a specific kind of user performs an action From one type of user to another, the expectations (acceptance criteria) of an action / feature can be very different. You can not bring value if you do not know who the user is. Not identifying the persona may also lead to writing unusable features. In addition, it also introduces security issues due to a lack of permissions management. BDD is not a security methodology, but it has the very good side effect of encouraging people to think about the type of user that should be able to perform certain actions. In the example above, the identified persona is a cinephile. Let's assume our application is also used by the projectionists of the movie theatre. For this type of users, knowing what the next showings will mean which projections they need to run in order to be synchronized with their work schedule. The virtuous double loop BDD is a development process based on two loops. In the first loop we identify the user needs that are expressed as scenarios with examples as follow: : As a cinephile In order to not loose a single minute of a movie I want to know what are the next showings of the movie theater : a cinephile : Looking for the next showings before the start of the last one the last showing is not started yet the user asks for the next showings the returned showings are not started yet the returned showings are not starting in the next 15 minutes Feature Background Given Scenario Given When Then And For each step of a scenario, we start writing the fixtures - also called step definitions - i.e. implementing the steps from a technical stand point. These fixtures will constrain and therefore define the behavior of the application. { {

        Given( , () -> me = Cinephile( ));
        Given( , 
           () -> lastShowing = Showing(now().plusHours( )));

        When( ,
                () -> nextShowings = searchShowings.searchNextShowingsFor(me));

       Then( , () -> {
            assertThat(allShowings.stream().filter(showing -> !showing.hasStarted()))
               .containsAnyElementsOf(nextShowings);
        });
       
        Then( , () -> {
            assertThat(nextShowings)
               .noneMatch( showing -> showing.startsBefore(now().plusMinutes( )));
        });
    }
} public class StepDefs implements En public StepDefs () "^a cinephile$" new "me" "^the last showing is not started yet$" new 3 "^the user asks for the next showings$" "^the returned showings are not started yet$" "^the returned showings are not starting in the next 15 minutes$" 15 Note that we are not talking about the production code here. The above example is written in a cucumber java8 style. The scenario with its related fixtures represent an acceptance test. Then we start the inner TDD loops by implementing the production code used in the step definitions until the functionality is developed. That's why we call this a "double loop". Behavior-Driven Development is a methodology that has emerged from TDD. It has the same strengths as Test-Driven Development - assures good test coverage and code design - while getting rid of the limitation of the bottom-up approach of TDD. When working with TDD, it can sometimes be very difficult to determine where to start, what may be the appropriate granularity of the method we want to build. Have you ever had this feeling: after performing a major refactoring during the last TDD iteration, you would have converged faster if you had thought of the problem with a top-down approach? This is where BDD helps a little, by forcing you to think first from the consumer's point of view, you can say this methodology is totally complementary to TDD and, more importantly, it drives your TDD iterations. Best Practices: the practical part All user stories and epics must have scenarios This is the basis. BDD offers a real framework for expressing user needs. By integrating a scenario into a user story, you make sure that it has been sufficiently prepared to be developed. Although it is very difficult to find anything in tools such as JIRA, the scenario will ensure the functional documentation of your software which, by design, is automatically checked. Always have a persona in a Given As mentioned earlier, the identification of the persona is the key, so it must be explicitly define inside a Given directive. This will also avoid confusion. Beware that persona are important from a security point of view. It will define the required privileges to trigger a feature. : Persona are not defined by fancy user names, rather use roles, it avoids mental mappings. NOTE Only one “when”! Your scenario must describe a single responsibility for a feature and you must have one scenario per use case. By not following these rules, you will end up with an accidental complexity. Remember that a When clause is describing a feature. If you need multiple Whens in a single scenario, it may mean you have a design problem. Then in the passive voice with no user action Thens are dedicated to the assertions on the outputs of a feature, the user is not a part of it. returned showings are starting next minutes the not in the 15 The subject must be the functional object on which we must check the post-conditions of the feature. The passive voice is best to emphasize the functional objects affected by the execution of the feature. You should never have any user actions here, they are dedicated to Whens directives. Putting a user action in a Then means that you can control what your user will do exactly after running your feature. Think about the unhappy paths first We usually start with the happy path to reassure ourselves, but unfortunately the process ends here. A large number of bugs are due to a lack of analyzis around the errors. Thinking first about the unhappy paths ensures that your functionality is built robustly. Do not forget to fail fast. expected Avoid the tables frenzy Some Gherkin frameworks allow you to use tables, although this can be very convenient and justified when you want to iterate over several input examples, it is totally inadvisable to use it to represent a complex object because you would lose the functional intent. In the folowing example: the showing:
  | | | | | |
  | | | | | | a child tries to buy a ticket for this showing the purchase is forbidden Given name schedule director rating duration Gore Movie 21H00 Jhon Doe NC-17 PT2H5M When Then A showing is built in the Given using the content of the table. Unfortunately we do not know what characteristics of the showing are important for the rest of the scenario, until we read the When and the Then. Here, only the rating is important, the goal being to prevent a child from buying a ticket for a movie rated NC-17. A better approach would be: a NC-17 rated showing a child tries to buy a ticket for this showing the purchase is forbidden Given When Then It is totally fine to hide inside the fixture the initialization of the showing fields as soon as they have no impact on the behavior of the scenario. Do not let an unimplemented step pass Some frameworks such as Cucumber will not fail your scenario by default if one of the steps is not implemented. But what can you say about a passing Scenario in which the only Then directive is not implemented? Never postpone the implementation of a step definition, it goes against the principles of the double loop first and no one will ever implement it later. Usually, we discover that an implementation is missing when we try to debug the given functionality. If you use Cucumber, make sure to turn on the . strict option Step Definitions must be component based Most of the time, users split their step definitions in the same way as their scenarios. Unfortunately, it couples everything together and the steps you made cannot be reused for another scenario. The step definitions should be organised in the same way than your domain concepts and, moreover, named after them. This way your step definitions code will be factorized, easier to find and navigate through, you'll get easier steps to manage and reuse. In other words, the step definitions must be designed with the same processes we use to create our business domain! Do not create a And or But fixture And and But are fine for chaining Givens and Thens in a scenario. However, do not define them as fixtures: And( , ...) "^the returned showings are not starting in the next 15 minutes$" In this case, it is really difficult to understand if this directive can be used as a Given or a Then and you can not usually use the same directive for different purposes. A Given usually initiates the input of a feature while a Then makes assertions on the outputs. Use an ubiquitous language The scenario must be understandable from the point of view of the user. Words must therefore be carefully chosen from the concepts of the business domain. Avoid technical terms! Expose your scenarios as a living documentation The natural language enforced by gherkin with the use of an ubiquituous language provides a powerful functional documentation of your software which is always up-to-date. This makes it easy to integrate new members into the team and helps your product owner / user understand what features your software supports. But do not just keep it in your version control repository, expose it as a static website generated from your CI/CD. You can use tools like for this purpose. Pickles Be Agile: Do not write all your scenarios at the same time Some teams lock up an entire afternoon in a room to produce a dozen scenarios needed for software. We do not live in the dark age of Waterfall, it is not necessary to specify everything in advance. Generally, these teams commit these scenarios directly in their repository. This has the disadvantage of confusing what the application is currently doing. Even adding a tag like to the top of a scenario is counterproductive for two reasons: first, the feature file name is meaningful enough to make someone think that it is really developed; secondly, invariants or simply the understanding of a functionality evolves over time. In some cases, the features are descoped... and your repository is not a backlog ... @wip A scenario must be written just before implementation - just in time - to save time and money. Do not mix the concerns (integration test/functional test) Sometimes you can see a scenario connected to the controller of an application and, as you can imagine, it's a bad practice. The scenarios have a business purpose: connecting it to the application's infrastructure level changes them into integration tests, and this methodology is not intended to meet an integration goal. A scenario is used to describe the use case of a feature. It should not fail because your JSON serializer is not well defined in a controller. If you are already "implementing" BDD and find yourself with hard-to-maintain tests, it's usually because of this lack of separation of concerns. Be aware that even Cucumber discourages this usage: documentation Cucumber is not an API automation tool In , it is generally advisable to connect the scenario directly to the domain modules of your application. hexagonal architecture If you like to write integration tests using a Gherkin style, you should take a look at . A really powerful tool with many predefined step definitions that handles some HTTP operations for you. KarateDSL Here is an example of a KarateDSL feature file: : Profile endpoint : creating a profile path 'profiles' request method post assert responseStatus == 201 match responseHeaders['Location'] contains only (profilesUrl + userId) Feature Scenario Given And """
    { "topics": [{"name": "DDD"},{"name": "Hexagonal Architecture"}],"talksFormats": ["QUICKIE","CONFERENCE"] }
    """ When Then And NOTE: All step definitions are built into KarateDSL, which means that the integration test does not require more than this file. Nutshalling BDD This is a methodology that helps you first define the behavior of a feature using with . It is complementary to the which takes a bottom-up approach while BDD comes from the top. development scenarios examples TDD The ensure the creation of a robust and tested software and the scenarios provide a that facilitates communication with non-technological stakeholders. double loop living documentation Further readings You can also read this really good article which also cover some other best practices in BDD: Cucumber Anti-Patterns Cucumber is a testing tool that allows you to test your assumptions and understanding of the problem before you…www.thinkcode.se and browse this , where you'll find a full implementation of and . repository Cucumber KarateDSL

Behavior-Driven Development From Scratch [101]

is the area of study focused on developing computer technology based on the principles of quantum theory. The quantum computer, following the laws of quantum physics, would gain enormous processing power through the ability to be in multiple states, and to perform tasks using all possible permutations simultaneously. Quantum computing A Comparison of Classical and Quantum Computing Classical computing relies, at its ultimate level, on principles expressed by Boolean algebra. Data must be processed in an exclusive binary state at any point in time or bits. While the time that each transistor or capacitor need be either in 0 or 1 before switching states is now measurable in billionths of a second, there is still a limit as to how quickly these devices can be made to switch state. As we progress to smaller and faster circuits, we begin to reach the physical limits of materials and the threshold for classical laws of physics to apply. Beyond this, the quantum world takes over. In a quantum computer, a number of elemental particles such as electrons or photons can be used with either their or acting as a representation of 0 and/or 1. Each of these particles is known as a quantum bit, or , the nature and behavior of these particles form the basis of quantum computing. charge polarization qubit Quantum Superposition and Entanglement The two most relevant aspects of quantum physics are the principles of and . superposition entanglement : Think of a qubit as an electron in a magnetic field. The electron’s spin may be either in alignment with the field, which is known as a spin-up state, or opposite to the field, which is known as a spin-down state. According to quantum law, the particle enters a superposition of states, in which it behaves as if it were in both states simultaneously. Each qubit utilized could take a superposition of both 0 and 1. Superposition Particles that have interacted at some point retain a type of connection and can be entangled with each other in pairs, in a process known as . Knowing the spin state of one entangled particle — up or down — allows one to know that the spin of its mate is in the opposite direction. Quantum entanglement allows qubits that are separated by incredible distances to interact with each other instantaneously (not limited to the speed of light). No matter how great the distance between the correlated particles, they will remain entangled as long as they are isolated. Taken together, quantum superposition and entanglement create an enormously enhanced computing power. Entanglement: correlation Where a 2-bit register in an ordinary computer can store only one of four binary configurations (00, 01, 10, or 11) at any given time, a 2-qubit register in a quantum computer can store all four numbers simultaneously, because each qubit represents two values. If more qubits are added, the increased capacity is expanded exponentially. Difficulties with Quantum Computers — During the computation phase of a quantum calculation, the slightest disturbance in a quantum system (say a stray photon or wave of EM radiation) causes the quantum computation to collapse, a process known as . A quantum computer must be totally isolated from all external interference during the computation phase. — Given the nature of quantum computing, error correction is ultra critical — even a single error in a calculation can cause the validity of the entire computation to collapse. — Closely related to the above two, retrieving output data after a quantum calculation is complete risks corrupting the data. Interference de-coherence Error correction Output observance The Future of Quantum Computing The biggest and most important one is the ability to factorize a very large number into two prime numbers. That’s really important because that’s what almost all of internet applications use and can be de-encrypted. A quantum computer should be able to do that relatively quickly. Calculating the positions of individual atoms in very large molecules like polymers and in viruses. The way that the particles interact with each other — if you have a quantum computer you could use it to develop drugs and understand how molecules work a bit better. encryption Even though there are many problems to overcome, the breakthroughs in the last 15 years, and especially in the last 3, have made some form of practical quantum computing possible. However, the potential that this technology offers is attracting tremendous interest from both the government and the private sector. It is this potential that is rapidly breaking down the barriers to this technology, but whether all barriers can be broken, and when, is very much an open question. , Author the Books : Ahmed Banafa Secure and Smart Internet of Things (IoT) Using Blockchain and AI Blockchain Technology and Applications Read more articles at Technology Trends by Prof. Ahmed Banafa References http://www.fastcolabs.com/3013214/why-quantum-computing-is-faster-for-everything-but-the-web http://www.theguardian.com/science/2014/mar/06/quantum-computing-explained-particle-mechanics http://www.economist.com/news/science-and-technology/21578027-first-real-world-contests-between-quantum-computers-and-standard-ones-faster http://whatis.techtarget.com/definition/quantum-computing http://physics.about.com/od/quantumphysics/f/quantumcomp.htm

What is Quantum Computing?

Where are we at? This is what we did so far In , we downloaded our data from , did some EDA and created our user item matrix. The matrix has 671 unique users, 9066 unique movies and is 98.35% sparse part 0 MovieLens In , we described 3 of the most common recommendation methods: ser ased ollaborative iltering, tem ased ollaborative iltering and atrix actorization part 1 U B C F I B C F M F In part 2, this part, we will implement atrix actorization through ALS and find similar movies M F Matrix Factorization We want to factorize our user item interaction matrix into a User matrix and Item matrix. To do that, we will use the lternating east quares (ALS) algorithm to factorize the matrix. We could write our own implementation of ALS like how it’s been done in or , or we can use the already available, fast implementation by . The ALS model here is from and can easily be added to your Python packages with or with Anaconda package manager with . A L S this post this post Ben Frederickson implicit pip conda import implicit model = implicit.als.AlternatingLeastSquares(factors=10,iterations=20,regularization=0.1,num_threads=4)model.fit(user_item.T) Here, we called ALS with the following parameters: 10 factors. The number of latent factors to be used 20 iterations 0.1 regularization. This regularization term is the lambda in the loss function 4 threads. This code can be parallelized which makes it super fast. it takes about 5 sec to train. One thing to note is that the input for the ALS model is a movie user interaction matrix, so we just have to pass the transpose of our item movie matrix to the model fit function Recommending similar movies It’s time to get some results. We want to find similar movies for a selected title. The implicit module offers a ready to use method that returns similar items by providing the movie index in the movie user matrix. However, we need to translate that index to the movie ID in the movies table movies_table = pd.read_csv(“data/ml-latest-small/movies.csv”)movies_table.head() def similar_items(item_id, movies_table, movies, N=5):“””Input item_id: intMovieID in the movies table movies_table: DataFrameDataFrame with movie ids, movie title and genre movies: np.arrayMapping between movieID in the movies_table and id in the item user matrix N: intNumber of similar movies to return Output recommendation: DataFrameDataFrame with selected movie in first row and similar movies for N next rows “”” Get movie user index from the mapping array user_item_id = movies.index(item_id) Get similar movies from the ALS model similars = model.similar_items(user_item_id, N=N+1) ALS similar_items provides (id, score), we extract a list of ids l = [item[0] for item in similars] Convert those ids to movieID from the mapping array ids = [movies[ids] for ids in l] Make a dataFrame of the movieIds ids = pd.DataFrame(ids, columns=[‘movieId’]) Add movie title and genres by joining with the movies table recommendation = pd.merge(ids, movies_table, on=’movieId’, how=’left’) return recommendation Let’s try it! Let’s see what similar movies do we get for a James Bond Movie: Golden Eye df = similar_items(10, movies_table, movies, 5)df Interesting recommendations. One thing to notice is that all recommended movies are also in the Action genre. Remember that there was no indication to the ALS algorithm about movies genres. Let’s try another example df = similar_items(500, movies_table, movies, 5)df Selected movie is a comedy movie and so are the recommendations. Another interesting thing to note is that recommended movies are in the same time frame (90s). df = similar_items(1, movies_table, movies, 5)df This is a case where the recommendations are not relevant. Recommending Silence of the Lambs for a user that just watched Toy Story does not seem as a good idea. Make it fancy So far, the recommendations are displayed in a DataFrame. Let’s make it fancy by showing the movie posters instead of just titles. This might help us later when we deploy our model and separate the work into Front End and Back End. To do that we will download movies that I found on Kaggle. We will need the following data: metadata movies_metadata.csv links.csv metadata = pd.read_csv(‘data/movies_metadata.csv’)metadata.head(2) From this metadata file we only need the and columns. imdb_id poster_path image_data = metadata[[‘imdb_id’, ‘poster_path’]]image_data.head() We want to merge this column with the movies table. Therefore, we need the links file to map between imdb id and movieId links = pd.read_csv(“data/links.csv”)links.head() links = links[[‘movieId’, ‘imdbId’]] Merging the ids will be done in 2 steps: First merge the poster path with the mapping links Then merge with movies_table But first we need to remove missing imdb ids and extract the integer ID image_data = image_data[~ image_data.imdb_id.isnull()] def app(x):try:return int(x[2:])except ValueError:print x image_data[‘imdbId’] = image_data.imdb_id.apply(app) image_data = image_data[~ image_data.imdbId.isnull()] image_data.imdbId = image_data.imdbId.astype(int) image_data = image_data[[‘imdbId’, ‘poster_path’]] image_data.head() posters = pd.merge(image_data, links, on=’imdbId’, how=’left’) posters = posters[[‘movieId’, ‘poster_path’]] posters = posters[~ posters.movieId.isnull()] posters.movieId = posters.movieId.astype(int) posters.head() movies_table = pd.merge(movies_table, posters, on=’movieId’, how=’left’)movies_table.head() Now that we have the poster path, we need to download them from a website. One way to do it is to use the to get movie posters. However, we will have to make an account on the website, apply to use the API and wait for approval to get a token ID. We don’t have time for that, so we’ll improvise. TMDB API All movie posters can be accessed through a base URL plus the movie poster path that we got, and using HTML module for Python we can display them directly in Jupyter Notebook. from IPython.display import HTMLfrom IPython.display import display def display_recommendations(df): images = ‘’for ref in df.poster_path:if ref != ‘’:link = ‘ + refimages += “<img style=’width: 120px; margin: 0px; \float: left; border: 1px solid black;’ src=’%s’ />” \% linkdisplay(HTML(images)) http://image.tmdb.org/t/p/w185/' df = similar_items(500, movies_table, movies, 5)display_recommendations(df) Recommendations for `Mrs Doubtfire` Put all of it into one small method def similar_and_display(item_id, movies_table, movies, N=5): df = similar\_items(item\_id, movies\_table, movies, N=N)

display\_recommendations(df) similar_and_display(10, movies_table, movies, 5) Recommendations for `Golden Eye` Conclusion In this post we implemented ALS through the implicit module to find similar movies. Additionally we did some hacking to display the movie posters instead of just DataFrame. In the next post we will see how to make recommendations for users depending on what movies they’ve seen. We will also see how we can set up an evaluation scheme and optimize the ALS parameters for. Stay tuned!

Metflix: How to recommend movies — Part 2

We recently compiled and analyzed a selection of some of the most widely used software metrics (you can download our report here ). As part of the work, we asked our users what shortcuts they had taken and wish they hadn’t. Disclosure: , the automated code reviews platform, has previously sponsored Hacker Noon. For Hacker Noon readers, they’re Codacy offering 15% off using this code: HACKERNOON . 1. Duplicated code “ ” I think one of the most valuable rules is avoid duplication. “Once and only once” is the Extreme Programming phrase. - Martin Fowler This is easily the number one rule. If you want your code to be rock solid and live forever, then duplicated code is the best way to torpedo your objectives. That’s because if you find a bug in one copy, you’ll have to fix it in all the other copies. That’s assuming that you can find them in the first place, or that you are even aware of them. This multiplies the time — and therefore costs — to test and debug. Things to consider Aim for zero duplicated code in production environment. You probably won’t get there 100%, but it’s a worthwhile goal to have. Factorize, factorize, and factorize your code. Test, test, and test your code. Food for thoughts Refactoring. Martin Fowler. . Link “Coming Full Circle On Code Duplication”. . Link When Code Duplication Is Acceptable. (Fagner Brack, 2016) . Link 2. Not measuring — or managing — code complexity “ ” Complexity kills. It sucks the life out of developers, it makes products dif cult to plan, build and test, it introduces security challenges, and it causes end-user and administrator frustration. - Ray Ozzie A measure of code complexity is cyclomatic complexity, which is the number of logical branches that the code can go through. In practice, it’s often approximated by the number of expressions such as IF, WHILE, FOR, FOREACH. The thinking is that the more control statements you have within a method, more complex it is, the harder to read and understand the code, and therefore the higher the risk of something backfiring. Things to consider Keep an eye on cyclomatic complexity. Like any metric, it is far from perfect, but at the very least it will flag potential hotspots and may even trigger some healthy re-thinking. software Remember: a method or a class is never too small. As much as possible, follow the . single responsibility principle Food for thoughts Path Testing: Independent Paths. Jeff Nyman. . Link A complexity measure. Thomas J. McCabe. IEEE Transactions on Software Engineering, Vol. SE-2, No 4, December 1976. . Link 3. Forgetting code design “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.” - C.A.R. Hoare The story goes like this: Bob X edits package A, which depends on package B. John Y edits package B, which turns out to depend on package A. Hilarity ensued. Not. Code is the way the different packages, modules, classes and methods are linked to each other. Having an idea of how a specific area of the code is connected to other areas of the code reduces the odds of turning your work into a game of Jenga. design Things to consider No silver bullet here. A healthy mix of code reviews, testing, and a pinch of software metrics (whether it’s afferent/efferent coupling, distance from the main sequence, etc.) might be helpful. Food for thoughts Single responsibility principle. Wikipedia. . Link A validation of object-oriented design metrics as quality indicators. Basili et al. 1995. . Link Software package metrics Wikipedia article. . Link Law of Demeter Wikipedia article. . Link Design Principles and Design Patterns. Object Mentor. Robert C. Martin (2000). . Link OO Design Quality Metrics. An Analysis of Dependencies. Robert C. Martin (1994). . Link 4. No tests. No code coverage “ ” Program testing can be used to show the presence of bugs, but never to show their absence. - Edsger Dijkstra Test test, and test your code. Unit tests. Functional tests. Regression tests. And keep an eye on code coverage. Code coverage is only as good as the tests that underpin it but it’s a great way to foster a culture of continuous testing. It happens to have other benefits too: it provides some level of confidence in the code, and it helps identify areas of the code that need further testing early in the development lifecycle. Things to consider Aim for code coverage of 80% approx. but remember that code coverage is only as good as the quality of the tests that drive it. High code coverage is no guarantee of quality. But low code coverage should make you pause and think. Developers should aim to spend 10% approx of their time on unit tests until they reach the targeted code coverage threshold. Food for thoughts Testing, fun? Really? . Link Code coverage. Wikipedia. . Link Testing and code coverage, Paul Johnson. . Link 5. No style guide Let’s start with the bleeding obvious: good code is code that does its job well but is also easy maintain, extend and debug. And for code to be easy to maintain, extend or debug, it must be easy to read and understand. Having style guides goes a long way towards enabling this, which is why companies like have adopted with gusto. For starters, by limiting choice, it removes unnecessary sources of distraction. Developers can focus on functionality as opposed to, say, naming conventions. It also makes sharing code easier for them. Last but not least, it provides a certain degree of : if all your functions use camelCase, then anyone who reads your code knows that `sendMessage` is a function, not a parameter or a variable. Google predictability Things to consider The style guide advice is somewhat controversial, and there are valid arguments on both sides of the debate. One side sees the benefits extolled in the above paragraph. The other side highlights the fact that style guides are like straight jackets, with no impact in the final product. Both arguments have their merits, so in the following section we list two articles with the two opposing views. Food for thoughts Coding standards horror story. . Link Why I Have Given Up on Coding Standards. Richard Rodger. . Link 6. Too many comments, or no comments at all “ ” Everything is complicated if no one explains it to you. - Fredrik Backman If writing new code is the fun bit, code commenting is the tedious chore that most of us would rather do without. In an ideal world, the code is so self-explanatory that no comments are required. However, there are times when you need to clarify what the purpose of that code is. It makes code maintenance easier and facilitates collaboration on projects. No matter how you look at it however, good code also means good comments. As a result, the time needed to comment your code should count towards the total development time. Things to consider Too much comment is counter-productive, it makes the code difficult to read and maintain. Rather than setting rigid guidelines on the number of lines of instructions per comment, follow common sense rules such as good code style on naming variables and methods, self-explanatory code, and the occasional comment explaining this bit of code does what it does. why Food for thoughts The Fine Art of Commenting. Bernhard Spuida, 2002. . Link Code as documentation. Martin Fowler. 2005. . Link 7. No code reviews or code reviews that are too long If you read this, you probably know that code reviews serve several purposes: They are the single best practice to ensure code quality. They facilitate team collaboration They help apply code standards They help identify bugs early in the development process. They help with developer onboarding and learning. But let’s sprinkle the above with some hard, cold, data points, taken from McConnell’s code complete book: “ … software testing alone has limited effectiveness — the average defect detection rate is only 25 percent for unit testing, 35 percent for function testing, and 45 percent for integration testing. In contrast, the average effectiveness of design and code inspections are 55 and 60 percent. Case studies of review results have been impressive: In a software-maintenance organization, 55 percent of one-line maintenance changes were in error before code reviews were introduced. After reviews were introduced, only 2 percent of the changes were in error. When all changes were considered, 95 percent were correct the first time after reviews were introduced. Before reviews were introduced, under 20 percent were correct the first time. In a group of 11 programs developed by the same group of people, the first 5 were developed without reviews. The remaining 6 were developed with reviews. After all the programs were released to production, the first 5 had an average of 4.5 errors per 100 lines of code. The 6 that had been inspected had an average of only 0.82 errors per 100. Reviews cut the errors by over 80 percent. The Aetna Insurance Company found 82 percent of the errors in a program by using inspections and was able to decrease its development resources by 20 percent. IBM’s 500,000 line Orbit project used 11 levels of inspections. It was delivered early and had only about 1 percent of the errors that would normally be expected. A study of an organization at AT&T with more than 200 people reported a 14 percent increase in productivity and a 90 percent decrease in defects after the organization introduced reviews. ” Jet Propulsion Laboratories estimates that it saves about $25,000 per inspection by finding and fixing defects at an early stage. Things to consider Apply code reviews systematically, for example with every commit. Also, consider adding a review column on your scrum board and make sure everyone sticks to it, including the team leader. Different people bring different perspectives, so make sure it’s not always the same people who review the code. Break the code reviews into small daily sessions and keep them short 4–5 hours per week is about right. Food for thoughts Seven Truths About Peer Reviews (Karl E. Wiegers). . Link Code complete. (Steve McConnell). . Link Code Reviews: Just Do It. Link. Coding Horror. . Link Thanks for reading! If you enjoyed it, hit that heart button below :) Also, we’ve also compiled a review of the most widely used software metrics in an ebook, free for all to download: Click to get the ebook

7 Coding Shortcuts That Don’t Work

Google is a giant in artificial intelligence. Every day, their exploits in AI make the news. As a result, AI startups can feel overshadowed by this mega-competitor, and their vision can be cloudy. Fortunately, to navigate through those murky waters, they can rely on Dr Fei-Fei Li, Director of . She is also known as the teacher of an online course on . Recently, she became the Chief Scientist of AI & Machine Learning at , but since , she still benevolently offered her expert advice about how to compete with her new employer. Stanford’s AI Lab (SAIL) neural networks for computer vision Google Cloud Google is not evil At the , she declared: Startup Grind Global conference (sponsored by Google) Each specific industry has a relatively small market size for AI, so getting deep into such market is not interesting for tech giants like Google. They prefer to build ‘horizontal’ AI, general-purpose models and Cloud infrastructure, which are used by the startups involved in the various verticals. In this context, AI startups need less AI expertise and data storage capacity, because they can find it on Google Cloud. That’s how AI is democratized, according to Fei-Fei Li. AI ‘democratization’ is a mask for Google domination With this viewpoint, startups respect the hegemony of big players in big ‘horizontal’ AI markets. In this AI democracy, startups-citizens remain quietly within their assigned vertical, subjected to the discipline imposed from the above, by the President Google Cloud. Here again, democracy is used as a rhetorical mask for imperialism. Fei-Fei Li’s philanthropic speech about democratizing AI reminds the way , while he invaded it and took over its oil. , and Google Cloud is the Bush of AI. the US President George W. Bush spoke about ‘democratizing’ Iraq in 2003 Data is the new oil American soldier guarding an oil field as a way to ‘democratize’ Iraq. Likewise, Google Cloud guards data as a way to ‘democratize’ AI. However, things should not go as planned. As Bush in Iraq, Google Cloud should meet fierce resistance on the ground. They have a quagmire ahead. More precisely, Fei-Fei Li should have a look at “ ” by the American philosopher Thomas Kuhn. In this book, she will be reminded that science is subjected to frequent and disruptive , which affect its evolution. Therefore, Google presidency on AI can be overthrown by the next scientific revolution. Google Cloud can be swept by the next desert storm. Let’s see how. The Structure of Scientific Revolutions paradigm shifts The next AI will not need Google data that much Google owns a lot of proprietary data, but maybe it is not so important in the long term. Current AI technology is based on supervised learning: in order to learn the desired behavior, the AI must be shown a large number of similar instances. A more advanced AI needs much less training data. After all, in order to learn to recognize cars, babies do not need to see them by thousands. Babies don’t need Google data to become intelligent, and neither should an advanced AI. For example, are a promising step towards this next AI. After being shown a large amount of data in some domain (e.g., millions of images, sentences, or sounds, etc.) a generative model can produce even more data like it. Therefore, a generative model can produce synthetic data to train a second model, based on supervised learning. If startups can rely on this free synthetic data, they won’t need Google Cloud. generative models Synthetic birds That’s why generative models are a strategic direction of research for the AI startup community. We are investigating generative models on the open platform, and everyone is welcome to join this effort. Startcrowd Google is struggling to incorporate fundamental research Most of Google research culture is unfit for disruptive science. Google has a ‘ ’: research is product-oriented, and it proceeds by incremental changes to existing solutions. Google can still have long-term research goals, but such large goals must be ‘factorized’ into a sequence of short-term goals, each needing to impact Google products. hybrid approach to research This research culture is unattractive for some talent. For example, many AI academics like or do not want to join a tech giant: they prefer to keep their independent research agenda, without pressure to impact company products. Yoshua Bengio Jürgen Schmidhuber Moreover, this Google science policy is vulnerable to paradigm shifts. A lot of major technological progress comes from peripheral topics, which had little impact before, and which suddenly move to the central stage. One such example is the meteoric rise of AI and neural networks. Within a couple of years, this academic curiosity became the backbone of a new industrial revolution. Mainstream technologists did not see AI coming, and Google founder Sergey Brin was not different. Google knows the limits of their short-term approach to research. That’s why they establish separate divisions for long-term projects. In AI, their main one is the London-based DeepMind. However, the problem with this policy is that ideas do not circulate well enough between DeepMind and the rest of Google. DeepMind remains quite insulated from the Google mothership. How many Googlers are really following DeepMind research? I guess not much, and that would be interesting to survey Googlers about it. To remedy this lack of knowledge transfer within Google itself, DeepMind opened a , and future will tell if it’s effective. dedicated team at Google HQ in California AI startups can better combine fundamental and applied research The boldest startups can find inspiration from this situation. They can dedicate part of their resources to fundamental research. This perk can attract the right kind of talent, and keep teams stimulated. Startups can go beyond the usual 20% projects allowed to Googlers, and establish a policy of 50% of fundamental research projects. Google has 20% projects, AI startups should raise to 50% When fundamental and applied research are conducted by the same people, with dual specializations, the bandwidth between the two is increased, which helps provoking improbable but high-impact breakthroughs. This takes Google’s ‘hybrid approach to research’ one step further: now there is no pressure to impact company products. This science policy is easier to implement in a small startup with a flat hierarchy. In order to climb the corporate ladder of a large organization like Google, it is a bad strategy to perform research off the beaten path. On the other hand, in a startup, there is no ladder to climb, and much less incentives to conformism. Some conservative Venture Capitalists might dislike this approach to research, but for bold startuppers, it will certainly be a lot of fun, and they may even change the world. AI startups can use the Cloud to subvert Google Finally, AI startups can use the Cloud to ‘democratize’ AI within Google itself. AI can be ‘democratically’ spread the other way round. For example, by doing their R&D in the open, AI startups can attract curious Googlers with their AI challenges, and distract them away from their daily jobs. For example, like OpenAI, they can propose their own ‘ ’. They can even go one step further, and use a dedicated Cloud platform: . Requests for Research Startcrowd With Startcrowd, Googlers just sit from an exciting AI startup adventure. They can start awesome AI side-projects, while staying comfortably within the walls of the Googleplex, with their monthly paycheck from Google. On Startcrowd, Googlers can enjoy a frictionless startup experience, until they decide to work on their AI project full-time. one click away What if angry managers find out? No problem, they won’t. Participation can be anonymous, and for additional discretion, user interface will soon be customizable, for camouflage purposes. Forget about Facebook and YouTube: Startcrowd is the place to go for smart procrastination. Life is short. Time to liberate your creativity. Work on problems you love. The Googleplex. a good place for . startcrowdination Between Google Cloud and Startup Crowd, the battle will be epic. They can acquire us with their money and data, but we can infiltrate them with our side-projects. Googlers can start their side-projects in AI on www.startcrowd.club

Stories, News & Mentions about FACTORIZE

Factorize’s Stories on HackerNoon