The Answers Platform for Hyperlocal Questions

We're a UC Berkeley-incubated startup reshaping the future of retail with the world’s first fully conversational AI platform designed exclusive

Dotlas

dotlas

Async/await in C# is a framework used for writing asynchronous C# code that is both readable and maintainable. These tips will help you to integrate async/await programming more effectively in the # projects: 1. ValueTask for Lightweight Operations Use ValueTask instead of Task for asynchronous methods that often complete synchronously, reducing the allocation overhead. public async ValueTask<int> GetResultAsync()
    {
        if (cachedResult != null)
            return cachedResult;
        int result = await ComputeResultAsync();
        cachedResult = result;
        return result;
    } 2. ConfigureAwait for Library Code Use ConfigureAwait(false) in the library code to avoid deadlocks by not capturing the synchronization context. public async Task SomeLibraryMethodAsync()
    {
        await SomeAsyncOperation().ConfigureAwait(false);
    } 3. Avoiding async void Prefer async Task over async void except for event handlers, as async void can lead to unhandled exceptions and is harder to test. public async Task EventHandlerAsync(object sender, EventArgs e)
    {
        await PerformOperationAsync();
    } 4. Using IAsyncDisposable For asynchronous cleanup, implement IAsyncDisposable and use await to ensure resources are released properly. public async ValueTask DisposeAsync()
    {
        await DisposeAsyncCore();
    }
    
    private async ValueTask DisposeAsyncCore()
    {
        if (resource != null)
        {
            await resource.DisposeAsync();
        }
    } 5. Efficiently Combine Tasks Use Task.WhenAll for running multiple tasks in parallel and waiting for all to complete, which is more efficient than awaiting each task sequentially public async Task ProcessTasksAsync()
    {
        Task task1 = DoTask1Async();
        Task task2 = DoTask2Async();
        await Task.WhenAll(task1, task2);
    } 6. Cancellation Support Support cancellation in asynchronous methods using CancellationToken. public async Task DoOperationAsync(CancellationToken cancellationToken)
    {
        await LongRunningOperationAsync(cancellationToken);
        cancellationToken.ThrowIfCancellationRequested();
    } 7. State Machine Optimization For performance-critical code, consider structuring your async methods to minimize the creation of state machines by separating synchronous and asynchronous paths. public async Task<int> FastPathAsync()
    {
        if (TryGetCachedResult(out int result))
        {
            return result;
        }
        return await ComputeResultAsync();
    } 8. Avoid Blocking Calls Avoid blocking on async code with .Result or .Wait(). Instead, use asynchronous waiting through the stack. public async Task WrapperMethodAsync()
    {
        int result = await GetResultAsync();
    } 9. Eliding Async/Await In simple passthrough scenarios or when returning a task directly, you can elide the async and await keywords for slightly improved performance. public Task<int> GetResultAsync() => ComputeResultAsync(); 10. Custom Task Schedulers For advanced scenarios, like limiting concurrency or capturing synchronization contexts, consider implementing a custom TaskScheduler. public sealed class LimitedConcurrencyLevelTaskScheduler : TaskScheduler
    {
        // Implement the scheduler logic here.
    } 11. Using Asynchronous Streams Leverage asynchronous streams with IAsyncEnumerable<T> for processing sequences of data asynchronously, introduced in C# 8.0. public async IAsyncEnumerable<int> GetNumbersAsync()
    {
        for (int i = 0; i < 10; i++)
        {
            await Task.Delay(100); // Simulate async work
            yield return i;
        }
    } 12. Avoid Async Lambdas in Hot Paths Async lambdas can introduce overhead. In performance-critical paths, consider refactoring them into separate async methods. // Before optimization
    Func<Task> asyncLambda = async () => await DoWorkAsync();
    
    // After optimization
    public async Task DoWorkMethodAsync()
    {
        await DoWorkAsync();
    } 13. Use SemaphoreSlim for Async Coordination SemaphoreSlim can be used for async coordination, such as limiting access to a resource in a thread-safe manner. private readonly SemaphoreSlim semaphore = new SemaphoreSlim(1, 1);
    
    public async Task UseResourceAsync()
    {
        await semaphore.WaitAsync();
        try
        {
            // Access the resource
        }
        finally
        {
            semaphore.Release();
        }
    } 14. Task.Yield for UI Responsiveness Use await Task.Yield() in UI applications to ensure the UI remains responsive by allowing other operations to process. public async Task LoadDataAsync()
    {
        await Task.Yield(); // Return control to the UI thread
        // Load data here
    } 15. Asynchronous Lazy Initialization Use Lazy<Task<T>> for asynchronous lazy initialization, ensuring the initialization logic runs only once and is thread-safe. private readonly Lazy<Task<MyObject>> lazyObject = new Lazy<Task<MyObject>>(async () =>
    {
        return await InitializeAsync();
    });
    
    public async Task<MyObject> GetObjectAsync() => await lazyObject.Value; 16. Combining async and LINQ Be cautious when combining async methods with LINQ queries; consider using asynchronous streams or explicitly unwrapping tasks when necessary. public async Task<IEnumerable<int>> ProcessDataAsync()
    {
        var data = await GetDataAsync(); // Assume this returns Task<List<int>>
        return data.Where(x => x > 10);
    } 17. Error Handling in Async Streams Handle errors gracefully in asynchronous streams by encapsulating the yielding loop in a try-catch block. public async IAsyncEnumerable<int> GetNumbersWithErrorsAsync()
    {
        try
        {
            for (int i = 0; i < 10; i++)
            {
                if (i == 5) throw new InvalidOperationException("Test error");
                yield return i;
            }
        }
        catch (Exception ex)
        {
            // Handle or log the error
        }
    } 18. Use Parallel.ForEachAsync for Asynchronous Parallel Loops Utilize Parallel.ForEachAsync in .NET 6 and later for running asynchronous operations in parallel, providing a more efficient way to handle CPU-bound and I/O-bound operations concurrently. await Parallel.ForEachAsync(data, async (item, cancellationToken) =>
    {
        await ProcessItemAsync(item, cancellationToken);
    }); 22. Avoid Excessive Async/Await in High-Performance Code In performance-critical sections, minimize the use of async/await. Instead, consider using Task.ContinueWith with caution or redesigning the workflow to reduce asynchronous calls. Task<int> task = ComputeAsync();
    task.ContinueWith(t => Process(t.Result)); 23. Async Main Method Utilize the async entry point for console applications introduced in C# 7.1 to simplify initialization code. public static async Task Main(string[] args)
    {
        await StartApplicationAsync();
    } 24. Optimize Async Loops For loops performing asynchronous operations, consider batching or parallelizing tasks to improve throughput. var tasks = new List<Task>();
    for (int i = 0; i < items.Length; i++)
    {
        tasks.Add(ProcessItemAsync(items[i]));
    }
    await Task.WhenAll(tasks); More Cheatsheets Cheat Sheets - .Net C# Programming🚀 Thank you for being a part of the C# community! Before you leave: Follow us: | | | Visit our other platforms: More content at Youtube X LinkedIn Dev.to GitHub C# Programming Also published . here

Async/Await For Basic to Advanced C# Developers: 24 Essential Tips

Introduction Remember when Femtech was a brand-new term nobody knew? It feels like we have been using this word forever, but it has been around for less than a decade! Yet the history of products, technologies, and services addressing health issues specific or prevalent to women started long ago, ages before this umbrella term was first used in 2016 to address this rapidly evolving market. Since then, this industry has had major plot twists, from Maven Clinic becoming the first unicorn in 2021 to venture capital infusions shaking up the game; the sector is sizzling hot! Femtech And let us give a thumbs-up to the growing acceptance and issues. It is no longer a niche but a booming market, with its size estimated at over $55 billion in 2022 and . de-stigmatization of women’s health expected to surpass $100 billion by 2030 Why is it important to me? I am a digital tech specialist coming from the Artificial Intelligence/ Machine Learning field with a rich background in building and successfully deploying voice recognition, AI cameras, and virtual assistant products catering to millions. Lately, I have shifted my focus, stepping into the intriguing world of Femtech and healthcare drawn by the promise of AI to simplify and enhance healthcare solutions, changing stagnant approaches to women’s healthcare through daring initiatives. With this article providing a thoughtful inquiry into how advanced technologies like LLMs shape modern healthcare and Femtech, I invite you to join me on this exciting journey. Together, we will take a look at how boosts the Femtech market and sift through the new directions and strategies reshaping this burgeoning sector. Our aim will be not only to learn about the tendencies of the industry but also to identify key patterns that could offer thrilling opportunities. And so, if you are curious like me about how novel technologies improve healthcare experiences for women and what strategic footholds exist in the Femtech market, let us dive in and explore. AI/ML Current trends and thrilling developments in Femtech In the last few years, the Femtech sector has been undergoing a transformative shift. Gone are the days when Femtech products predominantly focused on menstrual cycle tracking and fertility planning. Today, the industry is blossoming with more specialized solutions, from advanced home-testing kits to cutting-edge wearables. Technological advances have attracted a range of strong players in a multitude of niche markets, each aiming to revolutionize different aspects of women’s health. 1. From Basic Tracking to Comprehensive Health Solutions Take Sonio, a Paris-based Femtech firm. They recently received FDA clearance for their AI product, Sonio Detect, which aims to redefine prenatal ultrasounds. What sets Sonio apart is its manufacturer-independent tool that simplifies the screening process while raising its efficiency and accuracy, offering an easy-to-use yet powerful solution. San Francisco startup Mira is pioneering another monumental shift in Femtech by embedding lab technology into a handheld fertility monitor. With a user base exceeding 76,000, Mira’s advanced fluorescent detector goes beyond traditional ovulation kits. The technology allows for daily health checks and has accumulated an impressive amount of data — over 8.5 million hormone data points and analysis of more than 670,000 menstrual cycles. With AI-driven insights into hormone fluctuations, Mira is offering more personalized and accurate predictions than ever before, quite literally putting a mini hormone lab in the palms of its users. Recently, The Inc. placed Mira at number 437 on its 2023 list of fastest-growing private companies in the US, ranking it the highest among Femtech companies. More complex products that require significant medical research are also quickly gaining momentum. Teal Health exemplifies this with its recent $8.8 million seed funding round, backed by prominent investors like Emerson Collective and Serena Ventures. The funds aim to build a comprehensive platform, starting with the crucial issue of cervical cancer. women’s telemedicine 2. The Rise of Wearable Innovations Wearable technology is another frontier where Femtech is making significant strides. One famous example is Bloomlife, a clinically validated diagnostic device for pregnant women that allows future mothers to track contractions, giving them a better awareness of their progress. Movano Health has captured attention with its Evie Ring — a wearable gadget currently under examination by the FDA. Upon receiving FDA clearance, the Evie Ring will have the potential to set new standards in women’s health monitoring, offering an integrated, round-the-clock health monitoring and tracking solution right at the user’s fingertips. Ava, a fertility and pregnancy tracking bracelet with an accuracy rate of over 90%, has already been cleared by the FDA. Established in 2014, well before we even started using the term ‘Femtech,’ Ava is a prime example of a successful Femtech startup. Having attracted over $42 million in funding, the company reported to have helped over 70 thousand women in 36 countries to get pregnant by 2020. 3. The Expansion to Different Age Groups One of the noteworthy trends in Femtech recently is the surge in startups focusing on menopause. According to Female Insights, over 30% of companies in this sector were founded in the last three years alone. This uptick is accompanied by increasing diversity in the types of solutions being offered to tackle menopausal symptoms — from mobile apps that track hormones and symptoms and platforms facilitating user interaction and community building to consumer goods and therapeutic medications. According to Pubmed, by 2030, up to 1.2 billion women will be in or approaching menopause globally, with 47 million new women joining every year. By 2025, it is estimated that over 1 billion people worldwide—about 12% of the population—will be going through menopause. Financially, the menopause product market was valued at $15 billion in 2020 and is projected to skyrocket to $23 billion by 2028. These figures illustrate a maturing market with significant financial potential, underscoring why it has become a magnet for innovation and investment. An Irish startup, IdentifyHer, offers menopausal women sensor solutions. Controlled through a smartphone app, the sensor passively gathers information on menopause symptoms without interfering with the wearer's daily activities. The app also makes use of the data gathered to track progress and assess the effectiveness of lifestyle changes. The collected data is presented in periodic reports and can be used by medical professionals to diagnose and treat menopausal symptoms. AI-driven Midday is another customized menopause product developed by US-based startup Lisa Health that offers comprehensive management of the menopause journey. Midday’s solution, including a wearable device, an AI-based symptom tracking system, and a decision support tool for hormone therapy, uses advanced tracking to determine the menopause stage and anticipate symptoms. The app is used by doctors to manage the entire menopause process and provide decision support, while users are provided with a variety of therapeutic options and informational resources to give them personalized insights and get them ready for every stage of menopause. 4. Going Beyond Just Health Although Femtech is considered, first of all, a rising star in the Medtech industry, the possibilities of women-oriented digital technology products are not limited to healthcare alone. Recently, the term started to see new applications, one of which is Women’s Safety. Indian startup Shakti Wearables manufactures Shakti, a women’s defensive wearable device that increases women's safety in dangerous situations. When the wearer activates the button on the device, a small, non-lethal current is released, electrifying an attacker, while the internal insulation of the bangle keeps the wearer safe; this gives women a chance to flee their attackers and call for assistance. Additionally, Shakti sends the wearer's most important three contacts a WhatsApp message with their most recent live location. Another interesting personal safety app is being developed by Australian startup Safely, offering women peer-to-peer support in tense or frightening circumstances. With their contacts, users of the app can create private groups or circles for internal communication and location sharing. In case of emergency, users can also switch on the high-alert mode and notify each of their designated guardians. The app then uses live monitoring, records live video and audio, sends GPS updates, and saves all of this data to the cloud for later use as evidence. Additionally, users receive safety alerts from locations of interest to help them plan their commute or travel safely. Expanding healthcare’s reach with new technologies The world of healthcare is rapidly evolving, especially when it comes to Femtech. This dynamic transformation is powered by modern innovative technologies that are reshaping how women engage with healthcare and how healthcare providers offer services tailored to them. 1. The Personalization Revolution Fueled by LLMs One of the most disruptive advancements in this domain is the rise of Large Language Models (LLMs) like ChatGPT and GPT-4. Far from being a mere tech fad, LLMs are revolutionizing healthcare, with Femtech embracing this new technology. LLMs have the potential to serve as virtual nursing assistants, supporting busy doctors in responding to patients' basic queries about medications or health conditions. As you will see from the examples below, LLMs are proving to be instrumental in facilitating the creation of highly promising Femtech products, from personalized consulting products to innovative community engagement tools. One of the most intriguing examples of the implementation of this technology in Femtech is Soula, a Cyprus-based startup that provides women with support during pregnancy, childbirth, and motherhood with its "AI-powered" assistant. Powered by Med-PaLM, an LLM developed by Google to generate answers to medical questions, the app is expected to be able to respond to inquiries about pregnancies and maternity issues and provide information on a range of health-related concerns. It will also be able to answer questions about diet, sleep, physical activity, and emotional self-care, as well as track fetal movement and kick frequency. Mexican startup Kuna utilizes an LLM-powered chatbot to give women a secure way to access information and advice on parenting and sexual health. The startup's , Kuna Chat, offers details on sex, reproduction, pregnancy, and parenthood. The startup also provides online consultations with experts for more complex or difficult questions. In this way, Kuna encourages sex education for the uneducated while offering trustworthy, practical, and concrete answers to users' queries. AI-powered chatbot 2. Cost-Effective Health Management An extraordinarily important aspect of building successful medical products is to make services affordable to the customer, reaching broader but less solvent user segments and geographies. In this vein, Femtech offers not only personalized but also cost-effective healthcare solutions. Apps often provide free basic versions, with premium services available through subscription fees. Quite obviously, the use of LLMs trained on medical data sets and specialized in specific product cases can provide a boost to the more affordable cost of the products themselves. For example, the Ava fertility bracelet mentioned in the previous section of this article costs around $249 in the U.S., and the accompanying app is free with the bracelet purchase. When considering that the average cost of infertility treatment in the U.S. is around $11,000—often not covered by basic healthcare plans—Femtech presents a more affordable starting point for many women. 3. Breaking Barriers in Remote and Rural Areas Digital technology in Femtech is also crucial for improving healthcare access in remote and rural areas, especially in low- or middle-income countries in Africa and Asia — two regions with the highest female populations. Femtech provides innovative solutions such as virtual prenatal visits and remote monitoring, to manage their health more cost-effectively, regardless of their location. empowering women One successful example of such a product is M♡THer – a virtual care product facilitating the diagnosis of gestational diabetes mellitus by measuring blood glucose levels, blood pressure, weight, and more. GDM affects up to 20% of pregnant women, leading to various birth complications, and M♡THer, developed by the Australia E-Health Research Centre, has already benefited thousands of women in remote areas of the region. 4. A Multifaceted Approach to Chronic Diseases Femtech is not solely about reproductive health; it also targets female-specific chronic diseases through a mix of prevention, societal change, and technology. Mobile apps and wearable devices make it easier to manage and monitor conditions over time, a vital component of tackling chronic diseases that affect women disproportionately. To name an example of a health issue specific to women, it is estimated that 1 in 10 women have endometriosis – the growth of endometrial tissue outside of the uterus that can cause significant problems from pelvic pain to infertility. DotLab, a US-based medical diagnostics company, pioneers medical tests in this area, applying machine learning mechanisms to diagnose and assess this currently incurable disease. On the other side of the Atlantic Ocean, Kheiron Medical Technologies, a UK Medtech startup, combines data science and radiology to help with the diagnosis of breast cancer, another terrible disease affecting predominantly women. According to published clinical trial results, the company’s breakthrough technology achieves previously unseen heights in malignancy detection in breast cancer screening. Applied Conclusions Recent years have shown rapid exponential growth in the number of Femtech companies and products, both through the use of innovative technologies and through the discovery of more and more new applications and solutions. With the Femtech market estimated to reach an imposing $100 billion globally by 2030, it is one of the areas of the modern technology economy that is expected to provide maximum lucrative opportunities for new bright ideas and prosperous businesses in the coming years. While deciding whether to follow this attractive route, aspiring Femtech entrepreneurs should consider the following market characteristics. While there are highly competitive niches in the Femtech market, specifically well-established fields like menstrual cycle and pregnancy tracking, there are also areas that are more attractive for entry. Examples of such areas include app-controlled wearables and menopause management. The largest share of the global women’s health market surprisingly pertains to chronic women’s health conditions (forecast to reach $218 billion by 2027), and only 1 percent of existing Femtech companies address this area. Did you hear that? That is an opportunity knocking. AI and specifically Large Language Models can be a powerful boost for Femtech products as (a) with proper customization, LLMs assist in scaling the product or service itself and strengthen the core part of the product, and (b) they provide magnificent tools for community development, and the community is said to be the most important part of a product! Organic communities built around products are both a way of monetization and a tool that increases retention, helping build durable community-led growth. Taking into account the fact that the Femtech market is rapidly developing with a simultaneous increase in competition, and given the general market conditions (in particular, subscription applications revenues are currently declining; even if positive dynamics have remained in the Medtech segment, it has become more difficult to attract and retain users), companies need to be cost-efficient and provide a highly personalized, secure service to survive and rise above the competitors. A relevant and extremely effective tool employed by more and more technology startups for achieving these goals, again, is AI technologies. Whether it's addressing questions about menstrual health, pregnancy, or chronic conditions, LLMs can fill the information gap quickly and efficiently. Statistically, only 6% of Femtech companies used AI in 2021, mostly in diagnostics and data analytics, and while this share has seen stable growth over the years, there is enough room for new inspired ideas and success stories in this area. All in all, the advent of heralds an exciting era for Femtech entrepreneurs. LLMs have the potential to revolutionize user engagement and content delivery, supporting busy healthcare professionals by handling preliminary patient communication, serving as virtual nursing assistants or health advisors, and simply bringing a personalized touch to technology. With cutting-edge AI technologies charging Femtech products, humanity has the chance to drastically improve the efficiency of the entire healthcare system catering to women. Finally, there is a chance to overcome the millennia-old barriers plaguing this area of medicine, such as high costs, gender bias, stigmatization of women's health-related topics, and a lack of resources. Large Language Models Powered by the latest technological advancements, Femtech waits for its next hero — and it may be you!

Beyond Wellness: How Femtech is Shaping the Future of Women's Healthcare

সুস্থতার বাইরে: কীভাবে ফেমটেক মহিলাদের স্বাস্থ্যসেবার ভবিষ্যত গঠন করছে

Beyond Wellness: Wie Femtech die Zukunft der Frauengesundheit gestaltet

Más allá del bienestar: cómo Femtech está dando forma al futuro de la atención médica de las mujeres

Au-delà du bien-être : comment Femtech façonne l'avenir des soins de santé des femmes

कल्याण से परे: कैसे फेमटेक महिलाओं के स्वास्थ्य देखभाल के भविष्य को आकार दे रहा है

ウェルネスを超えて: フェムテックが女性のヘルスケアの未来をどのように形作るか

웰니스를 넘어서: Femtech가 여성 건강 관리의 미래를 형성하는 방법

Além do bem-estar: como a Femtech está moldando o futuro da saúde feminina

За пределами здоровья: как Femtech формирует будущее женского здравоохранения

Wellness'ın Ötesinde: Femtech Kadın Sağlığı Hizmetlerinin Geleceğini Nasıl Şekillendiriyor?

Ngoài sức khỏe: Femtech đang định hình tương lai của ngành chăm sóc sức khỏe phụ nữ như thế nào

超越健康：Femtech 如何塑造女性医疗保健的未来

's Goroutines make it easy to run code concurrently. We can simply add the keyword “go” in front of a function call to make it run in a separate routine, or asynchronously. Golang However, Go routines on their own don't offer much benefit unless we use primitives to help us either share data between routines or make routines coordinate with each other. In this blog post, we will look at a few handy primitives provided by the sync package. Mutex Sometimes two threads might attempt to access the same data simultaneously. The most representative example of this is when two routines are manipulating a variable holding your bank balance. The two routines will perform the following operations: Read the balance to a local variable; Add 100; Write the local variable to the shared variable. Now when this operation is performed by two routines interleaving the write after the read operation, it can produce a wrong result. A simple way to prevent this issue is using a . A Mutex, once it is locked, ensures that no other routine can lock the same Mutex until it is unlocked. Mutex is one of the most basic primitives and is used in the implementation of most other synchronization primitives that will be listed below. Mutex Note: in Golang it is common to use the defer keyword with mutex.Unlock(). You will see the following code at the beginning of a function: defer mu.Unlock() mu.Lock() Adding this at the top of a function ensures that only one routine can access the function at a time. Calling unlock with defer is also convenient since it ensures that the mutex is unlocked in the event of an error or unexpected return from the function. Channels Channels are by far the most practical synchronization primitive offered by the sync package. We can use channels to handle most of the synchronizing we need. If you encounter a synchronization problem that is difficult to solve, it can likely be restructured to make use of channels. So let's first look at how a channel works and the different ways it can be used. Mental Model Channels allow us to enable communication between two goroutines. One way to think about a channel is as a queue data structure that supports only the enqueue and dequeue methods. If we were to implement a queue data structure we would have to handle the below cases: What happens when we call dequeue on an empty queue; What happens when we call enqueue on a queue that is full. In a typical queue data structure, dequeue can be handled by returning nil or an error while enqueue can be handled by returning an error or more elaborately by resizing the queue. Channels handle these scenarios by blocking the routine until the condition is no longer true. Specifically: Calling dequeue on an empty channel blocks the calling routine and switches to another routine (i.e. a context switch). Once the channel is no longer empty, Golang returns to the calling routine and the function returns an available value; Calling enqueue on a full channel blocks the calling routing and switches to another routine (i.e a context switch). Once the channel is not full, Golang returns to the calling routine and returns from successfully enqueuing data to the channel. So, enqueuing on a full channel and dequeuing on an empty channel are necessary conditions because they determine when a routine will block (or when a context switch will occur). This makes the second parameter, size, to the make function quite important. make(chan int, 10) creates a channel of size 10 make(chan int) creates a channel with a default size of zero For a channel of size zero, these two conditions are true for each enqueue and dequeue operation, i.e. the queue is simultaneously full and empty. In other words, any attempt to read or write from a channel of size 0 will cause the routine to block immediately unless there is a reader or writer blocked awaiting such an operation on the other side of the channel. This can therefore be used as a signalling mechanism.Channels are a reference type and therefore can be passed between functions without the need for a pointer. Non-Blocking Writes In Go, we can use the select statement to perform non-blocking operations on channels. This can be useful when we don't want a routine to get blocked while attempting to read from or write to a channel. To do a non-blocking write, we can use a select statement with a default case. This way if the channel is full the routine will not get blocked. select { }
} // define a channel of size 2, which can buffer a maximum two numbers numbers := make(chan int, 2 ) data := []int{ 1 , 2 , 3 , 4 } for x := range data { case numbers <- x: fmt.Println( "wrote value" , x) default : fmt.Println( "didn't write since channel is full" ) /* Output: wrote value 0 wrote value 1 didn't write since channel is full didn't write since channel is full */ Non-Blocking Reads Sometimes it's important for our routine to attempt to read a value if one exists or proceed to do something else. That is a non-blocking enqueue function. This can also be achieved via the select function: ​ select { channel := make(chan int) // Attempt a non-blocking read from our channel case val := <-channel: fmt.Println("value in chan is ", val) default: fmt.Println("No value in channel") } ​ // Output: // No value in channel As you can see the calling function will not block and will hit the default case in the select statement. We may spawn N workers and check if any one of them passed back an error, which may or may not exist. Reading from Multiple Channels The Select statement can also be used to read from multiple channels. We can use a select statement around two channels to block until either one produces a value. We can also add a default to the above code to make the select statement non-blocking, in which case we will attempt to get a value from channel A; if it’s unavailable, we then attempt to get a value from channel B, and if unavailable, hit the default case. ​ select { fmt.Println("value from ChannelA", valA)
   case valB := <-channelB:
      fmt.Println("value from ChannelB", valB)
   }
}()
​
channelA <- 1
time.Sleep(1) channelA := make(chan int) channelB := make(chan int) // Attempt a non-blocking read from our channel go func ( ) { case valA := <-channelA: Channel Closing Closing a channel is a great way to indicate that we are done sending data as well as the workers listening on these channels, and stop listening to them. Below we will look at an example of reading from channels manually and by using a range loop. if !ok {
          break
      }
      fmt.Println("1: read value from channel val", val)
   }
   fmt.Println("1: channel is closed")
}
​
func forRangeConsumer(channel chan int) {
   for val := range channel {
      fmt.Println("2: read value from channel val", val)
   }
   fmt.Println("2: channel is closed")
}
​
func main() {
   numbers := make(chan int)
​
   // Spawn two consumers
   go manualConsumer(numbers)
   go forRangeConsumer(numbers)
​
   // write all data into channel
   data := int{1, 2, 3, 4, 5}
   for x := range data {
      numbers <- x
   }
​
   // close channel using go provided 'close' function 
   // to indicate the end of data to consumers.
   close(numbers)
​
   // Give some time for consumer routines to exit
   // we can do this more beautifully with use of waitGroups 
   // but for simplicity sake use sleep
   time.Sleep(1)
} func manualConsumer ( channel chan int ) { for { val, ok := <-channel A few points to note here: Once a channel is closed it can't be closed again. Otherwise, this will cause a panic. When reading from a closed channel, all data pushed into the channel is first returned. Once all data is read, subsequent reads will return the default value of channel type as the first parameter and false as the second parameter. Data can’t be pushed into a closed channel. Wait Groups Wait Groups are a very simple synchronization primitive. Their primary use case is to wait for all threads to complete before proceeding to the next step in the program. Mental Model A wait group is nothing more than a class with an integer. The class provides three functions: Add, Done, and Wait. These functions manipulate the integer internally in a thread-safe manner (you can think of this as an atomic int or an integer with a lock). Let me expand a bit on the 3 main functions: Add(delta int) adds  the "delta"  value to the atomic integer; Done() decrements the atomic integer value by one; Wait() blocks the caller until the atomic integer value is zero. The Done function cannot be called more times than the delta value. In other words, the internal integer value cannot be zero. When passing a to a function, pass it by reference to avoid making a copy of the WaitGroup (i.e is not a reference type). WaitGroup WaitGroup is commonly used in situations where the main thread should wait for workers to complete. WaitGroup wg.Done()
}
​ go worker(&wg)
   }
​ wg.Wait()
} func worker ( wg *sync.WaitGroup ) { fmt.Println( "working ..." ) func main ( ) { wg := sync.WaitGroup{} for i := 0 ; i < 5 ; i++ { wg.Add( 1 ) fmt.Println( "I am doing some work" ) Condition Variables Condition variables allow multiple routines to coordinate with each other. A condition variable has the following functions: : This is a helper function or constructor used to create a new condition variable; NewCond(mu Locker) blocks a routine until a different routine calls Broadcast or Signal. This function also ensures that the Locker is released upon blocking and reacquired before returning from the function; Wait() notifies all blocked routines to start; Broadcast() notifies a single blocked routine to start; Signal() The that is passed when constructing the object is public and accessible via the field. Locker L Note that we can mimic the signal function above with the use of channels. i.e. by executing a non-blocking write to a channel with a buffer size of 0. If a routine is blocked on reading from the channel, it will get unblocked. However, it's difficult to implement the broadcast operation with the use of channels. You might feel like it's possible to implement a broadcast by calling close on a channel, which in turn would unblock all routines that are blocked on attempting to read the channel. This is true, but the channel cannot be reused for a second broadcast which a condition variable can do. To better understand the usefulness of condition variables, let's look at an example where a group of workers are working on two tasks, A and B. Task B is dependent on Task A, meaning all workers must finish working on Task A before they can start working on Task B. This is essentially the implementation of a barrier. By using a condition variable, we can easily implement this synchronization logic and ensure that all workers complete Task A before moving on to Task B. }
​ cond.L.Lock()
   defer cond.L.Unlock() *sharedCounter--
​ cond.Broadcast() } cond.Wait()
}
​ ​ ​ notifyAllOrWaitForOthers(workerID, sharedCounter, cond)
​ ​
   wg.Done()
}
​ mu            = &sync.Mutex{}
      cond          = sync.NewCond(mu)
      wg            = &sync.WaitGroup{} sharedCounter = &numWorkers
   )
​
   wg.Add(numWorkers) go worker(i, &activelyWorking, cond, wg)
   }
   wg.Wait()
} func doTask ( workerID int, taskName string ) { fmt.Printf( "worker %d is doing task %s\n" , workerID, taskName) taskDuration := time.Duration(rand.Intn( 5 )) time.Sleep(taskDuration) // represents work workerID doing taskName func notifyOrWaitForOthers ( workerID int, sharedCounter *int, cond *sync.Cond ) { // Worker has finished his job so decrements the shared counter // You are the last to complete the task, best notify the rest of the workers. if *sharedCounter == 0 { fmt.Printf( "worker %d is the last to complete work\n" , workerID) fmt.Printf( "last worker signals all workers that they can now proceed with next task\n" ) return // You finished early, let's wait till the rest are done. /* Wait is a special function. Three things will happen upon calling it 1. cond.L.Unlock will be called 2. scheduler will switch to a different routine and will run it 3. Before the wait call returns, cond.L.Lock will be called. */ func worker ( workerID int, sharedCounter *int, cond *sync.Cond, wg *sync.WaitGroup ) { // All workers start on task A doTask(workerID, "A" ) // Once task A is done, you either wait for others or if your the last // to complete the task notify the rest they can start working on task B // All workers start on task B doTask(workerID, "B" ) func main ( ) { var ( numWorkers    = 3 for i := 0 ; i < numWorkers; i++ { You might feel like it is possible to implement the broadcast by calling close on a channel since this would unblock all routines blocked on attempting to read the channel. Although it is possible, the channel cannot be reused for a second broadcast, unlike a condition variable that can be used repeatedly. The example shown above demonstrates how condition variables can be used to coordinate multiple routines working on two tasks A and B. However, this concept can be extended to handle more complex scenarios where the result of the work done in the previous task is important to complete the task that follows. Sources The Go Standard Library Documentation: https://pkg.go.dev/sync An Introduction to Programming in Go, Chapter 10, Concurrency: https://www.golang-book.com/books/intro/10 Luke Champine, The Pros of Cond’s: https://lukechampine.com/cond.html The featured image was generated with Kadinsky 2 Prompt: Illustrate a screen displaying lines of code.

Mastering Synchronization Primitives in Go 

“‘Better social media’ is the new category.” Okay, sounds like some far-fetched idea from a Silicon Valley CEO that is about to pour millions of dollars into the next , even though they are so removed from the average user experience they wouldn’t understand why an edit button is needed. Twitter Wait, that was a 16-year-old that hasn’t finished high school yet!? When I sat down for in October to talk to Spencer and Sophia Rascoff, a father and daughter duo that mixes an impressive resume with impressive ideas, I didn’t know what I was in for. Success Story I assumed it would be Spencer driving the conversation — after all, he is a co-founder of Zillow, Hotwire, and a handful of other successful startups. But it was Sophia, his incredibly articulate daughter that had me scribbling quotes and advice. They have recently launched which is what they call a “vertical social media network.” That’s something focused on a single space (in this case, food), instead of a broad reach, and it just might be the next evolution of how we interact online. Recon Food, Teenaged success story “I’ve had a few lemonade stands.” That’s Sophia again with a perfect one-liner to explain her experience at such a young age. She’s not your average 16-year-old. In middle school, she decided to start an entrepreneurship program based on the kinds that are in high schools and colleges. She is a recipient of the President’s Volunteer Service Award, a leader in the Latin American/Hispanic Student Organization, and has taught herself software development on the side. She’s not even a senior yet. It helps to have a dad with such a strong resume. Spencer went from Goldman Sachs to private equity, founding Hotwire, working for Expedia, launching Zillow, and on and on. He created (and taught) the Harvard Business School course , and now hosts his successful podcast on the dot.LA network (which he also founded). Managing Tech Ventures If you are thinking that Sophia might take over the world, you’re right. But after talking to her for a little while, there might not be any better hands to put it in. Don’t mix your cupcakes with climate change So what does it mean to evolve social media? The Rascoffs aren’t trying to create a new . They aren’t trying to outdo . While those platforms chase down broad engagement and encourage outrage, she just wants to share photos of her favorite food. Instagram Facebook It’s not that simple, but it kind of is, too. I’ll let Spencer explain: “We baked cupcakes this weekend, right? And posted them on Recon Food. But Instagram was pretty heavy on Sunday. When you were making those cupcakes, there were a lot of terrible things happening in the world. Recon Food is meant to be a respite from all of that, and bring social media back to when it wasn’t the noisy, scary place that it is today.” If that seems like pie-in-the-sky thinking (sorry for the pun), it’s not. It’s what teenagers, like Sophia and her friends, are looking for. When internal documents from Facebook were leaked, an internal research presentation “Teens blame Instagram for increases in the rate of anxiety and depression. This reaction was unprompted and consistent across all groups.” included a slide that stated The data continued to show how much the big platforms were harming young people (young girls especially), as they felt immense pressure to post regular content and fight for likes. Stressors, not depressors When Spencer explained that posts on Instagram to do with climate change, the Ukraine war, or gun violence in schools depresses him, Sophia jumped on to identify a key generational gap that is being overlooked. “I describe it as more of a stressor than a depressor. It makes me sad, I guess, but it makes me way more stressed than sad because it’s about the future. Something that weighs on me is this idea of having to fix these problems, or at least having my generation fix these problems.” It’s something that I hadn’t ever considered. How differently a 15 or 16-year-old kid would feel about something like climate change, knowing that — as Spencer tragically put it — “if their generation doesn’t fix it, we’re all doomed as a planet.” For so many users that are arguing on social media today, there isn’t the built-in pressure of having to fix the problem. It can just be a topic to yell about or regurgitate talking points within your echo chamber. Vertical-driven socials So what is the answer? In their minds, it’s to “reduce the noise” and to think of social media as a vertical, instead of horizontal, platform. Focus on a specific category or topic and don’t let it spread beyond that. In their case it is food, but they were quick to point out some other examples. LinkedIn has built a vertical platform based solely on career growth. Strava is one based on running and cycling. Alltrails, for hiking. This is why it can be called evolution, not necessarily an outlier. The next wave of platforms is already on its way, Sophia and Spencer are just leading the charge. If that’s the case, then there will be readers of this article that want to know: how do you do it? How do you build that next evolution and successfully roll it out? Well, they were kind enough to share some wisdom. Building community Everything always comes back to community. Every startup, every brand, and every marketplace always needs to create loyalty and investment from its users. For Recon Food, that means four key methods Iteration There is nothing worse than a stagnant service. If you are trying to build a community, you have to serve your existing users first. That means frequently iterating on your product or service, whether it’s adding new features, fixing bugs, or simply changing the design. The key is to show that you are constantly listening to feedback and trying to improve the user experience. By having their suggestions heard and implemented, you’ll create evangelists for your product — ones who will go out and spread the gospel to others. PR Listen, even in today’s age you still need a good PR game. That’s why I could book the Rascoffs for Success Story, and why you’ll probably see them in some other publications going forward. If you’re not constantly selling your story, then you’re not doing enough. Make sure everyone on your team is aware of your community efforts and can speak to them confidently. In-person events But even if you are the co-founder of a billion-dollar company, there’s still room for showing up at a trade show and shaking some hands. When it comes to community, you need to put a face to the name. That’s why in-person events are so important for startups. It could be as simple as attending a meetup or throwing an event. But whatever you do, make sure you show up and represent your company well. Convert existing groups There are already probably thousands upon thousands of people in the world that are using a group chat to share pictures of food. They’ve created their own small-scale social media platform and housed it in another service. If you can target those people, and show them that there is someone out there building something that specifically meets their needs, you can poach them from the existing group and get them invested in your own. Doing this will also give you access to their network, which could be full of other potential users for your product or service. Interestingly, Sophia also pointed out an unexpected factor that has inhibited Recon’s growth. People like anonymity on social media, at least when they are so used to the angry shouting matches on other platforms. It sometimes leads to users consciously not sharing it with their friends, as they would rather get feedback on their recipes and photos from someone they didn’t know. To get around this, you’ll have to build trust over the long haul and show them that it’s safe to bring people they know into the sphere — there won’t be any political arguments here. Topic-specific features “Instagram’s never going to build a run-tracking feature because that’s not what their app is for. But that is what Strava is for.” Next on the list after building community is creating features that won’t be found elsewhere. These are topic-specific things that don’t belong on other platforms but can add value to the vertical you are chasing. For Recon, it’s recipes, restaurant maps, and food collections. The possibilities here are endless because so are the topics. Cater to your audience, not the whole world. Optimize for enjoyment While the broad social media apps are just trying their best to keep you from moving on, the next era will be focused on giving you the best experience. Sophia, again, puts it perfectly: “It’s such a powerful tool to connect people across borders, across countries, across worlds. There’s such potential there, but the way that it’s currently being implemented and executed is not it. It’s not maximized for user success and enjoyment. It’s maximized for clicks, likes, views, and time spent on app.” Make them love to be there, not scared to leave. Audience first, monetization later In terms of building it into a business, Spencer is clear about one thing. Don’t worry about monetization until you’ve secured your audience. There will be ways to make money down the line, but if you’re building something with that in mind from the start, you’ll fall into the same habits that so many others have. Final thoughts That’s enough to fill an entire podcast but I touched on so much more with Spencer and Sophia. The challenges of balancing school and business, the importance of continued education, the benefits (and challenges) of working with a family member, and what characteristics make the best startup CEOs. If you want to hear the entire discussion, head over to the and marvel at one of the next generation of business leaders. Success Story YouTube channel That’s it for this week, talk soon! Also published here.

Optimizing for Positivity: Niches and the Future of Social Media

Microtransactions have been a big part of the gaming industry over the last few years. They can come in all shapes and sizes, from the most common cosmetic effects, such as skins for your characters, all sorts of accessories and housing if that’s something on the game, all the way to the often controversial pay-to-win - or pay-to-fast - transactions. It is a business model that not only offers revenue to sustain free-to-play game developers, but is also used on huge titles from AAA companies, and all combinations in between. The point of this text is not to discuss whether or not microtransactions should be a thing in gaming, if they should be used only in ways that do not interfere with gaming mechanics, which companies should use them, nor the particularities in legislation and the fact that some countries have banned certain types of microtransactions. All that is intended is to provide some context and certain ideas and stay open to the possibility and thoughts brought by those reading it, after all, a paradigm change rarely, if ever, happens with a single person's point of view. Valve and Epic: Battle Pass Perspectives on DOTA 2 and Fortnite It's almost impossible to discuss business models in gaming without talking about the great behemoth casting its dark shadow over us all. And then there's Valve on top of it. Jokes aside, Gaben Newell's company did something remarkable almost two decades ago when Steam was released. The way Steam was able to bring together developers and players, creating opportunities for new games to be discovered and almost extinguishing physical media for PC games was something else. For years Valve reigned alone, and indie developers or massive companies had to follow their will to sell their games, or else. Of course, some tried to create other solutions and substitutes for Steam, but it wasn't until 15 years later, in December 2018, that a real alternative appeared when Epic Games released its Epic Store. Let's not talk numbers right now, it's a fact that Steam is still way bigger and holds a huge amount of games titles more than Epic, but for the first time, some developers have an alternative, with Epic's better rates in sales and even more than that when studios used their proprietary gaming engine, Unreal. The fact that these two companies have a and , makes them, in my opinion, perfect for observing small decision changes that impact their community. That takes me to two of the crown jewels of these companies: Valve's DOTA 2 and Epic Games' Fortnite. Store Side Game Development Side It's important to observe that the two games have hugely different demographics, mechanics, and universes. For that reason, I will not talk about Epic's borderline magic abilities to create skins in partnership with any company and/or personality. The focus will be on the differences and similarities between the companies' Battle Passes, and just a bit on their general cosmetics store. Fortnite: Seasonal Battle Pass Fortnite's Battle Pass was reworked in Chapter 2: Season 7 (June, 2021-September, 2021), and since then, it follows this formula: 100 levels, divided into 10 pages, each with 10 rewards. After the 100 levels, there are 100 more levels where players can get Bonuses Rewards, usually alternate styles for already owned skins. Check below: Seasons on Fortnite usually last around 3 months, with some lasting slightly less or more. They are wrapped around the lore of Fortnite and have quests that earn players experience points (XP). There are all kinds of quests, some explore the lore of the game, others are "daily" or "weekly" quests that are just mechanical stuff players have to do in their matches. All levels need the same amount of XP, players can buy levels in order to get their rewards faster if they so want, but it's important to note that playing Fortnite and doing the quests daily for around 30min to 1 hour and a half, during the season, will surely get you all the rewards from the Battle Pass and likely a lot, if not all, of the Bonuses as well. Trust me, I'm not even good at the game. In April 2022, sportskeeda published an showing how much would be needed to get all the rewards of the Battle Pass in Fortnite. Since each level costs around 150 V-Bucks (Fortnite Currency), in order to get all 100 levels one would need to get 14,850 V-Bucks, since after purchasing the Battle Pass, for 950 V-Bucks, you already unlock level 1. Considering the rates of the currency, one would spend close to $100 to get all the rewards. If you add the 100 levels from the Bonuses Rewards, it goes up to around $200. article DOTA 2: The International Battle Pass Dota's Battle Pass, formerly known as The International Compendium, was released for the first time in 2013 as a way to reward players and members of the community and also increase the prize pool of The International (TI), basically Dota's World Cup and also the highest paying esports event in the world. This collaborative way of increasing the prize pool for TI was a huge success, with prizes going from $1,600,000 on the first edition to a whopping $40,018,195 last year alone. One can figure then that Battle Pass in DOTA was a huge deal. It was released once a year, with all sorts of quests, special game modes, cosmetics, skins, special effects, etc. While at some point most of the rewards were already completed, if players kept leveling up their Battle Pass, they would get some unique rewards on the highest of levels, such as even a real - TI's trophy. miniature of the Aegis of Champions The release of the Compendium was that big moment when players felt that not only they were participating in the game they loved so much, but they also were rewarded because of it. The first three Compendiums were not based on levels; instead, they were focused on goals for the International Prize Pool. Only the first one didn't reach all the goals, the other two did it with ease, and the last one went 3.5 million dollars more than the highest tier Valve created. From 2016 onward, the Compendium became the Battle Pass, focusing on levels that rewarded the players with all sorts of rare/unique skins that would, for the most part, only be available then. Some could be traded or gifted later, but would never be in the game store. It was an interesting change, that felt welcomed by the community for the most part. Battle Passes had more than 2000 levels, but the biggest rewards were around level 200~300, and going beyond that were special cosmetic effects that would vanish after a time period. At some point though, around 2019, Valve decided that Arcanas - DOTA's most sought skins that completely change the hero's model, voice, and skins - would be released on the Battle Pass as well, and if one couldn't get them at that point it would never be able to. As if that was not problematic enough, the huge bummer for DOTA's community was the fact that it was virtually impossible to get the Arcana on the Battle Pass without paying for it with real money. The number of levels one could get without it, only playing the game, would never be enough. Before that, the next hero to receive an Arcana was voted by the community on the Compendium and the prized skin would come later at the game store, for everyone to purchase at any time, for the standard price of $34.99. With this 'update' on their decision-making, Arcanas would be prized around level 330 of the Battle Pass, but not spending money players could get only around 111 levels. This number is not precise though, because there was a random reward that could get players from a quarter of a level up to 10 levels in one go, so if you're lucky enough... The values to get the Arcanas on the last two Battle Passes, which were in fact Seasonal, not contributing to The International prize pool or any other improvement to the pro scene, were quite similar to Fortnite's price to get all the rewards, , depending on if the player wants to get the prize right away or get the max number of levels for free before spending their money. around $75 to $150 Do game characters dream of virtual clothing? I hope readers are still with me after this not-so-small exposure. My point from here on out, is to bring some facts and some conjectures, that I hope are good enough ideas that can make the microtransaction market a bit fairer with its real users, the gamers. A big element of both Fortnite and DOTA's Battle Pass is FOMO (Fear Of Missing Out), since players are aware that the only chance they will have to get some skins is " ", they go crazy in order to obtain the skins they want. I'm not personally a big fan of this model, but I can for sure claim that it works, my wallet is a sad witness of it, and I would bet an Arcana that some readers have suffered with it as well. now It's important to note that there's a big difference, at this point in time, between Steam and Epic, and that is Steam's Community Market. For those uninitiated, the Community Market is where Steam users can sell their skins to other users for "real money". It's indeed real money, but you can only use it on the Steam Store, and more than that, every and each transaction on the Market has a fee that Valve takes. It's not possible to trade, sell, or gift Fortnite skins that you already own (it is possible to gift to a friend skins that are on the game store). But in DOTA there's a whole economy that occurs in the Community Market and even on third-party sites where people trade their skins for real money outside of Steam, a practice that is not permitted but it's highly overlooked. But maybe another strategy could work out in order for these huge companies still earn obscene amounts of money. And that is . Epic and Steam have their on blockchain gaming at this moment, Valve is seemingly against it on their platform (though Mir4 is still there), and Epic claims to welcome games on their store, but not a single one is there yet. blockchain own perspective Blockchain Gaming Play-to-Earn games, built on the blockchain, have a strong claim: to give ownership of game resources back to their players. That is a double-edged blade since many started to see them as investments instead of games, but that is something I would prefer to get in-depth in another article, what should be noted here is that just like cryptocurrencies, P2E games are going downhill in their value and players recently. Even though shows that the activity in this first quarter of the year has grown 2000% compared with last year, and investments are still happening, one can notice that few are the projects that are capable of keeping their players for more than a few months. DappRadar Report This is not to say that they will all fail, that the projects are not interesting, and that the games aren't fun. The point is that if all projects are following more or less the same model, then maybe they should start to think differently. Blockchain is not a "new" technology, but it is new in regard to its presence in day-to-day life. Like all new technologies, it takes a while to learn what are the best applications to it, most of the time the first idea is not the best idea. The Titan's Embrace But what if companies like Valve and Epic, instead of rejecting or welcoming this new technology, embrace it? What if, for instance, Valve develops its blockchain, transforming game skins in NFTs? They don't need to do this with ALL their skins, they can do this for the ones with limited marketability, special ones, as the P2E game do. Blankos The implications of such a decision, at a first glance, could be that of players feeling to have real ownership of their stuff, knowing that if at some point they need to leave their gaming community, all the investment they did would not be lost unless they went after shady marketplaces that negotiate accounts. Another point is that players will continue to be aware of skins scarcity. These big companies would still be able to use the FOMO strategy, skins will only be available during their Battle Passes period, the number will not get any higher after that time, so it's as exclusive as before, but people will be able to sell it if they want to. For Valve this would mean staying a pioneer in the game industry, being the bastion they so much like to be, and showing how to use blockchain in gaming legitimately. They would also be able to continue to get its chunk out of transactions by applying as many gas fees as they wanted to. For Epic Games this would mean opening up a new universe of possibilities for their already immense skin portfolio on Fortnite. They would not only keep producing their skins, but be able to show other developers that they really stand by their claims of supporting blockchain games, and will continue to earn from skins of past Battle Passes as they are traded on their chain. I would love to see a game like those portrayed on Korean Webtoons, where the game currency can easily become real-world money and people can decide if they want to work as an IT tech or as a blacksmith in a VR game. I know that I would craft the best items around. But maybe it's not the time to put finances and games together, maybe we should address certain social issues in the physical world first. Regardless of that, I believe that "simply" mixing successful, well-known, current games from great industries, not intending on transforming them into an investment but simply by giving players the awareness that their money is respected and they are purchasing a real asset, could be the first step into P2E right now and rebranding NFTs as something useful and good, not a scam.

Paradigm Change: Microtransactions

The Journey Of The CTO (Podcast Transcript)

An In-Depth Look Into the World's Most Popular Dating Apps

Bitfinex, , has more than 180 open-source repositories in GitHub–from a to . The most sophisticated trading toolkit developed by Bitfinex, though, is Honey Framework with many useful packages such as , , , and . one of the largest cryptocurrency digital exchanges Tezos JS library UI goodies bfx-hf-ui bfx-hf-strategy bfx-hf-indicators bfx-hf-strategy-exec In this tutorial, we'll use the Honey Framework ecosystem to define and backtest a trend trading strategy on historic data for the past 500 days. Requirements Let's install the libraries we're going to use. First, run the following command to install Honey Framework libraries. npm install bfx-hf-strategy bfx-api- bfx-hf-util bfx-hf-indicators node -models We'll also need the following packages to display the trades on the command line. npm install cli- colors table Eventually, install library as well to retrieve the data from Bitfinex API. node-fetch npm install node -fetch Strategy We use a simple trend trading strategy based on the following logic: position if the price is between -1 SD and -2 SD Bollinger Bands® and short SMA is below the long SMA. Open short position if the candle is green and more than 75% of its body crossed above the -1 SD band. Close short position if the price is between +1 SD and +2 SD bands and the short simple moving average is above the long simple moving average. Open long position if the candle is red and more than 75% of its body crossed below +1 SD band. Close long The logic behind the strategy is fairly straightforward: if the price is between  -1 SD and +1 SD bands (No Man's Land), we close our position. If the price is in the sell-zone (between the lower bands) and the market trend is bearish (short SMA is below long SMA), we open a short position. On the other hand, if the price is between upper bands and the market is bullish, we open a long position (in other words, we only trade in direction of the overall trend). Check to find more information about trend trading based on Bollinger Bands®. this article from Investopedia Code Let's get our hands dirty. First, create that defines a dummy strategy. strategy.js { SYMBOLS, TIME_FRAMES } = ( ); { BollingerBands, SMA } = ( ); HFS = ( ); .exports = HFS.define({ : , : ,
    symbol,
    tf, : { : BollingerBands([ , ]), : BollingerBands([ , ]), : SMA([ ]), : SMA([ ]),
    },
  }); // strategy.js const require "bfx-hf-util" const require "bfx-hf-indicators" const require "bfx-hf-strategy" module ( ) => { symbol = SYMBOLS.BTC_USD, tf = TIME_FRAMES.ONE_DAY } id "bbands" name "bbands" indicators bbands2 new 20 2 bbands1 new 20 1 smaShort new 25 smaLong new 100 Let's break it down. We define the default symbol (BTCUSD) and timeframe (one day). Then, we define the indicators we're going to use. We need two Bollinger Band® bands, both in the period of 20 days. We give a multiplier of 2 to the first (±2 standard deviations) and a multiplier of 1 to the second one  (±1 standard deviations). In the end, we add two SMAs, one in a 25-day period (short SMA) and the other in a 100-day period (long SMA). to find a list of all indicators available in Bitfinex Honey Framework. Check this We can enter the market now. The strategy we defined accepts five handler methods to update positions and make trades on each event: - called when no position is open onEnter - called when a long position is open onUpdateLong - called when a short position is open onUpdateShort - called when any position is open onUpdate - called on every tick onPriceUpdate Event handlers receive two parameters, (the strategy context) and (point-in-time data), they look for trade opportunities and update if there's a trade to be made. state update state Enter Into the Market Let's define the strategy to enter the market. First, add to encapsulate the reusable logic. Define in it to extract Bollinger Band® bands values from the state. helper.js getBBands HFS = ( ); getBBands = { indicatorValues = HFS.indicatorValues(state); { bbands1, bbands2 } = indicatorValues; { : plusOne, : minusOne } = bbands1; { : plusTwo, : minusTwo } = bbands2; {
    plusOne,
    plusTwo,
    minusOne,
    minusTwo,
  };
}; .exports = {
  getBBands
} // helpers.js const require "bfx-hf-strategy" const ( ) => state const const const top bottom const top bottom return module returns a map of all indicators defined in the strategy. We then extract the Bollinger Bands objects from them, which include three values: top, bottom, and middle. Eventually, the method returns ±1 and ±2 SD bands. indicatorValues Define another method to extract uptrend/downtrend data from the state. checkIfIsUptrend = { indicatorValues = HFS.indicatorValues(state); { smaShort, smaLong } = indicatorValues; isUpTrend = smaShort > smaLong; isUpTrend;
}; .exports = {
  getBBands,
  checkIfIsUptrend,
} // helpers.js /*
getBBands here
*/ const ( ) => state const const const return module The method simply checks if short SMA is above long SMA and infers the trend from it. Add . We'll use it to find trade opportunities. look_for_trade.js HFS = ( ); { getBBands, checkIfIsUptrend } = ( ); .exports = (state = {}, update = {}) => { { price, : timestamp } = update; orderParams = { : timestamp, : ,
    price,
  }; bands = getBBands(state); isBuyZone = bands.plusTwo >= price && price >= bands.plusOne; isSellZone = bands.minusTwo <= price && price <= bands.minusOne; isUptrend = checkIfIsUptrend(state); (isSellZone && !isUptrend) { HFS.openShortPositionMarket(state, orderParams);
  } (isBuyZone && isUptrend) { HFS.openLongPositionMarket(state, orderParams);
  } state;
}; // look_for_trade.js const require "bfx-hf-strategy" const require "./helpers" module async const mts const mtsCreate amount 1 const // between +1 SD and +2 SD from mean, the trend is up const // between -1 SD and -2 SD from mean, the trend is down const const if return if return return The method checks if the price is in the trade zone (between +1 and +2 SD or -1 SD and -2 SD bands). If the price is in the sell zone and the trend is bullish, it opens a short position, and if the price is in the sell zone and the trend is bearish, it opens a long position. If none is true, it simply returns the previous state. Notice that we use Honey Framework helper functions to open positions. You can find the list of all helper functions . Add to use the function we just defined to handle onEnter events. here on_enter.js lookForTrade = ( ); .exports = (state = {}, update = {}) => {
  newState = lookForTrade(state, update); newState;
}; // on_enter.js const require "./look_for_trade" module async await return Now add the handler to the strategy. on_enter = ( ); .exports = HFS.define({ onEnter: on_enter,
  }); // strategy.js /* other imports */ //  handlers const require "./on_enter" module ( ) => { symbol = SYMBOLS.BTC_USD, tf = TIME_FRAMES.ONE_DAY } /*
    other strategy properties
    */ Update Positions Well done! We've just exposed ourselves to the market. But we should add other handlers to react to market updates while we have an open position. Define another method in to check if 75% or more of the candle body crossed the inner bands. helpers.js HFS = ( ); checkIfShouldClose = { candleBody = .abs(close - open); closeToNoMansLandDist =
    position == ? close - bands.minusOne : bands.plusOne - close; shouldClose = closeToNoMansLandDist / candleBody >= ; shouldClose;
}; .exports = {
  getBBands,
  checkIfIsUptrend,
  checkIfShouldClose,
}; // helpers.js const require "bfx-hf-strategy" /*
 Other methods
*/ const ( ) => close, open, bands, position const Math const "short" // check if 75% of candle body entered to the No Man's Land. // see: https://www.investopedia.com/thmb/AtmrjGx96Tbz8f1cfnGOGzpqHqU=/2378x0/filters:no_upscale():max_bytes(150000):strip_icc():format(webp)/dotdash_Final_Using_Bollinger_Bands_to_Gauge_Trends_Oct_2020-02-f76c639116734ccfb8493dce32ed149a.jpg const 0.75 return module calculates the candle body height by finding the distance between open and close values. Then, it checks how much of the candle entered the No Man's Land. If more than 75% of the candle body is in between two inner bands, it closes the position. checkIfShouldClose It's time to create . on_long.js HFS = ( ); lookForTrade = ( ); { getBBands, checkIfShouldClose } = ( ); .exports = (state = {}, update = {}) => { { candle, : timestamp, price } = update; { open, close } = candle; orderParams = { : timestamp, : ,
    price,
  }; bands = getBBands(state); isCandleGreen = open <= close; (isCandleGreen) state; isCloseAbove1Std = bands.plusOne < close; (isCloseAbove1Std) state; shouldClose = checkIfShouldClose(close, open, bands, ); (!shouldClose) state; newState = HFS.closePositionMarket(state, orderParams); newState = lookForTrade(newState, update); newState;
}; // on_long.js const require "bfx-hf-strategy" const require "./look_for_trade" const require "./helpers" module async const mts const const mtsCreate amount 1 const const if return const if return //  check if 75% of candle body is below 1 std const "long" if return let await //  look if should open short await return The method we just defined checks if: The candle is green. The candle is above the No Man's Land (i.e., the closing value is above the +1 SD band). Less than 75% of the candle is in the No Man's Land. And if any of these are correct, it doesn't touch the position. Otherwise, it closes the long position using helper method and then calls to check if a short position should be opened. Now, let's define our strategy while shorting. closePositionMarket lookForTrade HFS = ( ); lookForTrade = ( ); { getBBands, checkIfShouldClose } = ( ); .exports = (state = {}, update = {}) => { { candle, : timestamp, price } = update; { open, close } = candle; orderParams = { : timestamp, : ,
    price,
  }; bands = getBBands(state); isCandleRed = open >= close; (isCandleRed) state; isCloseBelowStd = bands.minusOne > close; (isCloseBelowStd) state; shouldClose = checkIfShouldClose(close, open, bands, ); (!shouldClose) state; newState = HFS.closePositionMarket(state, orderParams); newState = lookForTrade(newState, update); newState;
}; // on_short.js const require "bfx-hf-strategy" const require "./look_for_trade" const require "./helpers" module async const mts const const mtsCreate amount 1 const const if return const if return //  check if 75% of candle body is below 1 std const "short" if return let await //  look if should open long await return checks if: on_short.js The candle is red. The candle is below the No Man's Land (i.e., the closing value is below the -1 SD band). Less than 75% of the candle is in the No Man's Land. And if all above are false, it closes the short position and looks for new trade opportunities. That's it. We've finished defining the strategy. Add the new event handlers to . Then we can start testing the algorithm. strategy.js on_enter = ( ); on_long = ( ); on_short = ( ); .exports = HFS.define({ onEnter: on_enter, : on_long, : on_short,
  }); // strategy.js /* other imports */ //  handlers const require "./on_enter" const require "./on_long" const require "./on_short" module ( ) => { symbol = SYMBOLS.BTC_USD, tf = TIME_FRAMES.ONE_DAY } /* 
      Other strategy properties
    */ onUpdateLong onUpdateShort Test on Historical Data Add to retrieve historic data from Bitfinex API. get_data.js ; fetch = ( ); url = ; get_500_days = { date = (); end = date.getTime();

  date.setDate(date.getDate() - ); start = date.getTime(); { start, end };
}; .exports = { { { start, end } = get_500_days(); pathParams = ; queryParams = ; req = fetch( ); response = req.json(); response;
  } (err) { .log(err);
  }
}; // get_data.js "use strict" const require "node-fetch" const "https://api-pub.bitfinex.com/v2/" const => () let new Date const 500 const return module async ( ) function getData { symbol, tf } try const const `candles/trade: : /hist` ${tf} ${symbol} const `start= &end= &limit=500` ${start} ${end} const await ` / ? ` ${url} ${pathParams} ${queryParams} const await return catch console fetches OHLC data of the past 500 days, given a symbol and a timeframe.  You can modify the parameters to meet your needs. Find more information about Bitfinex API's public endpoint for candle data . getData here Add to define the execution logic. exec.js process.env.DEBUG = ; { Candle } = ( ); { SYMBOLS, TIME_FRAMES } = ( ); logTrades = ( ); HFS = ( ); getData = ( ); BBandsStrategy = ( ); market = { : SYMBOLS.BTC_USD, : TIME_FRAMES.ONE_DAY,
}; getCandles = () => { rawCandleData = getData(market); candles = rawCandleData
    .sort( a[ ] - b[ ])
    .map( ({
      ...new Candle(candle).toJS(),
      ...market,
    })); candles;
}; run = () => { candles = getCandles(); strategyState = BBandsStrategy(market); ( i = ; i < candles.length; i += ) {
    strategyState = HFS.onCandle(strategyState, candles[i]);
  }

  logTrades(strategyState);
}; {
  run();
} (e) { .error(e);
} // exec.js "*" const require "bfx-api-node-models" const require "bfx-hf-util" const require "bfx-hf-strategy/lib/debug/log_trades" const require "bfx-hf-strategy" const require "./get_data" const require "./strategy" const symbol tf const async const await // attach market data const ( ) => a, b 0 0 ( ) => candle return const async const await let for let 0 1 await try catch console method retrieves the data and attaches market information to them. During live execution, we may receive data from multiple sources for different symbols and different timeframes, thus, it's necessary to attach this information to raw data. getCandles Then, gets candle data and the strategy and passes them to helper method that orchestrates relevant strategy event handlers for the new price action data. Eventually, logs the results using function. run onCandle run logTrades Let's test the algorithm. Open the command line in the project's directory and run the following command. node exec The result will be something like this: You can find a list of all positions along with the amount, price, fee, and profit/loss figure of each one. At the bottom, general information about the strategy is logged. The bot has made 37 trades, opened 19 positions, and gained 3960.91$. You can modify and change how it displays the results. helper function log_trades.js That's it! We've defined and backtested a simple trading strategy using Bitfinex Honey Framework toolkit. The example pieces of code in the article are available . here

Define and Backtest Crypto Algo Trading Strategies With Bitfinex Honey Framework Toolkit

Gradle is used very often in the industry and I felt it will be good to cover the basics of Gradle in this article so that it can be used efficiently😄 What is Gradle Gradle is a Build Automation tool and is also Open source. This Article will cover the following topics in Gradle. Creating a Java Application by using Gradle as the build tool Running Unit tests for the Java Application Gradle Build Script and Tasks Installing Gradle You can Install Gradle from https://docs.gradle.org/current/userguide/installation.html Also ensure Gradle is set in the PATH so that gradle commands work. you can verify if gradle is installed and can be accessed using the command gradle -v Also ensure is set. JAVA_HOME Creating a Java Application by Using Gradle as Build Tool Create a folder called as ‘simple-gradle-java-app’ and enter into the folder in command prompt. This will be our project folder. mkdir simple-gradle-java-app simple-gradle-java-app cd Use the following Command to Initialize Gradle for this project gradle init This will ask you to select the type of project to generate. Choose . This is because we are creating an application ‘application’ Next you will be asked to select Implementation Language. Choose . This is because we are creating a Java Application. ‘Java’ Next you will be asked to Choose Build Script DSL. Choose . In this Demo we will be using Groovy as the build script language. ‘Groovy’ Next you will be asked to Select a Test Framework. Choose . ‘Junit Jupiter’ For now you can leave the project name and source package as default. Folder Structure Once the project is created. It would have the following Folder Structure Some of the notable items created in this folder structure are : This is the folder which has all the Java Code in the Application src/main/java : This is folder which has all the test cases for the Above Java code src/test/java : This is the gradle wrapper which is used the run various gradle tasks and Commands. works in Linux and MacOS and is used for Windows. gradlew and gradlew.bat gradlew gradlew.bat : This file indicates which projects to include in the build. In our case we have only one project which is ‘simple-gradle-java-app’. This is set in settings.gradle as rootProject.name = 'simple-gradle-java-app' settings.gradle : This is the file in which the Build Script is written. We will be discussing more on this later in this Blog. build.gradle : This is the folder which has all the Gradle Wrapper files gradle Modifying the Java Class and Its Test Class In The Init Script has automatically Created a Java Class called as . src/main/java App.java This is the Java class which is Automatically generated for us. simple.gradle.java.app; { { ;
    } {
        System.out.println( App().getGreeting());
    }
} package public class App String public getGreeting () return "Hello world." public static void main (String[] args) new Replace the Above Java class with the following Class simple.gradle.java.app; { { (num1 + num2);

    } {
            System.out.println( + getSum( , ));
    }
} package public class App public static int getSum ( num1, num2) int int return public static void main (String[] args) "Sum is " 4 5 We have a function here called as which returns sum of 2 numbers. The function calls this function. getSum main In the Init Script has automatically created a Java Test class for us Called as src/test/java AppTest.java The class this is generated is given below simple.gradle.java.app; org.junit.jupiter.api.Test; org.junit.jupiter.api.Assertions.*; { {
        App classUnderTest = App();
        assertNotNull(classUnderTest.getGreeting(), );
    }
} package import import static class AppTest @Test void appHasAGreeting () new "app should have a greeting" Replace the Above java class with the following simple.gradle.java.app; org.junit.jupiter.api.Test; org.junit.jupiter.api.Assertions.*; { { result = App.getSum( , ); expected = ;
       assertEquals(expected,result);
    } {
        App.main( String[]{});
    }
} package import import static class AppTest @Test void getSum () int 10 12 int 22 @Test void main () new Here we have written a unit test to check if function is working as expected. getSum Running The Java Application First let us build the project using Gradle. This can be done using the following command Windows gradlew.bat build Linux and MacOS ./gradlew build Once the build is done, a folder called as is created which will have the built project and also it would contain information like the test reports. build Then we can run the Application using the following Command Windows gradlew.bat run Linux and MacOS ./gradlew run This will Run the application and Print the output as Shown Below > Task: run
Sum is 9 If needed we can clean the build using the below Command Windows gradlew.bat clean Linux and MacOS ./gradlew clean This will delete the folder. Running The Unit Tests build When we Built the project using the command, it would automatically run the Test classes as well and Generate the Test reports build The Test reports can be found in . Open the index.html file using a browser to see the test results. build/reports/tests/test/index.html Now if you want to run only the tests, then you can use the following command. Windows gradlew.bat test Linux and MacOS ./gradlew test This will also generate the file. The Test report will look as shown below index.html Now Let us make an Error on Purpose In the code Replace with the following App.java simple.gradle.java.app; { { num1;

    } {
            System.out.println( + getSum( , ));
    }
} package public class App public static int getSum ( num1, num2) int int return public static void main (String[] args) "Sum is " 4 5 As you can see here, we are always returning num1 instead of num1 + num2 in function which is wrong. getSum Let’s Run the tests now Windows gradlew.bat test Linux and MacOS ./gradlew test You would notice that now it says that the . Let’s check the report in now to see what it is showing Task: Test failed index.html The Error Clearly shows that it Expected the sum to be 22, But then our App logic is returning 10. Hence the Test has Failed. Now modify getSum Function in App.java to return ( num1 + num2 ) and re-run the test and it will pass Gradle Build Script and Tasks has the build script. In this demo we are using Groovy for the build script. build.gradle The build.gradle which was generated has the following snippets. All of these are already Automatically generated for us. So here I will be explaining all the snippets generated Plugins plugins { id id } // Apply the java plugin to add support for Java 'java' // Apply the application plugin to add support for building a CLI application. 'application' help to add additional functionality to Groovy. Here we have a java plugin to Support Java Classes. We also have an Application Plugin so that We can build Applications using Groovy Plugins Dependencies dependencies { implementation testImplementation testRuntimeOnly } // This dependency is used by the application. 'com.google.guava:guava:28.1-jre' // Use JUnit Jupiter API for testing. 'org.junit.jupiter:junit-jupiter-api:5.5.2' // Use JUnit Jupiter Engine for testing. 'org.junit.jupiter:junit-jupiter-engine:5.5.2' is where we add additional libraries need in the project. For Example here we are mentioning that we need dependency for Unit Testing. If you wanted to make a Database connection, then you will need to use a Library like JDBC for it. In that case JDBC will be added as a dependency. Dependencies junit jupiter Repositories repositories { jcenter()
} // Use jcenter for resolving dependencies. // You can declare any Maven/Ivy/file repository here. is the location from where Groovy will fetch and download the dependencies. Here we have mentioned that the repository we are using is Jcenter. You can read more about Jcenter . Another Commonly used Repository is Maven Central. You can read more about Maven Central Repositories here here Application application { mainClassName = } // Define the main class for the application. 'simple.gradle.java.app.App' Since we have created a Java application, we need to mention the Java class containing the main class in the build script. In this case it is App.Java. Tasks A Task is a piece of work that the build needs to do. In currently we do not see any tasks defined. But a lot of default tasks already exist. You can see them by typing the following command build.gradle Windows gradlew.bat tasks Linux and MacOS ./gradlew tasks This will give the following output Application tasks
-----------------
run - Runs this project as a JVM application

Build tasks
-----------
assemble - Assembles the outputs of this project.
build - Assembles and tests this project.
clean - Deletes the build directory.
jar - Assembles a jar archive containing the main classes.
testClasses - Assembles classes.

Documentation tasks
-------------------
javadoc - Generates Javadoc API documentation the main code.

Help tasks
----------
tasks - Displays the tasks runnable from root project .

Verification tasks
------------------ - Runs the unit tests. test for source 'simple-gradle-java-app' test We have already used some of these tasks like , , and . build clean run test The Flexibility of groovy is that we can add Custom Tasks as well. Let’s add a Custom Task and then Run the Task. Add the following code snippet in build.gradle task projectupper {
    group = description = doLast {
        String projectName = project.name
        println println }
} 'Custom Tasks' 'Task to get Project Name and Convert it to upper case' "Project Name: $projectName" "Upper case: ${projectName.toUpperCase()}" We have written a simple task here which takes the project name , Coverts the name to upper case and then prints the Name in Upper case. is name of the Task projectupper and are very important since it will list this task in the tasks report whenever we run gradlew.bat tasks group description is where we define the task action. We first get the project Name using , then we convert it to upper case and print it. doLast project.name In order to run the task use the following command Windows gradlew.bat projectupper Linux and MacOS ./gradlew projectupper This will generate the following output > Task :projectupper
Project Name: simple-gradle-java-app
Upper : SIMPLE-GRADLE-JAVA-APP case This task is a very simple task. We can create much more complex tasks in groovy as well and I will cover those in future articles. Code The entire code discussed here is available in this repo https://github.com/aditya-sridhar/simple-gradle-java-app References Gradle Offical Guide: https://gradle.org/ Gradle Tasks : https://docs.gradle.org/current/userguide/tutorial_using_tasks.html Building Java Projects with Gradle: https://docs.gradle.org/current/userguide/building_java_projects.html Congrats 😄 You know how to use Gradle now. This Article covered some of the basic Concepts of Gradle. Please go over the documentation of Gradle to know more about Gradle. Happy Coding 😄 Feel free to connect with me in or follow me in This Article was originally published in LinkedIn Twitter adityasridhar.com

Getting Started With Gradle for Build Automation [A How-To Guide] 

The day after I came home from the hospital, I quickly realized that all the housework I had been doing until the day before I delivered was suddenly impossible. I could barely walk let alone stay awake! Photo by on Julie Johnson Unsplash Getting help at home during parental leave My mom who lives close by, took a couple days off from work to help out around my home. My grandma who lives with me took care of all the cooking. This gave my husband and me time to rest and take care of our little one. My husband and I had decided to take 3 months off for parental leave. During that time, various family members came to visit and help out. We were fortunate to have family that wanted to help and were willing to do it on our terms. I know it’s hard to set boundaries with family members, and so much of it is cultural. But it is was important for my sanity. With my own parents, I pushed them to let me know at least 48 hours in advance instead of dropping in anytime they felt like it. I also asked them to watch my little one for 2–4 hour blocks of time, so I could take a nap or just get out of the house. With my in-laws I kept my terms straightforward: come anytime after the first month, enjoy the baby, help out with household chores, and watch the baby, so I could get some rest. I was really thankful to have as much help during those months, because our little one was quite colicky and it was hard to soothe him and take care of ourselves and our home. I’ve also heard from some of my friends who didn’t have family close by or could take time off, ended up hiring nannies or postpartum doulas. Before my delivery, I kept hearing the following from close friends and family: “I struggled with bouts of the baby blues months after my delivery.” “My wife suffered from postpartum depression.” “It took me more than 2 years to heal my body.” “I had to go back to work 6 weeks after my baby was born and I was still exhausted.” Instead of thinking that things would be different for me, I faced reality and decided that I wasn’t special. I needed help to heal I’ll admit it felt great to get back to working out 5 weeks postpartum. Having that time to getaway and decompress worked wonders. It also felt great to not have to worry about making dinner, taking out the trash, cleaning the house, and so on. For once in my life, I gave myself permission to sit on my butt. There were a few times I’d get worked up, and was fortunate to have a partner remind me to relax. As parental leave came to an end, I wasn’t sure if I was going to be returning to part-time or full-time work, but I knew I’d need help nonetheless. How I found a childcare provider that met my needs My husband I figured we’d start with a part-time nanny and then ramp up. We chose this over daycare because we preferred having someone come to our home. It would give us a chance to take breaks throughout our workday to see our little one. And we were fortunate to be able to afford this option. Photo by on Jordan Whitt Unsplash We did a lot of due diligence as we reviewed nannies I checked out a few different care providers including Care.com, Bambino, and SitterCity. SitterCity seemed to provide the most straightforward experience. For about $100, I could post the type of care provider I was looking for and I’d receive responses for people who met the criteria. I likened it to recruiting for my business. We immediately received a handful of people who were interested. We interviewed a few either in-person or on the phone. There were two that we liked in particular based on their experience, demeanor, availability, and of course price point. We did reference calls on both as well as a background check. One of them was farther away then we would have liked. We were concerned about rush hour traffic and being flexible enough to get to us at a moment’s notice if needed or stay later if we needed the help. With the remaining care provider, we decided to do a test run. We had them come and take care of our little one for half a day. Unfortunately, the day before our first test run, they called to tell us they had accepted an offer for a full time job elsewhere. We were disappointed but they were considerate enough to recommend someone else to help. So we went through the test run with the care provider they recommended, who turned out to fantastic! Photo by on Laura Lee Moreau Unsplash Parents talk a lot about turnover and absenteeism with childcare providers It’s hard to have guaranteed childcare. Daycares won’t take a sick kid, have long waitlists, and a lot of nannies burn out. While it’s hard to guard against sickness and emergencies, I wanted to make sure that our childcare provider was aligned with us. To that end, we setup an LLC to provide all the benefits a business would pay its employees. Hence, we pay our nanny the hourly rate requested, provide paid vacation time, and pay into other benefits such as health insurance and unemployment. Yes it’s an upfront expense, but I think of it as an investment. An investment in my little one, and my ability to continue to have the freedom to run my company. I eased back into work by working part-time, about 3 days a week. I spent the first two weeks at home training our care provider. Since our little one had colic we figured it would be challenging for anyone who was getting to know him. During those two weeks, I was able to suss out our care provider’s experience level and see how coachable they were. And again, we were pretty fortunate to have found the right fit. After the first month of work, even with all the help at home, I was exhausted. Though I was only working 3 days a week, I had a 2 day commute, and I was spending 4 days caring for my little one. I didn’t have a lot of down time for myself. I decided to add 2 half days to our childcare schedule, and it has helped tremendously. I work 4 days a week and have a little extra time during the week to either work, workout, run some errands, or just relax. Taking time for myself Because I love my baby I always want to be with him if I’m not working. It took me awhile to realize that this can be exhausting. So once a month, I get a whole day to myself. I like to call this mommy’s day off (hat tip to Ferris Bueller). On this day, my todo list is no longer than 3 items and those are usually activities like reading, yoga or getting a massage. I’m fortunate to have a partner who supports mommy’s day off! Gone are the days of my 60-day Bikram Yoga challenges, but I do my best to squeeze in 3–4 workouts a week. 2 days of strength training and 2 days of cardio or just a long hike or walk. Aside from working out, I’ve become a pretty voracious reader. . Reading helps me relax, stay current, and can be squeezed in-between or during other activities like commuting, pumping, and when my little one naps. You can check out my GoodReads account here All other hobbies got put on the back burner. So that’s my new life at home, and I gotta say I love it! I don’t feel like anything is really missing. I’m very fortunate to not feel overwhelmed because I have a lot of help at home thanks to my partner, my family, and our nanny. Seeing the struggles my parents went through, I always aspired to be a calm and patient mom. Running my own business has given me the freedom to be that so far. And I’m sure in a few years, my little one will be curious to get out and explore more of the world, and I’ll be happy to tag along with him! Got questions for me? Feel free to let me know in the comments below, and I’ll do my best to answer them! And if you’d like to receive additional posts like this one. subscribe to my weekly newsletter here Enjoyed this post and want more? Help others enjoy it too by hitting the 👏🏽 ! Check out these additional resources on managing parenthood and your career: Breastfeeding maybe best, but boy does it put new moms to the test! The Privileges Of Parental Leave How To Pause For Parenthood Without Killing Your Career How To Change Careers Later In Life And Transition Into A Technical Role Startups And Mom’s Can Mix Pregnancy and startups I’m CEO of an early stage SaaS startup and I’m about to have a baby. This is what I’m doing for maternity leave.

As a new mom — I take all the help I can get!

Giving the Elephant Some Elixir MapReduce is a common Big Data pattern for analyzing a data set concurrently. This tutorial will introduce you to Elixir and the principals behind Hadoop. We will be building the equivalent of Hello World in MapReduce which is a word count program. Map and Reduce are also common higher order functions in the world of functional programming. Map is a function that takes a list and an anonymous function or lambda as arguments, applies the function to each element in the list, and returns a new list with the output of the lambda on each element. Reduce is a similar function in that it takes the same arguments with one additional argument in Elixir, an accumulator, but returns an accumulated value instead of a list. Elixir is a great language to learn concurrency and MapReduce is both a useful example and shows off many of Elixir’s features. MapReduce Flow MapReduce is a pipeline through which data flows and is processed. It can be broken down into roughly 5 steps which correspond to 5 modules we will write in Elixir. Our first step is the Input Reader. This takes in data, splits it into a form that our Map process can read, and concurrently launches Map processes. Our Map process reads the data given to it, runs a function on each piece of data, and outputs a key value pair to a Partition/Compare process. The Partition process accumulates key value pairs from all Map processes, compares the pairs, and spawns Reduce processes for each unique key. Each Reduce process runs a function on each value that adds up all the values for the given key, and emits these values to the Output Writer. Finally, the Output Writer yields your data in a format of your choice. Step 0: Setup First make sure you have Elixir installed for your current system. Instructions can be found . An excellent introduction to the language is available from the main site and hexdocs.pm provides Elixir module documentation . also provides an excellent Elixir syntax primer. This tutorial assumes little to no experience with Elixir but familiarity with basic programming concepts and the command line of your operating system. All code for this project is available on . here here here Learnxiny Github Once you have Elixir installed create a new project using . mix new mix new mapreduce --module MapReduce Next move into your directory and edit with this line inside . mapreduce mix.ex def project do [] escript: [main_module: MapReduce] Step 1: The Parent Process Once you have a new project navigate into in your project directory and open which should have been autogenerated by . First remove everything from the file that is not and . Let’s add a couple of imports for modules we will be writing later to the top of the module. lib/ mapreduce.ex mix defmodule MapReduce do end defmodule MapReduce dorequire InputReaderrequire Partition Next let’s create a main function and pipe its argument into several functions. , , is an operator that acts very similarly to bash’s pipe. Pipe |> def main(args) doargs |> parse_args |> pipelineend After that let’s write a private function (denoted by instead of ) to parse our command line arguments. Here we are creating a variable with type tuple which contains the result of . Our command line argument is and so we set our switches in the arguments to the parser function accordingly. We only need the first output from the parser which we will be returning from the other outputs are represented with an underscore to indicate we won’t need them. defp def OptionParser.parse --file=example.txt parse_args defp parse_args(args) do{options, _, _} = OptionParser.parse(args,switches: [file: :string])optionsendend The final part of our parent process is the pipeline. The first function is a pattern match case that checks for an empty file argument. The next function launches a Partition process but only stores the process id in a variable. We are using to give us the second element of the tuple returned by starting our process because we don’t need the atom, , first element. Then the Partition process id and the file name are passed to an Input Reader that we will write in the next step. Finally we use a recursive function, , to keep the parent process alive while the rest of our MapReduce flow executes. pipeline pipeline elem :ok forever defp pipeline([]) doIO.puts "No file given"end defp pipeline(options) dopartition = elem(Partition.start_link, 1)InputReader.reader("#{options[:file]}", partition)forever()end defp forever doforever()end Step 2: The Input Reader This module is fairly simple, it contains one function that takes a filename and process id. This function will attempt to open a file, sending a message to STDERR if this fails. If our file is successfully opened we will execute a function on every line of the file. We use a regular expression to parse lines from our file and return them as a list. is identical to map except that it returns an atom if it completes successfully instead of a list. For every line in our list we will a Map process with the line and the Partition process id as arguments. Enum.each spawn defmodule InputReader dorequire Mapper def reader(file, partition) docase File.read(file) do{:ok, body}      -> Enum.each(Regex.split( ~r/\r|\n|\r\n/, String.trim(body)), fn line -> spawn(fn -> Mapper.map(line, partition) end) end){:error, reason} -> IO.puts :stderr, "File Error: #{reason}"endendend Step 3: The Mapper The Mapper module is quite short, first we send the process id of the current Map process to the Partition process. This will enable us to check whether our Map processes are still running later on in Partition. After that we will again use to send individual words from a list generated by splitting each line based on the space character to the Partition process. Enum.each defmodule Mapper dodef map(line, partition) dosend(partition, {:process_put, self()})Enum.each(String.split(line, " "), fn key -> send(partition, {:value_put, key}) end)endend Step 4: The Partition The Partition module is the most complex module we will be creating. Here we will be using the module instead of just . We are using instead of because we want the parent process to be killed when this process is killed. Our linked processes are MapReduce, Partition, and OutputWriter in that order. OutputWriter will check if all Reduce processes have finished before exiting its own process. This will exit all linked processes all the way back to the parent. We use a lambda in to start a recursive function that takes 2 lists as arguments. Note that end is necessary to close all lambdas. Task spawn start_link start start_link defmodule Partition dorequire Reducerrequire OutputWriter def start_link doTask.start_link(fn -> loop([], []) end)endend Next we will write our recursive function. This will first check the length of the mailbox of messages sent by our Map processes. If it has processed all messages it will launch a check to see if we should launch our Reduce processes yet. We use to remove all null or whitespace characters that have snuck into our key value pairs. Note the use of a , , to represent a whitespace character. Next we have some pattern matching code that checks all of our received messages for specific tuples. If we receive the atom we will append the process id of the caller Map process to our process list inside of a recursive call. If we instead receive the atom we will append a containing the key sent to us by Map and a value of 1 for the word count. Any other message will produce an error. loop Keyword.delete sigil ~s(\s) :processor_put loop :value_put Keyword defp loop(processes, values) domailbox_length = elem(Process.info(self(), :message_queue_len), 1)if (mailbox_length == 0), do: (mapper_check(processes, Keyword.delete(Keyword.delete(values, String.to_atom(~s(\s))), String.to_atom(""))))receive do{:process_put, caller} ->loop([caller | processes], values){:value_put, key} ->loop(processes, [{String.to_atom(key), 1} | values])error -> IO.puts :stderr, "Partition Error: #{error}"endend The final piece of Partition is . This function checks if all of our Map processes are dead and launches Reduce processes for each unique word if they are. First we use to return a list, ,of any process that is still alive. Then we create a list, , of every unique key/word. If we have a non-zero number of keys and no Map process is alive then we OutputWriter and pass its process id to every Reduce process we spawn. After this we use on uniques and use to pull out every instance of each unique and then spawn a Reduce process with a list of all of those instances. mapper_check Enum.filter check unique start_link Enum.each Keyword.take defp mapper_check(processes, values) docheck = Enum.filter(processes, fn process -> Process.alive?(process) == true end)uniques = Enum.uniq(Keyword.keys(values))if (length(check) == 0 && length(uniques) != 0), do: (output_writer = elem(OutputWriter.start_link, 1)Enum.each(uniques, fn unique -> spawn(fn -> Reducer.reduce(Keyword.to_list(Keyword.take(values, [unique])), output_writer) end) end))endend Step 5: The Reducer The second to last module is our Reduce process. This takes a list of tuples (key value pairs) and the process id of Output Writer. Similar to Map we send Output Writer the process id of Reduce to keep track of its status. Then we check to make sure is not empty with a case pattern match. If is not empty we send a string to Output Writer. This string might look a little strange but that’s because we are using Elixir’s string interpolation syntax, , to place 2 expressions separated by a space. Let’s break down the two expressions in the string we send to Output Writer. First we use , which we’ve seen before, to get the key from the first tuple in the list using or head. All of the keys in the list should be the same so it doesn’t matter which one we use. Second we use to add up all of the values from our key values tuples using the accumulator given as the second argument in both and the lambda in . tuples tuples "#{}" elem hd Enum.reduce reduce reduce defmodule Reducer dodef reduce(tuples, output_writer) dosend(output_writer, {:process_put, self()})case tuples do[] ->  IO.puts :stderr, "Empty List"tuples ->send(output_writer, {:value_put, "#{elem(hd(tuples), 0)} #{Enum.reduce(tuples, 0, fn ({_, v}, total) -> v + total end)}"})endendend Step 6: The Output Writer This module should look very familiar since it’s a slightly modified version of Partition. The only difference is in where we open a file, write each word and its count to STDOUT and the file we just opened, and then close the file and the whole process chain all the way back to our parent process. One important detail is the use of to give us consistent file paths across various operating systems. Another is the use of which is Elixir’s string concatenation operator. reducer_check Path.join <> defmodule OutputWriter dodef start_link doTask.start_link(fn -> loop([], []) end)end defp loop(processes, values) domailbox_length = elem(Process.info(self(), :message_queue_len), 1)if (mailbox_length == 0), do: (reducer_check(processes, values))receive do{:process_put, caller} ->loop([caller | processes], values){:value_put, value} ->loop(processes, [value | values])endend defp reducer_check(processes, values) docheck = Enum.filter(processes, fn process -> Process.alive?(process) == true end)if (length(check) == 0 && length(processes) != 0), do: ({:ok, file} = File.open(Path.join("test", "output.txt"), [:write])for value <- values doIO.puts valueIO.write(file, value <> ~s(\n))endFile.close(file)Process.exit(self(), :kill))endend Testing and Wrap-up Congratulations! You’ve successfully written a small but non-trivial Elixir program that does something useful. Let’s do some testing and then we’re finished. First create a directory called test and a file inside it filled with random text. I used to for my file. Next compile your project with . Finally run your code using . You should see several hundred lines of text on your command line like this. randomtextgenerator.com mix escript.build ./mapreduce --file=test/input.txt Notice the exit message at the bottom. Finally open your directory and there should be a new file name with output identical to your command line except that the exit message won’t be at the end. test output.txt Thanks for reading, please leave a clap or several if this tutorial was helpful to you! Joe Cieslik is the CEO of Whiteboard Dynamics, a full stack development team specializing in functional programming and Android. You can find out more about us, how we can help you, and our past projects at whiteboarddynamics.co .

Build a MapReduce flow in Elixir

When Programming is Not Fun When the iteration loop is long, programming becomes frustrating. Let’s say you have to change a low-level system that lacks a way to test it directly. To test the new code, you have to manually execute a complex series of time-consuming and onerous steps. When something goes wrong, all you know is either your code is broken (for some unknown reason), or you exposed an existing issue in the monstrous system that depends on your change. It’s even worse when the code cannot be tested in a reproducible way, and you get a different behavior with each run. If that isn’t frustrating enough, imagine you are unable to implement the solution you envisioned. You can’t directly access the low-level system because a series of components — meant to abstract, simplify and create a safe alternative to your low-level system — are in the way. The components accomplish little of their stated purposes, but force your final design to be slower and more complex than necessary. Finally, you learn that the dev environment you are testing doesn’t match the behavior of production in crucial, important ways. This is what feels like to write queries for many websites. It’s not fun. Let’s fix that. database In this post I’ll show how to make database fun by using packages that make testing easy. I’ll walk through a example of building a durable queue backed with PostgreSQL. programming Isolated Temporary Databases To make database programming fun, we need a quick way to iterate. We need to bring up instances of our database, PostgreSQL in my case, and directly test the queries we are writing. One excellent option is . I’m going to utilize a Haskell analog I wrote called and the test helper based on it: . [pg_tmp](http://ephemeralpg.org/) [tmp-postgres](http://hackage.haskell.org/package/tmp-postgres) [hspec-pg-transact](http://hackage.haskell.org/package/hspec-pg-transact) provides a helper for setting up a test suite that creates a temporary process. Tests can use the optional , which adds a test and runs the db transaction used for the test. [hspec-pg-transact](http://hackage.haskell.org/package/hspec-pg-transact) postgres [itDB](https://hackage.haskell.org/package/hspec-pg-transact-0.1.0.0/docs/Test-Hspec-DB.html#v:itDB) Here is a simple example of testing db queries with a temporary database. migrate "Query” $ "work" $ do [ |INSERT INTO thingsVALUES (‘me’) |] [sql|SELECT nameFROM things |]` ` [Only "me"] describeDB itDB execute_ sql query_ shouldReturn (You need to make sure that is on the . This is case for on macOS, but for Ubuntu you must add to the ) initdb PATH brew /usr/lib/postgresql/VERSION/bin PATH The full code, including import statements, is . Let’s walk through a larger example. here Making postgresql-simple-queue In one of my previous posts, I discussed an in . The problem with the queue, as , is that it’s not . However, PostgreSQL is durable, so let’s use it to make our queue as Cale did. memory queue for sending emails Cale Gibbard pointed out durable We’ll add new payloads with . will grab the latest payload for us to process. We’ll call on the payload once the processing is finished: enqueueDB tryLockDB dequeueDB data Payload = Payload{ pId         :: PayloadId, pValue      :: Value, pState      :: State, pCreatedAt  :: UTCTime, pModifiedAt :: UTCTime} deriving (Show, Eq) enqueueDB :: Value -> DB PayloadIdenqueueDB value = dopid <- liftIO randomIOexecute[sql| INSERT INTO payloads(id, value) VALUES (?, ?)|](pid, value)return $ PayloadId pid tryLockDB :: DB (Maybe Payload)tryLockDB = listToMaybe <$> query_[sql| UPDATE payloadsSET state='locked'WHERE id in( SELECT idFROM payloadsWHERE state='enqueued'ORDER BY created_at ASCLIMIT 1)RETURNING id, value, state,created_at,modified_at|] dequeueDB :: PayloadId -> DB ()dequeueDB payloadId = void $ execute[sql| UPDATE payloadsSET state='dequeued'WHERE id=?|]payloadId All the operations occur in a monad so we can compose them with other db operations and have them run in the same transaction. For instance, we could use this library to and create a customer in a single transaction. [DB](http://hackage.haskell.org/package/pg-transact-0.1.0.0/docs/Database-PostgreSQL-Transact.html#t:DB) enqueueDB Back to Testing We can utilize to write our tests: [hspec-pg-transact](http://hackage.haskell.org/package/hspec-pg-transact) main :: IO ()main = hspec spec spec :: Specspec = describeDB init "Queue" $ doitDB "empty gives nothing" $ dotryLockDB`shouldReturn` Nothing itDB "enqueue/lock/dequeue" $ dotheId <- enqueueDB $ String "!"Just Payload {..} <- tryLockDB pId \`shouldBe\` theId  
pValue \`shouldBe\` String "!"  
tryLockDB   
  \`shouldReturn\` Nothing

dequeueDB pId \`shouldReturn\` ()  
tryLockDB   
  \`shouldReturn\` Nothing We can load up our single test file, , and run it in isolation: [test/Database/QueueSpec.hs](https://github.com/jfischoff/postgresql-queue/blob/master/test/Database/PostgreSQL/Simple/QueueSpec.hs) > :l test/Database/QueueSpec.hs> :main --match=locksDatabase.Queuepostgresql:///test?host=/tmp/tmp-postgres26345&port=57473 empty locks nothingenqueue/lock/dequeueShutting Down Finished in 1.8647 seconds2 examples, 0 failures Two seconds isn’t considered great in the unit-testing world, but for database testing we are in good shape. Not only are the tests fast, but we can also quickly iterate by reloading and rerunning a single file, all in single-digit seconds. This actually scales quite well. The queries themselves take milliseconds; almost all of the time is taken for the temporary db startup, which we only need to do once. Stress Tests Simple tests are easy, but let’s write a more complicated stress test. Unlike this simple test, the stress test will need to run multiple transactions, so we can’t use the test helper. itDB In this test we create consumer threads that read from the queue and threads that messages: 10 1000 enqueue it "stress queue" $ \testDB -> dolet withPool' = withPool testDBelemCount = 1000 :: Intexpected = [0 .. elemCount-1] ref <- newIORef [] -- Make 10 queue consumer threadsloopThreads <- replicateM 10$ async$ fix $ \next -> dompayload <- withPool' tryLockcase mpayload ofNothing -> nextJust Payload {..}  -> dolast <-atomicModifyIORef ref $\xs -> ( pValue : xs, length xs + 1)withPool' $ \conn ->dequeue conn pIdwhen (last < elemCount)next -- Fork a 1000 threads and-- enqueue an indexforM_ [0 .. elementCount - 1] $\i -> forkIO $ void $withPool' $ flip enqueue $toJSON i waitAnyCancel loopThreadsJust decoded <-mapM (decode . encode)<$> readIORef refsort decoded`shouldBe` sort expected This test works, but makes clear that there is an issue with our design. We have to poll the db constantly to see if there is new data. Going Full Postgres Luckily PostgreSQL has a solution for us, as : and . We make a new function, , to utilize PostgreSQL’s pubsub feature: lpsimth pointed out [NOTIFY](https://www.postgresql.org/docs/current/static/sql-notify.html) [LISTEN](https://www.postgresql.org/docs/current/static/sql-listen.html) lock notifyPayload :: Connection-> IO ()notifyPayload conn = doNotification {..} <-getNotification connunless (notificationChannel== "enqueue") $notifyPayload conn lock :: Connection-> IO Payloadlock conn = bracket_(Simple.execute_conn "LISTEN enqueue")(Simple.execute_conn "UNLISTEN enqueue")$ fix $ \continue -> dom <- tryLock conncase m ofNothing -> donotifyPayload conncontinueJust x -> return x We extend to also our channel after it enqueues a payload enqueueDB NOTIFY enqueue enqueueDB :: Value -> DB PayloadIdenqueueDB value = dopid <- liftIO randomIOexecute[sql| INSERT INTO payloads(id, value) VALUES (?, ?)NOTIFY enqueue;|](pid, value)return $ PayloadId pid We can write a similar test for , and our loop becomes: lock loopThreads <- replicateM 10 $async $ fix $ \next -> do-- blocks waiting for-- new payloadx <- withPool testDB locklastCount <-atomicModifyIORef ref $\xs -> ( pValue x : xs, length xs + 1)withPool testDB $ \conn ->dequeue conn (pId x)when (last < elemCount)next Much nicer (ignoring the super small margins for mobile formatting :p). The final queue is packaged up as on and . postgresql-simple-queue hackage github Fun is Hard to Write We went through a bunch of intermediate states before I pushed it to Hackage. I started with something simple, wrote stress tests, which thankfully found bugs, and added pubsub. Most incredibly, it was fun and not frustrating. Don’t take my word for it: try it. and are on Hackage/Stackage. [pg-transact](https://hackage.haskell.org/package/pg-transact) [hspec-pg-transact](https://hackage.haskell.org/package/hspec-pg-transact) Extra Credit: Email Queue Take 2 We can now write a better email process queue, discussed in my other post. First, we need a processing loop: forever $ dopayload <- withResourceconnectionPool locksendEmail payloadwithResource connectionPool $\conn -> dequeue conn(pId payload) ’s, provides this consumer loop for us and also does command line argument processing. We can make a queue consumer executable easily, like so: postgres-simple-queue [defaultMain](https://github.com/jfischoff/postgresql-queue/blob/master/src/Database/PostgreSQL/Simple/Queue/Main.hs#L133) main :: IO ()main = doenv <- newEnv DiscoverrunResourceT $ runAWS env $ defaultMain "aws-email-queue-consumer"$ \payload _ -> docase fromJSON $ pValue payload ofSuccess email -> doresp <- AWS.send $ makeEmail emaillogFailedRequest respError x -> throwIO $ userError$ "Failed to decode payload as an Email: "++ show x See for the full example. [EmailQueue](https://github.com/jfischoff/postgresql-queue/blob/master/examples/EmailQueue.hs)

Testing PostgreSQL for Fun

(Edit) I have added some gif to this article. I’m glad you like ! In fact, I am a die hard fan and I’ve been using it for ages (since ST2). I have a few tricks I would like to share them with you. Sublime Text 3 This is not a language oriented article. This article ranges from use of ST3 and we are going to learn and do: basics to advanced Fuzzy searching basics. What the files under folder are. User Some important functionalities as a reminder. Working with words and lines. Selection and deleting techniques. Useful keys and advance techniques. navigation Project Management. Install a few handy plugins. Optimisation by changing a few settings. Add missing to the commands Command Palette. In case you don’t know how to open the Command Palette, hit . The is for files, otherwise known as . ctrl+shift+p ctrl+p go to go to anything Fuzzy searching It works like this: if you are after a term in any palettes, for instance , you don’t need to type in the whole thing. I would do which matches the term. Also there is no space needed. Preferences: Key Bindings kybi Files under User folder There are some under . I’m using GNU/Linux, but the principals are the same in other OSs. The contents should be in valid format, otherwise they don’t work. files ~/.config/sublime-text-3/Packages/User JSON The is for global settings. It can be accessed by command palette ( ). Preferences.sublime-setting Preferences: Settings The is where you can add key shortcuts. It can be accessed in command palette ( ). Default (Linux).sublime-keymap Preferences: Key Bindings The contains the mouse actions. For now you only can open it by browsing to the directory and edit. However in this article I will show you how to add it to the Command Palette with ease. Default (Linux).sublime-mousemap There is another kind of file that doesn’t exist in that folder by default but you can create one, which the functionalities, I will explain it in below. extends Command Palette Basic functionalities and concepts can be invoked in any of these ways: Multi cursor Select multiple lines, then hit ctrl+shift+l on different places ctrl+left-click Find under expand. Put your cursor on a word, then hit it will select that word. Then hit a few more times to find similar words while creating a new cursor. It also works with partially selected words (a few characters or a few words/lines). ctrl+d ctrl+d Find all: and click on ctrl+f Find All shows the current file’s location in the sidebar and highlights it. Right click on the view and choose . We will create a command for it below as it doesn’t exists (yet). Reveal file in sidebar Reveal In Sidebar by hitting . You can create a command for this one too if you want. Toggle Sidebar ctrl+k ctrl+b is an important one. You can invoke its palette by . It shows you a list of available files in your current project (see below). It too obeys the fuzzy searching rules. Here are more things you can do with it: Go To Anything ctrl+p shows a list of symbols (classes/functions/methods) you can navigate to. @ takes you to a line by typing the line number. : shows you a list of token words (they can be set up in files and they usually come with plugins you install) # .tmLanguage If you don’t type in any file names, they work on the current file. Always have a mental image of your project and possible symbol names and navigate with this palette. Example 1: If you want to go to a file called and you know you want to land on a function called , type this in the palette: . This is just an example and the letters don’t have any significance other than matching. You need to make sure the destination file is focused then add to the mix. brew_beer.go Ingredients() bre@ingrd @ Example 2: If you want to go to the line of a file called , you can type in . 18 drunk_snake.py drnksnk:18 extremely useful for using in multiple panes (more about it in the origami section) New View into File while keeping the cursor at its position can be done by . Note that if you don’t have any selection, the cursor will travel by the edge of the view. In case you want to avoid and keep the cursor where it is, select one word and scroll. It retains the cursor position. Scrolling ctrl+up/down is handy when your lines are long and don’t fit into the view. You can invoke it with Command Palette. Wrap lines can be done with . It uses the ruler position to wrap. Wrapping a paragraph alt+q More useful actions will be mentioned in other sections of this article, please read on. Working with Words and Lines Let’s assume you have your cursor on a word/line and you want to: Go to next/previous word: ctrl+right/left Go to matching bracket: . It also works if you are not a bracket. It will navigate you to the closest bracket. ctrl+m Select the : hit . Please do not go to the end/beginning of the word and do , those combinations have a different purpose. It’s not what a would do! current word ctrl+d ctrl+shift+left/right ninja Select to next/previous word as you go: ctrl+shift+right/left Delete to the : end of a word ctrl+delete Delete to the : beginning of a word ctrl+backspace Delete to the : end of the line ctrl+shift+delete lines: Duplicate ctrl+shift+d Delete current line all together: ctrl+x up/down**:** Swap lines ctrl+shift+up/down . It expands the selection in blocks, and expands more if you hit it more times until it selects the whole file. Expand Selection: ctrl+shift+space Delete a block: hit until you covered the block you want to remove, then delete. ctrl+shit+space Indent line(s) left: and Indent line(s) right: ctrl+[ ctrl+] Jump forward and back (settings all the way down on this article): as you are moving around in your project, you can always go back and forth to the place you were. This is handy for inspecting the project. Insert a tab where ST3 insists on a completion: shift+tab (Edit 1) When you copy/cut stuff in ST3, it keeps them in a history. If you do it shows you a pop up of the history and you can choose to paste it. This is super useful! ctrl+k ctrl+v Join Lines: ctrl+j Project Management Projects Projects contain a series of files/folders added into a project. You can add multiple folders to a project if you wish. I have a folder in my folder called where I keep all my ST3 projects. I also put them into their own sub-folder to manage different workspaces. I suggest you do the same as it clutters the whole folder if you work with a lot of projects. You can open the project settings by invoking in command palette. Here are a few tips: Projects Sublime Text Project: Edit "folders": [{"file_exclude_patterns": ["*.test", "*.out"],"folder_exclude_patterns": ["tmp"],],"path": "/path/to/project/folder",}], Workspaces You can have multiple workspaces per project. They share the same settings as their parent project, therefore when you change the project settings it applies to all of its workspaces. Use them if you have different layouts or you have different concerns while working on a project. You can save them in the same folder you keep your ST3 project by choosing menu. You can also create a brand new workspace for the current project from the menu. Project> Save Workspace As.. Project Fast Switching Projects Every time you open/create projects or workspaces, ST3 keeps a history of what you’ve been working on. You can then hit to navigate between projects or workspaces easily. ctrl+alt+p Handy Plugins is useful for aligning cursors. I have changed the key binding to Alignment { "keys": ["super+a"], "command": "alignment" } Make a multi cursor and hit to align them. It will move the text with which is really handy. You can create such sight in your file: super+a shows you the opening and closing pairs of brackets. I use it to identify the block I am working on. Bracket Highlighter can change word/character cases. It has these modes: Case Conversion snake_case camelCase PascalCase dot.case dash-case Separate words Separate with forward slashes Separate with backslashes Toggle between snake_case, camelCase and PascalCase can change the file name. I use this to depend less on my mouse. File Rename shows git changes in the gutter. You also can view the diffs and roll them back, really handy. Git Gutter I use it every all the times. You can land your cursor on an indented area (maybe a bracket or beginning of a line) and hit to jump to the respective indention. Jump Along Indent: alt+up/down is a must have one if you do on github. Just generate a token on your github account (with only gist enabled permission) and add it to the plugin. You never need to browse to gists web page for your gists any more. You also can load a gist, edit and save it. It updates the gist automatically. HaoGist gists can move your tabs like in your browser. MoveTab is useful if you increment numbers. Create a bunch of cursors and type then select all those zeros and hit . It makes a series of incremented numbers. You can seed it with any numbers. Increment Selection 0 ctrl+alt+i is another must have plugin to make todo lists. It also tracks the time spent on each item. It is very straightforward and starts with its own tutorial when you install it, so there is no need to explain it here. PlainTasks will enhance your sidebar. Make sure you install this one. SideBarEnhancements is a better version of sublime’s bookmark system. You can name your bookmarks and navigate them in the command palette. Very handy. Sublime Bookmarks looks really cool, give it a go. Material Theme : this awesome plugin adds some cool functionalities to multi-cursor functionality. For instance you can edit individual cursors while retaining the who multi-cursor state. You also can cycle through the cursors and edit them! Head over to its page to see the full description. (Edit 1) MultiEditUtils : I mention this the last, because I also wanted to tell you a little story. Please read this part as I explain the decisions I have made to set this very plugin. Origami important I can’t live without . The way ST3 handles panes is way primitive in comparison to tmux and my ST3 work flow has been a bit slow before I started using origami. Before using origami, I rarely utilised panes and only relied on goto anything. The very first time I saw origami, it blew my mind, but soon I left it for the following reasons: tmux The shortcuts were different than my tmux set up It made me slower to code because it didn’t feel natural My tmux prefix is and I couldn’t use it for origami because it is also functionality. Until I asked myself these questions: ctrl+a select all How many times am I selecting the whole file’s contents in a day? Which one contributes to my productivity: Selecting all or having a more flexible workspace? The answer to the second question is obvious, duh! However the answer to the first question turned out very interesting: at most 10 times a day! Now the other obstacle: ? Here is what I came up with, which worked amazingly well for me and I’ve been a happier programmer. I changed the select all to: How am I going to set these two up in such a way I can have best of both { "keys": ["ctrl+a", "ctrl+a"], "command": "select_all" } And changed all prefixes of origami to . Now here is what happens: ctrl+b ctrl+a My pane control/navigation is like the tmux set up, and it feels very natural to me. exactly If I want to select all, I do twice, why not?! ctrl+a Problem resolved. I also disabled creating empty panes, which is really annoying. This means if you do a it creates an empty pane with nothing in it, which is not useful unless you want to create a new file. You can achieve that by creating a new file and move it to a new pane by . In origami’s user settings I have: ctrl+a right ctrl+a shift+right "create_new_pane_if_necessary": false, And in the preferences I added this to close empty panes when the last file is closed: "origami_auto_close_empty_panes": true Now here is a few handy tricks for origami. I call it here, the key I have set to which is by default. Lets call arrow keys a direction. For instance this combination means you hit then hit . prefix ctrl+a ctrl+b up/down/left/right prefix, ctrl+direction ctrl+a ctrl+up creates pane on the direction. prefix, ctrl+direction destroys pane on that direction. prefix, ctrl+shift+direction travels to the direction. prefix, direction travels while carries the file to the direction. prefix, shift+direction clones (makes a new view on the file) while bringing the new view to the direction. prefix, alt+direction zooms the pane. prefix, ctrl+z un-zooms the pane back to normal. prefix, ctrl+shift+z shows you an input box to put in the proportion of pane’s horizontal (column) size. I’ll explain in a bit. prefix, ctrl+c shows you an input box to put in the proportion of pane’s vertical (row) size. prefix, ctrl+r The sizes work like this: If you are on a pane that has only one pane of its side (or above/below): 0.3 means the the occupies 30 percent of the space in respect to the whole space. pane you are on If you are on the middle pane: 0.2, 0.9 means the left (or top) most pane gets 20 percent, the right one gets 90 percent and you get the rest. In the first point by whole space I mean if you have three panes and you are on the first one, it accounts the percentage as if there is only two, and it pushes the other panes to make room. : You can make more and put them side by side in order to work on different portion of the file and navigate through the panes as opposed to jumping up and down. A tip Views Into a File Origami has a concept of saving layouts. You also can create a layout and save the workspace. Either one works, but I prefer to save the layout. Now you can get rid of the sidebar and enjoy more space. Changing Some Settings I found these settings to be useful: {"bold_folder_labels": true,"close_windows_when_empty": true,"detect_slow_plugins": false,"enable_tab_scrolling": false,"ensure_newline_at_eof_on_save": true,"fade_fold_buttons": false,"font_face": "Inconsolata-g for Powerline","font_options":["subpixel_antialias","gray_antialias","directwrite"],"font_size": 11,"line_padding_bottom": 3,"line_padding_top": 3,"margin": 0,"caret_extra_bottom": 0,"caret_extra_top": 0,"caret_extra_width": 1,"caret_style": "wide","tab_size": 4,"spell_check": true,"dictionary": "Packages/Language - English/en_GB.dic","file_exclude_patterns":["*.sqlite3","*.exe","*.pyc","*.pyo","*.dia"],"folder_exclude_patterns":[".svn",".git",".gitignore",".hg","CVS","_build","dist","build","site"],"highlight_line": true,"highlight_modified_tabs": false,"indent_guide_options":["draw_active"],"match_brackets_angle": true,"material_theme_accent_orange": true,"material_theme_contrast_mode": true,"material_theme_small_statusbar": true,"material_theme_small_tab": true,"material_theme_tabs_autowidth": true,"origami_auto_close_empty_panes": true,"save_on_focus_lost": false,"shift_tab_unindent": false,"show_encoding": true,"show_line_endings": true,"show_tab_close_buttons": true,"theme": "Material-Theme-Darker.sublime-theme","translate_tabs_to_spaces": true,"trim_trailing_white_space_on_save": true,"use_tab_stops": true,"word_wrap": "false"} Notice the file_exclude_patterns and folder_exclude_patterns are also in project settings, when you set it here it applies it to every projects you have. I have excluded .git because I don’t want ST3 to search in those files when I do to search in everything. ctrl+shift+f Commands Create a file by any name and extension. Like so: and add this to the file: sublime-command arsham.sublime-command [{"caption": "Open Mouse Settings", "command": "open_file", "args": {"file": "${packages}/User/Default (Linux).sublime-mousemap"}},] Then invoke the Command Palette and type in . It will open your mouse map file (I have mentioned this earlier in this article). There is a wide range of commands you can do. In fact, I would like you to do an experiment. Open the by hitting then type this in there: Open Mouse Settings Console Panel ctrl+` sublime.log_commands(True) Without closing the console, do something to sublime. Like toggling the The short key is . You can see in the console it says: SideBar. ctrl+k ctrl+b command: toggle_side_bar Now you know the command name, you can add this to your commands: {"caption": "Toggle My sidebar please", "command": "toggle_side_bar"} And invoke it with command palette. Go ahead and try a few more! Here are some useful commands: {"caption": "Reveal File In SideBar", "command": "reveal_in_side_bar"},{"caption": "Open Mouse Settings", "command": "open_file", "args": {"file": "${packages}/User/Default (Linux).sublime-mousemap"}},{"caption": "Open Project", "command": "prompt_open_project_or_workspace"},{"caption": "Fold Comments", "command": "fold_comments"},{"caption": "Unfold Comments", "command": "unfold_comments"},{"caption": "Fold One", "command": "fold_by_level", "args": {"level": 1}},{"caption": "Fold Two", "command": "fold_by_level", "args": {"level": 2}},{"caption": "Fold Three", "command": "fold_by_level", "args": {"level": 3}},{"caption": "Fold Four", "command": "fold_by_level", "args": {"level": 4}},{"caption": "Fold Five", "command": "fold_by_level", "args": {"level": 5}},{"caption": "Fold Six", "command": "fold_by_level", "args": {"level": 6}},{"caption": "Fold Seven", "command": "fold_by_level", "args": {"level": 7}},{"caption": "Fold Eight", "command": "fold_by_level", "args": {"level": 8}},{"caption": "Fold Nine", "command": "fold_by_level", "args": {"level": 9}}, Here are some useful key bindings: { "keys": ["super+a"], "command": "alignment" },{ "keys": ["alt+left"], "command": "jump_back" },{ "keys": ["alt+right"], "command": "jump_forward" },{ "keys": ["ctrl+0"], "command": "reset_font_size" }, Please let me know in the comments if you have any suggestions how to improve this article, thanks! Conclusion Sublime Text 3 is really fast and comes with a lot of functionalities. When extended with plugins can be the best editor around. Please try to use your keyboard and rely less on mouse navigations. Enjoy!

Supercharge your Sublime Text 3 to increase your productivity

is a statistical method used for analyzing a dataset in which there are one or more independent variables that can be used to predict the outcome of a categorical dependent variable. In simpler terms, it's a technique used for classification problems where the target variable is binary (i.e., it can take on two possible values, such as 0 or 1). Logistic regression The main goal of logistic regression is to find the best-fitting model to describe the relationship between the independent variables and the probability of a particular outcome. It uses the logistic function (also known as the sigmoid function) to map predicted values to probabilities. Difference Between Linear Regression and Logistic Regression is used when the target variable is continuous and can take any numerical value. It predicts a value that is unbounded, which means it can be any real number. Linear regression uses a straight line to model the relationship between the independent variables and the dependent variable. Linear regression , on the other hand, is used when the target variable is categorical and represents two classes (binary classification). It predicts the probability that a given instance belongs to a particular class. The output of logistic regression is bounded between 0 and 1, thanks to the logistic (sigmoid) function. Logistic regression Real World Applications of Logistic Regression Logistic regression finds applications in various fields due to its simplicity, interpretability, and effectiveness in binary classification problems. Some real-world applications include: : Medical Diagnostics Predicting whether a patient has a particular disease based on symptoms and test results. : Finance Credit scoring to determine if a borrower is likely to default on a loan. Predicting the likelihood of a customer subscribing to a service based on demographic data. The logistic regression model uses the logistic function (also known as the sigmoid function) to model the relationship between the independent variables and the probability of a particular outcome. Mathematical Formula for Logistic Regression The logistic regression model can be mathematically expressed as: Logistic Function (Sigmoid Function): The logistic function is defined as: Here, ( ) is the output (probability), and is the linear combination of the independent variables and their coefficients: S t t The logistic function transforms any value of to a value between 0 and 1. This is crucial in logistic regression because we want to model probabilities. t Transformation of Linear Combination into Probabilities The linear combination t represents the weighted sum of the independent variables. When we apply the logistic function to t, we obtain a value between 0 and 1. This value represents the estimated probability of the event =1 given the values of the independent variables. Y In logistic regression, the goal is to find the best-fitting set of coefficients that maximizes the likelihood of the observed data. This is typically done through optimization techniques like gradient descent. Binary and Multinomial Logistic Regression Binary Logistic Regression: is used when the dependent variable has two possible categories or classes. These classes are typically coded as 0 and 1, representing the absence or presence of an outcome or event. Binary logistic regression For example, binary logistic regression can be used for tasks like: Predicting whether an email is spam (1) or not spam (0). Determining if a patient has a certain disease (1) or does not have it (0). In binary logistic regression, the logistic function is used to model the probability of belonging to one of the two classes. Multinomial Logistic Regression: , also known as softmax regression, is used when the dependent variable has more than two categories. It's an extension of logistic regression that can handle multiple classes. Multinomial logistic regression For example, multinomial logistic regression can be used for tasks like: Predicting the type of fruit in a basket (e.g., apple, banana, orange). Classifying different species of animals (e.g., cat, dog, bird). In multinomial logistic regression, the logistic function is extended to handle multiple classes. The softmax function is used to compute the probabilities of each class, and the class with the highest probability is predicted. Here's how it works: : Softmax Function The softmax function takes a vector of scores (one for each class) and transforms them into probabilities that sum to 1. It assigns a probability to each class based on the scores. : Prediction The class with the highest probability is predicted as the outcome. Code Step 1: Import necessary libraries import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score Step 2: Load and prepare the Titanic dataset data = pd.read_csv('train.csv') Step 3: Select relevant features and target variable X = data[['Pclass', 'Sex', 'Age', 'SibSp', 'Parch']]
y = data['Survived'] Step 4: Data Cleaning and Encoding # Convert categorical variable 'Sex' to numeric
X['Sex'] = X['Sex'].map({'male': 0, 'female': 1})

# Handle missing values in 'Age' (replace with mean)
X['Age'].fillna(X['Age'].mean(), inplace=True) Step 5: Split Dataset # Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) Step 6: Initialize the Model model = LogisticRegression()
model.fit(X_train, y_train) Step 7: Evaluate the Model # Step 5: Make predictions
y_pred = model.predict(X_test)

# Step 6: Evaluate the model
accuracy = accuracy_score(y_test, y_pred)
print(f'Accuracy: {accuracy * 100:.2f}%')

# ------------------ OUTPUT -------------- #
Accuracy: 82.68%

Mastering Logistic Regression: A Comprehensive Guide with Practical Example

Linear regression is a fundamental statistical technique used to model the relationship between a dependent variable and one or more independent variables. In simple terms, it helps us understand how changes in the independent variables are associated with changes in the dependent variable. In machine learning and data analysis, linear regression holds significant importance. It serves as a foundational tool for various tasks, including prediction, forecasting, and understanding the underlying patterns in data. It provides a simple yet powerful framework for making predictions based on historical data, making it a cornerstone in the field of predictive modeling. Moreover, linear regression also serves as a basis for more complex models and techniques in machine learning, making it a crucial concept to grasp for anyone working with data. What is Linear Regression The basic idea behind linear regression is to find the best-fitting line (or hyperplane in higher dimensions) that minimizes the difference between the predicted values and the actual values of a dependent variable based on one or more independent variables. In simple linear regression, there is one dependent variable (Y) and one independent variable (X). The relationship between them is modeled as: Here, Y represents the dependent variable (the one we want to predict). X represents the independent variable (the one used to make predictions). Beta_0 ​ is the intercept (the point where the line crosses the Y-axis). Beta_1 is the slope (the rate of change of Y with respect to X). epsilon represents the error term, which accounts for the variability in Y that is not explained by X. The goal of linear regression is to find the best values for Beta_0 and Beta_1 that minimize the sum of squared errors (the vertical distance between the actual data points and the predicted values on the line). In multiple linear regression, there are multiple independent variables. The relationship is extended as: Here, X_1, X_2, …, X_n represents the independent variables and Beta_1, Beta_2,…,Beta_n are the respective coefficients. Linear regression allows us to quantify the relationship between the dependent variable and the independent variable(s). It helps us understand how changes in the independent variables are associated with changes in the dependent variable. This understanding is crucial for making predictions, drawing insights, and making informed decisions based on data. Now, lets start implementing simple linear regression and multiple linear regression using sklearn. In this article, I’ll use the Dataset from Kaggle. California House Price Simple Linear Regression with Sklearn To demonstrate simple linear regression using the sklearn library, we'll use a California house price prediction dataset from Kaggle. Importing Libraries import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error Load the Dataset data = pd.read_csv('california_housing.csv')

# Explore the dataset
print(data.head()) Prepare the Data For this demonstration, we'll use the 'median_income' column as the independent variable ( ) and the 'median_house_value' column as the dependent variable ( ). X y X = data[['median_income']]
y = data['median_house_value'] Split the Data into Training and Testing Sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) Initialize and Train the Model # Initialize the model
model = LinearRegression()

# Train the model on the training data
model.fit(X_train, y_train) Make Predictions # Use the trained model to make predictions on the test set
y_pred = model.predict(X_test) Evaluate the Model # Calculate the Mean Squared Error (MSE) to evaluate the model's performance
mse = mean_squared_error(y_test, y_pred)
print(f'Mean Squared Error: {mse}')

# -------------- OUTPUT -------------- #
Mean Squared Error: 7091157771.76555 Multiple Linear Regression with Skearn Multiple linear regression extends the concept of simple linear regression to incorporate multiple independent variables. This allows us to model relationships between a dependent variable and two or more independent variables. Let's use the California housing dataset for this demonstration. Import Libraries import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error Load Dataset data = pd.read_csv('california_housing.csv')

# Explore the dataset
print(data.head()) Prepare the Data For this demonstration, we'll use 'median_income', 'total_rooms', and 'population' as the independent variables ( ) and 'median_house_value' as the dependent variable ( ). X y X = data[['median_income', 'total_rooms', 'population']]
y = data['median_house_value'] Splitting Dataset X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) Initialize and Train the Model # Initialize the model
model = LinearRegression()

# Train the model on the training data
model.fit(X_train, y_train) Make Predictions # Use the trained model to make predictions on the test set
y_pred = model.predict(X_test) Evaluate the Model # Calculate the Mean Squared Error (MSE) to evaluate the model's performance
mse = mean_squared_error(y_test, y_pred)
print(f'Mean Squared Error: {mse}')

# ---------------- OUTPUT -------------- #
Mean Squared Error: 7056856633.523101 I used Mean Squared Error evaluation metric to evaluate both the models, but there are multiple evaluation metrics that you can use for a model like this. Lets now learn about all the different evaluation metrics. Model Evaluation and Performance Metrics Evaluating the performance of a linear regression model is crucial to understand how well it fits the data and makes accurate predictions. Here are some common metrics used for this purpose: Mean Absolute Error (MAE): : The Mean Absolute Error is the average of the absolute differences between the predicted and actual values. It measures the average magnitude of errors. Definition : MAE provides a straightforward and easy-to-interpret measure of model performance. It's expressed in the same units as the target variable, making it intuitive to understand. Significance Mean Squared Error (MSE): : The Mean Squared Error is the average of the squared differences between the predicted and actual values. It gives more weight to large errors compared to MAE. Definition : MSE penalizes larger errors more heavily. It is useful when larger errors are considered more problematic (e.g., in situations where outliers have a significant impact on the model's performance). Significance Root Mean Squared Error (RMSE): : The Root Mean Squared Error is the square root of the MSE. It provides a measure of the average magnitude of the errors, but in the same units as the dependent variable. Definition : RMSE is particularly useful because it is in the same units as the target variable. This makes it easy to interpret and compare with the original data. Significance R-squared (R²): : R-squared is a statistical measure that represents the proportion of the variance in the dependent variable that is predictable from the independent variables. It ranges from 0 to 1, where 1 indicates a perfect fit. Definition : R-squared provides an overall indication of how well the model explains the variability in the data. A higher R-squared value indicates a better fit. However, it doesn't tell you about the magnitude or direction of individual errors. Significance Adjusted R-squared: : Adjusted R-squared is an adjusted version of R-squared that takes into account the number of independent variables in the model. It penalizes the inclusion of unnecessary variables. Definition : It provides a more accurate measure of the model's explanatory power when there are multiple independent variables. It helps prevent overfitting by accounting for the complexity of the model. Significance Interpretation: : "On average, our model's predictions are off by X units." MAE : "The average squared prediction error of our model is X." MSE/RMSE : "X% of the variance in the dependent variable is explained by the independent variables in our model." R-squared Choosing the appropriate evaluation metric depends on the specific problem and the importance of different types of errors. For instance, in some cases, minimizing the impact of large errors (MSE/RMSE) might be more critical, while in others, a clear and intuitive interpretation (MAE) might be preferred. R-squared provides a broad overview of the model's fit, but it's important to consider other metrics in conjunction with it.

Linear Regression Explained With Sklearn

In the vast landscape of data analysis and machine learning, uncovering meaningful patterns hidden within datasets is often akin to discovering buried treasures. One such powerful tool that aids in this pursuit is the K-Means clustering algorithm. At its core, K-Means is a technique that allows us to group similar data points together, shedding light on the underlying structure of the data. This article delves into the intricacies of the K-Means algorithm, exploring its mechanics and rationale, and highlighting why this method is a pivotal asset in the toolkit of any data enthusiast or machine learning practitioner. Explaining K-Means Clustering Imagine a scenario where we are presented with a vast collection of data points, each with a multitude of features. These data points might seem like an indistinguishable sea of information. However, within this sea, there are often clusters of data points that share intrinsic similarities. These clusters might represent customer segments, disease subtypes, or any other latent patterns. K-Means clustering emerges as the navigator that helps us steer through this sea, revealing these clusters with remarkable precision. The Essence of K-Means At its essence, K-Means is a method that groups data points based on their similarity in feature space. The algorithm accomplishes this by iteratively categorizing data points into clusters and refining these assignments until convergence. By doing so, it identifies natural groupings and assigns each data point to the cluster whose centroid (representative point) is closest in feature space. Why Care About K-Means? The significance of K-Means lies in its broad range of applications and its ability to simplify complex datasets: : K-Means uncovers hidden structures in data, making it easier to understand and interpret. Data Exploration : Businesses can segment customers based on their purchasing behavior, tailoring strategies for distinct groups. Customer Segmentation : K-Means can identify data points that deviate significantly from the norm, highlighting potential outliers or anomalies. Anomaly Detection : In image processing, K-Means is used to reduce the number of colors in an image while preserving its essence. Image Compression : Understanding consumer preferences and behavior is vital, and K-Means can assist in forming market segments. Market Research : K-Means can be used as a tool to create new features for machine learning models, capturing underlying data patterns. Feature Engineering In a world awash with data, K-Means clustering empowers us to distill information into actionable insights. It serves as a gateway to understanding the intricate relationships between data points, aiding us in making informed decisions and driving innovation. As we traverse the landscape of data analysis, K-Means remains a guiding star that illuminates the path to discovering the hidden treasures buried within our data. The K-Means Algorithm: Unveiling Data Clusters K-Means clustering algorithm is a fundamental unsupervised learning technique that partitions a dataset into distinct clusters based on similarity. The algorithm is iterative in nature and aims to group data points with similar features together. It's a powerful tool for exploring data patterns, segmenting datasets, and understanding the inherent structure within complex data. Algorithm Steps: : Begin by selecting the number of clusters 'K' you want to identify within the data. Initialize 'K' centroids, which serve as the representative points for each cluster. These centroids can be randomly chosen or using more advanced techniques like K-Means++. Initialization : In this step, each data point is assigned to the nearest centroid. The distance metric commonly used is the Euclidean distance, although other distance metrics can be employed based on the nature of the data. Assignment Step : After all data points have been assigned to centroids, calculate new centroids for each cluster. This is done by computing the mean of all data points assigned to a particular centroid. These new centroids represent the updated center of each cluster. Update Step : The assignment and update steps are repeated iteratively until a stopping condition is met. This can be a predefined number of iterations or until the centroids stop changing significantly. Convergence indicates that the algorithm has reached a stable solution. Convergence Minimizing Within-Cluster Variance: The K-Means algorithm aims to minimize the within-cluster variance or the sum of squared distances between data points and their respective centroids. As the algorithm progresses, data points tend to gravitate towards centroids that best represent their cluster, leading to tighter and more cohesive clusters. Selecting the Optimal K: Choosing the right number of clusters 'K' is a crucial step. An improper choice of 'K' can lead to either over-segmentation or under-segmentation of the data. Techniques like the elbow method, silhouette analysis, or cross-validation can help determine the optimal number of clusters by evaluating the trade-off between the number of clusters and the clustering quality. Pros and Cons of K-Means: Pros: Simple and easy to understand. Efficient for large datasets. Applicable to a wide range of data types. Can handle non-linear data, although it's primarily designed for convex clusters. Cons: Sensitive to initial centroid placement, which can lead to suboptimal solutions. Assumes clusters are spherical and equally sized, which might not reflect complex data distributions. Can converge to local optima, especially when using a random initialization of centroids. Conclusion: The K-Means algorithm is a cornerstone of data clustering, offering an intuitive approach to uncover hidden patterns within datasets. By iteratively refining clusters and centroids, K-Means transforms seemingly chaotic data into organized groupings, making it an invaluable technique in data analysis, customer segmentation, and various other fields. Selecting the Optimal Number of Clusters: Methods for Choosing K in K-Means Choosing the right number of clusters, denoted as 'K', is a critical step in the K-Means clustering algorithm. Selecting an inappropriate 'K' value can lead to misleading results and misinterpretation of data patterns. To address this challenge, several methods have been developed to aid in determining the optimal number of clusters. Here, we explore three commonly used methods: the Elbow Method, Silhouette Analysis, and Cross-Validation. 1. The Elbow Method: The Elbow Method is a straightforward technique to visualize how the variance within clusters changes as the number of clusters increases. The idea is to plot the sum of squared distances (inertia) between data points and their assigned centroids for different values of 'K'. As 'K' increases, the inertia tends to decrease since more centroids provide a closer fit to data points. However, there's a point where the rate of decrease slows down, forming an "elbow" in the plot. The 'K' value at the elbow can be considered as the optimal number of clusters, as it balances capturing variance and avoiding over-segmentation. 2. Silhouette Analysis: Silhouette Analysis is a metric that quantifies how similar an object is to its own cluster compared to other clusters. It ranges from -1 to 1, where a high silhouette score indicates that the object is well matched to its own cluster and poorly matched to neighboring clusters. By calculating silhouette scores for various 'K' values, you can identify the 'K' that maximizes the average silhouette score across all data points. This approach considers both the cohesion within clusters and the separation between clusters, leading to a more holistic assessment of cluster quality. 3. Cross-Validation: Cross-Validation involves splitting the dataset into training and validation subsets and then evaluating the clustering quality using various 'K' values. This process is repeated multiple times with different data splits to mitigate the impact of randomness. The 'K' that consistently yields the best clustering results across different validation sets is chosen as the optimal number of clusters. Cross-validation provides a more robust method for selecting 'K', as it considers the stability of clusters across different subsets of the data. Considerations and Trade-offs: While these methods offer valuable guidance, it's important to note that there is no one-size-fits-all solution for choosing 'K'. Different methods might yield slightly different results, and the final choice of 'K' can also depend on the context and goals of your analysis. It's advisable to apply multiple methods and analyze the results collectively to make an informed decision. Code Now, let’s start coding, here I’ll use the from kaggle. Mall Customer Segmentation Data Importing Libraries import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler In this block, essential libraries are imported. is used for data handling, for numerical computations, for data visualization, from for K-Means clustering, and from for feature scaling. pandas numpy matplotlib.pyplot KMeans sklearn.cluster StandardScaler sklearn.preprocessing Load the Dataset data = pd.read_csv('/kaggle/input/customer-segmentation-tutorial-in-python/Mall_Customers.csv') The dataset from the given path is loaded into a Pandas DataFrame named . It's assumed that the dataset contains columns like 'Annual Income (k$)' and 'Spending Score (1-100)'. data Select Relevant Features X = data[['Annual Income (k$)', 'Spending Score (1-100)']] The relevant features for clustering are selected and stored in the variable . In this case, the 'Annual Income (k$)' and 'Spending Score (1-100)' columns are chosen for clustering. X Standardize Features scaler = StandardScaler()
X_scaled = scaler.fit_transform(X) The features are standardized using to ensure that they have a mean of 0 and a standard deviation of 1. This preprocessing step is important for K-Means clustering. StandardScaler Choosing Optimal Number of Clusters Using Elbow Method inertia_values = []
num_clusters_range = range(1, 11)
for num_clusters in num_clusters_range:
    kmeans = KMeans(n_clusters=num_clusters, random_state=42)
    kmeans.fit(X_scaled)
    inertia_values.append(kmeans.inertia_) The code calculates the inertia (within-cluster sum of squares) for different numbers of clusters (1 to 10) using the Elbow Method. For each , a K-Means model is created and fitted to the standardized data. The inertia value for each configuration is appended to . num_clusters inertia_values Visualizing the Elbow Method plt.plot(num_clusters_range, inertia_values, marker='o')
plt.xlabel('Number of Clusters')
plt.ylabel('Inertia (Within-Cluster Sum of Squares)')
plt.title('Elbow Method for Optimal Number of Clusters')
plt.show()

optimal_num_clusters = int(input("Enter the optimal number of clusters: ")) Applying k-means Clustering with optimal number of clusters kmeans = KMeans(n_clusters=optimal_num_clusters, random_state=42)
cluster_labels = kmeans.fit_predict(X_scaled) A K-Means model is created using the optimal number of clusters, and the method is used to perform clustering on the standardized data. The resulting cluster labels are stored in the variable . fit_predict cluster_labels Visualize Clusters plt.figure(figsize=(10, 6))
colors = ['b', 'g', 'r', 'c', 'm', 'y', 'k', 'orange', 'purple', 'brown']
for i in range(optimal_num_clusters):
    cluster_data = data[data['cluster'] == i]
    plt.scatter(cluster_data['Annual Income (k$)'], cluster_data['Spending Score (1-100)'],
                color=colors[i], label=f'Cluster {i + 1}')
plt.xlabel('Annual Income (k$)')
plt.ylabel('Spending Score (1-100)')
plt.title(f'K-Means Clustering of Mall Customer Segmentation Data (Clusters: {optimal_num_clusters})')
plt.legend()
plt.show() The clusters are visualized using a scatter plot. Each cluster is assigned a different colour, and the 'Annual Income (k$)' is plotted on the x-axis while the 'Spending Score (1-100)' is plotted on the y-axis. Cluster data points are plotted, and a legend is added to the plot to indicate the clusters. Result Interpretation After performing the K-Means clustering on the Mall Customer Segmentation Data and selecting the optimal number of clusters, you can interpret the segmentation results based on the characteristics of each cluster. In this case, let's assume you chose the optimal number of clusters to be 5. Here's how you could interpret the segmentation: Cluster 1: High Income, High Spending Score (High Value Shoppers) Customers in this cluster have both high annual incomes and high spending scores. These individuals are likely to be high-value shoppers who are willing to spend more on products and services. Businesses could target this segment with premium offerings and personalized shopping experiences to enhance customer loyalty. Cluster 2: Moderate Income, Moderate Spending Score (Average Shoppers) This cluster comprises customers with moderate annual incomes and moderate spending scores. They represent a broad range of shoppers who spend reasonably in the mall. Businesses might focus on providing value-oriented products and promotions to attract and retain these customers. Cluster 3: Low Income, High Spending Score (Careful Spenders) Customers in this cluster have relatively lower incomes but exhibit a high spending score. These individuals are likely to be careful spenders who prioritize their purchases and make the most of their budget. Offering affordable yet quality products and targeted discounts could appeal to this segment. Cluster 4: Low Income, Low Spending Score (Frugal Shoppers) This cluster consists of customers with low incomes and low spending scores. They are cautious shoppers who spend conservatively. Businesses could consider introducing budget-friendly options and incentives to attract and engage this segment. Cluster 5: High Income, Low Spending Score (Potential for Growth) Customers in this cluster have high incomes but relatively low spending scores. This group might have potential for growth, and businesses could focus on understanding their preferences and interests to encourage higher spending. Personalized offers and experiences could help unlock this segment's potential. Overall, the K-Means segmentation has grouped mall customers into distinct categories based on their spending behavior and income levels. This information can guide businesses in tailoring their marketing strategies, product offerings, and customer engagement initiatives to effectively cater to the diverse needs and preferences of each customer segment.

Mastering K-Means: Data Clustering Simplified

Understanding Principal Component Analysis

In the realm of machine learning and data science, feature engineering stands as a cornerstone of model performance improvement. While algorithms and models often grab the spotlight, it's the carefully crafted features that lay the foundation for predictive power. Imagine trying to build a house on a weak foundation – without strong features, even the most advanced algorithms may crumble when faced with complex datasets. Understanding Feature Engineering Feature engineering is the creative and strategic process of selecting, transforming, and creating features (input variables) from raw data to maximize a model's performance. It's the art of converting data into actionable insights, giving machines the contextual understanding they need to make accurate predictions. Whether it's predicting house prices, classifying customer preferences, or diagnosing medical conditions, feature engineering holds the key to unlocking the potential hidden within your data. The Importance of Careful Craftsmanship Imagine you're tasked with predicting house prices based on a variety of factors. You might initially be drawn to the obvious features like square footage, number of bedrooms, and location. However, the magic of feature engineering lies in unearthing the subtler aspects that influence the target variable. Does the ratio of bathrooms to bedrooms impact the price? What about the presence of a fireplace or the age of the roof? Feature engineering isn't merely about creating more features; it's about discerning which facets of the data truly matter. This process often requires domain knowledge, creativity, and a deep understanding of the problem you're tackling. By refining existing features and crafting new ones, you're essentially teaching your model to understand the data like an expert. Why should you care about feature engineering? The answer lies in model performance. Well-engineered features can lead to faster convergence during training, reduced overfitting, and ultimately, more accurate predictions. A machine learning model is only as good as the data it's fed, and well-engineered features provide a richer, more nuanced representation of that data. In this article, we'll delve into the world of feature engineering using the advanced house price prediction dataset from Kaggle. By following along, you'll gain insights into various techniques that can transform raw data into valuable predictors and you’ll see how the results of your model improve by applying different feature engineering methods. So, without wasting anytime, let’s start learning about the different methods of feature engineering. Methods Now that we have set the stage, it's time to delve into the exciting world of advanced feature engineering techniques. In this section, I'll guide you through the step-by-step implementation of four powerful methods that can supercharge your predictive models. Each method serves a unique purpose, offering insights and improvements that can make a substantial difference in your model's performance. Method 1: Mutual Information - Extracting Information from Relationships Imagine being able to select the most influential features for your model with surgical precision. Mutual Information allows you to achieve just that. By quantifying the relationship between each feature and the target variable, you can identify the key factors that impact your predictions. We'll walk you through the code and provide a detailed explanation of every step, helping you master this insightful technique. Method 2: Clustering - Discovering Patterns through Grouping Clusters often hide valuable patterns within your data. With clustering, you can uncover these hidden gems and leverage them to enhance your model's understanding. I'll guide you through the process of applying KMeans clustering to group similar instances together. You'll learn how to create new features based on these clusters and observe their impact on model performance. Method 3: PCA Dimensionality Reduction - Condensing Complexity High-dimensional data can be overwhelming, but Principal Component Analysis (PCA) offers a solution. By identifying the most influential dimensions, you can reduce complexity without sacrificing predictive power. This tutorial will lead you through the transformation of your data using PCA, providing insights into how this technique can streamline your model while retaining its accuracy. In addition to these methods, I'll also introduce you to —an often overlooked technique that can wield powerful results. By applying mathematical operations to selected columns, you can shape your data to better align with your model's assumptions. You'll explore logarithmic, square root, and inverse transformations, illustrating how they can uncover hidden relationships and boost your model's accuracy. Mathematical Transformations Throughout this section, I'll offer comprehensive explanations to ensure that you not only grasp the technical aspects but also understand the reasoning behind each method's application. By the end, you'll have gained a valuable toolkit of advanced feature engineering techniques that can be confidently applied to enhance your predictive models in various scenarios. Are you ready to explore the intricacies of each method, uncover their potential, and equip yourself with the expertise to engineer features that translate to more accurate and powerful predictive models? Let's begin this enlightening journey! Code Importing Packages Let’s first import all the packages that you need. import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
from sklearn.feature_selection import SelectKBest, mutual_info_regression
from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
from sklearn.preprocessing import LabelEncoder, StandardScaler
import numpy as np Getting The Data Now, let’s get the dataset from kaggle and preprocess it. # Load the dataset
data = pd.read_csv('train.csv')

# Data preprocessing
def preprocess_data(df):
    # Handle missing values
    df = df.fillna(df.median())

    # Handle categorical variables
    df = pd.get_dummies(df, drop_first=True)

    return df

data = preprocess_data(data)

# Prepare the data
X = data.drop('SalePrice', axis=1)
y = data['SalePrice']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) To start, let's run a baseline model without any feature engineering and assess its accuracy. After that, we'll apply each of the four feature engineering methods (mutual information, clustering, PCA dimensionality reduction, and label encoding) individually and compare their effects on model performance. Base Line Model Before applying any feature engineering, we'll start with a baseline model. Here, we'll use a simple linear regression model to predict house prices using the original dataset. baseline_model = LinearRegression()
baseline_model.fit(X_train, y_train)
baseline_predictions = baseline_model.predict(X_test)
baseline_rmse = mean_squared_error(y_test, baseline_predictions, squared=False)
print(f"Baseline RMSE: {baseline_rmse}")

# ------------------ OUTPUT ---------------- #
Baseline RMSE: 49204.92 In the baseline model, you're starting with a simple linear regression model that uses the original features as they are. You're training the model using the training data, making predictions on the test data, and calculating the root mean squared error (RMSE) to measure how well the model performs on unseen data. Mutual Information Model # Method 1: Mutual Information
mi_selector = SelectKBest(score_func=mutual_info_regression, k=10)
X_train_mi = mi_selector.fit_transform(X_train, y_train)
X_test_mi = mi_selector.transform(X_test)
mi_model = LinearRegression()
mi_model.fit(X_train_mi, y_train)
mi_predictions = mi_model.predict(X_test_mi)
mi_rmse = mean_squared_error(y_test, mi_predictions, squared=False)
print(f"Mutual Information RMSE: {mi_rmse}")

# ------------------ OUTPUT ----------------- #
Mutual Information RMSE: 39410.99 Here, you're exploring the information each feature provides about the target variable. You select the top 10 features that have the highest mutual information scores with the target. Then, you train a new linear regression model using only these selected features. This helps ensure that your model focuses on the most informative features, and you calculate the RMSE to see how this model's predictions compare to the baseline. Clustering Model # Method 2: Clustering
num_clusters = 5
kmeans = KMeans(n_clusters=num_clusters, random_state=42)
X_clustered = X.copy()
X_clustered['Cluster'] = kmeans.fit_predict(X)
X_clustered = pd.get_dummies(X_clustered, columns=['Cluster'], prefix='Cluster')
X_train_clustered, X_test_clustered, _, _ = train_test_split(X_clustered, y, test_size=0.2, random_state=42)
cluster_model = LinearRegression()
cluster_model.fit(X_train_clustered, y_train)
cluster_predictions = cluster_model.predict(X_test_clustered)
cluster_rmse = mean_squared_error(y_test, cluster_predictions, squared=False)
print(f"Clustering RMSE: {cluster_rmse}")

# ------------------- OUTPUT -------------- #
Clustering RMSE: 47715.30 You're looking at grouping similar instances in your data using clustering. Specifically, you use the KMeans algorithm to divide your data into clusters. Each instance is assigned to a cluster, and you add this cluster information as a new categorical feature. By doing this, you're giving the model a way to consider the relationships between instances in terms of their clusters. After training a linear regression model on this clustered data, you calculate the RMSE to evaluate its performance. PCA - Dimensionality Reduction Model # Method 3: PCA Dimensionality Reduction
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X_train)
pca = PCA(n_components=10)
X_train_pca = pca.fit_transform(X_scaled)
X_test_pca = pca.transform(scaler.transform(X_test))
pca_model = LinearRegression()
pca_model.fit(X_train_pca, y_train)
pca_predictions = pca_model.predict(X_test_pca)
pca_rmse = mean_squared_error(y_test, pca_predictions, squared=False)
print(f"PCA RMSE: {pca_rmse}")

# --------------- OUTPUT -------------- #
PCA RMSE: 40055.78 PCA helps you reduce the complexity of your data by summarizing it into fewer dimensions. You standardize your data to make sure all features are on the same scale. Then, you use PCA to identify the most important patterns in your data and reduce the number of features to 10 principal components. Training a linear regression model on these components, you're able to capture the most significant information while simplifying the model. The RMSE helps you gauge if this approach is effective. Mathematical Transformations Model # Method 5: Mathematical Transformations
X_train_transformed = X_train.copy()
X_test_transformed = X_test.copy()

# Apply logarithmic transformation
log_columns = ['GrLivArea', 'LotArea']  # Specify columns to apply log transformation
X_train_transformed[log_columns] = np.log1p(X_train_transformed[log_columns])
X_test_transformed[log_columns] = np.log1p(X_test_transformed[log_columns])

# Apply square root transformation
sqrt_columns = ['GarageArea', '1stFlrSF']  # Specify columns to apply square root transformation
X_train_transformed[sqrt_columns] = np.sqrt(X_train_transformed[sqrt_columns])
X_test_transformed[sqrt_columns] = np.sqrt(X_test_transformed[sqrt_columns])

# Apply inverse transformation
inv_columns = ['YearBuilt', 'OverallQual']  # Specify columns to apply inverse transformation
X_train_transformed[inv_columns] = 1 / X_train_transformed[inv_columns]
X_test_transformed[inv_columns] = 1 / X_test_transformed[inv_columns]

math_transform_model = LinearRegression()
math_transform_model.fit(X_train_transformed, y_train)
math_transform_predictions = math_transform_model.predict(X_test_transformed)
math_transform_rmse = mean_squared_error(y_test, math_transform_predictions, squared=False)
print(f"Mathematical Transformations RMSE: {math_transform_rmse}")

# ------------------ OUTPUT --------------- #
Mathematical Transformations RMSE: 47143.21 Mathematical transformations involve altering feature values using mathematical operations to bring out underlying patterns. You apply logarithmic, square root, and inverse transformations to specific columns. For example, logarithmic transformation helps normalize skewed data, square root transformation can help with outliers, and inverse transformation can emphasize relationships with small values. You train a linear regression model using these transformed features and calculate the RMSE to assess whether the transformations have improved the model's predictive power. In all these methods, you're experimenting with different techniques to make your data more suitable for modeling. The goal is to find the method that leads to the lowest RMSE, indicating that your model's predictions are closer to the actual target values and hence more accurate. Comparing Results This is the root mean squared error (RMSE) of the baseline model, where no feature engineering or transformation has been applied. The model uses the original features as they are. An RMSE of 49204.92 indicates the average prediction error of the baseline model on the test data. Baseline RMSE: 49204.92 This RMSE represents the performance of the model after applying the mutual information feature selection method. It's significantly lower than the baseline RMSE, indicating that selecting the top k features based on their mutual information scores has led to improved model performance. Mutual Information RMSE: 39410.99 The RMSE here corresponds to the model's performance after introducing a new categorical feature based on clustering. The RMSE is close to the baseline RMSE, suggesting that the introduction of clustering did not lead to a significant improvement in this case. Clustering RMSE: 47715.30 This RMSE reflects the performance of the model after applying PCA for dimensionality reduction. It's a bit higher than the mutual information RMSE but lower than the baseline RMSE. The model using PCA-transformed features seems to perform moderately well. PCA RMSE: 40055.78 The RMSE here shows the performance of the model when categorical variables are label encoded. The RMSE matches the baseline RMSE, indicating that using label encoded features didn't lead to a noticeable improvement in this case. Label Encoding RMSE: 49204.92 This RMSE represents the performance of the model after applying various mathematical transformations to selected columns. The RMSE is lower than the baseline RMSE, suggesting that these transformations have led to improved model performance. Mathematical Transformations RMSE: 47143.21 In summary: seems to be the most effective feature selection method among the methods tried, as it significantly reduced the RMSE. Mutual Information and both resulted in improved model performance compared to the baseline. PCA Mathematical Transformations did not show a significant improvement in this particular scenario. Clustering Keep in mind that the actual RMSE values and their interpretation depend on various factors such as the dataset, the complexity of the model, and the nature of the target variable. The goal is to experiment with different feature engineering methods and select the one that leads to the best performance on unseen data.

Stories, News & Mentions about DOTLAS

Dotlas’s Stories on HackerNoon