Get the most recent info and news about Threadbox on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

Threadbox.com is the best of email, IM, and the social web, combined for smarter workgroup communication. A single web page tracks, organizes, and shares everything related to the conversation - messages, documents, links, comments, people and more. Unlike email, threads are organized, easy to follow, and always in sync. Follow a thread like email, checking when it's best for you. Or view and send updates in real time - like IM. Need a quick team decision? Have a group chat, right inside the thread. You can share documents, images, links, and media instantly. It's all captured, organized, and shared with all of the participants, automatically.

Threadbox

Every June, tech bros, pension portfolio managers and Twitter threadbois the world over turn their eyes to Santa Clara as hosts its annual Worldwide Developer Conference (WWDC, or ‘dub-dub’). Apple Among other things, dub-dub is best known as being the birthing ground of almost all of Apple’s suite of ubiquitous consumer technology. The original iPhone, iPad, Apple Watch, and Apple TV and subsequent upgrades have all been launched at the event. This year, this stable of consumer devices is joined by an entirely new product for the first time since 2015. That new product is the VisionPro. There is nothing I can tell you about the VisionPro or Apple’s vision for the future of extended reality that a quick browse of , or the internet at large cannot. The point of this article is instead to try and observe: Twitter LinkedIn how we got here; why people are building computers for your face and the many divergent paths this may lead us on. Content Overview XR: A Brief History Headsets Today: Code New Worlds The Next Frontier for Spatial Computers Requests for Startups XR: A Brief History For the uninitiated, XR refers to - a catch-all blend of (AR, e.g Pokemon Go), virtual reality (VR, e.g. Oculus Rift), and ‘capital R’ Reality (e.g. Google Street View). Mixed reality is another term that exists largely on the same dimension as AR and thus will mostly be ignored here. extended reality augmented reality The modern vision for XR begins around 1935. This was the year in which published . In this short story, protagonist Dan Burke is disillusioned with the reality in which he finds himself. To remedy this, he puts on a pair of glasses that transport him to a world of eternal youth and happiness. The only catch is that he obeys its rules (_T&C_s, in modern parlance). Stanley Weinbaum Pygmalion’s Spectacles Fast forward a few decades and we begin to see the first builds of XR in real environments. This begins with Morton Heilig’s ‘ ’ (excluded from the timeline below), a sort of 4D cinema experience that provided the viewer with smells and a vibrating chair in addition to the sounds & sight of the film. Sensorama Also excluded from the timeline below is the first military application of XR. In 1961, became the first headset with motion-tracking technology. Philco Headsight Though it has since been discontinued, the Oculus Rift was the first headset to gain real commercial appeal. Following Oculus, many of big tech’s household names joined the action. Curiously, most had different visions for who the primary customer base for these headsets would be. The dominant expected use case for most was in immersive entertainment applications, more specifically in gaming. This was the path followed by Oculus, Sony through their , and with the Meta Quest line of products. PlayStation VR Meta Others chose to focus predominantly on enterprise. Early pioneers in this segment included and These early movers focused their aims on things like enterprise collaboration platforms (much like some of the metaverse plays described below), emergency response training, and medical education. HTC Vive Magic Leap. Then, the hype cycle. Corporations the world over began to envision what it would look like if we took our lived environment but made it *virtual*. The vision was essentially , but this time with less porn and more corporate work stuff. metaverse SecondLife In the wake of COVID-19, the rise of remote work created a supposed need for collaborative digital environments. joined the party with its own industrial metaverse. Accenture launched a ‘metaverse services’ division. Facebook changed its name to Meta and began spending $1bn per month on this vision, culminating in this brilliant . Microsoft presentation I have had fun panning the metaverse, but the hype cycle around it was an important moment in raising awareness for the arrival of what Apple would refer to this week as spatial computing. As this space evolves, there is every chance that many of these ideas come back into vogue. Other use cases for commercial XR varied from the practical to the benign. Some of them are beginning to become commonplace in popular web back-ends. You take your pick as to which fits which description from the below: Virtual tourism Virtual home inspections Virtual clothing fitting (If you’re going to watch any of these videos, do yourself a favour and make it this one) Corporate training & education Surgical simulations Product design & prototyping Disney-fied sporting telecasts. Most of the existing devices listed above tried to achieve some sort of balance with the use cases they were targeting. The most successful to date by sales, the Meta Quest, did so by positioning hard as a gaming device. Normally, this would be a lesson. However, if anyone is to buck this trend it would be Apple. Apple has a broad suite of already ubiquitous products (1.6bn active iPhones, to name one) that can and will serve as secondary endpoints for VisionPro applications. While Apple’s inaugural demos for the VisionPro were all hosted within working environments, it is more than likely that it will end up being a general-purpose VR. I foresee its role as being a form of spatial augmentation for Apple’s existing ecosystem of products (e.g Maps, FaceTime, Siri et al) as its vision for computers shifts modalities. More on other potential applications later in this piece. Headsets Today: Code New Worlds Spatial Computing Rather than ride the wave of today’s tech buzzwords, Apple used its informational market-making power to stamp authority on a new one at WWDC. https://twitter.com/tim_cook/status/1665806600261763072?embedable=true While it sounds intuitive enough on the surface, let’s delve a little bit deeper into what this term may actually convey going forward. Besides the obvious branding benefits of coining a buzz phrase, the use of the term ‘computing’ feels extremely deliberate. It represents a logical next step from previous eras of ‘desktop computing’ and ‘mobile computing’. Just as Apple has done with the previous generations of personal computers, it will aim to make this concept and terminology ubiquitous. To quote Tim Cook from WWDC: “In the future, you’ll wonder how you led your life without augmented reality” Besides the change in hardware, how is spatial computing functionally different from its predecessors? Arguably the most important immediate shift will be from working in 2D to working in 3D. Tasks such as visualization, workspace navigation (i.e. clicking and moving things), and collaboration will all be enabled in ways not seen under previous modalities. In the longer term, and are more likely to prove the groundbreaking features of spatial computing. contextual awareness interconnectivity refers to the ability of the computer to; Contextual awareness i) adapt existing recommendations and notifications to a user’s sensory context (i.e recommendations based on location and movement patterns, automatic changes to screen presentations based on user habits, etc) and ii) provide customized (possibly agentic) assistance to users as they complete tasks in extended reality. refers to the user’s ability to extend extended reality beyond the device. Beyond just working and collaborating on holographic interfaces, users can control and interact with other devices in their environment. To take a boring example, users can change smart home settings from within the device. Interconnectivity In the longer run, interconnectivity may see Siri’s role as a virtual assistant evolve from voice commanded search aggregator to a living assistant that can help complete tasks from within the device based on the user’s previous habits and preferences. As the baseline capabilities of autonomous agents advance, the combination of Apple’s distribution network and the familiarity of Siri may make it the killer technology for bringing personal agents to the world en masse. Information Presentation As mentioned above, Apple’s demos of the VisionPro focused almost exclusively on working environments. Make no mistake, this first generation is very much geared toward information workers. How will they use it? Firstly, there is obvious and trodden ground. Apple will re-do and re-design all the enterprise XR solutions we have seen before. They will provide resources for institutions to train their employees, students, and leaders. They will provide a bevy of try-on solutions, gaming applications, and cloud TV solutions that we have seen before. One relatively novel example that was showcased as part of the launch was the ‘Minority Report’ style computer interfaces. Instead of having to boot up a physical device every time you want to work on a file, access it from anywhere in the cloud in an instant. But how does this make the core experience of personal computing itself different? Some early ideas: at any given point in time Unlimited monitors with information work (speech, physical movement etc) Multimodal forms of interacting Intelligent browser navigation through sensory (e.g eye) tracking Entirely automatically tailored to user preference custom 3D screen presentations (boring but important) 3D data visualization Each of these will present its own universe of custom ideas that will take the place of extensions and app stores before it. AppleLM This tweet from founder Sulaiman Omar feels prescient. Apple does indeed have a history of sacrificing first-mover advantage for the sake of developing a better product over a longer timeframe. The VisionPro is a perfect example of this, being released 13 years after the first commercially available Oculus Rift. Cognosys AI Integrating language models into its headsets will open up a number of possibilities for Apple that aren’t available to incumbents without their own proprietary LLMs (note: Meta’s LlaMa may put them in the same boat - Balaji Srinivasan has published a highlighting their potential role in the spatial computing race). great thread https://twitter.com/balajis/status/1666506719084843009?embedable=true&thread=true Among the possibilities that this may usher in: trained on users’ real-life habits. Virtual gaming experiences Immediate through the facial ‘browser’. queries built into the headset and trained on user habits. Agentic virtual assistants that recommend actions at a point in time. Spatial recommendation engines Real-time translation. allowing users to re-arrange objects in their space. Environmental augmentation that plays out immediately as the user speaks it. Generative content creation But could it be done more elegantly… The Next Frontier for Spatial Computers Many a commentator proclaimed the death of Google Glass upon seeing it. Who would want to wear a computer on the face? Why would you opt into looking a bit like a dork? Many have noted that not a single Apple executive even tried a VisionPro on as part of the launch. There has to be a better reason for strapping one on than “because everyone else is”. The key question in the long run then becomes: Why would anyone want to strap an XR computer to their face when you can have one inside your brain? The classic counter to this would be the idea of ‘ ’. When everyone is wearing an XR mask, we can reproject photons to make it look as though no one is. Because reprojection involves creating entirely new photons altogether, people can even make themselves look entirely different from what they actually do without the mask. This would become a killer app for catfishing. reprojection The techno-optimist’s concept of living in augmented reality is akin to living in a perpetual lucid dream. Sleep mask when you go to bed, extended reality mask when you get out of it. Examine exhibit A below: The current paradigm of XR relies on these exoskeletal aids for users to navigate alternative realities. The reality is that we are already so close to no longer needing to depend on these aids. Enter the brain-machine interface. BMIs In the long run, the envisioned reality is likely to trump any form of mixed reality that exists today. I use the term envisioned reality here to describe any form of environmental alteration that is; a) completely customizable to the user’s requirements or demands and b) requires no wearable aids. The most obvious conduit for envisioned reality today is (BMIs). For a longer primer, I cannot recommend Tim Urban’s piece on ‘ ’ from as far back as 2017. brain-machine interfaces Wizard hats for the brain BMIs offer many of the same ‘spatial computation’ benefits as tools like the VisionPro but without the need for strapping a weighty device to the face. existing prototypes for the first commercial BMIs involve invasive implants that, understandably, make many uncomfortable (even if the reward is superhuman intelligence and memory). What about when these become less invasive, more accessible, and thus more palatable to the human public? Neuralink’s Extended reality abounds. Requests for Startups Mentions of batteries were oddly absent from Apple’s inaugural launch of the VisionPro. It is less obvious for ‘in the moment’ demos and usage than other design features like interfaces, compute, and weight (as referenced in ) but equally important when used day-in-day-out. Better battery systems for today’s XR devices. this thread by Kyle Samani Not since first dates moved from cafés to online chat rooms has the core wiring of how humans interact with one another been so fundamentally rocked. How can behavior be moderated in a world where people can make themselves appear to be anything? (note: Apple already seems to be working on some form of protocol which may protect against this). What kind of content becomes more pervasive when the range of presentation methods expands so broadly? How do people come together online? Social infrastructure for an XR world. proof-of-identity SecondLife, Minecraft, and Roblox all created huger-than-expected businesses off the back of secondary marketplaces for digital assets. These marketplaces were enabled by creative tools baked into the very fabric of the platforms themselves. Arm the Rebels: Creative Tools for XR. While there are many developers today already working on designs for new experiences etc for headsets and metaverse-style projects, how can we make this process more accessible for the layman (more specifically the 8-year Roblocker) to create worlds of their own imagination? What will be the second coming of Minecraft for the augmented reality or ‘envisioned reality’ paradigm? A large part of the appeal of virtual/augmented/envisioned reality is the idea of living a different life from the one you have now. Much like Dan Burke in anyone will now be able to optimize their lived experiences within virtual worlds. Designs for Lived Experiences (i.e Qualia). Pygmalion’s Spectacles, In the event that such technologies do become ubiquitous, this creates a giant universal market for synthetic instances of subjective experience. The design scope is theoretically infinite. How many ways can you experience pure ecstasy? Nostalgia? The pool of rewards for those who can bring these kinds of experiences to the world through some form of extended reality are enormous. qualia - Also underappreciated may be the development of tools for users to capture or develop these qualia themselves. Is there a market for the ‘recording’ of one’s own lived experiences to share with others? How can people engineer new experiences to take to market and compose atop other open-source qualia? How can we provide guarantees of privacy for personal qualia if they begin to be used in recommendation engines? For the curious, check out the to learn more. Qualia Research Institute One thing that XR still has in common with other computing paradigms before it is its top-down nature. Users are at the whims of Apple for how they want to interact with their phones. If they want to protest this, they can select another one of the depressingly finite options on the market. To date, composability and customisability in hardware have been extremely limited. Open-Source Hardware & XR Software Marketplaces. But what if we were to open-source the development of headsets themselves through decentralized labs? Active participants could work to modify and iterate upon different hardware specifications to their preference. Modular designs could allow for customization at the layman's level. Just as importantly, these open-source labs would have no incentive to create closed-loop systems for software development. Developers could experiment with, ship, and deploy code that could be available to all headsets at the rate that they can build it. Such open-source development would represent a step change in the way people interact with both software and hardware. For some inspiration, check out the great work the team at is already doing. Auki Labs Linked to the point above, the top-down software ecosystem of today means that incentives built into software tend to be warped. Rather than going all out on user utility, market incentives corner developers into building sub-optimal applications that aim to maximize metrics like ‘average session time’ or ‘clickthrough rates’. Private User Agents. The opportunity that open-source development studios would have to outcompete legacy tech companies by building in private, natural language user agents that act as functional assistants to their users is immense. Also published here. The lead image for this article was generated by HackerNoon's AI Image Generator via the prompt "Augmented reality headsets"

Facial Computing: A Brief History, and the Promising Future, of Personal XR

ফেসিয়াল কম্পিউটিং: একটি সংক্ষিপ্ত ইতিহাস, এবং ব্যক্তিগত XR এর প্রতিশ্রুতিশীল ভবিষ্যত

Facial Computing: Eine kurze Geschichte und die vielversprechende Zukunft von Personal XR

Cómputo facial: una breve historia y el futuro prometedor de Personal XR

L'informatique faciale : une brève histoire et l'avenir prometteur du Personal XR

फेशियल कम्प्यूटिंग: ए ब्रीफ हिस्ट्री, एंड द प्रॉमिसिंग फ्यूचर, ऑफ पर्सनल एक्सआर

フェイシャル コンピューティング: パーソナル XR の簡単な歴史と有望な未来

얼굴 컴퓨팅: 개인용 XR의 간략한 역사와 유망한 미래

Computação facial: uma breve história e o futuro promissor do Personal XR

Yüz Bilişimi: Kişisel XR'nin Kısa Tarihi ve Umut Veren Geleceği

Điện toán khuôn mặt: Lịch sử tóm tắt và Tương lai đầy hứa hẹn của Personal XR

面部计算：个人 XR 的简史和充满希望的未来

Learn Multithreading in Python Like A Pro Using These 30 Questions

Mastering Scheduling and Tax Execution in Spring Boot

Requests are undeniably unmissable for anyone willing to start in the Python ecosystem. Yet, the community is broadly unaware of how sad the situation has become. The Incident Due to dramatic circumstances, it was partially locked and put in "maintenance mode" only, meaning it would no longer evolve (features speaking). Some of you may have heard that more than six years ago, Requests was working toward a major version, namely the V3. There was even r with a substantial amount, around $30k. a completed fundraise At the time, there was a plan to bring Async and HTTP/2 support (most notably). You'll guess that none of the exposed promises ever landed. Impacts This incident led to several impacts. Maintainers felt too much pressure to make steps toward a major; some of them said: . "We just can't deliver the promises, and it's going hard to explain why" All the back pressure landed at urllib3. Abruptly started to refuse any changes, saying: "Requests is feature complete, and it's popularity makes it nearly impossible to fix bugs that may be considered feature to this point." Newer projects started ditching HTTP client Requests by learning to use a new one. Some large-scale projects bound to Requests did not migrate because of the potentially high cost of doing it. Let's be clear, this article is not about saying "Requests maintainers are bad" but rather saying we don't agree, and everyone is entitled to their choices. For many years now, Requests have been frozen. Being left in a vegetative state and not evolving, from using more advanced features. this blocked millions of developers The Journey Into Waking Up Initially, like many other reasonable people, I said to myself "Let's wait, Requests will eventually unblock the situation!" Yet, the years are still passing, and nothing. Let's realize something, Yet, Requests is unable to do so. Internet Explorer 11 just celebrated its tenth anniversary last year, and you’ll never guess that it has support for HTTP/2 integrated! I said to myself: "Why are we waiting for this indefinitely? They don't owe us anything! There's a reasonable chance we are waiting... trapped in a circular reference… Let's act." Then, I tried to get a firm grip on the urllib3 base code, contributing and until I was ready to kick things up with that would have put urllib3 far ahead. Without any breaking changes. I was delusional. This was a bit of a shock, but six months passed between my initial kick-off and , and here's why in a nutshell: this there a proof of concept my formal give-up Every bit of idea brought to the table was dismissed, and never seriously analyzed. This felt as good as taking the coldest wind on your face while exiting your warm home. It's the hard truth of the OSS; it's not a democracy, and it's not ruled by pure logic, but by humans. Not far later on, urllib3 called out to help with HTTP/2 support (only) and claimed around $40k to be able to deliver. They probably meant well, but[...] a fundraiser This was the last straw for me, knowing the past mistakes, and actually the best incentive to break the cycle. Everything points to another year before having the slightest sign of a into our production pipes. They clearly lack the time to move forward (or incentives? or both?). No worries, as developers, we should share this burden for the greater good. PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n From my perspective (my opinion only), the OSS ecosystem should not fundraise money for a decade-old protocol, and most of the time should deliver prior to asking. Entities raising money with no solid proof of concept; that's rare! This felt like a bit of a hostage situation, but it could only be me seeing it this way. The Solution So many years and various expertise have been brought to Requests and urllib3… Do we throw all of it into the garbage disposal? Engineers should be reasonable scientists and not revisionists. We don't have to reinvent the wheel all over again; HTTP client Requests are established and really pleasant in their usage. We believe that Requests has the most inclusive and developer-friendly interfaces. So, how to propose an attractive continuity for Requests? How hard could this be to (i) bring a true upgrade experience and (ii) keep the compatibility as high as possible? Sometimes, the OSS ecosystem acts like it is trying to land a rover on the Moon, let's be more realistic and bring our vision down to firm ground. There's a chance offered to us to rehabilitate Requests (and urllib3) and give a fresh breath of air to us, the developers. Without having to learn something as basic as handling an HTTP client. So, the work started, without looking back. Just around my intent to give up on urllib3 evolution, we proposed a fork of both urllib3 and Requests, namely and . urllib3-future Niquests They are (truly) drop-in replacements, at the time of writing, those forks are six months old! And the whole work around this is roughly a year old. We listened to what the community needed by... triaging all the issues on major repositories (Requests, urllib3, and dependants of them) so that the upgrade would be truly useful. Are you seeing it coming? Great! against ! Let's look at the feature table comparison requests, httpx, and aiohttp Feature niquests requests httpx aiohttp HTTP/1.1 ✅ ✅ ✅ ✅ HTTP/2 ✅ ❌ ✅ ❌ HTTP/3 over QUIC ✅ ❌ ❌ ❌ Synchronous ✅ ✅ ✅ ❌ Asynchronous ✅ ❌ ✅ ✅ Thread Safe ✅ ✅ ❌ N/A Task Safe ✅ N/A ✅ ✅ OS Trust Store ✅ ❌ ❌ ❌ Multiplexing ✅ ❌ Limited ❌ DNSSEC ✅ ❌ ❌ ❌ Customizable DNS Resolution ✅ ❌ ❌ ✅ DNS over HTTPS ✅ ❌ ❌ ❌ DNS over QUIC ✅ ❌ ❌ ❌ DNS over TLS ✅ ❌ ❌ ❌ Multiple DNS Resolver ✅ ❌ ❌ ❌ Network Fine Tuning & Inspect ✅ ❌ Limited Limited Certificate Revocation Protection ✅ ❌ ❌ ❌ Session Persistence ✅ ✅ ✅ ✅ In-memory Certificate CA & mTLS ✅ ❌ Limited Limited SOCKS 4/5 Proxies ✅ ✅ ✅ ❌ HTTP/HTTPS Proxies ✅ ✅ ✅ ✅ TLS-in-TLS Support ✅ ✅ ✅ ✅ Direct HTTP/3 Negotiation ✅ N/A N/A N/A Happy Eyeballs ✅ ❌ ❌ ✅ Package / SLSA Signed ✅ ❌ ❌ ✅ And you would think... performance..? are bad? against ! Look at the performance comparison them Fetch a thousand requests using 10 tasks or threads, each with a hundred requests using a single pool of 10 connection. Scenario: High-Level APIs Client Average Delay to Complete Notes requests 987 ms ThreadPoolExecutor. HTTP/1.1 httpx 735 ms Asyncio. HTTP/2 niquests 470 ms Asyncio. HTTP/2 Simplified APIs Client Average Delay to Complete Notes requests core 643 ms ThreadPoolExecutor. HTTP/1.1 httpx core 550 ms Asyncio. HTTP/2 aiohttp 220 ms Asyncio. HTTP/1.1 niquests core 210 ms Asyncio. HTTP/2 Did you give up on HTTP/2 due to performance concerns? Think again! open up a wide range of possibilities! Multiplexing and response laziness And everything you are witnessing is in fact "Requests"-extended, and you have nothing to learn again to get started. We kept 98% of the test cases from both Requests and urllib3, then extended them to cover newer features. We are looking forward to seeing you at the Niquests repository! Let's say it's a promotional offer, get everything and more for $̶4̶0̶k̶  $0! #1 You depend on Requests? Discover instead. Niquests Drop-in replacement, simple patch needed, tiny effort required. #2 You depend on urllib3? Discover instead. urllib3.future Drop-in replacement, no patch required, atom-like sized effort required. By visiting the GitHub repository and documentation, you will be able to learn more about this, and hopefully make a decisive switch!

The Promise That Was Made Six Years Ago for Requests 3 Is Revived

After my latest post about how to build your own RAG and run it locally, today, we're taking it a step further by not only implementing the conversational abilities of large language models but also adding listening and speaking capabilities. The idea is straightforward: we are going to create a voice assistant reminiscent of Jarvis or Friday from the iconic Iron Man movies, which can operate offline on your computer. Since this is an introductory tutorial, I will implement it in Python and keep it simple enough for beginners. Lastly, I will provide some guidance on how to scale the application. Techstack First, you should set up a virtual Python environment. You have several options for this, including pyenv, virtualenv, poetry, and others that serve a similar purpose. Personally, I'll use Poetry for this tutorial due to my personal preferences. Here are several crucial libraries you'll need to install: : For a visually appealing console output. rich : A robust tool for speech-to-text conversion. openai-whisper : A cutting-edge library for text-to-speech synthesis, ensuring high-quality audio output. suno-bark : A straightforward library for interfacing with Large Language Models (LLMs). langchain , , and : Essential for audio recording and playback. sounddevice pyaudio speechrecognition For a detailed list of dependencies, refer to the link . here The most critical component here is the Large Language Model (LLM) backend, for which we will use Ollama. is widely recognized as a popular tool for running and serving LLMs offline. If Ollama is new to you, I recommend checking out my previous article on offline RAG: Basically, you just need to download the Ollama application, pull your preferred model, and run it. Ollama "Build Your Own RAG and Run It Locally: Langchain + Ollama + Streamlit." Architecture Okay, if everything has been set up, let's proceed to the next step. Below is the overall architecture of our application, which fundamentally comprises 3 main components: : Utilizing , we convert spoken language into text. Whisper's training on diverse datasets ensures its proficiency across various languages and dialects. Speech Recognition OpenAI's Whisper : For the conversational capabilities, we'll employ the Langchain interface for the model, which is served using Ollama. This setup promises a seamless and engaging conversational flow. Conversational Chain Llama-2 : The transformation of text to speech is achieved through , a state-of-the-art model from Suno AI, renowned for its lifelike speech production. Speech Synthesizer Bark The workflow is straightforward: record speech, transcribe to text, generate a response using an LLM, and vocalize the response using Bark. Sequence diagram for voice assistant with Whisper, Ollama, and Bark. Implementation The implementation begins with crafting a based on Bark, incorporating methods for synthesizing speech from text and handling longer text inputs seamlessly as follows: TextToSpeechService import nltk
import torch
import warnings
import numpy as np
from transformers import AutoProcessor, BarkModel

warnings.filterwarnings(
    "ignore",
    message="torch.nn.utils.weight_norm is deprecated in favor of torch.nn.utils.parametrizations.weight_norm.",
)


class TextToSpeechService:
    def __init__(self, device: str = "cuda" if torch.cuda.is_available() else "cpu"):
        """
        Initializes the TextToSpeechService class.
        Args:
            device (str, optional): The device to be used for the model, either "cuda" if a GPU is available or "cpu".
            Defaults to "cuda" if available, otherwise "cpu".
        """
        self.device = device
        self.processor = AutoProcessor.from_pretrained("suno/bark-small")
        self.model = BarkModel.from_pretrained("suno/bark-small")
        self.model.to(self.device)

    def synthesize(self, text: str, voice_preset: str = "v2/en_speaker_1"):
        """
        Synthesizes audio from the given text using the specified voice preset.
        Args:
            text (str): The input text to be synthesized.
            voice_preset (str, optional): The voice preset to be used for the synthesis. Defaults to "v2/en_speaker_1".
        Returns:
            tuple: A tuple containing the sample rate and the generated audio array.
        """
        inputs = self.processor(text, voice_preset=voice_preset, return_tensors="pt")
        inputs = {k: v.to(self.device) for k, v in inputs.items()}

        with torch.no_grad():
            audio_array = self.model.generate(**inputs, pad_token_id=10000)

        audio_array = audio_array.cpu().numpy().squeeze()
        sample_rate = self.model.generation_config.sample_rate
        return sample_rate, audio_array

    def long_form_synthesize(self, text: str, voice_preset: str = "v2/en_speaker_1"):
        """
        Synthesizes audio from the given long-form text using the specified voice preset.
        Args:
            text (str): The input text to be synthesized.
            voice_preset (str, optional): The voice preset to be used for the synthesis. Defaults to "v2/en_speaker_1".
        Returns:
            tuple: A tuple containing the sample rate and the generated audio array.
        """
        pieces = []
        sentences = nltk.sent_tokenize(text)
        silence = np.zeros(int(0.25 * self.model.generation_config.sample_rate))

        for sent in sentences:
            sample_rate, audio_array = self.synthesize(sent, voice_preset)
            pieces += [audio_array, silence.copy()]

        return self.model.generation_config.sample_rate, np.concatenate(pieces) : The class takes an optional parameter, which specifies the device to be used for the model (either if a GPU is available, or ). It loads the Bark model and the corresponding processor from the pre-trained model. You can also use the large version by specifying for the model loader. Initialization ( __init__ ) device cuda cpu suno/bark-small suno/bark : This method takes a input and a parameter, which specifies the voice to be used for the synthesis. You can check out other value . It uses the to prepare the input text and the voice preset, and then generates the audio array using the method. The generated audio array is converted to a NumPy array and the sample rate is returned along with the audio array. Synthesize ( synthesize ) text voice_preset voice_preset here processor model.generate() : This method is used for synthesizing longer text inputs. It first tokenizes the input text into sentences using the function. For each sentence, it calls the method to generate the audio array. It then concatenates the generated audio arrays, with a short silence (0.25 seconds) added between each sentence. Long-form Synthesize ( long_form_synthesize ) nltk.sent_tokenize synthesize Now that we have the set up, we need to prepare the Ollama server for the large language model (LLM) serving. To do this, you'll need to follow these steps: TextToSpeechService : Run the following command to download the latest Llama-2 model from the Ollama repository: Pull the latest Llama-2 model ollama pull llama2 . : If the server is not yet started, execute the following command to start it: . Start the Ollama server ollama serve Once you've completed these steps, your application will be able to use the Ollama server and the Llama-2 model to generate responses to user input. Next, we'll move to the main application logic. First, we need to initialize the following components: : We'll use the Rich library to create a better interactive console for the user within the terminal. Rich Console : We'll initialize a Whisper speech recognition model which is a state-of-the-art open-source speech recognition system developed by OpenAI. We'll use the base English model ( ) for transcribing user input. Whisper Speech-to-Text base.en : We'll initialize a Bark text-to-speech synthesizer instance, which was implemented above. Bark Text-to-Speech : We'll use the built-in from the Langchain library which provides a template for managing the conversational flow. We'll configure it to use the Llama-2 language model with the Ollama backend. Conversational Chain ConversationalChain import time
import threading
import numpy as np
import whisper
import sounddevice as sd
from queue import Queue
from rich.console import Console
from langchain.memory import ConversationBufferMemory
from langchain.chains import ConversationChain
from langchain.prompts import PromptTemplate
from langchain_community.llms import Ollama
from tts import TextToSpeechService

console = Console()
stt = whisper.load_model("base.en")
tts = TextToSpeechService()

template = """
You are a helpful and friendly AI assistant. You are polite, respectful, and aim to provide concise responses of less 
than 20 words.
The conversation transcript is as follows:
{history}
And here is the user's follow-up: {input}
Your response:
"""
PROMPT = PromptTemplate(input_variables=["history", "input"], template=template)
chain = ConversationChain(
    prompt=PROMPT,
    verbose=False,
    memory=ConversationBufferMemory(ai_prefix="Assistant:"),
    llm=Ollama(),
) Now, let's define the necessary functions: : This function runs in a separate thread to capture audio data from the user's microphone using the . The callback function is called whenever new audio data is available, and it puts the data into a for further processing. record_audio sounddevice.RawInputStream data_queue : This function utilizes the Whisper instance to transcribe the audio data from the into text. transcribe data_queue : This function feeds the current conversation context to the Llama-2 language model (via the Langchain ) and retrieves the generated text response. get_llm_response ConversationalChain : This function takes the audio waveform generated by the Bark text-to-speech engine and plays it back to the user using a sound playback library (e.g., ). play_audio sounddevice def record_audio(stop_event, data_queue):
    """
    Captures audio data from the user's microphone and adds it to a queue for further processing.
    Args:
        stop_event (threading.Event): An event that, when set, signals the function to stop recording.
        data_queue (queue.Queue): A queue to which the recorded audio data will be added.
    Returns:
        None
    """
    def callback(indata, frames, time, status):
        if status:
            console.print(status)
        data_queue.put(bytes(indata))

    with sd.RawInputStream(
        samplerate=16000, dtype="int16", channels=1, callback=callback
    ):
        while not stop_event.is_set():
            time.sleep(0.1)

def transcribe(audio_np: np.ndarray) -> str:
    """
    Transcribes the given audio data using the Whisper speech recognition model.
    Args:
        audio_np (numpy.ndarray): The audio data to be transcribed.
    Returns:
        str: The transcribed text.
    """
    result = stt.transcribe(audio_np, fp16=False)  # Set fp16=True if using a GPU
    text = result["text"].strip()
    return text

def get_llm_response(text: str) -> str:
    """
    Generates a response to the given text using the Llama-2 language model.
    Args:
        text (str): The input text to be processed.
    Returns:
        str: The generated response.
    """
    response = chain.predict(input=text)
    if response.startswith("Assistant:"):
        response = response[len("Assistant:") :].strip()
    return response

def play_audio(sample_rate, audio_array):
    """
    Plays the given audio data using the sounddevice library.
    Args:
        sample_rate (int): The sample rate of the audio data.
        audio_array (numpy.ndarray): The audio data to be played.
    Returns:
        None
    """
    sd.play(audio_array, sample_rate)
    sd.wait() Then, we define the main application loop. The main application loop guides the user through the conversational interaction as follows: The user is prompted to press Enter to start recording their input. Once the user presses Enter, the function is called in a separate thread to capture the user's audio input. record_audio When the user presses Enter again to stop the recording, the audio data is transcribed using the function. transcribe The transcribed text is then passed to the function, which generates a response using the Llama-2 language model. get_llm_response The generated response is printed to the console and played back to the user using the function. play_audio if __name__ == "__main__":
    console.print("[cyan]Assistant started! Press Ctrl+C to exit.")

    try:
        while True:
            console.input(
                "Press Enter to start recording, then press Enter again to stop."
            )

            data_queue = Queue()  # type: ignore[var-annotated]
            stop_event = threading.Event()
            recording_thread = threading.Thread(
                target=record_audio,
                args=(stop_event, data_queue),
            )
            recording_thread.start()

            input()
            stop_event.set()
            recording_thread.join()

            audio_data = b"".join(list(data_queue.queue))
            audio_np = (
                np.frombuffer(audio_data, dtype=np.int16).astype(np.float32) / 32768.0
            )

            if audio_np.size > 0:
                with console.status("Transcribing...", spinner="earth"):
                    text = transcribe(audio_np)
                console.print(f"[yellow]You: {text}")

                with console.status("Generating response...", spinner="earth"):
                    response = get_llm_response(text)
                    sample_rate, audio_array = tts.long_form_synthesize(response)

                console.print(f"[cyan]Assistant: {response}")
                play_audio(sample_rate, audio_array)
            else:
                console.print(
                    "[red]No audio recorded. Please ensure your microphone is working."
                )

    except KeyboardInterrupt:
        console.print("\n[red]Exiting...")

    console.print("[blue]Session ended.") Result Once everything is put down together, we can run the application as shown in the video above. The application runs quite slowly on my MacBook because the Bark model is large, even in its smaller version. Therefore, I have slightly sped up the video. For those with a CUDA-enabled computer, it might run faster. Here are the key features of our application: : Users can start and stop recording their voice input, and the assistant responds by playing back the generated audio. Voice-based interaction The assistant maintains the context of the conversation, enabling more coherent and relevant responses. The use of the Llama-2 language model allows the assistant to provide concise and focused responses. Conversational context: For those aiming to elevate this application to a production-ready status, the following enhancements are recommended: : Incorporate optimized versions of the models, such as whisper.cpp, llama.cpp, and bark.cpp, which are designed to boost performance, especially on lower-end computers. Performance Optimization : Implement a system that allows users to customize the bot's persona and prompt, enabling the creation of different types of assistants (e.g., personal, professional, or domain-specific). Customizable Bot Prompts : Develop a user-friendly GUI to enhance the overall user experience, making the application more accessible and visually appealing. Graphical User Interface (GUI) : Expand the application to support multimodal interactions, such as the ability to generate and display images, diagrams, or other visual content in addition to the voice-based responses. Multimodal Capabilities Finally, we have completed our simple voice assistant application, full code can be found at: . This combination of speech recognition, language modeling, and text-to-speech technologies demonstrates how we can build something that sounds difficult but can actually run on your computer. Let's enjoy coding, and don't forget to subscribe to so you don't miss the latest in AI and programming articles. https://github.com/vndee/local-talking-llm my blog Also published here

How to Build Your Own Voice Assistant and Run it Locally Using Whisper + Ollama + Bark

কীভাবে আপনার নিজের ভয়েস সহকারী তৈরি করবেন এবং হুইস্পার + ওল্লামা + বার্ক ব্যবহার করে স্থানীয়ভাবে এটি চালাবেন

So erstellen Sie Ihren eigenen Sprachassistenten und führen ihn lokal mit Whisper + Ollama + Bark aus

Cómo crear su propio asistente de voz y ejecutarlo localmente usando Whisper + Ollama + Bark

Comment créer votre propre assistant vocal et l'exécuter localement à l'aide de Whisper + Ollama + Bark

अपना खुद का वॉयस असिस्टेंट कैसे बनाएं और इसे व्हिस्पर + ओलामा + बार्क का उपयोग करके स्थानीय रूप से कैसे चलाएं

独自の音声アシスタントを構築し、Whisper + Ollama + Bark を使用してローカルで実行する方法

나만의 음성 어시스턴트를 구축하고 Whisper + Ollama + Bark를 사용하여 로컬에서 실행하는 방법

Como construir seu próprio assistente de voz e executá-lo localmente usando Whisper + Ollama + Bark

Как создать собственный голосовой помощник и запустить его локально с помощью Whisper + Ollama + Bark

Kendi Sesli Asistanınızı Oluşturma ve Whisper + Ollama + Bark Kullanarak Yerel Olarak Çalıştırma

Cách xây dựng trợ lý giọng nói của riêng bạn và chạy nó cục bộ bằng Whisper + Ollama + Bark

如何构建您自己的语音助手并使用 Whisper + Ollama + Bark 在本地运行

X Corp. v. Center for Countering Digital Hate, INC. Court Filing, retrieved on March 25, 2024 is part of . You can jump to any part in this filing . This part is 4 of 19. HackerNoon’s Legal PDF Series here B. Rule 12(b)(6) Under Rule 12(b)(6), the Court may dismiss a complaint for failure to state a claim upon which relief may be granted. The Court may base dismissal on either “the lack of a cognizable legal theory or the absence of sufficient facts alleged under a cognizable legal theory.” Godecke v. Kinetic Concepts, Inc., 937 F.3d 1201, 1208 (9th Cir. 2019) (cleaned up). A complaint must plead “sufficient factual matter, accepted as true, to state a claim to relief that is plausible on its face.” Ashcroft v. Iqbal, 556 U.S. 662, 678 (2009) (cleaned up). A claim is plausible “when the plaintiff pleads factual content that allows the court to draw the reasonable inference that the defendant is liable for the misconduct alleged.” Id. “Threadbare recitals of the elements of a cause of action, supported by mere conclusory statements, do not suffice” to survive a 12(b)(6) motion. Id. (citing Bell Atlantic v. Twombly, 550 U.S. 544, 555 (2007)). When evaluating a motion to dismiss, the Court “must presume all factual allegations of the complaint to be true and draw all reasonable inferences in favor of the nonmoving party.” Usher v. City of Los Angeles, 828 F.2d 556, 561 (9th Cir. 1987). “[C]ourts must consider the complaint in its entirety, as well as other sources courts ordinarily examine when ruling on Rule 12(b)(6) motions to dismiss, in particular, documents incorporated into the complaint by reference, and matters of which a court may take judicial notice.” Tellabs, Inc. v. Makor Issues & Rights, Ltd., 551 U.S. 308, 322 (2007). If a court dismisses a complaint for failure to state a claim, it should “freely give leave” to amend “when justice so requires.” Fed. R. Civ. Proc. 15(a)(2). A court may deny leave to amend due to “undue delay, bad faith or dilatory motive on the part of the movant, repeated failure to cure deficiencies by amendment previously allowed, undue prejudice to the opposing party by virtue of allowance of the amendment, [and] futility of amendment.” Leadsinger, Inc. v. BMG Music Pub., 512 F.3d 522, 532 (9th Cir. 2008). Continue Reading . Here About HackerNoon Legal PDF Series: We bring you the most important technical and insightful public domain court case filings. This court case retrieved on March 25, 2024, from is part of the public domain. The court-created documents are works of the federal government, and under copyright law, are automatically placed in the public domain and may be shared without legal restriction. storage.courtlistener

Rule 12(b)(6) and How it Impacts X's Lawsuit Against The Centre for Countering Digital Hate

If I would have to point out one of the most confusing Dispatchers in the world of Kotlin coroutines, that would be the Unconfined dispatchers. Called with Dispatchers.Unconfined when creating a coroutine scope, this a kind of dispatcher that doesn’t really care. The whole idea of the Unconfined dispatcher is simply to pick up a Thread for no other goal than to simply use it. There is no other criteria involved when using a Thread.Coroutines use a system called green threading or also known as user space threading. We can refer to routines as contructs that are designed to gain control of a Thread, but specifically they are managed by the schedulers to run in platform or system threads. By platform threads, we are talking about the threads accessible by the programming language or higher abstraction that we use in our development environment.When we talk about the unconfined dispatchers, it is important to understand what these terms are. And so let’s get clear to what is green threading and user space threading. These terms are used interchangeably. However, there is a very subtle difference: User space threading is used to emphasize that it is the user that schedules the threads. In other words, this is what we refer to when we start threads via our code. Green threading emphasizes on what goes on behind the curtain. Green threads are also called fibers and these are virtual threads that are multiplexed to be used on top of system threads. This is all something very difficult to visualize. But for the case of the Unconfined Dispatchers this is all we need to know about this part. Being aware of the existence of platform threads, virtual threads and system threads and how they work is important because the platform threads will then perform this multiplexing mechanism onto specific system threads and this is why in Kotlin, specifically, we can find dispatchers specifically to be used with the CPU, the Dispatchers.Default, dispatchers specific for IO operations called Dispatchers.IO and dispatchers specific for single threaded UI , such as Dispatchers.Main for other operations as alternatives to the Dispatchers.Unconfined context. Making tests using the Spring Framework we will very easily come across the fact that the Spring Framework uses Dispatchers.Unconfined. When I made it was generally said that Dispatchers.Unconfined were to be used in tests only due to their unpredictability. this video do not rely on a particular platform scheduling system oriented to a specific kind of Thread, it becomes very unpredictable how coroutines launched like this would perform. There were also mentions of the possibility that Dispatchers.Unconfined could eventually block the execution of the application in production code, but to be honest I have never ever seen that, even working so much with the Spring Framework for server-side programming as I currently do with Kotlin. And to top that up they mention precisely that, because of that risk, all nested coroutines form an event-loop. They explain that in their documentation here: Dispatchers.Unconfined https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-unconfined.html?embedable=true But hold on. Why is the Spring Framework using Dispatchers.Unconfined in their implementation instead of other Dispatchers like for example the Dispatchers.IO? And why not test this before anything? Well, first let’s have a look at an issue that was opened on GitHub about this: https://github.com/spring-projects/spring-framework/issues/32032?embedable=true I came across this subject when I was casually looking for something else and I bumped into another issue on StackOverflow: https://stackoverflow.com/questions/77810077/controller-code-using-unconfined-dispatcher-by-default?embedable=true Someone had found out that, for whatever reason, the Dispatchers.Unconfined was being used and so I created a service on my GitHub Repo here to prove that: https://github.com/jesperancinha/jeorg-spring-master-test-drives?embedable=true And this service is called car-parts-reactive and that is located in the root directory of the project: git clone https://github.com/jesperancinha/jeorg-spring-master-test-drives.git
cd car-parts-reactive
make b
make run The method we want to have a look at is this one in the controller CarController: @GetMapping("/break")
suspend fun getBreak(): String {
    println("Current Coroutine Context -> $coroutineContext")
    println("Current Coroutine Job -> ${coroutineContext.job}")
    println("Main Coroutine Context -> ${currentCoroutineContext()}")
    println("Main Coroutine Job -> ${currentCoroutineContext().job}")
    println("Current Thread -> ${Thread.currentThread()}")
    return "break"
} This method will print out a series of useful information to the console about the corroutine that has been called. The suspend function marked with the suspend modifier isn’t itself a coroutine, but it can only be called by coroutines in Kotlin. And that’s what happens when we call the method. This way we get access to the coroutine context and checkout what dispatcher is being used. Important to note is that using the currentCoroutineContext() function, we are accessing the coroutine context via a top level function in the kotlinx coroutines library. Using coroutineContext, we are accessing the getter of a value located in the kotlin standard library itself. They don’t share exactly the same implementation.In the root of the car-parts-reactive module we’ll find file generated-requests.http and if you are using IntellJ, you can easily make requests to the application with the code contents: ###
GET http://localhost:8080/break

###
GET http://localhost:8080/keys

###
GET http://localhost:8080/parts

###
GET http://localhost:8080/parts/list

###
GET http://localhost:8080/parts/suspend

###
GET http://localhost:8080/parts/correct When you have the application running, try to run a simple REST request against endpoint : http://localhost:8080/break There should be a result in the console similar to this: Current Coroutine Context -> [Context2{micrometer.observation=io.micrometer.observation.NoopObservation@52c86bb3, reactor.onDiscard.local=reactor.core.publisher.Operators$$Lambda$1234/0x000000080165d210@7166cb50}, MonoCoroutine{Active}@5474a700, Dispatchers.Unconfined]
Current Coroutine Job -> MonoCoroutine{Active}@5474a700
Main Coroutine Context -> [Context2{micrometer.observation=io.micrometer.observation.NoopObservation@52c86bb3, reactor.onDiscard.local=reactor.core.publisher.Operators$$Lambda$1234/0x000000080165d210@7166cb50}, MonoCoroutine{Active}@5474a700, Dispatchers.Unconfined]
Main Coroutine Job -> MonoCoroutine{Active}@5474a700
Current Thread -> Thread[#50,reactor-http-epoll-2,5,main] We can see in all counts that the dispatcher being used is the Dispatchers.Unconfined. If we go back to the issue that was created on GitHub, we’ll see that the answer to why the spring framework uses Dispatchers.Unconfined turns out to be the best solution for their cases, according to them. So what they mean, is that in spite of these dispatchers appear to be limited and restrictive, they apparently, and after testing are actually a very good fit for a reactive framework.Of course this is great to read because it makes it clear why actually, unconfined dispatchers are used in Spring.But let’s go back to the basics. I made a few quizzes about unconfined dispatchers and what I like about it is that they are still quite mysterious to me. And let’s get one thing clear before we move on. There can be some confusion regarding context and scope. Scope refers to where the coroutine is available and context to how it gets dispatched. There can be confusion about this because both terms are quite intimately connected to each other. For example, we cannot create a scope without a dispatcher and they go hand in hand. A scope allows me to launch coroutines with keywords like async and/or launch, but a context is related to how that coroutine gets assigned a Thread.Let’s make two quizzes. We start with this one in this video: https://www.youtube.com/shorts/7Rv_zcT_yPE?embedable=true In this video I show a simple implementation of some code that makes some calls and these different calls will print the words “master”, “mouse” and “cat” in different orders. The code is available here: https://github.com/jesperancinha/jeorg-kotlin-test-drives?embedable=true For this quiz, the code is available here: git clone https://github.com/jesperancinha/jeorg-kotlin-test-drives.git
cd jeorg-kotlin-coroutines/coroutines-crums-group-1
make b And the code is this one: private suspend fun runUnconfinedCoroutinesTest() {
    logger.infoTextSeparator("Unconfined means that all coroutines will execute concurrently on the starting thread in the scope of the parent thread")
    val job = CoroutineScope(Dispatchers.Unconfined).launch {
        launch {
            logger.infoThread("Cat Before thread ==> ${Thread.currentThread()}")
            delay(100)
            logger.info("This is cat @ ${LocalDateTime.now()}")
            logger.info("Cat context $coroutineContext")
            logger.infoThread("Cat After Resume thread ==> ${Thread.currentThread()}")
        }
        launch {
            logger.info("This is mouse @ ${LocalDateTime.now()}")
            logger.info("Mouse context $coroutineContext")
        }
        logger.info("This is master @ ${LocalDateTime.now()}")
        logger.info("Master context $coroutineContext")
        logger.infoThread(Thread.currentThread())

    }
    job.join()
} When we run this code, many things can happen, but one thing we can predict is how the logs are going to work out in the console. One thing about unconfined dispatchers is that they use the same context, same dispatchers, on the first sub coroutines. If these coroutines need to suspend, they may or may not return to the original Thread they were sharing. In the code above they will most likely not return to the original thread. Nesting in coroutines follows some very important rules as mentioned in their documentation: kotlinlang.org So this is the same as saying that the initial call executes in the same continuation in the same call-frame. This also means that the first 3 launches will run concurrently. The launch that logs a cat does this too, but beacause it has a delay, it will resume either in the same Thread or another thread. So this means that Mouse and Master will be the first things to see in the log and Cat will be the last log in the console. We can see all of this in this log sample: This is master @ 2024-03-21T18:49:00.661284475
Master context [StandaloneCoroutine{Active}@6ce139a4, Dispatchers.Unconfined]
Thread[#1,main,5,main]
Master After Resume thread ==> Thread[#1,main,5,main]
Cat Before thread ==> Thread[#1,main,5,main]
This is mouse @ 2024-03-21T18:49:00.703936722
Mouse context [StandaloneCoroutine{Active}@433d61fb, Dispatchers.Unconfined]
Mouse After Resume thread ==> Thread[#1,main,5,main]
This is cat @ 2024-03-21T18:49:00.801217959
Cat context [StandaloneCoroutine{Active}@52c68062, Dispatchers.Unconfined]
Cat After Resume thread ==> Thread[#28,kotlinx.coroutines.DefaultExecutor,5,main] And this explains the answer I give to this quiz on this video: https://www.youtube.com/shorts/MGxvkW0t8QM?embedable=true But moving on. I have another quiz for you and it is this one on this video: https://www.youtube.com/shorts/LEyEJkJA3xE?feature=share&embedable=true The code for this quiz is available here: git clone https://github.com/jesperancinha/jeorg-kotlin-test-drives.git
cd jeorg-kotlin-coroutines/jeorg-kotlin-coroutines-unconfined
make b And this is the class we are discussing: object UnconfinedCoroutineLauncher {
    private val logger = object {
        fun info(logText: Any?) = ConsolerizerComposer.out().yellow(logText)
        fun infoBefore(logText: Any?) = ConsolerizerComposer.out().green(logText)
        fun infoAfter(logText: Any?) = ConsolerizerComposer.out().red(logText)
        fun infoTitle(logText: String) = ConsolerizerComposer.outSpace()
            .cyan(ConsolerizerComposer.title(logText))
    }

    @JvmStatic
    fun main(args: Array<String> = emptyArray()) = runBlocking {
        logger.infoTitle(
            "Unconfined Coroutines scope tests / There are currently ${
                Runtime.getRuntime().availableProcessors()
            } CPU's available"
        )
        CoroutineScope(Dispatchers.Unconfined).launch {
            launch {
                logger.info("Running on context $coroutineContext")
                logger.infoBefore("Siamese Cat is launching on Thread-${Thread.currentThread().name} with id ${Thread.currentThread().threadId()}")
                delay(1)
                logger.infoAfter("Siamese Cat just ran on Thread-${Thread.currentThread().name} with id ${Thread.currentThread().threadId()}")
            }
            launch {
                logger.info("Running on context $coroutineContext")
                logger.infoBefore("Mouse is launching on Thread-${Thread.currentThread().name} with id ${Thread.currentThread().threadId()}")
                delay(20)
                logger.infoAfter("Mouse just ran on Thread-${Thread.currentThread().name} with id ${Thread.currentThread().threadId()}")
            }
            launch {
                logger.info("Running on context $coroutineContext")
                logger.infoBefore("Maine Coon is launching on Thread-${Thread.currentThread().name} with id ${Thread.currentThread().threadId()}")
                delay(30)
                logger.infoAfter("Maine Coon is just ran on Thread-${Thread.currentThread().name} with id ${Thread.currentThread().threadId()}")
            }
        }.join()
    }
} The best way to pick one or more options form the one I give in the quiz, is to look at the order the animals start. In this case the can all start in random orders. However they will all share the same thread because they are just being launched from that call stack. This way, the Siamese Cat, the Mouse and the Main Coon need to share he same Thread number on those calls and so that eliminates already options B and D. The Siamese cat only has a delay of 1 ms and that means that, although seemingly unlikely, option A is still very possible. It is actually very likely that this coroutine starts and ends before any of the other two start. 1 ms is just too much of a short time. We have to consider that possibility as well. And this is why the right answers to this questions are A and C, and this is only one example of that result: Running on context [StandaloneCoroutine{Active}@4de5031f, Dispatchers.Unconfined]
Siamese Cat is launching on Thread-main with id 1
Siamese Cat just ran on Thread-main with id 1
Running on context [StandaloneCoroutine{Active}@691a7f8f, Dispatchers.Unconfined]
Mouse is launching on Thread-main with id 1
Running on context [StandaloneCoroutine{Active}@50a7bc6e, Dispatchers.Unconfined]
Maine Coon is launching on Thread-main with id 1
Mouse just ran on Thread-kotlinx.coroutines.DefaultExecutor with id 21
Maine Coon is just ran on Thread-kotlinx.coroutines.DefaultExecutor with id 21 Find the video answer to this . quiz here My idea with this article is to show some interesting cases I found with . You may find these issues while working with Kotlin coroutines and very likely a lot of people will tell you just to use the Default dispatcher. That can lead to a miscommunication problem. There is such a thing called , which is specific for CPU operations. When people say that when in doubt, use the default dispatcher, they probably do not mean the context. Make sure to double check, because, especially if you are working in server-side programming with certain frameworks, what they probably mean is that you shouldn’t use any specific dispatchers. Most of frameworks like the Spring Framework already offer support to use suspend functions and so your concern as a developer is for the most cases only to implement the suspend functions and not use any specific context. Use the default of course and that is, in Spring, the . Have a good one everyone! Dispatchers.Unconfined Dispatchers.Default Dispatchers.Default Dispatchers.Unconfined Also published . here

They Said to Use the Default Dispatchers but I Found it Was Unconfined

Race conditions are the bitter truth of concurrent applications. But why would anyone call a race condition nice? The fascination of this race condition lies in its subtlety on the outside, countered by the intricate complexities under the hood. It is probably the reason why people call it a . Nice Race Condition Despite the disagreement on the naming, this race condition serves as an amazing case study of how seemingly innocent programs can fail in concurrent settings. In this blog, we will dissect this race condition. We will understand how and why this race condition occurs and what are some ways to eliminate this race condition in our applications. Let us Start… Consider the following Java code: public class NiceRC {

    public static void main(String[] args) {
        final Map<Object, Object> unsafeMap = new HashMap<>();
        final List<Thread> threads = new ArrayList<>();
        for (int i = 0; i < 5; ++i) {
            threads.add(new Thread(() -> {
                for (int j = 0; j < 100000; ++j) {
                    unsafeMap.put(new Object(), new Object());
                }
            }));
        }
        threads.forEach(Thread::start);
        threads.forEach(thread -> {
            try {
                thread.join();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        });
        System.out.println("Size of unsafeMap: " + unsafeMap.size());
    }

} I strongly suggest giving the code above a thorough read before proceeding. In this code, we start 5 threads and each thread adds around 100k entries to a shared map called . unsafeMap If you were to run this code multiple times, it won’t be long before you come across an instance where the code fails to terminate. It goes in some sort of infinite loop! But why? No section of our code could cause an infinite loop. Turns out there is a race condition at play. But before talking about the race condition, let us take a refresher on Java’s HashMap. This will be necessary to understand what is happening under the hood! HashMap’s Internal HashMap is a hash table-based implementation for a Map. A hash function is used to map a key to a bucket/slot from which the desired value can be found by traversing a Linked List. Following is the definition of the Entry class used to represent nodes of the Linked List inside a HashMap, static class Entry<K,V> implements Map.Entry<K,V> 
{
        final K key;
        V value;
        Entry<K,V> next;
        int hash;
} When the number of key-value pairs kept in a HashMap exceeds a threshold, it needs to resize its internal array and re-hash all the elements. This process is called or . rehashing resizing Here is a part of the code that performs this resizing: // Transfer method in java.util.HashMap -
  // called to resize the hashmap
  
  for (int j = 0; j < src.length; j++) {
    Entry e = src[j];
    if (e != null) {
      src[j] = null;
      do {
        Entry next = e.next;
        int i = indexFor(e.hash, newCapacity);
        e.next = newTable[i];
        newTable[i] = e;
        e = next;
     } while (e != null);
   }
 } Let us take an example to understand how this code works. rehashing Consider a map with two buckets and three elements. At this point, a resizing operation takes place to increase the number of buckets to 4. All three map elements will be rehashed. Keys A and B will be hashed to Bucket 3, while key C will be hashed to Bucket 1. After one iteration of the loop, the following will be the state of this map. Node A is plucked from the original bucket and is added to the front of the new bucket. After yet another iteration of this loop, we will have node B added to the head of Bucket 3. With one more iteration, the rehashing process is completed. This is the final state of the map. To the Race Condition… With all that knowledge of HashMap’s internals, we are all set to uncover the race condition that plagued our class. NiceRC Consider again the following map with two buckets and three elements. But this time, two threads attempt to perform the resizing/rehashing operation concurrently. Let us call these threads ‘Thread 1’ and ‘Thread 2’. Now we will see how interleaving of two resizing operations could lead to an infinite loop. Consider the first thread entering the resizing loop first. It initializes its and variables. Then for some reason, thread 1 stops. It gets kicked out of the CPU for any reason possible. e next ( ) For clarity variables for the thread are suffixed with thread number // Transfer method in java.util.HashMap -
  // called to resize the hashmap
  
  for (int j = 0; j < src.length; j++) {
    Entry e = src[j];
    if (e != null) {
      src[j] = null;
      do {
        Entry next = e.next;
        // THREAD 1 STOPS HERE
        int i = indexFor(e.hash, newCapacity);
        e.next = newTable[i];
        newTable[i] = e;
        e = next;
     } while (e != null);
   }
 } This is the state of thread 1 local variables at the time of eviction. Assume that while thread 1 is evicted out of CPU, thread 2 does not and completes the resizing process. This is now the new state of our map. Note that the resizing process does not clone Entry objects, so and still refer to valid objects in the new state. However, the next relation has now changed, in a wrong way. e1 next1 At this point, thread 1 is scheduled again and it starts its execution from where it left. It adds node A to the front of the new bucket. In the next iteration, variables and are updated to point to node B and node A respectively. With this, we add node B to the front of the new bucket. e1 next1 Next, we update the variable to pointer to node A. Since node A has no next node, now pointer to null. e1 next1 Our algorithm (again) tries to add node A to the front of the new bucket. With that the structure of our map is successfully corrupted! Any future operation that tries to add or get element from this corrupted bucket would end up in an infinite loop. This is exactly what happened with our class. NiceRC Utilizing non-thread-safe classes in a concurrent environment poses inherent risks. Even if your system can handle race conditions, there is always the possibility of unexpected interleaving of operations that could lead to system failures. Therefore, it is strongly recommended to employ thread-safe implementations for data structures when working in a multi-threaded environment. Also published here.

Why Would Anyone Call a Race Condition Nice?

THREADBOX

threadbox.com

1-10 employees

Since 2009

COMPANY RANKING

THREADBOX

EVERGREEN INDEX #10355

Representing this company? Click here to claim and customize this page!

Threadbox WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

THREADBOX

threadbox.com

1-10 employees

Since 2009

COMPANY RANKING

THREADBOX

EVERGREEN INDEX #10355

RELATED COMPANIESRANK

STORIES ABOUT THREADBOX

Representing this company? Click here to claim and customize this page!

Threadbox WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!