Microsec

microsec

Deep Learning: Edge Devices With mmWave FMCW Radars Part 1 - Signal Processing

ডিপ লার্নিং: mmWave FMCW রাডার সহ এজ ডিভাইস পার্ট 1 - সিগন্যাল প্রসেসিং

Deep Learning: Edge-Geräte mit mmWave-FMCW-Radaren, Teil 1 – Signalverarbeitung

Aprendizaje profundo: dispositivos perimetrales con radares mmWave FMCW Parte 1: Procesamiento de señales

Apprentissage profond : appareils Edge avec radars mmWave FMCW, partie 1 - Traitement du signal

डीप लर्निंग: एमएमवेव एफएमसीडब्लू रडार के साथ एज डिवाइस भाग 1 - सिग्नल प्रोसेसिंग

ディープラーニング: mmWave FMCW レーダーを搭載したエッジデバイス パート 1 - 信号処理

딥 러닝: mmWave FMCW 레이더를 사용한 엣지 장치 1부 - 신호 처리

Aprendizado profundo: dispositivos de borda com radares FMCW mmWave Parte 1 - Processamento de sinal

Глубокое обучение: периферийные устройства с радарами mmWave FMCW. Часть 1. Обработка сигналов

Derin Öğrenme: mmWave FMCW Radarlarına Sahip Uç Cihazlar Bölüm 1 - Sinyal İşleme

Học sâu: Thiết bị biên có radar mmWave FMCW Phần 1 - Xử lý tín hiệu

深度学习：配备毫米波 FMCW 雷达的边缘设备第 1 部分 - 信号处理

How Useful are Computational Models to Traders in Their Decision Making?

In previous articles, I explained how we could . But we started this exploration into MongoDB, covering a lot of the basics, and I wanted to start looking into more interesting aspects of how we use these things together. As I was putting video content together about all of these topics, one thing that caught my attention was the generic methods vs. methods that operated on BsonDocument, and I was curious if the performance was any different. So, to start, I figured we’d look at C# MongoDB insert benchmarks and see if any interesting patterns stand out. use C# and MongoDB to get records inserted Considerations For These C# MongoDB Benchmarks I was chatting with on Twitter the other day about benchmarks that he often posts on social media. It turns out there was another discussion floating around the Twitter-verse, where sync vs. async calls to databases were being debated. The proposal was that for very fast DB queries, the async variations are indeed slower. David Callan Of course, this got the gears turning. I have been writing about and creating videos for MongoDB in C# all week and hinting at performance benchmarks coming. This coupled with my conversation with Dave AND the conversation I saw on Twitter made me think that I had to give this more attention. I speculated it might have something to do with Task vs ValueTask, but I didn’t have a lot of grounds for that. However, I originally wanted to look into these benchmarks because I was curious if using BsonDocument compared to a dedicated DTO (whether it was a struct, class, or record variation) would have different performance characteristics. That meant that I would want to ensure I covered a matrix across: Synchronous Asynchronous Asynchronous with ValueTask Struct Class Record Struct Record Class BsonDocument And with that, I set off to go write some super simple benchmarks! Let’s check them out. The C# MongoDB Insert Benchmarking Code As with all benchmarking we do in C#, is our go-to tool! Make sure you start by installing the BenchmarkDotNet NuGet package. We’ll use this to ensure consistent setup, warmup, runs, and reporting for our MongoDB benchmarks. BenchmarkDotNet Since we’re trying to reduce as many possible external factors as we can for these benchmarks, we’re going to use Testcontainers to run an instance of MongoDB in a Docker container. Of course, interacting with anything outside of our code directly can lead to inconsistencies and more room for error to show up in our results. However, this should help minimize things. You’ll want to get the Testcontainers.MongoDB NuGet package for this as well. You can , but we’ll start with what our entry point looks like: find all the relevant code on GitHub using BenchmarkDotNet.Running;

using System.Reflection;

BenchmarkRunner.Run(
    Assembly.GetExecutingAssembly(),
    args: args); Nice and simple just to kick off the benchmarks. And the benchmarks are the most important part here: using BenchmarkDotNet.Attributes;

using MongoDB.Bson;
using MongoDB.Driver;

using Testcontainers.MongoDb;

[MemoryDiagnoser]
//[ShortRunJob]
[MediumRunJob]
public class InsertBenchmarks
{
    private MongoDbContainer? _container;
    private MongoClient? _mongoClient;
    private IMongoCollection<BsonDocument>? _collection;
    private IMongoCollection<RecordStructDto>? _collectionRecordStruct;
    private IMongoCollection<RecordClassDto>? _collectionRecordClass;
    private IMongoCollection<StructDto>? _collectionStruct;
    private IMongoCollection<ClassDto>? _collectionClass;

    [GlobalSetup]
    public async Task SetupAsync()
    {
        _container = new MongoDbBuilder()
            .WithImage("mongo:latest")
            .Build();
        await _container.StartAsync();
        
        _mongoClient = new MongoClient(_container.GetConnectionString());
        var database = _mongoClient.GetDatabase("test");
        _collection = database.GetCollection<BsonDocument>("test");
        _collectionRecordStruct = database.GetCollection<RecordStructDto>("test");
        _collectionRecordClass = database.GetCollection<RecordClassDto>("test");
        _collectionStruct = database.GetCollection<StructDto>("test");
        _collectionClass = database.GetCollection<ClassDto>("test");
    }

    [GlobalCleanup]
    public async Task CleanupAsync()
    {
        await _container!.StopAsync();
    }

    [Benchmark]
    public async Task InsertOneAsync_BsonDocument()
    {
        await _collection!.InsertOneAsync(new BsonDocument()
        {
            ["Name"] = "Nick Cosentino",
        });
    }

    [Benchmark]
    public async ValueTask InsertOneAsyncValueTask_BsonDocument()
    {
        await _collection!.InsertOneAsync(new BsonDocument()
        {
            ["Name"] = "Nick Cosentino",
        });
    }

    [Benchmark]
    public void InsertOne_BsonDocument()
    {
        _collection!.InsertOne(new BsonDocument()
        {
            ["Name"] = "Nick Cosentino",
        });
    }

    [Benchmark]
    public async Task InsertOneAsync_RecordStruct()
    {
        await _collectionRecordStruct!.InsertOneAsync(new RecordStructDto("Nick Cosentino"));
    }

    [Benchmark]
    public async ValueTask InsertOneAsyncValueTask_RecordStruct()
    {
        await _collectionRecordStruct!.InsertOneAsync(new RecordStructDto("Nick Cosentino"));
    }

    [Benchmark]
    public void InsertOne_RecordStruct()
    {
        _collectionRecordStruct!.InsertOne(new RecordStructDto("Nick Cosentino"));
    }

    [Benchmark]
    public async Task InsertOneAsync_RecordClass()
    {
        await _collectionRecordClass!.InsertOneAsync(new RecordClassDto("Nick Cosentino"));
    }

    [Benchmark]
    public async ValueTask InsertOneAsyncValueTask_RecordClass()
    {
        await _collectionRecordClass!.InsertOneAsync(new RecordClassDto("Nick Cosentino"));
    }

    [Benchmark]
    public void InsertOne_RecordClass()
    {
        _collectionRecordClass!.InsertOne(new RecordClassDto("Nick Cosentino"));
    }

    [Benchmark]
    public async Task InsertOneAsync_Struct()
    {
        await _collectionStruct!.InsertOneAsync(new StructDto() { Name = "Nick Cosentino" });
    }

    [Benchmark]
    public async ValueTask InsertOneAsyncValueTask_Struct()
    {
        await _collectionStruct!.InsertOneAsync(new StructDto() { Name = "Nick Cosentino" });
    }

    [Benchmark]
    public void InsertOne_Struct()
    {
        _collectionStruct!.InsertOne(new StructDto() { Name = "Nick Cosentino" });
    }

    [Benchmark]
    public async Task InsertOneAsync_Class()
    {
        await _collectionClass!.InsertOneAsync(new ClassDto() { Name = "Nick Cosentino" });
    }

    [Benchmark]
    public async ValueTask InsertOneAsyncValueTask_Class()
    {
        await _collectionClass!.InsertOneAsync(new ClassDto() { Name = "Nick Cosentino" });
    }

    [Benchmark]
    public void InsertOne_Class()
    {
        _collectionClass!.InsertOne(new ClassDto() { Name = "Nick Cosentino" });
    }

    private record struct RecordStructDto(string Name);

    private record class RecordClassDto(string Name);

    private struct StructDto
    {
        public string Name { get; set; }
    }

    private class ClassDto
    {
        public string Name { get; set; }
    }
} The benchmark code has as much as possible that we’re not interested in exercising pulled out into the GlobalSetup and GlobalCleanup marked methods. Results For C# MongoDB Insert Benchmarks Cue the drumroll! It’s time to look at our MongoDB benchmark results and do a little bit of an analysis: Here are my takeaways from the benchmark data above: All async variations used ~5KB more than the normal versions of the methods that we had to use. There didn’t seem to be any difference in allocated memory for async vs async value task BUT Gen0 and Gen1 didn’t have any value for *some* of the ValueTask benchmarks — However, not for all of them. It almost looks like ValueTask combined with a struct data type for the insert results in Gen0 and Gen1 with no value, but plain old BsonDocument is an exception to this. The fastest and lowest memory footprint seems to be InsertOne_RecordClass, although the InsertOne_BsonDocument is only a few microseconds off from this. Async versions of the benchmarks seem slower than their normal versions across the board, as well. This seems to be very much aligned with some of Twitter's opening thoughts on Async operations! So, some hypotheses proved/disproved: Async *is* overall worse for very fast DB operations ValueTask doesn’t stand out as a consistent performance optimization in these situations For single items, there’s no big difference in the memory footprint we’re seeing between any of these variations of the data types It’ll be a good exercise to follow up with benchmarking, inserting many items into MongoDB from C#. I think we may start to see some of these variations stand out in different ways once we’re working with collections of items — But this is still a hypothesis that needs to be proven! Wrapping Up C# MongoDB Insert Benchmarks This was a simple investigation of insert benchmarks for MongoDB using C#. Overall, there were some surprises for me, but I still think there’s more investigation to be done when we’re working with multiple records at a time. I truly was a little bit surprised to see async be worse across the board since I figured that perhaps any type of IO would mask the performance impact of async overhead. But this was a fun experiment to try out, and there is more to come! If you found this useful and you’re looking for more learning opportunities, consider and check out my ! subscribing to my free weekly software engineering newsletter free videos on YouTube Also published . here

Benchmarks for Inserting Documents Using C# for MongoDB

Introduction As a data analyst, I frequently encounter scenarios where seamless task scheduling is crucial for optimizing workflow efficiency. In my own experiments, I've encountered situations where I need to run two Python functions sequentially, with the second function scheduled to execute in the next nearest quarter-hour (15-minute interval) after the completion of the first function. This precise timing ensures that subsequent tasks are executed at optimal intervals, facilitating smooth data processing and analysis. If you're facing a similar challenge in your projects, the code snippet I've developed will provide a solution, streamlining your task scheduling process and enhancing overall productivity. Code Snippet: You can achieve this by calculating the next nearest quarter-hour (15-minute interval) after the completion time of the first function. Here's a Python code snippet to accomplish that: import datetime
import time

def first_function():
    # Your implementation for the first function
    pass

def second_function():
    # Your implementation for the second function
    pass

def get_next_quarter_hour(time):
    # Round up the time to the next quarter-hour
    next_quarter_hour = (time + datetime.timedelta(minutes=15)).replace(
        minute=(time.minute // 15) * 15 % 60, second=0, microsecond=0
    )
    return next_quarter_hour

# Call your first function
first_function()

# Get the current time
current_time = datetime.datetime.now()

# Calculate the next quarter-hour
next_quarter_hour = get_next_quarter_hour(current_time)

# Calculate the delay until the next quarter-hour
delay = (next_quarter_hour - current_time).total_seconds()

# Schedule the second function to run after the delay
time.sleep(delay)
second_function() This code first defines two placeholder functions and where you can place your actual function implementations. Then, it defines a utility function to calculate the next quarter-hour time from any given time. Finally, it calculates the delay until the next quarter-hour from the current time and schedules the execution of the second function after that delay. first_function second_function get_next_quarter_hour Conclusion: Automating time-based tasks with precise scheduling is essential for maintaining efficiency and productivity in data analysis projects. By leveraging the provided Python code snippet, you can ensure timely execution of sequential tasks, with the second function running at the nearest quarter-hour interval after the completion of the first function. This streamlined approach to task scheduling enhances workflow optimization and facilitates smooth data processing and analysis, empowering data analysts to accomplish their tasks with greater precision and effectiveness. Thank you for taking the time to explore data-related insights with me. I appreciate your engagement. If you find this information helpful, I invite you to follow me or connect with me on . Happy exploring!👋 LinkedIn

Automating Time-Based Tasks With Python: Scheduling Functions at Flexible Intervals

Software Engineering for OpenHarmony: The Research Roadmap

In today's ever-evolving tech ecosystem, the demand for real-time data processing solutions has reached unprecedented heights. As data flows in at staggering rates, businesses are relentlessly seeking solutions that not only keep pace but also push the boundaries of what's possible. At the recent event, hosted by , we witnessed significant strides in this direction with the latest updates to Ververica Cloud. Flink Forward 2023 Ververica Apache Flink, an open-source stream processing framework co-founded by the visionaries at Ververica, has emerged as a linchpin in the realm of real-time data processing. Its ability to process vast volumes of data with low latency has made it the go-to choice for businesses across a wide range of industries. With the surge in real-time data demands, enterprises are on a relentless quest for solutions that enhance Apache Flink's capabilities. The recent Flink Forward 2023 event, also led by the co-founders at Ververica, shed light on how Ververica Cloud's latest innovations are poised to redefine the landscape by introducing groundbreaking enhancements that amplify Apache Flink's prowess. Advancing the Speed of Insights Among the most intriguing aspects of these updates is the introduction of VERA (Ververica Runtime Assembly) technology. This innovative addition has ignited discussions within the tech community for its promise to accelerate data processing speeds, surpassing traditional open-source Apache Flink by a substantial margin. In a world where microseconds can make a world of difference, Ververica Cloud's millisecond-level data processing holds the potential to revolutionize our approach to real-time data analytics. Imagine a financial institution making lightning-fast investment decisions based on real-time stock market data. With insights delivered almost instantaneously, Ververica Cloud offers an edge that goes beyond mere efficiency—it represents a paradigm shift in how businesses engage with data. Redefining Efficiency in Data Management The current technological landscape presents new challenges, and one of the most pressing is the efficient management of data resources. Ververica Cloud introduces a flexible pay-as-you-go model that alleviates the financial burden of maintaining on-premises hardware. Beyond the financial aspect, this model streamlines resource allocation, enabling organizations to dynamically scale their infrastructure based on workload demands. The longstanding challenge of over-provisioning is now a relic of the past. This newfound resource management efficiency empowers businesses to optimize their operations while remaining agile in the face of evolving data requirements. Empowering Developers In an era where data fuels innovation, developers play a pivotal role in extracting value from real-time data. Ververica Cloud simplifies the development of stream processing applications, regardless of your preferred programming language—be it standard SQL, Java, or Python. Deployment becomes a hassle-free experience, liberating developers from the intricacies of infrastructure management. The platform's user-friendly interface offers a unified experience for deploying and managing streaming or Flink ML applications, allowing developers to concentrate on innovation rather than operational complexities. A Glimpse into the Future As we navigate the dynamic landscape of 2023, the demand for high-performance, cost-efficient, and developer-friendly real-time analytics solutions continues to surge. Ververica Cloud's commitment to innovation positions it at the forefront of this evolution. It's not merely about addressing today's demands; it's about envisioning and shaping the future of data analytics. According to Alexander Walden, CEO of Ververica, "These new features signify a significant leap forward in the realm of real-time data processing. Beyond the branding and buzz, these developments denote a significant shift in how we interact with data in a world that craves immediacy and efficiency." The Flink Forward 2023 event provided the perfect backdrop for these innovations, bringing together tech visionaries, developers, and industry leaders to explore the future of real-time data processing. It's clear that the advancements unveiled at the event mark a turning point in the way we harness data's power to drive innovation. These new features from an industry leader signify a substantial leap forward in the realm of real-time data processing. Beyond the branding and buzz, these developments denote a significant shift in how we interact with data in a world that craves immediacy and efficiency.

Unlocking the Future of Real-Time Data Processing: Ververica Cloud's Latest Advancements 

In the ever-evolving realm of , the efficiency of interactions can be the linchpin for a high-performing application. This article delves into the quintessential aspects of two prominent Redis libraries: Go-Redis and RedisPipe. While each library possesses its distinct strengths, RedisPipe's prowess in efficiency comes with nuanced considerations, particularly concerning timeout management microservices Redis Comparing Go-Redis and RedisPipe Let’s explore a detailed comparison of Go-Redia and RedisPipe with the following criteria - performance and throughput, ease of use, flexibility & extensibility, and, finally, context timeout in microservices architecture. Performance and Throughput: Go-Redis: Go-Redis stands as a widely-embraced Redis client library, lauded for its robust support and mature codebase. Employing a traditional request-response model for Redis server communication, Go-Redis, despite its reliability and battle-tested nature, may, in high-load scenarios, introduce latency owing to its synchronous essence. Nonetheless, Go-Redis redeems itself through features such as connection pooling, pipelining, and a plethora of optimizations, rendering it a commendable choice for most applications. In the context of ultra-high-throughput systems, however, alternative options might warrant consideration. RedisPipe, in stark contrast, embraces a distinctive approach, placing its focal point on the minimization of latency and the maximization of throughput. Achieving this feat through asynchronous I/O and a pipeline-centric design, RedisPipe adeptly handles a profusion of Redis commands in parallel without obstructing the primary Go routine. For high-load applications, this translates to substantial performance gains, rendering RedisPipe an enticing selection when every microsecond holds paramount importance. RedisPipe: Ease of Use: Go-Redis: Go-Redis shines through its well-documented and widely-adopted presence in the Go community. Its API, intuitive and comprehensible, endears it to developers who prioritize the comforts of familiarity and ease of use. Augmented by comprehensive documentation and unwavering community support, Go-Redis simplifies the quest for assistance and resources when the need arises. RedisPipe: RedisPipe, architected with a view to simplicity and performance, may necessitate a marginally steeper learning curve for initiates due to its asynchronous nature. However, developers already conversant with Go's concurrency paradigms will find RedisPipe's learning curve reasonably accommodating. The trade-off for this initial learning investment lies in the prospect of heightened performance in scenarios where load becomes the chief adversary. Flexibility and Extensibility: Go-Redis: Go-Redis unveils an expansive repertoire of features, encompassing support for Redis clustering, Pub/Sub, and diverse connection options. Its versatility renders it apt for a spectrum of Redis use cases, and its rich feature set champions compatibility with diverse application prerequisites. RedisPipe: RedisPipe, while unwaveringly dedicated to performance, might not inherently endorse certain advanced Redis functionalities. However, it acts as the canvass upon which custom solutions can be crafted when the exigency arises. This adaptability elevates RedisPipe's appeal in projects where exacting performance requisites govern the day. Context Timeout in Microservices Architecture: Go-Redis: In a microservices ecosystem adopting Go-Redis, the setting of context timeouts for Redis operations within individual services finds aptitude. This dexterity permits each microservice to chart its own timeout strategy in resonance with its unique requirements and dependencies. For instance, a service entrusted with user authentication might demand briefer Redis operation timeouts compared to a counterpart handling background data processing. Go-Redis extends its benevolence to the realm of context timeout propagation between microservices. When one microservice invokes another, the torchbearer is the context timeout itself, ensuring the downstream Redis operations hew to the same timeout constraints. This preservative orchestration fosters consistency in the response times of your application, bolstering resilience in the face of adversity. RedisPipe: RedisPipe, in stark contrast, refrains from bestowing built-in support for context timeouts at the individual Redis command level. Given its dependence on efficient, non-blocking Redis communication, avenues to impose timeouts become relatively limited. To navigate this conundrum in a microservices tapestry, the onus shifts to custom timeout management at the microservices level. The remedial strategy often involves the construction of supplementary layers or middleware to adeptly handle timeouts governing RedisPipe commands. While feasible, this approach unfurls an added layer of complexity and necessitates the choreographed coordination of efforts across the microservices ensemble. Conclusion: In the realm of microservices, where every millisecond matters, the choice of a Redis library can significantly impact the performance and reliability of your applications. Go-Redis and RedisPipe represent two compelling options, each with its own set of strengths and considerations. presents a mature and reliable choice, offering a battle-tested request-response model. It boasts an intuitive API, extensive documentation, and a broad feature set that includes support for Redis clustering and Pub/Sub. However, in ultra-high-throughput scenarios, its synchronous nature may introduce latency. Go-Redis , on the other hand, positions itself as a performance-oriented solution. Its asynchronous I/O and pipeline-centric design make it a powerhouse for high-load applications, where minimizing latency and maximizing throughput are paramount. While it may require a slightly steeper learning curve, RedisPipe's performance gains can be a game-changer. RedisPipe When choosing between these libraries, consider the nature of your microservices architecture and your specific performance requirements. Go-Redis shines in terms of ease of use and versatility, making it an excellent choice for many projects. RedisPipe, with its focus on performance, is the go-to solution when squeezing every ounce of performance from your Redis interactions is essential. However, it's crucial to be aware of RedisPipe's automatic request cancellation behavior when timeouts are exceeded. This feature enhances responsiveness but necessitates careful planning and the implementation of resilience mechanisms to ensure the robustness of your microservices, even in challenging conditions. In conclusion, both Go-Redis and RedisPipe have their places in the toolkit of a developer. Evaluate the trade-offs and select the library that aligns best with your project's unique requirements and performance demands. microservices

Choosing the Right Redis Library for Microservices: Go-Redis vs. RedisPipe

LONDON, United Kingdom, September 8th, 2023/Chainwire/--One Trading, the crypto asset exchange formerly operating under the Bitpanda Pro brand, today announced its new product, Instant Trade. Having recently raised €30M and fully separated from Bitpanda, Instant Trade is another step forward for One Trading as it diversifies its offering and establishes its footprint as a household name providing crypto assets to both retail and institutional investors in Europe. With backers including the likes of Peter Thiel’s Valar Ventures, MiddleGame Ventures, Speedinvest, Keyrock, and Wintermute Ventures, One Trading is drawing a lot of attention. Led by TradFi heavyweight and ex-JP Morgan exec Josh Barraclough, One Trading aims to address the lack of regulated venues in Europe by offering more sophisticated crypto products, including derivatives and spot trading, to both institutions and retail customers. With many traditional finance players lacking experience in the crypto market, there is a significant gap in providing long-term product confidence and success. What The CEO Had to Say The latest product being launched is Instant Trade. Josh had the following to say about Instant Trade: “We’re really excited to be unraveling our latest product, Instant Trade, which brings our OTC product, typically reserved for institutional clients, to all customer types. Our customers can now trade 100+ pairs between crypto-assets and fiat seamlessly with great fiat on and off ramps at the click of a button, and commission-free trading. Most brokers charge a dealing fee when customers buy or sell assets, however, Instant Trade will have 0% commission, meaning there is no additional dealing fee charged on Instant Trade aside from the market spread applied. We’ve built up a deep pool of liquidity with a range of partners over the years, which everyone can access without having to trade hundreds of millions a day or hold ecosystem tokens. We have a simplified, really easy to use UI and customers can even trade up to ten million euros in one trade. We’ll also be regularly adding new assets and our customers can rest assured that with our VASP license, we are a safe, secure, and regulated service, for anyone that isn’t a customer, you can test out Instant Trade on our website and compare our prices. We’re really excited to hear feedback from our customers so do let us know what you think via our socials and community channels. We will also be releasing an article on the One Trading blog page specifically about Instant Trade, so keep an eye out for that on the website.” "We want to make crypto trading safe and accessible for all and bridge the gap to provide an institutional-grade platform for all customer types. We have been rolling out a rebrand and a number of big infrastructure changes. We will be going live shortly with what we believe to be the fastest and most scalable exchange (< 250-microsecond order create/cancel) based on real-world metrics available to a shared retail and institutional audience. We aim to become a utility for large liquidity providers to exchange unlimited amounts of risk under a membership model instead of pay-per-trade and have low fees and deep books for retail with a number of liquidity protections. We will then start listing more products with appropriate controls and vetting as we move into derivatives. Above all we want a regulated, institutional-grade platform where people feel safe to trade with unique product options"Josh Barraclough, CEO, One Trading. https://www.youtube.com/watch?v=7rInXR-V2yo&embedable=true One Trading intends to operate as a MiFID Trading Venue, which will augment its existing VASP license, and would allow its platform to offer capital-efficient spot and derivative products for all customer types. It also offers a higher level of protection for customers who will be vetted through stringent anti-money laundering (AML) and Know-your-customer (KYC) procedures as well as client appropriateness checks. The full-scale MiFID license will also allow One Trading to list financial instruments – removing the debate over whether a DLT asset is a token or a security – and will permit new products to be designed from the ground up with appropriate vetting, transparency, and customer protection in mind. One Trading’s platform also offers access to a high-touch and secure OTC trading service, enabling high-net-worth individuals and institutions access to a wide range of digital assets, with competitive fees, and deep pools of liquidity. About One Trading The One Trading team is based and headquartered in Europe with offices across Amsterdam, London, Milan, and Vienna. One Trading is regulated in the EU and has built its product and venue to meet the needs of European customers. One Trading’s mission is to bridge the gap between crypto and traditional asset trading with a single regulated trading platform for all customer types and all products. Users can learn more about One Trading at their website: www.onetrading.com and for more updates, follow the One Trading social channels via Linktree. Contact Marketing Contact Andy Keelaghan One Trading info@3webdigital.com This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program.https://business.hackernoon.com/business-blogging?ref=hackernoon.com

One Trading Debuts Instant Trade to the World: What You Need to Know

One Trading presenta el comercio instantáneo al mundo: lo que necesita saber

One Trading lance le commerce instantané dans le monde : ce que vous devez savoir

वन ट्रेडिंग ने दुनिया में त्वरित व्यापार की शुरुआत की: आपको क्या जानना चाहिए

One Trading がインスタント取引を世界にデビュー: 知っておくべきこと

One Trading estreia comércio instantâneo para o mundo: o que você precisa saber

Một giao dịch ra mắt giao dịch tức thì với thế giới: Những điều bạn cần biết

One Trading 向全球推出即时交易：您需要了解什么

An Adventure Amidst the Industrial Revolution

In this blog post, we will be covering how you can combine and leverage the open-source streaming solution, , with , to improve the quality of your streaming flows. Buckle up! bytewax ydata-profiling Stream processing enables real-time analysis of data in-flight and before storage and can be or . stateful stateless is used for recommendations, pattern detection, or complex event processing, where the history of what has happened is required for the processing (windows, joining by a key, etc.). Stateful stream processing real-time is used for transformation that doesn’t require knowledge of other data points in the stream like masking an email or converting a type. Stateless stream processing inline Overall, data streams are widely used in the industry and can be found applied to use cases such as , , or . fraud detection patient monitoring event predictive maintenance One Crucial Aspect That All Data Streams Must Consider Is the Quality of the Data Unlike traditional models where data quality is usually assessed during the creation of the data warehouse or dashboard solution, . streaming data requires continuous monitoring It is essential to maintain data quality throughout the entire process, from collection to feeding downstream applications. After all, the cost of bad data quality can be high for organizations: “The cost of bad data is an astonishing 15% to 25% of revenue for most companies. (…) Two-thirds of these costs can be eliminated by getting in front on data quality.” — Thomas C. Redman, author of “Getting in Front on Data Quality” Throughout this article, we will show you how you can combine with to profile and improve the quality of your streaming flows! bytewa ydata-profiling Stream Processing for Data Professionals With Bytewax is an OSS stream processing framework designed specifically for Python developers. Bytewax It allows users to with capabilities similar to Flink, Spark, and Kafka Streams while providing a friendly and familiar interface and build streaming data pipelines and real-time applications 100% compatibility with the Python ecosystem. Using built-in or existing Python libraries, (Kafka, RedPanda, WebSocket, etc.) and out to various downstream systems (Kafka, parquet files, data lakes, etc.). connectors you can connect to real-time and streaming data sources write transformed data For the transformations, Bytewax with , , and methods and comes with familiar features such as recovery and scalability. facilitates stateful and stateless transformations map windowing aggregation Bytewax and is p . facilitates a Python-first and data-centric experience to data streams urposely built for data engineers and data scientists It allows users to and create customizations necessary to meet their needs without having to learn and maintain JVM-based streaming platforms like Spark or Flink. build streaming data pipelines and real-time applications Bytewax is well suited for many use cases, namely, , , , and more. Embedding Pipelines For Generative AI Handling Missing Values in Data Streams Using Language Models in a Streaming Context to Understand Financial Markets For use case inspiration and more information like documentation, tutorials, and guides, feel free to check . the bytewax website Why Data Profiling for Data Streams? and refers to the step of : its structure, behavior, and quality. Data Profiling is key to a successful start of any machine learning task thoroughly understanding our data In a nutshell, involves analyzing aspects related to the data’s format and basic descriptors (e.g., number of samples, number/types of features, duplicate values), its (such as the presence of missing data or imbalanced features), and other complicating factors that may arise during data collection or processing (e.g., erroneous values or inconsistent features). data profiling intrinsic characteristics , where circumstances might change fast and may require immediate action (e.g., healthcare monitoring, stock values, air quality policies). Ensuring high data quality standards is crucial for all domains and organizations, but is especially relevant for domains operating with domains outputting continuous data For many domains, data profiling is used from an exploratory data analysis lens, considering historical data stored in databases. On the contrary, for data streams, , where data needs to be checked at different time frames or stages of the process. data profiling becomes essential for validation and quality control continuously along the stream By embedding an , we can on the current state of our data and be alerted for any potentially critical issues — whether they are related to (e.g., corrupted values or changing formats), or to (e.g., data drifts, deviation from business rules and outcomes). automated profiling into our data flows immediately get feedback data consistency and integrity events happening in short periods of time In real-world domains — — automated profiling might save us from multiple brain puzzles and systems needing to be taken out of production! where you just know Murphy’s law is bound to strike and “everything can definitely go wrong” In what concerns data profiling, has consistently been a , either for or data. And no wonder why — ydata-profiling crowd favorite tabular time-series it’s one line of code for an extensive set of analysis and insights. Complex and time-draining operations are done under the hood: ydata-profiling and depending on the feature types (either numeric or categorical), it that are shown in the profiling report. automatically detects the feature types comprised in the data, adjusts the summary statistics and visualizations Fostering a , the package also , focusing on their pairwise and , and provides a thorough , from or values to and features. data-centric analysis highlights the existing relationships between features interactions correlations evaluation of data quality alerts duplicate constant skewed imbalanced It’s really a — with minimal effort. 360º view of the quality of our data Putting It All Together: bytewax and ydata-profiling Before starting the project, we need to first set our Python dependencies and configure our data source. First, let’s install the and packages ( bytewax ydata-profiling You might want to use a virtual environment for this — if you need some extra guidance!) check these instructions pip install bytewax==0.16.2 ydata-profiling==4.3.1 Then, we’ll upload the (License — CC0: Public Domain), which contains several measurements of from different IoT devices: Environmental Sensor Telemetry Dataset temperature, humidity, carbon monoxide liquid petroleum gas, smoke, light, and motion wget https://raw.githubusercontent.com/bytewax/air-quality-sensor/main/data/iot_telemetry_data_1000 , and the input would look like what we expect in a streaming platform . In this article, and create a dataflow using bytewax. In a production environment, these measurements would be continuously generated by each device such as Kafka to simulate the context we would find with streaming data, we will read the data from the CSV file one line at a time (As a quick side note, a dataflow is essentially a data pipeline that can be described as a directed acyclic graph — DAG) First, let’s make some : necessary imports from datetime import datetime, timedelta, timezone

from bytewax.dataflow import Dataflow
from bytewax.connectors.stdio import StdOutput
from bytewax.connectors.files import CSVInput
from bytewax.testing import run_main Then, we define our dataflow object. Afterward, we will use a stateless map method where we pass in a function to convert the string to a DateTime object and restructure the data to the format (device_id, data). The map method will make the change to each data point in a stateless way. The reason we have modified the shape of our data is so that we can easily group the data in the next steps to profile data for each device separately rather than for all of the devices simultaneously. flow = Dataflow()
flow.input("simulated_stream", CSVInput("/content/iot_telemetry_data_1000"))

# parse timestamp
def parse_time(reading_data):
    reading_data["ts"] = datetime.fromtimestamp(float(reading_data["ts"]), timezone.utc)
    return reading_data

flow.map(parse_time)


# remap format to tuple (device_id, reading_data)
flow.map(lambda reading_data: (reading_data['device'], reading_data)) Now, we will take advantage of the stateful capabilities of to gather data for each device over the duration of time that we have defined. expects a snapshot of the data over time, which makes the window operator the perfect method to use to do this. bytewax ydata-profiling In , we are able to produce summarizing statistics for a dataframe that is specified for a particular context. For instance, in our example, we can produce snapshots of data referring to each IoT device or to particular time frames: ydata-profiling from bytewax.window import EventClockConfig, TumblingWindow

# This is the accumulator function, and outputs a list of readings
def acc_values(acc, reading):
    acc.append(reading)
    return acc


# This function instructs the event clock on how to retrieve the
# event's datetime from the input.
def get_time(reading):
    return reading["ts"]

  
# Configure the `fold_window` operator to use the event time.
cc = EventClockConfig(get_time, wait_for_system_duration=timedelta(seconds=30))

# And a tumbling window
align_to = datetime(2020, 1, 1, tzinfo=timezone.utc)
wc = TumblingWindow(align_to=align_to, length=timedelta(hours=1))

flow.fold_window("running_average", cc, wc, list, acc_values)

flow.inspect(print) After the snapshots are defined, leveraging is as simple as calling the for each of the dataframes we would like to analyze: ydata-profiling ProfileReport import pandas as pd
from ydata_profiling import ProfileReport

def profile(device_id__readings):
    print(device_id__readings)
    device_id, readings = device_id__readings
    start_time = readings[0]['ts'].replace(minute=0, second=0, microsecond=0).strftime('%Y-%m-%d %H:%M:%S')
    df = pd.DataFrame(readings)
    profile = ProfileReport(
        df,
        tsmode=True,
        sortby="ts",
        title=f"Sensor Readings - device: {device_id}"
    )

    profile.to_file(f"Ts_Profile_{device_id}-{start_time}.html")
    return f"device {device_id} profiled at hour {start_time}"


flow.map(profile) In this example, we are writing the images out to local files as part of a function in a map method. These could be reported out via a messaging tool or we could save them to some remote storage in the future. Once the profile is complete, the dataflow expects some output so we can use the built-in to print the device that was profiled and the time it was profiled at that was passed out of the profile function in the map step: StdOutput flow.output("out", StdOutput()) There are multiple ways to execute Bytewax dataflows. In this example, we use the same local machine, but Bytewax can also run on multiple Python processes, across multiple hosts, in a , using a , and . Docker container Kubernetes cluster more In this article, we’ll continue with a local setup, but we encourage you to check our helper tool which manages Kubernetes dataflow deployments once your pipeline is ready to transition to production. waxctl Assuming we are in the same directory as the file with the dataflow definition, we can run it using: python -m bytewax.run ydata-profiling-streaming:flow We can then use the profiling reports to validate the data quality, check for changes in schemas or data formats, and . compare the data characteristics between different devices or time windows In fact, we can leverage the that highlights the differences between two data profiles in a straightforward manner, making it easier for us to detect important patterns that need to be investigated or issues that have to be addressed: comparison report functionality snapshot_a_report = ProfileReport(df_a, title="Snapshot A")
snapshot_b_report = ProfileReport(df_b, title="Snapshot B")

comparison_report =snapshot_a_report(snapshot_b_report)
comparison_report.to_file("comparison_report.html") Ready to Explore Your Own Data Streams? Validating data streams is crucial to identify issues in data quality in a continuous manner and compare the state of data across distinct periods of time. For organizations in , , , and — all working with continuous streams of data — a , from quality assessment to data privacy. healthcare energy manufacturing entertainment utomated profiling is key to establishing data governance best practices This requires the analysis of snapshots of data which, as showcased in this article, can be achieved in a seamless way by combining and . bytewax ydata-profiling takes care of all the processes necessary to handle and structure data streams into snapshots, which can then be summarized and compared with through a comprehensive report of data characteristics. Bytewax ydata-profiling Being able to appropriately process and profile incoming data opens up a plethora of use cases across different domains, from the to the highlighting and mitigation of additional issues that derive from real-world activities, such as (e.g., fraud or intrusion/threats detection), , and other events that deviate from the expectations (e.g., data drifts or misalignment with business rules). correction of errors in data schemas and formats anomaly detection equipment malfunction Now, you’re all set to start exploring your data streams! Let us know what other use cases you find, and as always, feel free to drop us a line in the comments, or find us at the for further questions and suggestions! Data-Centric AI Community See you there! Acknowledgments ) and Oli Makhasoeva (Developer Relations @ ) -- developing . This article was written by Fabiana Clemente (Co-founder & CDO @ ) and Miriam Santos (Developer Relations @ ) -- developing -- and Zander Matheson (CEO & Founder @ YData YData ydata-profiling Bytewax Byetwax bytewax You may find additional information about the OSS packages in the respective documentations: & . ydata-profiling docs bytewax docs Also published here

How to Understand Your Data in Real-Time Using bytewax and ydata-profiling

Cómo entender sus datos en tiempo real usando bytewax y ydata-profiling

Comment comprendre vos données en temps réel en utilisant bytewax et ydata-profiling

बाइटवैक्स और वाईडेटा-प्रोफाइलिंग का उपयोग करके वास्तविक समय में अपने डेटा को कैसे समझें

bytewax と ydata-profiling を使用してデータをリアルタイムで理解する方法

Como entender seus dados em tempo real usando bytewax e ydata-profiling

Cách hiểu dữ liệu của bạn trong thời gian thực bằng cách sử dụng bytewax và ydata-profile

如何使用 bytewax 和 ydata-profiling 实时了解您的数据

Welcome to our guide on building a hardware-software Geiger counter emulator of module based on ESP8266 This emulator can be useful for testing and tuning Geiger counters or for educational purposes. Let’s get started! GGreg20_V3 When writing this article, we decided to make a separate commercial product – the Geiger counter emulator . Updated: GCemu20_V1 Part 1: Introduction and Overview It is known that the Espressif ESP8266 module has a very high quality random number generator. We could not find any official details about the actual implementation of this generator, so we propose to read the material on this subject, published by a radio amateur and saved on the Internet: https://web.archive.org/web/20170321162556/http://esp8266-re.foogod.com/wiki/Random_Number_Generator Two different concepts with the same name are used in the text. Historically, there is a NodeMCU hardware module on the market as well as NodeMCU software firmware. The NodeMCU firmware is a compilable, open-source microprogram that can be written to hardware modules with Espressif’s ESP8266 controller. One of the many products based on the ESP8266 controller, is the NodeMCU hardware module. Thus the NodeMCU firmware can be written to the NodeMCU hardware module. Later in this article we will try, if possible, to clarify what is meant by the name NodeMCU: module or firmware. What is a Geiger counter emulator for Before proceeding to create a Geiger counter emulator, it is necessary to consider what for and who may need such a hardware-software device. The main idea of any emulator in the field of DIY electronics is to temporarily, at certain stages, use a virtual substitute component instead of a real module in the process of IoT device development or experimenting / learning to reproduce the operation and characteristics of a real device with high accuracy. The emulator should simplify and speed up development, as well as add convenience in the initial stages of a planned project or performing unit tests. Below we present some theses explaining our motives for developing the GGreg20_V3 Geiger counter module emulator. 1. No high voltage The GGreg20_V3 emulator, unlike the real module, has no high voltage on the board, so it is very convenient to use when developing on the table without fear of accidental electrocution. At the same time, in terms of the electrical interface of the outgoing pulses, the emulator is a completely similar radiation sensor module, generating random pulses of the same shape and duration as the genuine GGreg20_V3. 2. Simplified learning process The Geiger counter emulator is suitable for educational institutions. The instructor can gradually deepen practical lessons with students. First, all classes can be conducted using safe and cheap Geiger counter emulators, and then with real modules with Geiger tubes and high voltage on the tube side. Just like in the Army: familiarization and training is done first with blank cartridges, and only then with live ones. The emulator fully reproduces the results of the real GGreg20_V3 module, so there is no problem of insufficient number of real modules in the classroom: each student can work with his module independently and complete the learning process without dividing into groups or queues, as it usually happens when resources of training devices and stands in the lab are limited. 3. Lower cost The emulator has a lower cost than the real GGreg20_V3 module, so it is convenient to use it primarily for debugging and harmonizing the circuit or software of the system being designed or studied. Debugging a test bench or a project with such an emulator can be performed without the need for specially qualified personnel. If the emulators are used by students, then the teacher will not need to control the work with the devices, as would be necessary to do with the real Geiger counter module. Low-cost emulators can be purchased in large quantities and given to students for the duration of their studies, not just for the time they work in the school’s laboratory. 4. No real source of radiation is needed When we work with a real GGreg20_V3, we need a true source of radiation to reproduce the various situations that need to be considered in the software of the measurement system that is being designed or tested. In order to buy a test radiation source, the radio amateur must first solve a number of problems: to determine by physical and chemical parameters which source is needed; to find a supplier and buy a test source; some countries require customs notification and an import permit; the test source can only be of a certain power; you must properly store and dispose of the radioactive source; the price of the radiation source and its storage system exceeds many times the price of the Geiger counter. With the emulator, none of these problems will occur. The device can simulate 5 different modes of ionizing radiation power. The Geiger counter emulator generates output pulses similar to those of the real GGreg20_V3 module in the range from 0 to 1.5 µSv/h (taking the SBM-20 tube as a similar reference). At the output are generated chaotic (using a true random number generator ESP8266) pulses of the same amplitude with a total pulse period of 10 microseconds each. 5. Emulator resource cannot be depleted SBM-20 tube, installed in GGreg20_V3, has a relatively large, but still limited resource of particles, which it can detect during its lifetime. The resource of the SBM-20 tube declared in the documentation is at least 2*1010 pulses. . With a background radiation of 0.15 μSv/h with a coefficient of 0.0057, the SBM-20 tube detects about 27 pulses per minute. Note 1 Thus, under normal conditions, the resource of the tube will be enough for 2*1010 / (27 * 60 * 24) = 20 000 000 000 / (26 * 60 * 24) = 514403 days. And it would seem that this is quite a sufficient potential life of one tube. A completely different picture is observed when it comes to operating a tube with a test source of radiation. Here’s an example: On its website, Images Scientific Instruments provides a list of the radiation limits allowed for import for test sources: https://www.imagesco.com/geiger/radioactive-sources-int.html Let’s take two sources from the list at the link: Co-60 1.00 uCi, 37000 Bq; Cs-137 0.25 uCi, 9250 Bq; If we assume that under the influence of such a test source, half of the particles from radioactive decays enter the Geiger tube SBM-20 then the lifetime of such a tube would be: 20 000 000 000 / (37000 / 2 * 60 * 60 * 24) = 12 days (for Co-60); 20 000 000 000 / (9250 / 2 * 60 * 60 * 24) = 50 days (for Cs-137). As you can see, if we want to experiment (or test our devices that include a Geiger counter) on a real tube, then we deplete its available resource with a true source of radiation rather quickly. There is no such problem with the emulator. Using the emulator first instead of the tube is economically feasible. 6. Debugging data in the UART The emulator of GGreg20_V3 module outputs operational data (pulses, time, counts, cycles, etc.) to the UART console port during operation, which, if necessary, can be recorded and processed by the student, developer or radio amateur for training or comparison with the measurements of his tested or developed system. Genuine GGreg20_V3 counter modules and similar modules of other manufacturers do not and cannot have such a built-in feature. . To connect to the UART console, the , on which the emulator is based, has separately led UART interface lines (2.54 mm pitch holes for soldering) on the PCB. To connect the emulator to the computer the user additionally needs to have his own USB-UART converter. Note 2 ESP12.OLED module Further in the text the connection diagram will be given. Also published . here

Geiger Counter Emulator of GGreg20_V3 Module by Means of ESP8266 (Part 1 of 3)

Emulador de contador Geiger del módulo GGreg20_V3 por medio de ESP8266 (Parte 1 de 3)

Émulateur de compteur Geiger du module GGreg20_V3 au moyen d'ESP8266 (partie 1 sur 3)

ESP8266 के माध्यम से GGreg20_V3 मॉड्यूल का गीगर काउंटर एमुलेटर (3 का भाग 1)

ESP8266 による GGreg20_V3 モジュールのガイガー カウンター エミュレーター (パート 1/3)

Emulador de contador Geiger do módulo GGreg20_V3 por meio de ESP8266 (parte 1 de 3)

Trình giả lập bộ đếm Geiger của Mô-đun GGreg20_V3 bằng ESP8266 (Phần 1/3)

使用 ESP8266 的 GGreg20_V3 模块的盖革计数器仿真器（第 1 部分，共 3 部分）

Authorization is a crucial pain point software developers inevitably encounter when designing software. As much as it needs to be custom to every application, it also needs to be easily scalable and secure. For the uninitiated, authorization is the mechanism for checking what a user is allowed to do — anything from the pages you’re allowed to see, to the data that you are allowed interact with. Without proper authorization, an application can become vulnerable to unauthorized access, data breaches, and other security threats. Typically, stateful authorization, a system where permissions are granted based on stored information about a user's past actions or status, is utilized by development teams. These are often built by hand, ad-hoc and don’t take into account future scalability. Stateless security Yet as the number of application users balloons, the possibility of state desynchronization increases. That could result in possible leaks of sensitive information, users denied access or access enabled based on incorrect data. This is where stateless authorization comes into play. Companies like Cerbos provide an API for an authorization layer that relies on contextual information in a user request and is self-contained; meaning that any server can process them without drawing on stored user and application data. That makes it not only secure but also scalable, increasing application performance across the board. That’s what staff engineer at youth culture ecommerce platform NTWRK Steve High found when he opted for Cerbos, commenting that “if you imagine 10,000 people trying to buy the same instance of a physical product on our platform at the same time, that's what we have to deal with. Using Cerbos as a sidecar, we’ve been able to get permissions-checking latency down to microseconds… in turn, NTWRK is able to provide a great user experience to our customers, both internal and external.” In stateless authorization, each request is evaluated independently without having to interact with a database, bumping up performance in applications. No longer bogged down by expensive database queries, stateless authorization can handle large volumes of requests, making it ideal for high-traffic applications like NTWRK where the speed of response time could be a deciding factor as to whether the user stays or goes with a competitor. Unlocking flexibility As any startup founder knows, flexibility is key in scaling a business and, with stateless authorization, policies can be easily changed and updated. With requests now relying on context to make a decision on user access, policy modifications can be made that do not invalidate existing sessions. This makes it ideal for companies building in dynamic and fast-changing industries such as fintech, SaaS and ecommerce. For B2B fintech platform Loop, which aims to streamline the financial operation of businesses, flexibility in authorization processes is of particular importance as it deals with financial transactions. “Unless there's a change in customer requirements, or our internal requirements, in which we say that, we need to allow this person to do this — we just need to tweak some policies and deploy it,” says CTO Mohsin Kalam after implementing Cerbos’ stateless authorization, “And it hardly requires a core application code change on our end. We just deploy it and push the policies to the server and everything just works out of the box.” If authorization isn’t built in keeping with the demands of a rapidly-changing startup, it can be a headache for the developer teams. As highlighted by both NTWRK and Loop, stateless authorization offers a lightweight, scalable and flexible approach to controlling access — it’s time that the industry caught onto that.

The Future is Stateless: It’s Time To Ensure Your Security Is As Scalable as Your Business

You’ve probably used the command multiple times to test the connection to the server. This is a widely available tool. Have you ever thought about how exactly it works and how to implement it yourself? I find exploring some unknown things helpful in understanding how the system works, and in this article, I'll try to explain how exactly it works and how to recreate it from scratch in ping Node.js. This is not a deep dive by any means, but we will cover some basics on how to implement the binary protocol, manipulate bits, debug network requests, measure time precisely, and more. So let's dive in. What is ping? In case you have never used before, " is a computer network administration software utility used to test the reachability of a host on an Internet Protocol (IP) network... Ping measures the round-trip time for messages sent from the originating host to a destination computer that are echoed back to the source." (c) ping ping wiki Let's give it a try. Just open a terminal and execute . You should see something similar to this: ping 1.1.1.1 $ ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=8.322 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=19.255 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=59 time=7.433 ms
... Behind the scenes, we send ICMP packets to and await a response. At the same time, we measure the time it takes, TTL, and some statistics across all requests. There is a high chance that you never heard about packets before. This protocol is primarily used by network devices to share operational information. It differs from more common TCP and UDP protocols but works on the same IP stack. Sounds fun, let's dive into implementation details. 1.1.1.1 ICMP Let's take a look at the protocol description on wiki. We will focus on IPv4 for simplicity, but it works similarly for IPv6. A lot is going on here. However, most of the work will be done automatically for us by OS or socket library. Let's focus on the red parts that we will have to implement ourselves. Now we need to figure out how to send those bits and bytes Debugging network with Wireshark Now, we have a good theory behind us to get started. However, to debug our program, it would be helpful to see the actual bits and bytes we send to some IPs. The easiest way to check ANY traffic your machine is using is to use It is free and open-source software that works on both Windows and macOS. Before we start, let's check if theory and practice align on how the protocol works. Wireshark. Two easy steps: Start Wireshark and choose what to capture (in my case wifi that is connected to the Internet) Add filter ICMP in the top bar as it will produce a lot of noise Now execute command in the terminal. This is more or less what you will see: $ ping 1.1.1.1 You can see all requests and responses of the ping command. Even better, Wireshark is quite powerful and knows how to handle ICMP commands. So it will help you parse bits to more meaningful and easier-to-read structures. You can also clearly see the layered structure of network protocols (Ethernet -> IP -> ICMP). This will be very helpful in validating our setup. You can also hover over Packet Bytes to highlight what they are responsible for. In this case, you can see the Type of request is equal to , which is exactly what we would expect from our protocol described above. 8 Let's code Now we have all the building blocks; we can start coding. Unfortunately, Node.js has no native support for raw sockets to use ICMP protocol. However, the npm package supports it using node-gyp to access system-level APIs. raw-socket So practically, we need to: create socket subscribe to messages send messages import raw from "raw-socket";

const socket = raw.createSocket({ protocol: raw.Protocol.ICMP });

socket.on("message", function (buffer, source) {
  // TODO Decode
});

socket.on("error", (e) => {
  console.error("Socket Error: ", e);
  socket.close();
});

const pingBuffer = createPingBuffer(1234, 0, "Payload");
socket.send(pingBuffer, 0, buffer.length, "1.1.1.1", function (error, bytes) {
  if (error) console.error("Unable to send message: ", error.toString());
});

function createPingBuffer(identifier, sequence, payload) {
  // TODO Encode
} This is the main part of the app. Here we create an ICMP socket and start listening for new messages. At the same time, we send our ping command to the IP address. 1.1.1.1 So now we need to implement encoding and decoding of the messages. Encoding ping protocol As you saw in the diagram above, we must provide 8 bytes of information: Type, Code, Checksum, Identifier, and Sequence Number. We can also provide an optional payload that will be returned to us in the response. This will be helpful later. Type = 8 (Request) = 00001000 (binary)
Code = 0 = 00000000 (binary)
Checksum = dynamic value
Identifier = 1 = 0000000000000001 (binary)
Sequence Number = 0 = 0000000000000000 (binary) For clarity, we show this information in multiple rows with a nice table. But just to be clear what we actually send is the following 64 bits: 0000100000000000111101111111111000000000000000010000000000000000 The easiest way to manipulate bits is to use Node.js . It is a subclass of JavaScript's Uint8Array and adds helpful methods to manipulate raw bytes. Buffers const ICMP_HEADER_SIZE = 8;

function createPingBuffer(identifier, sequenceNumber, payload) {
  // Allocate empty 8 bytes buffer filled with 0
  const buffer = Buffer.alloc(ICMP_HEADER_SIZE);

  buffer.writeUInt8(8, 0); // Type 1 byte with offset 0 bytes
  buffer.writeUInt8(0, 1); // Code 1 byte with offset 1 byte
  buffer.writeUInt16BE(0, 2); // Checksum 2 bytes with offset 2 bytes
  buffer.writeUInt16BE(identifier, 4); // Identifier 2 bytes with offset 4 bytes
  buffer.writeUInt16BE(sequenceNumber, 6); // Sequence Number 2 bytes with offset 6 bytes

  // Override 0 checksum with correct value based on full request
  raw.writeChecksum(buffer, 2, raw.createChecksum(buffer));

  return buffer;
} You might notice a weird BE suffix in the function name; it stands for Big-Endian. This is . There are two ways how you can pack multi-byte values in the buffer. This defines which byte should be first/last. endianness That is it! We should be able to send ping requests now. Let's check if it works using Wireshark. As you can see, it works as expected, and we were able to send a request. However, we still need to decode a response Decode ping protocol We can use similar helper methods on the Buffer class to decode the ping packet. However, there is a caveat. We will get a full IP protocol packet as a response. Yet, IP protocol has a dynamic header size. It means we will have to find where our ping message starts. Let's take a closer look at the datagram IHL stands for . It has 4 bits that specify the number of 32-bit (4 bytes) words in the header. Unfortunately, Node.js Buffers lack helper methods to read 4-bit data. It means that we will have to implement it ourselves. Internet Header Length Here we will need to use . To get the last 4 bits of a byte, we will need to use bitwise bitwise operations AND function getIPProtocolHeaderSize(buffer) {
  const versionAndIHL = buffer.readUInt8();
  const IHL = versionAndIHL & 0b00001111; // will remove everything in the first 4 bits -> 0x0000XXXX
  return IHL * 4; // multiply by 4 as it contains number of 32-bit (4 bytes) words to get total size in bytes
} Now we have everything we need to decode a response: function decodePingPacket(buffer) {
  const ipOffset = getIPProtocolHeaderSize(buffer);

  // IP level TTL
  const ttl = buffer.readUInt8(8);

  const type = buffer.readUInt8(ipOffset);
  const code = buffer.readUInt8(ipOffset + 1);
  const checksum = buffer.readUInt16BE(ipOffset + 2);
  const identifier = buffer.readUInt16BE(ipOffset + 4);
  const sequenceNumber = buffer.readUInt16BE(ipOffset + 6);

  return {
    ttl,
    type,
    code,
    checksum,
    identifier,
    sequenceNumber,
  };
} That is it! We need to precisely measure the time between request and response to have a fully functional ping client. Time measurement If you pay closer attention to the original ping implementation, you will notice a special trick to make our client stateless and don't require us to track state for request/response time as ping protocol requires sending back the same payload we send as part of the request. We can send time as part of the payload! This is what the original UNIX implementation did. Unix implementation of ping is open source, so we can learn some tricks from reading . Here, it adds as part of the payload. source code timeval struct timeval {
  time_t      tv_sec;     /* seconds */
  suseconds_t tv_usec;    /* microseconds */
}; Let's use the same approach with our client. We will send a precise time when we send the command and compare it when we get a response. Standard will not work well in this case as we deal with high-precision data. To get accurate time, we can use API. So let's put it into practice and add it to the payload together with the payload itself. Date.now() Performance const time = performance.timeOrigin + performance.now();

const uint32 = new Uint32Array(2);
uint32[0] = time / 1000; // seconds
uint32[1] = (time - uint32[0] * 1000) * 1000; // microseconds

// write it as first 8 bytes of payload
buffer.writeUInt32BE(uint32[0], 8);
buffer.writeUInt32BE(uint32[1], 12);

// write payload itself
buffer.write(payload, 16);

// compute checksum
raw.writeChecksum(buffer, 2, raw.createChecksum(buffer));

return buffer; Conclusion I intentionally simplified some things and cut some corners. Ideally, we will need to add error logging, keep track of statistics, etc. We also need to make sure to filter out all other ICMP packets that are not part of our ping request by . checking the type of request and identifier However, I hope it was an interesting journey on how to implement low-level binary protocols, measure time, debug network requests, do bit manipulations, and read the source code. There is a lot to learn from other open-source projects. I put together all the code here in . It has some extra code to connect all the pieces together. Just run this repo node index.js 1.1.1.1 $ node index.js 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 28 data bytes
36 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=32.622314453125 ms
36 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=9.591552734375 ms
36 bytes from 1.1.1.1: icmp_seq=2 ttl=59 time=16.101806640625 ms
...

Building Ping Command in Node.js: Practical Guide to Buffers, Sockets, and Bit Manipulation

Recently I encountered an issue where a task contained a loop. It would make many requests to a third-party API endpoint. Within the documentation of this , like many others, was a rate limit of 2 requests per second. CRON API Previous we had a solution that looked something like this, which I have seen in a lot of other places: } foreach ( $productVariants as $variant ) { sleep( 2 ); // avoid rate limit sendRequestToAPI( $variant ->id); Very simple, pause for an adequate amount of time ﻿to avoid hitting the rate limit. Sometimes such operations might have a loop that iterates over a dataset that could contain tens of thousands of entries. It already takes a long time to run, so every wasted millisecond can add up and can make a script longer than it needs to be. A waster half second over a thousand products is eight minutes! So one could appreciate how this could snowball. With a few adjustments, you can simply have the script wait a much more precise amount of time. Still keeping under the rate limit, but also not wasting any more time than necessary. foreach($productVariants as $variant) { $elapsedTime = microtime(true) - $timeStart; }

    $timeStart = microtime(true);

    sendRequestToAPI($variant->id);
} $timeBetweenEachRequest = 2 ; if (isset($timeStart)) { $ remainingTime = $timeBetweenEachRequest - $elapsedTime; sleep($ remainingTime * 1000000); // Convert seconds to microseconds In the example above we simply time how long the loop took to run, and wait the remaining amount of time necessary. ﻿By simply subtracting the time already spent executing the logic within the loop already, we avoid wasted time whilst still being safe from hitting our limits. I hope this little snippet helps someone out there. The lead image for this article was generated by HackerNoon's AI Image Generator via the prompt "software development".

A Saner Method for Keeping Under API Rate Limits

Dallas Johnson is Chief Engineer for the Alien Worlds metaverse Alien Worlds’ engineers are on a quest for smart contract efficiency Alien Worlds is the most popular , with over 240,000 unique active wallets connecting daily. Within the game, its most solicited smart contract action – mining – gets called up to 13,000,000 times a day, making it the most frequently executed smart contract action in the world across all smart contracts and all blockchains. web3 game Suffice it to say; scalability is one of the technical aspects that we pay particular attention to. Ensuring that the game runs smoothly and is ready to accept not thousands but millions of new users is a continuous challenge. Of course, building on a highly scalable like WAX helps: with 0.5 second block time and 8,000 transactions per second, it is particularly well-suited for gaming. However, even WAX can get submerged by a massive transaction flow. blockchain To save precious blockchain resources, we are constantly optimizing our smart contracts, making sure they are as efficient and lean as possible. This starts with quantitative analysis to identify areas for improvement and continues with designing and testing new solutions, such as simplifying smart contracts or breaking up actions. Searching for the hot spots To make the search for efficiency a continuous process, we have integrated a mechanism measuring the time each smart contract action takes into our development tools. It allows us to collect the runtime statistics of all smart contract actions each time we run our unit tests (testing the smallest pieces of code that can be logically isolated in a system). The results, measured in CPU microseconds, show us the sections of code which could be improved. This approach also allows us to quantitatively measure the performance impact of every change we make to the codebase at the end of the optimization process. Keep it simple Mining in Alien Worlds refers to a common action our players perform to strategically use a combination of NFTs to earn rewards in the form of the in-game token, Trilium. Mining rewards split between the user and the owner of the land on which they mine used to be sent via token transfer as part of every mining transaction. However, this obvious-at-first-glance method is, in reality, quite cumbersome; every mining action creates two token transfers, which in turn initiate 3-4 additional notification actions (a WAX chain peculiarity), and then those token transfers can trigger a smart contract on the recipient side. This chain reaction, common to so many smart contracts involving token transfers, could have been dismissed because it did not impact the game itself. However, every action, even lightweight, can add a lot of overhead to the blockchain and impact its performance if repeated millions of times. We experienced this firsthand when Alien Worlds started to gain momentum. Realizing how burdensome the inline token transfers could be prompted us to redesign the mining rewards distribution mechanism, removing token transfers altogether. Now, the contract lets mining rewards accumulate internally, allowing users to claim them in batches rather than after every mine action and transfer the mined tokens to their wallets themselves. By this change alone, the number of actions in each mining transaction dropped from over 10 to 2, increasing the overall smart contract’s efficiency by a staggering 25%. Breaking it up Sometimes efficiency gains can be achieved by breaking up the smart contract, especially if this allows us to reduce the number of times it gets called. For example, many – including our mining contract – send out messages with game details, allowing off-chain APIs to record and store them. This logging action is often a loopback, sent outside the contract and then calling back to it again (a valid action must call an existing smart contract action, and calling itself is often the easiest option). However, each time an action is called on any smart contract, even in a callback, it must first generate a new instance of that smart contract, including all its memory storage and instance variables. smart contracts In our effort to increase efficiency, we have isolated the logging, creating a tiny new notification contract with just a few lines of code to simply receive and log that action in the blockchain history. Now, instead of instantiating a whole new instance of the heavy mining contract, the logging action calls this new lightweight contract, reducing the temporary memory load on the blockchain. This fairly small engineering change allows us to save valuable blockchain resources on every mine. As the game scales up, the importance of small details increases exponentially. A tiny performance gain becomes a sizable efficiency boost when multiplied by 13,000,000 and an important contribution to the blockchain’s stability when multiplied by 13,000,000,000. With an unrelenting focus on smart contract efficiency, we are ready to welcome Alien Worlds’ next millions of users. Play now or visit to apply for a development grant. Alien Worlds Galactic Hubs

Scaling to Millions: Behind the World’s Most Solicited Smart Contract 

In the , we looked at the possibilities of writing tests for MySQL, and eventually created a script that provides the results in a structured way. previous article However, there are a number of refinements that can improve the test execution experience. One of them concerns unique values. Reasons for Using Unique Values It makes sense to generate unique values for some text fields for each run of the script for three main reasons: For fields requiring uniqueness To delete test data in tables without primary keys To delete test data in case of an error while executing the script In the first case, there is a possibility that the text data of the script will coincide with those already existing in the database being tested. In the second, you need a field that is an alternative to the primary key. In the third, due to the linearity of the script structure, if an error occurs at any of the stages, the script execution will be interrupted, and the test data deletion block will not be executed. In this case, the IDs of the created test records will be lost. There are also less obvious reasons for generating unique values. Let's consider the ways of their generation. Ways of Generation Based on date and time: SELECT DATE_FORMAT(NOW(6), '%Y%m%d_%H%i%s.%f'); In most cases, the tests are not run very frequently, then you can use the date and time with microsecond precision. Based on a random value: SELECT MD5(RAND()); Based on a universal unique identifier using a built-in database function: SELECT UUID(); Based on connection ID: SELECT CONNECTION_ID(); You can also use the connection ID to distinguish between the data generated by different connections executing the test script. The Length The length of a unique value can be limited using an appropriate format or function (such as uuid_short) and can be extended by concatenating two or more unique values. Based on the previous statement, it follows that these methods can be combined. The examples given allow us to achieve uniqueness, but it is still difficult to select records generated in this way. To do this, it is convenient to use a certain prefix, which will be constant, but it will not be contained in the real data in the database. This may be a combination of characters that is impossible (or unlikely) to type from the keyboard or the way in which real data gets into the database. For example, a prefix like this: set @prefix = '─TEST─'; It looks like a regular string with a dash, but when typed from the keyboard, a normal dash (or minus sign) has code 45, while here, the character code is 196. Currently, most text fields are stored in utf8 encoding (or similar, which supports a large set of characters), so it will not be difficult to find the desired combination. Actually, the Process As a result, unique values will be set by the commands: set @prefix = '─TEST─';
set @uid = CONCAT(@prefix, DATE_FORMAT(NOW(6), '%Y%m%d_%H%i%s.%f')); An example of a test string identifier: ─TEST─20230314_170915.645455 Eventually, to delete test data, it is enough to execute commands like this on all the tables where this data was created: set @prefix = '─TEST─%';
DELETE FROM users WHERE NAME LIKE @prefix; In the case of foreign keys, you can select IDs by prefix, and delete them: DELETE FROM user_items WHERE user_id IN (SELECT id FROM users WHERE name LIKE @prefix);
DELETE FROM users WHERE NAME LIKE @prefix; Some Limitations It should be noted that commands with a search by like, in most cases, will be executed without using indexes, i.e., lead to a full scan, which may affect performance. Therefore, it is reasonable to adhere to the following rules when deleting test data by prefix: Do not perform deletion every time you run the test, if possible. Perform deletions at specific intervals. Use the flag of correct completion of the test script, and perform the deletion only in case of an error. Restrict search with additional filters, e.g., by date (last week or day) Using these principles will help make the test data management process more efficient.

Generating Unique Values for Test Data in MySQL

The Moon has long fascinated humankind, inspiring countless myths, stories, and scientific inquiries. Our understanding of the Moon has shifted gears in recent years owing to several outer space missions that explored geology, surface, and, mineral resources. The idea of building a lunar economy has existed for some time now. To make it a reality, scientists will have to set up a communication and navigation infrastructure with a standardized way to determine time. NASA is currently in the early stages of developing , a vital initiative to create an internet network for the Moon. This project goes beyond enabling astronauts to take pictures; it is a crucial component of the program. The agency intends to build various infrastructures on and around the Moon, such as a , a new , and a lunar web service that can maintain connectivity between all elements. LunaNet Artemis human habitat space station “LunaNet is a framework of mutually agreed-upon standards, protocols, and interface requirements allowing future lunar missions to work together, conceptually similar to what we did on Earth for joint use of GPS and Galileo,” Javier Ventura-Traveset, ESA's Navigation Manager. explains LunaNet: Enhancing Connectivity and Empowering Missions at the Moon “As NASA establishes a sustained lunar presence on the Moon, creating a robust infrastructure becomes increasingly important. The LunaNet communications and navigation architecture is extensible and flexible,” D.J. Israel, Exploration and Space Communications Projects Division Architect in a conference paper titled “LunaNet: a Flexible and Extensible Lunar Exploration Communications and Navigation Infrastructure.” states He further explains how LunaNet will provide three types of services: networking, position, timing and navigation, and science utilization. Users, both human and robotic, will experience network functionality similar to that experienced on Earth. The ESA(European Space Agency) opened an last week,  inviting private companies to help with its program, aimed at launching  three or four satellites to be “carried into lunar orbit by a space tug and deployed one by one, to form a constellation of lunar satellites.” The satellite constellation's orbits will be strategically optimized to encompass the lunar south pole, which has been designated as the primary landing site for  NASA’s future Artemis missions. application portal “Moonlight” What is ESA’s Moonlight initiative? “Many of the proposed lunar missions will require reliable navigation and telecommunication capabilities. Building these capabilities independently would be inefficient and costly due to complex, recurrent developments,” ESA’s announcement blog post titled “Lunar Economy Applications” . read Not only ESA, but NASA has also adopted public-private models for providing space services, such as and to the International Space Station. These partnerships have also been established for future endeavors, including the and to the lunar surface. delivering cargo transporting crew delivery of scientific experiments astronauts NASA has invested $1.2 billion in to develop a human landing variant of to land American astronauts on the Moon under Artemis III,” marking humanity’s first return to the lunar surface in more than 50 years.” SpaceX Starship Of the total investment in space over the last decade, which , 99% was directed toward the development of satellites and their launch vehicles for Earth's orbit. The emerging markets of space habitats, satellite servicing, and the fledgling lunar market accounted for $3.3 billion of this investment. amounted to $272 billion The US-based space agency anticipates that the lunar network will expand along with the increase in activity on and around the Moon. It projects that with the establishment of the Lunar Gateway, its newly planned lunar space station, the Moon will require approximately 210 megabits per second and 4.6 terabytes per day, as stated in a from the agency in the previous year. As NASA prepares for a human mission to Mars around 2035, it foresees the Moon requiring as much as 950 megabits per second and 12 terabytes per day. presentation "In 2021 there were 13 landers, orbiters, and rovers on and around the Moon," Kelly Larson, CEO of Aquarian Space, in the statement. "By 2030, we will have around 200, creating a multibillion-dollar lunar economy.” said NASA is devising a plan to enhance its services by launching lunar satellites that will interconnect with each other and eventually link back to the communication infrastructure on Earth. will be especially beneficial for the challenging-to-reach parts of the Moon, such as the far side and the lunar south pole, which the agency aims to explore with the aid of astronauts in the future. Furthermore, the agency intends to establish a range of ground stations that will function like cell towers on the Moon's surface. The idea is to create a lunar communication system that operates similarly to an internet network rather than a phone line. The relay spacecraft Private companies, including Aquarian Space and Nokia, are contributing to the development of lunar internet infrastructure. Aquarian Space to launch its initial lunar communication satellites by Q2 2025, while Nokia is collaborating with NASA on a 4G cellular network for the Moon. Nokia a contract of over $14 million from NASA and is scheduled to deliver its first base station and radio equipment via a SpaceX rocket next year. plans has won Accurate navigation depends majorly on precise timekeeping as it is crucial in determining the exact location of a . The satnav receiver measures the time it takes for multiple satellite signals to reach it and converts this time measurement into an estimate of distance. satnav receiver Currently, every new expedition to the Moon is conducted on Earth's timeline, with the assistance of deep space antennas to ensure that the spacecraft's clocks are aligned with terrestrial time and to enable communication between the spacecraft and Earth. However, this approach will not be viable in the future lunar environment. There are over planned in the coming decade. These missions will not only occur simultaneously on or around the Moon, but they will also frequently interact with each other. This interaction may involve relaying communications, conducting joint observations, or carrying out rendezvous operations. 250 lunar missions Considering this, space agencies have initiated discussions on how to measure time on the Moon. These discussions began with a meeting at ESA's ESTEC technology center in the Netherlands in November of last year. "During this meeting at ESTEC, we agreed on the importance and urgency of defining a common lunar reference time, which is internationally accepted and towards which all lunar systems and users may refer to. A joint international effort is now being launched towards achieving this,” ESA navigation system engineer Pietro Giordano. said “Timekeeping and time distribution are essential for navigation performance and for maintaining synchronization across multiple assets. Time knowledge significantly impacts observation accuracy. Inaccuracies and differences among sources used in measurement time-tagging impart offsets relative to the true orbit location,”  states the conference paper. Observations commonly used in radio and optometric technologies include one-way and two-way range and Doppler measurements. These observations heavily rely on precise time-stamping, which must be referenced to a standardized time scale such as International Atomic Time (TAI). Establishing a universal lunar time presents a unique challenge as clocks on Earth and the Moon operate at different speeds due to the varying gravitational fields. As such, the form that an official lunar time would take is not clear. One possibility is that lunar time could be based on a clock system that is designed to synchronize with Coordinated Universal Time (UTC). Alternatively, lunar time could be entirely independent of Earth time, resulting in the creation of a unique time system specifically designed for the Moon. According to , a lunar clock would gain approximately 56 microseconds or millionths over 24 hours. The exact rate will be a function of their position on the moon, ticking differently on the lunar surface and orbit. Jeff Gramling's estimation Other than technical standpoints, geopolitics will play a major role in building the lunar economy. The entity that shapes the internet infrastructure on the Moon will wield significant influence, as it will have the authority to establish guidelines for how network providers collaborate and allocate lunar web addresses. Despite the U.S.'s efforts to bring other nations and space agencies into alignment with its , a set of international principles aimed at guiding lunar and outer space exploration and development, there is no assurance that all parties will agree to its terms. Notably, Russia and China are collaborating on their own . Artemis Accords lunar initiatives For effective collaboration, the international community must agree upon a shared "selenocentric reference frame," similar to the International Terrestrial Reference Frame used on Earth, which allows for accurate measurement of precise distances between points across the Moon. Such customized reference frames are crucial components of modern GNSS (Global Navigation Satellite Systems). Once the scientists are able to move past the challenges, the construction of stations will be open to astronauts’ stay and will be resupplied with periodic Artemis launches. Numerous uncrewed missions will also be in place – each Artemis mission alone will release numerous lunar – and ESA will be putting down its Gateway CubeSats . Argonaut European Large Logistics Lander

The Future on the Moon: How LunaNet and Moon Standard Time will Shape the Lunar Economy

চাঁদের ভবিষ্যত: লুনানেট এবং চাঁদের মানক সময় কীভাবে চন্দ্র অর্থনীতিকে রূপ দেবে

Die Zukunft auf dem Mond: Wie LunaNet und die Mond-Standardzeit die Mondwirtschaft prägen werden

Futuro en la Luna: cómo LunaNet y el horario estándar lunar darán forma a la economía lunar

L'avenir sur la Lune : comment LunaNet et l'heure normale de la Lune façonneront l'économie lunaire

चंद्रमा पर भविष्य: कैसे लूनानेट और चंद्रमा मानक समय चंद्र अर्थव्यवस्था को आकार देंगे

月の未来: LunaNet と月標準時が月の経済をどのように形成するか

달의 미래: LunaNet과 달 표준시가 달 경제를 어떻게 형성할 것인가

Futuro na Lua: como o LunaNet e o horário padrão lunar moldarão a economia lunar

Ay'daki Gelecek: LunaNet ve Ay Standart Saati Ay Ekonomisini Nasıl Şekillendirecek?

Tương lai trên Mặt trăng: LunaNet và Giờ chuẩn Mặt trăng sẽ định hình nền kinh tế Mặt trăng như thế nào

月球上的未来：LunaNet 和月球标准时间将如何塑造月球经济

When I first encountered the book (GTD), by David Allen, I was blown away. The personal productivity method described in the book—also called Getting Things Done—clicked with me immediately, and I started implementing it in my daily life. The result was fast: after only a few days, I observed an increase in focus and a decrease in work-related stress. I can’t measure it, but I’m sure this book helped me save thousands of hours. Getting Things Done Personal productivity is a life-long learning process. Over the years, I’ve tried different methods or adapted existing ones to make the most of my working time, but each new technique reminds me of some aspects of the GTD method. I’ve also seen people struggle with their personal organization until they understand the concepts of this book. In this post, I will introduce you to GTD and show you why it’s still the top-ranking productivity method. Why you need an organization method The average worker is interrupted between 50 and 60 times per day. Each one of these interruptions creates an input that needs to be understood and sometimes transformed into a task. If we all receive so many inputs, why do some people seem to be better at turning them into action? The secret lies within their personal organization system. Feeling overwhelmed by those inputs is natural, but it’s also your responsibility to organize them. Without a way to capture them, you either let them overwhelm you or you forget them. In both cases, you will add stress due to too many tasks or unhappy colleagues. A good organization system will help you to capture inputs as they come and organize them so they don’t take over your day. If inputs are sorted and stored somewhere safe, you can always have a look at them later. With more time to spend on tasks and less time spent on interruptions, you can finally get this big task done in time. And once it’s done, you can rely on your system to tell you what to do next. How to use Getting Things Done According to David Allen, Getting Things Done is “ .” It’s a framework to organize your life and work systematically. To put his method into practice, David Allen developed a simple logic tree diagram shown below. I adapted it to streamline some of its components, but the original is very similar and can be found in the book. a personal productivity methodology that redefines how you approach your life and work The 5 principles of GTD For me, the genius of this method starts at the top, with the “stuff.” David Allen built an organization system where all kinds of inputs are treated the same way. By moving these inputs along the logic tree, you sort them into specific places that can be used in the corresponding context. At first, it can seem a bit daunting, but after some days of practice, this logic tree becomes second nature. The GTD method can be divided into 5 steps: Capture “Stuff” should be captured as soon as it comes to your attention. “Stuff” is coming from all directions: emails, instant messaging apps like Slack or Microsoft Teams, notes on your desk, phone calls, meetings, etc. Even ideas you got in the shower are “stuff” that needs to be captured as soon as possible (we all know how fast these ideas go away). To capture them, you will need a system of inboxes I will explain later. Clarify All “stuff” needs some clarification. Pausing to understand an input is an easy way to get the most out of it. This step can be as fast as a few seconds or take a bit longer for more complex inputs. The logic tree described above uses a series of question to clarify each input: “ ”: This is an informal question and usually takes a microsecond. The goal is to make you think about each “stuff” more deeply and understand its nature. It’s also helping you to clarify whether or not an input is worth your time. What is it? “ ”: You can’t always have a direct action linked to an input—sometimes the information doesn’t need a follow-up action. This is the first junction point in the clarification of “stuff.” Non-actionable inputs will be sorted in a sub-system I will describe below. Is it actionable? “ ”: If an input is actionable, you need to ask what the next action is. The answer will give you a task you can then organize. What’s the next action? “ ”: Some of the “next actions” are more complex than they seem and could lead to a multi-step project. In this case, they should be organized in a project system, described later in this post. Is it part of a multi-step project? “ ”: This is a critical part of the GTD method. When designing the system, David Allen was conscious that an organization system with too much maintenance is overkill. The two-minute rule fixes this issue. If a task takes less than two minutes to complete, you better do it right away. The effort required to organize it in a system will cost you more than the time it takes to complete it! Will it take less than two minutes? Organize With a clear picture of where your attention is needed, you can park the “stuff” in the right place, where you are sure to find it when you need it. For more details on how to organize your system, refer to the next step. Review Frequently reviewing your work is key to keep the system running. Priorities change, and reflecting on what’s left on your plate is important in order to keep your system up to date. GTD uses a system of daily and weekly reviews: : At the end of each day, take five to ten minutes to reflect on what you have done during the day and reorganize priorities for the next day (like uncompleted tasks or new priorities). You can also keep a daily log—it’s a great way to create a feeling of completion, and it helps for your weekly review. Daily reviews : Somewhere in your week, you need a deeper time to entirely review your system. The best time is typically on Friday before leaving the office or over the weekend. During the weekly review, you look at what has been done during the week (your daily log could be a great place for that) and prepare for the coming week. To do so, you should review your entire system. Update your project plans and next actions and select some for the coming week. Assign a day to each of these tasks so that when you start your week, you have a clear picture of what to do. Weekly reviews Engage This system would not be called Getting Things Done without involving a lot of action! When your personal organization system is running smoothly, you have more time to do the things you want. The weekly review also helps a lot to keep your priorities up to date. A lot of work-related stress is due to the impression of urgency with tasks. GTD helps you remove this feeling by gaining a clear idea of what to do and when to do it. Without the constant uncertainty, you can find the time to focus on a task you selected for yourself. How to set up your organization system To make the system work, you first need to build a few “boxes” to store your “stuff”: : You need inboxes to capture the “stuff.” You should already have used some of them, like your email inbox. What most people miss is that software inboxes are meant to be temporary storage places. It’s also common to have several inboxes, but to work, they need to be consolidated in one organized system. With GTD, you can use one or several inboxes as long as they are treated the same way, meaning that every bit of information you collect in them goes through the logic tree. Here are the inboxes I use: Inbox(es) : I use my email inbox as temporary storage to receive emails. Once I read them, the information they contain goes through the GTD logic tree and the email is archived (that’s called the zero inbox method). Email inbox : I use the inbox in my to-do list app ( ) to capture ideas, personal tasks, or verbal inputs from meetings or chats. It’s also accessible on my phone, so I can add an input at any time. To-do list inbox Things 3 : I treat the whole app as an inbox—every message in there is an input to be sorted, like an email. Slack messages : We use this app with my team to manage our projects (it’s similar to Asana or Trello). The built-in communication system notifies you if someone mentions you on a project note or tags you somewhere. Each notification I receive is an input as well and must be sorted with the GTD method. Monday.com notifications : Sometimes you want to keep some information for later, like a meeting note or an article you read online. That’s how the reference system comes in. and are great apps to use to build a reference system. Some people also like to keep an analog reference system, and the book goes into detail on how to create one. I keep everything digital as it’s easily searchable and accessible. Your reference system is like a second brain. It’s also used to keep notes on things you read in books or heard in a podcast, for example. I recommend keeping a system that requires as little maintenance as possible. Using tags is a good way to keep everything centered in one place without having to manually sort into folders. Reference Evernote Notion : Some “stuff” may be relevant, but not actionable right now. In those cases, it’s recommended to keep them in a special list called “someday/maybe.” I keep this in my to-do list. What’s crucial with this list is to remember to review it regularly. It’s a great place for fresh ideas you may have forgotten. Someday/maybe : Trashing unnecessary emails or messages is the best way to remove them from your mind. Don’t forget to press the trash button to declutter your system. Trash : Multi-step projects require their own system. It’s important to break down big projects into smaller bits that can be done independently. For those, a dedicated box is important. You can use a project management software (good for teams), a to-do list with project folders, or even a notebook. Project plans : It’s frustrating to delegate and task and realize weeks later that it hasn’t been done. The waiting list can solve your problem and allow you to delegate tasks in confidence. Delegation needs follow-up, and the waiting list keeps track of everything you’ve delegated. Like inboxes, you can have different waiting lists depending on how you work. I keep a waiting label in my emails and in my to-do list, for example. Every Thursday, I go through them and follow up if the task hasn’t been done. Waiting list : This is a no-brainer, as most of us are already using a digital or analog calendar. GTD makes use of it for “stuff” that is time-related. Calendar : This is “stuff” going on your to-do list. An important lesson I learned from GTD is to put due dates on a task only when it has a real deadline (meaning something bad will happen if it’s not done on time). In all other cases, the review system will help you keep your to-do list without a due date, which is a great way of reducing your stress. Next actions With those different systems set up, it’s easy to keep consistent organization with GTD. GTD is everywhere If you are still not convinced about the benefits of GTD, let me show you how it became ubiquitous in our work environment. Several popular productivity methods are inspired by GTD and use a similar logic tree to organize inputs and chose what to do next: : The key concept is to use your inbox as temporary storage and then sort, archive, or delete emails. It’s basically GTD applied to email. Inbox Zero : It’s an analog alternative to GTD (using a paper notebook and a pen). Similarly to GTD, the Bullet Journal method offers an organization system to sort all your inputs in different buckets called “collections.” Like GTD, it uses a recurring review principle to keep the system up to date (it’s great for people struggling with digital tools). Bullet Journal Even older techniques like Agile (or its derivatives like Scrum) can be applied more easily by using GTD. Agile uses a similar concept: collecting data on a project, planning the different steps, and then doing them in a short cycle called “sprint.” At the end of a sprint, the team can review the project and start preparing the next cycle. It’s very similar to a GTD workflow, but applied to a team instead of an individual. GTD is also built in the tools we use. Most of the apps we use for work use an inbox, an archive, and a trash function. GTD is everywhere, and ignoring it could cost you a lot of time. Make GTD part of a bigger framework The key of a good productivity system is consistency. GTD is sometimes criticized for being too complex to implement in your daily life. I agree that strictly following the GTD methodology can be too much, but you can already achieve a lot with the basics. For instance, I don’t use the “context” option described in the book. David Allen explains that you can tag each “next action” with a context label like “home/work,” “on the phone/in car,” or “low energy” / “high energy.” My work-life balance and agenda are well-structured, and my weekly review is enough to plan tasks into contexts without having to keep an active tag system. I chose to remove it from my GTD after realizing that it was taking too much time to maintain. Your GTD can evolve in its own customized version as well. GTD gives you the power to clear the inputs you receive in an organized way, but if you let inputs interrupt you all the time, your day will still look like a mess. On top of GTD, you can use other techniques that will let you focus on the “next actions.” I recommend three methods that work well with GTD: : GTD is solving the problem of to sort your “stuff;” asynchronous communication answers to sort it. And the answer is simple: once or twice a day, no more. By batching your communication time, you can free the rest of your day for deep work. if you want to go into more details. Asynchronous communication how when I wrote a post about it : If used properly, asynchronous communication will give you back the time you need to complete your “next actions.” But how do you choose what to do next? The 1-3-5 rule is a great way to select your next actions. The rule says that every day, you should select: The 1-3-5 rule 1 big task that takes several hours of focus 3 medium tasks that take five to thirty minutes each 5 small tasks that take under five minutes each, mainly errands “ ”: American author Mark Twain once said, “ at a live frog first thing in the morning and nothing worse will happen to you the rest of the day.” The “frog” is the big task in your 1-3-5 rule. It’s the most difficult one because it will take time and focus. To make it easier, tackle this big task first thing in the morning. Don’t check your emails or the latest news on your phone when you come to the office—just sit down and eat that frog. It works, believe me (disclaimer: I write all my blog posts first thing in the morning). Eat that frog E With these additional techniques under your belt, you should be equipped to regularly reach a state of flow where your concentration is at its highest level. Stress related to deadlines or fake due dates should also become a bad memory. Always feel free to experiment with new techniques or revert to something that worked better in the past. Getting Things Done is the backbone of your productivity system. Don’t wait to try it! Photos by and Viktor Talashuk Bernard Hermant This article was originally published on my Substack on November 6th, 2020. I since decided to migrate all my English content to Hackernoon (because it’s cool). 🇫🇷 🇫🇷 If you are looking for similar content in , you can visit my personal blog at French https://joachimeeckhout.com/ Also published . here

The Only Productivity Method You'll Ever Need Is 'Getting Things Done'

El único método de productividad que necesitará es "hacer las cosas"

La seule méthode de productivité dont vous aurez besoin est de « faire avancer les choses »

उत्पादकता का एकमात्र तरीका जिसकी आपको हमेशा आवश्यकता होगी वह है 'काम पूरा करना'

あなたが必要とする唯一の生産性メソッドは「物事を成し遂げる」ことです

O único método de produtividade que você precisará é 'fazer as coisas'

Phương pháp năng suất duy nhất mà bạn cần là 'Hoàn thành công việc'

你唯一需要的生产力方法就是“把事情做好”

Stories, News & Mentions about MICROSEC

Microsec’s Stories on HackerNoon