StarTree

startree

What data management in 2024 and beyond will look like hangs on one question. Can open data formats lead to a best-of-breed data management platform? It will take Interoperability across clouds and formats, as well as on the semantics and governance layer. Sixth Platform. Atlas. Debezium. DCAT. Egeria. Nessie. Mesh. Paimon. Transmogrification. This veritable word soup sounds like something that jumped out of a role-playing game. In reality, these are all terms related to data management. These terms may tell us something about the mindscape of people involved in data management tools and nomenclature, but that’s a different story. Data management is a long-standing and ever-evolving practice. The days of “Big Data” have long faded and data has now taken a back seat in terms of hype and attention. is the new toy people are excited about, shining a light on AI and machine learning for the masses. Generative AI Data management may seem dull in comparison. However, there is something people who have been into AI before it was cool understand well, and people who are new to this eventually realize too: AI is only as good as the data it operates on. There is an evolutionary chain leading from . Some organizations were well aware of this a decade ago. Others will have to learn the hard way today. data to analytics to AI Exploring and understanding how, where, and when data is stored, managed, integrated and used as well as related aspects of data formats, interoperability, semantics, and governance may be hard and unglamorous work. But it is what generates value for organizations, along with -like word soups. RPG Peter Corless and Alex Merced understand this. Their engagement with data goes back to way before their current roles as Director of Product Marketing at and Developer Advocate at , respectively. We caught up to talk about the state of data management in 2024, and where it may be headed. StarTree Dremio https://www.youtube.com/watch?v=Meq247f31-A&embedable=true The sixth data platform For many organizations today, data management comes down to handing over their data to one of the “Big 5” data vendors: , Azure, and , plus Snowflake and Databricks. However, analysts David Vellante and George Gilbert believe that the need of modern data applications coupled with the evolution of open storage management may lead to the emergence of a “sixth data platform”. Amazon Microsoft Google The notion of the “sixth data platform” was the starting point for this conversation with Corless and Merced. is that open data formats may enable interoperability, leading the transition away from vertically integrated vendor-controlled platforms towards independent management of data storage and permissions. The sixth data platform hypothesis It’s an interesting scenario and one that would benefit users by forcing vendors to compete for every workload based on the business value delivered, irrespective of lock-in. But how close are we to realizing this? To answer this question, we need to examine open data formats and their properties. In turn, to understand data formats a brief historical overview is needed. https://www.youtube.com/watch?v=j7s8xSTxkYk&embedable=true We might call this table stakes, and it’s not just a pun. Historically, organizations would resort to using data warehouses for their analytics needs. These were databases specifically designed for analytics, as opposed to transactional applications. Traditional data warehouses worked, but scaling them was an issue as Merced pointed out. Scaling traditional data warehouses was expensive and cumbersome because it meant buying new hardware with storage and computing bundled. This was the problem Hadoop was meant to solve when introduced in the early 2010s, by separating storage and compute. Hadoop made scaling easier, but it was cumbersome to use. This is why an SQL interface was eventually created for Hadoop via , making it more accessible. Hive used metadata and introduced a protocol for mapping files, which was what Hadoop operated on, to tables, which is what SQL operates on. That was the beginning of open data formats. Apache Hive Data lakes, data lakehouses, and open data formats Hadoop also signified the emergence of the data lake. Since storage was now cheap and easy to add, that opened up the possibility of storing all data in one big Hadoop store: the data lake. Eventually, however, , which is the realm the “Big 5” operate on. Hadoop and on-premise compute and storage gave way to the cloud “The idea of a data lakehouse was, hey, you know what? We love this decoupling of compute and storage. We love the cloud. But wouldn’t it be nice if we didn’t have to duplicate the data and we could just operate over the data store you already have, your data lake, and then just take all that data warehouse functionality and start trying to move it on there?”, as Merced put it. However, as he added, that turned out to be a tall order, because there were elements of traditional data management missing. Other cloud-based data management platforms introduced notions like automated sharding, partitioning, distribution, and replication. That came as part of the move away from data centers and file systems towards the cloud and APIs to access data. But for data lakehouses, these things are not a given. This is where open data formats such as and come in. They all have some things in common, such as the use of metadata to abstract and optimize file operations and being agnostic about storage. However, Hudi and Delta Lake both continued to build on what Hive started, while Iceberg departed from that. Apache Iceberg, Apache Hudi, Delta Lake https://www.youtube.com/watch?v=043GGYTZCbk&embedable=true A detailed open data format comparison is a nuanced exercise, but perhaps the most important question is: Does interoperability exist between open data formats, and if yes, on what level? For the sixth data platform vision to become a reality, Interoperability should exist among clouds and formats, as well as on the semantics and governance level. Interoperability of the same format between different cloud storage vendors is possible, but not easy. It can be done, but it’s not an inherent feature of the data formats themselves. It’s done on the application level, either via custom integration or by using something like Dremio. And there will be egress and network costs from transferring files outside the cloud where they reside. Interoperability of different formats is also possible, but not perfect. It can be done by using a third-party application, but there are also a couple of other options like or . Neither works 100%, as Delta Uniform batches transactions and OneTable is for migration purposes. Delta Uniform OneTable As Merced noted, creating a solution that works 100% would probably incur a lot of overhead and complexity as metadata would have to be synchronized across formats. Merced thinks these solutions are motivated at least partially by Iceberg’s growth and the desire to ensure third-party tools can work with the Iceberg ecosystem. To federate or not to federate? What’s certain is that there’s always gonna be sprinkles of data in other systems or multiple clouds. That’s why federation and virtualization that can work at scale is needed, as Corless noted. Schema management is complicated enough on one system, and trying to manage changes across three different systems would not make things easier. Whether data is left where it originally resides and used via federated queries, or ingested in one storage location is a key architectural question. As such, there are tradeoffs involved that need to be understood. Even within federation, there are different ways to go about it – Dremio, GraphQL, Pinot, Trino, and more. “How do data consumers want to size up these problems? How can we give them predictive ways to plan for these trade-offs? They could use federated queries, or a hybrid table of real time and batch data. We don’t even have a grammar to describe those kinds of hybrid or complex data products these days”, Corless said. Corless has a slightly different point of view, focusing on real-time data and processing. This is what StarTree does, as it builds on t. Pinot is a real-time distributed OLAP datastore, designed to answer OLAP queries with low latency. Apache Pino Corless also noted that the choice of data format depends on several parameters. Some formats are optimized for in-memory use, like Others like are optimized for disk storage. Apache Arrow. Apache Parquet There are attempts to have common representations of data both in-memory and on storage, motivated by the need to leverage tiered storage. Tiered storage may utilize different media, ranging from in-memory to SSD to some sort of blob storage. “People want flexibility in where and how they store their data. If they have to do transmogrification in real time, that largely defeats the purpose of what they’re trying to do with tiered storage. Is there something like the best, most universal format? Whenever you’re optimizing for something, you’re optimizing away from something else. But I’m very eager to see where that kind of universal representation of data is taking us right now.”, Corless said. Semantics and governance Regardless of where or how data is stored, however, true interoperability should also include semantics and governance. This is where the current state of affairs leaves a lot to be desired. As Merced shared, the way all table formats work is that the metadata has information about the table, but not about the overall catalog of tables. That means they are not able to document the semantics of tables and how they relate to each other. A standard format of doing that doesn’t exist yet. Merced also noted that there is something that addresses this gap, but it only works for Iceberg tables. is an open-source platform that creates a catalog, thus making tracking and versioning Iceberg tables and views possible. Project Nessie Nessie also incorporates elements of governance, and Merced noted that Nessie will eventually have Delta Lake support too. Databricks on its part offers which works with Delta Lake, but it’s a proprietary product. There is no lack of data catalog products in the market, but none of that can really be considered the solution to semantics and data interoperability. Unity Catalog Corless on his part noted that there a standard called DCAT. DCAT, which stands for Data Catalog Vocabulary, is an RDF vocabulary designed to facilitate interoperability between data catalogs published on the Web. DCAT was recently updated to v.3, it’s been around for over a decade, and it’s precisely aimed at interoperability. https://www.youtube.com/watch?v=Pb53L77_nS8&embedable=true The fact that DCAT is not used widely has probably more to do with vendors reinventing the wheel and/or aiming for lock-in, as well as clients not being more proactive in requiring interoperability standards than with DCAT itself, as per Corless. Unfortunately, it seems like what data is to AI, governance is to data for most organizations: a distant second at best. Forty-two percent of data and analytics leaders do not assess, measure or monitor their data and analytics governance, according to a . Those who said they measured their governance activity mainly focused on achieving compliance-oriented goals. 2020 Gartner survey The onset of . was an effort to standardize governance for data lakes leveraging DCAT and other metadata vocabularies. Today, however, both Data Lakes and Atlas seem to have fallen out of fashion. GDPR in 2018 marked the opportunity for governance to leverage metadata and semantics to rise up Apache Atlas A new project called was spun out of Atlas, aiming to address more than data lakes. Furthermore, there is another open standard for metadata and data lineage called and several lineage platforms that support it, including Egeria. Egeria OpenLineage Conclusion So, what is the verdict? Is the sixth data platform possible for data management in 2024 or is it a pipedream? Merced thinks that we’re already starting to see it. In his view, using open formats like Apache Iceberg, Apache Arrow, and Apache Parquet can create a greater level of interoperability. This interoperability is still imperfect, he noted, but it’s a much better state of things than in the past. Dremio’s thesis is to be an open data lakehouse, not just operating on the data lake, but across tools and data sources. Corless thinks that we’re going to see a drive toward clusters of clusters, or systems of systems. There’s going to be a reinforced drive towards the automation of integration. More like Lego bricks that snap together easily, rather than Ikea furniture that comes with a hex wrench. But for that to happen, he noted, we’ll need a language and a grammar so that systems and people can both understand each other. The ingredients for a sixth data platform are either there, or close enough. As complexity and fragmentation explode, a language and grammar for interoperability sounds like a good place to start. Admittedly, interoperability and semantics are hard. Even that, however, may be more of a people and market issue than a technical one. DCAT and OpenLineage are just some of the vocabularies out there. Even things as infamously hard to define as data mesh and data products have a vocabulary of their own – the . Data Product Descriptor Specification Perhaps then the right stance here would be cautious optimism. Also published . here

Data Management in 2024: Will Open Data Formats Shape a “Sixth Platform”?

Traffic-Based Anomaly Detection in Log Files

Error Rate-Based Anomaly Detection in Log Files

Keyword-Based Anomaly Detection in Log Files

A decentralized AI bears an amazing opportunity:  Do it better than traditional centralized AI such as Open AIs Chat GPT, Copilot Midjourney, or Runway, which squeeze out value derived from users’ data and IP they do not own. Better than extracting billions of dollars for its owners alone. With Web3 technology and attitude, we can make sure that users and IP contributors are more than just clients and/or the product. Let’s take a look at this from the perspective of AI-generated 3D Avatars from PFP IP. Why Do We Need an AI to be Decentralized? This quote meme-ing around for decades is now, in the AI hype cycle of the 2020s, more relevant than ever: "If you're not paying for the product, you are the product." Holders of valuable IP should be very aware of this, as AI is one of the major value extractors of the digital age, not only since we know what Jaron Lanier has to say on this. What has been happening with the intellectual property (IP) created for decades, by millions of people? It is fed into AI ‘black boxes’ that in return produce value for millions of people - while the earnings made from these services end up in the hands of a few shareholders. And what about the creative work in social networks - AI-algorithm-based value extraction happens with only a few exceptions completely with the creatives uninvited to the party. But this can be done very differently. Blockchain technology gives us all the tools we need to build an AI that fixes this imbalance, for tech that is currently disrupting almost any industry - no matter if it is image creation, coding, writing, etc. A decentralized AI in the shape of a DAO gives us an amazing opportunity to take care of everyone involved with the interplay of an AI entity: builders, users, contributors, and IP holders. This is not just a question of compensation, but also of power. While the alignment problem is on the radar of AI experts and critics, the power of the few who own an AI must be taken just as seriously to prevent a significant imbalance, with potentially negative consequences. This is How a Generative, Decentralized AI May Look Like Blockchain provides us with all we need to build something better. To build an AI for the users, owned by the users. This is because the users of a project are just as important as the creators, especially when it comes to AI which is reliant on human interaction. An AI DAO is a beneficial symbiosis, where every stakeholder has a benefit and a vote. Take the example of an AI service that is generating 3D avatars from PFP: Without PFP, no avatars. Without IP, no avatars. That is why users and IP creators should become part of the project, and one example to do this is to give them governance power, which is in web3 mostly done via token distribution. This is how the power of an AI can be effectively distributed. The Era of the Open Metaverse In a world, where the internet is embedded in our everyday, 3-dimensional life, the projection of our digital identity becomes dramatically more important. And so will 3D avatars and skins. With more than 250 open metaverse projects, thousands of games in the making, and Virtual and Augmented Reality (VR/AR/ MR) devices on the verge of mass adoption, with 34 million such devices estimated to be shipped in 2024, such a timeline seems highly likely. And AI is the obvious solution to make 3davatars highly accessible. And as even niche AI services are facing similar problems such as the ones mentioned above, we need to think about decentralization. Symbiosis of IP and Tech: Decentralized Generative AI An AI service likely has many shareholders: Developers, investors, users, and IP providers. As centralized AI services are not delivering a fair distribution of the generated wealth, in web3, there is a way to do it better. As one example of doing it differently, take the example of Beamit.space which is forming a DAO with all people involved, suggesting a more fair and balanced distribution of power and value: Users who are minting 3D Avatars from their IP
As the users are minting from their IP, they effectively provide crucial data for the datasets our AI is trained and optimized on.


Other supporters of the project This is a model if an incentivized engagement/social media campaign is already used by many decentralized projects. Investors At the pre-launch stage of almost any project, there is already capital demand, especially when it comes to AI with high render costs and skilled devs. Even in this case, though, the involvement of VCs can be avoided to achieve maximum decentralization, finding micro investors. With the example of the 3D avatars, this could be ideally the IP-holders themselves. Team & other ecosystem contributors Finally, developers and other team members, plus other entities contributing to the DAO ecosystem, have to be part of the DAO as well, to have a stake in the tool they build, and co-govern its evolution Conclusion: AI Should Be Decentralized for a Fair Power Distribution With such a model a governance system for a decentralized AI is most effective: Governed by the people who are involved with its creation, usage, optimization, and evolution. Let’s do it better than most Web2 corporations did before, and make sure AI is not in the power of a few. Disclosure: The Author is the founder of  STARBREEDER PTE. LTD, a Singapore-based startup that is currently building the decentralized AI Beamit AI. A decentralized AI bears an amazing opportunity:  Do it better than traditional centralized AI such as Open AIs Chat GPT, Copilot Midjourney, or Runway, which squeeze out value derived from users’ data and IP they do not own. Better than extracting billions of dollars for its owners alone. A decentralized AI bears an amazing opportunity:  Do it better than traditional centralized AI such as Open AIs Chat GPT , Copilot Midjourney , or Runway , which squeeze out value derived from users’ data and IP they do not own. Better than extracting billions of dollars for its owners alone. Open AIs Chat GPT Copilot Midjourney Runway With Web3 technology and attitude, we can make sure that users and IP contributors are more than just clients and/or the product. Let’s take a look at this from the perspective of AI-generated 3D Avatars from PFP IP. With Web3 technology and attitude, we can make sure that users and IP contributors are more than just clients and/or the product. Let’s take a look at this from the perspective of AI-generated 3D Avatars from PFP IP. Why Do We Need an AI to be Decentralized? Why Do We Need an AI to be Decentralized? This quote meme-ing around for decades is now, in the AI hype cycle of the 2020s, more relevant than ever: "If you're not paying for the product, you are the product." Holders of valuable IP should be very aware of this, as AI is one of the major value extractors of the digital age, not only since we know what Jaron Lanier has to say on this. Jaron Lanier What has been happening with the intellectual property (IP) created for decades, by millions of people? It is fed into AI ‘black boxes’ that in return produce value for millions of people - while the earnings made from these services end up in the hands of a few shareholders. And what about the creative work in social networks - AI-algorithm-based value extraction happens with only a few exceptions completely with the creatives uninvited to the party. But this can be done very differently. Blockchain technology gives us all the tools we need to build an AI that fixes this imbalance, for tech that is currently disrupting almost any industry - no matter if it is image creation, coding, writing, etc. A decentralized AI in the shape of a DAO gives us an amazing opportunity to take care of everyone involved with the interplay of an AI entity: builders, users, contributors, and IP holders. This is not just a question of compensation, but also of power. While the alignment problem is on the radar of AI experts and critics , the power of the few who own an AI must be taken just as seriously to prevent a significant imbalance, with potentially negative consequences. alignment problem is on the radar of AI experts and critics This is How a Generative, Decentralized AI May Look Like Blockchain provides us with all we need to build something better. To build an AI for the users, owned by the users. This is because the users of a project are just as important as the creators, especially when it comes to AI which is reliant on human interaction. An AI DAO is a beneficial symbiosis, where every stakeholder has a benefit and a vote. Take the example of an AI service that is generating 3D avatars from PFP: Without PFP, no avatars. Without IP, no avatars. That is why users and IP creators should become part of the project, and one example to do this is to give them governance power, which is in web3 mostly done via token distribution. This is how the power of an AI can be effectively distributed. The Era of the Open Metaverse In a world, where the internet is embedded in our everyday, 3-dimensional life, the projection of our digital identity becomes dramatically more important. And so will 3D avatars and skins. With more than 250 open metaverse projects, thousands of games in the making, and Virtual and Augmented Reality (VR/AR/ MR) devices on the verge of mass adoption, with 34 million such devices estimated to be shipped in 2024, such a timeline seems highly likely. And AI is the obvious solution to make 3davatars highly accessible. And as even niche AI services are facing similar problems such as the ones mentioned above, we need to think about decentralization. Symbiosis of IP and Tech: Decentralized Generative AI An AI service likely has many shareholders: Developers, investors, users, and IP providers. As centralized AI services are not delivering a fair distribution of the generated wealth, in web3, there is a way to do it better. As one example of doing it differently, take the example of Beamit.space which is forming a DAO with all people involved, suggesting a more fair and balanced distribution of power and value: Beamit.space Users who are minting 3D Avatars from their IP
As the users are minting from their IP, they effectively provide crucial data for the datasets our AI is trained and optimized on. Other supporters of the project Users who are minting 3D Avatars from their IP As the users are minting from their IP, they effectively provide crucial data for the datasets our AI is trained and optimized on. Users who are minting 3D Avatars from their IP Users who are minting 3D Avatars from their IP As the users are minting from their IP, they effectively provide crucial data for the datasets our AI is trained and optimized on. Other supporters of the project Other supporters of the project Other supporters of the project This is a model if an incentivized engagement/social media campaign is already used by many decentralized projects. Investors Investors Investors At the pre-launch stage of almost any project, there is already capital demand, especially when it comes to AI with high render costs and skilled devs. Even in this case, though, the involvement of VCs can be avoided to achieve maximum decentralization, finding micro investors. With the example of the 3D avatars, this could be ideally the IP-holders themselves. Team & other ecosystem contributors Team & other ecosystem contributors Team & other ecosystem contributors Finally, developers and other team members, plus other entities contributing to the DAO ecosystem, have to be part of the DAO as well, to have a stake in the tool they build, and co-govern its evolution Conclusion: AI Should Be Decentralized for a Fair Power Distribution With such a model a governance system for a decentralized AI is most effective: Governed by the people who are involved with its creation, usage, optimization, and evolution. Let’s do it better than most Web2 corporations did before, and make sure AI is not in the power of a few. Disclosure: The Author is the founder of  STARBREEDER PTE. LTD, a Singapore-based startup that is currently building the decentralized AI Beamit AI. Disclosure: The Author is the founder of STARBREEDER PTE. LTD , a Singapore-based startup that is currently building the decentralized AI Beamit AI. STARBREEDER PTE. LTD

This Is Why the Decentralization of AI Matters

এই কারণেই AI বিষয়গুলির বিকেন্দ্রীকরণ

Darum ist die Dezentralisierung der KI wichtig

Por eso es importante la descentralización de la IA

C'est pourquoi la décentralisation de l'IA est importante

यही कारण है कि एआई का विकेंद्रीकरण महत्वपूर्ण है

AIの分散化が重要な理由

이것이 AI의 분산화가 중요한 이유입니다.

É por isso que a descentralização da IA é importante

Вот почему децентрализация ИИ имеет значение

Yapay Zekanın Merkezi Olmaması Bu Nedenle Önemlidir

Đây là lý do tại sao sự phân cấp của các vấn đề AI

这就是为什么人工智能的去中心化很重要

*Credit of image: Unsplash (Tian-Shan Range, Wensu, Aksu, China) *Landsat, Sentinel, CBERS, and Amazonia satellites in the context of orbital-level multispectral imaging Nowadays, there are plenty of ways to access and use satellite imagery—here's a rundown of some of the main options I use in my personal projects, all of which are free :-) A (very) Quick Overview of Satellite Options First off, let's talk about the satellites and sensors themselves. There are many options today, each with unique capabilities. For agricultural and environmental analysis (what I do on a daily basis), two of the most commonly used satellite programs are Landsat and Sentinel. Each has been part of multiple missions with evolving sensors over the decades, offering increasingly rich data for various applications. Sentinel Sentinel-2, for example, is part of the European Copernicus program and specializes in multispectral imaging. As of now, there are three satellites in the Sentinel-2 series: Sentinel-2A, Sentinel-2B, and Sentinel-2C (the last one launched very recently, in September 2024, here). (Usually my first choice, as it can combine the best spatial, temporal and spectral resolutions). Landsat Landsat, managed by the U.S. Geological Survey (USGS) under the National Land Imaging Program, has been active since the 1970s. Its most recent mission is Landsat-9, which, like its predecessors, provides multispectral imaging for a variety of uses. With so many satellite programs, it's essential to choose based on your specific project needs. Some satellites focus on particular regions of the planet, while others have more global objectives, which leads me to also include other less conventional options here (on the international scene). Brazilian Satellites: CBERS and Amazonia. If you're interested in national programs, Brazil offers two noteworthy options: CBERS (China-Brazil Earth Resources Satellite): This Sino-Brazilian collaboration has launched several missions. The most recent, CBERS-4A, provides orbital-level multispectral imaging alongside earlier CBERS satellites with different sensors. Amazonia-1: Launched in 2021, this is Brazil's first entirely locally-developed satellite. While it offers multispectral imaging like the others, its spatial resolution is relatively lower. Its main goal is to monitor vast forested areas like the Amazon (as the name suggests). Future missions include Amazonia-1B and Amazonia-2. Of all the options above, I would say that the ideal scenario would be to use all available options to have the largest possible amount of imagery for your area. However, this approach raises at least two important questions to consider: What is the purpose of your analysis? (and whether a higher frequency of images is even necessary)
How will you handle processing so that all the different images are compatible for analysis? Where to Get Satellite Imagery Once you know your project's requirements and the sensors that meet them, there are several ways to access the data. Whether you need occasional imagery for reports or frequent data for large-scale analysis, here are the main platforms and acquisition methods: Official Data Portals Landsat: Earth Explorer
Sentinel: Copernicus Data Space Ecosystem
CBERS and Amazonia: INPE Image Catalog Other options include GloVis, for example. 2. Cloud Platforms Google Earth Engine (GEE): Access a huge catalog of satellite data, including Landsat, Sentinel, CBERS, and many other products. It includes a web-based code editor for quick use and APIs for JavaScript (there is a lot already ready to use in JS, so it's pretty much doable even if you're not familiar with the language—I, personally, am not) and Python.
Amazon Web Services (AWS): Landsat and Sentinel data are available on AWS. Check out the AWS Open Data Registry for details. APIs and Programmatic Access For more technical users, APIs and Python libraries are powerful tools: Sentinel API: Learn more
Landsat API: Learn more
CBERS and Amazonia API: INPE STAC Browser
Python Libraries: Libraries like pystac-client are great because they support multiple catalogs via the SpatioTemporal Asset Catalog (STAC) standard. With one library, you can access all these providers and much more. Third-Party Tools and Software Sentinel Hub: A commercial service offering easy access to Sentinel and other satellite imagery
Earth Data (NASA): A great option for additional datasets, similar to catalogs in GEE
GIS Software: Tools like QGIS and ArcGIS often integrate directly with satellite data repositories. In QGIS, you may need a plugin to fetch and process the data. Be sure to check the plugin's maintenance status for continuity. Final Thoughts The satellite and sensor landscape offers an incredible variety of options, but navigating them, especially in the beginning, can feel overwhelming. The good news? Once you identify your needs and the right sensors (lots of work to be done here!), there are plenty of tools and platforms—most of them free—to access the data and put it to work for your project. *Credit of image: Unsplash ( Tian-Shan Range, Wensu, Aksu, China) *Credit of image: Unsplash ( Tian-Shan Range, Wensu, Aksu, China ) Tian-Shan Range, Wensu, Aksu, China *Landsat, Sentinel, CBERS, and Amazonia satellites in the context of orbital-level multispectral imaging *Landsat, Sentinel, CBERS, and Amazonia satellites in the context of orbital-level multispectral imaging Nowadays, there are plenty of ways to access and use satellite imagery—here's a rundown of some of the main options I use in my personal projects, all of which are free :-) A (very) Quick Overview of Satellite Options A (very) Quick Overview of Satellite Options First off, let's talk about the satellites and sensors themselves. There are many options today, each with unique capabilities. For agricultural and environmental analysis (what I do on a daily basis), two of the most commonly used satellite programs are Landsat and Sentinel. Each has been part of multiple missions with evolving sensors over the decades, offering increasingly rich data for various applications. Sentinel Sentinel-2, for example, is part of the European Copernicus program and specializes in multispectral imaging. As of now, there are three satellites in the Sentinel-2 series: Sentinel-2A, Sentinel-2B, and Sentinel-2C (the last one launched very recently, in September 2024, here ). (Usually my first choice, as it can combine the best spatial, temporal and spectral resolutions). here Landsat Landsat, managed by the U.S. Geological Survey (USGS) under the National Land Imaging Program, has been active since the 1970s. Its most recent mission is Landsat-9, which, like its predecessors, provides multispectral imaging for a variety of uses. With so many satellite programs, it's essential to choose based on your specific project needs. Some satellites focus on particular regions of the planet, while others have more global objectives, which leads me to also include other less conventional options here (on the international scene). Brazilian Satellites: CBERS and Amazonia. If you're interested in national programs, Brazil offers two noteworthy options: Brazilian Satellites: Brazilian Satellites: CBERS (China-Brazil Earth Resources Satellite): This Sino-Brazilian collaboration has launched several missions. The most recent, CBERS-4A, provides orbital-level multispectral imaging alongside earlier CBERS satellites with different sensors. Amazonia-1: Launched in 2021, this is Brazil's first entirely locally-developed satellite. While it offers multispectral imaging like the others, its spatial resolution is relatively lower. Its main goal is to monitor vast forested areas like the Amazon (as the name suggests). Future missions include Amazonia-1B and Amazonia-2. Of all the options above, I would say that the ideal scenario would be to use all available options to have the largest possible amount of imagery for your area. However, this approach raises at least two important questions to consider: ideal use all available options What is the purpose of your analysis? (and whether a higher frequency of images is even necessary) How will you handle processing so that all the different images are compatible for analysis? What is the purpose of your analysis? (and whether a higher frequency of images is even necessary) purpose How will you handle processing so that all the different images are compatible for analysis? processing Where to Get Satellite Imagery Where to Get Satellite Imagery Once you know your project's requirements and the sensors that meet them, there are several ways to access the data. Whether you need occasional imagery for reports or frequent data for large-scale analysis, here are the main platforms and acquisition methods: Official Data Portals Official Data Portals Official Data Portals Landsat: Earth Explorer Sentinel: Copernicus Data Space Ecosystem CBERS and Amazonia: INPE Image Catalog Landsat: Earth Explorer Earth Explorer Sentinel: Copernicus Data Space Ecosystem Copernicus Data Space Ecosystem CBERS and Amazonia: INPE Image Catalog INPE Image Catalog Other options include GloVis , for example. GloVis 2. Cloud Platforms 2. Cloud Platforms Google Earth Engine (GEE): Access a huge catalog of satellite data, including Landsat, Sentinel, CBERS, and many other products. It includes a web-based code editor for quick use and APIs for JavaScript (there is a lot already ready to use in JS, so it's pretty much doable even if you're not familiar with the language—I, personally, am not) and Python. Amazon Web Services (AWS): Landsat and Sentinel data are available on AWS. Check out the AWS Open Data Registry for details. Google Earth Engine (GEE) : Access a huge catalog of satellite data, including Landsat, Sentinel, CBERS, and many other products. It includes a web-based code editor for quick use and APIs for JavaScript (there is a lot already ready to use in JS, so it's pretty much doable even if you're not familiar with the language—I, personally, am not) and Python. Google Earth Engine (GEE) Amazon Web Services (AWS): Landsat and Sentinel data are available on AWS. Check out the AWS Open Data Registry for details. AWS Open Data Registry APIs and Programmatic Access APIs and Programmatic Access APIs and Programmatic Access For more technical users, APIs and Python libraries are powerful tools: Sentinel API: Learn more Landsat API: Learn more CBERS and Amazonia API: INPE STAC Browser Python Libraries: Libraries like pystac-client are great because they support multiple catalogs via the SpatioTemporal Asset Catalog (STAC) standard. With one library, you can access all these providers and much more. Sentinel API : Learn more Sentinel API Landsat API : Learn more Landsat API CBERS and Amazonia API: INPE STAC Browser INPE STAC Browser Python Libraries: Libraries like pystac-client are great because they support multiple catalogs via the SpatioTemporal Asset Catalog (STAC) standard. With one library, you can access all these providers and much more. pystac-client Third-Party Tools and Software Third-Party Tools and Software Third-Party Tools and Software Sentinel Hub: A commercial service offering easy access to Sentinel and other satellite imagery Earth Data (NASA): A great option for additional datasets, similar to catalogs in GEE GIS Software: Tools like QGIS and ArcGIS often integrate directly with satellite data repositories. In QGIS, you may need a plugin to fetch and process the data. Be sure to check the plugin's maintenance status for continuity. Sentinel Hub: A commercial service offering easy access to Sentinel and other satellite imagery Sentinel Hub: Earth Data (NASA) : A great option for additional datasets, similar to catalogs in GEE Earth Data (NASA) GIS Software: Tools like QGIS and ArcGIS often integrate directly with satellite data repositories. In QGIS, you may need a plugin to fetch and process the data. Be sure to check the plugin's maintenance status for continuity. QGIS Final Thoughts The satellite and sensor landscape offers an incredible variety of options, but navigating them, especially in the beginning, can feel overwhelming. The good news? Once you identify your needs and the right sensors (lots of work to be done here!), there are plenty of tools and platforms—most of them free—to access the data and put it to work for your project.

How to Get Started with Free Satellite Imagery: Sentinel, Landsat, CBERS, and More

বিনামূল্যে স্যাটেলাইট ছবি দিয়ে কীভাবে শুরু করবেন: সেন্টিনেল, ল্যান্ডস্যাট, সিবিইআরএস এবং আরও অনেক কিছু

Cómo empezar a utilizar imágenes satelitales gratuitas: Sentinel, Landsat, CBERS y más

Ilmaisten satelliittikuvien käytön aloittaminen: Sentinel, Landsat, CBERS ja paljon muuta

Como comezar coas imaxes de satélite gratuítas: Sentinel, Landsat, CBERS e moito máis

無料の衛星画像を使い始める方法: Sentinel、Landsat、CBERS など

ວິທີການເລີ່ມຕົ້ນດ້ວຍຮູບພາບດາວທຽມຟຣີ: Sentinel, Landsat, CBERS, ແລະອື່ນໆ

Хиймэл дагуулын үнэгүй зургийг хэрхэн ашиглаж эхлэх вэ: Sentinel, Landsat, CBERS болон бусад

Jak zacząć korzystać z bezpłatnych obrazów satelitarnych: Sentinel, Landsat, CBERS i inne

د وړیا سپوږمکۍ انځورونو سره څنګه پیل کول: سینټینیل، لینډات، CBERS، او نور

Как начать работу с бесплатными спутниковыми снимками: Sentinel, Landsat, CBERS и другие

Jinsi ya Kuanza na Picha Zisizolipishwa za Setilaiti: Sentinel, Landsat, CBERS, na Mengineyo

Paano Magsimula sa Libreng Satellite Imagery: Sentinel, Landsat, CBERS, at Higit Pa

In Part 1 of this series, we went through the process of setting up and installing Appium, along with the necessary dependencies and drivers for mobile automation testing. Now that we have everything installed, let's take a closer look at what exactly we’ve set up and understand the role each component plays in our mobile test automation workflow. Understanding these components will give us a clearer picture of how Appium enables us to automate tests for both Android and iOS applications. 1. Appium Appium is the heart of our mobile automation setup. It acts as a bridge between the test scripts you write and the mobile device or simulator/emulator that the tests are being executed on. Think of Appium as a tool that translates your test scripts (which are written in programming languages like Java, Python, or JavaScript) into commands that the mobile operating system can understand. When we write a test case, Appium takes each action—such as tapping a button, swiping a screen, or entering text—and converts it into a sequence of commands that the mobile OS interprets. Appium is designed to support both Android and iOS, making it a cross-platform solution for mobile test automation. This means that we can write one test and run it across different devices and platforms with minimal changes to our code. 2. UiAutomator Driver The UI Automator driver is a powerful tool for interacting with the UI components on an Android device or emulator. It provides access to all the features needed to automate the interaction with Android app elements. This driver enables actions such as tapping buttons, scrolling through lists, entering text in forms, and verifying the presence of elements on the screen. One key benefit of the UI Automator Driver is its ability to interact with both the app's UI and system-level features, such as accessing notifications and interacting with native Android components. If you’re focusing on testing Android apps, the UI Automator Driver is a critical component of your setup. 3. XCUITest Driver XCUITest is a testing framework developed by Apple for iOS applications. When we need to interact with elements in an iOS app, we use the XCUITest Driver. Just like the UI Automator Driver does for Android, XCUITest translates the actions from your test script into commands that the iOS operating system can execute. XCUITest supports a wide range of iOS-specific features, allowing you to automate gestures, verify UI elements, and even test complex interactions such as push notifications or in-app purchases. Since iOS and Android are very different platforms, the XCUITest Driver provides a tailored approach to testing iOS apps, ensuring that your automation scripts can work seamlessly on Apple's devices. 4. Wait Plugin The Wait Plugin is an essential part of test automation, especially for mobile applications where delays or changes in the UI might cause timing issues. This plugin allows you to pause your test execution until a specific condition is met, such as waiting for an element (like a button or text field) to appear or become clickable. Without such a feature, your tests could fail prematurely, particularly in scenarios where elements load dynamically or after a delay (such as network latency or complex animations). By incorporating the Wait Plugin, you can avoid flaky tests—tests that fail intermittently because the necessary elements haven't loaded in time. The plugin ensures that the test doesn't proceed until it is sure the element is available for interaction, improving the reliability of your automated tests. 5. Appium Inspector Think of Appium Inspector as a visual tool to help you inspect the app's user interface during the test design phase. It provides a graphical interface that displays the app's components, such as buttons, text fields, and images, making it easy to identify elements you want to interact with. You can use the Appium Inspector to locate elements by inspecting their properties like their ID, class, or XPath. This tool is invaluable when designing your test scripts, as it allows you to visually explore the app’s UI and interact with specific elements without writing code immediately. With Appium Inspector, you can quickly determine the exact properties and attributes of UI components, and once you have identified the elements, you can use them in your test scripts to automate interactions. It significantly simplifies the process of writing and debugging automation tests. Conclusion In summary, the components we've installed: Appium, the UI Automator Driver, the XCUITest Driver, the Wait Plugin, and the Appium Inspector, each serve a unique purpose, but they work together to enable powerful mobile automation testing. Appium provides the foundation by connecting the test script with the device, while the drivers for Android and iOS (UI Automator and XCUITest) provide platform-specific functionality. The Wait Plugin ensures reliable test execution by managing timing issues, and the Appium Inspector makes it easier to identify and interact with UI elements. In Part 1 of this series, we went through the process of setting up and installing Appium, along with the necessary dependencies and drivers for mobile automation testing. Now that we have everything installed, let's take a closer look at what exactly we’ve set up and understand the role each component plays in our mobile test automation workflow. Understanding these components will give us a clearer picture of how Appium enables us to automate tests for both Android and iOS applications. Part 1 1. Appium Appium is the heart of our mobile automation setup. It acts as a bridge between the test scripts you write and the mobile device or simulator/emulator that the tests are being executed on. Think of Appium as a tool that translates your test scripts (which are written in programming languages like Java, Python, or JavaScript) into commands that the mobile operating system can understand. When we write a test case, Appium takes each action—such as tapping a button, swiping a screen, or entering text—and converts it into a sequence of commands that the mobile OS interprets. Appium is designed to support both Android and iOS, making it a cross-platform solution for mobile test automation. This means that we can write one test and run it across different devices and platforms with minimal changes to our code. 2. UiAutomator Driver The UI Automator driver is a powerful tool for interacting with the UI components on an Android device or emulator. It provides access to all the features needed to automate the interaction with Android app elements. This driver enables actions such as tapping buttons, scrolling through lists, entering text in forms, and verifying the presence of elements on the screen. One key benefit of the UI Automator Driver is its ability to interact with both the app's UI and system-level features, such as accessing notifications and interacting with native Android components. If you’re focusing on testing Android apps, the UI Automator Driver is a critical component of your setup. 3. XCUITest Driver XCUITest is a testing framework developed by Apple for iOS applications. When we need to interact with elements in an iOS app, we use the XCUITest Driver. Just like the UI Automator Driver does for Android, XCUITest translates the actions from your test script into commands that the iOS operating system can execute. XCUITest supports a wide range of iOS-specific features, allowing you to automate gestures, verify UI elements, and even test complex interactions such as push notifications or in-app purchases. Since iOS and Android are very different platforms, the XCUITest Driver provides a tailored approach to testing iOS apps, ensuring that your automation scripts can work seamlessly on Apple's devices. 4. Wait Plugin The Wait Plugin is an essential part of test automation, especially for mobile applications where delays or changes in the UI might cause timing issues. This plugin allows you to pause your test execution until a specific condition is met, such as waiting for an element (like a button or text field) to appear or become clickable. Without such a feature, your tests could fail prematurely, particularly in scenarios where elements load dynamically or after a delay (such as network latency or complex animations). By incorporating the Wait Plugin, you can avoid flaky tests—tests that fail intermittently because the necessary elements haven't loaded in time. The plugin ensures that the test doesn't proceed until it is sure the element is available for interaction, improving the reliability of your automated tests. 5. Appium Inspector Think of Appium Inspector as a visual tool to help you inspect the app's user interface during the test design phase. It provides a graphical interface that displays the app's components, such as buttons, text fields, and images, making it easy to identify elements you want to interact with. You can use the Appium Inspector to locate elements by inspecting their properties like their ID, class, or XPath. This tool is invaluable when designing your test scripts, as it allows you to visually explore the app’s UI and interact with specific elements without writing code immediately. With Appium Inspector, you can quickly determine the exact properties and attributes of UI components, and once you have identified the elements, you can use them in your test scripts to automate interactions. It significantly simplifies the process of writing and debugging automation tests. Conclusion In summary, the components we've installed: Appium, the UI Automator Driver, the XCUITest Driver, the Wait Plugin, and the Appium Inspector, each serve a unique purpose, but they work together to enable powerful mobile automation testing. Appium provides the foundation by connecting the test script with the device, while the drivers for Android and iOS (UI Automator and XCUITest) provide platform-specific functionality. The Wait Plugin ensures reliable test execution by managing timing issues, and the Appium Inspector makes it easier to identify and interact with UI elements.

Mobile App Test Automation - Getting Started With Appium Part 1B

If you have been avoiding mobile test automation because it sounds too complex or you are unsure where to start, fear not! When it comes to end-to-end testing for mobile apps, it can feel a lot like preparing for a trek through an unknown jungle. Whether you are looking to test an Android app or an iOS app, this guide will simplify the setup process and get you kicking with mobile app test automation. Without wasting much time, let’s dive into it. What is Appium? Appium is an open-source, free automation testing tool for mobile. It supports cross-platform testing, meaning it can test native apps and hybrid apps. Pre-requisite Before you begin, ensure you have met the following requirements: Download and Install the latest version of nodejs from the official site here
Download and Install Java Development Kit (JDK) from the official site here
XCode Installed
Android Studio Installed N/B: To verify you have installed node, in your terminal, write node -v which will tell you the current version installed. Installing Appium Open your terminal, and write this command npm install -g appium@2.11.4 to install Appium.
In your terminal write this command appium driver install uiautomator@3.7.2 to install the UI Automator driver.
In your terminal write this command appium driver install xcuitest@7.26.4 to install the XCUITest driver.
In your terminal write this command appium plugin install --source=npm appium-wait-plugin@2.0.3 to install a wait plugin.
Download, and install Appium Inspector from here. That’s it! You are good to go. To run Appium, simply write appium in your terminal, and the server loads up. Getting Started Now that we have installed Appium, we can now write test scripts to test mobile apps. But before we can write any code, we need an IDE. An IDE is a code editor that allows us to write scripts to test and build software. There are many IDEs out there, but for a quick start, we will be using IntelliJ IDE. You can download and install IntelliJ IDE here. You can make use of the community version. Once you have downloaded and installed IntelliJ IDE, open the software. Create a New Project on IntelliJ Click on new project.


Ensure that “Java” is selected as shown below.


Give the project a name.


Select the build as Maven.


Select the JDK installed or registered when you click the drop-down.


Once you are done, click on the Create button. You will be provided with a pom.xml file. Now, the pom.xml file is going to contain the necessary dependencies we need to use to run Appium. Open the pom.xml file, and add the following dependencies: <dependencies>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-java</artifactId>
        <version>7.13.0</version>
        <scope>test</scope>
    </dependency>

    <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.9.0</version>
            <scope>test</scope>
        </dependency>

    <dependency>
        <groupId>io.appium</groupId>
        <artifactId>java-client</artifactId>
        <version>8.6.0</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-remote-driver</artifactId>
        <version>4.24.0</version>
    </dependency>
</dependencies> Once you add the dependencies above in the pom.xml file, an “m” button may pop up by the top right corner, click on it so that it installs the dependencies. Conclusion So far, we have installed Appium with the related drivers and plugins we need to be able to successfully run Appium on our machine. We have also installed a code editor - IntelliJ. Finally, we have been able to create a new project on add dependencies in the pom.xml file. If you have been avoiding mobile test automation because it sounds too complex or you are unsure where to start, fear not! When it comes to end-to-end testing for mobile apps, it can feel a lot like preparing for a trek through an unknown jungle. Whether you are looking to test an Android app or an iOS app, this guide will simplify the setup process and get you kicking with mobile app test automation. Without wasting much time, let’s dive into it. What is Appium? Appium is an open-source, free automation testing tool for mobile. It supports cross-platform testing, meaning it can test native apps and hybrid apps. Pre-requisite Before you begin, ensure you have met the following requirements: Download and Install the latest version of nodejs from the official site here Download and Install Java Development Kit (JDK) from the official site here XCode Installed Android Studio Installed Download and Install the latest version of nodejs from the official site here here Download and Install Java Development Kit (JDK) from the official site here here XCode Installed Android Studio Installed N/B: To verify you have installed node, in your terminal, write node -v which will tell you the current version installed. N/B: To verify you have installed node, in your terminal, write node -v which will tell you the current version installed. Installing Appium Open your terminal, and write this command npm install -g appium@2.11.4 to install Appium. In your terminal write this command appium driver install uiautomator@3.7.2 to install the UI Automator driver. In your terminal write this command appium driver install xcuitest@7.26.4 to install the XCUITest driver. In your terminal write this command appium plugin install --source=npm appium-wait-plugin@2.0.3 to install a wait plugin. Download, and install Appium Inspector from here. Open your terminal, and write this command npm install -g appium@2.11.4 to install Appium. npm install -g appium@2.11.4 In your terminal write this command appium driver install uiautomator@3.7.2 to install the UI Automator driver. appium driver install uiautomator@3.7.2 In your terminal write this command appium driver install xcuitest@7.26.4 to install the XCUITest driver. appium driver install xcuitest@7.26.4 In your terminal write this command appium plugin install --source=npm appium-wait-plugin@2.0.3 to install a wait plugin. appium plugin install --source=npm appium-wait-plugin@2.0.3 Download, and install Appium Inspector from here . here That’s it! You are good to go. To run Appium, simply write appium in your terminal, and the server loads up. appium Getting Started Now that we have installed Appium, we can now write test scripts to test mobile apps. But before we can write any code, we need an IDE. An IDE is a code editor that allows us to write scripts to test and build software. There are many IDEs out there, but for a quick start, we will be using IntelliJ IDE. You can download and install IntelliJ IDE here . You can make use of the community version. here Once you have downloaded and installed IntelliJ IDE, open the software. Create a New Project on IntelliJ Click on new project. Ensure that “Java” is selected as shown below. Give the project a name. Select the build as Maven. Select the JDK installed or registered when you click the drop-down. Once you are done, click on the Create button. Click on new project. Click on new project. Ensure that “Java” is selected as shown below. Ensure that “Java” is selected as shown below. Give the project a name. Give the project a name. Select the build as Maven. Select the build as Maven. Select the JDK installed or registered when you click the drop-down. Select the JDK installed or registered when you click the drop-down. Once you are done, click on the Create button. Once you are done, click on the Create button. You will be provided with a pom.xml file. Now, the pom.xml file is going to contain the necessary dependencies we need to use to run Appium. Open the pom.xml file, and add the following dependencies: <dependencies>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-java</artifactId>
        <version>7.13.0</version>
        <scope>test</scope>
    </dependency>

    <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.9.0</version>
            <scope>test</scope>
        </dependency>

    <dependency>
        <groupId>io.appium</groupId>
        <artifactId>java-client</artifactId>
        <version>8.6.0</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-remote-driver</artifactId>
        <version>4.24.0</version>
    </dependency>
</dependencies> <dependencies>
    <dependency>
        <groupId>io.cucumber</groupId>
        <artifactId>cucumber-java</artifactId>
        <version>7.13.0</version>
        <scope>test</scope>
    </dependency>

    <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.9.0</version>
            <scope>test</scope>
        </dependency>

    <dependency>
        <groupId>io.appium</groupId>
        <artifactId>java-client</artifactId>
        <version>8.6.0</version>
    </dependency>

    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-remote-driver</artifactId>
        <version>4.24.0</version>
    </dependency>
</dependencies> Once you add the dependencies above in the pom.xml file, an “m” button may pop up by the top right corner, click on it so that it installs the dependencies. Conclusion So far, we have installed Appium with the related drivers and plugins we need to be able to successfully run Appium on our machine. We have also installed a code editor - IntelliJ. Finally, we have been able to create a new project on add dependencies in the pom.xml file.

Mobile App Test Automation - Getting Started With Appium Part 1

Getting Started with Data Analytics in Python Using PyArrow

Hi! If you read my previous article about why I chose Ruby, you know why I didn’t follow JS even though it was still deep in my memory after finishing the Calculator project and the whole Foundations from The Odin Project. Introduction to Ruby with The Odin Project I have to tell you that I couldn’t wait for that moment and was super excited to learn Ruby and to really see how Yukihiro “Matz” Matsumoto managed to make it so programmer-friendly, as many say. Getting Started: Ruby Installation and Documentation Firstly I had to install everything and it wasn’t difficult with TOP guide. After a short while, everything was set and ready to go. Firstly, I was introduced to Ruby documentation, which looked a little bit different from JS. (What would I expect, though?! It’s a different language! :D) It took me some time to get used to that. Especially when it comes to all methods and classes, at that point, I could tell the OOP lesson was approaching because it seemed different from what I was used to. Once you get the hang of everything, it becomes really easy to find the information you’re looking for. The next step was to learn what IRB is. It is a Ruby interactive model that lets you type your program in the console and run it without creating any files. It is a very useful thing that Rubyists use all the time, and I have liked it since the beginning as I didn’t need to open a browser to check my code every single time like I was doing with JS. Tackling the Basics: Data Types and Initial Challenges After I was sorted with all those, I was ready to tackle basic data types, and after a couple of Pomodoro sessions, my hype began to slow down. Why? It felt super boring! I felt as if I was doing JS from the beginning. I thought I’d done this before, so why waste time doing the same thing again? I understand that I am learning a new language, and I need to start from the basics, but c’moooon! Overcoming Boredom and Staying Motivated I’ll admit I was a LITTLE disappointed at first, but I had no choice but to stick to the plan and carry on. I knew it would get more exciting at some point, but I was only afraid not to lose motivation. Why would I? It took me already so long to get to this point that there is no chance I am stopping or changing my mind. I didn’t want to fight with my thoughts any longer at this point and just decided to put in more hours and focus harder as procrastination started to crippling in, especially between my Pomodoro sessions. The next lessons were about conditionals, loops, arrays, hashes, methods, and debugging. I cannot say it was the most exciting time, and at this time, I missed making some projects. I felt as if I was forgetting how to write code while learning how to write code. Strange, isn’t it? I hope that you know what I mean. Maybe there are more of you who have had the same struggles. I then approached Basic Ruby Projects, and this news made me happy. I was finally able to give it a try and solve some bigger exercises like I used with Java Script some time ago. Well, I can’t remember how long it took me to finish them, but it wasn’t too long before I got back to TOP lessons. From Basics to Object-Oriented Programming (OOP) The next one was object-oriented programming, which I couldn’t wait for. At that moment, I knew what it was going to be all about, more or less, and I was eager to try it. Moreover, I was really curious if it’s so difficult to grasp, as people sometimes say. Since then, I planned to push through the curriculum faster during the weekends, but something was always stopping me, like my car broke, and I had to buy another one as it wasn’t worth repairing my old one. Shortly after, my family wanted to visit me, and we were planning to go to Scotland for a week. I knew already that the time I wanted to spend on learning was limited that month. When we were there, I did my best to learn Ruby for 1 hour a day at least. After we came back and I dropped my family at the airport, my friend from Birmingham came to me for the weekend as he needed me to give him a hand fixing garden furniture he produced and sold to clients not far away from where I live. On top of that, my newly bought car decided to start leaking, and to save up money, I decided to fix it myself over another weekend. Guess what… By this point, I had already lost three weeks of productivity. I was a little bit mad and scared that I wouldn’t be able to get my routine going again, but fortunately, it seemed to start picking up nicely after I started again. I continued with OOP, which didn’t seem difficult. I could see the clear, organized approach, and I really liked it! I knew, however, that once I got to 1st bigger project using this, I would probably be stuck and have to return to these lessons. Well, that’s learning, isn’t it? Ruby vs. JavaScript: My Favorite Ruby Features Apart from time management, I knew Ruby was a great choice, and even though I still had JavaScript syntax in my brain, I could see how rewarding Ruby is. It was so easy to see how things are done in Ruby, and I can promise you that if you decide to learn this, you will have plenty of AHA and ‘’why couldn’t it be implemented to JS?!” moments. Look at these examples I liked the most so far: Range Operations: In Ruby, it’s super simple! (1..5).to_a And now, look at JavaScript: let range = Array.from({length: 5}, (v, k) => k + 1); Symbol vs string usage in hashes Ruby again makes it easier to read, especially in large objects: user = { name: "Alice", age: 25 } JavaScript: let user = { "name": "Alice", "age": 25 }; Another example is a case statement. I loved it since I found out about this for the first time. I finally didn’t need to use console.log, which was quite annoying. case language
when "Ruby"
  "Great choice!"
when "Python"
  "Not bad!"
else
  "Consider Ruby!"
end Javascript switch (language) {
  case "Ruby":
    console.log("Great choice!");
    break;
  case "Python":
    console.log("Not bad!");
    break;
  default:
    console.log("Consider Ruby!");
} Tips for Staying Motivated on Your Coding Journey It took more work to get to this point of The Odin Project. I would even say that the whole Foundation was easier than the beginning of Ruby. However, after these many lessons, my willingness to learn more has returned, and I want to push further. I also stopped procrastinating so much. If you are in a similar place with your learning schedule and feeling tired and thinking about doing something else, I encourage you not to give up! Really! Maybe it sounds blatant, and I might sound like a cheap coach or something, but it works. At the moment, I am working on a Mastermind project, and it is fun. Tic-Tac-Toe helped me to refresh all my knowledge about OOP, and it seems like Mastermind feels much more fun as, at this point, I know how to solve many problems. Keep going. Everything will click at some point! I’ll keep you updated after I’m done with another challenge. Hi! If you read my previous article about why I chose Ruby, you know why I didn’t follow JS even though it was still deep in my memory after finishing the Calculator project and the whole Foundations from The Odin Project . previous article The Odin Project Introduction to Ruby with The Odin Project I have to tell you that I couldn’t wait for that moment and was super excited to learn Ruby and to really see how Yukihiro “Matz” Matsumoto managed to make it so programmer-friendly, as many say. Yukihiro “Matz” Matsumoto Getting Started: Ruby Installation and Documentation Firstly I had to install everything and it wasn’t difficult with TOP guide. After a short while, everything was set and ready to go. Firstly, I was introduced to Ruby documentation, which looked a little bit different from JS. (What would I expect, though?! It’s a different language! :D) It took me some time to get used to that. Especially when it comes to all methods and classes, at that point, I could tell the OOP lesson was approaching because it seemed different from what I was used to. Once you get the hang of everything, it becomes really easy to find the information you’re looking for. The next step was to learn what IRB is. It is a Ruby interactive model that lets you type your program in the console and run it without creating any files. It is a very useful thing that Rubyists use all the time, and I have liked it since the beginning as I didn’t need to open a browser to check my code every single time like I was doing with JS. Tackling the Basics: Data Types and Initial Challenges After I was sorted with all those, I was ready to tackle basic data types, and after a couple of Pomodoro sessions, my hype began to slow down. Why? It felt super boring! I felt as if I was doing JS from the beginning. I thought I’d done this before, so why waste time doing the same thing again? I understand that I am learning a new language, and I need to start from the basics, but c’moooon! Overcoming Boredom and Staying Motivated I’ll admit I was a LITTLE disappointed at first, but I had no choice but to stick to the plan and carry on. I knew it would get more exciting at some point, but I was only afraid not to lose motivation. Why would I? It took me already so long to get to this point that there is no chance I am stopping or changing my mind. It took me already so long to get to this point I didn’t want to fight with my thoughts any longer at this point and just decided to put in more hours and focus harder as procrastination started to crippling in, especially between my Pomodoro sessions. The next lessons were about conditionals, loops, arrays, hashes, methods, and debugging. I cannot say it was the most exciting time, and at this time, I missed making some projects. I felt as if I was forgetting how to write code while learning how to write code. Strange, isn’t it? I hope that you know what I mean. Maybe there are more of you who have had the same struggles. I then approached Basic Ruby Projects, and this news made me happy. I was finally able to give it a try and solve some bigger exercises like I used with Java Script some time ago. Well, I can’t remember how long it took me to finish them, but it wasn’t too long before I got back to TOP lessons. From Basics to Object-Oriented Programming (OOP) The next one was object-oriented programming, which I couldn’t wait for. At that moment, I knew what it was going to be all about, more or less, and I was eager to try it. Moreover, I was really curious if it’s so difficult to grasp, as people sometimes say. Since then, I planned to push through the curriculum faster during the weekends, but something was always stopping me, like my car broke, and I had to buy another one as it wasn’t worth repairing my old one. Shortly after, my family wanted to visit me, and we were planning to go to Scotland for a week. I knew already that the time I wanted to spend on learning was limited that month. When we were there, I did my best to learn Ruby for 1 hour a day at least. After we came back and I dropped my family at the airport, my friend from Birmingham came to me for the weekend as he needed me to give him a hand fixing garden furniture he produced and sold to clients not far away from where I live. On top of that, my newly bought car decided to start leaking, and to save up money, I decided to fix it myself over another weekend. Guess what… By this point, I had already lost three weeks of productivity. I was a little bit mad and scared that I wouldn’t be able to get my routine going again, but fortunately, it seemed to start picking up nicely after I started again. I continued with OOP, which didn’t seem difficult. I could see the clear, organized approach, and I really liked it! I knew, however, that once I got to 1st bigger project using this, I would probably be stuck and have to return to these lessons. Well, that’s learning, isn’t it? Ruby vs. JavaScript: My Favorite Ruby Features Apart from time management, I knew Ruby was a great choice, and even though I still had JavaScript syntax in my brain, I could see how rewarding Ruby is. It was so easy to see how things are done in Ruby, and I can promise you that if you decide to learn this, you will have plenty of AHA and ‘’why couldn’t it be implemented to JS?!” moments. Look at these examples I liked the most so far: Range Operations: In Ruby, it’s super simple! (1..5).to_a (1..5).to_a And now, look at JavaScript: let range = Array.from({length: 5}, (v, k) => k + 1); let range = Array.from({length: 5}, (v, k) => k + 1); Symbol vs string usage in hashes Ruby again makes it easier to read, especially in large objects: user = { name: "Alice", age: 25 } user = { name: "Alice", age: 25 } JavaScript: let user = { "name": "Alice", "age": 25 }; let user = { "name": "Alice", "age": 25 }; Another example is a case statement. I loved it since I found out about this for the first time. I finally didn’t need to use console.log, which was quite annoying. case language
when "Ruby"
  "Great choice!"
when "Python"
  "Not bad!"
else
  "Consider Ruby!"
end case language
when "Ruby"
  "Great choice!"
when "Python"
  "Not bad!"
else
  "Consider Ruby!"
end Javascript switch (language) {
  case "Ruby":
    console.log("Great choice!");
    break;
  case "Python":
    console.log("Not bad!");
    break;
  default:
    console.log("Consider Ruby!");
} switch (language) {
  case "Ruby":
    console.log("Great choice!");
    break;
  case "Python":
    console.log("Not bad!");
    break;
  default:
    console.log("Consider Ruby!");
} Tips for Staying Motivated on Your Coding Journey It took more work to get to this point of The Odin Project. I would even say that the whole Foundation was easier than the beginning of Ruby. However, after these many lessons, my willingness to learn more has returned, and I want to push further. I also stopped procrastinating so much. If you are in a similar place with your learning schedule and feeling tired and thinking about doing something else, I encourage you not to give up! Really! Maybe it sounds blatant, and I might sound like a cheap coach or something, but it works. At the moment, I am working on a Mastermind project, and it is fun. Tic-Tac-Toe helped me to refresh all my knowledge about OOP, and it seems like Mastermind feels much more fun as, at this point, I know how to solve many problems. Keep going. Everything will click at some point! I’ll keep you updated after I’m done with another challenge.

Falling in Love with Ruby: Getting Started with The Odin Project

Stories, News & Mentions about STARTREE

Startree’s Stories on HackerNoon