FireStart empowers people and technology to perform as one™

FireStart

firestart

What Is Fracture Labs? Fracture Labs was founded back in 2018 and is an independent studio of games industry veterans with previous experience working on a number of successful AAA videogames for PC, console, and mobile. Fracture Labs’ studio is the official developer and publisher of its first videogame, . Decimated Meet Fracture Labs Team Fracture Labs’ team is made up of over fifty individuals including Unreal Engine developers who have worked on some of the most successful games and franchises in recent memory for PC, console, and mobile. The team also works with production partners in the videogame industry with AAA game development experience, ensuring the highest possible quality visuals and gameplay. We have developers in Ukraine, Portugal, the United Kingdom, Germany, Argentina, and Brazil, as well as a network of freelancers from across the globe collaborating on the project. We also work with some of the best production partners in the industry for art, animation, and co-development. Decimated Game Overview Decimated is a role-playing game set in the hostile environment of a grim cyberpunk world. Players are tasked with ensuring the survival of their character, with the title offering them the option to play as either a human or a cyborg. In order to survive, they must acquire resources and technology. With these, they can then repair vehicles, build bases, fortify their shelters, hunt, raid, loot, and trade. The 256 km² online world, which can be explored by land, sea, and air, offers players a wealth of possibilities. In the beginning, they will have to explore them mainly on foot. But there is also the option of so-called fast travel, which is offered by NPCs for a fee. In the wilderness, there is then the possibility of finding vehicle pieces, spare parts, and building materials. All the necessary items can be found on the map or purchased on the marketplace. To survive, players must also search for food, water, and in some cases, oxygen. If a player is injured, they must treat the injury, otherwise, they risk bleeding to death. Drugs and medications are addictive and addiction can be formed, which affects their effectiveness, among other things. If you die in the game, you will respawn with only a few items. Later in the game, there are vaults and safes where you can store items to avoid losing them. There is also the option to choose your own respawn location, such as where a guild has a base or one of the green zones. In order to survive in Decimated, players must use their skills and wits, either by cooperating with others, or conversely, while competing against them. Online environments will include crumbling cities, deserts, abandoned areas, caves, and safe zones protected by police cyborgs. Players will be able to freely explore and loot this vast area, trade, construct buildings, or complete missions that appear randomly on the map. Players are rewarded for missions, and the missions themselves can be completed either solo or as part of a team. If the missions are completed as a team, the player will get a share of the loot if they survive to the end and are not betrayed by other players. Players and NPCs in the game can have friendly or hostile relationships with each other. Everything will depend on the player’s reputation. Not only do NPCs react to reputation, but also other players. The title will be free to play. However, players will start the game with nothing at the beginning. They will have to secure all items mainly through paid missions, or they can purchase items such as vehicles or weapons with the in-game token DIO. If players want to progress, they will have to earn or purchase everything with the DIO token. The game is built in Unreal Engine 5 and is currently in the alpha stage. Everything that has been presented so far is only provisional and will be improved as the game develops. When creating the game, the team was inspired by ARMA, DayZ, and Escape from Tarkov. In the future, players will be able to buy and sell a number of virtual assets (NFTs) in our integrated trading platform. While many common items like ammo and consumables will not be tokenized and will never leave the game, certain items like vehicles, weapons, character customizations, and legendary items will be and can be sold on external marketplaces like and opensea.io solsea.io We will create and sell high-quality, digitally scarce items during development and for several years after launch on a regular basis. Gamers will be able to purchase these items, customize their own vehicles, weapons, apartments, and characters, and a unique token address will be assigned to them when they leave the Decimated world. When we run auctions on certain items, such as rare ships, we will set a deadline for players to bid on each item, and the highest bidder will win the item at the end of the auction, ownership will be assigned, and he/she will be able to use it in-game, customize it, or trade it with other collectors. We are aiming for the Alpha of the game to be ready in 2023 and the Beta in 2024, at which point it'll begin to be ported to Playstation and Xbox, and then mobile. Decimated Tokenomics There will only be one type of token in the game, called DIO, and represented by the symbol Ð. The token is currently based on the Solana blockchain, which will be integrated into the game. Our currency is to act as the main means of exchange inside the Decimated videogame, and it will give value to all other items in the game. The total supply of the token is 1,000,000,000. The maximum supply of DIO Tokens has been minted in a single genesis block. Unlike other online games which generate virtual money or loot every time an NPC or monster is killed, in Decimated, the DIO token is recirculated into the economy when it is spent, and there is a finite amount. Players will be rewarded DIO tokens in the following circumstances: Performing “legal” or “illegal” missions with different risk/reward ratios Stealing shipments or raiding convoys Escorting shipments and protecting them Collecting bounties for imprisoning/killing players and specified NPCs Freeing prisoners Trading items with other players or on the marketplace Players will be debited DIO in the following circumstances: Repairing weapons, vehicles, or bases Being killed by other players and NPCs Paying for fast travel to other locations Transactions fees with vendors and other players Fines from in-game authorities Fracture Lab’s Funding Round and Investors Fracture Labs launched a private token sale that received more than $34 million in offers in November 2021 and raised $4.1 million in the latest funding round – the independent game development studio received this funding for Decimated, their debut multiplayer third-person survival game. Before the release, the studio will use this funding to support development efforts and the launch of the NFT marketplace. In the funding round, there were over thirty Venture Capital firms that participated and made it a reality. The firms include: Spartan Group Cryptology AG Forward Analytic Alameda Research Mechanism Capital Israel Blockchain Association Gravity Finance Metaverse Capital Cointelligence Fund VLaunch Master Ventures Gains Associates Interstellar Ventures Spark Digital Capital Metavest Capital Polygon Hype Partners Israeli Blockchain Association Latitude Blockchain Services Three M Capital Bixin Ventures FireStarter Blockpact Capital GemCapital Cryptology Asset Group Gravity Finance Simplex Dutch Crypto Investors Titans Ventures Good Games Guild Forward Analytics Cointelligence PlutoDigital Fracture Lab’s Community At Fracture Labs we encourage community feedback, organize Q&A sessions, and share progress through live broadcasts on our Twitch channel and posts through the official social media channels. Join 100k+ other Decimators to learn more and contribute! Official Links: • Official website: • Decimated Merch: • Decimated Pitch Deck (Public): • Linktree (all links in one): https://www.decimated.net/ https://decimated.myspreadshop.com/ https://www.canva.com/design/DAEtjWTJPWk/1LfTVtLgbeJFFePrXLaA1A/view#1 https://linktr.ee/decimatedgame • Steam: • Discord: • Twitch: • Twitter: • Instagram: • Linkedin: • Youtube: • Facebook: • Reddit: • Patreon: • Medium: • Whitepaper: • Knowledge base (FAQ): • Telegram Decimated Official: • Telegram Decimated Community: Social Media: https://store.steampowered.com/app/1354980/Decimated/ https://discord.gg/decimated https://www.twitch.tv/decimatedofficial https://twitter.com/decimated_game https://www.instagram.com/decimated_game https://www.linkedin.com/company/fracture-labs-games/ https://www.youtube.com/c/DecimatedVideogame https://www.facebook.com/DecimatedOfficial/ https://www.reddit.com/r/decimated_game/ https://www.patreon.com/decimated_game https://medium.com/@fracturelabs https://www.decimated.net/updates/decimated-whitepaper https://decimated.zendesk.com/hc/en-us/ https://t.me/decimated_game https://t.me/decimatedofficial • DIO on Raydium: • DIO on : • DIO on Huobi: • DIO on Coinmarketcap: Buy DIO: https://raydium.io/swap/?inputCurrency=Es9vMFrzaCERmJfrF4H2FYD4KCoNkY11McCe8BenwNYB&outputCurrency=BiDB55p4G3n1fGhwKFpxsokBMqgctL4qnZpDH1bVQxMD&fixed=in Gate.io https://www.gate.io/trade/DIO_USDT https://www.huobi.com/en-us/exchange/dio_usdt/ https://coinmarketcap.com/currencies/decimated/

Decimated, the Next-Gen Gaming Metaverse to Look Out For

I wanted speed. I speed. needed All this clicking around, all this searching, all the repetitive thinking and mental overhead was cutting into actually getting the job done. I needed the robots to do the work for me. I’d recently moved over to macOS after many years on Windows where I’d tinkered around with AutoHotkey. As a relative Mac noob I started hacking around with , and quickly found solace in the form of Hammerspoon. Alfred Karabiner Ah, . Yes, I had consumed the code and modules — “Spoons” — from the super-smart people coding and extending Hammerspoon and that gained me some great mileage, yet I needed . Hammerspoon more Then, there was Lua. I wanted to upskill in Lua. The best way to do so? Code something. Enter a collection of 3 mini-apps built on top of so you can automate tasks, boost productivity and eliminate time-suck by programming shortcuts into your daily workflows. , Hellfred Hammerspoon It’s a way to map repetitive, time-consuming tasks to key sequences, commands or searchable texts. TL;DR and the branch. So you want quick-fire? Skip to the installation and tutorial section below or download the repo checkout basics git checkout basics The Apps Hellfire A quick-fire, mode-based, hotkey-to-action mapping utility. Supports single key triggers as well as key chord sequences as triggers. Exposes virtually every key on the keyboard including modifier keys to use as triggers. Modes allow the same trigger to have different behaviours under different contexts. English, please? When I type a character or a sequence of characters, then execute a specific function, but only if I am in a particular mode. Ok. An example maybe? When I type then run function (but only if I am in Default Mode) c launchGoogleChromeApp When I type followed by then run function (again, whilst in Default Mode) w m changeToWindowManagerMode When I type then run function (whilst in WindowManager mode) c centerWindowOnScreen Hellfuzz A fuzzy-search chooser utility with choice-to-action mapping. Supports multi-level (nested) choice sets. English, please? When I search through a list of choices and select one, then execute a specific function. If my choice has subchoices (think: ), then show me those so I can search through them. parent => children Ok. An example maybe? Suppose you have this structure: When I type ‘ ’, then highlight the choice ‘ ’. Selecting this option will execute the function . goog Open Google openGoogleInBrowser Alternatively, if I type ‘ ’, then highlight the choice ‘ ’. Selecting this option will replace the current choices with , , and (the subset of choices for ) app Launch Apps Terminal Notes Calendar Launch Apps When I fuzzy search through and select one, Hellfuzz will execute with the selected app. those launchOrOpenApp Hellprompt A commandline-like utility with basic string matching support. English, please? When I type out a command and submit it, then inspect my command for any matching string patterns and execute functions related to that command. Ok. An example maybe? When I type the command ‘ ’ and then hit , then execute any function with a (e.g. command must start with the word ‘ ’) and behaviour (e.g. open app associated with ‘ ’) suitable to open the Notes app. open notes enter filter open notes When I type ‘ ’ and then hit , then execute any function with a filter (e.g. command starts with the word ‘ ’) and behaviour (open url associated with ‘ ’) suitable to open the link. browse github enter browse github Installation: Firestarter Download and install Hammerspoon Install Hellfred: Clone the repository to your ` ` directory: ~/.hammerspoon git clone https://github.com/braddevelop/hellfred.git ~/.hammerspoon Bootstrap: Light it up There is a for Hellfred with a pre-configured setup. Let’s reference it in Hammerspoon’s file. bootstrap file init.lua https://gist.github.com/braddevelop/bac92b6797c087ca42f9363aab4972e0 Save the file and reload the configuration (or save yourself some time and use ) fancy reload What’s in the box? Try out the pre-configuration Out-the-box the 3 Hellfred apps are ready to use and are pre-configured with a quick-start example. Let’s test it out to make sure everything is wiring and firing. Try Hellfire Open with the hotkey + + Hellfire shift ⌘ h Type the character c The repo for Hellfred will open in a browser. Try Hellfuzz Open with the hotkey + + Hellfuzz shift ⌥ h Type in the word ‘ ’ wiki This highlights the option ‘ ’ Open Hellfred wiki Hit and the wiki for Hellfred will open in a browser enter Try Hellprompt Open with the hotkey + + Hellprompt shift ^ h Type ‘ ’ open code Hit and the code repo for Hellfred will open in a browser enter What , and achieve is map a or to an or , , and whilst we have just demonstrated using each app to achieve the same outcome, you will find each app more suited to certain use cases than others. Hellfire Hellfuzz Hellprompt trigger input action behaviour if-this-then-that Tutorial: A basic setup What we will be programming Now let’s turn up the heat and configure something a little more useful. We are going to program to solve the following scenarios so that you can get the hang of things: each app A simple app launcher A url launcher for commonly visited links You can find the final files for this walkthrough in the directory on the repository’s branch. hellfred/extend/basics basics Patterns to note Each app follows a similar set of steps. Initialise the app with a hotkey binding Configure ` ` objects (This could be done in but we will be creating separate files to keep things squeaky clean. We’ll leverage factory methods to make object creation a breeze) Subscriber hellfred-bootstrap.lua Register the subscribers with the app Hotkey to run the app Destroy time-sucking tasks Setup for Hellfire Hellfire works a little something like this: Application Launcher Initialise the app This is already done with the pre-configuration in . Feel free to change the hotkey to something else. hellfred-bootstrap.lua https://gist.github.com/braddevelop/49c7e7c7c62350d93e7798ff2bf40541 Configure Subscriber objects and are user-defined and wrapped inside simple configuration objects. These objects, act as when registered with the respective app and notified whenever something important happens inside the app. triggers callbacks subscribers Subscriber objects for Hellfire follow this structure: https://gist.github.com/braddevelop/20c9c5b97a85a9a5c4cb924b511e494a : If is not defined, Hellfire will set the mode by default, meaning the callback will fire in any mode when triggered. Note fireIfModeIs ANY Create the following directory structure if it does not exist: . hellfred/extend/basics Then create a new Lua file inside the directory called — the file naming convention has no importance. basics hellfirepack-applications.lua Add this code: https://gist.github.com/braddevelop/c12d37d64ce75b83a195666317a10aa2 Register the subscribers with the app Back in , we need to register the pack of subscribers we have just configured. hellfred-bootstrap.lua https://gist.github.com/braddevelop/2298469b805978062421f5a09d01d11a Run the app Enter Hellfire ( + + ) and type any of the new triggers: to open Finder app, to open Terminal or to open Notes app. shift ⌘ h f t n Link Launcher (using Hellfire Modes) Configure Subscriber objects Alright now let’s configure the subscribers for our Common Links url launcher. Create a new Lua file in called hellfred/extend/basics hellfirepack-common-links.lua Add this code: https://gist.github.com/braddevelop/7d637464f45d2d9a4641e365ad2f20a1 Register the subscribers with the app Back in , we need to register the pack of subscribers we have just configured. hellfred-bootstrap.lua https://gist.github.com/braddevelop/f0dafb518797620747044afbc20da060 Run the app Enter Hellfire ( + + ) and type any of the new triggers: , , or . shift ⌘ h t g h s Did you notice that typing the trigger launched opened the website? That’s probably not what we want to happen. Let’s take advantage of . Hang on! t Terminal as well as TechCrunch Hellfire’s Mode feature Modes offer a way to have the same trigger behave differently under different contexts. By default, Hellfire initialises in a mode called…you guessed it… ‘ ’ mode. We can configure some custom modes to use with Hellfire so that triggers can behave differently under different modes — or ‘namespaces’ if you like. Default Consider this flow: Create a new Lua file in called . hellfred/extend/basics hellfire-modes-extended.lua We will create a separate mode for the to fire in. common links triggers Add this code: https://gist.github.com/braddevelop/1e3f345076a8cd5c4cdf843cad785c9e We are going to require this file in so it is accessible. We will do the same with the so that we have access to Hellfire’s built-in modes in other parts of our application. hellfred-bootstrap.lua globally Hellfire Modes file Add the following code in (under the metadata section, towards the top of the file) hellfred-bootstrap.lua https://gist.github.com/braddevelop/04a4ec21aab06f50760d1082b216426f Now we need a way to change the mode to our new Common Links mode. We’ll use the key sequence of followed by . c l We also need to be able to get back to mode Default We’ll use the semi-colon as a trigger. ; That’s next… Configure Subscriber objects that trigger mode changes Create a new Lua file in called hellfred/extend/basics hellfire-mode-triggers.lua Add this code: https://gist.github.com/braddevelop/283acacb9634590208c3b29b6039dc12 Register the subscribers with the app In , register the subscribers for the new mode triggers: hellfred-bootstrap.lua https://gist.github.com/braddevelop/8afc4a576d550415f22c3c6fbf4152f1 Test switching between modes Now enter Hellfire ( + + ) and toggle between the two modes. shift ⌘ h Modes FTW! Update subscribers to work in modes Now we need to update our subscribers in so that they only fire when is active. hellfirepack-common-links.lua Common Links mode We will update the factory method and assign to instead of . _G.HELLFIRE_MODES_EXTENDED.COMMON_LINKS fireIfModeIs nil The updated method should look like this: https://gist.github.com/braddevelop/152a16f87efe55f08e7ed7139225f757 We also need to update our subscribers in so that they only fire when is active. hellfirepack-applications.lua Hellfire’s Default mode The updated method should look like this: https://gist.github.com/braddevelop/8ed8dea04397b0471299c8c89f8a9ae0 Run the app Enter Hellfire ( + + ) and toggle between the modes. The trigger now behaves differently depending on the mode that Hellfire is in. shift ⌘ h t Hell yeah! Setup for Hellprompt This is how Hellprompt functions: Application and URL Launcher Initialise the app This is already done with the pre-configuration in . Feel free to change the hotkey to something else. hellfred-bootstrap.lua https://gist.github.com/braddevelop/11e9b4871182f90c57313ab6ffa939a4 Configure Subscriber objects Subscribers for Hellprompt take a different structure to those for Hellfire. Consider this structure: https://gist.github.com/braddevelop/c247d1d1429234c96f3e8c7d8a0b48df : If is not defined then the callback will always be executed. Note filter Create a new Lua file in called hellfred/extend/basics hellpromptpack-commands.lua Add this code: https://gist.github.com/braddevelop/6b18219f83fcf69826083d5dfce5d06f Register the subscribers with the app Back in , we need to register the pack of subscribers we have just configured. hellfred-bootstrap.lua https://gist.github.com/braddevelop/f1b75646b7330fcb1c993a5737fd7d61 Enter Hellprompt ( + + ) and test out those commands. Run the app shift ^ h Inferno! Try: browse news and: open terminal Setup for Hellfuzz This is how Hellfuzz works: Application Launcher Initialise the app This is already done with the pre-configuration in . Feel free to change the hotkey to something else. hellfred-bootstrap.lua https://gist.github.com/braddevelop/b18f575ec347503628e7457217b6e187 Configure Subscriber objects Subscribers for Hellfuzz take a different structure to the other apps. Consider this structure: https://gist.github.com/braddevelop/56e9c22c6f921575637c73d60920d208 : If is defined then is ignored. Note nextChoicesFn callback To make things easier we’ll use a helper method to configure subscribers for Hellfuzz Create a new Lua file in called hellfred/extend/basics hellfuzzpack-apps-and-links.lua Add this code: https://gist.github.com/braddevelop/3564ad1ec286a30eca1abb94cfd6a69a Register the subscribers with the app In , register the pack of subscribers. hellfred-bootstrap.lua https://gist.github.com/braddevelop/a90f9d5635377d74b145a892845c8273 Run the app Enter Hellfuzz ( + + ) and type in a command. For example start typing the word , you’ll see the option to open Terminal is highlighted. Press and opens. shift ⌥ h ‘Terminal’ enter Terminal Smoking hot! Link Launcher (using nested choice sets) A handy feature of Hellfuzz is the ability to nest sets of choices. Consider this updated flow: Let’s try this out on our task, we’ll create the following hierarchical choice structure: Link Launcher Update the code in to the following: hellfuzzpack-apps-and-links.lua https://gist.github.com/braddevelop/384b0ae9629faca95d5d4cb519cd8e61 Now enter Hellfuzz ( + + ) and start searching for ‘ ’. You can select the ‘ ’ choice, and the sub-set of choices from will be displayed and can be fuzzy searched. Selecting any of the link options will open the respective url. shift ⌥ h Common links Common links commonLinkNextChoices Extensions: Add fuel to the fire. Look out for upcoming Hellfred experiments and extensions on the repo by checking out the branch. extend git checkout extend Now go raise hell \m/

Hellfred or: How I Learned To Automate macOS and Become Hellishly Productive

Hellfred o: Cómo aprendí a automatizar macOS y volverme tremendamente productivo

Hellfred ou : Comment j'ai appris à automatiser macOS et à devenir extrêmement productif

हेलफ्रेड या: मैंने मैकोज़ को स्वचालित करना कैसे सीखा और नारकीय उत्पादक बनें

Hellfred または: macOS を自動化し、地獄のように生産的になる方法を学んだ方法

Hellfred ou: Como aprendi a automatizar o macOS e me tornar incrivelmente produtivo

Hellfred or: Tôi đã học cách tự động hóa macOS và trở nên hiệu quả như thế nào

Hellfred 或：我如何学会自动化 macOS 并变得非常高效

NFT Platform Raises Funding to Expand Operations in Asia , a celebrity NFT platform that offers various additional services and experiences, has acquired over $5 million in funding, allowing the team to construct the infrastructure needed to push the NFT world into the mainstream. A customized virtual museum, numerous play-to-earn games, virtual competitions, and the metaverse are available via the authorized celebrity NFT platform. Colexion The global NFT market generated over $23 billion in trading volume in 2021, according to crypto-data analysis platform DappRadar. NFTs quickly bridge the gap between our physical reality and the widely anticipated metaverse. With NFTs becoming more mainstream worldwide, several brands have forayed into the space. Adidas, Nike, Taco Bell, Coca-Cola, and Campbell are some international brands that have entered the industry. Colexion has  become Asia’s biggest NFT platform, drawing the interest of a number of private investors and supporters. The seed round brought in $1.8 million, while a private round brought in $3.2 million. Polygon, Jump Trading Investments, Alan Howard of Brevan Howard, HyperEdge Capital, C2 Ventures, GSR, Oracles Investment Group (OIG), Gains Associate, and a number of other notable names such as ZBS Capital, ICO Pantera, Titan Ventures, Good Games Lab, Maven Capital, and others contributed to both rounds. Several launchpads, including Redkite, Prostarter, Scaleswap, Firestarter, and others, will support the native $CLXN utility coin. The token will be traded on a number of major exchanges. The token and NFTs will also be traded on the built-in marketplace as well as other Tier A exchanges. The platform’s major goal is to license famous NFTs while keeping them accessible to the general public. Major Indian movie stars, international cricket players, and other worldwide sports and movie stars are among the celebrities, with each NFT being unique. The platform has also secured exclusive rights to a number of international cricket teams, Bollywood and Tollywood stars, TV personalities, different global and Indian sports leagues, and other great athletes. The Future of Digital Art and NFTs NFTs have gained widespread awareness due to large art transactions, which now account for about a quarter of all NFT sales. For example, an American artist known as Beeple sold his digital artwork for $69.3 million at Christie’s earlier this month, putting him among the “top three most valuable living painters.” Every time the owner of the NFT changes, he will earn a ten percent royalty. Winkelmann’s success in the digital world is undoubtedly not unique. CryptoPunk #7804, one of the 24x24 pixel art pieces, sold for $7.5 million in March, while TrevorJones” Bitcoin Angel sold for $3.2 million. In 2022, the NFT market is expected to grow even further. Like any other sector, it has its own set of hazards that purchasers should be aware of before investing. However, when it comes to art, it’s necessary to look at the item first – and then at its properties – because possessing something you don’t like has little value. Catch all the breaking news, and Don’t forget to like the story!

Colexion a Celebrity NFT Platform Raises $5 Million from Polygon, Titan Ventures and Maven Capital

We’ve been flying under the radar for the last couple of months over at HQ, hard at work optimising our app and generally trying to give our users the best experience possible. Flamelink We’ve also been tinkering away at a brand spanking new feature that we believe is going to be adding a lot of value to your Flamelink and projects. (And one of the most requested features by our Flamelink community - after Firestore support.) Firebase Introducing Multi-Languages by Flamelink. From 7 March 2019, you’ll be able to build your Firebase and Flamelink projects in with Flamelink’s Multi-Language feature. Any Language You Can Think Of Upload your content in one central place, then only query the specific language/languages you require - instead of creating two or more separate projects and trying to keep track of which language you’ve updated what content for. Simples. How Will This Help You And Your Project? Flamelink’s Multi-Language feature lets you innovate by creating Multilingual apps / sites to reach a global audience, taking your project to a whole new market. Create environment specific content by publishing unique content across multiple platforms, giving your end users a bespoke user experience. Build more diverse teams by finally letting that dude that speaks flawless Quenya join your team. I’m not too sure how that would work for you, but it’s nice to know that you now have those kinds of options. How to Get Multi-Languages by Flamelink The Multi-Languages feature is available on the Firestarter Plan. Get up to any 3 languages right out the box for only $25 per month, plus all the other amazing features that come with Flamelink’s Firestarter plan. Sign up or upgrade to Flamelink’s Firestarter plan now. As always, if you need any assistance, feel free to get in touch with us. ’Til next time. Cheers. Totsiens. Ukudigada. Ma’as Salaam. Agur. Abar Dekha Hobe. zbogom. Donadagohvi. Zai Jian. Zai Geen. Na Shledanou. Do viđenja. Farvel. Näkemiin. Ni Sa Moce. Paalam na. Au revoir. Ba’adan Mibinamet. Auf Wiedersehen. Kan Ga Waanaa. antio. Aavjo. Aloha. Shalom. Namaste. viszontlátásra. Arrivederc. selamat tinggal. Slán Leat. Le-Hitra-Ot. Sayonara. Annyeong-hi Kashipshio. Qapla’! Vale. ardievas. sudie. Adiós. Bayartai. Kìyevame! farvel. Adeus. Do Widzenia. La Revedere. Dasvidaniya. macsalaameyn. Adiós. gihin ennan. Kwaheri. Hej då. Tschüss. Minnan. Sawatdi. Eyvallah. Do Pobatchenya. Khuda Hafiz. Xin chào ông. Hwyl fawr. Rhonanai. Zay Gesunt. Hamba Kahle.

Introducing Multi-Languages by Flamelink, the Firebase CMS.

In the last four articles or so, we’ve done a real whirlwind tour of Haskell libraries. We created a database schema using Persistent and used it to write basic SQL queries in a type-safe way. We saw how to expose this database via an API with Servant. We also went ahead and added some caching to that server with Redis. Finally, we wrote some basic tests around the behavior of this API. By using Docker, we made those tests reproducible. In this article, we’re going to review this whole process by adding another type to our schema. We’ll write some new endpoints for an type, and link this type to our existing type with a foreign key. Then we’ll learn one more library: . Esqueleto improves on Persistent by allowing us to write type-safe SQL joins. Article User Esqueleto As with the previous articles, there’s a specific branch on the for this series. Go there and take a look at the to see the complete code for this article. Github repository esqueleto branch Adding Article to our Schema So our first step is to extend our schema with our type. We’re going to give each article a title, some body text, and a timestamp for its publishing time. One new feature we’ll see is that we’ll add a foreign key referencing the user who wrote the article. Here’s what it looks like within our schema: Article PTH.share [PTH.mkPersist PTH.sqlSettings, PTH.mkMigrate "migrateAll"] [PTH.persistLowerCase| User sql=users   ...

 Article sql=articles   title Text   body Text   publishedTime UTCTime   authorId UserId   UniqueTitle title   deriving Show Read Eq|] We can use as a type in our schema. This will create a foreign key column when we create the table in our database. In practice, our Article type will look like this when we use it in Haskell: UserId data Article = Article { articleTitle :: Text , articleBody :: Text , articlePublishedTime :: UTCTime , articleAuthorId :: Key User } This means it doesn’t reference the entire user. Instead, it contains the SQL key of that user. Since we’ll be adding the article to our API, we need to add ToJSON and FromJSON instances as well. These are pretty basic as well, so you can check them out if you’re curious. If you’re curious about JSON instances in general, take a look . here at this article Adding Endpoints Now we’re going to extend our API to expose certain information about these articles. First, we’ll write a couple basic endpoints for creating an article and then fetching it by its ID: type FullAPI =       "users" :> Capture "userid" Int64 :> Get '[JSON] User :<|> "users" :> ReqBody '[JSON] User :> Post '[JSON] Int64 :<|> "articles" :> Capture "articleid" Int64 :> Get '[JSON] Article :<|> "articles" :> ReqBody '[JSON] Article :> Post '[JSON] Int64 Now, we’ll write a couple special endpoints. The first will take a User ID as a key and then it will provide all the different articles the user has written. We’ll do this endpoint as . /articles/author/:authorid ... :<|> "articles" :> "author" :> Capture "authorid" Int64 :> Get '[JSON] [Entity Article] Our last endpoint will fetch the most recent articles, up to a limit of 10. This will take no parameters and live at the route. It will return tuples of users and their articles, both as entities. /articles/recent … :<|> "articles" :> "recent" :> Get '[JSON] [(Entity User, Entity Article)] Adding Queries (with Esqueleto!) Before we can actually implement these endpoints, we’ll need to write the basic queries for them. For creating an article, we use the standard Persistent function: insert createArticlePG :: PGInfo -> Article -> IO Int64createArticlePG connString article = fromSqlKey <$> runAction connString (insert article) We could do the same for the basic fetch endpoint. But we’ll write this basic query using Esqueleto in the interest of beginning to learn the syntax. With Persistent, we used list parameters to specify different filters and SQL operations. Esqueleto instead uses a special monad to compose the different type of query. The general format of an esqueleto select call will look like this: fetchArticlePG :: PGInfo -> Int64 -> IO (Maybe Article)fetchArticlePG connString aid = runAction connString selectAction where   selectAction :: SqlPersistT (LoggingT IO) (Maybe Article)   selectAction = select . from $ \articles -> do     ... We use and then provide a function that takes a table variable. Our first queries will only refer to a single table, but we'll see a join later. To complete the function, we’ll provide the monadic action that will incorporate the different parts of our query. select . from The most basic filtering function we can call from within this monad is . This allows us to provide a condition on the query, much as we could with the filter list from Persistent. where_ selectAction :: SqlPersistT (LoggingT IO) (Maybe Article)   selectAction = select . from $ \articles -> do     where_ (articles ^. ArticleId ==. val (toSqlKey aid)) First, we use the lens to specify which value of our table we’re filtering. Then we specify the value to compare against. We not only need to lift our into an , but we also need to lift that value using the function. ArticleId Int64 SqlKey val But now that we’ve added this condition, all we need to do is return the table variable. Now, returns our results in a list. But since we’re searching by ID, we only expect one result. We’ll use so we only return the head element if it exists. We’ll also use once again to unwrap the article from its entity. select listToMaybe entityVal selectAction :: SqlPersistT (LoggingT IO) (Maybe Article)   selectAction = ((fmap entityVal) . listToMaybe) <$> (select . from $ \articles -> do     where_ (articles ^. ArticleId ==. val (toSqlKey aid))     return articles) Now we should know enough that we can write out the next query. It will fetch all the articles that have written by a particular user. We’ll still be querying on the table. But now instead checking the article ID, we’ll make sure the is equal to a certain value. Once again, we’ll lift our user key into an and then again with to compare it in “SQL-land”. articles ArticleAuthorId Int64 SqlKey val fetchArticleByAuthorPG :: PGInfo -> Int64 -> IO [Entity Article]fetchArticleByAuthorPG connString uid = runAction connString fetchAction where   fetchAction :: SqlPersistT (LoggingT IO) [Entity Article]   fetchAction = select . from $ \articles -> do     where_ (articles ^. ArticleAuthorId ==. val (toSqlKey uid))     return articles And that’s the full query! We want a list of entities this time, so we’ve taken out and . listToMaybe entityVal Now let’s write the final query, where we’ll find the 10 most recent articles regardless of who wrote them. We’ll include the author along with each article. So we’re returning a list of of these different tuples of entities. This query will involve our first join. Instead of using a single table for this query, we’ll use the constructor to combine our table with the table. InnerJoin users articles fetchRecentArticlesPG :: PGInfo -> IO [(Entity User, Entity Article)]fetchRecentArticlesPG connString = runAction connString fetchAction where   fetchAction :: SqlPersistT (LoggingT IO) [(Entity User, Entity Article)]   fetchAction = select . from $ \(users `InnerJoin` articles) -> do Since we’re joining two tables together, we need to specify what columns we’re joining on. We’ll use the function for that: on fetchAction :: SqlPersistT (LoggingT IO) [(Entity User, Entity Article)]   fetchAction = select . from $ \(users `InnerJoin` articles) -> do     on (users ^. UserId ==. articles ^. ArticleAuthorId) Now we’ll order our articles based on the timestamp of the article using . The newest articles should come first, so we'll use a descending order. Then we limit the number of results with the function. Finally, we’ll return both the users and the articles, and we’re done! orderBy limit fetchAction :: SqlPersistT (LoggingT IO) [(Entity User, Entity Article)]   fetchAction = select . from $ \(users `InnerJoin` articles) -> do     on (users ^. UserId ==. articles ^. ArticleAuthorId)     orderBy [desc (articles ^. ArticlePublishedTime)]     limit 10     return (users, articles) Caching Different Types of Items We won’t go into the details of caching our articles in Redis, but there is one potential issue we want to observe. Currently we’re using a user’s SQL key as their key in our Redis store. So for instance, the string “15” could be such a key. If we try to naively use the same idea for our articles, we’ll have a conflict! Trying to store an article with ID “15” will overwrite the entry containing the User! But the way around this is rather simple. What we would do is that for the user’s key, we would make the string something like . Then for our article, we’ll have its key be . As long as we deserialize it the proper way, this will be fine. users:15 articles:15 Filling in the Server handlers Now that we’ve written our database query functions, it is very simple to fill in our Server handlers. Most of them boil down to following the patterns we’ve already set with our other two endpoints: fetchArticleHandler :: PGInfo -> Int64 -> Handler ArticlefetchArticleHandler pgInfo aid = do maybeArticle <- liftIO $ fetchArticlePG pgInfo aid case maybeArticle of   Just article -> return article   Nothing -> Handler $ (throwE $ err401 { errBody = "Could not find article with that ID" })

createArticleHandler :: PGInfo -> Article -> Handler Int64createArticleHandler pgInfo article = liftIO $ createArticlePG pgInfo article

fetchArticlesByAuthorHandler :: PGInfo -> Int64 -> Handler [Entity Article]fetchArticlesByAuthorHandler pgInfo uid = liftIO $ fetchArticlesByAuthorPG pgInfo uid

fetchRecentArticlesHandler :: PGInfo -> Handler [(Entity User, Entity Article)]fetchRecentArticlesHandler pgInfo = liftIO $ fetchRecentArticlesPG pgInfo Then we’ll complete our like so: Server FullAPI fullAPIServer :: PGInfo -> RedisInfo -> Server FullAPIfullAPIServer pgInfo redisInfo = (fetchUsersHandler pgInfo redisInfo) :<|> (createUserHandler pgInfo) :<|> (fetchArticleHandler pgInfo) :<|> (createArticleHandler pgInfo) :<|> (fetchArticlesByAuthorHandler pgInfo) :<|> (fetchRecentArticlesHandler pgInfo) One interesting thing we can do is that we can compose our API types into different sections. For instance, we could separate our into two parts. First, we could have the type from before, and then we could make a new type for . We can glue these together with the e-plus operator just as we could individual endpoints! FullAPI UsersAPI ArticlesAPI type FullAPI = UsersAPI :<|> ArticlesAPI

type UsersAPI =      "users" :> Capture "userid" Int64 :> Get '[JSON] User :<|> "users" :> ReqBody '[JSON] User :> Post '[JSON] Int64

type ArticlesAPI = "articles" :> Capture "articleid" Int64 :> Get '[JSON] Article :<|> "articles" :> ReqBody '[JSON] Article :> Post '[JSON] Int64 :<|> "articles" :> "author" :> Capture "authorid" Int64 :> Get '[JSON] [Entity Article] :<|> "articles" :> "recent" :> Get '[JSON] [(Entity User, Entity Article)] If we do this, we’ll have to make similar adjustments in other areas combining the endpoints. For example, we would need to update the server handler joining and the client functions. Writing Tests Since we already have some user tests, it would also be good to have a few tests on the Articles section of the API. We’ll add one simple test around creating an article and then fetching it. Then we’ll add one test each for the “articles-by-author” and “recent articles” endpoints. So one of the tricky parts of filling in this section will be that we need to make test object. But we'll need them to be functions on the User ID. This is because we can’t know a priori what SQL IDs we'll get when we insert the users into the database. But we can fill in all the other fields, including the published time. Here’s one example, but we’ll have a total of 18 different “test” articles. Article testArticle1 :: Int64 -> ArticletestArticle1 uid = Article { articleTitle = "First post" , articleBody = "A great description of our first blog post body." , articlePublishedTime = posixSecondsToUTCTime 1498914000 , articleAuthorId = toSqlKey uid }

-- 17 other articles and some test users as well… Our before hooks will create all these different entities in the database. In general, we’ll go straight to the database without calling the API itself. Like with our users tests, we’ll want to delete any database items we create. Let’s write a generic after-hook that will take user IDs and article IDs and delete them from our database: deleteArtifacts :: PGInfo -> RedisInfo -> [Int64] -> [Int64] -> IO ()deleteArtifacts pgInfo redisInfo users articles = do void $ forM articles $ \a -> deleteArticlePG pgInfo a void $ forM users $ \u -> do   deleteUserCache redisInfo u   deleteUserPG pgInfo u It’s important we delete the articles first! If we delete the users first, we’ll encounter foreign key exceptions! Our basic create-and-fetch test looks a lot like the previous user tests. We test the success of the response and that the new article lives in Postgres as we expect. beforeHook4 :: ClientEnv -> PGInfo -> IO (Bool, Bool, Int64, Int64)beforeHook4 clientEnv pgInfo = do userKey <- createUserPG pgInfo testUser2 articleKeyEither <- runClientM (createArticleClient (testArticle1 userKey)) clientEnv case articleKeyEither of   Left _ -> error "DB call failed on spec 4!"   Right articleKey -> do     fetchResult <- runClientM (fetchArticleClient articleKey) clientEnv     let callSucceeds = isRight fetchResult     articleInPG <- isJust <$> fetchArticlePG pgInfo articleKey     return (callSucceeds, articleInPG, userKey, articleKey)

spec4 :: SpecWith (Bool, Bool, Int64, Int64)spec4 = describe "After creating and fetching an article" $ do it "The fetch call should return a result" $ \(succeeds, _, _, _) -> succeeds `shouldBe` True it "The article should be in Postgres" $ \(_, inPG, _, _) -> inPG `shouldBe` True

afterHook4 :: PGInfo -> RedisInfo -> (Bool, Bool, Int64, Int64) -> IO ()afterHook4 pgInfo redisInfo (_, _, uid, aid) = deleteArtifacts pgInfo redisInfo [uid] [aid] Our next test will create two different users and several different articles. We’ll first insert the users and get their keys. Then we can use these keys to create the articles. We create five articles in this test. We assign three to the first user, and two to the second user: beforeHook5 :: ClientEnv -> PGInfo -> IO ([Article], [Article], Int64, Int64, [Int64])beforeHook5 clientEnv pgInfo = do uid1 <- createUserPG pgInfo testUser3 uid2 <- createUserPG pgInfo testUser4 articleIds <- mapM (createArticlePG pgInfo)   [ testArticle2 uid1, testArticle3 uid1, testArticle4 uid1   , testArticle5 uid2, testArticle6 uid2 ] ... Now we want to test that we when call the articles-by-user endpoint, we only get the right articles. We’ll return each group of articles, the user IDs, and the list of article IDs: beforeHook5 :: ClientEnv -> PGInfo -> IO ([Article], [Article], Int64, Int64, [Int64])beforeHook5 clientEnv pgInfo = do uid1 <- createUserPG pgInfo testUser3 uid2 <- createUserPG pgInfo testUser4 articleIds <- mapM (createArticlePG pgInfo)   [ testArticle2 uid1, testArticle3 uid1, testArticle4 uid1   , testArticle5 uid2, testArticle6 uid2 ] firstArticles <- runClientM (fetchArticlesByAuthorClient uid1) clientEnv secondArticles <- runClientM (fetchArticlesByAuthorClient uid2) clientEnv case (firstArticles, secondArticles) of   (Right as1, Right as2) -> return (entityVal <$> as1, entityVal <$> as2, uid1, uid2, articleIds)   _ -> error "Spec 5 failed!" Now we can write the assertion itself, testing that the articles returned are what we expect. spec5 :: SpecWith ([Article], [Article], Int64, Int64, [Int64])spec5 = describe "When fetching articles by author ID" $ do it "Fetching by the first author should return 3 articles" $ \(firstArticles, _, uid1, _, _) ->   firstArticles `shouldBe` [testArticle2 uid1, testArticle3 uid1, testArticle4 uid1] it "Fetching by the second author should return 2 articles" $ \(_, secondArticles, _, uid2, _) ->   secondArticles `shouldBe` [testArticle5 uid2, testArticle6 uid2] We would then follow that up with a similar after hook. The final test will follow a similar pattern. Only this time, we’ll be checking the combinations of users and articles. We’ll also make sure to include 12 different articles to test that the API limits results to 10. beforeHook6 :: ClientEnv -> PGInfo -> IO ([(User, Article)], Int64, Int64, [Int64])beforeHook6 clientEnv pgInfo = do uid1 <- createUserPG pgInfo testUser5 uid2 <- createUserPG pgInfo testUser6 articleIds <- mapM (createArticlePG pgInfo)   [ testArticle7 uid1, testArticle8 uid1, testArticle9 uid1, testArticle10 uid2   , testArticle11 uid2, testArticle12 uid1, testArticle13 uid2, testArticle14 uid2   , testArticle15 uid2, testArticle16 uid1, testArticle17 uid1, testArticle18 uid2   ] recentArticles <- runClientM fetchRecentArticlesClient clientEnv case recentArticles of   Right as -> return (entityValTuple <$> as, uid1, uid2, articleIds)   _ -> error "Spec 6 failed!" where   entityValTuple (Entity _ u, Entity _ a) = (u, a) Our spec will check that the list of 10 articles we get back matches our expectations. Then, as always, we remove the entities from our database. Now we call these tests with our other tests, with small wrappers to call the hooks: main :: IO ()main = do ... hspec $ before (beforeHook4 clientEnv pgInfo) $ after (afterHook4 pgInfo redisInfo) $ spec4 hspec $ before (beforeHook5 clientEnv pgInfo) $ after (afterHook5 pgInfo redisInfo) $ spec5 hspec $ before (beforeHook6 clientEnv pgInfo) $ after (afterHook6 pgInfo redisInfo) $ spec6 And now we’re done! The tests pass! …After creating and fetching an article The fetch call should return a result The article should be in Postgres

Finished in 0.1698 seconds2 examples, 0 failures

When fetching articles by author ID Fetching by the first author should return 3 articles Fetching by the second author should return 2 articles

Finished in 0.4944 seconds2 examples, 0 failures

When fetching recent articles Should fetch exactly the 10 most recent articles Conclusion This completes our overview of useful production libraries. Over these articles, we’ve constructed a small web API from scratch. We’ve seen some awesome abstractions that let us deal with only the most important pieces of the project. Both Persistent and Servant generated a lot of extra boilerplate for us. This article showed the power of the Esqueleto library in allowing us to do type-safe joins. We also saw an end-to-end process of adding a new type and endpoints to our API. In the coming weeks, we’ll be dealing with some more issues that can arise when building these kinds of systems. In particular, we’ll see how we can use alternative monads on top of Servant. Doing this can present certain issues that we’ll explore. We’ll culminate by exploring the different approaches to encapsulating effects. Be sure to check out our ! It’ll show you how to use Stack, so you can incorproate all the libraries from this series! Haskell Stack mini-course! If you’re new to Haskell and not ready for that yet, take a look at our and get going! Getting Started Checklist

Join the Club: Type-safe Joins with Esqueleto

I recently did something that was rather surprising to me. I ended up writing a book without even realizing. In this post, I will discuss how my book/interactive course, , came to be, and what writing a book looks like as a programmer from a technical point of view. I will also talk a bit about creating online courses as it was the starting point of my writing journey. If you are short on time, you can skip to the third part of this article where I talk about what kind of tooling I ended up using for writing and publishing a book or check out the tl;dr below. But remember, sometimes the journey is more rewarding than the destination. Coding for Visual Learners tl;dr: Begin with the tools that you are most comfortable with. It is important to keep the friction low when you are just starting out. Your emerging needs will inform your specific tooling decisions so just start creating instead of getting stuck at planning. I tried and for writing. Settled on for its Markdown support, cloud syncing and offline editing capabilities. It was a good enough choice but can fall short when gathering feedback on your material. Workflowy Notion.so StackEdit.io Leveraged to include metadata inside the text and language to be able to use conditionals and variables. YAML front matter Nunjucks templating Created node.js scripts to move documents around, and to pre&post process them according to the desired output format (offline vs online). My process was automated but not overly engineered. Remember, d . o things that don’t scale I worked with a proofreader on Fiverr Used to self-publish and to generate an unbranded ebook in multiple formats to be sold on the . Leanpub Amazon Kindle store This linked post by is a must-read if you are planning on self-publishing. Zapier Used to create an interactive and animated course out of my already existing material. Educative.io Part 1: How did I write a book without even realizing? Let’s start with a proper introduction. My name is and I am a software developer, an educator and now an author. I have been teaching an Introduction to Python course at a college for more than two years now, and a couple of months ago I started to create online courses/video tutorials on the side as well. I usually find myself scripting the entire course before doing the recording to be able to have a more predictable and a repeatable outcome. In other words, I am not great at improvisation. These scripts that I end up writing are usually very comprehensive and contain virtually every single word that I plan to say during recording. I published my first online course last year on Pluralsight, . You should check it out if you haven’t done so yet. It’s okay, I will wait for you to come back. Engin Automating the Web using PhantomJS and CasperJS This time I wanted to create another online course, but I wanted to do everything by myself to have a more end-to-end experience in course creation. I have been working for someone else for my entire life and never really handled any transaction all by myself. I figured creating a course could be a good exercise in experimenting with the business side of the things as it would need me to handle things like marketing or distribution as well. In my preliminary research, I identified a market gap in visual ways of learning to code and choose to use the excellent JavaScript library, , to teach programming to beginners. As a self-taught programmer, I remember clearly the struggles that new programmers face. I set out to create the kind of course that would have helped me when I was starting out. p5.js To write the scripts I would be reading during recording, I have been using a service called . It is essentially a list creator where you can create . I found that this kind of an interface maps well to how I structure my thinking. It allows me to create a new list item for every couple of sentences. This makes reorganizing your text really easy just by dragging and dropping. And collapsing lists helps with hiding and managing the complexity. Workflowy deeply nested collapsible lists <a href="https://medium.com/media/5b848ef565256e54c2e731c4976e42a7/href">https://medium.com/media/5b848ef565256e54c2e731c4976e42a7/href</a> Workflowy Interface Workflowy mapped really well to how I did my initial recording as well. I was recording only a couple of sentences at a time since it felt like recording continuously for a long time would increase the chance of errors in my speech. #ESL : Later I found out that this is not the greatest way of going about recording. This level of granularity increases your editing and recording time a lot. An audio engineer recommended me that I should try to record in batch (like even trying to do the entire thing in one go) and try to edit/patch it at a later time. Learning One Workflowy is a great tool and I have been using it for many years now for other productivity related purposes as well. It makes writing and note-taking easy and friction-free however it is not great at showing code snippets, images, or anything other than plain text. A couple of months ago, I came across and was super impressed by its functionality. It is basically like Workflowy as it can also create infinitely nested lists but also pretty much anything else as well (checklists, formatted code snippets, Markdown documents, etc…) as far as organization/productivity tools go. I decided to migrate my work to Notion and continue writing over there. Its polished interface made me a bit more proud of my work in progress and seeded the idea that this body of text that I am developing could be more than what it’s intended to be. With features like code highlighting, image importing, etc. Notion made me see what my work could look like if it was on paper (that is digital paper). Notion Aesthetics do matter. <a href="https://medium.com/media/651a0235c403dcbfcc4c1327162c69e9/href">https://medium.com/media/651a0235c403dcbfcc4c1327162c69e9/href</a> At this point, I noticed I had written more than 150 pages. Using list item was keeping my approach very iterative and their easily collapsible nature was hiding my own progress from myself. I noticed I wrote a book without even realizing. So that’s how you write a book. One sentence at a time, my friend. One sentence at a time. Little things add up. I was only writing a couple of talking points every day, but over time, they gave me enough material to create a book out of them. Having systems that allowed me to work in a consistent manner really helped with achieving a remarkable end result. The choice of the medium was also helpful as well. Lists were keeping things atomic and collapsible lists were hiding the increasing body of work from me, preventing me from self-sabotaging by triggering anxiety. Learning Two: This is where things started to get a bit technical since I had a problem at hand. A book requires a continuous text whereas what I had was a huge bullet list of sentences. I needed to keep both formats for different reasons. A list was helping me with my approach to recording online courses at the time, whereas it was problematic if I were to create a body of text from this document. Enter coding. Part Two: Choosing a Platform for Writing I never really understood why programming might be needed for self-publishing a book. But it was starting to dawn on me. Notion had an option to export my script as a Markdown file. I needed to convert this Markdown file which was essentially a huge nested list into a flat body of text. For this purpose, I used a node.js .) That worked well but the remaining problem was that it was now an HTML document with a huge unsorted list. Still a long haul from being usable. Markdown library . At this point, I started to use another node.js library called to be able to manipulate the resulting HTML into the desired form. Cheerio is a library that offers a JQuery-like syntax on the server-side for the manipulation of hypertext data. Even though Cheerio was helpful to a certain level, it still didn’t get me all the way there. Cheerio As a result, I decided to add type hints to my text. Basically, providing a set of tags to guide Cheerio to do the correct manipulations. I would be adding a {{ h1 }} to indicate the list header needs to be an h1 title, or add a {{ + }} symbol in front of some of the sentences to indicate that an item needs to be merged with the previous one to create a paragraph. Around this time, something terrible happened, though. Notion decided to roll a set of poorly communicated updates that blocked me from working on my text for several consecutive days #firstworldproblems. At the time, Notion was a relatively new service so their desktop application was not great in caching the online data as well. I found myself unable to work on my productive hours because a cloud service was unavailable. This was a hard lesson in not putting too much faith in cloud services. Notion failed me several times over a couple of consecutive days making me realize that I needed to find a new home for my writing. Now, this is not to say Notion is a bad service. It is actually an amazing product. But I made the mistake of relying on it too much at an early stage in their product development where things were still very much in flux and their offline application wasn’t great at syncing with the online platform. You need to have a way of working offline if things are to become inaccessible. You can’t assume cloud will always be there. Learning Three: I started to look for alternatives that would replace my workflow and quickly settled with . It allows you to write Markdown in the browser using the local cache. It can sync with Google Drive in an easy manner as well. This is around the time that I also found out that my previous way of doing recordings wasn’t very inefficient. Recording (and writing) a script only a few sentences at a time is a bit too granular, and slows you down when it comes to editing. This prompted me to transfer all my writing into StackEdit.io as a flat markdown text, completely getting rid of the list structure. StackEdit.io There are no right or wrong tools when you are just starting out. The concept of , , exists in the realm of clickbaity titles of the internet. If I were to get obsessed with my tooling from the get-go, I wouldn’t be able to do any meaningful work. You will get things wrong, and that’s alright. It is part of the journey and the learning process. Learning Four: right wrong best Part Three: Coding for Publishing At this point I had all my writing in StackEdit, I also got rid of all my previously added style hints as I could just be leveraging the Markdown syntax moving on. StackEdit can automatically sync with Google Drive so I had a way of fetching the files I was working on for offline processing as well. I finally had a reliable way of building a pipeline around things. This is where coding came into play in full force. I wanted to be able to create multiple outputs from my Markdown source file. I was already working on sections in the book that were more in-depth which I started to consider having as a premium offering. I wanted to create an ebook out of the material but I also wanted to be able to display the same material on my Jekyll website as well. This already implied four different variations to the source text, like , , , . Another difference between the online and offline output was going to be that I could be using video (gif) files in the online version but these same visuals needed to be static images in the offline version. online-free online-paid offline-free offline-paid Now I was starting to understand how programming can help with my process. I had a single authoritative source and multiple output formats that needed to be managed and this started to look more and more like an automation problem. I started to build node.js scripts to establish my workflow. I was already syncing my Google Drive to a folder on my disk. So I wrote a script that would fetch the desired files from Google Drive to my working folder and place it in a staging area. 'use strict';

const fs = require('fs');

const SOURCE_DIR = '/Users/username/Google Drive';
const TARGET_DIR = './staging';

// remove the target content synchroniously
let targetContent = fs.readdirSync(TARGET_DIR);
targetContent.forEach((file, index) => {
    let targetFile = `${TARGET_DIR}/${file}`
    let isFile = fs.lstatSync(targetFile).isFile();
    if (isFile) {
        fs.unlinkSync(targetFile);
    }
});

fs.readdir(SOURCE_DIR, (err, files) => {
    files.forEach((file) => {
        if (file.startsWith('p5js-') && !file.endsWith('.gdoc')) {
            fs.createReadStream(`${SOURCE_DIR}/${file}`)
                .pipe(fs.createWriteStream(`${TARGET_DIR}/${file}.md`));
        }
    });
}); I was hosting the images that were referenced inside the Markdown file on . I built yet another script that would download these images into the staging folder. I needed images to be on my local for compiling an ebook. imgur Then I created another script that would pre-process the files in the staging area and move them to their corresponding folder that is determined based on the target destination. To be able to render different content in between online and offline versions, as well as the paid and free versions, I decided to use templating language. This solved two main use cases for me: Nunjucks Using Nunjucks, I was able to conditionally render content based on the targeted output format. For example, a gif in my document might be represented like this: {% if online %} 
![01–01](http://i.imgur.com/<id>) 
{% else %} 
![01–01](images/01–01.jpg) 
{% endif %} Having an if-else statement, I can set a variable inside my preprocessing script to decide which conditional to execute and use this data to render the Nunjucks template. const data = {online: true}; 
let renderedContent = nj.renderString(content, data); Nunjucks also allowed me to create variables. I could use a variable like `{{ item }}` in my Markdown text which can render to a value of `book` or `course` depending on the destination I am targeting. (I ended up creating an interactive course using the book material where I needed to refer to the material as a ‘course’, more on that a bit later). Using Nunjucks variables in Markdown. Using the pre-processing script I was also able to manipulate the front matter I had in the original Markdown file using the node library called . This needed to be done since one of my target output format was a .md file for Jekyll and I wanted to automatically add some additional front matter attributes for Jekyll to parse. front-matter This all might sound like terribly over-engineered and unnecessary. And there is a chance that it might very well be. But one thing that I was happy about this process was that even though I yielded to the developer inclination of tooling up in the face of the slightest problem, I didn’t get overly obsessed about building generic, scalable solutions. All this code I wrote is actually pretty ugly and embarrassing to be shared here but the point is to move fast and build automated solutions in a way that’s not going to waste your time. The primary objective is not building systems to deliver your content, it is delivering the content, however, possible. . You should be doing things that don’t scale Definitely read Paul Graham’s essay on doing things that don’t scale. Things that are not efficient can get you massive leverage in the short run. But if you are to get bogged down by the concerns of scalability when you are only starting out then you might miss out on opportunities of growth and sources of motivation like the sense of delivering value to people which might impede and even ruin your progress. Learning Five: One technical decision I regret is using Promises for my file system operations. I think I was trying to prove myself that I was comfortable with using them but they were completely an overkill for my circumstances as I didn’t have any performance concerns. The excessive usage of Promises started to have a mental toll, where I just wanted to move fast, as they are not as straightforward as synchronous operations would have been. Game developer Jonathan Blow has a on optimizing for the cognitive load when developing personal projects. Granted this is not at the scale of anything he is working for but if you are just working on something that needs to work regardless of how you need to make sure that it is usable as possible. Don’t try to be smart, because most days you will dumber than your smartest self. great talk I also created a post-processing file that I would run for some specific purposes. For example sending the document to a copy editor ( I worked with a freelancer on , I didn’t need to have the code snippets in there as they were ballooning up my word count and hence affecting the pricing. Having an automated workflow allowed me to remove them easily. There was an instance where I was faced with an opposite problem where I just needed the code snippets. This was again solved quickly by using the post processing file to selectively remove the target elements (anything that is not a code snippet) from the files. Fiverr I ended up publishing my work on three different platforms. I first published the book on . Having already established a pipeline, it was very easy to integrate my work with Leanpub and use their Github integration. After pushing my work on Leanpub, I exported an unbranded version of the book using their tools and placed it on Amazon Kindle Store. has an amazing blog post on the self-publishing platforms. It is a must-read for anyone that is interested in this space. Leanpub Zapier The most amazing discovery for me was coming across through a post on Hacker News. Educative.io is an online course creation site where you can build interactive courses using blocks that allow you to add executable code snippets (among many other things) inside a document in an easy manner. This allows you to create documents that are similar to Jupyter notebooks. Transferring my source text to their platform was a breeze as it uses the Markdown format as well. Educative.io I am not claiming that my workflow was perfect. One big shortcoming was that gathering online feedback on my text from other people was pretty hard. For that use case, working in Google Docs would have been much more useful. But unfortunately, Google Docs doesn’t offer a great way of working with Markdown files. Also using Nunjucks templating in your source text introduces a little bit of an overhead as you can’t just copy-paste text, you need to process — compile — it first. But considering the efficiencies that were gained, I find this to be a reasonable tradeoff. This summarizes my journey. If there is a final lesson to be derived here I think it is not to be overly obsessed with tools and best-practices from the get-go and just start with creating things. It is the content and the product that really matters and all the other concerns are secondary, at least initially. Don’t let yourself get slowed down by choices, you probably don’t know enough at the beginning to inform your decisions so it is important to get started and iteratively adjust according to your emerging needs. for making it this far! Feel free to check out and feel free to reach out to me on or through with your comments, suggestions and questions. Thank you Coding for Visual Learners Twitter my website Also to and Leigh at for reading drafts of this. thanks Hoi-En Myplanet

Writing a Book as a Programmer

The United States v Meta Platforms Court Filing October 24, 2023 is part of . You can jump to any part in this filing . This is part 78 of 100. HackerNoon’s Legal PDF Series here COUNT XXXIII: VIOLATIONS OF THE NEW JERSEY CONSUMER FRAUD ACT, N.J. STAT. ANN. §§ 56:8-1 to 227 1045. New Jersey realleges and incorporates by reference each and every factual allegation in the paragraphs above as if the same were fully set forth herein. 1046. As set forth above and at all relevant times, Meta engaged in and continues to engage in commercial practices pursuant to the New Jersey Consumer Fraud Act (CFA), N.J. STAT. ANN. §§ 56:8-1 to 227. 1047. These commercial practices were and continue to be made in connection with the sale and advertisement of merchandise. 1048. These commercial practices constitute unconscionable or abusive commercial practices in violation of the CFA. 1049. These commercial practices constitute acts of deception, fraud, false pretense, false promise, and misrepresentation in violation of the CFA. 1050. These commercial practices knowingly conceal, suppress, and omit material facts with the intent that consumers relied upon the concealed, suppressed, and omitted material facts. 1051. The conduct described in Count I is conclusively presumed to be to be an unlawful act in violation of the CFA. N.J. STAT. ANN. § 56:8-4b. 1052. These commercial practices were and continue to be material to the sale and advertisement of merchandise. 1053. While engaging in the acts and practices alleged in this Complaint, Meta knew or should have known that that its conduct was of the nature prohibited by N.J. STAT. ANN. § 56:8-2, subjecting itself to enforcement and penalties as provided in N.J. STAT. ANN. §§ 56:8-8, 11, 13, 14, and 15. 1054. Each unlawful practice alleged herein constitutes a separate violation of the CFA. Continue Reading . Here About HackerNoon Legal PDF Series: We bring you the most important technical and insightful public domain court case filings. This court case 4:23-cv-05448 retrieved on October 25, 2023, from is part of the public domain. The court-created documents are works of the federal government, and under copyright law, are automatically placed in the public domain and may be shared without legal restriction. Washingtonpost.com

New Jersey Unleashes Legal Firestorm on Meta for Violations of Consumer Fraud Act

Use , and to display real-time changes in an application. This is the main theme of today's blog. Streams Firebase Cloud Functions, FireStore Intro Hello and Welcome to the 12th of the Flutter App Development Tutorial Series. This is Nibesh from Khadka's Coding Lounge. We traveled a long way to be here. We created a , wrote a , made a made an , set up a connection with firebase , , and accessed s on firebase projects. splash screen theme custom app bar, authentication screen cloud and emulators authenticated users users’ location We'll pick right where we left off and toggle the favorite icon for temples. We'll use Firestore to store all the favorites list for each user's document in the " " collection. The Firebase Function will help us fetch the immediate list and update it. While Stream will provide the changes in real-time for users to see. You can find the source code so far . last time users here Firebase Cloud Functions To Update  Firestore In RealTime Now, we'll create a cloud function in our file. This function will take an "id" as input. This id is provided by Google Maps Places API. index.js place_id It'll search and fetch the current user's document. We've already created an array while registering with the onCreate trigger. favTempleList The function then will look through the list and toggle the value. // Add temple to my fav list:
exports.addToFavList = functions.runWith({
    timeoutSeconds: 120,
    memory: "128MB"
}).https.onCall(async (data, context) => {
    const templeId = data.templeId;
    try {
        // Get user doc
        let userDocRef = await db.collection('users').doc(context.auth.uid).get();

        // extract favTempleLis from the doc
       // #1
        let favTempleList = userDocRef._fieldsProto.favTempleList;

        // if fav list is empty 
        // #2
     //============================//
        if (favTempleList.arrayValue.values.length === 0) {
            // Put the id in the list 
            const templeList = [templeId];
            functions.logger.log("Fav list is empty");
            
            // Update the favTemple list
            await db.collection('users').doc(context.auth.uid).set({ favTempleList: templeList }, { merge: true });
//============#2 ends here=====================//
        } else {
            functions.logger.log("Fav Temple List is not empty");
   
            // Make list of available ids
            // firebase providers arrays values as such fileName.arrayValue.values array 
            // consisting dictionary with stringValue as key and its value is the item stored
            // #3
            functions.logger.log(favTempleList.arrayValue.values[0]);
            let tempArrayValList = favTempleList.arrayValue.values.map(item => item.stringValue);

            // if not empty Check if the temple id already exists
           // #4
            let hasId = tempArrayValList.includes(templeId);

            // if so remove the id if no just add the list 
           // #5
//============================//
            if (hasId === true) {

                // Usr filter to remove value if exists
                let newTemplesList = tempArrayValList.filter(id => id !== templeId);

                await db.collection('users').doc(context.auth.uid).set({ favTempleList: newTemplesList }, { merge: true });
//==============#5 ends here===========//
            }
            // If the id doesnot already exists
           // #6
//============================//

            else {
                // first create a fresh copy
                let idList = [...tempArrayValList];

                // add the new id to the fresh list
                idList.push(templeId);

                // update the fresh list to the firesotre
                await db.collection('users').doc(context.auth.uid).set({ favTempleList: idList }, { merge: true });
//==============#6 ends here===========//

            }

        }

    } catch (e) { functions.logger.log(e); }

    // Return the Strig done. 
    //#7
    return "Done";
}); A couple of lines of code here are on other functions and triggers we've already done. So, let's only go over a few important ones: We are extracting the firebase field named "favTempleList" from the user documentation. That field is an array. We're checking if that array is empty. If so, we can just add the new place_id we got from the function call to the array and then update the doc. One thing to notice here is that I am using the merge option for an empty array. That's because this set() method doesn't just set/update a field in a doc, it sets/updates the whole document itself. So, if the **{merge: true} **option is not provided it'll overwrite every field there, in our case username, email, location, and so on. Now, we enter the section where the temple list is not empty. Then first we need to extract the list of values(id's) that are already present there. Check if the new ID, we got from the method call, is already present in the list we got from Firestore. If the id is already there then we need to remove it. We'll combine and methods. After filtering the current place_id, we'll update the Firestore user doc with the fresh list of ids. filter() includes() If the place_id is not there then we need to add it. We'll push the current place_id into the list. And then update the doc like before. Remember we should permanently terminate Firebase functions in the end. We don't need any data from this so we'll just terminate it by returning a string. Provider Now, we'll need to add a method in our Provider class that'll call the HTTPS callable function we just created. In our TempleProvider class let's add another method . addToFavList void addToFavList(String templeId) async {
    // Instantiate callable from index.js
    HttpsCallable addToFav = functions.httpsCallable('addToFavList');
    try {
      // Run the callable with the passing the current temples ID
      await addToFav.call(<String, String>{
        'templeId': templeId,
      });
    } catch (e) {
      rethrow;
    }
  } We're not updating or returning anything here. That's because we can get data from snapshots from a stream, as you'll see later. BTW, you could add this method to AuthStateProvider because it deals with user collection. Use Stream & StreamBuilder To Output Changes In RealTime We can simply use the for this. So, we'll connect with Firesotre with streams and update the screen with StreamBuilder.  So, where exactly are we using this StreamBuilder? Good question, you see getting real-time updates, means reloading(re-reading) the same collections on every change. It is obviously memory expensive. But it can cost expensive as well since Firebase charges for several reads. So, we don't want to load the list of temples, again and again, to toggle a single favorite icon. So, let's wrap only our favorite icon with stream builder instead. streams On temple_item_widget.dart make these changes. Create a function that'll call the addToFavList method from the provider class. // function to call addToFavList from provider class
  // It'll take id and providerclass as input
  void toggleFavList(String placeId, TempleProvider templeProvider) {
    templeProvider.addToFavList(placeId);
  } Inside the build method of class before the return statement. // Fetch the user doc as a stream
//#1
 Stream<DocumentSnapshot> qSnapShot = FirebaseFirestore.instance
        .collection('users')
        .doc(FirebaseAuth.instance.currentUser!.uid)
        .snapshots();

// Instantiate provider method to pass as argument for tooggle FavList
//#2
    TempleProvider templeProvider =
        Provider.of<TempleProvider>(context, listen: false); As mentioned before we'll get the Stream from Firestore, a document using the current user's ID. Instantiate provider method to pass as an argument for toggleFavList function. But make sure to turn off the listener. Replace FavIcon Section With StreamBuilder StreamBuilder(
                      // Use latest update provided by stream
                      // #1
                      stream: qSnapShot,
                      builder: (context, snapshot) {
                        if (snapshot.connectionState ==
                                ConnectionState.waiting ||
                            snapshot.connectionState == ConnectionState.none) {
                          return const CircularProgressIndicator();
                        } else {
                          // Get documentsnaphot which is given from the stream
                         // #2
                          final docData = snapshot.data as DocumentSnapshot;

                          // Fetch favTempleList array from user doc
                         // # 3
                          final favList = docData['favTempleList'] as List;

                          // Check if the curent widget id is among the favTempLlist
                         // #4
                          final isFav = favList.contains(widget.itemId);

                          return Expanded(
                            child: IconButton(
                                // Call toggleFavlist method on tap
                               // #5
                                onPressed: () => toggleFavList(
                                    widget.itemId, templeProvider),
                                icon: Icon(
                                  Icons.favorite,
                                  // Show color by value of isFav
                                 // #6
                                  color: isFav ? Colors.red : Colors.grey,
                                )),
                          );
                        }
                      }) Here, in the StreamBuilder Class we: Used the stream we got from Firestore 'users' collection as a stream. That way the latest change is
reflected asap. When the ConnectionState is done, first we get the latest data from the snapshot, which will be the current user's doc. Get the latest updated favTempleList array as a List from the document. Check if the current ID exists in the favItemList, which is the field passed down from the temples screen. itemId On pressing the heart icon button call the toggleFavList method of the class. Change the color of the heart icon based on the presence or absence of itemId on favTempleList. And with that latest real-time changes will be reflected in the app. Final Code The file looks like this after the changes. temple_item_widget.dart import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:firebase_auth/firebase_auth.dart';
import 'package:flutter/material.dart';
import 'package:provider/provider.dart';
import 'package:temple/screens/temples/providers/temple_provider.dart';

class TempleItemWidget extends StatefulWidget {
  // Fields that'll shape the Widget
  final String title;
  final String imageUrl;
  final String address;
  final double width;
  final String itemId;

  const TempleItemWidget(
      {required this.title,
      required this.imageUrl,
      required this.address,
      required this.width,
      required this.itemId,

      // required this.establishedDate,
      Key? key})
      : super(key: key);

  @override
  State<TempleItemWidget> createState() => _TempleItemWidgetState();
}

class _TempleItemWidgetState extends State<TempleItemWidget> {
  // function to call addToFavList from provider class
  // It'll take id and providerclass as input
  void toggleFavList(String placeId, TempleProvider templeProvider) {
    templeProvider.addToFavList(placeId);
  }

  @override
  Widget build(BuildContext context) {
    // Fetch the user doc as stream
    Stream<DocumentSnapshot> qSnapShot = FirebaseFirestore.instance
        .collection('users')
        .doc(FirebaseAuth.instance.currentUser!.uid)
        .snapshots();

// Instantiate provider method to pass as argument for tooggle FavList
    TempleProvider templeProvider =
        Provider.of<TempleProvider>(context, listen: false);

    return SizedBox(
      // Card will have height of 260
      height: 260,
      width: widget.width,
      child: Card(
        key: ValueKey<String>(widget.itemId),
        elevation: 4,
        shape: RoundedRectangleBorder(
          borderRadius: BorderRadius.circular(10),
        ),
        margin: const EdgeInsets.all(10),
        child: Column(
          // Column will have two children stack and a row
          children: [
            // Stack will have two children image and title text
            Stack(
              children: [
                ClipRRect(
                  borderRadius: const BorderRadius.only(
                    topLeft: Radius.circular(10),
                    topRight: Radius.circular(10),
                  ),
                  child: Image.network(
                    widget.imageUrl,
                    fit: BoxFit.cover,
                    width: widget.width,
                    height: 190,
                  ),
                ),
                Positioned(
                  bottom: 1,
                  child: Container(
                    color: Colors.black54,
                    width: widget.width,
                    height: 30,
                    child: Text(
                      widget.title,
                      style: Theme.of(context)
                          .textTheme
                          .headline2!
                          .copyWith(color: Colors.white),
                      // softWrap: true,
                      overflow: TextOverflow.fade,
                      textAlign: TextAlign.center,
                    ),
                  ),
                ),
              ],
            ),
            Row(
                // Rows will have two icons as children

                crossAxisAlignment: CrossAxisAlignment.center,
                mainAxisAlignment: MainAxisAlignment.spaceEvenly,
                children: [
                  Expanded(
                    child: IconButton(
                      onPressed: () {
                        print("Donate Button Pressed");
                      },
                      icon: const Icon(
                        Icons.attach_money,
                        color: Colors.amber,
                      ),
                    ),
                  ),
                  StreamBuilder(
                      // User the ealier stream
                      stream: qSnapShot,
                      builder: (context, snapshot) {
                        if (snapshot.connectionState ==
                                ConnectionState.waiting ||
                            snapshot.connectionState == ConnectionState.none) {
                          return const CircularProgressIndicator();
                        } else {
                          // Get documentsnaphot which is given from the stream
                          final docData = snapshot.data as DocumentSnapshot;
                          // Fetch favTempleList array from user doc
                          final favList = docData['favTempleList'] as List;
                          // Check if the curent widget id is among the favTempLlist
                          final isFav = favList.contains(widget.itemId);
                          return Expanded(
                            child: IconButton(
                                // Call toggleFavlist method on tap
                                onPressed: () => toggleFavList(
                                    widget.itemId, templeProvider),
                                icon: Icon(
                                  Icons.favorite,
                                  // Show color by value of isFav
                                  color: isFav ? Colors.red : Colors.grey,
                                )),
                          );
                        }
                      })
                ]),
          ],
        ),
      ),
    );
  }
} Summary In this exciting blog we: Created a Firebase HTTP Callable function to update a document in Firestore. We also wrote a simple instruction in the provider class to handle the callable. Lastly using stream and stream builder we displayed real-time changes on our screen. Show Support Alright, this is it for this time. The next part will be the last one for this series. We won't be doing anything new there. I'll give you some tasks to do on your own to continue this app and practice what you've learned so far. I'll also enlist some courses, blogs, and books that have helped me a lot to learn flutter. Please like, comment, and share the article with your friends. Thank you for your time and for those who are subscribing to the blog's newsletter, we appreciate it. Keep on supporting us.  This is Nibesh from Khadka's Coding Lounge, a freelancing agency that makes websites and mobile applications. Also Published here

How to Stream Real-time Changes With Firebase, Firestore and Flutter

Hello there, When I first working with FireStore I had to face many issues How can I filter documents by some criteria? How can I insert a new document from a JSON, in fact, I have to write a script to insert some new document I have to use GC Storge to backup the data, which is I need to spend more time learning about it. WTF 🤬 why not just export and import by a JSON file? If I edit the same field of many documents, I need to go through documents and documents to edit. Is there any tool to interact with FireStore like - A question pop on my head. But you know what TablePlus So yeah, I think many of you will phase the same issues like me, so I decided to make one. Check out it at https://refiapp.io Features The first version includes: 🔐 - Your privacy is our first priority. We do not keep any of your data Privacy We never ever send any of your data. Your data is your and always your 📜 - Easy view & edit a large amount of data like Excel Table view 🗒️ - You can add, edit documents like in your editor. No more wasted time clicking to add just a document JSON Editor We integrated - aka so you can feel like 🏠 Monaco Editor VS Code core 👨‍💻 - With hotkeys, you can do anything without leaving your keyboard. We are developer, we love keyboard Build for Developer Press or to open Cmd + Shift + P Cmd + / Command list 🚧 - Confidence edit data without making Production crash Preview changes P/S This is an early version, so let help me make it better by giving me some feedback. I think we can make it better P/S 2 This tool is built by Electron, Typescript, React, Vite And the Homepage is built by Cloudflare worker, Notion as backend There some many interesting when building it, I think it would be shared with the community. Which one make you interesting most?

Refi App Allows Devs to Get Through 4 Firestore Issues at Once: Learn More

Guide to Firebase Cloud FireStore Database in Android

I am a great reader, I love books and I try to read as much as I can, no matter the topic, whatever, fantasy, comedy, sci-fi, educational...Books take me to another world, they make me feel, make me think, make me relax and make me disconnect from the day by day schedule. I cannot live without them. This is why my first approach to Javascript was to create a Library to store my books, with some information about them ( title, author, number of pages…) and if I read it or not. I know, it was a small, simple project, but what do you expect? It was my very first one, so I only wanted to practice as much as I could and make it work. So, I created my HTML file, my `script.js` file and worked a bit on my CSS to build something nice to my eyes. After working on it for a while, I got my site looking good and my library working fine. I had a table with my books, with all the information related and a “NEW BOOK” button that brought up a form allowing me to input the details for a new book to add to the library. All the books were an instance of the constructor Books, with a prototype to toggle the read status and all of them stored in an array. Something like this: { .title = title; .author = author; .pages = pages; .read = read;
}

Book.prototype.info = { .title + + .author + + .pages + + .read;
}
Book.prototype.toggleReadStatus = { .read = .read == ? : ;
} myLibrary = []; ( ) function Book title, author, pages, read this this this this ( ) function return this " by " this ", " this " pages, " this ( ) function this this 'Read' 'Not read' 'Read' let Perfect! I was so happy so I started to add read books and books that I planned to read, I spent some time with it. Actually, it was my first JS creation and wanted to play around a bit. After all the work I went to have lunch and went came back to my computer and opened my page again...surprise! No one book there, all of them gone! My array was empty, and all my previous work adding content as well. Bad luck or just a beginner mistake, I thought all the data will remain in my array and would keep it there forever, but it is not working that way. After some research on the web, I understood that I needed to store the data somewhere or it would vanish every time I refreshed the page. LocalStorage My first approach to the storage was using , `LocalStorage` is a type of web storage that allows JavaScript sites and apps to store and access data right in the browser with no expiration date. This means the data stored in the browser will persist even after the browser window has been closed. LocalStorage Storage objects are simple key-value stores, similar to objects, but stored in `string` format. These web storage objects are not sent to the server with each request. Because of that, we can store much more. Most browsers allow at least 2 megabytes of data (or more) and have settings to configure that. The use of the `localStorage` is quite straight-forward, has no secrets, it provides some methods to work with it: setItem() With this method, you store a key/value pair into `localStorage`. It takes two parameters, the key and the value. Like this: .localStorage.setItem( , );
getItem() window 'book' 'The Lord of the Rings' With this method, you access to local storage by a key given. For example: .localStorage.getItem(‘book’); window // The Lord of the Rings removeItem() Providing the key, this method will remove the key and its values. .localStorage.removeItem(‘book’); window clear() This method, when invoked, clear the `localStorage` deleting everything. .localStorage.clear(); window length Returns the length of the `localStorage`. .localStorage.length; key() window // returns an integer This method requires one parameter, an index, and return the key on that position in the `localStorage`; .localStorage.key(index); window We have already said that `localStorage` store all the data in `string` format, so how do we do to store objects? My library array, for example... To do so we need to convert our object into a string, or into a JSON. We can do that: book = { :’ ’, :’George Orwell’
} .localStorage.setItem(‘books’, .stringify(book); const name 1984 author window JSON When we call method, it will return us a string with values, something like that: getItem() .localStorage.getItem(‘books’)
“{“book”:” ”,”author”:”George Orwell”}” window 1984 In this case, we need to convert it into an object using the method JSON.parse(). .parse( .localStorage.getItem(‘books’)); JSON window And we will get our object back. We neither can iterate over `localStorage`, but there is a way we can apply to do so, using the key and the length methods: ( i= ; i<localStorage.length; i++) { key = localStorage.key(i);
  alert( );
} for let 0 let ` : ` ${key} ${localStorage.getItem(key)} So...cool! After learning all of that I implemented it in my Library project, and this time it worked as I wished. Great!!! But there was a problem, I always want more and more, and the problem with `localStorage` is that is only accessible on the computer that it was created on. So I could not access my library from any other device or when I am not at home. LocalStorage was a solution, but not the one I was looking for. So, back to google and spend some more time researching on the web, until I found Firebase. Firebase Firestore is an online database that can be set up relatively easily, allowing you to save your data to a server in the cloud. Well, actually, it is not only that, Firebase is a Backend-as-a-Service (BaaS). That means that it offers different services like authentication, analytics, hosting, cloud storage…. Firebase I paid special attention to . So I started to read the documentation and learn from an online tutorial they provide (you can find it ). Cloud Firestore here When I finished it I went back to my project and spent some time thinking about how to implement it. I already had my Library working so it was only a few changes to make. The first step is to Sign into Firebase website using your Google account, for example. And create a Firebase project to connect to your app. No secrets to do that, just click on the ‘Add Project’ and add a Project Name,  you can also set up Google Analytics for your project, which enables you to use many of the Firebase’s products. After that, you need to add your app to your Firebase project. We do that by clicking the Web icon (</>). Then follow the on-screen instructions to add the app to the project. You have to provide a nickname for your app and they will give you the Firestore SDK scripts  <script src="https://www.gstatic.com/firebasejs/7.17.1/firebase-app.js"></script> <script src="https://www.gstatic.com/firebasejs/7.17.1/firebase-analytics.js"></script> <script src="https://www.gstatic.com/firebasejs/7.17.1/firebase-firestore.js"></script> <script src="https://www.gstatic.com/firebasejs/7.16.1/firebase-database.js"></script> of    And the firebase config firebaseConfig = { : , : , : , : , : , : , : , : ,
}; var apiKey "api-key" authDomain "project-id.firebaseapp.com" databaseURL "https://project-id.firebaseio.com" projectId "project-id" storageBucket "project-id.appspot.com" messagingSenderId "sender-id" appId "app-id" measurementId "G-measurement-id" Then initialize Firebase in the same script block where you added the configuration. firebase.initializeApp(firebaseConfig);
firebase.analytics(); You should add all of that lines at the bottom of your HTML file. At the Firebase console, on the left, click on ‘database’ and create a database. Select a starting mode for your Cloud Firestore Security Rules (test mode is recommended for web apps, and choose the location for your database). When the process completes, you'll be taken to the overview page for your Firebase project in the Firebase console. And your project in Firebase is ready and connected to your app Once we have set up all the firebase configuration the next step is add our data to our database, in my case, my library. Think that Cloud Firestore stores data in Documents, which are stored in Collections. Cloud Firestore creates collections and documents implicitly the first time you add data to the document. You do not need to explicitly create collections or documents. Check this for more information. video In my case, I added these lines of code to my js file: firestore = firebase.firestore(); docRef = firestore.collection( ); let const 'books' Where I initialize an instance of Cloud Firestore, and then I create my collection which I called ‘books’. Everything ready to add, receive and update data in my Cloud Firestore Database. Firebase assigns one ID to each entry in your collection, but for readability, I decided to change it and add the title of the book instead, so when I go to my console I can see, in my collection, all the entries with the title and not a bunch of letters and numbers. To add my data I just add the input of the form, when I create a book entry, and set it to the database. Like this: docRef.doc( ).set({ : newBook.title, : newBook.author, : newBook.pages, : newBook.read
      }).then( { .log( );
      })
      .catch( { .error( , error);
}); ` ` ${newBook.title} title author pages read ( ) function docRef console "Document successfully written!" ( ) function error console "Error adding document: " As you can see, here we use asynchronous functions to add the data to our database, setting our new book as an object. If you go to your database in the Cloud Firestore console you will see the collection ‘books’, all the books added as documents and for each document, we can find the title, author, pages and the read status. I had my library on the cloud and I could access it from my site and from the console. Cool! Time to update my library, every time I read a book I need to update the read status from ‘Not Read’ to ‘Read’. I added these lines to my script: docRef.doc( ).update({ : item.read
 }); ` ` ${book.title} read With the method update you can modify your documents by the key given, in my case I only wanted to modify the read status so I only changed that field. How do we remove a document? As everything with Firebase, it is not complicated, I only needed to use the delete() method and add the key I wanted to delete. And...deleted! No secrets! docRef.doc( ).delete(); ` ` ${book.title} And that was all, I got my library ready to access from everywhere and I could access my data from my app and from my Cloud Firestore Database. Conclusion As you can see store the data of your projects it will not be a problem from now on. You can use your browser’s `localStorage` and keep the data locally forever, and if you want to access it from everywhere you can easily implement Firebase Firestore in your project and access your data in an easy and secure way. I usually implement Firebase in most of my projects since it's an amazing tool. You can also use it for your hosting, authentication and many other things it offers. That will come in another article very soon. Thank you for reading. Happy coding!

How To Implement localStorage or Firebase Firestore into your JS project

For the past 20 hours, I have been trying to pick up Firebase Firestore and to build a queue system consisting only the View-layer of a typical MVC/MVT architecture, which is named ‘ ’. is a queue system that works similarly to the one we typically encounter in banks, government agencies, etc. However, instead of getting a small piece of paper with your queue number on it, you now capture your queue number from the screen. Schlange Schlange I build Schlange primarily for these purposes: to test if I could eliminate the web service or server layer by incorporating the logic into the View-layer and interact directly with the Data Store (Of course this is only applicable to insensitive data and simple logic); to test the usability of Firebase API; to test the real-time synchronicity of Firebase Firestore; and how feasible it is to integrate with Vue.js; Overview of Schlange There are 3 views of , a Display Panel view, an Agent view and a New Number Panel. The left window is the Display panel showing the queue to be served. The top right window is the Agent view for queue server and the top bottom right window is the panel for users to request for queue number. Schlange Whenever a new queue number is requested (by using the New Number Panel at the bottom right window), a read of latest next queue number is performed, the number is then enqueued into the ‘pending’ list. The ‘pending’ list is reflected in the ‘Next’ column of the display panel. Simple enough. My focus is not on the queue system but its architectural design. Architecture of Schlange In the case of , Vue.js is used as the View-layer framework. For real-time data store, I use Firebase Firestore. The users of interact with the 3 panels of and any IO operation to the data store is handled by the panels themselves. Schlange Schlange Schlange In , a new number request is a ‘creation of element’ in the ‘pending’ list. In Firestore, there isn’t any straightforward way to store a list. Instead, the ‘pending’ list is stored as a Firestore Collection. The real-time synchronicity of Firestore allows any changes to be reflected in no time. Since there is no complex logic in , the logic to perform this operation doesn’t have to be separated. Oh! It’s architecture! Schlange Schlange fat-client-thin-server Data Model of Schlange These are the data models of Schlange. "pending":{  1005: {    "number": 1005  },  1006: {    "number": 1006  }}"serving":{  1004: {    "number": 1004  },  1003: {    "number": 1003  }}"served": {  1002:{    "number": 1002  },  1001: {    "number": 1001  }} To enqueue, simply add a new Document into Firestore so the “pending” collection looks like: "pending":{  1005: {    "number": 1005  },  1006: {    "number": 1006  },  1007: {    "number": 1007  }} Whenever there is an update to the data store, it will be synchronized to the display panel. You can learn about the data model in Firestore from their documentation . How is Schlange Data-Centric? A data-centric application is an application that relies heavily on the connection to a database. is nothing different from many conventional systems dealing with data all the time. If there is a network failure, the disconnect of the data store is going to break the system. Therefore, the weakness of a data-centric is pretty obvious. Once the connection is down, the system is down. Schlange How is Schlange Serverless? According to , he describes a serverless application as an application that runs its logic in a stateless container or application that uses vast ecosystem of cloud accessible databases or services. He even exemplifies it with a client application that interacts directly with the data store. So, is one of it. Martin Fowler’s explanation on Serverless architecture Schlange How to Integrate Vue.js with Firestore? I will be posting this topic in near future and I will add a link here once it’s done. Stay tuned! 😊 What I’ve learnt? So, these are what I’ve learnt based on my purposes: to test if I could eliminate the web service or server layer by incorporating the logic into the View-layer and interact directly with the Data Store -> Yes, I could. But there is one caveat I have to take care of is the access permission of data. Good news is, it is enforceable by using Firestore Rules and incorporating Firebase Auth to test the real-time synchronicity of Firebase Firestore; -> The synchronicity is satisfying but the actual latency depends on how you integrate it with your application and how feasible it is to integrate with Vue.js; -> There is a way, but I suspect there is performance trade-off. I will look into it further to test the usability of Firebase API; -> Their JS API uses Promise extensively, a great mechanism to handle async calls. I have posted about promisifying callback API in a native way, you can check it out . here Demo You can find the demo at . http://schlangeplayground.herokuapp.com Your clap will definitely drive me further. Give me a clap if you like this post.

Building a Serverless Vuejs App with Firebase Firestore

Not so long ago Firebase team announced new called A new database has many improvements and new abilities. Even now in beta, it has almost the same limitation like the old brother (100 000 concurrent connections). From obvious advantages: document database Firestore. Firebase Realtime Database scalability multi-region support document-oriented data-structure! easy querying offline support (even for web) So, if you feel ready to dive into the future of this tutorial is for you! serverless technologies During this tutorial, I will call as . Firebase Realtime Database RTDB How to import data? The first question you might ask yourself is how to push data to the new database. Old was great with the features of export/import database as the single JSON file. RTDB For the new we need to implement a mechanism by yourself. Let’s assume you are trying to migrate from to , so you need to export your current database snapshot. If you don’t have RDBM instance you can just use any JSON data to test this approach. Cloud Firestore, RTDB Firestore Script to load data Why instead of Firebase admin SDK Firebase SDK? At the current point, there are still problems with pushing nested arrays to firebase using (tested in 4.6.1). If you try to set nested arrays using Firebase SDK you will get something like this: Firebase SDK _Function DocumentReference.set() called with invalid data. Nested arrays are not supported._ Authentication To authorize our application we need to export service key. Firebase Console In the click on the settings wheel next to the section and choose option. Firebase Console Overview Users and permissions You’ll be landed on . Choose tab and fill it with data as shown in the screenshot above. Fill checkbox so you will be able to your key and use it in the script. After creation, you will be asked for the location to save the key. Let’s save it as permissions page Service accounts Furnish a new private key, download service-key.json const admin = require('./node_modules/firebase-admin');const serviceAccount = require("./service-key.json"); admin.initializeApp({credential: admin.credential.cert(serviceAccount),databaseURL: "https://YOUR_DB.firebaseio.com"}); — property will be filled by just created credentials service account key — is the name of your Database instance + firebaseio.com databaseURL and https://cf-studio-dev.firebaseio.com https://cf-studio.firebaseio.com Rest of the script As long as Arrays are also objects this script is fully ready to use. $ node json-to-firestore.js Keep in mind that script relies on few files: — npm module, you can install it locally to the folder of the script firebase-admin — file we’ve generated in the section service-key.json Authentication — actually the data we want to push to our data.json Firestore Conclusion looks very promising. And even despite that fact that it’s doesn’t mean you should not get your hands on it. Already well known Firebase product called (still in beta) have already been used by thousands of projects. Cloud Firestore still in beta Cloud Functions In this article, I’ve covered an easy way to submit data to . If you are interested in configuring with the Framework Learn . JSON Cloud Firestore Firebase hosting Angular How Get to From Zero to Production with Angular, Firebase, and GitLab CI

How to import data to Cloud Firestore?

I recently started to build several projects in React and React Native. The first one is a . The second on is a set of . With the third one, currently in progress, I’m trying to use the latest novelties in the space. Novelties such as the . So when Firebase announced its latest baby, Cloud Firestore, I started to migrate my code to the new product. While doing so, I noticed a redundancy when doing the data binding between MobX and Firebase as well as an opportunity to factor it. fitness app premium starter kit for React Native wonderful Material UI v1-beta If you use Firestore with React and MobX, a big chunk of your code is likely following the pattern below. @observerclass MyComponent extends Component<MyComponentProps, {}> {    render() {        const {loading, data} = this.store;        if (loading) {            return <LoadingIndicator />;        } else if (!data) {            return <NotFound />;        } else {            return <DisplayData {...{data}} />;        }    }} When the user load a screen, you request the necessary data from Firebase. While waiting for the answer, you are likely to display a loading indicator of some kind. The data might not be found or an error might happen. In that case, you need to display a status message to the user, typically a “not found” or error screen. If the data is successfully delivered, you can finally display the component. This means that there can be a lot of redundant code for every screen of your app. The code snippet below is a typical example of how this flow looks like. @observable loading = true;@observable data: Data;async componentWillMount() {    try {        const snapshot = await fb.doc(`collection/doc`).get();        if (snapshot.exists) {            this.data = snapshot.data();        }    } catch(e) {        // There can be some extra error processing    }    this.loading = false;} However this logic we just described can be captured by the following interface: interface FireDocument<T> {        loading(): boolean;        exists(): boolean;        get(): T;} And below is an example of this interface in action. @observerclass MyComponent extends Component<MyComponentProps, {}> {    store: FireDocument<Data>;        componentWillMount() {        this.store = new FireDocument(fb.doc(`collection/doc`));    }        render() {        const {store} = this;        if (store.loading()) {            return <LoadingIndicator />;        } else if (!store.exists()) {            return <NotFound />;        } else {            return <DisplayData {...{data}} />;        }    }} Pretty slick right? I published an implementation of and . Please let me know if this is useful to you and if you would like me to publish this as a proper NPM package. FireDocument FireCollection here Filter & Map The is almost identical to . It takes a query snapshot as argument. FireCollection FireDocument const query = firebase.db.collection(`collection`)                                .where("prop", "==", "value");this.store = New FireCollection(query); When dealing with collections, there are a couple some minor things to consider. You might want to filter some of the items on the client side and transform the result to another representation. this.store = New FireCollection(query);this.store.setFilter(doc => doc.data().prop === true);this.store.setTransform(docs => ({  docs: docs.map(doc => doc.data())})); Do you find this wrapper useful? Should I publish it as proper package? What are the other possible ways to integrate Firestore and MobX together? One last question This package takes care of the data binding part of the equation. What about the rendering? Maybe we can build a decorator that would make our previous example look like this: const config = {  Loading: <LoadingIndicator />,  notFound: <PageNotFound />,  resolve: (routerParams) => `users/${routerParams.uid}`};@bindDocument<User>(config)class User extends Component<UserProps, {}> {    render() {        const {user} = this.store;        return <div>Hello ${user.name}</div>;    }} Could a decorator like this one be useful? I’m looking forward to read your thoughts on this topic. Looking for a premium React Native starter kit that integrates with Cloud Firestore? Checkout the story below. _I recently built a new version of the React Native Fiber starter kit that integrates with Firebase. You can check it…_hackernoon.com Live Coding — Integrating Firebase in React Native

A Useful Wrapper for MobX and Firebase Cloud Firestore

This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Salim Chouaki, LIX, CNRS, Inria, Ecole Polytechnique, Institut Polytechnique de Paris; (2) Oana Goga, LIX, CNRS, Inria, Ecole Polytechnique, Institut Polytechnique de Paris; (3) Hamed Haddadi, Imperial College London, Brave Software; (4) Peter Snyder, Brave Software. Table of Links Abstract and Introduction Background Measurement Methodology Results Limitations Related Work Conclusion, Acknowledgments, and References Appendix 6 RELATED WORK Search engines and online tracking received a lot of research attention. We review studies closest to our work. A first line of work has measured to which extent we can observe personalization in search engine results [23, 34] and ads [22]. For instance, Hannak et al. [23] have developed a methodology for measuring personalization in search results, applied it to Bing, Google, and DuckDuckGo, and found that Bing results are more personalized than Google ones while they did not notice any personalization for DuckDuckGo. A second line of work has focused on solutions to protect users’ privacy from search engines and prevent web profiling. Castellà-Roca et al. [12] presented a computationally efficient protocol that provides a distorted user profile to the search engine to preserve users’ privacy. Finally, several studies have proposed privacy-preserving search-personalizing solutions for search engines. For instance, Shen et al. [36] analyze various software architectures for personalized search and envision possible strategies with a client-sided personalization. Xu et al. [40] suggest helping users choose the content and degree of detail of the profile information built by search engines. To the best of our knowledge, there is no study investigating the privacy properties of the advertising systems used on private search engines. Search engines. Several works analyzed the usage of cross-site tracking techniques in the wild [15]. Chen et al. [13] propose a data flow tracking system to measure user tracking performed through first-party cookies. They found that more than 97% of the websites they have crawled have firstparty cookies set by third-party javascript and that on 57% of them, there is at least one cookie containing a unique user identifier diffused to multiple third parties. Roesner et al. [35] measured how user tracking occurs in the wild. They found that multiple parties track most commercial pages and estimate that several trackers can each capture more than 20% of a user’s browsing behavior. Koop et al. [26] analyzed a dataset of redirection chains in the wild and found that 11% of websites redirect to the same 100 top redirectors. Moreover, they demonstrate that these top redirectors could identify users on the most visited websites. Randall et al. [33] measure the frequency of UID smuggling in the wild and find that it is performed on more than 8% of all navigations in their dataset. We use a similar method to identify user identifiers among all cookie values and query parameters by implementing automatic filtering followed by a manual inspection. All these studies were conducted in the wild, and to the best of our knowledge, no study focuses on navigational tracking techniques performed on search engines. Online tracking.

Understanding the Privacy Risks of Popular Search Engine Advertising Systems: Related Work

This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Salim Chouaki, LIX, CNRS, Inria, Ecole Polytechnique, Institut Polytechnique de Paris; (2) Oana Goga, LIX, CNRS, Inria, Ecole Polytechnique, Institut Polytechnique de Paris; (3) Hamed Haddadi, Imperial College London, Brave Software; (4) Peter Snyder, Brave Software. Table of Links Abstract and Introduction Background Measurement Methodology Results Limitations Related Work Conclusion, Acknowledgments, and References Appendix 2 BACKGROUND This section briefly discusses the policies of the main search engines alongside popular tracking approaches. 2.1 Private search engines We study the two dominant search engines that rely on user tracking for personalized search results and advertisements, namely Google and Bing, and three of the most popular privacy-branded search engines that provide users with nonpersonalized results and ads: DuckDuckGo, StartPage, and Qwant [11, 29]. Private search engines can either build their own independent search indexes or use big tech search engines like Bing, Google, or Yahoo to provide search results. Both types of private search engines claim not to store users’ search histories and not to collect nor share tracking and personal data. We now describe the advertising systems employed by the different private search engines and present a summary of their data-sharing policies outlined in their respective About pages. DuckDuckGo is a standalone search engine that maintains and uses its own search index alongside other indexes, such as Bing’s, to provide search results [31]. DuckDuckGo relies on Microsoft’s advertising system but only serves ads based on the search results and not the behavioral profiles of users [30]: "search ads on DuckDuckGo are based on the search results page you’re viewing instead of being based on you as a person" When clicking an ad on DuckDuckGo, the user is redirected to the ad’s landing page through Microsoft Advertising’s platform. DuckDuckGo claims Microsoft does not store ad-click behaviors from DuckDuckGo for purposes other than accounting and does not associate ad-clicks with users’ profiles [18]: "When you click on a Microsoft-provided ad that appears on DuckDuckGo, Microsoft Advertising does not associate your adclick behavior with a user profile. It also does not store or share that information other than for accounting purposes." This implies that Microsoft can, though currently chooses not to, link the ad-click to an existing Microsoft user profile. The privacy policy is signed by both DuckDuckGo and Microsoft. is a standalone EU-based search engine that allows users to access online resources without being tracked nor profiled [32]. Qwant relies on Microsoft’s advertising system to deliver ads in their search results pages. Although Qwant reports transmitting some information concerning search queries to Microsoft to enable the latter to present pertinent advertisements, it remains unclear which specific information is shared. In addition, to detect fraud, Qwant uses a specialized service offered by Microsoft, which has access to the user’s IP address and the browser "User-Agent". Qwant assures that this service does not have access to the search query, which is sent to another service that does not know the IP address of the user [32]. Qwant Unlike DuckDuckGo, which also uses Microsoft advertising, we did not find any mention to ad click information on Qwant’s privacy policy. They do not mention whether Microsoft stores this data and for what purposes they use it. is a meta-search engine that allows users to obtain non-personalized search results from Google’s search index while protecting their privacy. StartPage relies on Google AdSense to show ads to users. According to StartPage’s privacy policy, the search engine serves strictly non-personalized ads since it does not share any identifiable information with Google. Therefore, ads displayed on the search results page are solely based on the user’s search query [38]. StartPage Regarding ad-click behavior data, the privacy policy does not make any reference to whether Google tracks or profiles users based on this information. Nevertheless, StartPage emphasizes that by clicking on an ad, users leave the protection of StartPage’s privacy policies and become subject to the practices of the website they are redirected to [37]. "By clicking on an ad, like any other external website you click on after performing a StartPage search, you leave the privacy protection of StartPage and are subject to those websites’ data collection policies." 2.2 Cross-site tracking Cross-site tracking refers to the practice of following a user across multiple first-party websites and associate their browsing activities to a unique identifier. Web tracking practices require first party websites (e.g. the content providers) to share data about a user’s activity with third parties (the trackers). Online tracking has been traditionally implemented through browser cookies. However, due to increasing adoption of cookie-blocking browsers and extensions, and the push on adopting partitioned cookies storage on web browsers, more and more trackers started to rely on navigational tracking techniques. We next discuss how these techniques work. To enable cross-site cookie tracking, whenever a user visits a first-party website, the website makes a request to the third-party website (the tracker). This allows the tracker to set a cookie, which will identify the user and will be associated with the browsing activity of the user. For example, when the user visits a website A that makes a request to the tracker T, the tracker associates the cookie identifier of the user with the fact that the user visited website A (see Figure 1). Later, when the user visits website B, which also makes a request to the tracker T, the tracker will be able to associate the cookie identifier of the user with the fact that the user visited website B. Hence, the tracker will be able to know that the user visited websites A and B. 2.2.1 Cookie tracking. This was initially possible because browsers had a common cookie storage containing all cookies, and trackers could read their corresponding cookies regardless of which firstparty website allowed the tracker cookie to be set (see Figure 1). However, several browsers, such as Safari, Firefox, and Brave, have implemented partitioned storage to prevent using cookies for cross-site tracking [33]. These browsers use a partitioned cookies storage with a hierarchical namespace where a tracker accesses a different storage area on each website that loads it, preventing trackers from matching or assigning the same identifiers to users across multiple websites. Hence, cross-site tracking based on cookies can no longer be performed on these browsers. Chrome -the most used web browser- is in the process of testing partitioned cookies storage but does not use it by default [27, 28]. Navigational tracking refers to tracking techniques that use one or more URL navigations to share user information across sites. Navigational tracking does not require third-party cookies and can be used to circumvent browsers’ privacy protections from cross-site tracking using partitioned cookies storage. 2.2.2 Navigational tracking. is a navigational tracking technique that refers to redirecting users through one or more redirectors when navigating from one website to another. To allow this, a website A containing links to another website B does not directly link to the target B but instead links to an intermediary redirector (R)–the tracker (see Figure 2). When users click on a link on website A, they are taken to the redirector first, which then redirects them to the intended destination (website B) or other intermediary redirectors. The website A can directly change the actual link of the destination (b.com) to a redirection link (r.com), or a redirector’s third-party script can do it. On its turn, the redirector can change the destination link again and send it further to other redirectors. Bounce tracking Hence, from the link in the ad on the website A, one cannot know all the different redirectors the users will pass through when they click on an ad. We call the redirection path all the websites a user navigates through to arrive from A to B. Since, from a browser perspective, the redirector is the first-party domain, it can read or set cookies in its own partition [26]. In the following, we describe what data redirectors can infer according to the redirector’s behavior. (1) If the redirector does not set a first-party cookie, it will only know that a user went from website A to website B and will not be able to link this to other user browsing activities. (2) If the redirector sets a first-party cookie, it will be able to aggregate all the activity of the user that is redirected through it (either from website A or other websites that use it as a redirector), hence, it will allow cross-site tracking. (3) If the redirector also sets third-party cookies on websites A and B, it will not be able to link the activity of the user on website A with the activity of the user on website B, and with the activity of the user that goes through its own site (through redirects) since they do not share the same user ID [33]. Hence, while bounce tracking allows to a certain degree, cross-site tracking, it does not have the same coverage as the traditional third-party cookie tracking. is a navigational tracking technique that modifies users’ navigation requests by adding information to the navigation URLs in the form of query parameters. In addition, similar to bounce tracking, UID smuggling may redirect the user to one or more third-party trackers before UID smuggling redirecting the user to the intended destination. Figure 3 describes this process. When a user clicks on a link on a website A, the originator page itself or a tracker on the page–through a script–decorates the URL by adding the originator’s user identifier (UID) as a query parameter. The user then passes through zero or more redirectors which are invisible to him. Each of these redirectors can get the UID from the query parameter and has permission to store it in a first-party cookie under the redirector’s domain. Finally, the user is sent to the destination website B, and the redirector can forward or not to website B the UID it received from A. All the trackers on website B will be able to read the UID from the query parameter and know that it was the UID sent by the originator (through request headers). UID smuggling is more powerful than bounce tracking. Trackers using UID smuggling regain the ability to share UIDs across websites with different domains and can circumvent restrictions from partitioned cookie storage spaces [33]. For example, they can link the user’s visits to the website A with the user’s visits to website B and the user’s activity that goes through its site (through redirects) since they can all be linked to the same user ID. In addition, UID smuggling can help other trackers on website B (and website A) to link users’ browsing activity across all the websites that received the UID as a query parameter.

Understanding the Privacy Risks of Popular Search Engine Advertising Systems: Background

This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Salim Chouaki, LIX, CNRS, Inria, Ecole Polytechnique, Institut Polytechnique de Paris; (2) Oana Goga, LIX, CNRS, Inria, Ecole Polytechnique, Institut Polytechnique de Paris; (3) Hamed Haddadi, Imperial College London, Brave Software; (4) Peter Snyder, Brave Software. Table of Links Abstract and Introduction Background Measurement Methodology Results Limitations Related Work Conclusion, Acknowledgments, and References Appendix ABSTRACT We present the first extensive measurement of the privacy properties of the advertising systems used by privacy-focused search engines. We propose an automated methodology to study the impact of clicking on search ads on three popular private search engines which have advertising based business models: StartPage, Qwant, and DuckDuckGo, and we compare them to two dominant data-harvesting ones: Google and Bing. We investigate the possibility of third parties tracking users when clicking on ads by analyzing firstparty storage, redirection domain paths, and requests sent before, when, and after the clicks. Our results show that privacy-focused search engines fail to protect users’ privacy when clicking ads. Users’ requests are sent through redirectors on 4% of ad clicks on Bing, 86% of ad clicks on Qwant, and 100% of ad clicks on Google, DuckDuckGo, and StartPage. Even worse, advertising systems collude with advertisers across all search engines by passing unique IDs to advertisers in most ad clicks. These IDs allow redirectors to aggregate users’ activity on ads’ destination websites in addition to the activity they record when users are redirected through them. Overall, we observe that both privacy-focused and traditional search engines engage in privacy-harming behaviors allowing cross-site tracking, even in privacy-enhanced browsers. CCS CONCEPTS • Security and privacy → Privacy protections; Privacy protections; • Networks → Network measurement; Network measurement. KEYWORDS Search engines, advertising systems, cross-site tracking, privacy, measurement. ACM Reference Format: Salim Chouaki, Oana Goga, Hamed Haddadi, and Peter Snyder. 2023. Understanding the Privacy Risks of Popular Search Engine Advertising Systems. In Proceedings of the 2023 ACM Internet Measurement Conference (IMC ’23), October 24–26, 2023, Montreal, QC, Canada. ACM, New York, NY, USA, 15 pages. https://doi.org/10. 1145/3618257.3624823 1 INTRODUCTION Privacy-focused search engines such as DuckDuckGo, StartPage, and Qwant [3, 9, 10] promote a strategy of respecting users’ privacy and promise not to track users’ search and browsing behavior, all while delivering relevant search results. However, private search engines rely on advertising for revenue, and use traditional advertising platforms to deliver ads: DuckDuckGo and Qwant use Microsoft’s advertising system, while StartPage uses Google’s advertising system. These search engines are often ambiguous on the privacy properties of the ads that appear on their search page, and their consequent privacy properties remain unexplored to the best of our knowledge. In this work, we aim to fill this gap by conducting the first study of the privacy properties of the advertising systems of three major privacy-focused search engines: DuckDuckGo, StartPage, and Qwant, and how they compare to more popular search engines: Bing and Google. We investigate the privacy properties of these search engines when they: (i) present search ads to users, (ii) when a user clicks on an ad, and (iii) when the user lands on the advertiser’s page. We implement an automated measurement methodology to measure if and how users can be re-identified (hence, their privacy is compromised) when clicking on search ads on each search engine (see Section 3). We build an opensource implementation of this methodology in the form of a Puppeteer-based pipeline that simulates search queries and ad clicks. We apply this crawling methodology to the five search engines, providing a full dataset with visited websites, cookies created, locally stored values, and web requests to search engines’ servers and/or other third parties when clicking ads. We use filter rules from several major open-source lists to detect web requests to online trackers, and we propose a methodology to differentiate user identifiers from non-tracking values in query parameters and cookies values. We then present in Section 4 a systematic analysis of our dataset to investigate privacy harms before clicking an ad, during clicking an ad, and after clicking an ad and reaching the advertiser’s website. We find that users’ privacy is not harmed until users click on an ad. Privacy-focused search engines do not appear to attempt to re-identify users across visits or queries and do not include resources from, or make network requests to known trackers. However, we find that users’ privacy is compromised by studied search engines in various ways once users click on an ad. all Disappointingly, we find that all search engines record additional information about the user and/or the users’ clicks after the user has clicked on an ad. Private search engines capture data related to the clicked ad, including the ad provider, destination URL, and the ad’s position within the search results page, along with the user’s browsing data, such as the search query, device type, and browser language. Private search engines do not store user identifiers upon ad clicks, in contrast to traditional search engines that record user identifying values. Furthermore, we find that all search engines engage in navigation-based tracking. Navigation based tracking refers to tracking techniques that are redirecting users through one or more redirectors when navigating from one website to another in order to share user information across sites [33]. Navigation-based tracking does not require third-party cookies and can be used to circumvent browsers’ privacy protections from cross-site tracking using partitioned cookies storage. Alarmingly, we observe that privacy-focused search engines engage in more navigationbased tracking than non-privacy-focused ones: We observe navigational tracking on 4% ad clicks on Bing, on 100% ad clicks on Google, on 100% ad clicks on DuckDuckGo, on 86% ad clicks on Qwant, and on 100% ad clicks on StartPage. On the destination page, we check whether the search engine requires advertisers to abide by privacy-respecting practices by measuring whether advertisers include trackers or other known privacy-harming resources. We found that 93% of ads destination pages (across all five search engines) included tracker and privacy-harming resources. Finally, we check whether search engines or redirectors aid advertisers in profiling visitors by measuring the data they receive in the form of user-describing query params. We find that advertisers receive user identifiers in 68%, 92%, and 53% of cases for DuckDuckGo, StartPage, and Qwant, respectively. This practice, known as UID smuggling, enables redirectors to aggregate more user behavior data if they have scripts on the ads’ destination websites and they store the user-identifying parameters they receive. Notably, in the case of private search engines, the user-identifying parameters are not set by the search engine but by the redirectors encountered between the search engine’s and the advertiser’s sites. Our results indicate that private search engines’ privacy protections do not sufficiently cover their advertising systems. Although these search engines refrain from identifying and tracking users and their ad clicks, the presence of ads from Google or Microsoft subjects users to the privacyinvasive practices performed by these two advertising platforms. When users click on ads on private search engines, they are often identified and tracked either by Google, Microsoft, or other third parties, through bounce tracking and UID smuggling techniques. Particularly, advertisers receive unique user identifiers through query parameters in most ad clicks, which can enable cross-site tracking even in privacyenhanced browsers that block third-party cookie tracking.

Understanding the Privacy Risks of Popular Search Engine Advertising Systems: Abstract and Intro

In a whirlwind of activity within the cryptocurrency market, Bitcoin (BTC) has smashed through expectations, soaring to an all-time high of $71,000. This monumental milestone comes amidst a broader surge in the digital asset realm, with Ethereum (ETH) notching a notable 7% increase in its value. As established players like Bitcoin and Ethereum continue to make waves, emerging tokens like are also making headlines. FURR, in the midst of its presale phase, is on the cusp of completing its 6th stage, signaling a growing interest in its unique approach to infusing the crypto space with adorable charm. Furrever Token (FURR) Bitcoin Surge Sparks Frenzy: Analysts Divided as BTC Nears $71,000 Milestone Bitcoin (BTC) sustains its upward momentum, buoying the cryptocurrency community's expectations, particularly amidst the anticipation of significant US inflation data and the looming release of FOMC minutes. However, as BTC nears the $71,000 threshold, a cautionary note emerges from a prominent analyst warning of potential volatility. Analysts foreseeing a surge in investors' risk appetite ignited today's momentum in Bitcoin's price. Yet, despite the uptick, a cautious stance prevails among some. Warnings echo regarding significant liquidation potential should BTC breach $71,000. Leading analyst Ali Martinez highlights the risk, projecting roughly $72 million in liquidations at this juncture. This underscores market fragility, urging investor vigilance as Bitcoin approaches crucial levels. Martinez's caution contrasts with another expert's bullish outlook. Captain Faibik predicts Bitcoin's all-time high, citing a bullish pennant formation on the daily chart, suggesting surpassing the $70,000 resistance could propel BTC to unprecedented levels. This optimism diverges from Martinez's prudent approach. At present, Bitcoin's price has registered a 0.64% increase over the last 24 hours, trading at $69,743.67, with trading volume rising by 3.55% to $20.87 billion. Amidst conflicting sentiments, the cryptocurrency market remains dynamic, with investors closely monitoring Bitcoin's trajectory. Ether's Meteoric Rise: Ethereum (ETH) Surges Past $3,600 Mark Amid Foundation's Strategic Shift The onset of the new trading week witnessed a notable surge in the cryptocurrency market, with Ethereum emerging as a prominent beneficiary. Displaying robust performance, ETH experienced a commendable uptick in value, surpassing the $3,600 threshold, signaling potential bullish momentum. Amid this market exuberance, attention turned to the Ethereum Foundation, a cornerstone institution within the Ethereum ecosystem. Reports from Peckshield Alert unveiled a strategic move by the foundation, whereby a portion of its ETH holdings was converted into stablecoins, translating to a substantial sum of 354,000 DAI from 100 ETH. Further insights from Arkham Intelligence corroborated this action, highlighting the subsequent transfer of DAI to the foundation's established wallet. The rationale behind this decision remains subject to conjecture, with speculation centering on operational exigencies. This strategic maneuver potentially aligns with past instances in which Ethereum cofounder Vitalik Buterin hinted at the necessity for the foundation to address operational expenses by offloading ETH holdings. The market's response to this insider activity amid ETH's bullish ascent becomes a focal point of interest. Observers can now better closely monitor whether such measures will persist amid the ongoing uptrend of the major altcoin's price. Nevertheless, as Ethereum's value goes up, the foundation's decision to convert a portion of its ETH holdings into stablecoins has ignited a firestorm of speculation. With Ethereum's price soaring, all eyes are on the foundation's next move. Furrever Token (FURR): A Purr-fect Blend of Cuteness, Community, and Investment Potential has recently gained significant popularity in the cryptocurrency market thanks to several key factors contributing to its appeal. Furrever Token (FURR) Firstly, FURR stands out with its unique concept of infusing cuteness into the crypto space. In a market dominated by serious projects, FURR brings a refreshing and light-hearted approach centered around adorable cat imagery. This unique concept has resonated with users, attracting attention and interest. Additionally, FURR has actively engaged its community, fostering a warm and friendly environment. Through regular challenges, launches, and engagement activities, FURR involves its community members, creating a sense of belonging and shared joy. This active community engagement has contributed to FURR's growing popularity and strong community support. Furthermore, FURR offers a whimsical ecosystem filled with cute cat-themed stickers, emojis, and visuals. This adds an element of fun and charm to the user experience, making it more enjoyable and engaging for participants. The platform's focus on providing a delightful user experience has helped it stand out in the crowded crypto market. FURR's success in its presale stages, having raised over $690,000, also speaks to its popularity and investor confidence. This strong fundraising performance demonstrates the market's belief in FURR's vision and potential for growth. Moreover, FURR offers up to 15X returns, making it an attractive investment opportunity for many. With a current price of $0.00048, FURR presents investors with the potential for significant returns on their investment. Additionally, FURR prioritizes transparency and security, with the smart contract audited and safety measures implemented to ensure a secure experience for all users. Active community management by admins and moderators on platforms like Telegram further enhances trust and credibility within the FURR community. Overall, Furrever Token (FURR) has become a popular choice for investors and crypto enthusiasts due to its unique concept, active community engagement, potential for significant returns, transparency, and potential for growth in the evolving cryptocurrency landscape. Discover the Most Exclusive Presale Opportunity of 2024 Now: | | Furrever Token Official Website Visit Furrever Token Presale Join Official Telegram Group Follow Official X Account This story was distributed as a release by ZEX MEDIA under HackerNoon’s Brand As An Author Program. Learn more about the program . here

Stories, News & Mentions about FIRESTART

Firestart’s Stories on HackerNoon