This is Part 6 of the Let’s Explore ARK Core series which documents the development of the next major release of the ARK Core Codebase alongside some tips & tricks on how to get started with contributing and building your next idea today. Introduction In the , we explored how the infrastructure of Core has changed, how it affects testing and what the benefits for developers are by making these changes. Today, we'll take a look at some of the major changes in regards to tooling and how and why certain dependencies were chosen and integrated or in some cases removed and replaced. previous article ARK Core v3.0 Github Repository Replacing oclif with our own pluggable CLI When we started to move away from running Core from source via git we had to find an easy way to integrate a solution that would get us up and running quickly. There are 2 big players in the JavaScript world when it comes to building CLIs, the most popular by far is because it follows the spirit of JavaScript by not enforcing any structure on the user besides how they register their command with the CLI. Commander.js The other is , a CLI framework developed by the folks over at Heroku. It has superb TypeScript support and enforces a structure that developers have to follow to develop their commands and plugins. oclif At the time this choice was made, it was clear that going with oclif was the right way as we had migrated to TypeScript just before implementing the CLI so the strictness it enforced on our codebase was a welcomed change as it would force every developer to do things in a structured way. Our CLI integration is almost exactly 1 year old and our needs have now outgrown oclif in terms of control and extensibility. Extensibility When we initially started to build the CLI we only had a very limited set of features we required. Things like checking for updates, suggesting a command if the user has a typo in their command, and obviously the ability to quickly add new commands. oclif made all of those easy with and . The way plugins work in oclif makes it very easy to add any functionality you need on the fly and the most common scenarios are already covered by official and community plugins. oclif plugins oclif hooks Hooks allow you to intercept into the starting process of the CLI and perform things like update checks before the actual CLI starts working and executes any commands. All of those things worked great and still do but we needed more control over how the CLI could be extended and interacted with to allow other developers to implement new commands for our CLI through their own plugins. To solve this problem, we set out to build our own small CLI framework which allows us to provide tight integration into Core to provide the best experience we can whilst giving developers full control over how the CLI works. { Commands, Container } ; { Networks } ; Joi ; { parseFileSync } ; { existsSync } ; .injectable() Command Commands.Command { signature: = ; description: = ; configure(): { .definition .setFlag( , , Joi.string().default( )) .setFlag( , , Joi.string().valid(...Object.keys(Networks))) .setFlag( , , Joi.string()); } execute(): < > { envFile: = .app.getCorePath( , ); (!existsSync(envFile)) { .components.fatal( ); } env: object = parseFileSync(envFile); key: = .getFlag( ); (!env[key]) { .components.fatal( ); } .components.log(env[key]); } } import from "@arkecosystem/core-cli" import from "@arkecosystem/crypto" import from "@hapi/joi" import from "envfile" import from "fs-extra" /** * @export * @class Command * @extends {Commands.Command} */ @Container export class extends /** * The console command signature. * * @type {string} * @memberof Command */ public string "env:get" /** * The console command description. * * @type {string} * @memberof Command */ public string "Get the value of an environment variable." /** * Configure the console command. * * @returns {void} * @memberof Command */ public void this "token" "The name of the token." "ark" "network" "The name of the network." "key" "The name of the environment variable that you wish to get the value of." /** * Execute the console command. * * @returns {Promise<void>} * @memberof Command */ public async Promise void const string this "config" ".env" if this `No environment file found at .` ${envFile} const const string this "key" if this `The " " doesn't exist.` ${key} this With the ability to create plugins for both Core and the CLI, we hope to see plugins that leverage both which could, for example, be that the transaction pool plugin exposes the transaction pool for Core to use and also some CLI plugins to interact with it, such as flushing all stored transactions. Testability Testing commands has been a major pain point with oclif. It does provide some tooling to ease testing as can be seen at but they are targeted specifically for MochaJS. We use and don't intend to move away from it anytime soon so all of our toolings should play nicely with it. If it doesn't, we'll need to resolve that problem. oclif testing Jest The result of those problems is that the new CLI has been built from the ground up with TDD. Applying TDD to this problem was an obvious choice because it would reveal any testing problems and unnecessary complexity while building out the tooling. This has greatly helped to reveal some older bugs in the existing CLI and to make testing of commands and plugins a lot easier. The result of this process is a new package called which is responsible for all tooling that revolves around building a CLI. Accompanying to this package there are a few helpers that have been added to core-test-framework which make it a breeze to manipulate and execute commands in our test suite. core-cli { Console } ; { Command } ; { ensureDirSync, ensureFileSync, writeFileSync } ; { dirSync, setGracefulCleanup } ; cli; beforeEach( { process.env.CORE_PATH_CONFIG = dirSync().name; cli = Console(); }); afterAll( setGracefulCleanup()); describe( , { it( , () => { writeFileSync( , ); message: ; jest.spyOn( , ).mockImplementationOnce( (message = m)); cli.withFlags({ key: }).execute(Command); expect(message).toBe( ); }); it( , () => { ensureFileSync( ); expect(cli.withFlags({ key: }).execute(Command)).rejects.toThrow( , ); }); it( , () => { ensureDirSync( ); expect(cli.withFlags({ key: }).execute(Command)).rejects.toThrow( , ); }); }); import from "@arkecosystem/core-test-framework" import from "@packages/core/src/commands/env-get" import from "fs-extra" import from "tmp" let => () new => () "GetCommand" => () "should get the value of an environment variable" async ` /.env` ${process.env.CORE_PATH_CONFIG} "CORE_LOG_LEVEL=emergency" let string console "log" => m await "CORE_LOG_LEVEL" "emergency" "should fail to get the value of a non-existent environment variable" async ` /.env` ${process.env.CORE_PATH_CONFIG} await "FAKE_KEY" 'The "FAKE_KEY" doesn\'t exist.' "should fail if the environment configuration doesn't exist" async ` /jestnet` ${process.env.CORE_PATH_CONFIG} await "FAKE_KEY" `No environment file found at /.env` ${process.env.CORE_PATH_CONFIG} As you can see, testing anything related to the CLI is now a breeze and isn't coupled to any specific testing tools or framework. This will hopefully make it easier for us and other developers to ensure that the CLI is working as intended and not performing any unwanted actions on a user's system. Replacing pg-promise with TypeORM When Core 2.0 was initially developed, we were using JavaScript so was an obvious choice for our database layer as it was well maintained and the most popular package for PostgreSQL integrations into node.js applications. The other choice was but was discarded later on as it had major performance issues and a lot of known bugs and unresolved issues. PG Promise Sequelize The problem with it is that it locks us into a single database engine without the ability to swap it out for something like SQLite while running tests or on devices with lower hardware specifications. After the move to TypeScript in early 2019, we started to look at TypeScript alternatives to enforce more strictness on the structure and to provide a cleaner interface for developers that want to interact with the database. The result of that search is . As the name suggests is an ORM that supports a wide variety of platforms and database engines which made it a perfect fit for us to achieve our goals of having a database-agnostic backend. The integration of TypeORM was fairly straightforward and the result is an integration that provides developers with more access to the internals that are responsible for the database. TypeORM With TypeORM, we will be able to support more platforms for development that have fewer resources available and continue to remove dependencies that are required to run Core. Overall this will help us to lower the barrier of entry and make development easier by removing the need for Docker. Replacing various third-party dependencies Core 2.0 heavily relied on third-party packages even for the most basic tasks, simply for convenience. Lodash and dozens and dozens of other small packages are littered all over the codebase and there just is no chance to know what all of those dependencies do and a lot of them will have poor performance when used with larger datasets. This is where comes in. This package provides performance-oriented implementations of commonly used functions in TypeScript. The aim of this package is to free us from the plethora of third-party dependencies that we use in all our projects and replace them with alternative implementations with a focus on performance. ARK’s Utils Lodash has been completely removed from Core, smaller packages have been replaced by native functions or smaller alternative implementations and many more functions are being added. This ensures that all of our developers know what our dependencies are doing and that performance adjustments can be made as needed without having to rely on others or waiting for a PR to be merged. Relying less on third-party dependencies also means that we can tailor everything specifically to our use-cases and don't need to worry about what people might want to do and thus ending up with a very generic solution that has performance issues that will bite us sooner or later. What’s next? This concludes Part 6 of the Let’s Explore Core series. This is the last part of this series and we hoped you enjoyed and learned something new. I want to help with development That's great news! If you want to help out, are wide open, but that is not all, we also have special Monthly Development GitHub Bounties on-going where you can earn money for every valid and merged Pull-Request. our GitHub repositories To learn more about the program please read our blog post and to learn more about ARK Core and blockchain visit our . Bounty Program Guidelines Learning Hub
![featured image - Let's Explore ARK Core v3: Tooling [Part 6]](https://hackernoon.imgix.net/images/o1f6329g.jpg?auto=format&fit=max&w=3840)
![Let's Explore ARK Core v3: Maintainability & Testability [Part 5]](https://hackernoon.imgix.net/drafts/23ic32j0.png?auto=format&fit=max&w=3840)
