We Built an IDE to work with API Design. Here's What We Did

Hello there! Today is the day when we’re ready to present to the IT community TestMace, our IDE for API design. Some of you may know about it from the previous article. However, we haven’t yet provided you with an overall review of our tool, so here you go.

Motivation

We’d like to start by telling you the story on how we ended up with this feeling of creating our own advanced API tool at the first place. Here’s the list of features we suppose every IDE for API design should have:

• creating and sending requests and scenarios (request chains);
• writing different kinds of tests;
• test generation;
• working with API description, including importing fromSwagger, OpenAPI, WADL, etc.;
• mocking requests;
• great support of one or several scripting languages and integration with the most popular libraries;
• and so on.

Follow your preferences to complete the list. Apart from IDE, it is also quite important to create a specific infrastructure like cloud synchronization, CLI tools, online monitoring service, etc. After all, the latest trends are about not only powerful features, but also a nice interface.

Who will benefit from it? Apparently, those who are in any way connected with API development and testing, developers and testers in particular. While developers are often limited to sending single requests and running simple scenarios, in their main tool testers are going to need something more advanced, including powerful features for writing tests and ability to run them in CI.

Having all that in mind, we started designing our product. Let’s look at what we have achieved so far.

Quick Start

Let’s start with simply looking at our app. You can download it from our site. For now, all 3 main platforms are supported – Windows, Linux, and MacOS. Download the app, install it and run. Running it the first time, you’ll see this window:

Click on the + button in the top of content area to create your first request. The request tab should look like this:

Let’s take a closer look at it. The request interface resembles popular rest clients interface, which makes migration from that kind of tools easier. Let’s send our first request to https://next.json-generator.com/api/json/get/NJv-NT-U8

Apparently, the response panel has nothing to surprise us with. But still you have to know, that:

• The response body represents a tree, that makes it more informative and allows to add some curious features described down below.
• There is the Assertions tab, that has a list of tests for this particular request.

As you can see, our tool can be used as a well-organized rest client. But sending requests is not what we’re here for. Now let’s talk about the main features and concepts of TestMace.

TestMace’s Main Features and Concepts

Node

TestMace’s workflow is distributed into different node types. The previous example illustrates how RequestStep node works. Some other node types are now available in the app:

• RequestStep. This is a node that you use to create a request. It can have only one Assertion node as a child element.
• Assertion. This one is used for writing tests. It can be a child node only for a RequestStep node.
• Folder. It allows to group Folder and RequestStep nodes under it.
• Project. This is a root node, that is created automatically when you start a new project. It is functionally identical to a Folder node.
• Link. This is a link to Folder or RequestStep nodes. It allows to reuse requests and scenarios.

You can find nodes in scratches (a bottom left panel for quick creating one-time requests) and in project (a top left panel), that I’m going to show you next.

Project

After running the app, you can see a Project line at the top left of the screen. That is the project tree root. When you run your project, a temporary project is created, and the path to it depends on your operating system. You can change it whenever you want.

The main purpose of the project is to give you an ability to save your drafts in the file system, synchronize it via the version control system, run scenarios in CI, review changes, etc.

Variables

Variables is one of the core mechanisms of our app. Those of you who work with TestMace-like tools might already have it in mind. Variables is a way of saving shared data and communication between nodes. Environment variables in Postman and Insomnia are a great example of this. But we have gone beyond all that. In TestMace you can define your variables at the node level. Any node. There is also a mechanism of inheritance variables from parents and overriding variables in child elements. In addition, there are several default variables, which names start with 

$

• $prevStep

   – a reference to the previous node variables;

• $nextStep

   – a reference to the next node variables;

• $parent

   – a reference to the parent node variables;

• $response

   – a server response;

• $env

   – current environment variables;

• $dynamicVar

   – dynamic variables, that are being created during scenario or request run.

• $env

   are basically just the Project node variables, but the variables set is different for every environment.

You can access a variable using 

${variable_name}

The value of that variable might contain another variable or even an expression. For instance, an

url

http://${host}:${port}/${endpoint}

It should be mentioned, that you can assign variables while running scripts. For example, quite often it is important to save authorization data (a token or full header) that come from the server after a successful login. TestMace allows you to store that data in parent’s dynamic variables. To avoid collision with existing static variables, dynamic variables have been placed in a separate object – 

$dynamicVar

Scenarios

Taking advantage of the features stated above, you can run request scenarios. For example, create an entity -> request an entity -> delete an entity. In this case, you can use a Folder node to group several RequestStep nodes.

Autocomplete Feature and Expression Value Highlighting

To make working with variables more convenient, we added an autocomplete feature and variables’ values highlighting, that shows you what value each variable has. A picture paints a thousand words here:

Note that not only variables, but also such things as headers, some headers’ values (like Cotent-Type), protocols are autocompleted. With development of our product, the list is regularly updated.

Undo/Redo

Ability to undo/redo changes is a practically useful feature, yet not every tool comes with it (including API tools). We’re here to change it!  Undo/redo works within the whole project, which allows you to undo not only changes to a particular node, but also its creating, deleting, moving and so on. The most critical operations need to be confirmed.

Creating Tests

Assertion node takes care of creating tests. One of the major features here is creating tests without programming using build-in editors.

An assertion node is a number of assertions, each of which has its own type. There are several types:

1. Compare values – it simply compares two values. We have several comparison operators:

   equal to

   ,

   not equal to

   ,

   greater than

   ,

   greater than or equal to

   ,

   less than

   ,

   less than or equal to

   .
2. Contains value – checks if a substring is present in a string.
3. XPath – checks if there is a certain value in XML via selectors.
4. JavaScript assertion – a random JavaScript script that returns true on success and false on failure.

Note that only the latter requires some programming skills, the other three are created with the help of graphical user interface. Here is an example of creating compare values assertion dialog:

Quick assertions creation right from the response would be our icing on the cake. Just look at this!

However, these assertions clearly have certain limitations, so you can use javascript assertions. Here TestMace provides you with a nice environment with autocomplete feature, syntax highlighting and even static analyzer.

API Description

TestMace allows not only to use API, but also to write documentation. The description itself has a hierarchical structure and fits the project naturally. In addition to that, you can import API descriptions from Swagger 2.0 / OpenAPI 3.0 formats. The description is not left aside, but fully integrates with the project: autocompletion of urls, HTTP headers, query parameters is available, and we’re planning to add tests to check if a response matches API description.

Nodes Sharing

Let’s say you need to share a bugged request or scenario with a colleague or just attach it to the bug. TestMace’s got it: it allows you to serialize any node or a subtree in url. Just copy-paste and you have your request shared with another machine or project.

Human-Readable Format for Project Storing

Each node is stored either in a separate .yml file (like Assertion node is) or in a folder with the node’s name containing index.yml file.

That is how the file with the request from the previous example looks like:

children:  []
variables: {}
type: RequestStep
assignVariables:  []
requestData:
  request:
    method: GET
    url: 'https://next.json-generator.com/api/json/get/NJv-NT-U8'
  headers:  []
disabledInheritedHeaders:  []
  params:  []
  body:
    type: Json
jsonBody: ''
xmlBody: ''
textBody: ''
formData:  []
    file: ''
formURLEncoded:  []
strictSSL: Inherit
authData:
  type: inherit
name: Scratch 1

As you can see, it’s totally readable. You can easily edit it manually if you need.

The directory hierarchy is the same as the nodes hierarchy. For example, this scenario:

Is mapping in the file system to the following structure (only directory hierarchy is shown, but it should be clear):

That simplifies the project review process.

Import from Postman

Having read all this, some of you may be wanting to try out our new product (aren’t you?) or even use it in your project (why the hell not). Well, you must be hesitating since you’ve got all your work done in Postman. In this case, TestMace supports importing collections from Postman. It is without tests for now, but we may include it in our future releases.

Our Plans

We’re hoping after reading all this many of you have found our product interesting. And it’s just a beginning! At the moment we’re fully involved in developing and I’m going to tell you about what we’re going to do next.

Cloud Synchronization

That’s one of the most desirable features. Now we suggest you use the version control system for it, working on making everything in our app compatible with it. However, some of you may not be particularly happy with this workflow, so we’re planning to add cloud synchronization mechanism that you used to.

CLI

As I already mentioned, it’s hard to imagine any IDE product without integration with existing projects or workflow. CLI is essential for integrating tests written in TestMace into continuous integration process. We’re currently working on it, but in our early versions projects will run with a simple console report. In the future you’ll see reports in JUnit format.

Plugin System

Despite all functionality, our product can’t cover all possible use cases. Each project is unique, and there may be too many specific tasks. That is why we’ve been thinking to add SDK for plugins development, so that each individual user could meet their own needs.

New Node Types

Some use cases may require more node types, than we have now. We’re planning to add:

• Script Node, which transforms and stores data using js and the corresponding API. This node type might be useful for writing something like pre-request and post-request scripts in Postman;
• GraphQL Node for graphql support;
• Custom Assertion Node to expand the set of existing assertions in the project;

The list is definitely not complete yet, and your feedback would help us update it.

FAQ

How is your product different from Postman?

• The node concept allows to scale your product functionality with no limits.
• The human-readable format of the project and storing it in the file system makes it easier to work with the version control system.
• Creating tests without programming and advanced js support in the test editor (autocomplete feature, static analyzer).
• More sophisticated autocomplete feature and highlighting of current variables’ values.

Is this an open-source product?

No, source files are not available, but in the future we’re considering to add this option.

What are you living on? :)

Along with the free version, there will soon be a paid one. It’ll first of all include the features, like synchronization that require the server side.

Conclusion

We’re working hard to finally release a stable version of TestMace. We’ve received a positive feedback from our first users, so you can already safely try out our product. We would really appreciate your feedback as well, since it is impossible to create a great tool without close cooperation with the community. Contact us:

• Official site: https://client.testmace.com
• Telegram: https://t.me/testmace
• Slack: https://testmaceslackin.herokuapp.com
• Facebook: https://www.facebook.com/testmace
• Issues-tracker: https://github.com/TestMace/TestMace-issues

We’re looking forward to your comments!