Vue Amsterdam 2022 - Part VI: It’s a (Testing) Trap!

Welcome! Happy to see you in the sixth part of my Vuejs Amsterdam Conference 2022 summary series, in which I share a summary of all the talks with you.

You can read my JSWorld Conference 2022 Summary series (in four parts) here, where I summarized all the first day’s talks. You can also find the previous Talks of the Vue Amsterdam conference 2022 in my blog.

(Recurring) Introduction

After two and a half years, JSWorld and Vue Amsterdam Conference were back in Theater Amsterdam between 1 and 3 June, and I had the chance to attend this conference for the first time. I learned many things, met many wonderful people, spoke with great developers, and had a great time. On the first day the JSWorld Conference was held, and on the second and third days, the Vue Amsterdam.

The conference was full of information with great speakers, each of whom taught me something valuable. They all wanted to share their knowledge and information with other developers. So I thought it would be great if I could continue to share it and help others use it.

At first, I tried to share a few notes or slides, but I felt it was not good enough, at least not as good as what the speaker shared with me. So I decided to re-watch each speech, dive deeper into them, search, take notes and combine them with their slides and even their exact words in their speech and then share it with you so that what I share with you is at least at the same level as what I learned from them.

A Very Important Point

Everything you read during these few articles is the result of the effort and time of the speaker itself, and I have only tried to learn them so that I can turn them into these articles. Even many of the sentences written in these articles are exactly what they said or what they wrote in Slides. This means if you learn something new, it is because of their efforts. (So if you see some misinformation blame them, not me, right? xD)

Last but not least, I may not dig into every technical detail or live codings in some of the speeches. But if you are interested and need more information, let me know and I’ll try to write a more detailed article separately. Also, don’t forget to check out their Twitter/Linkedin.

Here you can find the program of the conference.

It’s a (Testing) Trap! Common Testing Pitfalls and How to Solve Them

Ramona Schwering - Software developer at shopware

“It’s a trap” - a call or feeling we all might be familiar with, not only when it comes to Star Wars. It’s signalizing a sudden moment of noticing imminent danger.

This situation is an excellent allegory for an unpleasant realization in testing. Imagine having the best intentions when it comes to testing, but still ending up with tests failing to deliver you any value at all?

Tests who are feeling like a pain to deal with?

When writing frontend tests, there are lots of pitfalls on the way. In sum, they can lead to lousy maintainability, slow execution time, and - in the worst-case - tests you cannot trust.

But the worst pain point is those tests that give you new value and result because they are flaky. You have a build that sometimes passes and sometimes fails and you did nothing in between.

Pain Points

You might be sure that testing has many perks and advantages, but all of those wonderful things can be out shadowed by pain points caused by various reasons. All the things you did with the best intentions but in the long run or even earlier they turned out painful or exhausting.

For example, slow tests can kill the fun. Imagine you have your pull request and you want it to be merged, but you need to wait hours or maybe days for the pipeline to finally be completed.

Even worse are tests that are painful to maintain. You think about your past self and you ask: What did you do with this test? What was the purpose?! What did you think about that one? Or other team members who are asking you questions about what you did in the past and you have no clue.

But the worst pain point is those tests that give you new value and result because they are flaky. You have a build that sometimes passes and sometimes fails and you did nothing in between.

Simple Tests

It doesn’t have to be this way. One of the most important mindsets and golden rules in JavaScript testing best practices is Simple.

Tests should be designed plain and simple in every case. In unit tests, integration tests, and End to End tests, keep it stupid simple.

Our goal should be that one is capable to understand your test and get its intent instantly without thinking about it.

Try to aim for a flat test design, which means only testing as much as needed and using few to no abstractions at all.

Traps

Let’s look at the first Trap.

describe('deprecated.plugin', () => {
	it('should throw error',() => {
		…
	});
});

It should throw an error if the deprecated plugin is in use.

When you look at the title — should throw an error — you don’t know what it wants to accomplish. But the Golden rule says you should know instantly what it’s doing.

We can make it more comprehensible with the “Three-Part Rule”. The test title should contain three things:

1. What is being tested
2. Under what circumstances should be tested
3. What is the expected result

With this rule in mind, this is what our test will look like:

describe('deprecated.plugin', () => {
	it('Property: Should throw an errorif the deprecated prop is used',
	() => {
		…
	});
});

The next Trap can be the test structure:

describe('Context menu', () => {
	it('should open the context menu on click', async () => {
		const wrapper = createWrapper();
		expect(wrapper.vm).toBeTruthy();
		await wrapper.trigger('click');
		const selector = '.sw-context-menu';
		expect(wrapper.find(selector).isVisible()).toBeTruthy();
	});
});

In this case, declarations, actions, and assertions are written all over the place without any clear structure. Especially in bigger tests, this can be a huge pain. We can make it more structured with AAA Pattern. You divide your test into three parts:

1. Arrange: Everything concerning the setup to the scenario the test aims to simulate.
2. Actions: Running your unit on the test and doing the steps to get to the results state.
3. Assert: Where you can do the assertions and check your test scenario.

This is what our test looks like with AAA Pattern:

describe('Context menu', () => {
	it('should open the context menu on click', () => {
		// Arrange
		const wrapper = createWrapper();
		const selector = '.sw-context-menu';
		// Act
		await wrapper.trigger('click');
		// Assert
		expect(wrapper.vm).toBeTruthy();
		expect(wrapper.find(selector).isVisible()).toBeTruthy();
	});
});

This looks much better!

But there is a problem. Think back to our test case. It is a context menu and it is hidden by default. So we need to open it to interact with it. But it means we need to do an assertion to make sure that it’s open before the act. Doesn’t it destroy the pattern?

If we are testing the DOM, we sometimes need to check the before and after states. So Let’s take a look at this pattern from another perspective.

• …arrange my test == what I’m given.
• …act in my test == when something happens.
• …assert the results == something happens then this is what I expect as the outcome.

This is a pattern taken from behaviour driven development: Given - When - Then

Our prior test with the usage of this pattern:

describe('Context menu', () => {
	it('should open the context menu on click', () => {
		// Given
		const contextButtonSelector = 'sw-context-button';
		const contextMenuSelector = '.sw-context-menu';
		// When
		let contextMenu = wrapper.find(contextMenuSelector);
		expect(contextMenu.isVisible()).toBe(false);
		const contextButton = wrapper.find(contextButtonSelector);
		await contextButton.trigger('click');
		// Then
		contextMenu = wrapper.find(contextMenuSelector);
		expect(contextMenu.isVisible()).toBe(true);
	});
});

E2E Testing

Avoid using placeholders and use realistic naming as much as possible.

it('should create and read product', () => {
	…
	cy.get('.sw-field—product-name').type('T-Shirt Ackbar');
	cy.get('.sw-select-product__select_manufacturer')
	.type('Space Company’);
	…
});

If you don’t want to come up with thousands of names, maybe you can use faker or other tools to generate some dummy data, or if it’s okay with the laws and your project, import it from the production state. Just make sure that your test stays comprehensible and easy to read and that you know what it does.

Let’s take a look at the same test for the next trap, which is selectors.

If you refactor your application and change some classes — which is common — this could cause your test to fail, and in this case, the test failing without a bug present in your app — which is called a false negative giving no reliable report of your app because it’s not broken, just your implementation changed.

Look at selectors you must! In other words, you shouldn’t test implementation details. Instead, think about using things a user would call attention to and navigating your application. For example some text inside your page. Even better, use selectors which are not that prone to change, like data attributes. You can even call it for example for cypress data-cy=”submit” so one instantly know that it is meant for testing.

Last but not least, don’t use fixed waiting times.

Cypress.Commands.add('typeSingleSelect', {
		prevSubject: 'element',
	},
	(subject, value, selector) => {
		…
		cy.wait(500);
		cy.get(`${selector} input`)
		.type(value);
});

On one hand, that can slow down your test way too much if a test case will be executed so many times. It can be even worse when your application is being used on weaker hardware. In this case, you may need more than that fix 500ms.

There are some solutions for that, for example waiting for the changes in the UI, like a for loading spinner to disappear or an animation to stop or something to appear.

Or even better, wait for API Requests and Responses.

Tests are there as an assistant to you, not a hindrance. They should feel like a routine not like solving a complex mathematical formula.

End of The Sixth Talk

I hope you enjoyed this part and it can be as valuable to you as it was to me. Over the next few days, I’ll share the rest of the talks with you. Stay tuned…

Also published here.