paint-brush
Property-based testing (with a sprinkle of JavaScript)by@riccardoodone
3,112 reads
3,112 reads

Property-based testing (with a sprinkle of JavaScript)

by Riccardo Odone
Riccardo Odone HackerNoon profile picture

Riccardo Odone

@riccardoodone

Software Maverick with no managers

January 7th, 2018
Read on Terminal Reader
Read this story in a terminal
Print this story
Read this story w/o Javascript
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

If you come from OO and have ever tested any piece of software chances are you’ve employed example-based testing. In other words, the function under test is called with some predefined input and the return value is checked against some predefined expectation. For instance, the function <code class="markup--code markup--p-code">sum</code> is called with <code class="markup--code markup--p-code">1</code> and <code class="markup--code markup--p-code">2</code> and the expected result is <code class="markup--code markup--p-code">3</code>.

People Mentioned

Mention Thumbnail

John Hughes

@johnhughes0303

Company Mentioned

Mention Thumbnail
YouTube
featured image - Property-based testing (with a sprinkle of JavaScript)
Riccardo Odone HackerNoon profile picture
Riccardo Odone

Riccardo Odone

@riccardoodone

Software Maverick with no managers

image

If you come from OO and have ever tested any piece of software chances are you’ve employed example-based testing. In other words, the function under test is called with some predefined input and the return value is checked against some predefined expectation. For instance, the function sum is called with 1 and 2 and the expected result is 3.

There are problems with this approach though. Firstly, only a limited amount of examples can be checked. Secondly, it’s a human being deciding what input-output to verify. This, unfortunately, reflects the developer’s bias. Therefore there’s a big chance some critical input is left unchecked.

But how is it possible to have expectations on the return value of a function if the input is not chosen by the developer?

Turns out, it’s possible to derive invariants (properties) out of the code that are always true, no matter what the input is. If you mix that with random values generators you get property-based testing.

In the case of sum, we would use a number generator to produce the input. Then, we would derive some properties to test such as commutativity (i.e. x + y == y + x) and associativity (i.e. (x + y) + z == x + (y + z)). Lastly, the property-based testing framework would generate some sets of x, y and z and check multiple times if the properties hold.

In presence of a failure, we would get back the first set of x, y and z for which the property didn’t hold.

The sum one is a simple example. But we could imagine something more complex that involved other function calls and less trivial input. In that case, just getting the input set that failed could be not enough.

That’s why most property-based testing frameworks provide a feature called shrinking. In other words, after a failure, the framework tries to shrink the input that made the property fail to its simplest form by removing or simplifying input data.

Show me the code

Let’s apply property-based testing to one of those stupid examples you are never going to see in real life. I’m going to use JavaScript and JSVerify.

Let’s say we have the following code which needs tests:

const div = (dividend, divisor) => dividend / divisor

Easy, right? The following example-based tests are green so we can call it a day!



describe('div', () => {it('with natural numbers', () => {const expected = 2

const actual = div(6, 3)

assert.strictEqual(actual, expected)  

})


it('with decimal numbers', () => {const expected = 2

const actual = div(6.3, 3.15)

assert.strictEqual(actual, expected)  


})})

Well, not really. Let’s see what happens with property-based tests.

Firstly, as the generator we can use jsc.nat which returns natural numbers.

Secondly, as the properties let’s just check the “right-distributive” one (i.e. (n1 + n2) / n3 == (n1 / n3) + (n2 / n3)).


describe('div', () => {const naturalNumber = jsc.nat






jsc.property('is right-distributive',naturalNumber, naturalNumber, naturalNumber,(n1, n2, n3) => div(n1 + n2, n3) === div(n1, n3) + div(n2, n3))})

BOOM!

Error: Failed after 4 tests and 4 shrinks. rngState: 08b51479f83d6a20ec; Counterexample: 0; 0; 0;

With n1 = 0, n2 = 0 and n3 = 0 something went wrong. From the node REPL


div(0 + 0, 0) === div(0, 0) + div(0, 0)// false


div(0 + 0, 0)// NaN


div(0, 0) + div(0, 0)// NaN


NaN === NaN// false

JavaScript, right? If we run the property-based test again

Error: Failed after 4 tests and 4 shrinks. rngState: 08b51479f83d6a20ec; Counterexample: 2; 32; 3;

BOOM AGAIN. But this time it’s a different failure. From the node REPL


div(2 + 32, 3) === div(2, 3) + div(32, 3)// false


div(2 + 32, 3)// 11.333333333333334


div(2, 3) + div(32, 3)// 11.333333333333332

JavaScript and floating point arithmetic, right?

Now, if property-based testing found out 2 bugs in 2 runs out of

const div = (dividend, divisor) => dividend / divisor

imagine what else it can find out of more complex code.

Outro

Property-based testing takes away bias. That way it enables discovering bugs the developer didn’t think to test for.

Also, it forces to consider the code from yet another point of view, which is a good thing.

image

At the same time, properties are somewhat more abstract than examples (e.g. commutativity in sum vs sum(1, 2) == 3). That’s why mixing the two styles is prolly the best idea.

Hungry for more? Check out I use property-based TDD in the follow up post.

Pointers

L O A D I N G
. . . comments & more!

About Author

Riccardo Odone HackerNoon profile picture
Riccardo Odone@riccardoodone
Software Maverick with no managers

TOPICS

THIS ARTICLE WAS FEATURED IN...

Permanent on Arweave
Read on Terminal Reader
Read this story in a terminal
 Terminal
Read this story w/o Javascript
Read this story w/o Javascript
 Lite
Cto
Hypothes

Mentioned in this story

companies
profiles