Why you should check the performance of your code?

Just like most of you, I usually don't care the performance of my codes. Until one day, I saw the comparison between my code and my teammates', I changed my mind since.

Comparison:
     Yunus's method :   119030.6 i/s
     Ebuka's method :    63232.0 i/s - 1.88x  slower
    Kelvin's method :     6848.0 i/s - 17.38x  slower

But beside competing with your peers and having fun, is there any other reasons for you to check the performance of your code?

Yes, Of course, there are few, please following through.

First at all, it is fun to play with.

As we all know, learn to code, especially learning algorithm is time consuming, and boring. Because in most of the time, we can't apply those knowledges instantly, we just learn the concept and move on.

When I was writing a sorting algorithm - "QuickSort" for a coding challenge, I have the same feeling. I only care about whether it can sort the array, I know nothing about the algorithm beside its concept.

But then one of my teammates introduce me Benchmark Testing tool "Benchmark-ips", claim somehow his code is run faster. I didn't believe it, so I run it the test to compare with the sorting method build-in Ruby. Here is the result -

"Benchmark test with [1, 4, 6, 9, 3]"
Warming up --------------------------------------
           Ruby Sort    86.628k i/100ms
          Quick Sort     9.179k i/100ms
Calculating -------------------------------------
           Ruby Sort      1.353M (± 5.8%) i/s -      4.072M in   3.018663s
          Quick Sort     95.023k (±14.1%) i/s -    284.549k in   3.083889s

Comparison:
           Ruby Sort:  1353445.0 i/s
          Quick Sort:    95023.3 i/s - 14.24x  slower

Ok, now I know my QuickSort is about 14 times slower than Ruby's sort, what is the big deal? I just learn ruby, and QuickSort isn't the best sorting method out there.

Then another day, I have learned a way to improve QuickSort 's performance, let's call it "Advance QuickSort", after I finish writing the code, I can't wait to check how much performance it gain.

"Benchmark test with [1, 4, 6, 9, 3]"
Warming up --------------------------------------
           Ruby Sort    77.027k i/100ms
          Quick Sort     9.312k i/100ms
      Adv Quick Sort    14.244k i/100ms
Calculating -------------------------------------
           Ruby Sort      1.325M (± 7.4%) i/s -      4.005M in   3.039960s
          Quick Sort    103.551k (± 4.7%) i/s -    316.608k in   3.064478s
      Adv Quick Sort    166.568k (± 6.0%) i/s -    512.784k in   3.089796s

Comparison:
           Ruby Sort:  1325148.8 i/s
      Adv Quick Sort:   166568.5 i/s - 7.96x  slower
          Quick Sort:   103551.0 i/s - 12.80x  slower

Holy moly, my Advance QuickSort gain about 60% performance!

I was hooked, because it was a quite useful feedback I can get from my codes. I wouldn't get it otherwise.

It might seem nothing for others, but it was significant for me. I can't stop to think about how to make my code run faster since.

You can check them out from here (QuickSort) and here (Advance QuickSort).

You may argue, that just my experience, you might not find the tests is as fun as I claim. But I will like to respond, don't you like feedback, don't you like to play around with your code and improve it as much as you can?

If "Yes", then you might want to do some tests occasionally, compare it with your own code, or your friend's code, I am sure you will find some funs out of it.

If "No", then you might want to continue the reading below, there are more I can tell.

Second, it will help you learn.

Learn to code is a long process, having some funs along the way certainly help, but that is not enough. Sometime you need to dive deeper to understand the mechanism behind the scene.

Traditionally, you will need to spend more time to study the theories, read more articles and do more practices to master them. But sometime this way is slow. Because it is hard to find a good resources to learn from, or there are just too little you can't find any.

Doing some tests by yourself first might be a better approach. Here are some examples.

The next day, I compared my "QuickSort" algorithm with my another sorting algorithm - "Insertion Sort".

"Benchmark test with [1, 4, 6, 9, 3]"
Warming up --------------------------------------
           Ruby Sort    86.270k i/100ms
      Insertion Sort    19.597k i/100ms
          Quick Sort     8.369k i/100ms
      Adv Quick Sort    13.374k i/100ms
Calculating -------------------------------------
           Ruby Sort      1.292M (± 7.3%) i/s -      3.882M in   3.020565s
      Insertion Sort    232.637k (± 7.4%) i/s -    705.492k in   3.049315s
          Quick Sort     93.083k (± 6.6%) i/s -    284.546k in   3.070232s
      Adv Quick Sort    155.587k (± 7.3%) i/s -    468.090k in   3.025360s

Comparison:
           Ruby Sort:  1292262.4 i/s
      Insertion Sort:   232637.3 i/s - 5.55x  slower
      Adv Quick Sort:   155586.9 i/s - 8.31x  slower
          Quick Sort:    93083.4 i/s - 13.88x  slower

It seems Insertion Sort work better ... really? Not quite, keep watching.

"Benchmark test with [9, 8, 6, 7, 3, 5, 4, 1, 2]"
Warming up --------------------------------------
           Ruby Sort    68.250k i/100ms
      Insertion Sort     6.703k i/100ms
          Quick Sort     4.120k i/100ms
      Adv Quick Sort     7.987k i/100ms
Calculating -------------------------------------
           Ruby Sort    958.828k (± 7.2%) i/s -      2.935M in   3.076884s
      Insertion Sort     74.428k (± 5.7%) i/s -    227.902k in   3.071941s
          Quick Sort     43.658k (± 7.4%) i/s -    131.840k in   3.037402s
      Adv Quick Sort     90.081k (± 5.3%) i/s -    271.558k in   3.023168s

Comparison:
           Ruby Sort:   958827.5 i/s
      Adv Quick Sort:    90080.9 i/s - 10.64x  slower
      Insertion Sort:    74428.5 i/s - 12.88x  slower
          Quick Sort:    43657.8 i/s - 21.96x  slower

So when test case size increase, Insertion Sort performance start to degrade (compare Quick Sort ), let's try it with an array with size of 100.

"Benchmark test with [406, 157, 415, 318, 472, 46, 252, 187, 364, 481, 450, 90, 390, 35, 452, 74, 196, 312, 142, 160, 143, 220, 483, 392, 443, 488, 79, 234, 68, 150, 356, 496, 69, 88, 177, 12, 288, 120, 222, 270, 441, 422, 103, 321, 65, 316, 448, 331, 117, 183, 184, 128, 323, 141, 467, 31, 172, 48, 95, 359, 239, 209, 398, 99, 440, 171, 86, 233, 293, 162, 121, 61, 317, 52, 54, 273, 30, 226, 421, 64, 204, 444, 418, 275, 263, 108, 10, 149, 497, 20, 97, 136, 139, 200, 266, 238, 493, 22, 17, 39]"
Warming up --------------------------------------
           Ruby Sort     9.924k i/100ms
      Insertion Sort   112.000  i/100ms
          Quick Sort   354.000  i/100ms
      Adv Quick Sort   499.000  i/100ms
Calculating -------------------------------------
           Ruby Sort    108.398k (± 6.7%) i/s -    327.492k in   3.035360s
      Insertion Sort      1.202k (± 6.0%) i/s -      3.696k in   3.086332s
          Quick Sort      3.700k (± 5.6%) i/s -     11.328k in   3.071347s
      Adv Quick Sort      5.185k (± 6.5%) i/s -     15.968k in   3.092826s

Comparison:
           Ruby Sort:   108398.1 i/s
      Adv Quick Sort:     5184.7 i/s - 20.91x  slower
          Quick Sort:     3699.9 i/s - 29.30x  slower
      Insertion Sort:     1201.6 i/s - 90.21x  slower

Wow, Insertion Sort become much slower when test case size moving up. Unexpected, right? I wouldn't know that if I don't run these tests.

Even I knew by definition, Insertion Sort is slower than Quicksort. But see the results it in numbers, knowing that smaller array is actually work better with Insertions Sort, how much degradation it will have when array size move up ... all these interesting details aren't written in definition, you have to run tests to find out by yourself.

At the end, learning "algorithms" is not just learn what it is, but also better to learn how to improve it, why one is better than the other and when to use it.

The other examples as you might saw at the beginning of this article, I tried to compare my codes with my teams. And the result is satisfied, not only we all enjoy the competitions, but also we cultivate a culture that we never stop to learn from each other.

Comparison:
    Kelvin-Ruby Sort:    67751.6 i/s
     Afani-Ruby Sort:    62771.8 i/s - same-ish: difference falls within error
    Yunus-Merge Sort:     5194.8 i/s - 13.04x  slower
       Model-No Sort:     1653.6 i/s - 40.97x  slower
   Kelvin-Adv Q-Sort:      744.4 i/s - 91.02x  slower

You might argue, what the "Performance Test" to do with such experiences? I can tell you that, such environment is very difficult to establish if we don't do the test to compare with, because they all giving the same result and finished in a blink of second.

The reason behind that is , once you have some quantifiable data about your code in hand, you can have something to talk about in a conversation, you can tell others how good your programming skill it is in a way everyone can understand even they are not programmer.

So if you got the chances, compare your code with your peers programmer, to see who's code run better. Learn from them if they are better, mentor them if yours is better.

I guess that is exactly why some big coding challenge platforms such as: LeetCode.com, HackerRank.com tried to give a detail performance test result on submission and build up communities that let members can share, compete and learn from each other.

But if you try to do the tests locally use the tool I mention above, you will have more freedoms on choosing who to compare with, what dataset you want to compare with, how long you can wait for it. In other word, you might learn more from it.

Finally, it will help in your career.

In general, when doing coding challenges or developing application, people will only employ very small scale of datasets to test their concept and functionalities.

But in reality, the data volume is usually much bigger. With performance test, you can test it out before the code go into production. It cloud be critical for some high traffic web application.

Below is one demonstration, let's just use the same sorting comparison I mention in early section.

"Sorting with array have 100 numbers"
Comparison:
           Ruby Sort:   105372.3 i/s
      Adv Quick Sort:     5437.3 i/s - 19.38x  slower
          Quick Sort:     3200.4 i/s - 32.92x  slower
      Insertion Sort:     1302.9 i/s - 80.87x  slower

When sample size is 100, the differences between different sorting methods are big, but still the codes can run at least one or few thousands time in 3 seconds (Time period I set per code test), it is human unnoticeable, end user can't tell the differences.

But when data size increase in scale, the differences are significant.

"Sorting with array have 1,000 numbers"
Comparison:
           Ruby Sort:     6370.7 i/s
      Adv Quick Sort:      370.7 i/s - 17.18x  slower
          Quick Sort:      297.9 i/s - 21.38x  slower
      Insertion Sort:       13.9 i/s - 459.65x  slower

It seems still acceptable, but it start to more noticeable.

"Sorting with array have 10,000 numbers"
Comparison:
           Ruby Sort:      477.7 i/s
      Adv Quick Sort:       29.0 i/s - 16.46x  slower
          Quick Sort:       27.1 i/s - 17.66x  slower
      Insertion Sort:        0.1 i/s - 3285.01x  slower

When size up to 10,000, one sorting method will be unbearable (finish the sorting over 30 seconds) .

Imaging 10k isn't much data for some high volume websites, using the wrong code (although it can pass all the unit Test ) would cause your website shut down due to bad performances.

I understand that in the reality, websites as big as I mention above usually have QA or QC team to do the performance tests before go production. But if you can do it from the beginning as a developer, you might help save a lot of time and effort of your teams, therefore save money for your company and elevate your chance getting promote.

Beside trying different size of datasets, you can also try different type of datasets (almost sorted data vs. completed random data, for example, like the example in section 2 above), which can help you better understand what code work better with certain scenarios.

Finally, I think even you just do it occasionally, keep performance test as a good practice in your development career will help you become better developer eventually and have higher market value compare to others only do unit test and integration test.

PS: I hope this article is helpful for you, I would love to hear any comment and ideas from you, and I wish all of you - Happy Coding! You can find the source code of testing from here.