Too Long; Didn't Read
At DB Benchmarks we specialize in latency testing. We make sure some query against some database takes 117ms today, tomorrow, and in a week. We also develop an open-source platform so you can do it too. In this article, I would like to share the 10 most important principles that we’ve formulated for ourselves that help us make high-quality benchmarks. The principles are: test different databases on the exactly same hardware, test with full OS cache purged before each test. Measure cold run time separately and test on a fixed CPU frequency.