Benchmarking
Benchmarking is an important feature for our CI pipeline; we need to understand when the performance of our application degrades. To do this, we are going to both run our benchmark tests and use the handy benchcmp tool, which compares two runs of tests. The standard version of benchcmp only outputs the difference between two test runs. While this is fine for comparison, it does not give us the ability to fail our CI job should this difference be within a certain threshold. To enable this ability, I have forked the benchcmp tool and added flag-tollerance=[FLOAT]. If any of the benchmarks change within the given tolerance, then benchcmp exits with status code 1, allowing us to fail the job and investigate why this change has taken ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access