Gauger provides developers complete freedom
with respect to the names, values and units of
the performance metrics to be monitored.
functions provided by the VCS could
be used to insert the number into the
source code.
Selecting Proper Units
Gauger provides developers complete
freedom with respect to the names,
values and units of the performance
metrics to be monitored. So, how do
you choose a good unit? Naturally, part
of the unit always is dictated by the
kind of performance characteristic you
are interested in—for example, execution
time (seconds) or space consumption
(bytes). However, generally it’s a good
idea always to relate this basic unit to
the amount of work performed as part
of the unit given to Gauger.
For example, suppose a benchmark
measures the execution time for 100
GET requests. In this case, it is better
to choose a name “GET request perfor-
mance” with unit “requests/second”
(and log the execution time divided by
100) instead of the name “Time for
100 GET requests” with unit “seconds”.
The reason for this is it’s quite possible
the benchmark will be adjusted in the
future—for example, to run 1,000 GET
requests. If performance is logged as
“requests/second”, such a change
would then not require any changes to
the name of the tracked metric. As a
result, the performance regression
analysis can continue to track the metric
in the same curve.
Related Projects
Gauger does not include for support
for actually automatically running
the benchmarks on various systems.
However, this is not a drawback,
because an excellent tool for that
purpose exists in the form of Buildbot.
Buildbot typically is used for regression
testing and, hence, is by itself not
suitable for identifying performance
regressions. Nevertheless, Buildbot
requires a similar network setup—that
is, clients that run the tests connect to
a public server. This makes it trivial to
