StatsD is an extremely popular open source tool for collecting metrics. The typical configuration is to run the Statsd daemon on each of your servers following the Sidecar pattern, and periodically forward the accumulated metrics to Graphite's Carbon daemon.
This pattern makes it possible to scale network and storage costs based on the cardinality of metrics stored as opposed to the number of metric observations taken. Said differently, this model means you can record each time a server performs an HTTP request, and you'll use the same amount of disk space on Graphite and the same amount of network bandwidth to transmit those metrics, whether you're doing 100 requests per minute or 10,000 requests per second.
Another impressive aspect of StatsD, and likely why it has remained popular and been reimplemented in numerous languages, is the simple protocol that clients use to communicate with it.
http.request:1|c http.request:1|c http.duration:103|ms http.duration:12|ms
Those four commands would increment the
http.request counter twice,
and add two timings for the
http.duration timer. These counters and times
accumulate until a flush is triggered, typically every ten seconds or so.
This system challenge is to build an implementation of StatsD, which takes commands from standard in,
supporting both the counter and timer syntaxes and also an added
flush commands prints the accumulated data to standard out, and then resets
accumulated state to empty.
A few more detailed rules about flushing, to get matching output. Flushed keys should be ordered alphabetically, and that there will be no timers and counters that share the same. Flushed timers should represent the average of all supplied timers for a given metric, and should be integers (effectively the floor, as you'll throw away any decimal level accuracy before flushing the output).
req:1|c req:3|c resp:89|c flush req:12|c flush req:201|c dur:100|ms dur:300|ms flush
# no output # no output # no output req: 4, resp: 89 # no output req: 12 # no output # no output # no output dur: 200, req: 201
Once you have that input/output working properly, you're ready to take on the complete challenge below!