Prometheus.jl
Introduction
This package is a Julia client for Prometheus. If you are not familiar with Prometheus it is recommended to browse the upstream documentation. The documentation here focuses on the Julia client.
Two of the basic concepts of a Prometheus client are Registries and Collectors. Registries are collections of collectors, and the collectors are the units responsible to record and capture metrics. Client libraries implement a default registry which all collectors implicity register with, so for basic usage there is no need to interact with a registry (see Default registry).
The third important concept is Exposition of the collected metrics. Typically metrics are exposed over a HTTP server, as in the Quickstart-example just below. See the section about Exposition for more details and examples on how metrics can be exposed.
Quickstart
Install Prometheus.jl and HTTP.jl using the package manager:
pkg> add Prometheus HTTPPaste the following code into a Julia REPL.
# Load the packages +using Prometheus, HTTP + +# Create a Counter metric +const request_counter = Prometheus.Counter("request_count", "Number of handled requests") + +# Start a HTTP server on localhost port 8000 to server the metrics +server = HTTP.listen!(8000) do http + Prometheus.inc(request_counter) # Increment the request counter + return Prometheus.expose(http) # Expose the metrics +endVisit http://localhost:8000 in your browser. You will see something like the following
# HELP gc_alloc_bytes_total Total number of allocated bytes +# TYPE gc_alloc_bytes_total counter +gc_alloc_bytes_total 365578814 + +[...] + +# HELP request_count Number of handled requests +# TYPE request_count counter +request_count 1The output contains some default metrics related to the running process, as well as the request counter that we added ourselves. Every time you refresh, the counter will increment its value.
close(server)will shutdown the server.
Collectors
This section documents the collectors that are currently supported. This include the "basic" collectors (Counter, Gauge, Histogram, Summary) as well as some custom collectors (GCCollector, ProcessCollector). There is also a section on how to implement your own collector, see Custom collectors.
Upstream documentation:
- https://prometheus.io/docs/concepts/metric_types/
- https://prometheus.io/docs/instrumenting/writing_clientlibs/#metrics
Counter
Quoting the upstream documentation:
A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart. For example, you can use a counter to represent the number of requests served, tasks completed, or errors.
Do not use a counter to expose a value that can decrease. For example, do not use a counter for the number of currently running processes; instead use a gauge.
Counter API reference
Prometheus.Counter — MethodPrometheus.Counter(name, help; registry=DEFAULT_REGISTRY)Construct a Counter collector.
Arguments
name :: String: the name of the counter metric.help :: String: the documentation for the counter metric.
Keyword arguments
registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Passregistry = nothingto skip registration.
Methods
Prometheus.inc: increment the counter.
Prometheus.inc — MethodPrometheus.inc(counter::Counter, v::Real = 1)Increment the value of the counter with v. The value defaults to v = 1.
Throw a Prometheus.ArgumentError if v < 0 (a counter must not decrease).
Gauge
Quoting the upstream documentation:
A gauge is a metric that represents a single numerical value that can arbitrarily go up and down.
Gauges are typically used for measured values like temperatures or current memory usage, but also "counts" that can go up and down, like the number of concurrent requests.
Gauge API reference
Prometheus.Gauge — MethodPrometheus.Gauge(name, help; registry=DEFAULT_REGISTRY)Construct a Gauge collector.
Arguments
name :: String: the name of the gauge metric.help :: String: the documentation for the gauge metric.
Keyword arguments
registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Passregistry = nothingto skip registration.
Methods
Prometheus.inc: increment the value of the gauge.Prometheus.dec: decrement the value of the gauge.Prometheus.set: set the value of the gauge.Prometheus.set_to_current_time: set the value of the gauge to the current unixtime.Prometheus.@time: time a section and set the value of the the gauge to the elapsed time.Prometheus.@inprogress: Track number of inprogress operations; increment the gauge when entering the section, decrement it when leaving.
Prometheus.inc — MethodPrometheus.inc(gauge::Gauge, v::Real = 1)Increment the value of the gauge with v. v defaults to v = 1.
Prometheus.dec — MethodPrometheus.dec(gauge::Gauge, v::Real = 1)Decrement the value of the gauge with v. v defaults to v = 1.
Prometheus.set — MethodPrometheus.set(gauge::Gauge, v::Real)Set the value of the gauge to v.
Prometheus.set_to_current_time — MethodPrometheus.set_to_current_time(gauge::Gauge)Set the value of the gauge to the current unixtime in seconds.
Prometheus.@time — MacroPrometheus.@time collector exprTime the evaluation of expr and send the elapsed time in seconds to collector. The specific action depends on the type of collector:
collector :: Gauge: set the value of the gauge to the elapsed time (Prometheus.set)collector :: Histogramandcollector :: Summary: add the elapsed time as an observation (Prometheus.observe)
The expression to time, expr, can be a single expression (for example a function call), or a code block (begin, let, etc), e.g.
Prometheus.@time collector <expr>
+
+Prometheus.@time collector begin
+ <expr>
+endIt is also possible to apply the macro to a function definition, i.e.
Prometheus.@time collector function time_this(args...)
+ # function body
+endto time every call to this function (covering all call sites).
Prometheus.@inprogress — MacroPrometheus.@inprogress collector exprTrack the number of concurrent in-progress evaluations of expr. From the builtin collectors this is only applicable to the Gauge – the value of the gauge is incremented with 1 when entering the section, and decremented with 1 when exiting the section.
The expression, expr, can be a single expression (for example a function call), or a code block (begin, let, etc), e.g.
Prometheus.@inprogress collector <expr>
+
+Prometheus.@inprogress collector begin
+ <expr>
+endIt is also possible to apply the macro to a function definition, i.e.
Prometheus.@inprogress collector function track_this(args...)
+ # function body
+endto track number of concurrent in-progress calls (covering all call sites).
Histogram
Quoting the upstream documentation:
A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values.
Histogram API reference
Prometheus.Histogram — MethodPrometheus.Histogram(name, help; buckets=DEFAULT_BUCKETS, registry=DEFAULT_REGISTRY)Construct a Histogram collector.
Arguments
name :: String: the name of the histogram metric.help :: String: the documentation for the histogram metric.
Keyword arguments
buckets :: Vector{Float64}: the upper bounds for the histogram buckets. The buckets must be sorted.Infwill be added as a last bucket if not already included. The default buckets areDEFAULT_BUCKETS = [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1.0, 2.5, 5.0, 7.5, 10.0, Inf].registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Passregistry = nothingto skip registration.
Methods
Prometheus.observe: add an observation to the histogram.Prometheus.@time: time a section and add the elapsed time as an observation.
Prometheus.observe — MethodPrometheus.observe(histogram::Histogram, v::Real)Add the observed value v to the histogram. This increases the sum and count of the histogram with v and 1, respectively, and increments the counter for all buckets containing v.
Prometheus.@time — MacroPrometheus.@time collector exprTime the evaluation of expr and send the elapsed time in seconds to collector. The specific action depends on the type of collector:
collector :: Gauge: set the value of the gauge to the elapsed time (Prometheus.set)collector :: Histogramandcollector :: Summary: add the elapsed time as an observation (Prometheus.observe)
The expression to time, expr, can be a single expression (for example a function call), or a code block (begin, let, etc), e.g.
Prometheus.@time collector <expr>
+
+Prometheus.@time collector begin
+ <expr>
+endIt is also possible to apply the macro to a function definition, i.e.
Prometheus.@time collector function time_this(args...)
+ # function body
+endto time every call to this function (covering all call sites).
Summary
Quoting the upstream documentation:
Similar to a histogram, a summary samples observations (usually things like request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window.
Summary API reference
Prometheus.Summary — MethodPrometheus.Summary(name, help; registry=DEFAULT_REGISTRY)Construct a Summary collector.
Arguments
name :: String: the name of the summary metric.help :: String: the documentation for the summary metric.
Keyword arguments
registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Passregistry = nothingto skip registration.
Methods
Prometheus.observe: add an observation to the summary.Prometheus.@time: time a section and add the elapsed time as an observation.
Prometheus.observe — MethodPrometheus.observe(summary::Summary, v::Real)Add the observed value v to the summary. This increases the sum and count of the summary with v and 1, respectively.
Prometheus.@time — MacroPrometheus.@time collector exprTime the evaluation of expr and send the elapsed time in seconds to collector. The specific action depends on the type of collector:
collector :: Gauge: set the value of the gauge to the elapsed time (Prometheus.set)collector :: Histogramandcollector :: Summary: add the elapsed time as an observation (Prometheus.observe)
The expression to time, expr, can be a single expression (for example a function call), or a code block (begin, let, etc), e.g.
Prometheus.@time collector <expr>
+
+Prometheus.@time collector begin
+ <expr>
+endIt is also possible to apply the macro to a function definition, i.e.
Prometheus.@time collector function time_this(args...)
+ # function body
+endto time every call to this function (covering all call sites).
GCCollector
A collector that exports metrics about allocations and garbage collection (for example number of allocations, number of bytes allocated, time spent in garbage collection, etc). These metrics have the julia_gc_ prefix in their name.
A GCCollector is registered automatically with the default registry, see Default registry for more details.
GCCollector API reference
Prometheus.GCCollector — MethodPrometheus.GCCollector(; registry=DEFAULT_REGISTRY)Create a collector that exports metrics about allocations and garbage collection.
Keyword arguments
registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. The default registry is used by default. Passregistry = nothingto skip registration.
A GCCollector is registered automatically with the default registry. If necessary it can be removed by calling
Prometheus.unregister(Prometheus.DEFAULT_REGISTRY, Prometheus.GC_COLLECTOR)ProcessCollector
A collector that exports metrics about a running process, for example CPU seconds and metrics about I/O operations. Metrics from this collector have the process_ prefix in their name. This collector is only available on Linux since it requires the /proc file system.
A ProcessCollector for the current process is registered automatically with the default registry, see Default registry for more details.
ProcessCollector API reference
Prometheus.ProcessCollector — MethodPrometheus.ProcessCollector(pid; registry=DEFAULT_REGISTRY)Create a process collector for the process id given by the pid function. The collector exposes metrics about the process' CPU time, start time, memory usage, file usage, and I/O operations.
Arguments
pid :: Function: a function returning a process id as a string or integer for which to collect metrics. By default the"self"pid is used, i.e. the current process.
Keyword arguments
registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. The default registry is used by default. Passregistry = nothingto skip registration.
A ProcessCollector for the current process is registered automatically with the default registry. If necessary it can be removed by calling
Prometheus.unregister(Prometheus.DEFAULT_REGISTRY, Prometheus.PROCESS_COLLECTOR)The process collector is currently only available on Linux since it requires the /proc file system. On Windows and macOS this collector will not expose any metrics.
Custom collectors
RandomCollector
Labels
Prometheus allows attaching labels to metrics, see the upstream documentation:
- https://prometheus.io/docs/practices/naming/#labels
- https://prometheus.io/docs/practices/instrumentation/#use-labels
- https://prometheus.io/docs/practices/instrumentation/#do-not-overuse-labels
In this package labeling of collectors is done with Prometheus.Family. A collector family consist of a number of regular collectors, the children, with unique labels.
A concrete example is a HTTP request Counter, where we might also want to keep track of the target resource and the status code of the request. Such instrumentation can be implemented as follows
# Custom label struct
+struct RequestLabels
+ target::String
+ status_code::Int
+end
+
+# Create the counter family
+request_counter = Prometheus.Family{Prometheus.Counter}(
+ "http_requests", "Total number of HTTP requests", RequestLabels
+)
+
+# Extract a Counter for a specific set of labels
+counter = Prometheus.labels(request_counter, RequestLabels("/api", 200))
+
+# Increment the counter
+Prometheus.inc(counter)Note that using a custom label struct is optional, refer to the constructor Prometheus.Family and Prometheus.labels for alternative methods.
Family API reference
Prometheus.Family — MethodPrometheus.Family{C}(name, help, args..., label_names; registry=DEFAULT_REGISTRY, kwargs...)Create a labeled collector family with labels given by label_names. For every new set of label values encountered a new collector of type C <: Collector will be created, see Prometheus.labels.
Arguments
name :: String: the name of the family metric.help :: String: the documentation for the family metric.args...: any extra positional arguments required forCs constructor, seePrometheus.labels.label_names: the label names for the family. Label names can be given as either of the following (typically matching the methods label values will be given later, seePrometheus.labels):- a tuple of symbols or strings, e.g.
(:target, :status_code)or("target", "status_code") - a named tuple type, e.g.
@NamedTuple{target::String, status_code::Int}where the names are used as the label names - a custom struct type, e.g.
RequestLabelsdefined as
where the field names are used for the label names.struct RequestLabels + target::String + status_code::Int +end
- a tuple of symbols or strings, e.g.
Keyword arguments
registry :: Prometheus.CollectorRegistry: the registry in which to register the collector. If not specified the default registry is used. Passregistry = nothingto skip registration.kwargs...: any extra keyword arguments required forCs constructor, seePrometheus.labels.
Methods
Prometheus.labels: get or create the collector for a specific set of labels.Prometheus.remove: remove the collector for a specific set of labels.Prometheus.clear: remove all collectors in the family.
Examples
# Construct a family of Counters
+counter_family = Prometheus.Family{Counter}(
+ "http_requests", "Number of HTTP requests", (:target, :status_code),
+)
+
+# Increment the counter for the labels `target="/api"` and `status_code=200`
+Prometheus.inc(Prometheus.labels(counter_family, (target="/api", status_code=200)))Prometheus.labels — MethodPrometheus.labels(family::Family{C}, label_values) where CGet or create the collector of type C from the family corresponding to the labels given by label_values. If no collector exist for the input labels a new one is created by invoking the C constructor as C(name, help, args...; kwargs..., registry=nothing), where name, help, args..., and kwargs... are the arguments from the family constructor, see Family.
Similarly to when creating the Family, label_values can be given as either of the following:
- a tuple, e.g.
("/api", 200) - a named tuple with names matching the label names, e.g.
(target="/api", status_code=200) - a struct instance with field names matching the label names , e.g.
RequestLabels("/api", 200)
All non-string values (e.g. 200 in the examples above) are stringified using string.
Base.getindex is overloaded to have the meaning of Prometheus.labels for the family collector: family[label_values] is equivalent to Prometheus.labels(family, label_values).
This method does an acquire/release of a lock, and a dictionary lookup, to find the collector matching the label names. For typical applications this overhead does not matter (below 100ns for some basic benchmarks) but it is safe to cache the returned collector if required.
Prometheus.remove — MethodPrometheus.remove(family::Family, label_values)Remove the collector corresponding to label_values. Effectively this resets the collector since Prometheus.labels will recreate the collector when called with the same label names.
Refer to Prometheus.labels for how to specify label_values.
This method invalidates cached collectors for the label names.
Prometheus.clear — MethodPrometheus.clear(family::Family)Remove all collectors in the family. Effectively this resets the collectors since Prometheus.labels will recreate them when needed.
This method invalidates all cached collectors.
Registries
Default registry
Exposition
Prometheus support
Prometheus.expose — FunctionPrometheus.expose(file::String, reg::CollectorRegistry = DEFAULT_REGISTRY)Export all metrics in reg by writing them to the file file.
expose(io::IO, reg::CollectorRegistry = DEFAULT_REGISTRY)Export all metrics in reg by writing them to the I/O stream io.
expose(http::HTTP.Stream, reg::CollectorRegistry = DEFAULT_REGISTRY; kwargs...)Export all metrics in reg by writing them to the HTTP stream http.
The caller is responsible for checking e.g. the HTTP method and URI target. For HEAD requests this method do not write a body, however.