Skip to main content

Metric graphs 101: Timeseries graphs

This is the first post in a series about visualizing monitoring data. This post focuses on timeseries graphs.
Observability is not just about having monitoring data—that data must be easily available and interpretable. Choosing the right visualization for your data is an important part of providing human-readable representations of the health and performance of your systems. There is no one-size-fits-all solution: you can see different things in the same metric with different graph types.
To help you effectively visualize your metrics, this first post explores four different types of timeseries graphs, which have time on the x-axis and metric values on the y-axis:
For each graph type, we’ll explain how it works, when to use it, and when to use something else.

Line graphs

Line graph
Line graphs are the simplest way to translate metric data into visuals, but often they’re used by default when a different graph would be more appropriate. For instance, a graph of wildly fluctuating metrics from hundreds of hosts quickly becomes harder to disentangle than steel wool. It’s nearly impossible to draw any useful conclusions about your systems from a graph like that.

When to use line graphs

WhatWhyExample
The same metric reported by different scopesTo spot outliers at a glanceCPU idle for each host in a cluster
CPU idle per host
Tracking single metrics from one source, or as an aggregateTo clearly communicate a key metric's evolution over timeMedian latency across all web servers
Median webapp latency
Metrics for which unaggregated values from a particular slice of your infrastructure are especially valuableTo spot individual deviations into unacceptable rangesDisk space utilization per database node
Disk space per node
Related metrics sharing the same unitsTo spot correlations within a systemLatency for disk reads and disk writes on the same machine
Read and write latency
Metrics that have a clear acceptable domainTo easily spot unacceptable degradationsLatency for processing web requests
Key page latency

When to use something else

WhatExampleInstead use...
Highly variable metrics reported by a large number of sourcesCPU from all hosts
Noisy CPU line graph
Heat maps to make noisy data more interpretable
CPU heat map
Metrics that are more actionable as aggregatesthan as separate data pointsWeb requests per second over dozens of web servers
Web server requests per node
Area graphs to aggregate across tagged groups
Web server requests per availability zone
Metrics that are often equal to zeroMetrics tracking relatively rare S3 access errors
Error count
Bar graphs to avoid jumpy interpolations
Error count bar graph

Stacked area graphs

Area graph
Area graphs are similar to line graphs, except the metric values are represented by two-dimensional bands rather than lines. Multiple timeseries can be summed together simply by stacking the bands, but too many bands makes the graph hard to interpret. If each band is only a pixel or two tall, the information conveyed is minimal.

When to use stacked area graphs

WhatWhyExample
The same metric from different scopes, stackedTo check both the sum and the contribution of each of its parts at a glanceLoad balancer requests per availability zone
Load balancer requests per availability zone
Summing complementary metrics that share the same unitTo see how a finite resource is being utilizedCPU utilization metrics (user, system, idle, etc.)
CPU utilization, stacked graph

When to use something else

WhatExampleInstead use...
Unaggregated metrics from large numbers of hosts, making the slices too thin to be meaningfulThroughput metrics across hundreds of app servers
Overloaded stacked area graph
Line graph or solid-color area graph to track total, aggregate value
Solid area graph

Heat maps to track host-level data
Throughput heat map
Metrics that can't be added sensiblySystem load across multiple servers
Stacked system load graph
Line graphs, or heat maps for large numbers of hosts
System load line graph

Bar graphs

Bar graph
In a bar graph, each bar represents a metric rollup over a time interval. This feature makes bar graphs ideal for representing counts. Unlike gauge metrics, which represent an instantaneous value, count metrics only make sense when paired with a time interval (e.g., 13 server errors in the past five minutes).
Bar graphs require no interpolation to connect one interval to the next, making them especially useful for representing sparse metrics. Like area graphs, they naturally accommodate stacking and summing of metrics.

When to use bar graphs

WhatWhyExample
Sparse metrics (e.g. metrics tracking rare events)To convey metric values without jumpy or misleading interpolationsBlocked tasks in Cassandra's internal queues
Blocked tasks bar graph
Metrics that represent a count (rather than a gauge)To convey both the total count and the corresponding time intervalFailed jobs, by data center (4-hour intervals)
Failed jobs bar graph

When to use something else

WhatExampleInstead use...
Metrics that can't be added sensiblyAverage latency per load balancer
Confusingly summed latency metrics
Line graphs to isolate timeseries from each host
Isolated latency metrics
Unaggregated metrics from large numbers of sources, making the slices too thin to be meaningfulCompleted tasks across dozens of Cassandra nodes
Bar graph of completed tasks
Solid-color bars to track total, aggregate metric value
Solid-color bar graph

Heat maps to track host-level values
Heat map of completed tasks

Heat maps

Heat map
Heat maps show the distribution of values for a metric evolving over time. Specifically, each column represents a distribution of values during a particular time slice. Each cell’s shading corresponds to the number of entities reporting that particular value during that particular time.
Heat maps are essentially distribution graphs, except that heat maps show change over time, and distribution graphs are a snapshot of a particular window of time. Distributions are covered in Part 2 of this series.

When to use heat maps

WhatWhyExample
Single metric reported by a large number of groupsTo convey general trends at a glanceWeb latency per host
Webapp latency heat map



To see transient variations across members of a groupRequests received per host
Requests per host heat map

When to use something else

WhatExampleInstead use...
Metrics coming from only a few individual sourcesCPU utilization across a small number of RDS instances
Sparse heat map of CPU
Line graphs to isolate timeseries from each host
Line graph of CPU
Metrics where aggregates matter more than individual valuesDisk utilization per Cassandra column family
Sparse heat map of CPU
Area graphs to sum values across a set of tags
Line graph of CPU

Conclusion

By understanding the ideal use cases and limitations of each kind of timeseries graph, you can present actionable information from your metrics more clearly, thereby providing observability into your systems.
In the next article in this series, we’ll explore other methods of graphing and monitoring metrics, including change graphs, ranked lists, distributions, and other visualizations.

Comments

Popular posts from this blog

OWASP Top 10 Threats and Mitigations Exam - Single Select

Last updated 4 Aug 11 Course Title: OWASP Top 10 Threats and Mitigation Exam Questions - Single Select 1) Which of the following consequences is most likely to occur due to an injection attack? Spoofing Cross-site request forgery Denial of service   Correct Insecure direct object references 2) Your application is created using a language that does not support a clear distinction between code and data. Which vulnerability is most likely to occur in your application? Injection   Correct Insecure direct object references Failure to restrict URL access Insufficient transport layer protection 3) Which of the following scenarios is most likely to cause an injection attack? Unvalidated input is embedded in an instruction stream.   Correct Unvalidated input can be distinguished from valid instructions. A Web application does not validate a client’s access to a resource. A Web action performs an operation on behalf of the user without checking a shared sec

CKA Simulator Kubernetes 1.22

  https://killer.sh Pre Setup Once you've gained access to your terminal it might be wise to spend ~1 minute to setup your environment. You could set these: alias k = kubectl                         # will already be pre-configured export do = "--dry-run=client -o yaml"     # k get pod x $do export now = "--force --grace-period 0"   # k delete pod x $now Vim To make vim use 2 spaces for a tab edit ~/.vimrc to contain: set tabstop=2 set expandtab set shiftwidth=2 More setup suggestions are in the tips section .     Question 1 | Contexts Task weight: 1%   You have access to multiple clusters from your main terminal through kubectl contexts. Write all those context names into /opt/course/1/contexts . Next write a command to display the current context into /opt/course/1/context_default_kubectl.sh , the command should use kubectl . Finally write a second command doing the same thing into /opt/course/1/context_default_no_kubectl.sh , but without the use of k

标 题: 关于Daniel Guo 律师

发信人: q123452017 (水天一色), 信区: I140 标  题: 关于Daniel Guo 律师 关键字: Daniel Guo 发信站: BBS 未名空间站 (Thu Apr 26 02:11:35 2018, 美东) 这些是lz根据亲身经历在 Immigration版上发的帖以及一些关于Daniel Guo 律师的回 帖,希望大家不要被一些马甲帖广告帖所骗,慎重考虑选择律师。 WG 和Guo两家律师对比 1. fully refund的合约上的区别 wegreened家是case不过只要第二次没有file就可以fully refund。郭家是要两次case 没过才给refund,而且只要第二次pl draft好律师就可以不退任何律师费。 2. 回信速度 wegreened家一般24小时内回信。郭律师是在可以快速回复的时候才回复很快,对于需 要时间回复或者是不愿意给出确切答复的时候就回复的比较慢。 比如:lz问过郭律师他们律所在nsc区域最近eb1a的通过率,大家也知道nsc现在杀手如 云,但是郭律师过了两天只回复说让秘书update最近的case然后去网页上查,但是上面 并没有写明tsc还是nsc。 lz还问过郭律师关于准备ps (他要求的文件)的一些问题,模版上有的东西不是很清 楚,但是他一般就是把模版上的东西再copy一遍发过来。 3. 材料区别 (推荐信) 因为我只收到郭律师写的推荐信,所以可以比下两家推荐信 wegreened家推荐信写的比较长,而且每封推荐信会用不同的语气和风格,会包含lz写 的research summary里面的某个方面 郭家四封推荐信都是一个格式,一种语气,连地址,信的称呼都是一样的,怎么看四封 推荐信都是同一个人写出来的。套路基本都是第一段目的,第二段介绍推荐人,第三段 某篇或几篇文章的abstract,最后结论 4. 前期材料准备 wegreened家要按照他们的模版准备一个十几页的research summary。 郭律师在签约之前说的是只需要准备五页左右的summary,但是在lz签完约收到推荐信 ,郭律师又发来一个很长的ps要lz自己填,而且和pl的格式基本差不多。 总结下来,申请自己上心最重要。但是如果选律师,lz更倾向于wegreened,