{"id":2930,"date":"2024-08-24T12:44:36","date_gmt":"2024-08-24T11:44:36","guid":{"rendered":"https:\/\/contentlabstudy.com\/soft\/?p=2930"},"modified":"2026-09-24T21:15:48","modified_gmt":"2026-09-24T20:15:48","slug":"software-metrics-monitoring","status":"publish","type":"post","link":"https:\/\/contentlabstudy.com\/soft\/software-metrics-monitoring\/","title":{"rendered":"Software Metrics Monitoring"},"content":{"rendered":"<p><strong>Software metrics<\/strong> are numbers that change how you act. If a dashboard cannot change a decision this week, it is decoration. Monitor the user\u2019s outcome first, then the resources that produce it.<\/p>\n<h2>Four golden signals (and cousins)<\/h2>\n<ul>\n<li><strong>Latency<\/strong> \u2014 p50 is the typical request; pages and SLOs use p95\/p99 and timeouts. Split success vs error latency.<\/li>\n<li><strong>Traffic<\/strong> \u2014 requests\/s, queue depth, concurrent users. Know the shape of a normal Tuesday.<\/li>\n<li><strong>Errors<\/strong> \u2014 rate and <em>kind<\/em> (5xx vs 429 vs client 4xx). Alert on burn rate, not a single blip.<\/li>\n<li><strong>Saturation<\/strong> \u2014 CPU, thread pool, DB connections, disk. Saturation explains the other three.<\/li>\n<\/ul>\n<p>RED (rate, errors, duration) and USE (utilization, saturation, errors) are the same idea from two ends.<\/p>\n<h2>SLI \/ SLO \/ SLA<\/h2>\n<p>An <strong>SLI<\/strong> is a measured indicator (e.g. \u201cshare of checkout HTTP 2xx under 300 ms\u201d). An <strong>SLO<\/strong> is the target you hold yourselves to. An <strong>SLA<\/strong> is a contract with money attached. Do not write an SLA you cannot measure. Error budgets exist so you can ship: when the budget is burned, freeze features and fix reliability.<\/p>\n<h2>Application metrics worth having<\/h2>\n<p>Queue consumer lag, cache hit ratio, dependency error rate by <em>name<\/em>, GC pause, row counts you care about (orders\/min). Cardinality kills: do not put user id on a Prometheus label.<\/p>\n<h2>Example<\/h2>\n<p>An SLI you can page on: the share of <code>POST \/checkout<\/code> responses that are 2xx and finish in under 300 ms.<\/p>\n<h2>What breaks it<\/h2>\n<p>A <code>user_id<\/code> label on the request metric creates one series per user. Keep <code>route<\/code> and <code>status<\/code>.<\/p>\n<pre><code>http_request_duration_seconds{user_id=\"1842\"}\nhttp_request_duration_seconds{route=\"POST \/checkout\", status=\"200\"}<\/code><\/pre>\n<h2>Pitfalls<\/h2>\n<ul>\n<li>CPU dashboard, no request latency.<\/li>\n<li>Averages hiding a bimodal distribution.<\/li>\n<li>Paging a human for a metric with no runbook.<\/li>\n<li>Vanity: lines of code, story points as \u201cproductivity.\u201d<\/li>\n<\/ul>\n<p>When the metric says \u201cslow,\u201d next: <a href=\"\/soft\/software-performance-issues\/\">performance issues<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Software metrics are numbers that change how you act. If a dashboard cannot change a decision this week, it is [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[12],"tags":[],"class_list":["post-2930","post","type-post","status-publish","format-standard","hentry","category-quality-assurance"],"_links":{"self":[{"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/posts\/2930","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/comments?post=2930"}],"version-history":[{"count":4,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/posts\/2930\/revisions"}],"predecessor-version":[{"id":3233,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/posts\/2930\/revisions\/3233"}],"wp:attachment":[{"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/media?parent=2930"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/categories?post=2930"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/contentlabstudy.com\/soft\/wp-json\/wp\/v2\/tags?post=2930"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}