18 — Monitoring: Health, Availability, Performance, and the Rest

"Add some logs and hope" was my monitoring plan, and hope is not an observability strategy. Writing it down gave the discipline a structure: monitoring tracks five dimensions — health, availability, performance, security, and usage — built on instrumentation, and made useful only when paired with visualization and alerts. [1] A system you cannot observe is a system you cannot operate.

The framing that landed is that monitoring is a diagnostic loop. The system emits signals (metrics, logs, traces) via instrumentation; those signals get aggregated into dashboards so an operator can spot trends; thresholds on those signals raise alerts when something needs attention. The five dimensions are the categories of question the signals answer, and instrumentation is the non-negotiable foundation — without it, there is nothing to aggregate.

The five dimensions

The roadmap splits monitoring into five dimensions, each answering a different question about the system [1][2][3][4][5]:

• Health monitoring generates a snapshot of the current health so I can verify all components are functioning as expected [2]. A system is healthy if it is running and capable of processing requests. This is the "is it up right now?" dimension.
• Availability monitoring tracks the availability of the system and its components to generate statistics about uptime over time [3]. Closely related to health, but longer-term — health is the immediate snapshot, availability is the track record. This is where the "nines" from the availability notes come from.
• Performance monitoring watches for decreases in performance under stress, because component failure is frequently preceded by a slowdown [4]. If I can detect the degradation early, I can take proactive steps before it becomes an outage.
• Security monitoring records sign-in attempts (success and failure), operations by authenticated users, resources accessed, and session end events [5]. A spike in failed sign-ins may indicate a brute-force attack; a surge in requests may be a DDoS. Every request to every resource should be monitored regardless of source.
• Usage monitoring tracks how features and components are used [6]. The data identifies heavily-used features (candidates for optimization, partitioning, or replication), infrequently-used features (candidates for retirement), operational patterns for capacity planning, and even user satisfaction signals (abandoned carts suggest a checkout problem). It also feeds billing and quota enforcement in multi-tenant systems.

Instrumentation: the foundation

The dimensions describe what to watch; instrumentation is how the watching happens. Instrumentation is the critical part of monitoring — I can only make meaningful decisions about performance and health if I first capture the data that enables those decisions [7]. The information must be sufficient to assess performance, diagnose problems, and make decisions without requiring me to sign in to a remote production server to trace manually. Instrumentation data typically comprises metrics and information written to trace logs.

The modern default is OpenTelemetry — a vendor-neutral standard for emitting traces, metrics, and logs from application code [8]. The point is not which tool; the point is that the instrumentation must exist. A system without instrumentation is opaque, and opacity in production is how small problems become outages.

Visualization and alerts: making monitoring useful

The final layer is what turns raw data into something an operator can act on. Visualization presents the data so trends and problems are visible at a glance; alerts inform the operator quickly when a significant event needs attention [9]. A dashboard nobody looks at is operations theater; an alert that fires for every hiccup is noise that trains operators to ignore it. The discipline is to alert on symptoms users experience (errors, latency) rather than causes (CPU is high), and to tune thresholds so alerts are rare enough to be taken seriously.

How I use this

When I build or operate a system, I instrument from the first commit — request latency, error rate, and request rate (the USE/RED basics: utilization, saturation, and errors for the box; rate, errors, and duration for requests) — because retrofitting instrumentation to a system already in production is painful and always incomplete. I route the five dimensions to dashboards grouped by service, and I set alerts on user-facing symptoms first. The habit that pays off most is reviewing alerts periodically and deleting the ones that fire without action, so that when an alert does go off, it means something. Monitoring that does not drive action is just storage cost.

References

[1] Microsoft, "Monitoring and diagnostics guidance," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/monitoring

[2] Microsoft, "Health monitoring of a system," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/monitoring#health-monitoring

[3] Microsoft, "Availability monitoring," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/monitoring#availability-monitoring

[4] Microsoft, "Performance monitoring," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/monitoring#performance-monitoring

[5] Microsoft, "Security monitoring," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/monitoring#security-monitoring

[6] Microsoft, "Usage monitoring," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/monitoring#usage-monitoring

[7] Microsoft, "Instrumenting an application," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/monitoring#instrumenting-an-application

[8] OpenTelemetry, "What is OpenTelemetry?," 2024. [Online]. Available: https://opentelemetry.io/docs/concepts/what-is-opentelemetry

[9] Microsoft, "Visualizing data and raising alerts," Azure Architecture Center. [Online]. Available: https://learn.microsoft.com/en-us/azure/architecture/best-practices/monitoring#visualizing-data-and-raising-alerts