12 — Load & Performance Testing — Metrics Before Tools

"Pick a load tool, point it at the server, see what breaks" was my performance-testing plan, and it produced numbers without a question. The inversion that fixed it: performance testing is metrics-first — p95 and p99 latency, throughput, error rate, Apdex — measured against defined service-level objectives, and the tool choice only matters once you know which metric answers your question [1][2]. Generating traffic without an SLO is just producing numbers; the discipline is deciding what "good enough" looks like before the first request fires, then choosing the load pattern that surfaces whether you meet it.

The framing that landed is the split between load testing the system (many users, server-side metrics — is the backend holding up?) and performance testing the page (one user, browser-side metrics — is the experience fast?). The roadmap's tool list mixes both, and they answer different questions. k6 and JMeter generate server load; Lighthouse and WebPageTest measure a single user's experience. Confusing the two is how teams "pass performance" on the load test while shipping a page that takes 8 seconds to become interactive.

The metrics that actually matter

Before any tool, the metrics. The roadmap's resources are emphatic that the raw average is misleading and percentiles — p95, p99 — are the real signal [3]:

• p50 (median) — half of requests are faster than this. Comforting, but hides the worst experiences.
• p95 — 95% of requests are faster; 1 in 20 is this slow or worse. This is where users actually live.
• p99 — 99% are faster; 1 in 100 is this slow. This is the tail, where churn happens.

The diagram shows the trap: a p50 of 200ms looks great in a dashboard, while the p95 of 800ms (past the SLO) and p99 of 1500ms mean 1 in 20 and 1 in 100 users are having a bad experience. Reporting only the median is reporting only the happy half.

The complementary metric is Apdex (Application Performance Index), a single score from 0 to 1 that buckets response times into "satisfied," "tolerating," and "frustrated" against a threshold [4]. Apdex is a way to communicate performance to non-technical stakeholders in one number, with the buckets defined explicitly so the score is interpretable.

The load patterns — what each one reveals

The roadmap's umbrella node and resources distinguish load patterns by shape of stress [1][5][6]:

• Load testing — apply expected real-life load and measure. "Can we handle our normal peak?" This is the baseline; every system should pass its own load test.
• Stress testing — push beyond normal capacity to find the breaking point. "What happens when we exceed it?" The output is how it fails — gracefully or catastrophically — which informs recovery design.
• Spike testing — apply sudden sharp surges. "Can we absorb a traffic spike from a launch or a viral moment?" Reveals whether autoscaling and queueing absorb the burst.
• Soak / endurance testing — sustained load over hours or days. Catches memory leaks, connection-pool exhaustion, and slow degradation that short tests never see.

Each pattern surfaces a different defect class. A team that only does load testing passes the happy case and ships a system that crashes on the first spike or leaks memory over a weekend.

The load-generation tools

With metrics and patterns clear, the tools divide by how scenarios are written:

• k6 (Grafana) — developer-centric, scenarios written in JavaScript, designed to catch performance regressions early in CI [7]. The "tests as code" framing makes k6 fit naturally alongside functional automation; the same PR that adds a feature adds a load test for it.
• JMeter (Apache) — the enterprise veteran, GUI-authored test plans, broad protocol support, mature reporting [8]. Heavyweight, Java-based, and ubiquitous in enterprise contexts. The GUI is great for authoring, awkward for version control.
• Gatling — Scala-based, scenarios written as code with a fluent DSL, version-controlled, known for efficient concurrent-user simulation and detailed HTML reports [9].
• Locust — Python-based, user behavior defined in plain Python, lightweight coroutines instead of threads for efficiency, with a real-time web UI [10]. The pick for Python teams who want load tests in their own language.
• Artillery — YAML or JavaScript scenarios, HTTP/WebSocket/Socket.io, designed to run in CI and integrate with monitoring [11].
• Vegeta — a versatile HTTP load testing tool, usable as a CLI or a Go library, built for constant request rate [12]. Minimal, focused, single-purpose.

The pattern: every modern load tool trends toward "tests as code" (k6, Gatling, Locust, Artillery) over the GUI-authoring model (JMeter). Code-based scenarios version alongside the application, review in PRs, and run in CI — the same advantages functional test automation realized a decade ago.

The page-performance tools

Distinct from load generation, the roadmap includes tools that measure a single user's experience:

• Lighthouse — Google's open-source auditor for performance, accessibility, PWA, SEO, running in Chrome DevTools, CLI, or as a Node module [13]. Its lab-based scores (the synthetic single-page run) differ from real-user data, and the roadmap's resources explicitly flag this gap — Lighthouse lab scores aren't field data [14].
• WebPageTest — deep diagnostic tool, runs from different global locations on real browsers over customizable network conditions [15]. Where Lighthouse gives a score, WebPageTest gives the filmstrip — frame-by-frame rendering — that lets you see why the page is slow.

The load tools answer "can the backend handle the traffic?"; Lighthouse and WebPageTest answer "is the page fast for one user on a real connection?" Both matter; neither substitutes for the other.

How I use this

The practical payoff is a strict ordering: metrics, then SLOs, then pattern, then tool.

1. Define the SLO first. "p95 login under 500ms." Without this, every load run produces numbers with no verdict.
2. Report percentiles, not averages. A dashboard that shows only the mean is hiding the tail where users suffer.
3. Match the pattern to the question. Load test for the baseline; stress test for failure mode; spike test for bursts; soak test for leaks.
4. Choose the tool by scenario language. JavaScript team → k6. Python team → Locust. JVM/enterprise → JMeter or Gatling. Code-based, version-controlled, CI-integrated.
5. Separate backend load from page experience. k6 tells you the backend holds; Lighthouse and WebPageTest tell you the page is fast. A passing load test with a slow page is still a slow product.
6. Gate on regressions in CI. Performance regression as a CI gate — the roadmap's resources explicitly call this out [16] — catches the slow drift that manual testing misses.

The deeper habit is refusing to run a load test without an SLO. "Let's see what it can do" is exploration, which is fine, but it's not performance testing until there's a defined threshold the result passes or fails against. Performance testing without SLOs is just generating traffic and hoping to feel something.

References

[1] Evaluat, "Performance testing: the complete guide," 2024. [Online]. Available: https://www.evaluat.com/blog/performance-testing-guide

[2] Evaluat, "What is performance testing? A QA engineer's guide," 2024. [Online]. Available: https://www.evaluat.com/blog/what-is-performance-testing

[3] Evaluat, "Reading p95 and p99 response time," 2024. [Online]. Available: https://www.evaluat.com/blog/p95-p99-response-time

[4] Evaluat, "What is an Apdex score?," 2024. [Online]. Available: https://www.evaluat.com/blog/what-is-an-apdex-score

[5] Evaluat, "What is spike testing? Preparing for traffic surges," 2024. [Online]. Available: https://www.evaluat.com/blog/what-is-spike-testing

[6] Evaluat, "Soak testing: catching memory leaks over time," 2024. [Online]. Available: https://www.evaluat.com/blog/soak-testing

[7] Grafana k6, "Grafana k6 — load testing for engineering teams," 2024. [Online]. Available: https://k6.io/

[8] Apache, "Apache JMeter," 2024. [Online]. Available: https://jmeter.apache.org/

[9] Gatling, "Gatling — load testing for HTTP services," 2024. [Online]. Available: https://gatling.io/

[10] Locust, "Locust — scalable load testing in Python," 2024. [Online]. Available: https://locust.io/

[11] Artillery, "Artillery — load testing for APIs and microservices," 2024. [Online]. Available: https://www.artillery.io/

[12] tsenart, "Vegeta — HTTP load testing," GitHub, 2024. [Online]. Available: https://github.com/tsenart/vegeta

[13] Evaluat, "Core Web Vitals: why Lighthouse scores differ from real users," 2024. [Online]. Available: https://www.evaluat.com/blog/core-web-vitals-lab-vs-field

[14] Evaluat, "Core Web Vitals under load, explained," 2024. [Online]. Available: https://www.evaluat.com/blog/core-web-vitals-load-testing

[15] WebPageTest, "WebPageTest," 2024. [Online]. Available: https://www.webpagetest.org/

[16] Evaluat, "Performance regression testing as a CI/CD gate," 2024. [Online]. Available: https://www.evaluat.com/blog/performance-regression-testing