---
title: "07 — Backend Performance: Code Optimization (Profile, Pick, Decompose, Stream, Bound)"
uid: backend-performance-code-optimization
tags: ["algorithms", "streaming", "architecture", "profiling", "batching", "timeouts", "roadmap:backend-performance", "backend", "performance"]
excerpt: "Profile before you optimize — then the eight code-optimization items are one move each against a specific way of asking the machine to do unnecessary work."
date: 2026-08-13T04:50:26+0000
source: https://www.aveshina.my.id/en/blog/backend-performance-code-optimization
---

Second-largest section, same shape as the database one: 8 Code Optimization items [1] that reduce to a thread. The thread: **profile before you optimize, and most items below are one move against a specific way of asking the machine to do unnecessary work.** The eight items split into four pairs.

- **Measure, then choose** (#35, #36) — profile and pick the right algorithm.
- **Big bodies and hot paths** (#34, #37) — stream large parsing, isolate critical endpoints.
- **Decisions about runtime and structure** (#38, #39) — pick a compiled language where it counts, decompose when one box can't carry the whole system.
- **Bound your dependencies on other slow systems** (#40, #41) — timeouts and retries, batch your calls.

## Measure, then choose

- **Profile your code to identify performance bottlenecks** — the rule that precedes all the others. The roadmap is explicit: the first move is prof, not optimization. The reasons are universal — without measurement, an "optimization" is as likely to be slower as faster (a cache lookup that's more expensive than the thing it caches, a manual loop unroll that defeats the JIT). Tools I use: pprof for Go, py-spy for Python, the JVM profilers (async-profiler is the modern choice), and Linux perf for system-wide CPU profiling. A flame graph of a representative load genuinely tells you which of the eight items below you need — usually only one or two — and which ones you can ignore.
- **Optimize algorithms and data structures used** — the move profiling usually points at. The big-O step — how the work grows as the input grows — matters more than any of the small constant-factor tweaks below. A linear search replaced by a hash lookup is a 1000× win that no caching or compiled-language move can match. The rule I keep: if profiling shows a hot function, ask "is its complexity what you expect, or did the inner loop quietly become quadratic — every element touching every other element?" The common wrong-big-O patterns: nested lookups where a dict would do, sorting inside a hot path (sort once outside), and linear scans through a list that should have been a set.

## Big bodies and hot paths

- **Implement streaming of large requests/responses** — when a body is bigger than a memory-budget threshold (a 500 MB upload, a 2 GB report), buffering the whole thing in RAM costs you both memory and concurrency — you can hold only a few concurrent requests before OOM. Streaming reads the body in chunks and processes each chunk before the next arrives (a parser SAX-style, a response piped to the client). The framework support is wide and quiet (Go's io.Reader, Node's streams, Python's StreamingResponse); the failure mode is a handler that calls .body()/.read() and reads the entire thing into memory. Streaming isn't free — backpressure and partial-failure handling cost thought — but on big bodies it's the difference between a handler that scales and one that dies at 50 concurrent uploads.
- **Identify and optimize critical paths or frequently accessed endpoints for overall system health** — the Pareto move: most requests hit a small set of endpoints, and the hot ones pay for the cold ones. Find the top three endpoints by volume from your access logs (or APM) and treat each as its own optimization project: indices, caching, fewer downstream calls. The users' p99 (the slowest 1% of requests) is dominated by the hot endpoints because they're called the most; you can ignore the cold ones until traffic shifts.

## Decisions about runtime and structure

- **Consider using compiled languages like Go or Rust for performance-critical parts of your backend** — the language item, with a careful "consider" doing real work. The trade isn't universal: Go/Rust beat Python/Ruby on raw CPU by 10-100× on hot loops, but the 5× is usually irrelevant when the dominant cost is IO or DB. The right framing: write everything in the productive language (Python, Ruby, TS) by default, and **rewrite the measured hot components in a faster language when profiling says the language is the bottleneck**. The boundary cases I've seen: image/media processing (move to Rust), data ingestion and parsing (move to Go), the request router/edge of a service (Go). Don't pick the language by performance in advance — let profiling tell you which bytes of code earn the rewrite.
- **Look into different architectural styles (SOA, microservices) and decompose services if required** — the structural cousin to the horizontal-scaling item. Decomposing a monolith into services has performance dimensions: it lets services scale independently (the search service can grow while the auth service stays small), it isolates failure domains (a runaway worker pool can't OOM the request routing layer), and it lets each service pick its own runtime (see the previous item). The trade is real and large: network latency between services, an operations tax (deploy observability for each), and distributed-system fallacies. Right when the monolith's hot endpoints dominate everything else or when one team's release cadence is bottlenecked by another's; rarely the right move before that pain. Same "if required" caveat as sharding in the database section.

## Bound your dependencies on other slow systems

- **Set appropriate connection timeouts and implement efficient retry mechanism to handle network issues** — the item that protects the rest of your system from one slow dependency. Three knobs matter: connect timeout (don't wait 60s on TCP), read timeout (don't wait 30s on a response that's clearly never arriving), and retry budget (do retry; don't retry infinitely). The recurring failure: defaults are huge (the OS connect timeout can be minutes), no retry means a transient blip fails the request, and infinite retry means a slow dependency floods itself with retries during an outage — a retry storm that's worse than the original problem. Use exponential backoff with jitter, cap retries at 2-3, and surface the difference between "dependency is slow" and "request should fail" in your dashboards. Timeouts are the most underused cheap perf win in distributed systems.
- **Batch similar requests together to minimize overhead and reduce the number of round trips** — the round-trip counterpart to keep-alive. Each call to a downstream service or DB has overhead (a network hop, a serialization cost, sometimes a cold cache); N calls of the same kind can usually become one batched call. _WHERE id IN (?)_ instead of _WHERE id = ?_ in a loop, a bulk multidownload instead of N GETs to object storage, a single MGET instead of N GETs to Redis. Same pattern, many specific instances. The rule: if you're calling the same downstream with the same operation more than once per request, batch it.

## The thread

Four pairs, eight items — and the item on top of the stack (profile) is the gate for the other seven. The observation I keep returning to: code optimization has a higher dishonesty rate than every other section of the backend roadmap. "I made it faster" is a claim that needs a flame graph next to it; almost every claimed optimization I've seen that didn't start with a profile either didn't move the metric or made it worse, and the team didn't notice because they didn't measure. The pattern that survives: profile, identify the hot function, ask which of these eight items it points at, fix exactly that, measure again, repeat. Repeat the cycle, not the menu. Eight items but only one or two actually apply to any given system; the rest are noise until they're not.

If I had to compress the eight items into one rule: **profiling is the only universal move; treat the other seven as the catalog you select from.** Pick when profiling points; ignore when it doesn't.

## References

- [1] roadmap.sh, "Backend Performance Best Practices — Code Optimization," roadmap.sh, 2024. [Online]. Available: [https://roadmap.sh/backend-performance-best-practices](https://roadmap.sh/backend-performance-best-practices)
- [2] Google, "Go diagnostics — pprof," Go Documentation, 2024. [Online]. Available: [https://go.dev/doc/diagnostics#profiling](https://go.dev/doc/diagnostics#profiling)
- [3] Brendan Gregg, "The USE Method," Brendan Gregg's site, 2024. [Online]. Available: [https://www.brendangregg.com/usemethod.html](https://www.brendangregg.com/usemethod.html)
- [4] AWS Architecture, "Exponential Backoff," AWS SDK, 2024. [Online]. Available: [https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html](https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html)
- [5] Martin Fowler, "Microservices," martinfowler.com, 2024. [Online]. Available: [https://martinfowler.com/articles/microservices.html](https://martinfowler.com/articles/microservices.html)