17 — The Standard Library — Doing More With Less

I expected a standard library to be a floor, and Go's turned out to be a ceiling most projects never need to leave. The model that clicked: Go ships an unusually complete standard library, built around small composable interfaces (io.Reader, io.Writer), so that a production HTTP service with JSON, file I/O, time handling, structured logging, and regex can be written with zero external dependencies. [1] The culture that flows from this — "use the stdlib first, reach for a third-party package only when the stdlib genuinely cannot do it" — is what keeps Go dependency trees small and code portable. Once I learned the shape of the library, I stopped reflexively searching for a package and started checking pkg.go.dev first.

io — the interface at the center

Everything in Go's I/O revolves around two tiny interfaces [2]:

type Reader interface { Read(p []byte) (n int, err error) }
type Writer interface { Write(p []byte) (n int, err error) }

A file is a Reader and a Writer. A network connection is too. An in-memory buffer, a gzip stream, a crypto cipher — all of them implement io.Reader or io.Writer. Because the interfaces are so small, anything that reads bytes can read from anywhere, and anything that writes bytes can write to anywhere. io.Copy(dst, src) works whether dst is a file, a network socket, or a buffer, and whether src is any of those. This uniformity is the single design decision that makes the whole library compose [2]. Higher-level helpers like io.ReadAll, io.MultiReader, and io.Pipe build on the same two interfaces.

os and bufio — files and buffered I/O

The os package wraps operating-system facilities: file operations, environment variables, process control, command-line arguments [3]. Opening a file is os.Open("path"), reading it is io.ReadAll, environment is os.Getenv("KEY"). The bufio package wraps io.Reader/io.Writer with buffering to reduce system calls, and ships bufio.Scanner — the idiomatic line-by-line reader [4]:

f, _ := os.Open("log.txt")
defer f.Close()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
    line := scanner.Text()
    // process line
}

bufio.Scanner handles the chunking and newline splitting for you; the bare Read loop is something you almost never write.

time — durations, layouts, and tickers

time handles instants, durations, parsing, formatting, timers, and timezones [5]. Two quirks catch newcomers. First, Go formats times by _reference layout_, not format codes: instead of YYYY-MM-DD, you write "2006-01-02" — a specific reference time whose components map to the fields. Second, durations are time.Duration (an int64 nanoseconds), so you write 5 * time.Second, not a bare 5. Timers (time.After, time.NewTimer) and tickers (time.Tick) integrate directly with select, which is how timeouts appear in concurrent code.

encoding/json — marshal and unmarshal

The JSON package converts between Go structs and JSON via two functions [6]:

type User struct {
    Name  string `json:"name"`
    Email string `json:"email,omitempty"`
}

u := User{Name: "Ave"}
data, _   := json.Marshal(u)          // struct → []byte
var u2 User
json.Unmarshal(data, &u2)             // []byte → struct

Struct tags (covered in the structs notes) control field names and omission. The streaming variants json.NewEncoder(w).Encode(v) and json.NewDecoder(r).Decode(&v) are preferred for HTTP handlers and large payloads — they write/read directly without buffering the whole document in memory. Decoding into map[string]any works for JSON of unknown shape, at the cost of losing the type system.

log/slog — structured logging

Go 1.21 added log/slog, the standard library's structured logger [7]. It emits key-value pairs (typically as JSON) instead of free-form strings, which makes logs machine-parseable and queryable in production:

import "log/slog"

slog.Info("user logged in", "user_id", 42, "ip", "10.0.0.1")
// {"time":"...","level":"INFO","msg":"user logged in","user_id":42,"ip":"10.0.0.1"}

slog supports leveled logging (Debug/Info/Warn/Error), context integration (so a trace ID in the context flows into logs), and pluggable handlers. It replaces ad-hoc log.Printf for any production service and removes the need for a third-party logger in most cases — though high-throughput services still reach for zap or zerolog (covered in the data-and-logging notes) for their lower allocation cost.

regexp and flag

regexp provides RE2-syntax regular expressions — safe, because RE2 avoids catastrophic backtracking (the slow, exponentially growing search that can hang other regex engines) and always finishes in time proportional to the input — and patterns are compiled for reuse [8]. The pattern is re := regexp.MustCompile(\patten\); re.MatchString(s) — compile once, reuse, because compiling on every call is wasteful.

flag parses command-line arguments for CLIs that do not need a full framework [9]:

port := flag.Int("port", 8080, "server port")
flag.Parse()

It auto-generates -h help. For anything beyond simple flags — subcommands, rich validation — you reach for cobra or urfave/cli (covered in the CLI notes).

net/http and go:embed

The library's crown jewel is net/http — a complete HTTP client and server in the standard library, with HTTP/2, TLS, and cookies built in [1]. A basic server is two lines: http.HandleFunc("/", handler); http.ListenAndServe(":8080", nil). This is why Go's web frameworks are thin wrappers rather than necessities — the standard library already does the heavy lifting.

go:embed is the modern tool for bundling static files into a binary [10]. A //go:embed directive compiles files, SQL migrations, HTML templates, or whole directories into the executable at build time, producing a single self-contained binary with no external file dependencies — exactly the deployment story Go was designed for.

How I use this

The reflex I built from the standard library is "check the stdlib first." For HTTP, JSON, files, time, logging, regex, and CLI flags, the built-in packages are production-ready and stay forever — no churn from dependencies going unmaintained. I reach for external packages for three things only: a web framework when I want routing/middleware ergonomics (gin/echo), a database driver (pgx), and a higher-performance logger when profiling shows slog allocating too much. Everything else comes from the standard library. The io.Reader/io.Writer idea is the key that unlocks the rest — once every stream looks the same, composition becomes obvious, and io.Copy, json.NewDecoder, and bufio.Scanner all click together.

References

[1] A. Joshi, "Building Robust APIs with Go's Standard Library," dev.to, 2024. [Online]. Available: https://dev.to/aaravjoshi/building-robust-apis-with-gos-standard-library-a-comprehensive-guide-3036

[2] A. Joshi, "Building High-Performance File Processing Pipelines in Go," dev.to, 2024. [Online]. Available: https://dev.to/aaravjoshi/building-high-performance-file-processing-pipelines-in-go-a-complete-guide-3opm

[3] Reintech, "An Overview of Go's os and io Packages," 2024. [Online]. Available: https://reintech.io/blog/an-overview-of-gos-os-and-io-packages

[4] B. Musyoka, "Go Fast with bufio: Unlocking the Power of Buffered I/O," Medium, 2024. [Online]. Available: https://medium.com/@emusbeny/mastering-bufio-in-go-the-art-of-buffered-i-o-17cae584ee4b

[5] The Go Authors, "time package," pkg.go.dev, 2024. [Online]. Available: https://pkg.go.dev/time

[6] The Go Authors, "encoding/json package," pkg.go.dev, 2024. [Online]. Available: https://pkg.go.dev/encoding/json

[7] The Go Authors, "Structured Logging with slog," The Go Blog, 2023. [Online]. Available: https://go.dev/blog/slog

[8] The Go Authors, "regexp package," pkg.go.dev, 2024. [Online]. Available: https://pkg.go.dev/regexp

[9] DigitalOcean, "How To Use the Flag Package in Go," 2024. [Online]. Available: https://www.digitalocean.com/community/tutorials/how-to-use-the-flag-package-in-go

[10] The Go Authors, "embed package," pkg.go.dev, 2024. [Online]. Available: https://pkg.go.dev/embed