HTTP services
Handlers and the Standard Library Router
http.Handler, HandlerFunc, the method and wildcard patterns Go 1.22 added to ServeMux, why handler state belongs on a struct, and testing routes with httptest.
Your first HTTP service works. Then you add a second route, and the handler needs the database handle, and the obvious move is a package-level variable. It compiles, it serves traffic, and six weeks later two tests interfere with each other and the race detector prints a wall of stacks on a map you forgot was shared. The routing API is not where that bug comes from. The shape of the handler is.
Go's net/http is small enough to hold in your head once you accept one
idea: a handler is a value with a method on it. Everything else in this lesson follows from
that, including why you can test a route without opening a socket and why the standard
library router is enough for most services.
A handler is an interface with exactly one method
http.Handler is this, in full:
type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}
No lifecycle, no registration hook, no context argument, because the request already
carries one. A type satisfies the interface by having that method, which means anything can
be a handler: a struct, a function type, a closure. The Server type you will
write below is a handler. So is a *http.ServeMux.
func (s *Server) routes() *http.ServeMux {
mux := http.NewServeMux()
mux.HandleFunc("GET /items", s.listItems)
mux.HandleFunc("POST /items", s.createItem)
mux.HandleFunc("GET /items/{id}", s.getItem)
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "ok")
})
return mux
}
http.HandlerFunc is the adapter that makes the function form work. It is a
function type with a ServeHTTP method, so a plain function can be passed
anywhere a handler is expected:
type HandlerFunc func(ResponseWriter, *Request)
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
f(w, r)
}
That is the whole trick, and it is worth internalising because you will write the same adapter for your own types. A type whose only difference from another is the method it needs gets a named function type and a one-line method.
The other half of the interface is http.ResponseWriter. Two of its
behaviours decide how your service behaves under load. WriteHeader sends the
status line and headers and can only take effect once; a second call logs a warning and is
ignored. And the first Write without an explicit
WriteHeader sends 200, which is how a handler that forgets to set a status
reports success for a validation failure. Set the content type before the status, always,
because both leave in the same byte stream.
Go 1.22 gave ServeMux methods and wildcards
For years http.ServeMux matched a path prefix and nothing else, which is
why a generation of Go services pulled in a router. Since Go 1.22 a pattern can name a
method, a host and wildcard segments, so the routing work that used to justify the
dependency is in the standard library.
mux.HandleFunc("GET /items/{id}", s.getItem)
func (s *Server) getItem(w http.ResponseWriter, r *http.Request) {
id, err := strconv.Atoi(r.PathValue("id"))
// ...
}
The pattern grammar is [METHOD ][HOST]/[PATH], all three parts optional.
Some details that trip people up on the first day:
| Pattern | What it matches |
|---|---|
GET /items/{id} | One path segment. {id} is read with r.PathValue("id"). /items/ does not match, so the empty value never reaches your handler. |
GET /files/{path...} | The rest of the path, slashes included. The ... wildcard must be last. |
GET /{$} | The root path exactly. Plain / matches everything, which is usually not what you want on the root route. |
GET /items | GET and HEAD requests. Other methods get 405 with an Allow header filled in for you. |
Two patterns that overlap are resolved by specificity, not registration order, and two patterns that are ambiguous panic at registration rather than picking one. That panic happens at startup, which is the right time to find out.
Run this service and the router's own behaviour shows up in the responses:
$ curl -i http://127.0.0.1:8080/items
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 52
[{"id":1,"name":"hammer"},{"id":2,"name":"chisel"}]
$ curl -i -X POST -d '{"name":"level"}' http://127.0.0.1:8080/items
HTTP/1.1 201 Created
Content-Type: application/json
Content-Length: 24
{"id":3,"name":"level"}
$ curl -i -X DELETE http://127.0.0.1:8080/items/1
HTTP/1.1 405 Method Not Allowed
Allow: GET, HEAD
Content-Type: text/plain; charset=utf-8
Method Not Allowed
$ curl -i http://127.0.0.1:8080/items/99
HTTP/1.1 404 Not Found
Content-Type: application/json
Content-Length: 23
{"error":"no item 99"}
The 405 in that transcript cost no code. The mux knew which methods
/items/1 accepts and said so. Note also that GET patterns answer HEAD, which
the Allow header reports honestly.
Package-level handler state is a bug with a delay
Handlers run concurrently, one goroutine per request. Anything a handler touches that lives outside the request is shared, and shared mutable state needs a lock or a different design. A package-level map has neither:
var seen = map[string]int{}
func bump(w http.ResponseWriter, r *http.Request) {
seen[r.URL.Path]++
fmt.Fprintf(w, "%s: %d\n", r.URL.Path, seen[r.URL.Path])
}
$ GORACE=halt_on_error=1 go run -race ./l10race
==================
WARNING: DATA RACE
Read at 0x00c000100b40 by goroutine 11:
runtime.mapassign_fast64ptr()
main.bump()
/tmp/go-m4-lab/l10race/main.go:14 +0x90
Previous write at 0x00c000100b40 by goroutine 36:
runtime.mapaccess2_faststr()
main.bump()
/tmp/go-m4-lab/l10race/main.go:14 +0xcc
==================
exit status 66
The same program without -race usually prints a plausible number and exits
zero. That is the delay: a counter that is slightly wrong for a month, and a corrupted map
the first time two writes land together.
The fix is not a mutex around the global. It is to stop having the global. State that belongs to the service becomes a field on a struct, and the struct gets passed to the handler that needs it:
type Server struct {
store *Store
}
type Store struct {
mu sync.Mutex
items map[int]Item
next int
}
This is the decision that makes the rest of the module easy. A test builds a fresh
Server with a fresh Store, so nothing carries between tests and
nothing is shared between requests except what you deliberately share. A handler that needs
a logger, a clock, a config value or a database takes it from the struct. When you catch
yourself reaching for a package variable inside a handler, that variable is a field you have
not added yet.
httptest tests the route, not the network
A handler is a function from a request to a response, and httptest gives
you both ends of that without a listener. httptest.NewRequest builds a request
that never left the process; httptest.NewRecorder captures the status, headers
and body.
func newTestServer(t *testing.T) http.Handler {
t.Helper()
srv := &Server{store: NewStore()}
srv.store.Add("hammer")
return srv.routes()
}
func TestGetItem(t *testing.T) {
h := newTestServer(t)
req := httptest.NewRequest(http.MethodGet, "/items/1", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
var item Item
if err := json.Unmarshal(rec.Body.Bytes(), &item); err != nil {
t.Fatalf("decode body %q: %v", rec.Body.String(), err)
}
if item.Name != "hammer" {
t.Errorf("item.Name = %q, want hammer", item.Name)
}
}
$ go test ./l10full/ -v
=== RUN TestGetItem
--- PASS: TestGetItem (0.00s)
=== RUN TestMethodMismatchIsRejectedByTheMux
--- PASS: TestMethodMismatchIsRejectedByTheMux (0.00s)
=== RUN TestCreateItemWithGarbageBody
--- PASS: TestCreateItemWithGarbageBody (0.00s)
=== RUN TestAgainstARealServer
--- PASS: TestAgainstARealServer (0.00s)
PASS
ok go-m4-lab/l10full 0.335s
Calling h.ServeHTTP directly keeps the test fast, and it is what you want
most of the time. httptest.NewServer starts a real listener on a random port
and returns its URL, which is the right tool when the thing under test is a client, when you
need a real Host header, or when you are not sure whether the failure is in
your handler or in how the server feeds it. Those two tests are the same handler; only the
transport differs.
What httptest cannot tell you: whether the handler is race free
(go test -race does that), whether it leaks goroutines, and whether it behaves
under a real client that disconnects halfway through the response.
A third-party router buys you middleware, not matching
Matching was the historical reason to install chi,
gorilla/mux or httprouter. That reason is gone. What those
libraries still offer is a middleware composition API and route groups, and whether that is
worth a dependency is a judgement about your service rather than about Go.
Stay with ServeMux when your routes are a handful of resource paths, when
you want zero dependencies for a service that must build offline, and when the middleware
chain is short enough to write as nested functions. The standard library also gives you
something the alternatives cannot: r.PathValue is part of
*http.Request, so a handler has no router type in its signature and stays
testable with httptest.NewRequest alone.
Reach for a router when you need middleware registered per route group with shared
prefixes, when you are porting a large route table and want it declarative, or when you need
something the standard mux does not model at all, such as host-and-subdomain routing with
per-group middleware in one place. Write the three middleware functions you actually need
before you decide; most services that adopt a router would have been fine with a twenty-line
Chain helper.
Build a service with three routes and no package-level state: GET
/items, POST /items and GET /items/{id}. Store the
items in a map guarded by a mutex, behind a struct that the handlers reach through their
receiver.
Seed it with two items, run it, and use curl -i to confirm each of
these: a listing, a create that returns 201, a request for a missing id that returns
404, and a DELETE that returns 405 with an Allow header.
Then write four tests with httptest: the happy path for one item, the
405, a POST with a body that is not JSON, and a check that
Content-Type is application/json. Run the suite twice and
confirm the second run does not see the first run's items.
Finally, move the store to a package-level variable and run go test
-race to see what changes.
Show the solution and the reasoning
The full server is in the transcript above; the parts worth explaining are the choices.
The store holds the lock rather than the handler, because the map is the shared thing.
Every method takes the lock for the whole read or write and returns a copy of the item, not
a pointer into the map. Returning &it from a map would hand the caller a
reference that later writes mutate, which is the same class of bug as the global, one
level down.
routes() returns a *http.ServeMux rather than registering
on http.DefaultServeMux. The package-level default is a global with all the
properties you just spent a section avoiding: two tests that register the same pattern
panic, and import side effects can register handlers you did not write.
Building the mux in a method also means a test gets a complete route table from
srv.routes() without touching any package state.
For the 405 test, the Allow header comes from the mux, so the assertion
is really a check that your pattern names the method. The exact value is
GET, HEAD, because a GET pattern answers HEAD as well. If you assert only the
status code, the test passes for a service that would answer the wrong thing to a
HEAD.
Moving the store back to a package variable is the instructive failure. Depending on
how your tests are written, either the second run of the suite sees items from the first
(len(items) is now four, not two) or -race fires when two tests
run in parallel. Both are the same root cause, and both disappear when the state goes back
on the struct. The detector is the better teacher here because it names the line.
Official reference
The Go standard library documentation on pkg.go.dev for what this lesson uses.
Progress
Mark this lesson done and it stays marked in this browser.