JSON Encoding and API Shape

Your endpoint accepts {"name":"hammer","price":9.5} and one client sends {"name":"hammer","price":"9.50"}. Your decoder returns an error mentioning a type, a field name and a Go type name, and now you have to decide what the client is supposed to see. Meanwhile a second client posts an extra field nobody asked for and your handler silently ignores it, so the typo in their integration stays invisible for a month. API shape is the set of decisions you make before either of those happens.

Go's encoding/json will happily do the obvious thing for every field you declare. Everything difficult is in the fields you did not think about: the ones that should be missing, the ones a client may not send, and the ones the client sends that you never declared.

Struct tags decide the wire format, not the Go type

The tag is the contract. json.Marshal reads the tag first and the field name second, so an untagged field goes out with its Go name and a tagged one goes out with yours. Two options carry most of the weight:

type Item struct {
	ID    int64    `json:"id"`
	Name  string   `json:"name"`
	Tags  []string `json:"tags,omitempty"`
	Notes string   `json:"-"`
}

a := Item{ID: 7, Tags: nil, Notes: "internal review note"}
b, _ := json.Marshal(a)
fmt.Println(string(b))

{"id":7,"name":""}

omitempty drops Tags because it is empty, and json:"-" drops Notes unconditionally, which is what you want for anything a client must never see. Name stays, encoded as the empty string, because omitempty means absent-or-zero and a required string has no sensible absent state.

The cost of omitempty is real: the field is now optional on the wire, so every consumer has to handle its absence. Use it for genuinely optional data such as Tags, and leave it off fields you always mean to send. A response that sometimes includes "price" and sometimes does not is harder to consume than one that always sends 0.

A pointer is how you tell absent from zero

Unmarshal loses information that the request contained. {"age":0} and {} both leave an int field at zero, and those mean opposite things for a partial update. A pointer keeps the distinction:

type Patch struct {
	Name *string `json:"name"`
	Age  *int    `json:"age"`
}

for _, in := range []string{`{}`, `{"name":""}`, `{"age":0}`} {
	var p Patch
	json.Unmarshal([]byte(in), &p)
	fmt.Printf("input %-12s name=%v age=%v\n", in, p.Name == nil, p.Age == nil)
}

input {}           name=true age=true
input {"name":""}  name=false age=true
input {"age":0}    name=true age=false

Read those booleans as "the client did not send this field". With values you would have three identical rows. This matters most on an update endpoint, where age=0 means set the age to zero and an absent age means leave it alone.

Null is the third case, and Go folds it into the first. {"age":null} leaves the pointer nil, exactly like an absent field, because there is nowhere else for null to go. If your API needs to distinguish "set this to null" from "do not touch this", you need a type that carries that state explicitly. Most APIs do not, and the honest move is to document the equivalence rather than build a tri-state around it.

Pointers are for request structs, not response structs. On the way out you are choosing values, so encode what you decided and let omitempty handle genuine optionality.

Strict decoding is one helper, and it is usually right

Two defaults of json.Decoder are permissive in ways that hide client bugs. An unknown field is ignored, so a client sending {"nmae":"hammer"} gets a successful create with an empty name. And the decoder reads one value and stops, so a body containing two values decodes the first and silently discards the rest.

func decodeStrict(w http.ResponseWriter, r *http.Request, dst any) error {
	r.Body = http.MaxBytesReader(w, r.Body, maxBody)
	dec := json.NewDecoder(r.Body)
	dec.DisallowUnknownFields()
	if err := dec.Decode(dst); err != nil {
		return err
	}
	var extra json.RawMessage
	if err := dec.Decode(&extra); err != io.EOF {
		return errors.New("body contains more than one JSON value")
	}
	return nil
}

DisallowUnknownFields turns the typo into an error naming the field: json: unknown field "admin". The second Decode is the lesser-known half. It aims at a value that must not be there, and the only acceptable result is io.EOF. Anything else, including a second valid object, is a client that is not sending what your API documents.

MaxBytesReader belongs on the same line as the decoder for a reason beyond request size: without a limit, a client can stream an arbitrarily large body into your decoder and your memory bill follows. It reports the failure as *http.MaxBytesError, which the handler can turn into a 413 rather than a generic 400.

The trade-off is compatibility. A strict decoder means you cannot add a field to a request without clients noticing, and you cannot accept a client that sends a superset. That is a feature for an internal API and a decision to make deliberately for a public one. If you choose leniency, choose it once and write it down, rather than getting strictness by accident.

One thing not to do: validate with the decoder. It knows about types and syntactically valid JSON, and nothing about whether a name may be empty or a price may be negative. Keep those checks in the handler, or in a Validate() method on the request type, so the error you return says "name must be a non-empty string" instead of naming a Go type.

The encoder streams, and the shapes you get back show it

json.Marshal builds a []byte you then write, which means the whole response lives in memory before the first byte reaches the client. json.NewEncoder(w).Encode(v) writes as it walks the value, so a large listing does not need a matching buffer and the client starts receiving bytes sooner. It also appends the newline you want after a JSON body, which Marshal does not.

func writeJSON(w http.ResponseWriter, status int, v any) {
	w.Header().Set("Content-Type", "application/json; charset=utf-8")
	w.WriteHeader(status)
	enc := json.NewEncoder(w)
	enc.SetEscapeHTML(false)
	if err := enc.Encode(v); err != nil {
		log.Printf("encode response: %v", err)
	}
}

Two subtleties in five lines. The content type goes before WriteHeader, because headers leave with the status line. And the encoder's error is logged rather than returned, because by the time Encode fails you have already sent a 200 and there is no status left to change. A half-written body with a 200 is the honest outcome of an encoding failure at that point; the log line is how you find out it happened.

SetEscapeHTML(false) is the one deliberate default override worth knowing about. Go escapes <, > and & in strings by default, which is a safety measure for the days when JSON was pasted into HTML:

default:              {"note":"a \u003c b \u0026 c \u003e d","url":"https://x.test/?a=1\u0026b=2"}
SetEscapeHTML(false): {"note":"a < b & c > d","url":"https://x.test/?a=1&b=2"}

For an API consumed by another service, the escaped form is noise that makes logs and diffs unreadable, and any correct JSON parser decodes both identically. Turning it off is safe when the response is JSON and never interpolated into HTML. Leave it on if the payload might be embedded in a page.

One error shape for every endpoint

A client that has to parse three different error formats will write three parsers and get one of them wrong. Decide the shape once and route every failure through one function:

type errorResponse struct {
	Error struct {
		Code    string `json:"code"`
		Message string `json:"message"`
	} `json:"error"`
}

func writeError(w http.ResponseWriter, status int, code, msg string) {
	var body errorResponse
	body.Error.Code = code
	body.Error.Message = msg
	writeJSON(w, status, body)
}

The nesting is not decoration. A wrapper object means you can add fields later, such as a request id or a list of field errors, without breaking clients that already read error.code. A bare string body has nowhere to grow.

The code is the part clients branch on, so it has to be stable and machine-readable. message is for the human reading a log, and it may change in any release. When both exist, a client that matches on the message text has made a mistake you gave it permission to make, so keep the code list short and documented.

Here is the whole API, including the decode helper, exercised with the failures that matter:

$ curl -i -X POST -d '{"name":"hammer","price":9.5,"tags":["tools"]}' localhost:8081/items
HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Content-Length: 54

{"id":1,"name":"hammer","price":9.5,"tags":["tools"]}

$ curl -X POST -d '{"name":"x","admin":true}' localhost:8081/items
{"error":{"code":"unknown_field","message":"json: unknown field \"admin\""}}

$ curl -X POST -d '{"name":42}' localhost:8081/items
{"error":{"code":"invalid_type","message":"field \"name\" expects string"}}

$ curl -X POST -d '{"price":1}' localhost:8081/items
{"error":{"code":"name_required","message":"name must be a non-empty string"}}

$ curl -X POST -d '' localhost:8081/items
{"error":{"code":"empty_body","message":"request body is empty"}}

$ curl -X POST --data-binary @big.json localhost:8081/items
{"error":{"code":"body_too_large","message":"body exceeds 4096 bytes"}}

Note what the 413 does not say. The handler knows the limit and the client does not, so the message names it. That is the useful half of a strict API: the failure tells the caller how to succeed.

json.Number keeps a 64-bit integer intact

Decode into any, or into a float64, and every number becomes a double. That is fine until an id or a ledger amount exceeds 2^53, at which point the value silently changes:

b := []byte(`{"balance":9007199254740993}`)

var loose struct {
	Balance float64 `json:"balance"`
}
json.Unmarshal(b, &loose)
fmt.Printf("as float64:     %.0f\n", loose.Balance)

var strict struct {
	Balance json.Number `json:"balance"`
}
dec := json.NewDecoder(bytes.NewReader(b))
dec.UseNumber()
dec.Decode(&strict)
fmt.Println("as json.Number: ", strict.Balance.String())

as float64:      9007199254740992
as json.Number:   9007199254740993

The float lost a 1 and reported no error. json.Number is a named string type holding the literal digits, and it converts on demand with Int64(), Float64() or String(), so you decide where the precision is allowed to end. Two habits follow from that.

First, when you decode into any, call dec.UseNumber() before Decode. Without it, any still receives float64 for every number and you have gained nothing. Second, use json.Number on the struct fields that carry identifiers or money, and plain int64 where the range is guaranteed to fit, which is most internal counters.

Encoders are not the problem here. Go writes an int64 as its exact digits, which is why the first line of that transcript shows the correct value. Precision is lost on the way in, and on any client in another language that parses numbers as doubles.

type patchRequest struct {
	Name  *string   `json:"name"`
	Price *float64  `json:"price"`
	Tags  *[]string `json:"tags"`
}

func patchItem(st *store) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
		if err != nil {
			writeError(w, http.StatusBadRequest, "bad_id", "id must be an integer")
			return
		}

		var req patchRequest
		if err := decodeStrict(w, r, &req); err != nil {
			writeError(w, http.StatusBadRequest, "invalid_json", err.Error())
			return
		}

		if req.Name != nil && *req.Name == "" {
			writeError(w, http.StatusUnprocessableEntity, "name_required",
				"name must be a non-empty string")
			return
		}
		if req.Price != nil && *req.Price < 0 {
			writeError(w, http.StatusUnprocessableEntity, "price_negative",
				"price must not be negative")
			return
		}

		item, ok := st.update(id, func(it *Item) {
			if req.Name != nil {
				it.Name = *req.Name
			}
			if req.Price != nil {
				it.Price = *req.Price
			}
			if req.Tags != nil {
				it.Tags = *req.Tags
			}
		})
		if !ok {
			writeError(w, http.StatusNotFound, "not_found", "no item with that id")
			return
		}
		writeJSON(w, http.StatusOK, item)
	}
}