10. Nested JSON and navigation
- Read values three or four levels deep without
KeyError/TypeError. - Use jq from the terminal to explore, filter and transform JSON.
- Know JSONPath to express paths declaratively.
- Understand how an API produces nested JSON by “following” foreign keys (
?expand=).
Theory
Section titled “Theory”Real JSON is never flat. Order 1 with its customer and the books of its lines is a tree four levels deep: order → lines → book → author. To reach the author name of the first book the path is items[0].book.author_name.
Three ways to walk a tree:
- Direct access (
o["items"][0]["book"]["author_name"]): fast, but if a piece is missing the program stops with an exception. - Safe access:
.get()with a default in Python,?.(optional chaining) and??in TypeScript. It returnsNone/undefinedinstead of blowing up. - A path language: JSONPath (
$.items[*].book.title) or jq (.items[].book.title). Ideal for exploring unfamiliar data or for terminal scripts.
How does nested JSON come about? The database has separate tables linked by foreign keys (orders.customer_id → customers.id, order_items.book_id → books.id). The server, if you ask with ?expand=, replaces the id with the whole object. Without expand you only get customer_id: 39 and need a second request: you will see this in detail in Section 2, chapter 5.
curl, explained
Section titled “curl, explained”curl -s 'https://corso-api-json.giuseppebosi.com/api/v1/orders/1?expand=items.book,customer' | jq .- '…?expand=items.book,customer'
- the URL is inside single quotes because it contains
?and,, which the shell might interpret - ?expand=
- start of the query string:
key=valueparameters after the path - items.book
- expand the lines (
items) and, inside each line, the book referenced bybook_id - ,customer
- several values for the same parameter, separated by commas: expand the customer too
- | jq .
.is the "identity" jq program: it prints the JSON as it is, but indented and colored
{ "id": 1, "customer_id": 39, "status": "delivered", "shipping_address": { "street": "Via Sestri 73", "city": "Bari", "zip": "15399", "country": "IT" }, "total": 12.04, "created_at": "2025-01-10T21:00:00Z", "customer": { "id": 39, "name": "Giovanni Costa", "email": "giovanni.costa39@example.com", "city": "Bari", "created_at": "2025-04-27T00:00:00Z" }, "items": [ { "id": 1, "order_id": 1, "book_id": 11, "quantity": 1, "unit_price": 12.04, "book": { "id": 11, "title": "Quello che resta di Montecarvo", "price": 12.04, "author_id": 14, "author_name": "Pietro Galli" } } ]}jq in eight moves
Section titled “jq in eight moves”We work on the list of books (/api/v1/books?limit=100, an object with items and meta).
URL='https://corso-api-json.giuseppebosi.com/api/v1/books?limit=100'
# 1. proiezione: da ogni libro tieni solo due campicurl -s "$URL" | jq '.items[] | {title, price}'
# 2. filtro + conteggiocurl -s "$URL" | jq '[.items[] | select(.price < 8)] | length'
# 3. ordina, prendi i primi 3, stampa testo grezzo (-r) con tabcurl -s "$URL" | jq -r '.items | sort_by(-.price) | .[0:3][] | "\(.price)\t\(.title)"'
# 4. variabile dalla shell con --argcurl -s "$URL" | jq --arg f tascabile '[.items[] | select(.attributes.format == $f) | .title] | length'
# 5. appiattisci array annidati e raggruppacurl -s "$URL" | jq '[.items[].attributes.awards[]] | group_by(.) | map({premio: .[0], libri: length})'
# 6. accesso sicuro: una chiave che non esiste vale null, non errorecurl -s 'https://corso-api-json.giuseppebosi.com/api/v1/books/1' | jq '.attributes.dimensions_cm.h, .attributes.missing'
# 7. aggregazione numericacurl -s "$URL" | jq '.items | map(.price) | add / length | . * 100 | round / 100'- URL='…'
- a shell variable; then
"$URL"in double quotes, so it gets expanded but?and&stay protected - .items[]
- goes into
itemsand "unpacks" the array: from here on jq works on one element at a time - {title, price}
- builds an object with those two keys (shorthand for
{title: .title, price: .price}) - [ … ]
- collects the elements back into an array (needed for
length,sort_by…) - select(cond)
- only lets through the elements for which the condition is true
- -r
- raw: strings come out without quotes (useful for tables and scripts)
- "\(.price)\t\(.title)"
- interpolation:
\( )inserts an expression into a string;\tis a tab - --arg f tascabile
- passes a value from the shell as the variable
$f(a string); for numbers/JSON use--argjson - .attributes.awards[]
[]on an array unpacks it: the awards of all the books end up in a single stream- group_by(.) | map(…)
- groups equal values, then turns each group into an object
- .a, .b
- the comma produces several outputs: one per expression
- add / length
- sum of the elements divided by their count = average
Output (one per command):
{ "title": "Il silenzio di Ustica", "price": 18.71 } … (80 oggetti)334.49 Quello che resta di Sestri34.36 Ustica e il Novecento33.49 La cucina di Ustica30[ { "premio": "Bancarella", "libri": 10 }, { "premio": "Campiello", "libri": 8 }, { "premio": "Strega", "libri": 14 }, { "premio": "Viareggio", "libri": 16 } ]19null19.9Python and TypeScript
Section titled “Python and TypeScript”# pip install httpx jsonpath-ngimport httpxfrom jsonpath_ng.ext import parse
BASE = "https://corso-api-json.giuseppebosi.com/api/v1"o = httpx.get(f"{BASE}/orders/1", params={"expand": "items.book,customer"}).json()
# accesso diretto: chiaro, ma KeyError se manca un pezzoprint(o["items"][0]["book"]["author_name"]) # Pietro Galli
# accesso sicuro: .get() a catena con defaultprint(o.get("customer", {}).get("city")) # Bariprint(o.get("customer", {}).get("phone", "n/d")) # n/d (campo inesistente)
# JSONPath: percorsi dichiarativi, anche con wildcard e filtriprint([m.value for m in parse("$.items[*].book.title").find(o)])print([m.value for m in parse("$.items[?(@.quantity>=1)].book.author_name").find(o)])
lista = httpx.get(f"{BASE}/books", params={"limit": 100}).json()economici = [m.value for m in parse("$.items[?(@.price<8)].title").find(lista)]print(len(economici)) # 3What you should see:
Pietro GalliBarin/d['Quello che resta di Montecarvo']['Pietro Galli']3Notes: jsonpath_ng.ext (not jsonpath_ng) is needed for [?(...)] filters. For a fixed path where you need no flexibility, chained .get() calls are more readable and faster.
// node --experimental-strip-types naviga.tsconst BASE = "https://corso-api-json.giuseppebosi.com/api/v1";const o = await (await fetch(`${BASE}/orders/1?expand=items.book,customer`)).json();
// accesso diretto: TypeError se un pezzo è undefinedconsole.log(o.items[0].book.author_name); // Pietro Galli
// optional chaining ?. e nullish coalescing ??console.log(o.customer?.city); // Bariconsole.log(o.customer?.phone ?? "n/d"); // n/dconsole.log(o.items?.[5]?.book?.title ?? "nessuna sesta riga");
// percorso da stringa, senza librerie: una funzione di 5 righefunction leggi(obj: unknown, percorso: string, fallback?: unknown) { return percorso.split(".").reduce<any>((acc, k) => (acc == null ? undefined : acc[k]), obj) ?? fallback;}console.log(leggi(o, "shipping_address.zip")); // 15399console.log(leggi(o, "items.0.book.price")); // 12.04console.log(leggi(o, "items.9.book.price", 0)); // 0
// "jq in TypeScript": map/filter/sort sui daticonst lista = await (await fetch(`${BASE}/books?limit=100`)).json();const top3 = [...lista.items].sort((a, b) => b.price - a.price).slice(0, 3).map((b) => `${b.price}\t${b.title}`);console.log(top3.join("\n"));What you should see:
Pietro GalliBarin/dnessuna sesta riga1539912.04034.49 Quello che resta di Sestri34.36 Ustica e il Novecento33.49 La cucina di UsticaFor real JSONPath in TS there is jsonpath-plus (npm install jsonpath-plus): JSONPath({ path: "$.items[*].book.title", json: o }).
Try it now
Section titled “Try it now”Response headers
Response headers
Exercise
Section titled “Exercise”Try it online, nothing to install:
The GET request straight in your browser: https://corso-api-json.giuseppebosi.com/api/v1/orders/1?expand=items.book,customer
Python on Google Colab (a free Google account is needed to run the cells); TypeScript on StackBlitz (it opens and starts by itself in the browser). If the practice API is switched off, turn it on from /en/api-attiva/.
With jq (or in Python/TS), starting from /api/v1/categories?tree=true:
- Print the names of the root categories only.
- Print, for each root, the names of its subcategories.
- Add up
books_countover all categories, at any level (jq hint:..recursively visits every node).
Solution
URL='https://corso-api-json.giuseppebosi.com/api/v1/categories?tree=true'curl -s "$URL" | jq -r '.items[].name' # 1curl -s "$URL" | jq -r '.items[] | "\(.name): \([.children[].name] | join(", "))"' # 2curl -s "$URL" | jq '[.. | .books_count? | numbers] | add' # 3 → 80.. produces every value in the tree; .books_count? tries to read the key, ignoring errors on non-objects; numbers discards the nulls.
Common mistakes
Section titled “Common mistakes”o["items"][0]on an empty list →IndexError/undefined: check the length or use?.[0].- Confusing
nullwith “missing key”: in jq and JS both arrive asnull/undefined; if the difference matters, usehas("k")(jq) /"k" in obj(JS) /"k" in d(Python). - Forgetting the quotes around a URL with
?and&: the shell silently splits the command. jq '.items[].price | add'does not work:addwants an array, so[.items[].price] | addor.items | map(.price) | add.
Summary
Section titled “Summary”Direct access when the shape is guaranteed, safe access (.get, ?.) when something may be missing, jq/JSONPath to explore. And remember: nesting comes from the relations between tables, which the API expands for you. Next chapter: when the JSON is too big to fit in memory.