Skip to content

15. Final project

Put together everything you have seen in Section 1 in a program you could actually use:

  • download data in streaming mode (ch. 11);
  • validate every record with a typed model (ch. 7–8), discarding the broken ones;
  • aggregate (counts, averages, top-N) with the language’s data structures;
  • write a readable JSON report (ch. 4–6);
  • verify the result against an independent source (/api/v1/stats).

From the Aurora Bookshop catalog, produce report.json with this shape:

{
"books": 80,
"avg_price": 19.9,
"by_category": [ { "category": "saggistica", "books": 14, "avg_price": 17.04 }, … ],
"top_authors": [ { "name": "Matteo Rinaldi", "books": 8 }, … ],
"formats": { "tascabile": 30, "brossura": 26, "rilegato": 24 }
}

Source: GET /api/v1/export/books.ndjson (one book per line, with author_name and category already resolved by the server). Check: GET /api/v1/stats, which computes the same things in SQL.

  1. Download the export in streaming mode and, line by line, validate it with a model (Libro). Count the discarded lines.
  2. Accumulate the aggregates while you read (do not keep all the books in memory, even if here there are only 80: it is the habit that counts).
  3. Build the report and write it to a file with indentation and UTF-8.
  4. Download /stats and compare the average price and books per category: they must match.

The two calls the program will make, seen “bare”:

terminal
curl -s https://corso-api-json.giuseppebosi.com/api/v1/export/books.ndjson | wc -l
curl -s https://corso-api-json.giuseppebosi.com/api/v1/stats | jq '{prices, cat: .books_by_category[0:2], top: .top_authors[0:2]}'
| wc -l
counts the lines of the export: 80, that is the records, without parsing
jq '{prices, cat: .books_by_category[0:2], top: .top_authors[0:2]}'
extracts the three pieces the report must reproduce; [0:2] is a slice: the first two elements
80
{
"prices": { "min": 7.13, "avg": 19.9, "max": 34.49 },
"cat": [ { "slug": "saggistica", "name": "Saggistica", "books": 14, "avg_price": 17.04 }, { "slug": "ragazzi", "name": "Ragazzi", "books": 12, "avg_price": 16.72 } ],
"top": [ { "id": 18, "name": "Matteo Rinaldi", "books": 8, "avg_rating": 4.22 }, { "id": 1, "name": "Elena Marchetti", "books": 6, "avg_rating": 3.12 } ]
}
"""Progetto finale: da NDJSON a report. pip install httpx pydantic · python progetto.py [report.json]"""
import json, sys
from collections import Counter, defaultdict
import httpx
from pydantic import BaseModel, Field, ValidationError
BASE = "https://corso-api-json.giuseppebosi.com"
class Libro(BaseModel): # il contratto: ciò che DEVE esserci per contare il libro
id: int
title: str = Field(min_length=1)
price: float = Field(ge=0)
author_name: str
category: str
in_stock: bool
attributes: dict
def scarica() -> list[Libro]:
libri, scartati = [], 0
with httpx.stream("GET", f"{BASE}/api/v1/export/books.ndjson", timeout=30) as r:
r.raise_for_status()
for riga in r.iter_lines():
if not riga:
continue
try:
libri.append(Libro.model_validate_json(riga)) # parsing + validazione in un colpo
except ValidationError as e:
scartati += 1
print("riga scartata:", e.errors()[0]["msg"], file=sys.stderr)
print(f"letti {len(libri)} libri, scartati {scartati}", file=sys.stderr)
return libri
def report(libri: list[Libro]) -> dict:
per_cat = defaultdict(list)
for l in libri:
per_cat[l.category].append(l.price)
autori = Counter(l.author_name for l in libri)
return {
"books": len(libri),
"avg_price": round(sum(l.price for l in libri) / len(libri), 2),
"by_category": sorted(
[{"category": c, "books": len(p), "avg_price": round(sum(p) / len(p), 2)} for c, p in per_cat.items()],
key=lambda x: -x["books"]),
"top_authors": [{"name": n, "books": k} for n, k in autori.most_common(3)],
"formats": dict(Counter(l.attributes.get("format", "n/d") for l in libri)),
}
def confronta(mio: dict) -> None:
ufficiale = httpx.get(f"{BASE}/api/v1/stats").json()
print("prezzo medio: mio", mio["avg_price"], "| server", ufficiale["prices"]["avg"], file=sys.stderr)
miei = {c["category"]: c["books"] for c in mio["by_category"]}
loro = {c["slug"]: c["books"] for c in ufficiale["books_by_category"]}
print("categorie uguali:", miei == loro, file=sys.stderr)
if __name__ == "__main__":
libri = scarica()
r = report(libri)
confronta(r)
destinazione = sys.argv[1] if len(sys.argv) > 1 else "report.json"
with open(destinazione, "w", encoding="utf-8") as f:
json.dump(r, f, indent=2, ensure_ascii=False)
print("scritto", destinazione, file=sys.stderr)

Output on stderr (the report goes to a file, so python progetto.py > altro does not mix them):

letti 80 libri, scartati 0
prezzo medio: mio 19.9 | server 19.9
categorie uguali: True
scritto report.json

Choices worth noticing: model_validate_json parses and validates in one go (faster than json.loads + model_validate); the libri list is kept only for clarity — for a huge export you would pass the data to report() as you read; ensure_ascii=False keeps accented letters readable in the file.

GET/api/v1/stats— the official numbers to compare your report with

Try it online, nothing to install:

Open the notebook in Google Colab Open the TypeScript project in StackBlitz

The GET request straight in your browser: https://corso-api-json.giuseppebosi.com/api/v1/stats

Python on Google Colab (a free Google account is needed to run the cells); TypeScript on StackBlitz (it opens and starts by itself in the browser). If the practice API is switched off, turn it on from /en/api-attiva/.

  1. Robustness: simulate a corrupted line (edit a local copy of the export: remove a comma, put "price": "n/d") and check that the program discards it without stopping.
  2. Top authors with ties: the server sorts authors with the same number of books by name. Do the same in your report and compare top_authors too.
  3. A second report: ordini.json from GET /api/v1/orders?limit=100 (paginated: follow links.next until it is null) with revenue by status and by city (shipping_address.city), to compare with /api/v1/stats (orders_by_status) and /api/v1/stats/sales?group_by=city.
Hints
  1. Save the export with curl -o libri.ndjson, edit it, and have the program read the file (open("libri.ndjson") / createReadStream): the rest of the code does not change — that is the benefit of having separated reading from validation.
  2. Python: sorted(autori.items(), key=lambda x: (-x[1], x[0]))[:3]. TS: .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])).
  3. A loop while url: r = get(url); …; url = r["links"]["next"]. Revenue by status: the sum of total grouped by status, excluding cancelled if you want to match /stats/sales.
  • Validating after aggregating: a broken line ends up in the totals before you notice. Validate at the boundary, right away.
  • Rounding halfway through: round only in the final report; use full numbers in the intermediate sums.
  • Printing the report on stdout together with the logs: use stderr for messages and a file/stdout for data.
  • sum(p) / len(p) with an empty p: division by zero; it does not happen here, but in a real dataset it does.

Download in streaming mode, validate at the boundary, aggregate while you read, write readable UTF-8, verify against an independent source. Next step: Section 2, chapter 1 — HTTP and REST.