The depth rewrite

Turning a corpus of accurate-but-shallow reference notes into a knowledge base worth reading. This file is the work order: it holds the roadmap, the decisions, and nothing a tool can compute.

Standing instruction

Work the roadmap below without asking. Take the next unchecked entry, do it properly, verify, commit, tick the box, and move to the next one. Do not stop to confirm which note to work on, and do not wait for a prompt between Articles - the choice is already made here.

Stop only when: the wave is finished, a check fails in a way you cannot fix, or the roadmap’s premise turns out to be wrong for a note.

bash

python tools/tiers.py --audit         # must say every Article meets its tier
python tools/check_links.py           # must pass
python tools/check_style.py --gate    # must be 0
python tools/progress.py --write      # regenerate PROGRESS.md, then commit it

PROGRESS.md is the report of what is done; this file is the work order for what is next.

Then the site build and node scripts/verify.mjs, a one-line commit, and push. Never deploy unless the user asks in that message.

The bar

Eight obligations, in AUTHORING.md under Tiers. The two that carry the most weight, and are most often skipped:

Show, don’t assert. If the claim is that something breaks, the repro and its output are in the note. Half the Articles so far found a real bug or a wrong claim in the note they replaced - that is the standard, not a bonus.
A mechanism section. What the system does underneath, not the API surface.

An Article is 250-400 lines, four worked code blocks, a production-failure section, follow-up chains in the Interview angle, and the trade you did not take. tiers.py --audit enforces the mechanical half.

The queue is a hypothesis

Read the note before trusting this file’s description of it. Twice the roadmap has been wrong: 02_indexes.md + 11_index_types.md looked like a merge and were a bad split; 24_functions_triggers_and_views.md looked like a trim and needed deleting. When the premise fails, do the right thing and record what changed here. That is the job, not a deviation.

If a note is legitimately Reference - a product survey, a comparison table, an API listing - leave it, tick it with a note saying why, and move on.

Roadmap

Ordered by what the job ads ask for. Each line is the file and the single question the Article should answer.

Wave A - the agent core · done 2026-08-19

The most-demanded cluster across every agentic ad.

ai_ml/10_agents_orchestration/02_the_agent_loop.md - what the loop actually is, and why the step budget and tool-error feedback are the loop
ai_ml/10_agents_orchestration/08_function_calling_and_structured_output.md - how a model calls a tool, and where schema guarantees stop
ai_ml/10_agents_orchestration/07_agent_failure_modes.md - how agents fail, and which failures are silent
ai_ml/12_context_engineering/01_context_management.md - what to keep when history outgrows the window
ai_ml/10_agents_orchestration/05_durable_execution_hitl.md - resuming across a restart, and where a human approves
ai_ml/10_agents_orchestration/04_multi_agent_patterns.md - when a second agent earns its cost

Wave B - RAG depth · done 2026-08-19

ai_ml/09_rag_embeddings/05_chunking_and_retrieval.md - was 433 lines of two questions, and its retrieval half duplicated 08_. Split: 05_ is now the chunking Article (228 lines), the retrieval sections were deleted, and time-aware ranking moved to 08_. Filename kept to avoid link churn
ai_ml/09_rag_embeddings/02_what_are_embeddings.md - what a vector encodes, and what it cannot
ai_ml/13_evaluation/02_building_eval_sets.md - where a golden set comes from when nobody labelled anything
ai_ml/13_evaluation/03_llm_as_judge.md - grading with a model, and the biases that make it wrong
ai_ml/07_training_finetuning/07_fine_tuning_vs_rag.md - the decision, argued with costs
ai_ml/09_rag_embeddings/04_vector_databases.md - left as Reference, correctly: it is a product landscape, and “which vector database” is a lookup question. Trimmed its hybrid-search section, which duplicated 08_. But it gave HNSW one table row, so the real gap was a missing Article: 14_ann_indexes_and_hnsw.md

Wave C - Python and async · done 2026-08-20

backend/04_async_concurrency/08_cpu_io_tasks.md - not a split; it was a 3x2 matrix walked exhaustively, and it never mentioned free-threading despite STACK_BASELINE saying to rewrite every GIL note around it. Rewritten around the mechanism, 375 -> 221 lines.
plaintext
```
**Folder redundancy noted:** `01_gil`, `02_concurrency_vs_parallelism`,
`03_threads_vs_processes`, `07_asyncio_vs_threads` and
`09_when_to_use_processes` all answer overlapping versions of "which
primitive". Worth a consolidation pass; added to Phase 3.
```
backend/04_async_concurrency/12_taskgroup_structured_concurrency.md - what structured concurrency prevents that gather does not
backend/06_web_frameworks/fastapi/03_dependencies_and_injection.md - how Depends resolves, and what per-request caching means
backend/06_web_frameworks/fastapi/10_fastapi_async.md - the sync endpoint in a threadpool, and the blocking call that stalls the loop
backend/02_python_core/22_eq_vs_hash.md - the contract, and what breaks when you violate it
backend/07_rest_apis/09_stateful_vs_stateless.md - what state costs at the load balancer

Wave D - distributed systems

backend/13_architecture_design/15_event_driven_saga.md - compensation when you cannot roll back
backend/10_message_queues/kafka/04_delivery_semantics.md - what exactly-once means and where it stops being true
backend/10_message_queues/celery/04_idempotency.md - the consumer side; cross-link, do not duplicate 18_idempotency_keys.md
backend/14_microservices/04_data_consistency_patterns.md - consistency without a distributed transaction
system_design/02_resilience/01_timeouts_retries_backoff.md - the retry storm you cause by retrying
backend/15_observability/07_opentelemetry.md - read for a split; decided against. Ten notes link to it as “the standard” and a survey is legitimately Reference, so it stayed Reference: fixed a broken TODO link, corrected the stale “OTel logs are immature” claim, and cut the sampling and propagation sections down to pointers at the notes that own them. The Article for this wave is the single-question note underneath it:
backend/15_observability/10_correlation_ids_async.md - why a trace breaks: the contextvars copy rule

Wave E - security and guardrails

ai_ml/14_guardrails_safety/01_prompt_injection.md - why prompting cannot solve it, and what to do instead
ai_ml/14_guardrails_safety/03_pii_privacy_and_compliance.md - personal data through an LLM pipeline
backend/25_security/06_jwt_pitfalls.md - the failures that are not about the algorithm
backend/25_security/05_oauth2_oidc.md - which flow, and why the others are wrong

Wave F - frontend

Do this wave. CLAUDE.md says frontend is maintained to the same standard, not de-scoped, and ten Articles passed 01_core_web_vitals.md sitting at the top of the queue. Skipping it again reverses a settled decision by attrition.

frontend/15_performance/01_core_web_vitals.md - what the metrics measure and what actually moves them
frontend/11_apis_data_fetching/05_abort_and_race_conditions.md - the stale response that overwrites the fresh one
frontend/05_react/server_components.md - what runs where, and the boundary people get wrong
frontend/11_apis_data_fetching/02_tanstack_query.md - cache as the source of truth for server state

Wave G - data and ML foundations

backend/08_databases/sql/02_indexes.md - now trimmed; promote as “how a B-tree index actually works”
ai_ml/03_feature_engineering/04_data_leakage.md - the bug that makes a model look brilliant
ai_ml/06_transformers_llm/05_kv_cache.md - why generation is memory bound
ai_ml/06_transformers_llm/08_context_windows.md - what long context costs, and where attention degrades
ai_ml/02_classical_ml/05_gradient_boosting.md - why it still wins on tabular data

Done

Phase 0 - rules and tooling · 2026-08-19

Code width 56 → 88, diagrams 60; code-wide fell 8,088 → ~1,200. Three tiers, with > Article. as the opt-in. tools/tiers.py computes the queue.

Phase 1 - the SQL pilot · 2026-08-19

Six Articles, folder renumbered contiguous, one survey note deleted, ~150 lines of duplication removed: 24_triggers_in_practice.md, 25_ctes_and_recursive_queries.md, 26_functions_and_procedures.md, 27_views_and_materialized_views.md, 10_explain_analyze.md (rewritten as Reading a query plan), 28_why_your_index_is_not_used.md.

Phase 2 - in progress

backend/13_architecture_design/18_idempotency_keys.md - found a real bug in its own code: SELECT ... FOR UPDATE on a row that does not exist takes no lock. Replaced with ON CONFLICT DO NOTHING RETURNING.
ai_ml/09_rag_embeddings/08_hybrid_search_and_reranking.md - latency budget, reranker fallback, pre/post-filter effect on HNSW recall, and a correction: Postgres core FTS is ts_rank, not BM25.
ai_ml/15_mlops_llmops/03_llm_observability.md - 94 → 262 lines. Context propagation across tasks, threads and processes; streaming vs TTFT; tail- vs head-based sampling. Its Verified marker was bold rather than a blockquote, so staleness.py had never counted it.
backend/13_architecture_design/16_transactional_outbox.md - corrected its own claim: sequences are non-transactional, so the outbox id does not give global ordering, and a high-water-mark cursor silently loses events.

Decisions already taken

Not everything becomes an Article. Most of the corpus is legitimately reference material.
techcheck/ is out of scope for prioritisation and coverage - a private folder for one specific interview.
Job descriptions are the demand signal. Nine real ads audited in August 2026; tools/coverage.py re-runs the check against any new one.
Frontend stays, on the same terms as everything else. Wave F exists because of this.
Length is not a gate. Completeness wins; see CLAUDE.md.
One-shot migration scripts do not live in tools/. normalize_frontend.py, normalize_headings.py and unwrap_comments.py did their jobs in August 2026 and were deleted 2026-09-11; git history keeps them. tools/ holds only what you would run again.

Wave H - the Interview angle backlog

This is the current wave. Work it the same way as the roadmap above.

tools/normalize_frontend.py put 157 frontend notes onto the house template on 2026-08-20, which renamed ## What a senior is expected to say to ## Interview angle. The structure is right and the content is not yet: those sections are unattributed statements, which is answers with the questions removed. You cannot quiz yourself from them, and quizzing yourself is the entire point of the section.

bash

python tools/check_style.py --only angle-no-questions   # the live count

144 notes at the start: 137 frontend, 6 backend, 1 behavioral. Cleared to zero on 2026-08-20, and angle-no-questions is now in GATE_CODES, so a note whose Interview angle carries no question fails the build.

Convert each bullet to the house shape:

markdown

- **"Can you revoke a JWT?"** - not while it stays stateless. Revocation means
  the server knows something the token does not say, and knowing is state.
  - *Follow-up: "So what do you actually do?"* - decide the window you can
    tolerate between revoking and it taking effect.

Rules for the conversion, so it stays honest:

Do not invent content. The question comes from the answer that is already there. If a bullet answers nothing an interviewer would ask, delete it rather than writing a question around it.
Merge duplicates. Several bullets often restate one point; one good question beats three thin ones. 2-4 questions per note is the target.
Add the follow-up where the note already contains one. Do not manufacture follow-ups to hit a count - that is the article-no-followup mistake in a different place.
Fix what you find. These files have not been read since they were written; stale versions and broken claims are in scope while you are there.

Work folder by folder, commit per folder, and let the count in --only angle-no-questions be the progress bar.

Wave I - the long tail, scheduled

This is the current wave. Phase 3 below was written as “opportunistic, when a note is already open”, and after eight waves of that it had not moved. The items are real defects, so they get scheduled like everything else.

Ordered by whether a reader would notice. Content overlap first, structural inconsistency second, width counts last.

backend/04_async_concurrency/ - three notes answer the same question as 08_cpu_io_tasks.md in worse form. Assessed 2026-08-20: 02_concurrency_vs_parallelism.md (243 lines) is a textbook essay whose GIL section, comparison table and hybrid example all duplicate 01, 08 and 14; 07_asyncio_vs_threads.md (358) has a Decision Framework and Performance Comparison that are strictly worse than 08’s decision tree; 03_threads_vs_processes.md (118) is sound content in bold-and-numbered headings. Keep all three files - each has a distinct question - and cut each to it.
Essay-shaped headings: ## Introduction (18 files), ## Conclusion (13), ## Summary (61), ## Overview (22), ## Table of Contents (38). A note opens with prose under the h1 and closes with ## Interview angle; a table of contents duplicates the site’s own navigation.
Bold and numbered headings: 41 files matching ^#{2,3} \*\*. The heading is already emphasis.

Then gate what was fixed, so it cannot come back. essay-heading and bold-heading are in GATE_CODES, both verified to fire.

plaintext

**Correction on `## Summary`.** The plan lumped it in with the others
on its count (61). Checked rather than assumed: 49 of the 53 carry a
comparison table or a rule list, only one was filler, and two were
really `### **Summary**` - a bold-heading case. So it is a generic name
over real content, not ceremony, and it stays. Renaming 53 headings for
no reader benefit is churn.

The width backlog - measured, and deliberately not worked. 1,136 code-wide across 410 files, so there is no concentrated win: the worst file has 20. Median overshoot is 9 characters and the 90th percentile is 27, which is a rounding error on a scrollable code block.

plaintext

The composition is the interesting part. 688 of the 1,136 are `text`
fences over the 60-char diagram cap, and sampling them splits roughly
176 real box-drawing diagrams against 132 blocks of sample output. The
cap exists because *diagrams cannot wrap*; sample output scrolls exactly
like code and does not need 60. So a meaningful slice of this backlog is
a rule that is broader than its own rationale, not content that is wrong.

Fixing that properly means a way to tell a diagram from output - a
separate fence tag - which is corpus-wide churn for a code that does not
gate. Left as it is, on purpose, and this note is here so the next
session does not re-derive it.

staleness.py: 16 -> 19 of 1,041. Stamped exactly the three async notes rewritten in this wave, whose free-threading and 3.14 claims were checked against STACK_BASELINE while writing them. Never bulk-stamp - it moves a few notes per wave and that is what makes the signal mean anything. STACK_BASELINE itself was re-checked: verified 2026-08-08 to 08-18, so well inside the ~3-month window and nothing needed re-verifying.

External links - checked for the first time 2026-08-21

check_links.py only ever covered links between notes. Nothing checked the outbound ones, and 17 of 557 were dead - a rot rate around 3%. All 17 fixed with replacements that were probed, not guessed.

Two of them were more than link rot. radix-ui.com/.../combobox 404s because Radix Primitives ships no Combobox, and three notes recommended it for exactly that widget - one of them a sentence I wrote during Wave H. The dead link is what surfaced the wrong advice. Corrected to Downshift, Headless UI, Ariakit and React Aria, which do have one.

tools/check_external.py now exists and runs weekly in CI rather than per push, since it is network-bound. It is deliberately conservative: HEAD then GET, 401/403/429 are not failures, one worker per host, and URLs inside code fences are skipped - the first run reported 38 of 40 “broken” because it was checking nginx $upstream and SOAP namespace URIs.

The width backlog - worked, and why it cannot reach zero

2026-08-21. Reduced from 3,348 to 3,033, and the two systemic fixes matter more than the count:

The diagram cap was measuring the wrong blocks. check_style.py applied the 60-char diagram limit to every ```text fence, including sample output and log excerpts, which scroll exactly like code. It now classifies the block first. That removed 171 false positives and left a number that means something.
tools/unwrap_comments.py was rebuilt for the 88-char cap and moved 101 trailing comments above their code line. The old version targeted 56, had to touch most lines in a file, and was rejected for mangling the aligned comment columns; at 88 it only reaches genuine outliers.

Then four of the worst files were redrawn by hand, including a nested clean-architecture diagram whose borders did not line up.

Why it will never be zero, and must never gate:

A diagram can be legitimately wide. The RFC 6455 frame layout is 65 characters because a 32-bit ASCII bit diagram is 65 characters. All 15 flags in the worst diagram file are that one correct diagram.
Code can contain an atomic token - a URL, an ARN, a hash - that cannot wrap.
table-cell, table-label and table-wide (1,854 of the 3,033) were measured in a browser: of 135 pages with a 4+ column table, six actually overflowed, all six were fixed, and the rest is a scannability preference.
too-long is advisory by design; CLAUDE.md says completeness outranks it.

What remains is 819 code-wide across ~350 files - roughly 439 diagram lines and 380 code lines - each needing an editorial judgment about how to re-lay-out a specific block. It is worth doing when a file is open for another reason. It is not worth a mechanical pass, and a mechanical pass is the one thing that would damage the corpus.

Question-shaped headings - checked, not a defect

2026-08-21: 1,285 ## ...? headings above the Interview angle across 259 files. Not worked, and the count is here so it is not re-derived as alarming. The concentration is in files named *_interview.md, which are question banks where that is the correct format. CLAUDE.md bans the quiz-transcript shape - numbered ### Q1: runs and **Answer:** blocks, both gated and at zero - not a descriptive question used as a section title.

Per-file judgment still applies: behavioral/07 was converted because five question headings were its entire structure, which made a method note read as a Q&A list.

Phase 3 - the long tail

Superseded by Wave I above; kept for the numbering gap item.

~1,200 code-wide lines still over 88.
table-cell and table-label, ~1,800 combined.
~~Numbering gaps.~~ Closed 2026-08-21. Two existed, not one: ai_ml/08_inference_serving/ skipped 09 from a restructure that moved Bedrock in at 10, and fastapi/ lost 12 when the two dependency-injection notes were merged. Renumbered 10 -> 09 and 13-19 -> 12-18, with the link rewrite done as a single mapped pass so one rename could not cascade into another’s result. check_links.py now fails on a gap or a repeated number, verified to fire. The duplicate 12_ in ai_ml/02_classical_ml/ was fixed 2026-08-20.
backend/04_async_concurrency/ has five overlapping “which primitive” notes; consolidate around 08_cpu_io_tasks.md.
staleness.py shows 10 of 1,041 notes verified. Stamp only what you actually re-checked; never bulk-stamp.