Answers — say these out loud

One page. Every question this folder names, with the answer written the way you would actually speak it.

Read each one aloud once. They are written to be said, not skimmed — the sentences are short on purpose so you do not run out of breath or lose the grammar halfway through.

Anything in [brackets] is a number only you know. Fill it in before the call; a real number is what separates senior from mid.

The three rules. Aim for 60–90 seconds. Never stop at “I don’t know” — bridge to the nearest thing you have done. Slower English reads as more fluent, not less.

The other half: The call itself — everything except the answers covers everything that is not a technical answer — the opening, the self-intro, handling follow-ups, and closing.

The 60-second scan

If you only have a minute before the call, read this. One line each — the sentence the whole answer is built around.

• Fine-tuning vs RAG — RAG changes what the model knows, fine-tuning changes how it behaves. If the answer changes when a document changes, it is RAG.
• LLM call vs agent — the defining property is who owns control flow. A chain of five calls is not an agent, because I decided the order.
• Estimating output — three layers: offline golden set, LLM-as-judge with its biases named, production signals. And it runs in CI or it degrades silently.
• Observability — trace and span level, not endpoint level. Debug retrieval first: was the right chunk even there?
• Hallucinations — detect and reduce, and you cannot reach zero, so design for what happens when it is unsure. Abstain and escalate.
• Model routing — small models for classification and extraction, frontier for reasoning. Output tokens cost several times more than input.
• Query optimisation — pg_stat_statements to find it, EXPLAIN (ANALYZE, BUFFERS) to understand it, then one specific fix with a number.
• RAG — two pipelines, ingest and query. Evaluate retrieval and generation separately or you cannot tell which half is broken.
• Money — never a float. Decimal or integer minor units.
• PII — redact before the call, or run the model inside our boundary.

Last project

I worked on a wealth-planning platform, on the backend and the AI side. The main thing I owned was the retrieval pipeline. We had [N] documents that advisers needed to search, and the first version returned results that were technically related but not useful, so people stopped trusting it.

I rebuilt it on Postgres with pgvector. I changed the chunking to [strategy], added hybrid search with reranking, and put an evaluation set in place so we could actually measure whether a change helped. The hardest decision was staying on Postgres instead of adding a dedicated vector database. It was slower in benchmarks, but it kept everything in one system we already operated, backed up and monitored.

The result was recall going from [X] to [Y], and p95 latency down to [Z]. If I did it again, I would build the evaluation set first, before touching the pipeline.

If they ask what was hardest: the honest answer is that the hard part was not the retrieval — it was agreeing with the team on what a good answer even looked like, because until we had that written down we were tuning blind.

Go deeper: How would you describe your strongest project, and what questions would you expect about it?

Which databases have you worked with

Postgres is my main one, and that includes pgvector for embeddings. I also use Redis for caching and rate limiting, and I have worked with [DynamoDB / MongoDB] where the access pattern was simple and the scale was high.

I would say my depth is in Postgres specifically — not just using it, but reading query plans, designing indexes, and knowing when the problem is the query and when it is the schema.

Go deeper: SQL and NoSQL

How do you optimise queries

I follow the same order every time, because guessing is expensive.

First I find out what is actually slow. I use pg_stat_statements to see which queries cost the most in total, not just the slowest single run — usually the real problem is a fast query called ten thousand times.

Then I run EXPLAIN (ANALYZE, BUFFERS) on it. I am looking for a sequential scan on a large table, a row estimate that is far from the actual count, or a sort spilling to disk.

Then I fix the specific thing. Usually it is an index — and the column order matters, equality columns first and range columns last. Sometimes it is the application: an N+1 from lazy loading, which I fix with selectinload.

A concrete example: we had an endpoint taking [X] seconds. The plan showed a sequential scan because the query had a function on the indexed column, so the index could not be used. I changed the query, added a composite index, and it went to [Y] milliseconds.

If they push on when to stop: at some point the query is not the problem. Then I look at denormalising, a materialised view, a read replica, or moving the work to a background job.

Go deeper: EXPLAIN and EXPLAIN ANALYZE

Your LLM experience

I have built production systems, not demos. The two main ones are a RAG pipeline over [N] documents, and agents built with LangGraph that call internal tools.

What I would emphasise is the operational side, because that is where most of the work actually is. I run evaluation sets so I can tell whether a change improved anything. I trace every request so I can debug a bad answer. And I route between models by task, because using a frontier model for everything is expensive and usually unnecessary.

The thing that broke most often was retrieval, not generation. The model was almost always doing a reasonable job with whatever context it received — the problem was that the right chunk was not in that context.

Go deeper: How exactly do you apply AI in your work?

RAG, end to end

There are two pipelines, and it helps to separate them.

Ingestion runs per document: I parse it, split it into chunks, embed each chunk, and store the vectors with metadata. On our stack that is pgvector with an HNSW index.

Query runs per request: I embed the question, retrieve the top candidates, rerank them with a cross-encoder, and pass the best few to the model with an instruction to answer only from that context and to cite it.

The part most people skip is evaluation. I measure retrieval and generation separately — recall@k and MRR for retrieval, faithfulness and answer relevancy for generation. Without that split you cannot tell which half is broken.

If they ask what improved it most: chunking and reranking, in that order. Hybrid search with reciprocal rank fusion was the single biggest win, because dense search alone misses exact matches like product names and error codes.

Go deeper: What is RAG?

Fine-tuning versus RAG

They solve different problems. RAG changes what the model knows at inference time. Fine-tuning changes how the model behaves. They are complementary, not alternatives.

The way I decide is simple: if the answer changes when a document changes, that is RAG. If the answer is wrong in style or format rather than in fact, that is fine-tuning.

In FinTech, RAG is almost always right first. You need citations, you need per-user access control at retrieval time, and you cannot revoke a fine-tuned weight once a document should no longer be visible.

My order of escalation is prompt engineering, then few-shot examples, then RAG, then tools, and fine-tuning last — because it is the most expensive to iterate on.

Go deeper: Fine-Tuning vs RAG vs Prompt Engineering

LLM call versus agent

A single LLM call is a pure function. One prompt in, one completion out, and I wrote the control flow.

An agent is a loop where the model controls the flow. It decides which tool to call, looks at the result, and decides whether to continue or stop.

The defining property is who owns control flow. A chain with five sequential calls is still not an agent, because I decided the order.

That loop has consequences: the number of steps is not deterministic, cost and latency are unbounded, and errors compound. So you need maximum step limits, timeouts, and tracing — which is exactly why observability stops being optional once you have agents.

The point I would add: I default to the least agentic thing that works. Most cases people call agents are really a fixed workflow with one routing decision, and an explicit graph is cheaper, faster and testable. I use a free-running loop only when the task genuinely cannot be planned in advance.

Go deeper: What is an agent

How do you estimate the result from an LLM

I think about it in three layers.

The first is offline evaluation on a golden set. I build 50 to 200 real question and answer pairs from actual user queries. I measure retrieval separately with recall@k and MRR, and generation with faithfulness and answer relevancy. Where the output is structured, I use deterministic checks — is the JSON valid, are the required fields present.

The second is LLM-as-judge for the things you cannot check mechanically. It works, but I would name the caveats, because they are real: position bias, verbosity bias, and self-preference bias when the judge scores its own family. I mitigate by randomising order and calibrating the judge against human labels on a subset.

The third is production signals: thumbs up and down tied to a trace id, retry rate, escalation to a human, and groundedness alerts.

The important part is that the offline set runs in CI. Any prompt change, model version bump or chunking change re-runs it, and I block the merge on a regression. Without that, an LLM system degrades silently.

Go deeper: Building eval sets

Observability

Standard monitoring tells me the endpoint took four seconds. It does not tell me the retriever returned nothing useful on step three of a seven-step agent run. So LLM observability has to be at the step level.

Concretely: one trace per request, with a span for each step — retrieval, reranking, each model call, each tool call. On each span I record the model and its version, the prompt version, tokens in and out, cost, latency, and the ids of the chunks that were retrieved.

The debugging workflow matters more than the tool. A user reports a bad answer, I open the trace by id, and the first thing I check is retrieval — did the right chunk come back at all? If it did, the problem is the prompt or the model. If it did not, the problem is chunking, embedding or the query.

I use Langfuse, and in FinTech I would point out that it can be self-hosted, so prompts and completions containing personal data never leave our boundary.

The loop that matters: every bad trace becomes a case in the evaluation set. Every production failure turns into a regression test.

Go deeper: LLM observability

Hallucinations — what do you do

I split it into detecting them and reducing them, because most people only talk about the second.

To detect, I score the answer for groundedness against the retrieved context — is each claim actually supported. I require citations and verify them programmatically, so I check that the cited passage really contains the claim. For structured output I validate the schema and apply business rules. And for anything numeric I check against the system of record rather than trusting the model’s arithmetic.

Go deeper: Hallucinations, Function Calling, and Tool Use

How do you decrease them

Grounding first — better retrieval, so the right chunk is actually in context. Hybrid search and reranking do more for hallucination than any prompt change.

Then prompting: I tell the model to answer only from the provided context, and I explicitly give it permission to say it does not know. A model forced to answer will invent one.

Then constraining the output: tool calling and JSON schemas instead of free text, and letting code do arithmetic rather than the model.

Then context hygiene — less but better context. Stuffing fifty chunks makes it worse, because irrelevant context causes drift.

And for anything high-stakes, a human review gate.

How I would close: you cannot get to zero. So the real design question is what the system does when it is unsure. I would rather build something that abstains and escalates than something that is confidently wrong three percent of the time — in FinTech, a wrong number is a liability, not a bad user experience.

Go deeper: Hybrid search and reranking

Which models for which tasks

I route by task rather than using one model everywhere.

Small fast models handle classification, routing, extraction and summarisation. That is most of the volume and it does not need a frontier model. Frontier models handle multi-step reasoning, code and anything where a mistake is expensive. Embeddings and reranking are separate models entirely, sized by cost rather than by capability.

I set the routing by task type, and I keep a fallback for when a provider is degraded. The metric I watch is the escalation rate — how often the cheap model was not good enough — because that tells me whether the routing rule is still right.

On cost, the lever that matters most is that output tokens are several times more expensive than input, so shortening the output usually saves more than trimming the prompt.

Go deeper: Which model for which task

FastAPI project structure

The official documentation defines one multi-file layout: an app package, routers split by resource, a dependencies.py, and include_router with a prefix and tags. The idea that there are two types is community shorthand for flat versus layered.

What I do is organise by resource rather than by technical layer, so everything about orders lives together instead of being split across a controllers folder and a services folder. Dependencies go in one module so they can be overridden in tests. And I keep routers thin — they validate input and call a service, so the business logic is testable without the framework.

Go deeper: FastAPI project structure

Do you have experience with international teams

Yes. I worked with a distributed team where English was the working language, across several time zones. Most communication was written and asynchronous — documents and pull request discussions rather than meetings — because with a time difference you cannot rely on being online together.

What I learned is that written communication has to be more explicit. If I have a question at ten in the evening for someone who starts in eight hours, I have to write it so it can be answered without a follow-up. That habit made me better at documentation generally.

Go deeper: Joining an Existing Project: How to Read and Understand the Codebase

Can you work independently

Yes, and I would give you a concrete example rather than just saying so. On the retrieval work, I scoped it myself, agreed the goal with the team, and delivered it in small reviewable pieces. I flagged the risk about the vector database choice early instead of after building it, and I wrote the documentation so nobody had to ask me how it worked.

For a part-time role, the way I would handle it is to over-communicate in writing — a short written update on what I did, what is next, and anything blocked, so you never have to ask.

Go deeper: How do you work independently without close management?

FinTech: how do you store money

Never as a float. Either Decimal, or integers in minor units — cents rather than euros. Floats cannot represent most decimal fractions exactly, so rounding errors accumulate, and in a financial system that shows up as a reconciliation failure nobody can explain.

Go deeper: Python performance for quantitative work

FinTech: what about personal data in prompts

The rule I work to is that personal data does not go to a third-party model unless we have explicitly decided it can. In practice that means redacting before the call, keeping identifiers out of the prompt and passing a reference instead, and knowing where the provider processes the data.

If the requirement is that data cannot leave our environment at all, then the answer is a model inside our boundary — Bedrock or Azure in our own tenancy, or a self-hosted model. That is a real architectural constraint, and it is better to know it before designing than after.

Go deeper: PII, privacy and the EU AI Act

Questions to ask them

Ask two or three. These are the ones that signal seniority.

• What does the AI part of the product do for the end user today, and what is the biggest quality complaint about it?
• Do you already have an evaluation set and tracing in place, or would that be part of what I build?
• Given this is FinTech — what are the constraints on data leaving your environment for inference? Are you on Bedrock or Azure, or is a self-hosted model an option?
• How would you like to split the part-time hours — a fixed overlap with your time zone, or asynchronous with checkpoints?

Phrases to connect your sentences

Learn these. They buy you thinking time and they make the English sound deliberate rather than hesitant.

• “The trade-off there was…”
• “What drove that decision was…”
• “To give you a concrete example…”
• “Let me walk you through it.”
• “I would push back on that slightly, because…”
• “I have not used X specifically, but I have done the same thing with Y.”
• “Let me think about that out loud for a second.”

If you do not know something

Never stop at “I don’t know”. Use this shape:

“I have not used Weaviate specifically. I have run the same pattern on pgvector with HNSW, where the trade-off was recall against latency, and I would expect the same question to come up there.”

Honest and still reasoning beats honest and silent. The last candidate was rejected partly for giving up too quickly, not for lacking knowledge.