Building a RAG pipeline that ships to Kubernetes, not just a notebook

It's easy to get a RAG pipeline working in a notebook: chunk some PDFs, embed them, query an LLM, done. It's a different problem when the output is an ESG score for a listed company that an analyst has to be able to defend.

The actual requirements

Building the ESG scoring platform, the interesting constraints weren't about retrieval quality in isolation — they were about what happens around it:

• Grounding beyond the document set. Filings don't contain everything relevant to an ESG score. We combined vector search over 2,300+ ingested PDFs, XBRL filings, and annual reports with live web grounding (Tavily search, Gemini Search Grounding) so the system isn't limited to stale documents.
• A score isn't a single LLM call. It has a lifecycle: Generated → In Review → Finalized, with full audit history. An analyst needs to see what changed, when, and why — not just the final number.
• It has to run like a service, not a script. The scoring pipeline is deployed as Kubernetes-based FastAPI microservices, the same way any other production backend in the company runs — with health checks, rollouts, and the ability to kubectl rollout undo when something goes wrong.

Ingestion is most of the work

The unglamorous part — and the part that actually determines reliability — is ingestion. Scraping and processing 2,300+ ESG PDFs, XBRL filings, and annual reports across multiple financial years, orchestrated in Airflow, with idempotent processing so a re-run doesn't double-count or corrupt state. Getting to 99% pipeline reliability had much less to do with prompt engineering than with treating ingestion like the data engineering problem it is: idempotency, validation, and graceful handling of malformed source files.

What I'd tell someone starting this

Treat the LLM as one component in a data system, not the whole system. The retrieval quality matters, but so does the database schema, the audit trail, the deployment target, and what happens when a dependency pin goes stale in production. RAG systems that only work in a notebook and RAG systems that survive being on-call for are built differently — and it shows.