← back to projects

cat esg-rag-platform/README.md

AI-Powered ESG Scoring Platform

DSP Investment Managers · 2025

RAG + live web grounding, scoring 2,000+ listed companies in production

PythonFastAPIKubernetesDockerPostgreSQLRAGGeminiTavilyAirflow

Problem

ESG evaluation for 2,000+ listed companies was a manual, analyst-heavy process spread across thousands of PDFs, XBRL filings, and annual reports — slow to scale and hard to audit.

Approach

  • Designed an end-to-end RAG pipeline: vector embeddings over ingested filings, combined with live web grounding via Tavily search and Gemini Search Grounding for facts outside the document set.
  • Shipped the scoring service as Kubernetes-deployed FastAPI microservices, with a Generated → In Review → Finalized workflow and full audit history for analyst sign-off.
  • Built the ingestion layer in Airflow to scrape and process 2,300+ ESG PDFs, XBRL filings, and annual reports across multiple financial years, with idempotent processing.
  • Modeled the data in PostgreSQL — schemas, audit tables, and materialized views — to back a real-time market dashboard with 1-day to 10-year historical charting.

Impact

  • Automated ESG evaluation for 2,000+ listed companies, cutting manual analyst effort.
  • 2,300+ source documents processed with 99% pipeline reliability.
  • Production service running on Kubernetes with a full audit trail from generation to publish.