← back to projects
cat esg-rag-platform/README.md
AI-Powered ESG Scoring Platform
DSP Investment Managers · 2025
RAG + live web grounding, scoring 2,000+ listed companies in production
PythonFastAPIKubernetesDockerPostgreSQLRAGGeminiTavilyAirflow
Problem
ESG evaluation for 2,000+ listed companies was a manual, analyst-heavy process spread across thousands of PDFs, XBRL filings, and annual reports — slow to scale and hard to audit.
Approach
- ▸Designed an end-to-end RAG pipeline: vector embeddings over ingested filings, combined with live web grounding via Tavily search and Gemini Search Grounding for facts outside the document set.
- ▸Shipped the scoring service as Kubernetes-deployed FastAPI microservices, with a Generated → In Review → Finalized workflow and full audit history for analyst sign-off.
- ▸Built the ingestion layer in Airflow to scrape and process 2,300+ ESG PDFs, XBRL filings, and annual reports across multiple financial years, with idempotent processing.
- ▸Modeled the data in PostgreSQL — schemas, audit tables, and materialized views — to back a real-time market dashboard with 1-day to 10-year historical charting.
Impact
- ✓Automated ESG evaluation for 2,000+ listed companies, cutting manual analyst effort.
- ✓2,300+ source documents processed with 99% pipeline reliability.
- ✓Production service running on Kubernetes with a full audit trail from generation to publish.