Intelligent Data Dictionary Agent
Link to open source: https://github.com/Harsh3456D/Intelligent-Data-Dictionary-Agent
Link to Live Project: https://idda-frontend-7x3fn0vh3-harsh3456ds-projects.vercel.app/
full-stack Natural Language to SQL (NL2SQL) application that lets users connect to enterprise databases, generate comprehensive data documentation, and query data using plain English — powered by AI.
Enterprise Data Agent bridges the gap between non-technical stakeholders and complex database systems. Users connect to any supported database, auto-generate a full data dictionary (schema, profiling, quality metrics), and then ask questions in natural language — the system translates them to SQL, executes safely, and returns results with explanations.
- Auto-Documentation — Connects to a database and generates a full data dictionary with column types, nullability, primary/foreign keys, sample values, and quality metrics.
- Natural Language Queries — Ask questions in plain English; the system generates, validates, and executes SQL safely.
- SQL Guard — Every generated query passes through multi-layer validation: blocks DDL/DML/admin commands, rejects multi-statement queries, enforces row limits.
- Auto-Correction — If a generated SQL query fails, the system retries with error context (configurable retries).
- PII Masking — Automatically detects and masks sensitive columns (email, phone, SSN, etc.) in query results.
- Data Profiling — Computes row counts, null percentages, unique counts, min/max values, and top-N frequency distributions.
- Sleek Dark Mode UI — Black/dark grey backgrounds with Electric Blue (#007BFF) accents, fully responsive.
- Versioned Outputs — Documentation snapshots are timestamped and stored with a
latest/symlink.
enterprise_data_agent/
├── api_server.py # FastAPI application entry point
├── package.json # Backend npm scripts (optional)
├── app/
│ ├── main.py # CLI entry + connector factory
│ ├── config/
│ │ └── settings.py # Pydantic Settings (env/CLI config)
│ ├── ai/
│ │ ├── llm_client.py # Gemini API wrapper with retry
│ │ └── prompts.py # System/user prompt templates
│ ├── chat/
│ │ └── handler.py # NL2SQL pipeline orchestrator
│ ├── connectors/
│ │ ├── base.py # DatabaseConnector ABC
│ │ ├── mysql.py # MySQL connector
│ │ ├── postgres.py # PostgreSQL connector
│ │ ├── sqlserver.py # SQL Server connector
│ │ ├── oracle.py # Oracle connector (SQLAlchemy)
│ │ ├── sqlite.py # SQLite connector
│ │ └── snowflake.py # Snowflake connector
│ ├── metadata/
│ │ └── extractor.py # Schema introspection via SQLAlchemy
│ ├── profiling/
│ │ └── profiler.py # Statistical data profiling
│ ├── docs/
│ │ ├── builder.py # Documentation assembly
│ │ └── markdown_gen.py # Markdown report generator
│ ├── security/
│ │ ├── sql_guard.py # SQL validation & safety checks
│ │ └── masking.py # PII detection & masking
│ ├── storage/
│ │ └── vector_store.py # Vector store integration
│ └── utils/
│ └── logger.py # Loguru configuration
├── frontend/
│ ├── src/
│ │ ├── App.tsx # Main layout
│ │ ├── main.tsx # React entry
│ │ ├── types.ts # TypeScript interfaces
│ │ └── components/
│ │ ├── Header.tsx
│ │ ├── ConnectionPanel.tsx
│ │ ├── TableExplorer.tsx
│ │ ├── SchemaGrid.tsx
│ │ ├── QueryAssistant.tsx
│ │ └── DocumentationCenter.tsx
│ ├── index.html
│ ├── package.json
│ ├── vite.config.ts
│ └── tailwind.config.js
├── tests/
│ └── test_db_connectivity.py # DB connectivity & SQL Guard tests
└── outputs/ # Generated documentation (git-ignored)
┌──────────────────────────────────────────────────────────┐
│ React Frontend │
│ (Vite + Tailwind CSS + TypeScript) │
│ ┌──────────┬───────────┬───────────┬──────────────────┐ │
│ │Connection│ Schema │ Query │ Documentation │ │
│ │ Panel │ Explorer │ Assistant │ Center │ │
│ └──────────┴───────────┴───────────┴──────────────────┘ │
└──────────────────────┬───────────────────────────────────┘
│ HTTP (Axios)
▼
┌──────────────────────────────────────────────────────────┐
│ FastAPI Backend │
│ ┌──────────┬───────────┬───────────┬──────────────────┐ │
│ │ API │ Chat │ Metadata │ Docs │ │
│ │ Server │ Handler │ Extractor │ Builder │ │
│ ├──────────┼───────────┼───────────┼──────────────────┤ │
│ │ SQL │ LLM │ Profiler │ Masking │ │
│ │ Guard │ Client │ │ │ │
│ └──────────┴───────────┴───────────┴──────────────────┘ │
│ Connectors Layer │
│ ┌────────┬────────┬───────┬────────┬───────┬──────────┐ │
│ │ MySQL │Postgres│ MSSQL │ Oracle │SQLite │Snowflake │ │
│ └────────┴────────┴───────┴────────┴───────┴──────────┘ │
└──────────────────────────────────────────────────────────┘This build was uploaded as a hackathon project
