Agentic AI · Data Quality
Shifting Data Quality Left with Autonomous Agents
Replacing reactive firefighting with an AI agent that finds, explains, and proposes fixes for data issues before they reach the business.
Problem
Data quality issues were discovered by the business after bad numbers had already shipped — expensive to fix and corrosive to trust in the platform.
Business Context
In a regulated, numbers-driven organisation, a single wrong figure in an executive report can trigger real financial and reputational cost. Detection speed was a business risk, not just an engineering annoyance.
Architecture
A LangGraph-orchestrated agent continuously profiles pipeline outputs, compares against expectations, and when it finds an anomaly, reasons about likely causes using an LLM grounded in schema and lineage context. It then files an enriched ticket and proposes a remediation.
Technical Decisions
Every write action requires human approval. The agent's tools are narrow and auditable. All reasoning is logged, so any decision can be traced and reviewed.
Trade-offs
We deliberately kept the agent advisory-first rather than fully autonomous. Slower to act, but it built the trust needed before granting broader autonomy.
Implementation
We started with a single high-value pipeline, proved the pattern, then generalised the profiling and reasoning components into a reusable framework across domains.
Challenges
Grounding the LLM well enough to avoid confident-but-wrong root-cause claims. Rich lineage and schema context, plus strict prompting, kept hallucinations in check.
Performance Improvements
Mean-time-to-detect for data incidents dropped sharply, and engineers received actionable, pre-diagnosed tickets instead of raw alerts.
Business Value
Fewer bad numbers reached decision-makers, on-call load fell, and institutional knowledge about failure modes was captured in code rather than lost.
Future Enhancements
Granting the agent scoped autonomy to auto-remediate well-understood, low-risk issues without waiting for human approval.
Let's build something intelligent together.
Whether you're modernising a data platform or bringing agentic AI into production, I'd love to hear what you're working on.