Pipelines
Every data flow and ETL process that connects your systems — sources, targets, schedules, and frequencies — extracted from your integration docs.
Pipelines are the connective tissue. When they're catalogued, lineage falls out for free.
Pipelines
2 of 4 Inventories
91
records
54
Approved
28
Pending
9
Manual
Live counts from the agent pipeline
How records get here.
Records land in the inventory two ways — extracted from documents by an agent, or entered by your team. Both paths feed the same approval queue.
Pipeline Inventory Agent
AI-powered extraction.
The agent reads integration documents and extracts pipeline definitions, source-to-target mappings, and scheduling information automatically.
Processes these document types
Automatic source-and-target detection across documents.
Human in the loop
Manual entry and review.
Data engineers and stewards can add pipelines directly or review and validate agent-extracted records before they feed lineage.
Human actions
- Add records manually
- Edit existing data
- Approve pending items
- Remove duplicates
Every AI suggestion requires human approval.
Trust scores, by source.
Every record carries a trust score based on the maturity of the document it came from. Approved technical specifications produce high-trust records. Informal notes produce low-trust ones.
High Trust
From approved specs and signed-off requirements.
Medium Trust
From draft documents and work-in-progress specs.
Low Trust
From informal docs, emails, and early drafts.
Pilot distribution
From pending to production.
Records start as pending. They must be marked use before downstream tools like lineage can pick them up.
Pending
Awaiting review
Use
Ready for lineage
Lineage
Connected to pipelines
The Pipelines table.
Track data flows with source-target mappings, frequencies, and full traceability — sortable, filterable, exportable.
| # | Trust | Action | Pipeline ID | Pipeline Name | Source System | Target System | Frequency | Source Documentation |
|---|---|---|---|---|---|---|---|---|
| 1 | High | Use | CRM-DWH-001 | CRM Customer Sync | CRM System | Data Warehouse | Daily | CRM Integration Spec v2.pdf |
| 2 | High | Use | ERP-DWH-001 | ERP Financial Extract | ERP Platform | Data Warehouse | Daily | ERP Data Pipeline Requirements.docx |
| 3 | Medium | Use | DWH-RPT-001 | Reporting Aggregation | Data Warehouse | Reporting Engine | Hourly | Reporting Pipeline Tech Spec.pdf |
| 4 | Medium | Pending | KYC-CRM-001 | KYC Customer Update | KYC System | CRM System | Real-time | KYC Integration Business Req.docx |
| 5 | Low | Pending | ATM-DWH-001 | ATM Transaction Feed | ATM System | Data Warehouse | Real-time | ATM Data Flow Specification.pdf |
| 6 | Low | Pending | DWH-ML-001 | ML Feature Pipeline | Data Warehouse | ML Pipeline | Daily | Manual entry via UI |
This is a preview — the real interface supports full CRUD operations
Three more inventories.
Pipelines is one of four. Together they form your end-to-end landscape — from the source system down to the column in the database.
Tools that depend on this inventory.
Pipelines are the edges in the lineage graph. The catalogue here drives every connection drawn there.
Ready to map your data flows?
We'll point IDA at your integration docs in a 30-minute call and show you the pipeline catalog that comes out.
Book a demo