II.
StackProfile overview
Reference · livestack-profile:master-data-management
Master Data Management (Python, PostgreSQL, RabbitMQ, Airflow, FastAPI) overview
A centralized master data management hub that maintains golden records for customers, products, suppliers, and organizational hierarchies across the enterprise. FastAPI exposes CRUD and match-merge APIs with configurable survivorship rules for conflict resolution. PostgreSQL stores the master records with full change history and lineage tracking. RabbitMQ distributes change events to downstream consumers ensuring eventual consistency across systems. Apache Airflow orchestrates batch data quality jobs, deduplication pipelines, and cross-reference reconciliation. The tradeoff is the political complexity of data ownership governance and the computational cost of probabilistic matching at scale.
Attributes
displayName
Master Data Management (Python, PostgreSQL, RabbitMQ, Airflow, FastAPI)
description
A centralized master data management hub that maintains golden records
for customers, products, suppliers, and organizational hierarchies
across the enterprise. FastAPI exposes CRUD and match-merge APIs with
configurable survivorship rules for conflict resolution. PostgreSQL
stores the master records with full change history and lineage tracking.
RabbitMQ distributes change events to downstream consumers ensuring
eventual consistency across systems. Apache Airflow orchestrates batch
data quality jobs, deduplication pipelines, and cross-reference
reconciliation. The tradeoff is the political complexity of data
ownership governance and the computational cost of probabilistic
matching at scale.
composes
Outgoing edges
applies_to2
- domain:data-engineering·DomainData Engineering
- domain:operations·DomainOperations
composed_of8
- language:python·LanguagePython
- tool:psql·Toolpsql
- tool:rabbitmq·ToolRabbitMQ
- tool:airflow·ToolApache Airflow
- framework:fastapi·FrameworkFastAPI
- library:pydantic·LibraryPydantic
- library:sqlalchemy·LibrarySQLAlchemy
- library:uvicorn·LibraryUvicorn
follows_workflow2
- workflow:data-quality-monitoring·WorkflowData Quality Monitoring
- workflow:data-governance-review·WorkflowData Governance Review
requires_skill_area5
- skill-area:data-quality·SkillAreaData Quality
- skill-area:etl-pipelines·SkillAreaETL Pipelines
- skill-area:api-design·SkillAreaAPI Design
- skill-area:data-governance·SkillAreaData Governance
- skill-area:messaging-queuing·SkillAreaMessaging and Queuing
used_by_role3
- role:data-engineer·RoleData Engineer
- role:analytics-engineer·RoleAnalytics Engineer
- role:architect·RoleArchitect
Incoming edges
None.