II.
LibraryProcess overview
Reference · livelib-process:data-engineering-analytics--data-lineage
data-lineage overview
Implement comprehensive data lineage mapping with automated extraction (OpenLineage/SQLLineage), graph visualization, impact analysis, and documentation
Attributes
displayName
data-lineage
description
Implement comprehensive data lineage mapping with automated extraction (OpenLineage/SQLLineage), graph visualization, impact analysis, and documentation
libraryPath
library/specializations/data-engineering-analytics/data-lineage.js
specialization
data-engineering-analytics
references
- - OpenLineage: https://openlineage.io/ - SQLLineage: https://github.com/reata/sqllineage - Apache Atlas: https://atlas.apache.org/ - Amundsen: https://www.amundsen.io/ - DataHub: https://datahubproject.io/ - dbt Lineage: https://docs.getdbt.com/docs/collaborate/data-lineage
example
const result = await orchestrate('data-engineering-analytics/data-lineage', {
dataSources: [
{ type: 'database', connection: 'postgres://...', schemas: ['public', 'analytics'] },
{ type: 'dbt', projectPath: '/path/to/dbt/project' },
{ type: 'airflow', dagFolder: '/path/to/dags' },
{ type: 'spark', jobsPath: '/path/to/spark/jobs' }
],
lineageTool: 'openlineage',
graphFormat: 'neo4j',
includeColumnLineage: true,
impactAnalysisEnabled: true,
visualizationTool: 'graphviz',
outputFormat: 'interactive-html'
});
usesAgents
- general-purpose
Outgoing edges
lib_applies_to_domain1
- domain:data-engineering·DomainData Engineering
lib_belongs_to_specialization1
- specialization:data-engineering-analytics·Specialization
lib_implements_workflow1
- workflow:data-pipeline-deployment·WorkflowData Pipeline Deployment
Incoming edges
None.