II.
LibraryProcess overview
Reference · livelib-process:data-engineering-analytics--obt-creation
obt-creation overview
One Big Table (OBT) Creation - Design and implement denormalized One Big Table (OBT) structures by joining fact and dimension tables, optimizing for analytical query performance, implementing appropriate denormalization strategies, and identifying optimal use cases for OBT patterns in data warehouse architecture.
Attributes
displayName
obt-creation
description
One Big Table (OBT) Creation - Design and implement denormalized One Big Table (OBT) structures
by joining fact and dimension tables, optimizing for analytical query performance, implementing appropriate
denormalization strategies, and identifying optimal use cases for OBT patterns in data warehouse architecture.
libraryPath
library/specializations/data-engineering-analytics/obt-creation.js
specialization
data-engineering-analytics
references
- - One Big Table Pattern: https://www.holistics.io/blog/the-one-big-table-gambit/ - Denormalization Best Practices: https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/ - Columnar Storage Optimization: https://docs.snowflake.com/en/user-guide/tables-clustering-micropartitions - BigQuery Denormalized Tables: https://cloud.google.com/bigquery/docs/best-practices-performance-patterns - Performance Optimization: https://www.databricks.com/blog/2022/05/20/five-simple-steps-for-implementing-a-star-schema-in-databricks-with-delta-lake.html
example
const result = await orchestrate('data-engineering-analytics/obt-creation', {
projectName: 'Sales Analytics OBT',
sourceSchema: 'star_schema',
targetSchema: 'obt_analytics',
performanceGoals: { queryTimeReduction: 80, targetLatency: '100ms' },
dataVolume: 'large', // 'small', 'medium', 'large', 'xlarge'
refreshStrategy: 'incremental', // 'full', 'incremental', 'streaming'
storageOptimization: 'columnar', // 'columnar', 'row-based', 'hybrid'
aggregationLevel: 'transaction' // 'transaction', 'daily', 'weekly', 'monthly'
});
usesAgents
- schema-analyst
- use-case-analyst
- denormalization-architect
- join-optimizer
- obt-schema-designer
- performance-optimizer
- materialization-architect
- ddl-generator
- pipeline-architect
- query-specialist
- validation-architect
- documentation-specialist
Outgoing edges
lib_applies_to_domain1
- domain:data-engineering·DomainData Engineering
lib_belongs_to_specialization1
- specialization:data-engineering-analytics·Specialization
lib_implements_workflow1
- workflow:data-pipeline-deployment·WorkflowData Pipeline Deployment
Incoming edges
None.