Python Data Pipelines

skill-area:python-data-pipelines

SkillAreadomain/skill-areas/python-data-pipelines.yaml·Open in Graph →

Attributes

displayName

Python Data Pipelines

description

Building Python data pipelines: pandas / polars dataframes, Airflow or Dagster orchestration, parquet/arrow IO, idempotent task design, and incremental backfills.

domains

domain:data-engineering

requiresLanguages

language:python

requiresLibraries

expertiseLevels

intermediate
expert

Outgoing edges (5)

applies_to1

domain:data-engineering·DomainData Engineering

uses_language2

uses_library2

library:pandas·Librarypandas
library:polars·LibraryPolars

Incoming edges (203)

addresses3

skill:xlsx-handling·SkillXLSX Handling
skill:python-data-analysis·SkillPython Data Analysis
skill:csv-analysis·SkillCSV Analysis

library_used_by2

library:pandas·Librarypandas
library:polars·LibraryPolars

requires_expertise7

role:data-scientist·RoleData Scientist
role:quantitative-analyst·RoleQuantitative Analyst
role:portfolio-manager·RolePortfolio Manager
role:risk-analyst·RoleRisk Analyst
role:research-analyst·RoleResearch Analyst
role:business-analyst·RoleBusiness Analyst
role:data-analyst·RoleData Analyst

requires_skill_area191

workflow:ad-tech-stack-setup·WorkflowAd Tech Stack Setup
workflow:paid-search-campaign-optimization·WorkflowPaid Search Campaign Optimization
workflow:display-retargeting-optimization·WorkflowDisplay Retargeting Optimization
workflow:mission-simulation-review·WorkflowMission Simulation Review
workflow:precision-agriculture-data-pipeline·WorkflowPrecision Agriculture Data Pipeline
workflow:crop-yield-forecasting·WorkflowCrop Yield Forecasting
workflow:algo-strategy-backtesting·WorkflowAlgorithmic Strategy Backtesting
workflow:algo-strategy-paper-trading·WorkflowAlgorithmic Strategy Paper Trading
workflow:execution-algorithm-optimization·WorkflowExecution Algorithm Optimization
workflow:market-making-parameter-tuning·WorkflowMarket-Making Parameter Tuning
workflow:momentum-signal-research·WorkflowMomentum Signal Research
workflow:algorithm-benchmarking·WorkflowAlgorithm Benchmarking
workflow:survey-methodology-review·WorkflowSurvey Methodology Review
workflow:flight-operations-review·WorkflowFlight Operations Review
workflow:mro-scheduling-optimization·WorkflowMRO Scheduling Optimization
workflow:clinical-data-pipeline-review·WorkflowClinical Data Pipeline Review
workflow:bizdev-crm-and-tracking-setup·WorkflowBizDev CRM and Tracking Setup
workflow:partnership-pipeline-review·WorkflowPartnership Pipeline Review
workflow:strategic-alliance-performance-review·WorkflowStrategic Alliance Performance Review
workflow:market-expansion-assessment·WorkflowMarket Expansion Assessment
workflow:business-case-development·WorkflowBusiness Case Development
workflow:process-automation-assessment·WorkflowProcess Automation Assessment
workflow:process-simulation-review·WorkflowProcess Simulation Review
workflow:structural-analysis-review·WorkflowStructural Analysis Review
workflow:renewable-energy-project-assessment·WorkflowRenewable Energy Project Assessment
workflow:circular-economy-audit·WorkflowCircular Economy Audit
workflow:gdpr-data-subject-request·WorkflowGDPR Data Subject Request
workflow:data-retention-review·WorkflowData Retention Review
workflow:privacy-impact-assessment·WorkflowPrivacy Impact Assessment
workflow:project-risk-assessment·WorkflowProject Risk Assessment
workflow:supply-chain-risk-monitoring·WorkflowSupply Chain Risk Monitoring
workflow:climate-risk-physical-assessment·WorkflowClimate Risk Physical Assessment
workflow:end-to-end-customer-journey-audit·WorkflowEnd-to-End Customer Journey Audit
workflow:customer-health-score-review·WorkflowCustomer Health Score Review
workflow:nps-survey-cycle·WorkflowNPS Survey Cycle
workflow:customer-win-loss-analysis·WorkflowCustomer Win/Loss Analysis
workflow:cyber-risk-scenario-modeling·WorkflowCyber Risk Scenario Modeling
workflow:ad-hoc-analysis-request-cycle·WorkflowAd-Hoc Analysis Request Cycle
workflow:dashboard-development-cycle·WorkflowDashboard Development Cycle
workflow:data-quality-investigation·WorkflowData Quality Investigation
workflow:cdc-pipeline-validation·WorkflowCDC Pipeline Validation
workflow:data-catalog-maintenance·WorkflowData Catalog Maintenance
workflow:dbt-model-review·Workflowdbt Model Review
workflow:model-explainability-review·WorkflowModel Explainability Review
workflow:dataset-versioning-governance·WorkflowDataset Versioning Governance
workflow:data-pipeline-monitoring·WorkflowData Pipeline Monitoring
workflow:data-governance-review·WorkflowData Governance Review
workflow:day-trading-journal-review·WorkflowDay Trading Journal Review
workflow:pre-market-preparation-routine·WorkflowPre-Market Preparation Routine
workflow:trading-performance-attribution·WorkflowTrading Performance Attribution
workflow:forecasting-model-calibration·WorkflowForecasting Model Calibration
workflow:customer-churn-analysis·WorkflowCustomer Churn Analysis
workflow:pay-equity-analysis·WorkflowPay Equity Analysis
workflow:dei-metrics-review·WorkflowDEI Metrics Review
workflow:inclusive-hiring-audit·WorkflowInclusive Hiring Audit
workflow:sdk-adoption-tracking·WorkflowSDK Adoption Tracking
workflow:student-engagement-analytics·WorkflowStudent Engagement Analytics
workflow:learning-outcomes-assessment·WorkflowLearning Outcomes Assessment
workflow:renewable-energy-forecasting·WorkflowRenewable Energy Forecasting
workflow:board-reporting-cycle·WorkflowBoard Reporting Cycle
workflow:product-market-fit-assessment·WorkflowProduct-Market Fit Assessment
workflow:environmental-impact-assessment·WorkflowEnvironmental Impact Assessment
workflow:sustainability-metrics-review·WorkflowSustainability Metrics Review
workflow:scope3-emissions-tracking·WorkflowScope 3 Emissions Tracking
workflow:climate-risk-scenario-analysis·WorkflowClimate Risk Scenario Analysis
workflow:esg-reporting-cycle·WorkflowESG Reporting Cycle
workflow:carbon-footprint-audit·WorkflowCarbon Footprint Audit
workflow:design-to-shelf-pipeline·WorkflowDesign-to-Shelf Pipeline
workflow:trend-forecasting-cycle·WorkflowTrend Forecasting Cycle
workflow:field-dispatch-optimization·WorkflowField Dispatch Optimization
workflow:accounts-payable-cycle·WorkflowAccounts Payable Cycle
workflow:accounts-receivable-aging-review·WorkflowAccounts Receivable Aging Review
workflow:tax-compliance-review·WorkflowTax Compliance Review
workflow:financial-close-cycle·WorkflowFinancial Close Cycle
workflow:financial-reporting-dashboard-setup·WorkflowFinancial Reporting Dashboard Setup
workflow:budget-planning-cycle·WorkflowBudget Planning Cycle
workflow:market-risk-daily-report·WorkflowMarket Risk Daily Report
workflow:credit-risk-assessment·WorkflowCredit Risk Assessment
workflow:liquidity-risk-monitoring·WorkflowLiquidity Risk Monitoring
workflow:model-risk-management-review·WorkflowModel Risk Management Review
workflow:stress-testing-scenario-design·WorkflowStress Testing Scenario Design
workflow:payment-reconciliation-audit·WorkflowPayment Reconciliation Audit
workflow:recipe-management-review·WorkflowRecipe Management Review
workflow:game-economy-balancing-review·WorkflowGame Economy Balancing Review
workflow:game-analytics-instrumentation·WorkflowGame Analytics Instrumentation
workflow:civic-data-publication·WorkflowCivic Data Publication
workflow:cyber-risk-quantification·WorkflowCyber Risk Quantification
workflow:product-led-growth-metrics-review·WorkflowProduct-Led Growth Metrics Review
workflow:referral-program-performance-review·WorkflowReferral Program Performance Review
workflow:seo-content-audit·WorkflowSEO Content Audit
workflow:conversion-funnel-optimization·WorkflowConversion Funnel Optimization
workflow:growth-experiment-review·WorkflowGrowth Experiment Review
workflow:growth-experimentation-platform-setup·WorkflowGrowth Experimentation Platform Setup
workflow:referral-program-design·WorkflowReferral Program Design
workflow:clinical-trial-data-management·WorkflowClinical Trial Data Management
workflow:guest-experience-feedback-analysis·WorkflowGuest Experience Feedback Analysis
workflow:property-portfolio-analysis·WorkflowProperty Portfolio Analysis
workflow:employee-engagement-pulse-check·WorkflowEmployee Engagement Pulse Check
workflow:workplace-diversity-metrics-review·WorkflowWorkplace Diversity Metrics Review
workflow:recruiting-pipeline-review·WorkflowRecruiting Pipeline Review
workflow:compensation-benchmarking·WorkflowCompensation Benchmarking
workflow:culture-survey-analysis·WorkflowCulture Survey Analysis
workflow:lean-manufacturing-kaizen·WorkflowLean Manufacturing Kaizen
workflow:production-scheduling-optimization·WorkflowProduction Scheduling Optimization
workflow:claims-processing-audit·WorkflowClaims Processing Audit
workflow:shareholder-engagement-cycle·WorkflowShareholder Engagement Cycle
workflow:e-discovery-workflow·WorkflowE-Discovery Workflow
workflow:port-turnaround-optimization·WorkflowPort Turnaround Optimization
workflow:email-marketing-campaign-cycle·WorkflowEmail Marketing Campaign Cycle
workflow:marketing-automation-platform-setup·WorkflowMarketing Automation Platform Setup
workflow:marketing-analytics-stack-setup·WorkflowMarketing Analytics Stack Setup
workflow:marketing-attribution-review·WorkflowMarketing Attribution Review
workflow:material-characterization-pipeline·WorkflowMaterial Characterization Pipeline
workflow:composite-failure-analysis·WorkflowComposite Failure Analysis
workflow:thermal-simulation-validation·WorkflowThermal Simulation Validation
workflow:content-distribution-optimization·WorkflowContent Distribution Optimization
workflow:resource-estimation-review·WorkflowResource Estimation Review
workflow:model-training-cycle·WorkflowModel Training Cycle
workflow:ab-experiment-lifecycle·WorkflowA/B Experiment Lifecycle
workflow:feature-store-management·WorkflowFeature Store Management
workflow:llm-eval-pipeline·WorkflowLLM Evaluation Pipeline
workflow:hyperparameter-tuning-cycle·WorkflowHyperparameter Tuning Cycle
workflow:data-labeling-pipeline·WorkflowData Labeling Pipeline
workflow:ml-experiment-tracking·WorkflowML Experiment Tracking
workflow:nanofabrication-process-audit·WorkflowNanofabrication Process Audit
workflow:nano-characterization-pipeline·WorkflowNano-Characterization Pipeline
workflow:grant-lifecycle-management·WorkflowGrant Lifecycle Management
workflow:impact-measurement-review·WorkflowImpact Measurement Review
workflow:key-risk-indicator-monitoring·WorkflowKey Risk Indicator Monitoring
workflow:risk-appetite-framework-review·WorkflowRisk Appetite Framework Review
workflow:ecosystem-health-assessment·WorkflowEcosystem Health Assessment
workflow:drug-trial-protocol-review·WorkflowDrug Trial Protocol Review
workflow:pharmacovigilance-signal-detection·WorkflowPharmacovigilance Signal Detection
workflow:computational-experiment-validation·WorkflowComputational Experiment Validation
workflow:particle-accelerator-data-pipeline·WorkflowParticle Accelerator Data Pipeline
workflow:developer-portal-analytics-review·WorkflowDeveloper Portal Analytics Review
workflow:portfolio-rebalancing-cycle·WorkflowPortfolio Rebalancing Cycle
workflow:portfolio-risk-attribution·WorkflowPortfolio Risk Attribution
workflow:investment-thesis-review·WorkflowInvestment Thesis Review
workflow:asset-allocation-strategy-review·WorkflowAsset Allocation Strategy Review
workflow:media-coverage-analysis·WorkflowMedia Coverage Analysis
workflow:kpi-dashboard-review·WorkflowKPI Dashboard Review
workflow:data-storytelling-workshop·WorkflowData Storytelling Workshop
workflow:tenant-experience-optimization·WorkflowTenant Experience Optimization
workflow:alpha-factor-research-cycle·WorkflowAlpha Factor Research Cycle
workflow:alternative-data-evaluation·WorkflowAlternative Data Evaluation
workflow:quant-model-peer-review·WorkflowQuant Model Peer Review
workflow:research-notebook-reproducibility·WorkflowResearch Notebook Reproducibility
workflow:signal-decay-monitoring·WorkflowSignal Decay Monitoring
workflow:quantum-algorithm-benchmarking·WorkflowQuantum Algorithm Benchmarking
workflow:error-correction-validation·WorkflowError Correction Validation
workflow:quantum-hardware-calibration·WorkflowQuantum Hardware Calibration
workflow:inventory-replenishment-cycle·WorkflowInventory Replenishment Cycle
workflow:lead-to-revenue-funnel-analysis·WorkflowLead-to-Revenue Funnel Analysis
workflow:pricing-strategy-review·WorkflowPricing Strategy Review
workflow:revenue-forecasting-model-calibration·WorkflowRevenue Forecasting Model Calibration
workflow:robotics-simulation-validation·WorkflowRobotics Simulation Validation
workflow:bioinformatics-pipeline-validation·WorkflowBioinformatics Pipeline Validation
workflow:sales-forecast-review·WorkflowSales Forecast Review
workflow:customer-renewal-management·WorkflowCustomer Renewal Management
workflow:sales-compensation-plan-design·WorkflowSales Compensation Plan Design
workflow:territory-planning·WorkflowTerritory Planning
workflow:experiment-reproducibility-review·WorkflowExperiment Reproducibility Review
workflow:athlete-performance-analytics-review·WorkflowAthlete Performance Analytics Review
workflow:competitive-intelligence-system-setup·WorkflowCompetitive Intelligence System Setup
workflow:competitive-intelligence-briefing·WorkflowCompetitive Intelligence Briefing
workflow:strategic-initiative-progress-review·WorkflowStrategic Initiative Progress Review
workflow:market-research-synthesis·WorkflowMarket Research Synthesis
workflow:demand-forecasting-cycle·WorkflowDemand Forecasting Cycle
workflow:inventory-optimization-review·WorkflowInventory Optimization Review
workflow:genomic-data-pipeline-validation·WorkflowGenomic Data Pipeline Validation
workflow:hotel-revenue-management-review·WorkflowHotel Revenue Management Review
workflow:livestock-monitoring-pipeline·WorkflowLivestock Monitoring Pipeline
workflow:carbon-credit-verification·WorkflowCarbon Credit Verification
workflow:scenario-planning-workshop·WorkflowScenario Planning Workshop
workflow:energy-trading-risk-review·WorkflowEnergy Trading Risk Review
workflow:actuarial-model-validation·WorkflowActuarial Model Validation
workflow:environmental-remediation-planning·WorkflowEnvironmental Remediation Planning
workflow:donor-stewardship-cycle·WorkflowDonor Stewardship Cycle
workflow:best-execution-review·WorkflowBest Execution Review
workflow:transaction-reporting-cycle·WorkflowTransaction Reporting Cycle
workflow:trading-surveillance-review·WorkflowTrading Surveillance Review
workflow:trade-reconciliation-cycle·WorkflowTrade Reconciliation Cycle
workflow:settlement-exception-management·WorkflowSettlement Exception Management
workflow:fleet-maintenance-scheduling·WorkflowFleet Maintenance Scheduling
workflow:route-optimization-review·WorkflowRoute Optimization Review
workflow:deal-flow-screening·WorkflowDeal Flow Screening
workflow:due-diligence-deep-dive·WorkflowDue Diligence Deep Dive
workflow:portfolio-company-review·WorkflowPortfolio Company Review
workflow:data-pipeline-monitoring·WorkflowData Pipeline Monitoring
workflow:data-governance-review·WorkflowData Governance Review