II.
SkillArea overview
Reference · liveskill-area:spark-jobs
Apache Spark Jobs overview
Authoring Spark jobs (Scala/Python/SQL) — partitioning, broadcast joins, AQE, and tuning shuffle and memory.
Attributes
displayName
Apache Spark Jobs
description
Authoring Spark jobs (Scala/Python/SQL) — partitioning, broadcast
joins, AQE, and tuning shuffle and memory.
domains
requiresLanguages
expertiseLevels
- intermediate
- expert
Outgoing edges
applies_to1
- specialization:data-engineering-analytics·Specialization
uses_language4
- language:scala·LanguageScala
- language:python·LanguagePython
- language:scala·LanguageScala
- language:python·LanguagePython
Incoming edges
lib_requires_skill_area1
- lib-skill:data-engineering-analytics--spark-jobs·LibrarySkillspark-jobs
prerequisite_for_learning1
- skill-area:data-analysis·SkillAreaData Analysis
requires_expertise1
- role:data-engineer·RoleData Engineer
requires_skill_area3
- stack-profile:data-lakehouse·StackProfileData Lakehouse Stack (Databricks, Spark, Delta Lake, dbt, Airflow)
- stack-profile:data-lake-stack·StackProfileData Lake Stack (Spark, Object Storage, Delta/Iceberg)
- workflow:etl-pipeline-cost-optimization·WorkflowETL Pipeline Cost Optimization
used_by_skill_area2
- language:python·LanguagePython
- language:scala·LanguageScala