Azure Databricks Data Engineer
- Zveřejněno
- Typ úvazku
- Plný úvazek
Shrnutí pozice
Cypher Consulting Europe is seeking an experienced Azure Databricks Data Engineer to build and operate secure, scalable batch and streaming data platforms on Azure. The role focuses on Spark/PySpark or Scala, Spark SQL, Delta Lake, bronze/silver/gold architectures, CDC, performance tuning, and production observability. You’ll work with Unity Catalog, ADLS, ADF/Synapse, Key Vault, Entra ID, and implement CI/CD and infrastructure automation through Azure DevOps or GitHub Actions. Candidates should have strong enterprise data platform, governance, security, lineage, and troubleshooting experience, plus the ability to work independently in a consulting environment.
Koho hledáme
- Strong hands-on expertise with Azure Databricks engineering
- Advanced Apache Spark experience using: PySpark or Scala Spark SQL
- Deep understanding of Delta Lake architecture and optimization techniques
- Experience building scalable enterprise data platforms and real-time pipelines
- Strong governance and security experience in multi-team environments
- Experience with service principals, managed identities, lineage, and auditability
- Strong troubleshooting and Spark performance optimization skills
- Azure integration experience across storage, orchestration, and security services
- Solid CI/CD and Infrastructure-as-Code exposure
- Comfortable working independently in consulting-style delivery environments
Co budeš dělat
- Design, build, and maintain scalable batch and streaming pipelines using bronze/silver/gold architectures
- Develop reusable transformation frameworks and production-grade Spark workloads
- Implement and optimize Delta Lake solutions, including: ACID transactions, Partitioning strategies, Schema evolution, OPTIMIZE/ZORDER, Incremental and CDC processing patterns
- Configure and manage Unity Catalog permissions, governance, lineage, and auditability
- Perform Spark performance tuning and observability monitoring
- Integrate Databricks with Azure services including ADLS, ADF/Synapse, Key Vault, and Microsoft Entra ID
- Implement CI/CD pipelines and deployment automation using Azure DevOps or GitHub Actions
- Collaborate with architects, analysts, data scientists, and engineers to align technical implementation with business needs
- Continuously improve platform performance, reliability, cost efficiency, and standardization
O pozici
We are looking for an experienced Azure Databricks Data Engineer to help build and operate scalable, production-grade data platforms on Azure Databricks and Microsoft Azure. The ideal candidate combines strong Spark engineering expertise with a production engineering mindset, treating data pipelines as reliable, enterprise-grade products rather than ad-hoc solutions. This role requires hands-on experience delivering secure, governed, high-performance batch and streaming data platforms at scale.