Azure Data Engineer – Databricks & Unity Catalog

Responsibilities

• Design and develop scalable data pipelines using Azure Data Factory and Databricks.
• Implement modern data architectures including Data Lake, Delta Lake, and Lakehouse patterns.
• Manage Databricks Unity Catalog for data sharing, permissions, and robust governance.
• Automate data ingestion, transformation, and deployment processes to enhance platform efficiency.
• Monitor and optimize resource consumption and performance within Azure and Databricks environments.
• Collaborate with data scientists and business stakeholders to industrialize new analytics use cases.

Requirements

• 5+ years of experience in data engineering with a focus on Azure cloud environments.
• You possess deep expertise in Azure Databricks and Unity Catalog, including Lakehouse design and security.
• You bring advanced skills in Spark, PySpark, and Delta Lake for large-scale data processing.
• You have strong proficiency in SQL and Python for data manipulation and platform automation.
• You possess experience with Azure Data Factory, Azure Storage, Azure SQL, and Key Vault.
• You have knowledge of CI/CD principles and Azure DevOps applied to data engineering.
• You bring familiarity with application integration patterns such as REST API and AMQP.
• You're proactive and possess an analytical mindset with strong communication skills.
• You are fluent in French or Dutch with professional proficiency in English.

Nice to Haves

• Possession of Microsoft Azure or Databricks certifications.
• Experience with Power BI administration and governance.
• Knowledge of Terraform or Infrastructure as Code.
• Familiarity with real-time data streaming and Azure Monitor.
• Basic knowledge of SAP data sources like S/4HANA or BW/4HANA.