Azure Data Engineer with Databricks

Your tasks

• Develop and implement scalable data processing solutions on Azure, transforming and cleaning large structured and unstructured datasets efficiently.
• Design, build, and maintain robust data pipelines integrating data from diverse sources (databases, APIs, streaming platforms).
• Collaborate with cross-functional teams including data engineers, data scientists, and cloud architects to deliver end-to-end data platform solutions.
• Engage at all critical stages of project development with business analysts, source system experts, data architects, engineers, and product owners.
• Clearly communicate and document technical work, factoring in performance and cost considerations in all design decisions.

Requirements

• Hands-on experience with implementing solutions with data lakes and pipelines (Databricks, Azure Data Factory).
• Proven experience in designing and implementing solutions in Azure cloud (knowledge of AWS, GCP clouds will be an additional advantage).
• Proficiency in SQL, PySpark, data modeling (3NF, star/snowflake schemas) and ELT/ETL design patterns (knowledge of popular ETL tools, including Azure Data Factory).
• Experience with Python or similar scripting languages for data transformation, orchestration, and automation.
• Familiarity with advanced Databricks features (Unity Catalog, MLflow, Jobs & Pipelines).
• Understanding cloud data architecture principles, including data Lakehouse concepts, Medallion architecture, API integration, and event-driven architecture.
• Experience with CI/CD and DevOps practices for data workflows (Git, Azure DevOps) and analytics integration (e.g., Power BI).
• Excellent communication and client-facing skills, with the ability to explain technical concepts clearly to team members and stakeholders.
• Strong collaboration skills across multidisciplinary teams.

Must have: SQL, PySpark, Scripting language, Data modeling, Snowflake, Data Lake, Databricks, Azure Data Factory, Azure Cloud, AWS, ETL, UNITY, Power BI, Python, MLflow, Cloud, API, Azure DevOps, CI/CD, Design Patterns, DevOps

Nice to have: GCP