DigiTribe - Data Engineer (Azure Databricks & Application Development)

Position overview

Digitribe is looking for a Data Engineer with deep expertise in Azure and Databricks to design, develop, and maintain scalable data engineering solutions. You will build production-grade ETL/ELT pipelines, develop data processing applications, and contribute to a modern lakehouse architecture. Working cross-functionally with Architecture, Data, AI, Cloud, and CI/CD teams, you will translate business and analytical requirements into robust, reusable, and cost-efficient data solutions.

Key responsibilities

• Design, develop, test, and maintain scalable data engineering solutions on Microsoft Azure and Databricks, using Python, PySpark, Scala, and SQL
• Implement reliable batch and streaming data workflows using Azure Databricks, Delta Lake, and modern data lakehouse patterns
• Develop data processing applications for ingestion, transformation, enrichment, validation, and serving of analytical datasets
• Build CI/CD pipelines for data applications using Azure DevOps YAML pipelines and apply software engineering best practices: modular design, clean code, Version Control, code reviews, and documentation
• Develop reusable libraries, utilities, and frameworks to accelerate data product delivery and support less experienced engineers through coaching and knowledge sharing
• Collaborate with Cloud, DevOps, and Platform teams on deployment, environment configuration, and operational readiness
• Technical skills:

• 5+ years of hands-on experience in data engineering, application development, or cloud-based data platform development
• Strong hands-on development experience with Python, PySpark, Scala, SQL, and ETL/ELT development
• Solid experience with the Azure Databricks ecosystem: notebooks, jobs, Spark clusters, Delta Lake, lakehouse architecture, and performance tuning
• Good knowledge of Azure data services: Azure Data Lake Storage, Azure Data Factory, Azure Key Vault, Azure DevOps, and Azure monitoring/logging capabilities
• Experience with CI/CD pipelines (Azure DevOps YAML), Version Control, automated testing, modular development, and code reviews
• Understanding of cloud security, access management, observability, and cost-aware development practices
• Working knowledge of TerraForm or infrastructure-as-code concepts is a plus

• Preferred qualifications:

• Experience with large-scale distributed data processing using Spark and streaming/real-time ingestion patterns
• Experience building reusable data engineering frameworks, libraries, or shared components
• Knowledge of data quality frameworks, metadata management, lineage, and governance practices
• Experience integrating data pipelines with AI, Machine Learning, or analytics use cases; familiarity with MLOps is an advantage
• Preferred certifications: Microsoft Azure Data Engineer Associate, Azure Developer Associate, Azure Solutions Architect Expert, Databricks Certified Data Engineer Associate/Professional, or Databricks Certified Developer for Apache Spark

• Soft skills:

• Strong communication and documentation skills, with the ability to explain technical data engineering concepts to diverse audiences
• Strong problem-solving mindset with a focus on automation, reliability, and maintainability
• Team-oriented, proactive, and comfortable working in a cross-functional environment
• Continuous improvement mindset and willingness to adopt evolving data platform technologies