Tech Lead Data - Databricks

Accountabilities Provide technical leadership to Data Engineering teams, supporting solution design, code reviews, technical decisions, and the resolution of delivery blockers.

Collaborate with the Data Architect to define the technical vision, coding standards, data modeling conventions, and best practices for the Lakehouse environment using Delta Lake and the Medallion architecture.

Design, develop, and oversee data pipelines in Databricks using PySpark and Spark SQL across Bronze, Silver, and Gold layers, ensuring scalability, reliability, security, and governance.

Establish and reinforce DataOps practices, including Git version control, CI/CD, automated unit and integration testing, data quality validation, and technical documentation.

Optimize pipeline performance, including partitioning strategies, Spark tuning, cloud cost management, and operational reliability through monitoring, observability, and alerting.

Ensure non-functional requirements are met, particularly regarding access control, security, and data policies across Azure and Databricks.

Partner with business and analytics stakeholders to translate requirements into scalable technical solutions and realistic delivery plans.

Mentor engineers, promote engineering excellence, and develop reusable templates, playbooks, and technical references to improve team consistency and productivity.

Requirements:

Strong experience in Data Engineering projects, including previous experience providing technical and operational leadership as a Tech Lead or Senior Engineer.

Extensive hands-on experience with Databricks, PySpark, and Spark SQL, particularly in designing and implementing batch data pipelines, with working knowledge of streaming when required.

Deep knowledge of Delta Lake and practical experience implementing the Medallion architecture across Bronze, Silver, and Gold layers.

Solid experience with Azure data services, including Azure Databricks, Azure Data Lake Storage (ADLS), and integration with identity and security services such as Azure Active Directory.

Strong SQL and Python skills, along with a solid understanding of dimensional data modeling, consumption-oriented models, and schema and data contract best practices.

Experience implementing DataOps practices, including Git, CI/CD for data pipelines, automated testing, and clear technical documentation.

Proven ability to review architecture and code, troubleshoot performance and scalability issues, and guide engineering teams toward effective solutions.

Advanced English proficiency for collaboration with stakeholders and the preparation of technical documentation.

Nice to have: Experience with Unity Catalog, Databricks Workflows and Repos, Delta Live Tables, data quality frameworks such as Great Expectations or Soda, and dbt on the Lakehouse.

Additional advantages: Familiarity with orchestration tools such as Airflow, observability practices involving metrics, logs, and alerts, and change data capture or streaming technologies such as Event Hubs and Kafka.

Consulting experience, exposure to international clients, and Databricks or Azure certifications are also valued.

Benefits:

Health and dental insurance.

Meal and food allowances.

Childcare assistance.

Extended paternity leave.

Access to gym memberships and health and wellness professionals through Wellhub (Gympass) and TotalPass partnerships.

Profit Sharing and Results Participation (PLR).

Life insurance.

Access to a continuous learning platform and professional development resources.

Discount club with partner offers.

Free online resources dedicated to physical, mental, and overall well-being.

Pregnancy and responsible parenting courses.

Partnerships with online learning platforms.

Access to a language-learning platform.

Additional benefits and well-being initiatives.

How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best!  Why Apply Through Jobgether? 

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1