Engenheiro(a) de Dados Sênior – Azure Databricks
Your tasks
- Design, develop, maintain, and optimize data pipelines using Azure Databricks, PySpark, Azure Data Factory (ADF), Python, and Azure Data Lake.
- Build and support ETL/ELT processes to integrate enterprise data and meet analytical and business requirements.
- Perform data ingestion, transformation, validation, and reconciliation across multiple source systems.
- Conduct data profiling and quality assessments to identify inconsistencies, resolve issues, and ensure data accuracy and reliability.
- Develop, write, and optimize complex SQL queries to improve data processing performance.
- Perform performance tuning and troubleshoot large-scale data workloads to ensure efficiency, scalability, and reliability.
- Provide Level 2 production support for incidents involving Azure Data Factory pipelines, Databricks jobs, and data processing workflows.
- Monitor and maintain production pipelines, ensuring availability, operational stability, and efficient resource utilization.
- Support deployment and release processes across development, testing, and production environments.
- Manage competing priorities across multiple projects, working independently while meeting deadlines and maintaining high delivery standards.
Requirements
- More than eight years of professional experience in data engineering, including the development and maintenance of ETL processes.
- Strong hands-on experience with Azure Databricks and Azure Data Factory (ADF).
- Extensive knowledge of ETL/ELT development, data transformation, and enterprise data integration.
- Advanced SQL skills, including complex query development, optimization, and performance tuning.
- Practical experience with Python and PySpark for data processing and engineering.
- Experience working with Azure Data Lake and integrating data from multiple enterprise systems.
- Proven ability to perform data profiling, validation, reconciliation, and quality assurance.
- Experience monitoring, maintaining, and troubleshooting data pipelines in production environments.
- Advanced English proficiency, both written and spoken, to collaborate effectively in a global working environment.
- Strong analytical and investigative skills, with a structured, solution-oriented approach to complex technical challenges.
- Ability to work autonomously, organize activities, manage competing demands, and prioritize effectively.
- Excellent verbal and written communication skills, with the ability to collaborate across multidisciplinary teams.
- Commitment to code quality, performance, reliability, and timely delivery.
Nice to have
- Experience implementing CI/CD pipelines and DevOps practices.
- Familiarity with Git, Azure DevOps, or similar version control and deployment tools.
- Knowledge of cloud monitoring, logging, and observability solutions.
What we offer
- Fully remote work arrangement.
- CLT employment contract in Brazil.
- Opportunity to contribute to large-scale enterprise data engineering initiatives in a global environment.
- Salary, healthcare coverage, paid time off, and additional benefits are not specified in the provided job description.
How to apply
How Jobgether works We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether?
Other
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.