Engenheiro de Dados GCP - Sênior
Accountabilities: As a Senior Data Engineer, you will be responsible for designing, modernizing, and optimizing data pipelines while ensuring data quality, scalability, and operational efficiency.
Reverse engineer complex PySpark-based data logic and translate it into optimized native SQL solutions within BigQuery.
Use AI-powered tools to accelerate data pipeline conversion, improve productivity, and automate validation processes.
Modernize and develop data ingestion workflows by replacing legacy tools with native Google Cloud services such as Datastream, Dataflow, and Cloud Run.
Build and manage data orchestration workflows using Cloud Composer (Apache Airflow).
Manage and optimize data stored in Apache Iceberg format through BigLake, ensuring performance, scalability, and cost efficiency.
Design and execute automated testing frameworks to validate data migration accuracy and business logic consistency.
Perform data reconciliation activities, including row count validation, partition verification, business key comparisons, checksums, and hash validations.
Validate key business aggregations through accuracy checks involving totals, distinct counts, minimums, and maximums.
Troubleshoot complex data performance issues, schema evolution challenges, and large-scale data processing scenarios.
Collaborate with technical stakeholders to ensure successful delivery of cloud data transformation initiatives.
Requirements:
The ideal candidate should have strong experience in cloud data engineering, advanced SQL development, and modern data architecture, with the ability to handle complex, high-volume data environments.
Minimum 6 years of professional experience working with Google Cloud Platform, especially BigQuery, Google Cloud Storage (GCS), and Cloud Composer (Airflow).
Strong expertise in advanced SQL and Python/PySpark development.
Proven experience designing, developing, and optimizing large-scale data pipelines.
Hands-on experience with open table formats, especially Apache Iceberg and Delta, including conversion and data management.
Experience with data transformation orchestration tools such as Dataform or dbt.
Strong understanding of cloud-native data services and migration strategies.
Ability to solve complex performance, scalability, and schema evolution challenges in Terabyte-scale environments.
Analytical mindset with strong problem-solving skills and attention to data quality.
Ability to work independently in remote environments while collaborating effectively with technical teams.
Benefits:
Fully remote work model.
Opportunity to work on challenging cloud data modernization projects.
Temporary project contract with an expected duration of up to 4 months, with potential continuation.
Exposure to modern Google Cloud technologies, AI-assisted engineering practices, and large-scale data platforms.
Opportunity to collaborate with experienced professionals in data, cloud, and artificial intelligence environments.
Continuous learning and professional development opportunities.
Inclusive environment open to professionals with diverse backgrounds.
How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1