Data Engineer – Data Contracts & Quality (GCP)

• Project being carried out for a large company from the telecommunications sector, related to the development and maintenance of solutions in the area of Data Governance and Data Quality. The main goal is the implementation and automation of Data Contracts in the Google Cloud Platform environment, ensuring data schema consistency and automatic quality validation. Collaboration includes Data Engineering, BI, and business teams.
• Hybrid work from Warsaw (to be determined with the Client), full time, long term

Daily tasks

• Implementation, deployment, and maintenance of Data Contracts in the GCP environment based on agreed specifications and adopted architecture and standards.
• Implementation and updating of data schemas and validation rules in Data Contracts in accordance with quality requirements agreed upon with data owners and technical teams.
• Implementation of automatic schema validation and data compliance with contracts, as well as collaboration on integrating these mechanisms with CI/CD processes.
• Creating SQL queries for data analysis, verification of quality requirements, and identifying non-compliance with contracts.
• Analysis, processing, and modeling of data for the purpose of defining and implementing contracts in Data Warehouse and Big Data solutions.
• Collaboration with product teams, Data Engineering, BI, and data owners when agreeing on requirements, defining contracts, and resolving detected non-compliances.
• Participation in the selection of tools supporting contract automation and improving the processes of their creation, updating, and maintenance.

Requirements

• Experience with GCP
• Practical knowledge of issues regarding the Data Contracts area (defined data contracts, quality requirements) and tools supporting contract automation (e.g., OpenAPI/JSON Schema, CI/CD tools, data QA)
• Experience in data analysis, processing, and modeling, knowledge of Data Warehouse and Big Data solution architecture
• Advanced level SQL query creation skills supported by practical professional experience
• English Level: B1

Must have: GCP, SQL, CI/CD, Data Warehousing / Big Data, DataContracts, JSON Schema / OpenAPI

Nice to have: dbt, Dataform, Knowledge Catalog (formerly Dataplex), Data Catalog, Python