Senior Data Engineer - #1630

Key Responsibilities

• Design, build and operate scalable data architectures and pipelines for ingesting, transforming and serving data across diverse source systems and use cases.
• Develop robust data models and reusable data capabilities for applications, analysts, data scientists and other data consumers.
• Apply proven architectural and engineering practices to improve the reliability, security, observability, performance and maintainability of data systems.
• Evaluate technologies and architectural approaches, make sound technical trade-offs, and contribute to the evolution of the Data Programmers architecture and engineering standards.
• Champion modern software engineering practices (automated testing, code review, CI/CD, infrastructure-as-code) and help the team consistently meet these standards through review and coaching.
• Work cross-functionally with engineers, Product Managers, Data Scientists, analysts and users, while providing technical leadership through design reviews, mentoring and knowledge sharing.

What we are looking for:

• Strong software engineering fundamentals and proficiency in Python and SQL, with hands-on experience building and operating complex production data systems.
• Strong experience in enterprise data architecture and engineering, with the ability to apply established patterns and practices to new technical problems.
• Experience designing data pipelines and data models, with a strong understanding of data warehouses, data lakes and lake house architectures.
• Experience with cloud platforms, preferably AWS, and modern data warehouse or data platforms such as Redshift, Snowflake, Databricks, BigQuery or equivalent.
• Experience with data orchestration, transformation and modelling using modern engineering approaches and tools.
• Strong understanding of production engineering practices including testing, CI/CD, monitoring, troubleshooting and data quality.

Good to have:

• Hands-on experience with workflow orchestration tools such as Apache Airflow or equivalent.
• Experience with transformation and analytics engineering frameworks such as dbt or equivalent.
• Experience with distributed data processing technologies such as Apache Spark.
• Deep experience with AWS data services and cloud infrastructure.
• Familiarity with BI and analytics tools such as Tableau, Power BI or equivalent.
• Experience with infrastructure-as-code, data observability, metadata, catalogue or lineage capabilities.
• Experience working with sensitive or regulated data, and an interest in using technology and data for public good.