Data Engineer (Iceberg, Snowflake and Python)
Must-have:PythonGitCloudDataCI/CDAISecurity
Must have : Data engineer with Iceberg, Snowflake and Python
Job Responsibilities
- Design, develop, and implement scalable data engineering solutions to support PIPL program requirements.
- Build and maintain end-to-end ETL/ELT pipelines using Python, SQL, and modern data engineering frameworks.
- Develop and manage data lakehouse architectures using Apache Iceberg and Snowflake.
- Create and support a dedicated China data environment on Alibaba Cloud (AliCloud) for data processing, analytics, machine learning, and reporting workloads.
- Automate data ingestion, transformation, validation, and reporting processes to improve operational efficiency.
- Optimize data pipelines, storage, and query performance to ensure scalability and reliability.
- Collaborate with architects, program leads, and business stakeholders to gather requirements and deliver data solutions.
- Ensure data quality, governance, security, and compliance with PIPL and enterprise standards.
- Monitor, troubleshoot, and resolve data platform and pipeline issues.
- Contribute to best practices in DataOps, CI/CD, automation, and cloud-native data engineering.
Job Requirements
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
- 5+ years of experience in Data Engineering, Data Warehousing, or Big Data environments.
- Strong hands-on experience with Apache Iceberg.
- Proven experience working with Snowflake for data warehousing and analytics.
- Advanced proficiency in Python and SQL development.
- Experience designing and building ETL/ELT pipelines and data integration solutions.
- Solid understanding of data lake, lakehouse, and data warehouse architectures.
- Experience working with cloud platforms, preferably Alibaba Cloud (AliCloud).
- Knowledge of distributed data processing technologies such as Apache Spark, Flink, or similar frameworks.
- Experience with data modeling, performance tuning, and data quality management.
- Familiarity with CI/CD pipelines, Git, and automation tools.
- Strong analytical, problem-solving, and stakeholder management skills.
- Experience in regulatory or data privacy-focused environments (PIPL/GDPR) is an added advantage.
Must-Have Skills:
- Apache Iceberg, Snowflake, Python, SQL, ETL/ELT Development, Data Engineering, Alibaba Cloud (preferred), Data Lakehouse Architecture.