Architect Lead - Data engineering
Must-have:CloudDataAILead
Job Description
We are looking for an experienced Architect Lead – Data Engineering to lead the design and implementation of enterprise-scale data platforms, data pipelines and cloud-based data solutions.
Key Responsibilities:
- Lead the architecture and design of scalable data engineering and data platform solutions.
- Design and develop high-volume data pipelines using Apache Spark and PySpark .
- Implement data warehousing and analytics solutions using Snowflake .
- Design workflow orchestration using Apache Airflow .
- Develop modern data lake/lakehouse solutions using Apache Iceberg .
- Design real-time and event-driven data pipelines using Apache Kafka .
- Develop ETL/ELT solutions using DBT and enterprise ETL frameworks .
- Work with GBI ETL Framework and ensure integration across enterprise data platforms.
- Leverage cloud computing technologies to build secure, scalable and highly available data solutions.
- Provide technical leadership, architecture guidance and mentoring to data engineering teams.
- Drive performance tuning, data quality, scalability and reliability of data platforms.
- Collaborate with business and technology stakeholders to translate requirements into technical solutions.
- Explore and implement appropriate Artificial Intelligence capabilities within data platforms.
Requirements:
- Minimum 10 years of experience in data engineering, big data or data architecture.
- Strong hands-on experience with Spark, PySpark, Snowflake, Airflow, Kafka and Iceberg .
- Strong experience in ETL/ELT frameworks and DBT .
- Experience with cloud-based data platforms and modern data architecture.
- Experience designing large-scale data pipelines and lakehouse architectures.
- Exposure to Artificial Intelligence / AI-based data solutions.
- Strong analytical, problem-solving, communication and technical leadership skills.
- Degree in Computer Science, Information Technology, Engineering or a related discipline.