Senior Data & Analytics Technical Lead
Riachtanach:PythonGitCloudDataCI/CDFinTech
Job Summary
We are looking for a data engineer to design, develop, and support enterprise-scale data engineering solutions. The role will focus on data pipelines, ETL processing, big data platforms, data migration, data quality, and optimization across banking and financial services environments.
Key Responsibilities
- Design, develop, and optimize scalable data pipelines and ETL processes.
- Develop and maintain data processing solutions using Python, PySpark, SQL, Hive, and Spark.
- Work with Big Data platforms, including Hadoop HDFS, Hive, and cloud/object storage environments.
- Perform data extraction, transformation, and loading from multiple source systems.
- Design and implement data migration and modernization solutions involving DataStage, Talend, Informatica, and Hadoop.
- Develop and optimize batch processing and scheduling workflows using Autosys and Control-M.
- Perform data profiling, data quality checks, and performance optimization.
- Support data warehouse and dimensional data modelling activities.
- Implement CI/CD processes for data pipeline deployments.
- Work with business, application, and infrastructure teams to deliver data solutions.
- Provide technical leadership and production support for enterprise data platforms.
Requirements
- Degree in Computer Science, Information Technology, Data Engineering, or a related discipline.
- Strong experience in data engineering, ETL, data warehousing, and Big Data technologies.
- Strong hands-on experience in Python, PySpark, SQL, and Hive/HQL.
- Experience with Hadoop, Spark, and enterprise data platforms.
- Experience with Talend, DataStage, and/or Informatica.
- Experience with Teradata, Oracle, or SQL Server.
- Experience with Unix/Linux and shell scripting.
- Experience with CI/CD tools such as Git and Jenkins.
- Strong understanding of data modelling, data quality, and data pipeline optimization.
- Banking or financial services experience will be an advantage.
Preferred Skills
- Experience with Snowflake and Cloudera.
- Experience with Autosys, Control-M, or similar scheduling tools.
- Experience in large-scale data migration and technology modernization.
- Strong stakeholder management and technical leadership skills.