Data Engineer

Your tasks

• Design, build and maintain efficient and reliable large-scale batch and real-time data pipelines
• Utilize data processing/engineering frameworks for data ingestion, cleaning, transformation to ensure the data pipelines are efficient, reliable and scalable
• Design, develop and deploy data tables in data lake / data warehouses.
• Design and develop backend APIs and database integrations to support applications/use cases.

Requirements

Prerequisites

• Degree or Diploma in Computer Science, Computer Engineering, Information Technology, Business Analytics or related disciplines.
• Minimum 2 - 8 years’ experience in designing, developing and/or maintaining data platforms.
• Played the role of a technical lead to a team of data engineers.
• Spark, Scala/Pyspark
• Big data frameworks/tools (e.g. Hadoop, Azure Data Factory, Azure Synapse)
• Data platform and data structure/model design and management (e.g. Data Mart, DataLake, Data Virtualization and Data Warehousing)
• Experience with building production-grade data pipelines, ETL/ELT data integration in a big data environment using Cloud platforms such as Azure.
• Experience working with Agile teams with strong collaboration skills.
• Knowledge/experience on data governance policies, access control and security best practices.
• Stakeholders’ engagement/management and requirements analysis skills.