Data Engineer - Databricks & Automation

U3 InfotechSingaporemycareersfuturepublished 11/19/2026
Must-have:PythonDataCI/CDAI

Key Responsibilities

  • Design, develop and maintain scalable ETL/ELT data pipelines using Databricks, Python, PySpark, SQL and Delta Lake.
  • Integrate data from multiple sources, including databases, APIs and files, with streaming ingestion where required.
  • Develop and maintain curated datasets and data models to support business reporting, analytics and AI applications.
  • Implement data quality checks, data validation, reconciliation and incremental data processing.
  • Monitor data pipelines, investigate failures, troubleshoot issues and ensure data accuracy and availability.
  • Optimise data processing performance, pipeline execution and compute utilisation.
  • Implement version control, automated testing, CI/CD and technical documentation.
  • Identify and automate repetitive data engineering and support activities.
  • Explore practical AI