ML/AI Engineer (Public Sector)

Your tasks

The ML/AI Engineer is part of the Data Platform and Architecture Team. You will productionise, operate and improve machine-learning and AI workloads on the SWDA data platform using AWS and Databricks.

This role combines ML/AI engineering with practical data engineering. Working closely with data scientists, data engineers, platform engineers, product teams and governance stakeholders, you will build reliable data and feature pipelines, prepare production-ready datasets, and establish reusable MLOps and AIOps capabilities. Your work will support secure, observable and cost-effective AI services throughout their lifecycle, from data ingestion and experimentation to deployment, monitoring, retraining and retirement.

• MLOps and AIOps platform engineering: Design and implement reusable platform components and standard workflows for experiment tracking, model registry, pipeline orchestration, deployment, monitoring, feedback and retraining using Databricks, MLflow and approved AWS services.
• Production engineering: Turn notebooks and prototypes into modular, tested, containerised and maintainable batch jobs, APIs, model-serving endpoints or agentic services, with clear interfaces and operational ownership.
• Feature engineering pipelines: Build and maintain reliable pipelines on Databricks and AWS to ingest, transform, validate and serve structured and unstructured data for ML/AI workloads.
• Performance and cost engineering: Optimise Spark and SQL jobs, file layouts, partitioning, cluster configurations and workload schedules to improve reliability and runtime, while managing AWS and Databricks costs.
• CI/CD/CT automation: Build automated workflows to test and promote code, data, features, prompts, models and infrastructure across environments. Include quality, security and evaluation checks, as well as human approval gates.
• Reproducibility and governance: Implement versioning for source code, data, models, prompts and environments. Capture lineage, evaluation evidence, approvals and deployment history, and provide rollback and retirement mechanisms.
• Monitoring and reliability: Build dashboards and alerts for system health, latency, throughput, errors, resource use, cost, data quality, drift and model performance. Maintain runbooks and support incident response and root-cause analysis.
• Evaluation and experimentation: Develop repeatable offline and online evaluation, regression testing, champion–challenger comparisons and A/B testing patterns, with agreed release and rollback criteria.
• Security and responsible AI: Apply least-privilege IAM, encryption, secrets management, network and environment controls, vulnerability scanning, audit logging, sensitive-data controls, and safeguards for untrusted AI inputs and tool use.
• Recommendation and search support: Support maintenance and retraining, package and dependency updates, new data ingestion, experimentation, filter and logic changes, and performance reporting for SWDA recommendation and search workloads.
• Continuous improvement: Improve scalability, resilience, deployment frequency and cost efficiency; reduce manual steps and duplicated patterns; and contribute to the team’s MLOps maturity roadmap.
• Documentation and capability transfer: Produce architecture decisions, model cards, release evidence, operational runbooks and user guidance. Coach DSAD teams so they can operate supported solutions sustainably.

Requirements

Education and Experience

• Education: Degree in Computer Science, Artificial Intelligence, Data Science, Software Engineering, Information Systems or a related technical discipline.
• Experience: Typically 4–7 years in software, data or ML engineering, including hands-on delivery and operation of ML/AI systems beyond the prototype stage. Candidates with fewer years of experience may also be considered if they can demonstrate strong end-to-end production ownership.

Essential Technical and Professional Competencies

• Strong Python and software engineering skills, including APIs, packaging, automated testing, Git, code review, debugging and modular design.
• Hands-on experience with Databricks and MLflow for experiment tracking, model registry, workflow orchestration or model serving.
• Strong hands-on experience with Spark/PySpark and SQL, including distributed data processing, query optimisation, partitioning, joins, incremental processing and pipeline troubleshooting.
• Experience designing and implementing lakehouse or data-lake pipelines using Databricks, Delta Lake and AWS S3, including Bronze, Silver and Gold data-layer patterns.
• Experience with data modelling, schema design, data contracts, metadata, lineage, data-quality validation and reconciliation for production analytics and ML datasets.
• Experience orchestrating batch or streaming workflows using Databricks Workflows, AWS Glue, Step Functions, EventBridge, Apache Airflow or an equivalent enterprise orchestration tool.
• Hands-on AWS data and cloud engineering experience with a relevant mix of services, such as S3, Glue, Lake Formation, IAM, KMS, ECR, ECS/EKS, Lambda, Step Functions, EventBridge, CloudWatch, API Gateway, Secrets Manager, Redshift, MSK/Kinesis or equivalent approved services.
• Experience with Docker, infrastructure as code such as Terraform, and CI/CD platforms such as GitLab CI, GitHub Actions, Jenkins or Harness.
• Understanding of ML lifecycle controls covering data validation, feature engineering, training, evaluation, deployment, monitoring, drift, retraining and retirement.
• Ability to ensure reproducibility and traceability across code, datasets, features, parameters, environments, prompts and model artefacts.
• Experience implementing system and model observability, alerts, runbooks, incident diagnosis, and rollback or recovery procedures.
• Working knowledge of cloud security, least-privilege access, encryption, secrets management, vulnerability management and audit logging.

Preferred Experience

• Databricks Unity Catalog, Delta Live Tables or Lakeflow; feature engineering or feature stores; change data capture; event streaming with MSK, Kinesis or Kafka; and cross-account or governed data-sharing patterns.
• Productionising generative AI or agentic systems, including orchestration, state, prompt and version management, evaluation, controlled tool use, guardrails and human approval.
• Kubernetes, OpenTelemetry, Prometheus, Grafana, model drift tooling, A/B testing or champion–challenger deployment.
• Supporting high-volume, low-latency or public-facing AI services, and optimising cloud inference and training costs.
• Experience in Singapore public-sector environments, including VAPT, IM8-aligned controls, data classification, system assurance or responsible-AI governance.

Soft Skills and Communication

• Strong engineering judgement and a pragmatic mindset, with the ability to distinguish a supportable production service from a demonstration.
• Structured problem-solving across data, model, application and infrastructure layers, including calm, evidence-based incident handling.
• Ability to work across data engineering and ML engineering, translating model requirements into dependable datasets, features, interfaces and service-level expectations.
• Clear written and verbal communication, including the ability to explain architecture, evaluation evidence, risk, cost and operational trade-offs to technical and management audiences.
• Collaborative approach when working with data scientists, data engineers, platform engineers, product teams, cybersecurity and governance stakeholders.
• Ability to work independently, document decisions, manage ambiguity and deliver through incremental, testable outcomes.
• Commitment to knowledge transfer, reusable engineering patterns and sustainable in-house capability.

Other

• 2-year contract, renewable
• Hybrid work arrangement
• Government project