AI Engineer – LLMOps & Azure

Responsibilities

• Monitor production AI services for availability and latency while managing incident root-cause analysis and resolution.
• Maintain evaluation sets and track answer quality to detect and mitigate performance regressions.
• Operate ingestion pipelines and manage the integration of new content sources and business domains.
• Oversee agent interactions and monitor tool calls to ensure seamless integration with business systems.
• Manage model version upgrades and configuration changes through validated deployment pipelines.
• Track token consumption and service costs to propose architectural and operational optimizations.
• Maintain infrastructure consistency using Terraform and Helm across multiple environments.

Requirements

• 3+ years of experience in AI development using Python and LangChain.
• 3+ years of experience with Azure services, specifically Azure OpenAI, Azure AI Search, and Azure API Management.
• 3+ years of experience managing containerized environments using Azure Kubernetes Service (AKS).
• 3+ years of experience in Infrastructure as Code with Terraform and Helm.
• Proven track record of operating RAG systems or AI agents in a production setting.
• Hands-on experience with MLflow, Databricks, or OpenTelemetry for monitoring and tracing.
• Practical knowledge of Azure DevOps for CI/CD pipeline management.
• Experience with Vector databases and data ingestion workflows.
• Proactive and analytical approach to troubleshooting complex technical issues.
• You are fluent in English with a level of Native Dutch or Native French.

Nice to Haves

• Experience with data protection impact assessments and access control logic.
• Understanding of prompt engineering best practices for production reliability.