Data Engineer

Description

We are seeking an experienced data engineer with strong expertise in Python, microservices architecture, data transformation, and backend engineering to join our data engineering team. The ideal candidate will be responsible for designing and developing scalable backend services, data processing frameworks, RESTful APIs, and data transformation pipelines that support enterprise data-driven applications.

This role requires strong hands-on experience with Python-based web frameworks, database technologies, ETL pipelines, microservices, API security, and modern software engineering practices. Candidates should have experience building production-grade applications with a focus on scalability, reliability, security, and performance.

Requirements

Key responsibilities

Design, develop, and maintain scalable backend applications and web frameworks using Python.

Build robust RESTful APIs and microservices to serve enterprise datasets and business applications.

Develop and optimize data transformation pipelines to process, clean, enrich, and prepare data for analytics and business consumption.

Design and implement scalable data access layers integrating multiple relational and NoSQL databases.

Collaborate with frontend developers to integrate user-facing components with backend services.

Ensure applications are secure, highly available, performant, and maintainable.

Design and maintain database schemas and data models aligned with business requirements.

Implement ETL processes for extracting, transforming, and loading large-scale datasets.

Optimize database performance, query execution, and data retrieval mechanisms.

Troubleshoot production issues and implement scalable solutions for long-term platform stability.

Participate in code reviews, testing activities, technical design discussions, and engineering best practices.

Work closely with business stakeholders, architects, and engineering teams within Agile delivery environments.

Required technical skills

Python development

Strong hands-on experience in core Python development.

Expertise in building backend services and APIs using Python.

Strong understanding of object-oriented programming principles and software design patterns.

Experience writing clean, reusable, maintainable, and production-quality code.

Web frameworks & API development

Strong experience with one or more Python frameworks including:

FastAPI

Flask

Django

Pyramid

Experience with:

RESTful API design and development

API versioning and documentation

Routing and middleware implementation

Authentication and authorization

Framework testing and debugging

Microservices architecture

Strong experience designing and developing microservices-based applications.

Deep understanding of distributed systems architecture.

Experience building scalable and resilient service-oriented platforms.

Knowledge of service communication patterns and API orchestration.

Experience handling concurrent requests and high-volume transaction processing.

Asynchronous programming

Strong understanding of asyncio and asynchronous programming concepts.

Experience handling concurrent connections and non-blocking operations.

Knowledge of performance optimization for I/O-intensive applications.

Security & API protection

Strong understanding of API and microservices security best practices.

Experience implementing:

OAuth 2.0

JWT authentication

API gateway integrations

Encryption and secure communication protocols

Experience implementing authentication, authorization, and access control mechanisms.

Database & ORM expertise

Strong experience with relational and NoSQL databases.

Experience designing database schemas and optimizing database performance.

Strong proficiency with:

SQL

PostgreSQL

MySQL

MongoDB

Redis

Hands-on experience with ORMs including:

SQLAlchemy

Django ORM

Ability to:

Define models

Perform CRUD operations

Write optimized queries

Handle transactions

Manage database migrations

Data engineering & transformation

Experience building and maintaining ETL pipelines.

Strong understanding of data transformation, enrichment, aggregation, and cleansing techniques.

Experience ensuring data quality, consistency, and integrity.

Ability to support analytical and data-driven applications through scalable processing frameworks.

DevOps & CI/CD

Experience building and maintaining CI/CD pipelines.

Experience deploying and supporting production-grade applications.

Understanding of infrastructure automation and modern software delivery practices.

Experience with release management and deployment workflows.

Big data & data technologies

Exposure to technologies such as:

Hadoop

Apache Spark

Apache Kafka

Understanding of:

Data warehousing concepts

Distributed data processing

Enterprise data architectures

Version control & development practices

Strong experience with Git version control.

Experience with:

Branching strategies

Code reviews

Pull requests

Merge conflict resolution

Collaborative development workflows

Testing & debugging

Experience with:

Unit testing

Integration testing

Logging

Monitoring

Performance troubleshooting

Production support

Nice to have

Experience with AWS, Azure, or GCP.

Experience with Docker and Kubernetes.

Exposure to cloud-native architectures.

Knowledge of data governance, compliance, privacy, and security standards.

Experience supporting large-scale enterprise data platforms.

Required competencies

Strong analytical and problem-solving skills.

Excellent communication and stakeholder management capabilities.

Strong collaboration and teamwork skills.

Ability to work effectively in Agile/Scrum environments.

Strong ownership mindset with attention to detail.

Ability to document solutions and communicate technical concepts clearly.

Education

Bachelor’s degree in computer science, information technology, engineering, data engineering, or a related field.