Data Engineer

🎯 Job Summary

We are looking for an experienced Data Engineer with 7+ years of experience in designing, developing, and maintaining scalable data pipelines and data infrastructure. The candidate will be responsible for building ETL/ELT processes, managing large datasets, optimising data workflows, and supporting data analytics and business intelligence initiatives.

🔧 Key Responsibilities

• Design, develop, and maintain scalable data pipelines and data integration solutions.
• Build and manage ETL/ELT workflows for structured and unstructured data.
• Develop data models and optimise data warehouse solutions.
• Integrate data from multiple sources, including databases, APIs, and cloud platforms.
• Implement data processing solutions using batch and real-time streaming technologies.
• Ensure data quality, accuracy, consistency, and reliability.
• Work with data scientists, analysts, architects, and business stakeholders.
• Develop and optimise SQL queries and database performance.
• Implement data governance, security, and access control practices.
• Monitor, troubleshoot, and resolve data pipeline failures and performance issues.
• Work with cloud-based data platforms and distributed processing frameworks.
• Maintain technical documentation and follow software development best practices.

💻 Technical Skills & Requirements

• Strong proficiency in SQL and Python.
• Hands-on experience with ETL/ELT tools and data pipeline development.
• Experience with Apache Spark, PySpark, and Hadoop.
• Knowledge of Apache Airflow or similar workflow orchestration tools.
• Experience with data warehouses such as Snowflake, Amazon Redshift, Google BigQuery, or Azure Synapse.
• Hands-on experience with AWS, Microsoft Azure, or Google Cloud Platform.
• Knowledge of relational and NoSQL databases.
• Experience with data integration, data modelling, and data transformation.
• Familiarity with Kafka or other real-time data streaming technologies.
• Experience with Git, CI/CD, and automated deployment processes.
• Understanding of data governance, data security, and data quality frameworks.

🎓 Education & Experience

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related discipline.
• Minimum 7 years of relevant experience in Data Engineering.
• Proven experience developing and managing enterprise-scale data pipelines.
• Experience working with cloud data platforms and big data technologies.

🤝 Required Skills

• Strong analytical and problem-solving abilities.
• Excellent SQL and programming skills.
• Ability to handle large and complex datasets.
• Strong communication and collaboration skills.
• Ability to work independently and manage multiple priorities.

⭐ Preferred Certifications

• AWS Certified Data Engineer.
• Microsoft Certified: Azure Data Engineer Associate.
• Google Cloud Professional Data Engineer.
• Databricks Data Engineer Certification.