Sr Data Specialist - R01564133

Lead Data Engineer

Primary Skills Athena, Step Functions, Spark - Pyspark, ETL Fundamentals, SQL (Basic + Advanced), Glue, Python, Lambda, Data Warehousing, EBS /EFS, AWS EC2, Lake Formation, Aurora, S3, Modern Data Platform Fundamentals, PLSQL, Data Modelling Fundamentals, Cloud front

Specialization AWS Data EngineerIng Basic: Senior Data Engineer

Job requirements Job Description – AWS Senior Data Engineer

Company: Brillio Technologies Role: Senior Data Engineer Experience: 5–10 Years Location: India (Hybrid/Remote) Employment Type: Full-time

Role Overview

We are looking for a Senior Data Engineer with strong Databricks and Big Data expertise to design, build, and optimize scalable data pipelines and platforms. The ideal candidate will have hands-on experience with Spark, PySpark, AWS ecosystem, and modern data architectures , along with exposure to APIs and real-time data processing.

Key Responsibilities

Design and build scalable data pipelines using Spark / PySpark on Databricks

Develop and optimize ETL/ELT workflows for large-scale data processing

Work with AWS EMR, S3, and Hadoop ecosystem for distributed data processing

Build and maintain data lake and data warehouse solutions

Develop and integrate APIs for data ingestion and consumption

Implement data processing workflows using Airflow / Autosys

Optimize data performance using partitioning, caching, and query tuning

Handle structured and semi-structured data from multiple sources

Ensure data quality, governance, and reliability of pipelines

Collaborate with cross-functional teams (Analytics, Product, Engineering)

Technical Skills (Must-Have)

Strong expertise in:

Apache Spark / PySpark

Databricks (mandatory)

SQL (PostgreSQL or similar)

Hands-on experience with:

AWS EMR, S3

Hadoop, Hive ecosystem

Programming skills:

Python (must-have)

Scala (good exposure)

Strong knowledge of:

UNIX / Shell scripting

ETL pipelines and data engineering fundamentals

Additional Skills

Experience with:

Elasticsearch (data storage & retrieval)

Workflow orchestration tools (Airflow, Autosys)

Exposure to:

API development & integration

Version control tools ( Git / SVN )

Basic knowledge of HTML (for web/data tasks)

Experience Required

Minimum 5+ years of Data Engineering experience

Strong hands-on experience in Databricks + Big Data stack

Proven experience working on large-scale distributed systems

Education

Bachelor’s or Master’s degree in Computer Science, Engineering, or related field

Why Join Brillio

Work on cloud-native, large-scale data platforms

Exposure to cutting-edge Databricks & AWS ecosystems

Collaborative and high-performance engineering culture