Solution Architect

This role requires strong technical knowledge of enterprise data platforms, excellent stakeholder management skills, and proven experience delivering complex projects within the banking and financial services industry.

• 10+ years of experience.
• 5+ years managing Enterprise Data Lake, Data Warehouse or Big Data delivery initiatives.

• Proven experience in Hadoop/Cloudera environments.
• Experience managing multiple concurrent project deliveries and BAU portfolios.

• Banking and Financial Services experience preferred.

Data Engineering & Integration

• Design, develop, and maintain scalable data pipelines within the Enterprise Data Lake platform using Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.
• Develop high-performance batch and near real-time data ingestion frameworks for onboarding enterprise source systems into the Data Lake.

• Develop complex transformation logic using Spark, HiveQL, SQL, and Java-based processing frameworks.
• Design and implement data ingestion patterns supporting structured, semi-structured, and unstructured datasets.

• Build and maintain batch and near real-time data integration solutions.
• Implement source-to-target mappings and ingestion specifications.

Hadoop & Cloudera Platform Development

• Develop and support solutions within the Cloudera Data Platform ecosystem.
• Build and maintain data pipelines leveraging:

• Hadoop HDFS
• Hive

• Impala
• Spark

• Kafka
• YARN, Oozie

• Perform deployment, troubleshooting, and performance tuning of applications running on the Hadoop ecosystem.
• Partner with platform teams to ensure optimal cluster utilization and workload management.

Data Quality & Governance

• Ensure adherence to Data Lake onboarding standards and governance requirements.
• Investigate and resolve data quality issues and production incidents.

• Ensure compliance with enterprise data governance standards.

DevOps & Platform Automation

• Integrate data engineering solutions into CI/CD pipelines using Git, Jenkins, Azure DevOps, or equivalent tools.
• Automate deployment, testing, monitoring, and operational support activities.

• Develop reusable frameworks and utilities to improve engineering productivity and platform scalability.

Production Support & Incident Management

• Provide L3 support for Data Lake application.
• Perform root cause analysis for production incidents and implement permanent fixes.

• Monitor pipeline execution, cluster health, and workload performance.
• Participate in release implementation, deployment validation, and post-production support activities.

• Strong hands on experience with Hadoop ecosystem and data pipeline orchestration.
• Hadoop/HDFS

• Hive
• Spark

• Unix/Linux scripting
• Shell scripting

• ETL/ELT development experience.
• Data modelling fundamentals.

• Performance tuning and optimization.
• Experience in Data Warehousing and Data Lake environments.

• Strong Java / Python competency with experiences.
• Strong SQL development skills.