Python & Apache Spark Engineer / Data Processing Specialist

About the Project You will join a dynamic technology team responsible for designing, developing, and deploying innovative enterprise technology and AI-driven solutions supporting the delivery of tax services. The team combines expertise across software engineering, data engineering, AI, tax technology, change management, and project management . The projects cover the full technology lifecycle — from solution design and development through deployment and optimization, to training, adoption, and stakeholder engagement. You will work in a modern, cloud-based environment and contribute to the development of scalable data processing solutions, with a strong focus on Python, Apache Spark, cloud technologies, microservices, and modern data platforms . Technology Stack Cloud: Microsoft Azure

Data Processing: Apache Spark, Databricks, Azure Synapse / Microsoft Fabric

Backend: Python, .NET 8, ASP.NET Core

Data: MongoDB, Azure SQL, Parquet, Delta Tables

Frontend: Angular 18, Kendo

AI / GenAI: LangGraph, LangChain, RAG Pipelines, Multi-modal LLMs

Architecture: Microservices

DevOps & Collaboration: GitHub Enterprise, GitHub Copilot

Containerization: Docker, Kubernetes

Daily tasks

• Software & Data Engineering
• Design, develop, and maintain robust, scalable, and production-ready Apache Spark applications.
• Develop efficient data processing, transformation, and analysis workflows using Python and Spark.
• Build solutions capable of handling large-scale datasets while maintaining high performance and reliability.
• Work with columnar data formats, particularly Parquet, and implement data solutions using Delta Tables.
• Develop clean, maintainable, testable, and efficient code in accordance with established engineering standards.
• Optimize Spark applications, data processing pipelines, and code for performance, scalability, and efficient resource utilization.
• Analyze and resolve technical issues affecting application reliability, performance, and data processing.
• Contribute to the design and implementation of scalable microservices-based solutions.
• Technical Leadership & Code Quality
• Define, promote, and enforce software engineering best practices and coding standards across the project.
• Conduct detailed code reviews, ensuring code quality, maintainability, security, performance, and compliance with project standards.
• Identify potential technical risks and proactively propose improvements.
• Support the team in solving complex technical and architectural problems.
• Provide technical guidance to other team members and help establish consistent development practices.
• Contribute to continuous improvement of development processes, workflows, and engineering standards.
• Collaboration & Stakeholder Management
• Work both independently and collaboratively within a distributed, cross-functional team.
• Cooperate closely with software engineers, data specialists, architects, project managers, and business stakeholders.
• Communicate clearly with team members, coordinate day-to-day technical activities, and help ensure smooth delivery.
• Maintain regular communication with the client and participate in technical discussions and project meetings.

Requirements

Professional Skills Strong analytical and problem-solving capabilities.

Detail-oriented approach with the ability to identify potential technical issues and bottlenecks.

Practical and pragmatic mindset - able to balance standardized processes with the flexibility required to achieve project objectives.

Strong organizational and time-management skills.

Ability to independently manage workload, prioritize tasks, and meet tight deadlines.

Excellent communication skills and the ability to explain technical topics clearly to both technical and non-technical stakeholders.

Comfortable working independently while also being an active member of a collaborative, international team.

Willingness to take ownership and proactively drive technical topics forward.

Nice to Have Experience with Microsoft Azure or other major cloud platforms.

Practical knowledge of Azure services, particularly: Azure Service Bus

Azure Data Lake

Azure Blob Storage

Azure Redis

Other Azure SaaS/PaaS services

Experience building APIs and backend services with FastAPI .

Hands-on experience with Docker .

Experience with Kubernetes and container orchestration.

Familiarity with modern AI/GenAI technologies , including LangChain, LangGraph, RAG pipelines, or LLM-based applications.

Experience with Azure Synapse Analytics or Microsoft Fabric .

Experience working in enterprise environments with complex data processing requirements.

Must have: Analytical skills, Problem-Solving, Organizational skills, Time management, Communication skills, Python, .NET, Apache Spark, ASP.NET CORE, MongoDB, Azure SQL, Angular, Kendo

Nice to have: Microsoft Azure, Azure, Azure Service Bus, Azure Data Lake, Azure Blob Storage, Azure Redis, Azure SaaS, PaaS, FastAPI, Docker, Kubernetes, AI, GenAI, LangChain, LangGraph, RAG Pipelines, Azure Synapse Analytics, Microsoft Fabric