Technical Lead Big Data Engineer (AWS)

Requirements

5+ years of experience in Big Data or Data Engineering, including supporting production data platforms

Strong proficiency in Python, PySpark, and Apache Spark for large-scale data processing

Hands-on experience with AWS data services such as Glue, S3, Athena, Aurora PostgreSQL, and Lambda

Experience with Apache Airflow for data pipeline orchestration and scheduling

Strong knowledge of SQL, data ingestion, ETL processes, and data quality practices

Experience with Docker, Terraform, and CI/CD pipelines using GitHub Actions or similar tools

Familiarity with Python development and testing tools such as pytest, Pydantic, ruff, or mypy

Understanding of healthcare data regulations such as HIPAA/HITRUST, multi-tenant architectures, or data retention requirements is a plus

Readiness to travel to the United States for a three-month business trip in Q1 2027

Upper-intermediate or higher level of English, with strong communication and problem-solving skills

Responsibilities

Support, maintain, and enhance an existing AWS-based data platform and data ingestion pipelines

Develop and optimize data processing workflows using Python, PySpark, Apache Spark, and AWS Glue

Monitor and troubleshoot data pipelines, ensuring reliability, performance, and data quality

Work with AWS services such as S3, Aurora PostgreSQL, Athena, Lambda, SQS, SNS, and EventBridge

Maintain and improve workflow orchestration using Apache Airflow

Support infrastructure and deployment processes using Docker, Terraform, and Amazon EKS

Apply software engineering best practices, including automated testing, code quality checks, and CI/CD processes

Utilize AI-assisted development tools such as Claude Code to support implementation, testing, and code reviews

Collaborate with remote engineering teams to resolve technical issues, implement improvements, and ensure platform stability

Support data security, governance, and compliance practices within a regulated healthcare environment

Some of our benefits

SoftServe University: Access continuous learning opportunities, certification support, and leadership development programs to grow your skills and advance your career

Medical insurance: Health coverage for you and your family

Food vouchers: Additional support for your everyday expenses

Extra days off: Enjoy additional time off to rest, recharge, and maintain a healthy work-life balance

Dare to Care: Our holistic well-being program, designed to support your overall wellness

Anniversary Program: Celebrating your milestones and recognizing your journey with SoftServe

SoftServe is a global digital services and consulting company with over 30 years of experience and more than 11,000 professionals across 16 countries, helping businesses worldwide solve complex challenges through technology.

Our Data & Analytics Center of Excellence (CoE) brings together data specialists to drive excellence in data, analytics, and business intelligence. We deliver solutions across Data Strategy & Governance, Data Platform Engineering, Data Migration, and Reporting & BI.

In this role, you’ll support and enhance a modern AWS-based data platform, focusing on data ingestion, processing, and platform reliability. You’ll use Python, PySpark, and Apache Spark to maintain and optimize existing data pipelines while collaborating with a distributed engineering team. You’ll work with healthcare data in a regulated environment, and the role includes a three-month business trip to the United States in Q1 2027.