Lead Data Engineer (Azure)

Requirements

5+ years of experience in Data Engineering, including hands-on development of ETL pipelines and data integration solutions

+2 years of experience as Data Engineer Lead

Advanced proficiency in SQL and Python for data processing, transformation, and optimization

Strong experience with Spark and PySpark for distributed data processing

Hands-on experience with Azure data technologies and cloud-based data platforms

Advanced proficiency with Azure Synapse, Azure Data Factory and/or Apache Airflow for workflow orchestration and scheduling

Strong understanding of Data Architecture, Data Modeling, Data Quality, Data Integration, and SQL and ETL performance optimization

Proven ability to provide technical guidance to engineering teams and serve as a technical reference throughout project delivery

Strong working proficiency in English for daily communication with international clients and teams

Client-facing, communication, analytical, and problem-solving skills

Responsibilities

Design and develop scalable ETL and data integration pipelines across Azure data environments

Own technical implementation and solution quality, ensuring reliable, scalable, and maintainable data processing

Collaborate with clients, Solution Architects, and engineering teams to translate business requirements into effective data solutions

Build and optimize data processing workflows using SQL, Python, Spark, PySpark, Azure Data Factory, and Apache Airflow

Develop data models and schemas aligned with business requirements, architecture standards, and downstream consumption needs

Implement data quality and validation controls to maintain data integrity and consistency across platforms

Optimize SQL queries, ETL processes, and data workflows to improve performance and reliability

Guide the Data Engineering team on technical decisions, engineering practices, troubleshooting, and successful solution delivery

Nice to have

Exposure to Microsoft Fabric, Hadoop, Hive, Kafka, Power BI, or Tableau as a plus

Experience with healthcare or pharmaceutical data environments as an advantage

Some of our benefits

SoftServe University, with learning opportunities, certification support and leadership opportunities to help you grow your skills

Medical insurance for you and your family

Food vouchers

Extra days off

Dare to Care, our holistic well-being program

Anniversary Program to celebrate your time with us

ABOUT THE ROLE

In this role, you will design and deliver scalable data solutions within Azure environments while serving as a technical reference for the engineering team. Combining hands-on Data Engineering with technical leadership, you will work closely with clients and architects to translate business needs into reliable solutions across the delivery lifecycle, with a focus on data integration, performance, and quality.