Job Description:

We are looking for an experienced and motivated Lead Data Engineer to lead our data engineering team. As the Lead Data Engineer, you will play a critical role in designing, implementing, and managing our data infrastructure and analytics platforms. You will lead a team of data engineers, collaborate with cross-functional teams, and drive the development of innovative solutions to address complex data challenges. The ideal candidate will have a strong technical background in data engineering, leadership experience, and a proven track record of delivering scalable and reliable data solutions.

What you will do:

·        Lead and mentor a team of data engineers, providing technical guidance, coaching, and support.

·        Design, develop, and maintain scalable data pipelines and ETL processes to support data analytics and machine learning initiatives.

·        Architect, design, develop, and implement software applications using cutting-edge technologies and frameworks.

·        Take complete ownership of individual projects end-to-end. Should become recognized for technical expertise within the company and considered a “go-to” person within the area of focus and individual modules within projects.

·        Collaborate with stakeholders to define data architecture, design scalable data pipelines, and develop data models to support business objectives.

·        Oversee the development and maintenance of ETL processes, data warehouses, and data lakes to ensure data availability, reliability, and performance.

·        Troubleshoot and debug complex software issues, identify root causes, and implement effective solutions.

·        Stay current with emerging technologies and industry trends in data engineering and provide recommendations for technology adoption and innovation.

·        Choose and integrate suitable big data solutions (such as Hadoop, Spark, Kafka) and cloud environments (such as Azure, GCP).

·        Ensure data quality, security, and compliance with relevant regulations.

Who you are:

·        Bachelor's degree or higher in Computer Science, Engineering, or a related field.

·      7 to 10 years of experience in data engineering or a similar role, with at least 2 years of experience in a leadership or managerial capacity.

·        Proficiency in at least one programming language, such as Java, Python, or Scala.

·        Strong understanding of object-oriented programming principles and design patterns.

·        Experience working with relational databases and SQL.

·        Experience with big data technologies such as Hadoop, Kafka, Spark, NiFi, Hive etc,.

·        Experience with streaming technologies such as Spark Streaming, Flink, and others.

·        Experience with cloud platforms such as Microsoft Azure

·        Excellent leadership and communication skills, with the ability to influence and inspire team members.

·        Should have excellent experience in CICD frameworks and container-based deployments.

·        Ability to thrive in a fast-paced and dynamic environment, juggling multiple projects and priorities.