Job Title: Hybrid Position - Data Engineer

Location: Gurgaon

Overview:

We are seeking a talented and experienced Data Engineer to join our team in Gurgaon. The ideal candidate will have a strong background in data engineering, with expertise in SQL, Python, Scala, Spark, and AWS. The primary responsibility will be to develop and maintain data pipelines, optimize database and Spark job performance, and contribute to the overall success of our data infrastructure.

Key Responsibilities:

1. Develop and maintain data pipelines to extract, transform, and load data from various sources into SQL Server or Postgres databases.
2. Write complex SQL queries and PL/SQL procedures to manipulate and analyze large datasets efficiently.
3. Utilize Informatica ETL tools for data integration and transformation, understanding basic concepts and ensuring smooth data flow.
4. Develop Spark jobs in Scala to process large-scale datasets efficiently.
5. Optimize database performance, including query tuning and index optimization, to ensure optimal data retrieval speed.
6. Identify and resolve issues related to Spark job performance, such as data skewness and inefficient partitioning.
7. Collaborate with cross-functional teams to understand data requirements and implement effective solutions.
8. Stay updated with emerging technologies and best practices in data engineering, particularly in AWS technologies.

Requirements:

• Bachelor's degree in Computer Science, Engineering, or related field.
• X years of experience working as a Data Engineer or similar role.
• Proficiency in SQL Server or Postgres, with hands-on experience in writing complex SQL queries and PL/SQL procedures.
• Strong programming skills in Scala, with the ability to write efficient Spark jobs.
• Experience with Informatica ETL tools is a plus.
• Good understanding of database performance optimization techniques.
• Familiarity with AWS technologies is preferred.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.