Responsibilities:

• Design and implement scalable and robust ELT processes using Python, Scala, and Apache Spark.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure the timely delivery of high-quality data solutions.
• Optimize and troubleshoot data-related issues to ensure data quality, reliability, and performance.
• Implement best practices for data security, privacy, and compliance.
• Work closely with cross-functional teams to integrate data solutions into existing systems and processes.
• Continuously evaluate and implement emerging technologies to enhance data processing efficiency.

Qualifications:

• Bachelor's or higher degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Data Engineer with a focus on designing and developing ELT processes using Python, Scala, and Spark.
• Solid understanding of data modeling, database design, and data warehousing concepts.
• Experience with ETL (Extract, Transform, Load) processes and tools.
• Strong knowledge of distributed computing and big data processing frameworks.