Company Overview

Career Mantra is a leading organization in the education industry, headquartered in Pune, specializing in career counselling, skill development, and recruitment services. We aim to bridge the gap between talent and opportunity by providing personalized guidance, resume building, interview preparation, and corporate training to empower individuals towards achieving their career goals.

Job Overview

Career Mantra is seeking a highly skilled Senior Data Engineer for a remote, full-time position based in Pune. The successful candidate will be responsible for developing, constructing, testing, and maintaining architectures, such as databases and large-scale processing systems. This role will be instrumental in enhancing our data processing capabilities to deliver effective solutions to our clients.

Qualifications and Skills

• Proven experience in ETL processes and data pipeline development (Mandatory skill).
• Expertise in using Azure Databricks for data engineering tasks (Mandatory skill).
• Comprehensive understanding of data modeling concepts and methodologies to optimize database efficiency.
• Proficiency in handling big data technologies, including Hadoop and Spark, to manage and analyze large volumes of data.
• Strong background in data warehousing concepts and technologies to integrate diverse data sources.
• Experience working with various cloud platforms such as AWS, Azure, or Google Cloud for deploying and managing data storage solutions.
• Ability to write and optimize complex SQL queries for data retrieval and manipulation.
• Excellent problem-solving skills and the ability to work independently as well as collaboratively in a team.

Roles and Responsibilities

• Design and implement effective database solutions and models to store and retrieve data efficiently.
• Develop and maintain scalable data pipelines and ETL processes to support ongoing data analytics.
• Collaborate with data analysts and data scientists to provide data-driven insights and solutions.
• Utilize Azure Databricks and big data technologies to process and analyze large datasets.
• Conduct research and provide recommendations on emerging data engineering trends and technologies.
• Ensure data integrity and security while maintaining compliance with industry standards and regulations.
• Optimize and maintain codebase and processes to improve data reliability, efficiency, and scalability.
• Document data flows, processes, and architectural designs to facilitate knowledge sharing and maintenance.