Company Overview

Career Mantra is a leading organization in the education industry, dedicated to empowering individuals to reach their career aspirations through tailored career counseling, skill development, and recruitment services. With a focus on bridging the gap between talent and opportunity, Career Mantra aids job seekers and businesses in their growth and success. Operating from its headquarters in Pune, the company employs between 51-200 individuals and is committed to professional excellence in career services.

Job Overview

We are seeking an experienced Senior Data Engineer to design, develop, and maintain scalable data pipelines and optimize data storage solutions. You will work with cloud platforms, big data technologies, and ETL processes to support business intelligence and analytics.

Key Responsibilities:

Design and implement scalable data pipelines using Python, PySpark, and Big Data tools.

Optimize data processing performance in AWS.

Collaborate with business teams to deliver reliable data solutions.

Develop and maintain ETL processes using tools like Airflow, Azkaban, and AWS Glue.

Troubleshoot and ensure data integrity across data pipelines.

Mentor junior team members and promote best practices.

Key Skills:

Strong experience with Python, PySpark, SQL, AWS (Redshift, Glue, S3).

Expertise in big data technologies (Hadoop, Hive, Spark).

Proficient in data modeling and ETL orchestration.

Experience with business intelligence tools (Tableau, Power BI).

Strong problem-solving and communication skills.

Qualifications and Skills

• Extensive experience with data mapping (Mandatory skill), facilitating accurate data translation and transformation across systems.
• Proficient in ETL processes (Mandatory skill) for efficient extraction, transformation, and loading of data.
• Strong knowledge of API Integration, enabling smooth communication between different software solutions.
• Expertise in System Integration, ensuring harmonious operation of various IT systems and platforms.
• Experience with Middleware Technologies to facilitate connectivity and collaboration between applications.
• Competency in Scripting Languages for automating routine processes and enhancing system functions.
• Proven Database Management skills to ensure data integrity and performance optimization.
• Adept at Troubleshooting to identify, diagnose, and resolve technical issues promptly.

Roles and Responsibilities

• Design and implement solutions for complex data integration projects to enhance operational efficiency.
• Collaborate with cross-functional teams to ensure smooth integration and alignment of IT systems.
• Develop and execute testing plans to verify the functionality and performance of integrated systems.
• Monitor system performance and suggest necessary improvements for optimal operation and stability.
• Serve as a technical expert, providing guidance and support to junior team members and stakeholders.
• Maintain comprehensive documentation of integration processes, configurations, and best practices.
• Stay current with technology trends and advancements to incorporate industry best practices.
• Ensure compliance with company policies and standards in all integration activities.