Company Overview

Career Mantra, founded in 2008, is a premier academic advisory board guiding students in UG and PG course selection in India and abroad. With a comprehensive database of over 1500 institutions and 25,000 courses, Career Mantra ensures a personalized experience through robust interaction and authentic information delivery. Headquartered in Gwalior, we are committed to bridging educational pathways and empowering students with the tools needed to achieve their dream academic and career goals.

Job Overview

We are seeking a Senior Data Engineer Migration to join our innovative and fast-paced team at Career Mantra in Pune. This is a remote, contract position where you will play a vital role in engineering data processes, ensuring seamless migrations and supporting critical phases of big data application development. The ideal candidate will be integral in designing efficient and robust data pipelines, as well as implementing data ingestion, preprocessing, and transformation solutions using cutting-edge technologies.

Qualifications and Skills

• Proven experience in developing and managing data solutions using big data technologies, including phases for application development, data ingestion, preprocessing, and transformation (Mandatory skill).
• Strong proficiency in AWS CloudFormation for defining infrastructure and deploying resources in the cloud environment.
• Solid understanding of AWS CloudWatch for monitoring and operational insights.
• Experience in designing and integrating GraphQL APIs for effective data retrieval and transformation.
• Hands-on experience with AWS Data Pipeline for orchestration and data workflow automation.
• Utilization of AWS Glacier for long-term data archiving and retrieval.
• Demonstrated ability to work remotely, independently, and collaboratively within a distributed team environment.
• Proficient analytical and problem-solving skills with an eagerness to learn and adapt to new technologies.

Roles and Responsibilities

• Design and implement efficient data ingestion pipelines to collect and process large volumes of data from various sources. 
• Hands-on experience with AWS Database Migration Service for seamless data migration between databases. 
• Develop and maintain scalable data processing systems, ensuring high performance and reliability. 
• Utilize advanced data transformation techniques to prepare and enrich data for analytical purposes. 
• Collaborate with cross-functional teams to understand data needs and deliver solutions that meet business requirements. 
• Manage and optimize cloud-based infrastructure, particularly within the AWS ecosystem, including services such as S3, Step-Function, EC2, and IAM. 
• Experience with cloud platforms and understanding of cloud architecture. 
• Knowledge of SQL and NoSQL databases, data modelling, and data warehousing principles. 
• Familiarity with programming languages such as Python or Java. 
• Implement security and compliance measures to safeguard data integrity and privacy. 
• Monitor and tune the performance of data processing systems to ensure optimal efficiency. 
• Stay updated with emerging trends and technologies in data engineering and propose adaptations to existing systems as needed. 
• Proficient in AWS Glue for ETL (Extract, Transform, Load) processes and data cataloging. 
• Hands-on experience with AWS Lambda for serverless computing in data workflows. 
• Knowledge of AWS Glue Crawler Kinesis RDS for batch / real-time data streaming 
• Familiarity with AWS Redshift for large-scale data warehousing and analytics. 
• Skillful in implementing data lakes using AWS Lake Formation for efficient storage and retrieval of diverse datasets. 
• Experience with AWS Data Pipeline for orchestrating and automating data workflows. 
• In-depth understanding of AWS CloudFormation for infrastructure as code (IaC) deployment. 
• Proficient in AWS CloudWatch for monitoring and logging data processing workflows. 
• Familiarity with AWS Glue DataBrew for visual data preparation and cleaning. 
• Expertise in optimizing data storage costs through AWS Glacier and other cost-effective storage solutions. 
• Hands-on experience with AWS DMS (Database Migration Service) for seamless data migration between different databases. 
• Knowledge of AWS Athena for interactive query processing on data stored in Amazon S3. 
• Experience with AWS AppSync for building scalable and secure GraphQL APIs.

Qualifications: 

• A minimum of 10 years of experience in data engineering or a related field. 
• Strong background in big data application phases, including data ingestion, preprocessing, and transformation. 

Education: 

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field. A Master’s degree is preferred.