Company Overview

Career mantra is an educational advisory board that simplifies course and college selection for students pursuing UG and PG courses in India and abroad. Since 2008, we have empowered education seekers with reliable information on over 1500 institutions and 25,000 courses. By offering personalized experiences, seminars, and counseling support, we help students make informed decisions. Headquartered in Gwalior, we aim to bridge the educational gap for students worldwide with confidence and motivation.

Job Overview

We are seeking a Senior Data Engineer specializing in data migration to join our team. This is a remote, contract position based in Pune. The ideal candidate will have experience in managing data applications, transformation, and ingestion. By joining Career mantra, you will play a pivotal role in advancing our data infrastructure to support our mission in empowering educational decisions globally.

Qualifications and Skills

• Proven experience in big data application phases such as development and deployment (Mandatory skill).
• Expertise in data ingestion techniques using modern technologies (Mandatory skill).
• Strong abilities in preprocess and transformation of data for efficient analysis (Mandatory skill).
• Proficient in using AWS CloudFormation for deploying infrastructure as code in the cloud environment.
• Experienced in monitoring applications using AWS Cloudwatch for improved performance insight.
• Strong familiarity with designing APIs using GraphQL for streamlined data querying and manipulation.
• Competent in designing workflow automation using AWS Data Pipeline for data processing at scale.
• Knowledgeable of AWS Glacier for archival storage and cost-effective data management solutions.

Roles and Responsibilities

• Design and implement efficient data ingestion pipelines to collect and process large volumes of data from various sources. 
• Hands-on experience with AWS Database Migration Service for seamless data migration between databases. 
• Develop and maintain scalable data processing systems, ensuring high performance and reliability. 
• Utilize advanced data transformation techniques to prepare and enrich data for analytical purposes. 
• Collaborate with cross-functional teams to understand data needs and deliver solutions that meet business requirements. 
• Manage and optimize cloud-based infrastructure, particularly within the AWS ecosystem, including services such as S3, Step-Function, EC2, and IAM. 
• Experience with cloud platforms and understanding of cloud architecture. 
• Knowledge of SQL and NoSQL databases, data modelling, and data warehousing principles. 
• Familiarity with programming languages such as Python or Java. 
• Implement security and compliance measures to safeguard data integrity and privacy. 
• Monitor and tune the performance of data processing systems to ensure optimal efficiency. 
• Stay updated with emerging trends and technologies in data engineering and propose adaptations to existing systems as needed. 
• Proficient in AWS Glue for ETL (Extract, Transform, Load) processes and data cataloging. 
• Hands-on experience with AWS Lambda for serverless computing in data workflows. 
• Knowledge of AWS Glue Crawler Kinesis RDS for batch / real-time data streaming 
• Familiarity with AWS Redshift for large-scale data warehousing and analytics. 
• Skillful in implementing data lakes using AWS Lake Formation for efficient storage and retrieval of diverse datasets. 
• Experience with AWS Data Pipeline for orchestrating and automating data workflows. 
• In-depth understanding of AWS CloudFormation for infrastructure as code (IaC) deployment. 
• Proficient in AWS CloudWatch for monitoring and logging data processing workflows. 
• Familiarity with AWS Glue DataBrew for visual data preparation and cleaning. 
• Expertise in optimizing data storage costs through AWS Glacier and other cost-effective storage solutions. 
• Hands-on experience with AWS DMS (Database Migration Service) for seamless data migration between different databases. 
• Knowledge of AWS Athena for interactive query processing on data stored in Amazon S3. 
• Experience with AWS AppSync for building scalable and secure GraphQL APIs.

Qualification

• A minimum of 10 years of experience in data engineering or a related field. 
• Strong background in big data application phases, including data ingestion, preprocessing, and transformation.

Education

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. A Master's degree is preferred.