Data Engineer – Duties & Responsibilities

 Design and implement scalable data pipelines to process and transform large volumes of

data using AWS services such as Glue, Kafka, S3, EMR, Athena.

 Collaborate with cross-functional teams including data scientists, analysts, and software

engineers to understand data requirements and deliver solutions that meet business

needs.

 Optimize data workflows for performance, reliability, and cost-efficiency on AWS cloud

platform. Performance Tune ETL Pipelines, database queries.

 Develop and maintain ETL processes to ensure data quality and integrity throughout the

data lifecycle.

 Implement best practices for data security, access control, and compliance in

accordance with industry standards and regulations.

 Continuously monitor and troubleshoot data pipelines to identify and resolve issues in a

timely manner.

 Stay updated with the latest AWS services and technologies and evaluate their potential

for improving our data infrastructure.

 Document data architecture, design decisions, and technical specifications for reference

and knowledge sharing.

Data Engineer – Experience & Qualifications:

 Design and implement scalable data pipelines to process and transform large volumes of

data using AWS services such as Glue, Kafka, S3, EMR, Athena.

 Collaborate with cross-functional teams including data scientists, analysts, and software

engineers to understand data requirements and deliver solutions that meet business

needs.

 Optimize data workflows for performance, reliability, and cost-efficiency on AWS cloud

platform. Performance Tune ETL Pipelines, database queries.

 Develop and maintain ETL processes to ensure data quality and integrity throughout the

data lifecycle.

 Implement best practices for data security, access control, and compliance in

accordance with industry standards and regulations.

 Continuously monitor and troubleshoot data pipelines to identify and resolve issues in a

timely manner.

 Stay updated with the latest AWS services and technologies and evaluate their potential

for improving our data infrastructure.

 Document data architecture, design decisions, and technical specifications for reference

and knowledge sharing.

Data Engineer – Educational Qualifications & Skills

 Bachelor or Masters degree in Computer Science Engineering, or related field.

 Proven experience as a Data Engineer or similar role, with at least 5 years of experience

in building and maintaining data infrastructure on AWS cloud platform.

 Strong proficiency in AWS services such as Glue, Redshift, Lambda, Athena, S3,

QuickSight etc.

 Experience with big data technologies such as Hadoop, Spark, Kafka, etc.

 In-depth knowledge of data architecture principles, data modelling, and database

design.

 Hands-on experience with SQL, Python, and any other programming languages for data

manipulation and scripting.

 Familiarity with DevOps practices and tools for CI/CD, infrastructure as code, and

automated deployment.

 Knowledge of machine learning concepts and techniques for data analysis and

predictive modelling.

 Excellent problem-solving skills and attention to detail, with a focus on delivering high-

quality solutions.

 Ability to work effectively in a fast-paced environment with tight deadlines and changing

priorities.

 Strong communication and collaboration skills, with the ability to work effectively in a

cross-functional team environment