Responsibilities:

· Design, develop, and deploy highly scalable and efficient data pipelines utilizing containerization technologies (Docker/Kubernetes)

· Leverage AWS services like MSK (Managed Streaming Kafka) and S3 (Simple Storage Service) to build real-time data pipelines

· Implement and manage access controls using AWS IAM (Identity and Access Management) with Roles and Policies

· Work with Apache Kafka for high-throughput messaging and streaming data

· Utilize Debezium for Change Data Capture (CDC) from relational databases

· Automate infrastructure provisioning and management using Terraform (Infrastructure as Code)

· Configure and maintain Jenkins for continuous integration and continuous delivery (CI/CD) of data pipelines

· Collaborate with data scientists, analysts, and software engineers to ensure data pipelines meet business needs

· Monitor and troubleshoot data pipelines to ensure data quality and availability

Qualifications:

· 3-6 years of experience as a Data Engineer or related role

· Strong understanding of data warehousing, data pipelines, and data modeling principles

· Proficient in scripting languages like Python, Bash, or similar

· Experience with containerization technologies (Docker, Kubernetes) is a plus

· Experience with AWS cloud services (MSK, S3, IAM) is a plus

· Experience with Apache Kafka and Debezium is a plus

· Experience with Terraform and Infrastructure as Code (IaC) principles is a plus

· Experience with Jenkins and CI/CD practices is a plus

· Excellent communication and collaboration skills

· Ability to work independently and as part of a team