company logo

Big Data Engineer (AWS)

Bangalore Urban
Bangalore Rural
Bangalore
Full-Time
Junior: 3 to 5 years
Posted on Mar 10 2025

About the Job

Skills

Python
PySpark
databricks
Amazon Redshift
AWS Glue
AWS Lambda
Hadoop
SQL

Role : Big Data Engineer

Location : Bangalore

Job Type : Fulltime, Hybrid

Skills : Python, Spark, SQL, Databricks, Hadoop, Hive, Spark, AWS, DSA


Job Overview

We are seeking a Junior Big Data Engineer specializing in AWS for a full-time role located in Bangalore Urban, Bangalore Rural, Bangalore. As a key contributor in this role, you will leverage your expertise in big data technologies to design, implement, and optimize complex data processing solutions. Your contribution will be pivotal in enhancing our clients' data architectures and ensuring robust data pipelines.


Qualifications and Skills

  • Proficiency in Python (Mandatory skill) is required to efficiently handle data processing tasks and pipeline creation.
  • Experience with PySpark (Mandatory skill) for scalable data analytics and large data set processing is essential.
  • Expertise in Databricks (Mandatory skill) to collaborate and transform data in a cloud-based environment is necessary.
  • Strong knowledge of Amazon Redshift for creating and managing data warehouses and ensuring data storage efficiency.
  • Experience with AWS Glue for developing and managing ETL jobs to prepare data for analytics.
  • Familiarity with AWS Lambda for executing code in response to triggers, ensuring a flexible data processing environment.
  • Experience with Hadoop frameworks for processing large volumes of data and managing data storage effectively.
  • Advanced SQL skills for querying, managing, and structuring data stored in relational databases.


Roles and Responsibilities

  • Design and implement scalable data processing pipelines to handle large data sets and support business goals.
  • Collaborate with clients and team members to gather data requirements and ensure accurate data structuring.
  • Optimize data architectures for performance, reliability, and scalability within cloud environments.
  • Utilize AWS services to develop innovative data solutions tailored to client needs.
  • Conduct regular data monitoring and maintenance to ensure data integrity and security.
  • Analyze and interpret complex data sets, providing actionable insights to stakeholders.
  • Stay updated with emerging trends and technologies in big data engineering to drive continuous improvement.
  • Participate in code reviews and contribute to maintaining coding standards and practices.

About the company

Being one of the top staffing consultancy firms in Bangalore, VidPro provides manpower solutions to both IT and non-IT sectors. We have a track record of sourcing top talent for several startups PAN India and ensuring exceptional customer experience of all our clients. Our mission is to support our clients business objectives through continuous talent enhancement and skill augmentation. Our vision ...Show More

Industry

Human Resources Services

Company Size

11-50 Employees

Headquarter

Bengaluru

Other open jobs from VidPro Consultancy Services