Data Engineer

Total Experience:           6+ yrs                  

Preferred domain:          Real Estate

Location:           PAN India (WFH / Remote)

Skills:  Python, Pandas, PySpark, REST API integration, SQL, JSON, AWS, Scripting for ETL, IaC.

Education:       Bachelor’s degree in Data Science / Statistics / Computer Science or related field.

Mandatory Skills:

❖ Python – Scripting for ETL, Automation & REST API integration

❖ SQL (Athena / RDS) – Query writing, Optimization & Schema design

❖ Pandas, PySpark – Data transformation, Aggregation & processing

❖ JSON & REST API – Integration for data retrieval & automation

❖ Amazon S3 – Data storage, partitioning, lifecycle policies, and cost optimization.

❖ AWS Athena – SQL-based querying, performance tuning (partitioning, compression, indexing), and cost-efficient querying.

❖ AWS Lambda – Serverless function development, event-driven processing, and integration with AWS services.

Programming & ❖  Amazon RDS – Database management, query optimization, replication, and backup

Scripting: strategies.

❖ Amazon DynamoDB – NoSQL data modelling, indexing, and performance tuning (provisioned v/s on-demand capacity).

❖ AWS Step Functions – Orchestration of workflows, state management, and error handling.

❖ AWS Cloud Formation – Infrastructure as Code (IaC), resource provisioning, and version control.

❖ AWS Data Pipelines – CI/CD setup for automated deployment of data pipelines and infrastructure.

❖ Azure Table Storage

❖ Azure AI Search

Good to have skills

❖ Azure Cognitive Services

❖ Azure Functions

❖ Azure Storage / Service Bus