Python / Airflow Data Engineer
Python / Airflow Data Engineer82
Applications
82
Applications
Bangalore
Pune
Full-Time
Mid-Level: 3 to 6 years
Posted on Jan 11 2025
About the Job
Skills
AWS
Python
Apache airflow
SQL
snowflake
AWS S3
AWS Glue
AWS AMR
- Design, develop, and maintain complex data pipelines using Python for efficient data processing and orchestration.
- Collaborate with cross-functional teams to understand data requirements and architect robust solutions within the AWS environment.
- Implement data integration and transformation processes to ensure optimal performance and reliability of data pipelines.
- Optimize and fine-tune existing data pipelines / Airflow to improve efficiency, scalability, and maintainability.
- Troubleshoot and resolve issues related to data pipelines, ensuring smooth operation and minimal downtime.
- Work closely with AWS services like S3, Glue, EMR, Redshift, and other related technologies to design and optimize data infrastructure.
- Develop and maintain documentation for data pipelines, processes, and system architecture.
- Stay updated with the latest industry trends and best practices related to data engineering and AWS services.
About the company
Epergne Solutions is a leading IT solution provider with niche, comprehensive and leading-edge capabilities in mobility, analytics and web solutions. Our primary focus is to be a trusted partner and strategic technology adviser to our clients. We aim at having long term and fruitful associations with our clients based on trust, genuine interest and understanding of client's busines ...Show More
Epergne Solutions is a leading IT solution provider with niche, comprehensive and leading-edge capabilities in mobility, analytics and web solutions. Our primary focus is to be a trusted partner and strategic technology adviser to our clients. We aim at having long term and fruitful associations with our clients based on trust, genuine interest and understanding of client's business needs.
...Show Less
Industry
IT Services and IT Consul...
Company Size
11-50 Employees
Headquarter
Singapore, Singapore
Other open jobs from Epergne Solutions