Objective:

As a Data Engineer, you will be responsible for building effective data solutions that will cater to the needs of Data Analysts, Data Scientists, and other Data Consumers across the organization. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. They assess a wide range of requirements and apply relevant database techniques to create a robust architecture and will ensure delivering of high quality ETL solutions.

Roles and Responsibilities:

• Should be comfortable in building and optimizing performant data pipelines which include data ingestion, data cleansing and curation into a datawarehouse, database, or any other data platform using Python
• Must have solid understanding of data structures and algorithms
• Must be extremely proficient with python core libraries and Object-Oriented Programming concepts
• Must have experience working with REST API
• Maintenance and optimization of existing processes
• Experience in writing production ready code in Python and test, participate in code reviews to maintain and improve code quality, stability, and supportability
• Experience in designing data warehouse/data mart
• Experience with any RDBMS preferably SQL Server and must be able to write complex SQL queries
• Leading the client calls to flag off any delays, blockers, escalations and collate all the requirements
• Expertise in requirement gathering, technical design and functional documents
• Experience in Agile/Scrum practices
• Experience in leading other developers and guiding them technically
• Experience in deploying data pipelines using automated CI/CD approach
• Ability to write modularized reusable code components
• Proficient in identifying data issues and anomalies during analysis
• Strong analytical and logical skills
• Must be able to comfortably tackle new challenges and learn
• Must have strong verbal and written communication skills

Technical Capabilities

Must Have –

• Python (primary)/Pyspark
• SQL
• Any ETL (ADF/Databricks/Airflow)
• DWH
• Redshift/Synapse/Snowflake/Google Big Query/Hive
• Object Oriented Programming
• AWS/Azure/GCP/ Any Cloud

Good to Have-

• Git/similar version control tool
• Basic knowledge in CI/CD, Microservices

Education: UG/PG preferably in Computer Science/Information Technology or any other related fields