Main Skill: Python

Experienced in end-to-end of ETL (Data Pipeline) QA testing using various technologies in Azure cloud involving ingestion from varied data sources.

Should be familiar with data warehousing target and intermediate structures ideally in environments u sing data lakes.

Well versed in Databricks, SQL Database, Azure Data Factory, Azure Data Lake

Must have hands-on experience in PySpark, SparkSQL, T-SQL

Should be able to go through requirements, design specs and create appropriate test scenarios across various layers/stages of data pipelines.

Should be able to write reusable PySpark and T-SQL script to validate data across layers.

Experience of working in Databricks Delta Lake will be highly desirable.

Ideally should have worked in a team using Agile methodology.

Appreciation of data, its quality, and its usage for business benefit

Very good communication and written skills.