Job Description:

We are searching for a highly skilled and experienced Data Engineer to join our growing team. You will

play a key role in designing, developing, and maintaining data pipelines using SQL and Databricks. You

will work closely with onshore and clients on requirement gathering, unit testing, UAT and deployments.

Responsibilities:

• Plan, build, and implement data solutions based on DataBricks.

• Design and build Modern Data Pipelines and Data Streams

• Develop and Maintain Data Warehouse

• Design and create ETL processes supplying Data Warehouse

• Implement effective metrics and monitoring processes.

• Preprocess of structured and unstructured data, create queries in databases.

Skills:

• 5+ years experienced resources in Databricks, Python, Big Data, Apache Spark, SQL

• and Spark SQL , AWS/Azure/ GCP

• Experience in Building Data Governance Solutions like Unity Catalog, StarBust …etc

• Build Very Strong Orchestration Layer in Databricks/ADF…. Workflows / AWS

• Build CICD for Databricks in Azure Devops / AWS / GCP

• Process near Real time Data thru Auto Loader, DLT Pipelines.

• Implement Security Layer in Delta Lake.

• Implement Massive Parallel Processing Layers in Spark SQL and PySpark.

• Implement Cost effective Infrastructure in Databricks.

• Ability to Build Solutions as Cloud Agnostic