Company Overview

Hiring for a global analytics services and solutions company. We are one of the fastest growing private companies in the country for three straight years according to the Inc. 5000 and we continue to set ourselves apart from our competitors by attracting the greatest talent in the data analytics and data science space. Our capabilities range from Data Visualization, Data Management to Advanced analytics, Big Data and Machine Learning. Our uniqueness is in building Scalable Big Data Solutions on Onprem/GCP/Azure cloud in a very cost effective and easily scalable manner for our clients. We also come in with some strong IP and pre-built analytics solutions in data mining, BI and Big Data. 

Job Overview

We are seeking a Senior Data Engineer to join our dynamic team at Covenant Consultants. This full-time position is available in various locations including Chennai, Bangalore, Pune, Hyderabad, Delhi, Gurgaon, and Kolkata. The ideal candidate will have extensive experience with data engineering tools and platforms. As a Data Engineer, you will play a crucial role in managing and optimizing our data pipeline architecture.

Qualifications and Skills

• Proven experience in using Databricks for building and optimizing data pipelines is crucial.
• Strong skills in Pyspark for efficient processing of large volumes of data with accuracy and speed.
• Expertise in SQL for querying and manipulating structured data is essential.
• Solid understanding of Python programming for customization and data manipulation tasks.
• Experience with Azure Databricks for cloud-based data engineering solutions.
• Strong problem-solving skills with a focus on innovative and practical solutions to complex issues.
• Experience in working collaboratively in a team-oriented environment, demonstrating excellent communication and interpersonal skills.
• Ability to manage multiple tasks and projects simultaneously, prioritizing workload effectively to meet demanding deadlines.

Roles and Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems.
• Develop unique and efficient solutions to problems involving substantial amounts of data.
• Establish data architecture standards through code review, unit tests, and performance tuning.
• Collaborate with data scientists and data analysts to integrate analytics tools with data-centric solutions.
• Continuously improve and optimize the performance of existing data engineering applications.
• Ensure data quality and integrity through rigorous testing and validation processes.
• Document and communicate data engineering standards, practices, and lessons learned.
• Keep up-to-date with the latest industry trends, tools, and technologies, integrating these into your work as necessary.