Job Title: Cloudera Administrator

Job Description:

Key Responsibilities:

1. Cloudera Hadoop Cluster Management:

• Install, configure, and maintain Cloudera Hadoop clusters (Cloudera Manager, CDH, Cloudera Data Platform).
• Ensure high availability, performance tuning, and disaster recovery of Hadoop clusters.
• Upgrade and patch Cloudera components (HDFS, YARN, Spark, Hive, HBase, Kafka, etc.).
• Monitor and manage cluster performance and capacity planning.
• Handle user access, security configurations, and permissions using Kerberos, Ranger, and other security tools.

2 . Monitoring and Performance Tuning:

• Set up and manage monitoring tools (Cloudera Manager, Nagios, Ambari, etc.) to track and optimize cluster performance.
• Conduct regular performance testing, tuning, and analysis of Hadoop clusters, nodes, and associated services.
• Troubleshoot issues related to HDFS, YARN, MapReduce, and Spark jobs.

3. Data Management:

• Manage HDFS file system, including file movements between HDFS and local systems or cloud storage.
• Oversee data backups, retention policies, and recovery procedures.
• Support data engineers in their ETL pipeline workflows using Hive, Impala, and Pig.

4. Security:

• Implement and manage security for Hadoop clusters (encryption, authentication, access control).
• Set up and configure Kerberos for secure authentication.
• Ensure compliance with industry standards for data protection and privacy (GDPR, CCPA).

5. Automation and Scripting:

• Develop automation scripts using Python, Shell, or Ansible to automate routine tasks (cluster health checks, user provisioning, etc.).
• Automate cluster provisioning and scaling tasks using infrastructure-as-code tools (Terraform, CloudFormation, etc.).

6. Collaboration:

• Work closely with data engineering teams, providing them with access to resources and troubleshooting support.
• Collaborate with DevOps and system administrators for hardware provisioning, network setup, and configuration.

1. Support and Documentation:

• Provide production support for Hadoop-related issues and resolve them within SLAs.
• Create and maintain cluster documentation, including topology diagrams, troubleshooting guides, and best practices.

Required Skills and Qualifications:

• Experience: 5-8 years of hands-on experience in managing Cloudera Hadoop clusters.
• Technical Expertise:
• Experience with Hadoop ecosystem components: HDFS, YARN, MapReduce, Hive, HBase, Spark, etc.
• Familiarity with Cloudera Manager, CDH (Cloudera Distribution of Hadoop), or Cloudera Data Platform (CDP).
• Experience with Linux/Unix system administration.
• Proficiency in scripting languages (Shell, Python, Perl).
• Strong understanding of network, security (Kerberos, SSL), and storage technologies.
• Tools: Experience with monitoring tools (Nagios, Ganglia, Prometheus), and automation tools (Ansible, Terraform).
• Certifications: Cloudera Certified Administrator for Apache Hadoop (CCAH) or similar certifications are a plus.
• Soft Skills: Strong troubleshooting, analytical, and communication skills. Ability to work in a fast-paced environment and collaborate across teams.
• Experience working in cloud environments (AWS, Azure, GCP) with Hadoop clusters.
• Knowledge of containerization (Docker, Kubernetes) and CI/CD processes.
• Experience with Apache Kafka, Oozie, and Sqoop.

Preferred Qualifications:

Work Location: Hyderabad