Responsibility:    

1. Need to write and Develop ETL Scripts for big data sources

2.     Workflow optimization Data Optimization ETL Optimization

3.     Design and Develop Scalable Distributed Architecture

Skills:

1.     Superior coding skills in Scala, Python

2.     In-Depth understanding of highly scalable and distributed architecture

3.     Writing ETL stack in scalable & optimized fashion using Apache Spark/Hadoop/Kafka etc

4.     Writing Distributed & optimized Apache Spark Jobs for various Machine Learning Algorithms. 

Good To Have:

1. Good Understanding and coding skills in any of the following NoSQL Database system Cassandra, MongoDB, Elastic, Any Graph DB
2. Thorough understanding of Hadoop distributed architecture, write optimal map reduce jobs
3. Understanding of RDBMS (MySQL) system SQL Query Language
4. Understanding of Caching technologies and working knowledge on Redis will be added advantage
5. Working knowledge of Docker and Kubernetes 

 Professional Attributes:

1.      Should have good communication skills

2.      Willing to take full responsibility of own task and show complete ownership