• 4 + years of expertise and hands on experience in Spark with Scala and Big data technologies

• Good working experience in Scala and object oriented concepts

• Good working experience in HDFS, Spark, Hive and Oozie

• Technical expertise with data models, data mining, and partitioning techniques

• Hands-on experience with SQL database

• Have good understand on CI/CD tools (Maven, Git, Jenkins) and SONAR

Responsibilities

• Analyze, organize and process raw data using Bigdata technologies and Scala

• Perform data validation, cleaning and transformation using Big data technologies, Spark and Scala

• Ingest and manage data in HDFS/Hive and analyze data using Spark and produce meaningful insights for the business team

• Combine raw information from different sources, Explore ways to enhance data quality and reliability

• Develop code using Software Craftmanship best practices such as Continuous delivery and Clean coding principles

• Help business operations team with data samples to create proof of concepts

• Ensure the new data sets are feeding the Business Intelligence setup i.e Dashboards on PowerBI

• Liaise with BA to ensure subjects are getting prioritized, Communicate to PO about progress and potential issues