I'm Hiring a Data Engineer for one of my reputed client.
office Location - Hyderabad
5 to 9 years hands on experience on Spark Core, Spark-SQL, Scala / Java-Programming and Streaming datasets in Big Data platform
Should have extensive working experience in Hive and other components of Hadoop eco system (HBase, Zookeeper, Kafka and Flume)
Should be able to understand the complex transformation logic and translate them to Spark-SQL queries.
Unix Shell Scripting and setting up CRON jobs
Should have worked on Cloudera distribution framework, Oozie Workflow (or any Scheduler), Jenkins (or any version controller)
Spark Developer should be able to understand the Business requirements, Functional and Technical requirements and translate them effectively using the Spark framework.
Should be able to understand the complex transformation logic and translate them to Hive / Spark-SQL queries. Hands on experience on dealing with live streaming datasets using Kafka / Spark Streaming.
Excellent understanding of technology life cycles and the concepts and practices required to build big data solutions
Prior experience in Consumer Banking Domain is an advantage
Prior experience in agile delivery method is an advantage
Familiar with Data Warehouse concepts
Core Java skillset is an added advantage