Ideally should not be more than 3-5)
1.Experience with Spark, or the Hadoop ecosystem and similar frameworks.
2.Hands-on experience in widely used BigData components such as Spark, Kafka, Hive, HBase, Pig, Flink.
3.Hands-on experience in building and optimizing batch and real-time big data’ data pipelines, architectures and data sets.
4.Hands-on experience in object-oriented / scripting languages such as Python, Java, Scala, etc.
Responsibility of / Expectations from the Role
Work closely with other data and analytics team members to optimize the data systems and pipeline architecture.
Desig n and build data pipeline for data extraction, preparation, and loading from a variety of sources using selected tools and frameworks as required and requested.
Perform tasks such as writing scripts, jar / package creation, calling APIs, write SQL queries, etc.
Support business decisions with ad hoc analysis as needed.