We are looking for a Data Platform Engineer to help design and power our big data platform. The responsibilities will include innovating in building out a scalable, fault - tolerant big data platform that can be leveraged for gaining deep insights for both external and internal needs.
Is Petabyte scale data processing just the problem you were hunting to solve
Have you always had a desire to find that needle in a gigantic haystack
Are you a compulsively data driven geek
Do you have a desire to collaborate with and learn from a team of very high caliber data scientists
If you answered YES to any of the above, you can start filling your look ahead buffer!
Skill Set :
You need to have a deep passion for coding and also be versatile enough to explore and solve problems across various facets of computer science, including :
Big Data Processing (MapReduce and Distributed File Systems)
Queueing, Publish / Subscribe systems.
Massive Graph Processing
Realtime data pipelines
Large scale Query Processing
Relational and Non - Relational (NoSQL) database systems
Scalable and Fault - Tolerant Serving systems.
Virtualization, Containerization and Micro Services
Large - scale system design
This is a living growing list : )
Maintain, Manage and Build out a data platform that can process petabyte - scale data.
Engineer a near - real - time system that can process massive amounts of data.
In - depth understanding of how large - scale query processing using Hive, Pig, Presto etc
Design and Implement the data processing, analytics and machine learning part of the platform
Designing, building, installing, configuring and supporting Hadoop.
Perform analysis of vast data stores and uncover insights.
Maintain security and data privacy.
Create scalable and high - performance web services for data tracking.
High - speed querying.
Total experience of 6 to 12 years with a minimum of 5years of hands - on high - performance development experience in Big data Platforms.
Excellent development and debugging skills using Java / Scala.
Knowledge / Contribution to Apache opensource projects such as Kafka, Zookeeper, Hadoop, Hive, Hbase, Spark etc is a big plus.
Prior experience being part of a team that delivered early versions of a product in a successful startup is a plus
Minimum Bachelors or Master s degree in computer science (or equivalent).