Job Description Data Engineer TransOrg Analytics
Why would you like to join us?
TransOrg Analytics, an award-winning - Big Data and Predictive analytics- company, offering advanced analytics solutions and services to industry leaders and Fortune 500 companies across India, US, Singapore Asia Pacific, and the Middle East.
Our Automated Machine Learning product - Clonizo' yields significant business benefits to our clients. We were recognized by the CIO Review magazine as the - Predictive Analytics Company of the Year- and by TiE for excellence in position is for a Data Engineer with experience in SQL / NoSQL, Cloud, Hadoop and / or Spark platforms
Design and implement data engineering projects.
Integrate multiple data sources to create data lake / data mart.
Perform data ingestion and ETL processes using SQL, Scoop, Spark or Hive
Knowledge of new components and various emerging technologies in on-premises and Cloud (AWS / Azure / Google)
Collaborate with various cross-functional teams : infrastructure, network, and database.
Work with various teams to set up and manage users, secure and govern platforms and data and maintain business continuity through contingency plans (data archiving, etc.)
Monitor job performances, manage file system / disk-space, cluster & database connectivity, log files, manage backup / security, and troubleshoot various user issues.
Design, implement, test and document performance benchmarking strategy for platforms as well as for different use cases
Setup, administer, monitor, tune, optimize and govern large scale implementations
Implement machine learning models on the real-time input data stream.
Drive customer communication during critical events and participate / lead various operational improvement initiatives.
Education & Skills Summary :
2 - 5 years of relevant experience in data engineering
Exposure to any or all latest data engineering ecosystem platforms such as AWS, Azure, GCP, Cloudera and Data bricks
Sound knowledge of Python / Scala / Java.
Good knowledge of SQL / NoSQL databases and data warehouse concepts
Hands-on experience of working on databases such as Sql Servers, PostgreSql, Cloud infrastructure, etc.
Excellent knowledge of data backup, recovery, security and integrity
Sound knowledge on Spark, HDFS / HIVE / HBASE, Shell Scripting, and Spark Streaming.
Excellent communication skills
Must be proficient with data ingestion tools like Sqoop, flume, talend, and Kafka.