As a Lead Data Engineer, you will be part of the team which is responsible for building the foundations that enable data driven Business and the digital transformation journey at Acko.
You will work with data strategists to understand the business requirements and execute the tactical aspects of delivering on the vision.
The team will be responsible for driving various tool selection initiatives, ingesting data from different source systems making it available in various ways to be analysed.
Deliver on the business data requirements
Investigate, identify and establish new tools and processes for data warehousing, data quality assurance, reporting, business intelligence, data governance and data cataloging.
Setup reliable data ingestion pipelines for new data sources and integrate them effectively with existing data sets
Assemble, aggregate and transform large datasets to meet the requirements of downstream data consumers
Build data quality tracking mechanisms and diligently execute the issue address processes
Address inevitable disruptions in data ingestion and processing
Develop a comprehensive data catalog
Develop robust internal documentation
Implement data governance strategies as required by the business
Address questions and concerns from downstream data consumers
Continue to adapt to changes based on emergence of new technologies, new competitors, artificial intelligence and alternative sources of data
5 to 8 years of experience in data engineering, data warehousing or data engineering
Advanced working knowledge of multiple SQL and NoSQL storages. Experience working with relational databases, query authoring as well as working familiarity with a variety of databases
Proficient in at least one programming language (Python, Scala, Java etc).
SME for one of the Data Warehouse Solutions : Redshift, Google Bigquery, Snowflake or equivalent
Should be able to build data platform for batch and streaming workloads using cloud services and cloud storage.
Big Data analytics : Hive, Hadoop, Spark or equivalent
Workflow management : Oozie, PySpark, Scala
Reporting and BI : Looker, Qlikview, Tableau, Microstrategy, PowerBI or equivalent.
Tools used to identify and manage unstructured data
Excellent written and verbal communication skills, empathy, initiative and ownership
Superb analytical skills, technical aptitude, influencing skills and attention to detail
Experience working with Analytics, Data Science or Reporting teams
Eager to learn new things and passionate about technology
Flexible and be able to embrace change effectively
Bachelors or master's degree in Computer Science, Engineering, Statistics, Informatics, Information Systems or in another quantitative fields
Strong technical skills in best of class technologies around building data platform, data lake, data warehousing, data wrangling, data quality and data governance
Ability to work effectively with varied set of teams
This job was posted by Priyanka from Acko.