Want to shape the future of Energy through Data Science? We all know that without good data there is no Data Science.
If you let garbage in, it will emit the garbage out.
60-70% efforts in a data science project are spent in Data Engineering & Feature Engineering.
That's where we need your skills to fetch the data from disparate sources, transform it the way business needs (that may also include applying lots of critical business logics per its source and nature) and load it in a data warehouse / big data systems.
These critical pieces of works complement the Data Scientist, with a continuous feedback loop based on how a model is performing and what fine tuning is needed in the data.
Â The Energy Exemplar (EE) data team is looking for an experienced Data Engineer to join our Pune office.
As a dedicated Data Engineer on our Research team, you will apply data engineeringÂ expertise, work very closely with the core data team to identify different data sources for specific energy markets and create an automated data pipeline.
The pipeline will then incrementally pull the data from its sources and maintain a dataset, which in turn providesÂ tremendous value to hundreds of EE customers.
Â At EE, you ll have access to vast amounts of energy-related data from our sources.
Our data pipelines are curated and supported by engineering teams.
We also offer many company-sponsored classes and conferences that focus on data science and ML.
There s great growth opportunity for data science at EE.Â ResponsibilitiesDevelop, test and maintain architectures, such as databases and large-scale processing systems using high-performance data pipeline.
Recommend and implement ways to improve data reliability, efficiency, and quality.Identify performant features and make them universally accessible to our teams across EE.
Work together with data analysts and data scientists to wrangle the data and provide quality datasets and insights to business critical decisions.
Take end-to-end responsibility for the development, quality, testing, and production readiness of the services you build.
Define and evangelize Data Engineering best standards and practices to ensure engineering excellence at every stage of development cycle.
Act as a resident expert for data engineering, feature engineering, exploratory data analysis.Â Qualifications2+ years of professional experience in developing data-pipelines for large-scale, complex datasets from varieties of data sources.
Data Engineering expertise with strong experience working with Big data technologies such as Hadoop, Hive, Spark, Scala, Python etc.
Experience working with Cloud based data technologies such as Azure Data lake, Azure Data factory, Azure Data Bricks highly desirable.
Knowledge and experience working with database systems such as Cassandra, HBase, Cosmos etc.Moderate coding skills.
SQL or similar required.
C# or other languages strongly preferred.Proven track record of designing and delivering large-scale, high quality systems and software products.
Outstanding communication and collaboration skills.
You can learn from and teach others.Strong drive for results.
You have a proven record of shepherding experiments to create successful shipping products / services.Experience with prediction in adversarial (energy) environments highly desirable.
A Bachelor or Masters degree in Computer Science or Engineering with coursework in Statistics, Data Science, Experimentation Design, and Machine Learning highly desirable.
Skills : - Python, Apache Spark, Apache Hive, Apache Hadoop, Data engineering, Azure Databricks and Data Pipeline