Job Role Data Engineer
Essential Duties and Responsibilities
Develop new applications and features within a scrum team providing data and data services to the enterprise, other engineering teams, data science, analysts, product, management / executives, and other business teams
Build high performing and scalable data systems to support multiple internal and 3rd party data pipelines and make the data available as soon as possible
Implement new technologies and practices to provide the best solutions to our customers
Responsible for following and implementing architecture / design and risk mitigation strategies
Develop and maintain solutions on our tech stack environments (Spark, Kafka, Hadoop, Vertica, SQL Server, etc.)
Work with business teams to deliver on commitments within time and scope
Work with IT Operations and Prod Support to ensure solutions are releasable, maintainable, and scalable
Work with Risk & Compliance to ensure necessary logging / security is in place to comply with audits
Help develop team members through code reviews, enforcing standards, best practices, policies, and processes
Perform functional testing, end-to-end testing, performance testing, and UAT of applications and code written by self and other members of the team
Depending on experience, you will have the opportunity to contribute to designing of new applications, setting / changing standards and architecture, and deciding on usage of new technologies either immediately or in the future.
Qualifications and Experience
2+ years of application software and data engineering experiences in Spark, Kafka, RDBMS in Scala / Java and SQL.
Some experience in diagnosing issues and tuning applications in various environments
Experience with AWS or other cloud vendors
Big pluses : MS SQL Server, Vertica, Docker, Kubernetes, Data Science / ML Knowledge, Snowflake, Hadoop, NoSQL
Proficient knowledge of optimization, concurrency, scale-up vs scale-out, encryption, application- and process-level security, etc.
in an Apache / Hadoop environment is a plus
You can follow and contribute to a detailed conversation about Web, REST, APIs, JSON, service buses, and other web developer technologies and ideas.
Education and Training
Bachelors degree in Computer Science, MIS or related field or equivalent relevant experience
ONLY PRODUCT COMPANIES