The Data Engineer is responsible for supporting Snaplogic ETL, AWS Redshift, RDS, and Aurora environments, exploring performance improvement opportunities, and troubleshooting data ingestion and user queries’ issues.
This individual will perform typical database administrative tasks such as managing user accounts, monitoring system capacity usage and performance, balancing system workload, identifying root cause of issues raised by users and provide solution proposals, as well as supporting and enhancing data ingestion processes developed by other development teams.
This individual will work with business users to identify performance improvement opportunities and implement them in the order based upon urgency and priority.
In addition, this individual will identify data integration, ETL and quality issues, investigate issues identified by others, research and implement solutions to correct problems (at the source where possible), collaborate with developers to correct historical data, develop reports and monitoring programs, and execute data integration initiatives.
Job Responsibilities :
Manage user accounts and security in the Redshift, RDS, and Aurora environments
Monitor and manage system workloads in order to keep all the systems running smoothly and minimize bottlenecks.
Configure system connections between data sources and targets.
Assist users in debugging query failure issues and data quality problems
Analyze objects such as tables and views designed and deployed by the development teams and users, provide best practice solution proposals
Analyze queries from users, identify performance improvement opportunities, and assist users to improve performance
Analyze and manage disk space usages by users, and provide best practice to development teams and business users
Maintain, modify and enhance data ingestion processes per development teams request
Develop and deploy data load processes per business users request, including tables, views, and data load ETL pipelines’ design and development
Create native connections between Redshift, RDS, and Aurora
Develop and maintain metrics programs to monitor system usages and performance
Manage data Redshift archiving processes
Develop Redshift Spectrum objects
Support Engineering and Manufacturing Datawarehouse and related data systems and initiatives
Solve moderate to complex data quality / data integration problems
Recognize and investigate anomalies and propose solutions
Research and recommend innovative, and where possible automated, approaches for data issues
Monitor ETL pipelines, troubleshoot and address any issues.
Liaison with business users and Subject matter Experts to find permanent solution to recurring issues.
Work closely with managed service providers in streamlining production support processes.
Required Qualifications :
Bachelor’s degree in computer science, Information Systems, (or a related field) or a minimum of seven years of relevant professional experience and training
7 to 9 years Data Engineer or Support Engineer roles
3+ year AWS Redshift experience
2+ year experience in performance tuning and optimization of any SQL databases
3+ year supporting ETL packaged software like Informatica or SnapLogic
Familiarity with concept of and experiences in performance tuning approaches related to MPP databases
Proficient in SQL
Experiences in data modeling, including logical and physical design, normalization and denormalization
Expertise in performance tuning in one or more RDBMS such as Postgres, MySQL, Oracle, DB2, MS SQL, etc.
Experiences in programing tools such as Linux shell scripting, Python, Java, etc.
Experiences in analysis, Design, Development and Implementation of ETL and exposure to any commercial ETL tools (Snaplogic, Informatica, Alteryx, Matillion) and reporting (Tableau, Spotfire) tools will be an advantage but not mandatory.
Effectively and efficiently work with an enterprise environment involving very large data sets - millions of records in both databases and flat files
Experienced in requirements analysis, design, and development in the data Management application areas
Responsible for working with cross-functional and cross-organizational teams to understand issues, implement solutions, and influence others to implement solutions
Able to be comfortable in a fast-paced environment with constantly shifting priorities and a mix of quick hit, short, medium and long-term projects of varying priorities
High-tech, or manufacturing industry experience preferred but not required
Excellent communication, facilitation, and interpersonal relations skills required
Western Digital is committed to providing equal opportunities to all applicants and employees and will not discriminate based on their race, color, ancestry, religion (including religious dress and grooming standards), sex (including pregnancy, childbirth or related medical conditions, breastfeeding or related medical conditions), gender (including a person’s gender identity, gender expression, and gender-related appearance and behavior, whether or not stereotypically associated with the person’s assigned sex at birth), age, national origin, sexual orientation, medical condition, marital status (including domestic partnership status), physical disability, mental disability, medical condition, genetic information, protected medical and family care leave, Civil Air Patrol status, military and veteran status, or other legally protected characteristics.
We also prohibit harassment of any individual on any of the characteristics listed above. Our non-discrimination policy applies to all aspects of employment.
We comply with the laws and regulations set forth in the poster.
Western Digital thrives on the power and potential of diversity. As a global company, we believe the most effective way to embrace the diversity of our customers and communities is to mirror it from within.
We believe the fusion of various perspectives results in the best outcomes for our employees, our company, our customers, and the world around us.
We are committed to an inclusive environment where every individual can thrive through a sense of belonging, respect and contribution.
Western Digital is committed to offering opportunities to applicants with disabilities and ensuring all candidates can successfully navigate our careers website and our hiring process.
Please contact us at to advise us of your accommodation request. In your email, please include a description of the specific accommodation you are requesting as well as the job title and requisition number of the position for which you are applying.