Visa is in the process of a technology transformation to use open source BigData technologies like Hbase, Hadoop, Spark, Kafka etc.
Visa has numerous processes that perform complex transformations on data in order to gain competitive information and insights.
The data goes through numerous different transformations into a multitude of tables. The Data-as-a-Service Insights and Analytics team owns platforms that allow for configuration of rules to detect data discrepancies, Metadata information that captures job lineages by analyzing information from schedulers, platform metrics and run time, stewardship data and other important information about the various tables and fields.
The Reference Data team has platforms to manage the core, reference tables at Visa and the workflow engine to manage the governance of additions and modifications to the reference tables.
With a portfolio of these products, the objective is to build a one-stop solution that provides real-time insights of running jobs, data availability, event correlation that can predict / analyze data anomalies, all of which will result in better operational efficiency.
Within the portfolio of products, the Data Quality team is responsible for building a platform where developers and business users can configure data quality checks at the table and field level.
The platform will need to scale to handle onboarding thousands of jobs, with checks that are in the 100s of thousands, ensure validation of the checks being configured in a self-
service manner, while keeping security concerns in mind.
We are looking for passionate, motivated and self-driven engineers who can continue to build these platforms to the next level of scale, and utilize these to build co-
relation and classification systems to drive insights into our systems. A few examples of such insights : drive optimization of our jobs, group alerts and identify the root cause issues.
Key Responsibilities :