So what does a Data Data Warehouse Engineer really do? Think of yourself as one of the people responsible for designing, developing, and maintaining data pipelines, data warehouses for use by the analytics infrastructure, so not just anyone is qualified for this role! We make sure we get the best of the best, after all, we are a ridiculously good company so we make sure our employees are top notch.
So come on, now we need your full concentration because it’s time to imagine what it’s like being a Data Engineer.
Imagine yourself going to work with one thing on your mind : you will assist in making accurate and meaningful data available for consumption of TaskUs Business Intelligence reporting methodology .
As you tackle your new tasks for the day, you know that it will lead to one thing : that everything you do is in order to meet an enterprise’s data delivery, business analysis and reporting needs.
As a Cloud Data Warehouse Engineer , you will create new data pipelines, database architectures and ETL processes, you will observe and suggest what the go-to methodology should be.
You will be gathering requirements, performing vendor and product evaluations, deliver solutions, training, and documentation.
You will also handle the design and development,tuning, deployment, and maintenance of information, advanced data analytics, and physical data persistence technologies.
You will establish analytic environments required for structured, semi-structured and unstructured data. You will implement the business requirements and business processes, build ETL configurations, create pipelines for the Data Warehouse, research on new technologies and build proof-of concepts around them.
Build and maintain extract-transform-load ( ETL ) and / or extract-load-transform ( ELT ) processes for DW and Data Lake.
You will carry out monitoring, tuning, and database performance analysis. You will perform the design and extension of data marts, metadata, and data models.
You will also ensure all data warehouse architecture codes are maintained in a version control system.
So, do you have what it takes to become a Data Warehouse Developer?
So what is exactly needed for this position? Well, first you must have at least 5 - 6 years of broad information technology experience.
What else? Strong knowledge of Data Warehousing concepts with understanding of traditional and MPP database designs, star and snowflake schemas with at least 4-5 years of data modeling .
You must have at least 2 years of hands-on development experience using ETL Tools such as Pentaho, SSIS, Informatica, Talend, Fivetran or Airflow.
Knowledge on the architecture, design and implementation of MPP Databases such as Teradata, Snowflake or Redshift. 2-year development using Cloud-based analytics solutions preferrable AWS or GCP.
Knowledge of designing and implementing streaming pipelines using Apache Kafka, Apache Spark, or Segment. At least 2 years experience of using Python in a cloud based environment is definitely a plus.
Knowledge on NoSQL DBs is not required but preferred.
We need someone who has a Bachelor’s degree in Computer Science (have an MBA or MS degree? That’s a plus!) or 8-10 years experience in lieu of degree.
We need someone who has knowledge of the following : Python, Apache Kafka, AWS Redshift, AWS Glue, AWS S3, Pentaho Data Integration.
We need someone who has knowledge in Database Tuning and ETL tuning. We need someone who has the ability to work effectively across internal functional areas in ambiguous situations.
Someone who is a structured thinker and effective communicator.