AdvaRisk - Tech Lead - Web Scraping & Crawling (5-7 yrs) Pune (Backend Developer)
SLO Technologies (P) Ltd
20h ago
source :


  • Manage individual projects priorities, deadlines, and deliverables
  • Gather and process raw data at scale (including writing scripts, web scraping, calling / create APIs, etc.) from the web / internet
  • Develop frameworks for automating and maintaining constant flow of data from multiple sources
  • Identify, analysis, design, and implement internal process improvements
  • Design and implement tooling upgrades to increase stability and data quality
  • Help team to fix issues that occur in test and production environments
  • Automate software development processes, including build, deploy, and test
  • Mange and guide the team members

  • 4+ years of web crawling / scraping experience is a must
  • Strong knowledge of scraping frameworks such as Scrapy, Beautiful Soup, HTQL, Jsoup, Web-Harvest and others
  • Excellent verbal, written, and interpersonal communication skills in English
  • Good to have Experience of complex crawling (like captcha, Mobile OTP based crawling, bypassing proxy)
  • Sound Knowledge in Bot Management Techniques
  • Experience in various data extraction methods (like data extraction from PDF Files, web pages, etc)
  • Good understanding of HTML DOM, CSS, Javascript, and RESTful web services
  • Good to have understanding of AWS
  • Experience with Linux
  • Experience with Java / Python
  • Report this job

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    My Email
    By clicking on "Continue", I give neuvoo consent to process my data and to send me email alerts, as detailed in neuvoo's Privacy Policy . I may withdraw my consent or unsubscribe at any time.
    Application form