Web Scraping Engineer

SPsoft is seeking a part-time web scraping engineer with expertise in Scrapy who can design and maintain a reliable website scraper. The ideal candidate will have a solid understanding of web scraping techniques, data extraction, and processing and proficiency in Python and Scrapy.

Responsibilities:

• Design and implement web scrapers to extract structured and unstructured data from the web pages.
• Ensure the scraper’s reliability, scalability, and maintainability.
• Handle anti-scraping measures and implement best practices.
• Collaborate with the team to integrate scraped data into our systems.

Requirements:

• Proven experience with Scrapy and Python.
• Familiarity with HTML, CSS, and XPath/Selectors for web scraping.
• Understanding of web protocols and handling CAPTCHAs or rate-limiting.
• Attention to detail and ability to meet project deadlines.