Companies using Scrapy

Here is a list of companies and web sites using Scrapy, in no particular order

  • Scrapinghub provides a platform for running Scrapy spiders in the cloud (still in closed beta)
  • Mydeco uses Scrapy to scrape more than 4000 furniture retailer sites daily
  • Insophia uses Scrapy for several scraping projects on-demand
  • Médialab Sciences Po in Paris is using Scrapy to develop a web mining tool for Social Sciences researchers (announcement here)
  • ZaoJiao100 - Daily for infants and young children crawling early education information
  • World Government Data - UK government data aggregation site (tweet)
  • Catalystic - used to develop bespoke scraping projects
  • NextWeekVacations uses Scrapy to collect the data of their vacation rentals
  • Oposicionesaldia - uses Scrapy to collect data from jobs postings, scholarships and online free courses in Spain.
  • Iberestudios - uses Scrapy to collect data from masters degrees, doctorates and postgradute degrees in Spain.
  • BiteHunter - real time aggregator and search engine for dining
  • Dig It Deals - to scrape deals from many sites
  • Tryolabs - for its Omega Spider product, and for consulting projects that need web crawling
  • MarketMate RS - to scrape job offers from many sites
  • Pricewiki.com - using it to scrape various websites for cost of living information
  • Weotta - used to crawl data for post-processing (tweet)
  • Dealshelve uses Scrapy to scrape daily deals from many sites
  • Zinmoo uses Scrapy to scrape thousands of real state properties a per day (blog post)
  • CareerBuilder.com uses Scrapy to scrape job offers from certain sites
  • GrabLab is Russian company which specializes in web scraping, data collection and web automation tasks.
  • SimpleSpot uses Scrapy to build their geolocalized information service
  • Monetate uses Scrapy daily to collect catalog information from their clients
  • ClanSlots uses Scrapy daily to collect levels and plugins for games they host
  • Alışveriş Robotu is a Turkish price comparison site that uses Scrapy to collect data from hundreds of retailers everyday
  • Thalasar Ventures uses Scrapy to build bespoke crawlers
  • ToTheGo is an aggregator of ads related to homes, jobs and cars trading. They use Scrapy to scrape everyday announces from the biggest portals and organize them inside their database.
  • JobMistral is an aggregator of jobs classified ads. The use Scrapy to scrape job ads hourly from many sites
  • TuvaLabs uses Scrapy to scrape RSS feeds of major newspapers from all over the world. They scrape the web to find the most interesting articles of significant news stories taking place around the world and transforming them into interactive math learning units.

If you use Scrapy, we'd like to include you on the list. Send us an email to info@scrapinghub.com.