Our web scraper is a very powerful tool however, it can be difficult for non-technical users to use. As you can see in this how-to video from 2016.
To overcome this issue we now hide the raw scrape instructions unless explicitly requested, with the Scrape Instruction tab now displaying scrape instructions in plain English by default. In fact scrape instructions can now be entirely added and deleted without ever writing any code.
We have also simplified and made our web scraper wizard more intuitive by merging four independent wizards into one. The new wizard now can perform multiple actions, with each action only showing when applicable for a web page. It is also designed to give the user more intuitive options based on the web page that is being scraped.
But what do you do if you are writing a scrape for several different types of web page? To solve this we created a new templating system that allows you to define what scrape instructions should be executed for what web pages and actions and all of this new functionality is available without writing manual scrape instructions.
Another common complaint was that it was difficult to tell if a scrape was working as expected or not, to solve this scrape results and logs are now also available in real time. So you know if a scrape is working before it ends giving you the option to abort the scrape early, if required.
All of these changes have resulted in a much more user friendly experience as shown in the latest scraping tutorial on the left.
However we haven’t finished improving the usability of our web scraper with these changes, so if anyone has any ideas on how it can be improved further please contact us!