Extensible web scraper platform.
Designer features:
- Easy configuration using configuration tree, drag and drop, context menus and wizards.
- Extensibility: use custom actions, converters, filters, data sources, user interface plugins
- HTML element filters (not just XPath!) and converters to extract any data in any format in a reliable way
- Custom actions to do anything that is possible on a web site (login, search, paging, submit, solve CAPTCHA, extract data, send emails & private messages, etc...)
- .Net macro support to implement custom action, filtering or conversion functionality
- Multiple database support
- Web-based configuration file storage
- Window docking
Crawler features:
- Multiple crawler processes
- Real time graphical crawler interface
- Per-page configurable browser options
- Multiple browser engine support
- Built-in proxy to filter and modify web pages
- HTML Tidy to fix invalid HTML DOM trees
- Customizable error handling and logging
- Constraints to ensure data validity
- Multiple data sources and record storages
- API to implement custom crawlers