dart_web_scraper
0.2.17, released 6 weeks ago by sukhcha.in
Config-based scraper for web pages and APIs using chainable parsers and transforms.
WASM-ready
What it does
Extracts data from web pages and APIs using simple key/value configurations and chained parsers. It supports over 15 built-in parsers including HTML, CSS selectors, JSONPath, tables, and response headers.
AI-written summary of version 0.2.17, based on the package's README and metadata. Report an inaccuracy
Use it when
- You need to scrape web pages or APIs using configuration-driven patterns.
- You want to extract structured data from HTML tables or JSON endpoints without writing boilerplate parsing logic.
Skip it when
- You need a full headless browser solution to execute complex client-side JavaScript interactions.
- You are scraping heavily authenticated applications requiring complex session management.
Strengths
- Over 15 built-in parsers with support for elements, attributes, text, JSON paths, and headers.
- Automatic browser-like request headers and user-agent generation.
- Supports data cleaning and transformations.
Weaknesses
- Relies on static HTML fetching and cannot execute JavaScript-heavy Single Page Applications natively.
- Config-based approach may become difficult to maintain for extremely complex scraping workflows.
Setup
- Kind
- Pure Dart
- Effort
- Low
- Code generation
- No
- Direct dependencies
- 5
import 'package:dart_web_scraper/dart_web_scraper.dart';
void main() async {
WebScraper webScraper = WebScraper();
Map<String, Object> result = await webScraper.scrape(
url: Uri.parse("https://quotes.toscrape.com"),
scraperConfig: ScraperConfig(
parsers: [
Parser(
id: "quotes",
parents: ["_root"],
type: ParserType.element,
selectors: [".quote"],
multiple: true,
),
Parser(
id: "quote",
parents: ["quotes"],
type: ParserType.text,
selectors: ["span.text"],
),
],
),
);
print(result);
}Alternatives
Packages that do the same job and could replace dart_web_scraper.
- socket_io_common
Socket.io common parser library for Dart.
47Compare - chaleno
Scrape HTML content from websites with a simple, cross‑platform API
45Compare
Works with
Packages commonly used together with dart_web_scraper: ones it builds on, ones built on it, and its own family.
- htmldart_web_scraper depends on itA Dart implementation of an HTML5 parser for strings and bytes.
- json_pathdart_web_scraper depends on itDart implementation of RFC 9535 JSONPath for querying and manipulating JSON
- json5dart_web_scraper depends on itLightweight library for encoding and decoding JSON5 with comments and trailing commas.