Skip to content
pubscout

dart_web_scraper

0.2.17, released 6 weeks ago by sukhcha.in

Config-based scraper for web pages and APIs using chainable parsers and transforms.

WASM-ready
$ flutter pub add dart_web_scraper
Source repository

What it does

Extracts data from web pages and APIs using simple key/value configurations and chained parsers. It supports over 15 built-in parsers including HTML, CSS selectors, JSONPath, tables, and response headers.

AI-written summary of version 0.2.17, based on the package's README and metadata. Report an inaccuracy

Use it when

  • You need to scrape web pages or APIs using configuration-driven patterns.
  • You want to extract structured data from HTML tables or JSON endpoints without writing boilerplate parsing logic.

Skip it when

  • You need a full headless browser solution to execute complex client-side JavaScript interactions.
  • You are scraping heavily authenticated applications requiring complex session management.

Strengths

  • Over 15 built-in parsers with support for elements, attributes, text, JSON paths, and headers.
  • Automatic browser-like request headers and user-agent generation.
  • Supports data cleaning and transformations.

Weaknesses

  • Relies on static HTML fetching and cannot execute JavaScript-heavy Single Page Applications natively.
  • Config-based approach may become difficult to maintain for extremely complex scraping workflows.

Setup

Kind
Pure Dart
Effort
Low
Code generation
No
Direct dependencies
5
import 'package:dart_web_scraper/dart_web_scraper.dart';

void main() async {
  WebScraper webScraper = WebScraper();

  Map<String, Object> result = await webScraper.scrape(
    url: Uri.parse("https://quotes.toscrape.com"),
    scraperConfig: ScraperConfig(
      parsers: [
        Parser(
          id: "quotes",
          parents: ["_root"],
          type: ParserType.element,
          selectors: [".quote"],
          multiple: true,
        ),
        Parser(
          id: "quote",
          parents: ["quotes"],
          type: ParserType.text,
          selectors: ["span.text"],
        ),
      ],
    ),
  );

  print(result);
}

Alternatives

Packages that do the same job and could replace dart_web_scraper.

Works with

Packages commonly used together with dart_web_scraper: ones it builds on, ones built on it, and its own family.