Get Started

Package · crates.io

crw_crawl

crates.io · v0.37.2 · AGPL-3.0

Async BFS web crawler with rate limiting and robots.txt support for CRW

Tentative categories were assigned with moderate confidence (0.5 to 0.75): they are shown, but they do not count toward a category’s momentum. How categories work

Technologies

Install

cargo add crw_crawl

Adoption

Downloads (90 days)
1.4K
Observeddownloads / 90 days
Popularity
Lower half on crates.io
Derived1st percentile within crates.io
Dependents
Collectinghistory
Collecting
All-time downloads
3K
Observedsince first publish
Maintenance
0.98
DerivedRelease cadence and recency, 0 to 1
Releases per year
129
75 versions over its life

Growth by period

Derived by Crest of Tech from two observations of crates.io’s own figure, a period apart. Nothing is stored or estimated: without an observation near the period’s start, the period is still collecting.

7 days
Collectinghistory
Collecting
30 days
Collectinghistory
Collecting
90 days
Collectinghistory
Collecting
1 year
Collectinghistory
Collecting

Adoption historydownloads / 90 days

Observations in this range began OCT 6, 2026. The chart appears after 7 observations; 1 so far. Collecting history.

Table of observations (30)
Daydownloads / 90 daysDependents
OCT 6, 20261,367Not observed
OCT 5, 2026Not observedNot observed
OCT 4, 2026Not observedNot observed
OCT 3, 2026Not observedNot observed
OCT 2, 2026Not observedNot observed
OCT 1, 2026Not observedNot observed
SEP 30, 2026Not observedNot observed
SEP 29, 2026Not observedNot observed
SEP 28, 2026Not observedNot observed
SEP 27, 2026Not observedNot observed
SEP 26, 2026Not observedNot observed
SEP 24, 2026Not observedNot observed
SEP 23, 2026Not observedNot observed
SEP 22, 2026Not observedNot observed
SEP 21, 2026Not observedNot observed
SEP 20, 2026Not observedNot observed
SEP 19, 2026Not observedNot observed
SEP 18, 2026Not observedNot observed
SEP 17, 2026Not observedNot observed
SEP 16, 2026Not observedNot observed
SEP 15, 2026Not observedNot observed
SEP 14, 2026Not observedNot observed
SEP 13, 2026Not observedNot observed
SEP 12, 2026Not observedNot observed
SEP 11, 2026Not observedNot observed
SEP 10, 2026Not observedNot observed
SEP 9, 2026Not observedNot observed
SEP 8, 2026Not observedNot observed
SEP 7, 2026Not observedNot observed
SEP 6, 2026Not observedNot observed

Facts from the registry

Latest version
v0.37.2
75 versions
First published
MAR 3, 2026
Last release
SEP 30, 2026
24d ago
Licence
AGPL-3.0
Links
crates.io page
Homepage · fastcrw.com
Source · github.com

Repository

In the Crest catalog

fastcrw/crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

Stars
1.1K
Forks
89
Language
Rust
Last push
24d ago
Last release
24d ago

Linked by the package’s own registry metadata.GitHub

Dependencies latest version · v0.37.2

Runtime 20

  • crw-core^0.37.2
  • crw-diff^0.37.2
  • crw-extract^0.37.2
  • crw-renderer^0.37.2
  • dashmap^6
  • flate2^1
  • futures^0.3
  • libc^0.2
  • moka^0.12
  • phf^0.14
  • publicsuffix^2
  • rand^0.10
  • regex^1
  • reqwest^0.13
  • scraper^0.27
  • serde_json^1
  • tokio^1
  • tracing^0.1
  • url^2
  • uuid^1

Related packages

Related news

No published news event mentions this package or its repository yet.