Skip to content
ArticleIntermediate

Build a web crawler with Queues and Browser Run

Build and deploy a web crawler with Queues, Browser Run and Puppeteer, storing results in Workers KV.

Last updated
Oct 12, 2026
Read the original

Submit links to a queue, and a consumer Worker crawls each one in a headless browser with Puppeteer through Browser Run, saving what it finds to KV — a scalable pattern for crawling only sites you’re allowed to.

What you’ll learn

  • Create a KV namespace and set up Browser Run
  • Set up a queue and its bindings
  • Submit links to crawl
  • Crawl with Puppeteer and deploy

Cloudflare products used

Queues, Browser Run, KV, Workers.

Follow the full, step-by-step tutorial in Cloudflare’s documentation.

Keep learning