Dataset

Common Crawl

1stories this week
1last 30 days
1all time

Timeline

  1. Oct 9, 2026 · Research paper · 1 source
    Host-Aware Web Crawling for LLM Pretraining Data
    Web crawling is a major source of large language model (LLM) pretraining data, yet data-selection pipelines discard a large share of crawled pages.