How Context.dev launched Batches on X

Batches is how Context.dev scrapes a large set of pages in one background job, for developers building agents and products that need live web data at volume. You submit up to 100,000 URLs, or a starting URL or sitemap to crawl from, and each page can return any combination of nine outputs: HTML, Markdown, screenshots, highlights, JSON, parsed data, images, original response bytes, and product data. The API returns a batch ID immediately. You poll for status or receive a webhook when the job finishes, then read results as paginated JSON or download them as files. This release raises the batch ceiling. Jobs added on July 30 stopped at 25,000 URLs and returned only Markdown or HTML. The October 15 update lifts that to 100,000 URLs and applies the same per-page options as the single-page Scrape Anything endpoint, so screenshots, JSON extraction, and product records can come back in the same run. Context.dev, a Y Combinator Summer 2026 company in San Francisco founded by Yahia Bakour, runs the rendering, proxies, and retries under that job. Callers send URLs and read structured page data without operating the browsers themselves.