Spider API
API by Spider · rest
Spider is a web crawling, scraping and search API. It turns a page or a whole site into markdown, text or structured JSON, and it runs a browser when a page needs JavaScript. Base URL https://api.spider.cloud. Every POST path also accepts a /v1/ prefix. Send "Authorization: Bearer YOUR_API_KEY". Create a key at https://spider.cloud/api-keys. Keyless access is /scrape only. It is rate limited. On a 429, honor Retry-After. Get a key for anything sustained. Without a key, every route other than /scrape returns 402, except /unlimited/*, which returns 401. Content routes return a JSON array with one object per page, /scrape included. The status on each object is the page's own HTTP status, separate from the status of the API call. An attempt bills for the bandwidth and compute it used, whatever status the page returns. A failed page is not free. Pay as you go starts at 10,000 requests per minute and rises through four levels to 80,000 as the account uses more. RateLimit-Limit on each…
Endpoint: https://api.spider.cloud/
Verification: observed. Source: crawl.
Capabilities (17)
- AI browser automation
- AI crawl
- AI links extraction
- AI scrape
- AI search
- Crawl a site
- Remaining credit balance
- Fetch a page by path with stored settings
- Collect links
- Scrape one page
- Take a screenshot
- Search the web
- Convert HTML you already have
- Unblock a page (deprecated)
- Unlimited crawl
- Unlimited links
- Unlimited scrape