Use DataFuel from AI agents and IDEs

Connect Claude Code, Cursor or any MCP client to the DataFuel MCP server, or hand an agent the skill file.

Written By Carmine Cella

Last updated About 3 hours ago

The same API is exposed as a Model Context Protocol server for agents and AI IDEs. It is streamable HTTP, stateless, and authenticates with the same X-API-Key header. Every tool call is billed exactly like its REST counterpart.

Endpoint: https://scraping-api.datafuel.ai/mcp

Claude Code

claude mcp add --transport http datafuel https://scraping-api.datafuel.ai/mcp \  --header "X-API-Key: df_key_your_key_here"

Cursor, Claude Desktop and other clients

Add this to the client's MCP configuration (mcp.json or equivalent):

{  "mcpServers": {    "datafuel": {      "url": "https://scraping-api.datafuel.ai/mcp",      "headers": { "X-API-Key": "df_key_your_key_here" }    }  }}

Tools

Tool	What it does	REST equivalent
`scrape`	Run one task and wait for the result. `type` and `attributes` as in `POST /task`. Defaults `include_images` to false.	`POST /task`
`get_task`	Fetch a task and its result by id.	`GET /task/{id}`
`map`	Discover the URLs of a site. Flat inputs: `url`, `search`, `limit`, `sitemap`, `include_subdomains`, `ignore_sitemap`, `sitemap_only`, `proxy_type`, `proxy_country`.	`POST /map`
`create_job`	Queue a batch. Plural targets in `attributes` (`urls` or `prompts`).	`POST /job`
`get_job`	Job status and counters.	`GET /job/{id}`
`get_job_results`	Per-task results of a job.	`GET /job/{id}/results`
`crawl`	Start a crawl. `attributes` is the crawl config plus unlocker options. Returns `job_id`.	`POST /crawl`
`get_crawl`	Crawl status, page counters, stop reason.	`GET /crawl/{id}`
`get_crawl_results`	Page through crawled pages. Pass `next_cursor` back as `cursor`.	`GET /crawl/{id}/results`
`get_balance`	Remaining credits.	`GET /users/@me/balance`

scrape and map block until the result is ready, up to ten minutes. The job and crawl tools return immediately and are polled.

Give an agent the skill file

For agents that call the REST API directly, DataFuel publishes a skill file that explains endpoint choice, request shapes, the result envelope and error handling in a form written for models:

https://scraping-api.datafuel.ai/skill.md
https://scraping-api.datafuel.ai/llms.txt (same content)

Drop it into your agent's context or skills directory. The full machine-readable schema is at https://scraping-api.datafuel.ai/docs/openapi.yaml.

Keep the key out of prompts

Configure the key in the MCP client or in the agent's environment. Never paste it into a chat message; transcripts get logged and shared.