Crawling infrastructure for AI agents

Clean, typed data from the sites that block everyone else.

datamint crawls your hardest targets and returns LLM-ready Markdown or schema-validated JSON in a single call. No proxy pools to maintain, no parsers to re-fix every sprint.

See how it works

1,000 free credits every month · No card required · Blocked requests cost nothing

mint.sh
$ curl -X POST https://api.datamint.xyz/v1/mint \
  -H "Authorization: Bearer mint_live_xxx" \
  -d '{
    "url": "https://example.com/item/123",
    "schema": "ecommerce_item",
    "js": true
  }'
response200
{
  "title": "Aeron Chair, Graphite",
  "price": 1795.00,
  "sku": "HM-AER-GRA",
  "availability": "in_stock",
  "confidence": 0.98
}
1,000credits / month
Free every month, not a one-off trial.
No cardto start
Sign up and make requests without a sales call.
On deliveryonly
A blocked request that returns nothing costs nothing.
Publicdata only
No paywalls, no authenticated sessions, ever.

What you get back

Your scraper gets a DOM. You get a record.

Bot-defended pages return the same thing to everyone: markup built for a browser, wrapped in a challenge. Stripping that into something a model can read is the part that keeps breaking.

raw response403 · challenged
<!DOCTYPE html><html class="no-js">
<head><script>window._cf_chl_opt=
{"cvId":"3","cZone":"xxxxx","cType":
"interactive","cNounce":"84719"}
</script></head><body>
<div id="challenge-running">
  <div class="lds-ring"><div></div>
  <span>Checking your browser…</span>
</div>
<!-- 412,880 bytes, 0 usable fields -->
datamint response200 · minted
{
  "title": "Aeron Chair, Graphite",
  "price": 1795.00,
  "currency": "USD",
  "sku": "HM-AER-GRA",
  "availability": "in_stock",
  "_mint_meta": {
    "confidence": 0.98,
    "minted_at": "2026-09-13T07:45:02Z"
  }
}

Same shape, every run

Fields are validated against your schema before the response leaves. A miss is a flagged request, never a silent null.

Confidence included

Every object carries a score. Below 0.80 the request reroutes through fallback rather than returning holes.

Provenance attached

Capture timestamp and source URL travel with the record, so any value can be traced to the page state that produced it.

How it works

Router first, browser last.

Most requests never touch a headless browser, which is why the per-page cost stays where it is. Escalation is automatic and only goes as far as the target forces it.

  1. 01

    Point at a URL

    One POST with the target and, optionally, the shape you want back. Nothing to install, no proxy pool to configure, no browser to keep alive.

  2. 02

    We pick the cheapest route that works

    A static TLS-spoofed fetch first. A headless browser only if the page needs JavaScript. Residential routing only if the target rejects everything else. The route is remembered per target.

  3. 03

    Get back something you can use

    Clean Markdown for a model to read, or JSON validated against your schema. Either way it arrives with a confidence score and provenance attached.

A request that cannot be served costs nothing. You are billed for pages that come back with usable data, not for attempts.

Ingestion

Four ways in. One normalized shape out.

Every surface runs the same normalization pipeline, so data arriving by synchronous call and data arriving by overnight batch crawl have identical structure.

POST /v1/mint800ms–2.5s

Synchronous extract

Single-page fetch with rendering options and an extraction schema. Returns cleaned Markdown or validated JSON in under 2.5s.

View documentation→
POST /v1/crawlAsync

Asynchronous deep crawl

Traverses subdomains or path patterns with breadth and depth controls, regex filtering, deduplication, and automatic sitemap discovery. Delivers by webhook or cloud storage dump.

View documentation→
POST /v1/extractSchema-driven

Zero-code schema extraction

Supply a JSON Schema or Pydantic model. The engine handles DOM traversal, heuristic extraction, and LLM-assisted structural mapping so the data shape comes back uniform.

View documentation→
MCP mcp.datamint.xyzLive · Agents

Model Context Protocol server

Native MCP interface so agents in Claude Desktop, Cursor, and LangChain call datamint as an internal tool for web retrieval without scraper boilerplate.

View documentation→

Build vs. buy

You can build this. The question is what it costs to keep.

The first version of a scraper is a day of work. The version that survives twelve months of anti-bot updates is a team.

When a target changes its markup
In-house

Someone debugs it. Usually mid-sprint, usually after a customer notices missing rows.

datamint

Schema drift is detected on the next run and flagged before your pipeline sees a gap.

When a target tightens its bot defence
In-house

New fingerprinting to chase, new proxy tier to buy, and a week of tuning success rates back up.

datamint

Routing escalates automatically. The work happens on our side of the boundary.

What it costs to run
In-house

Proxy bandwidth, browser infrastructure, and the engineer-hours that never leave the backlog.

datamint

Per page, quoted up front. A blocked request is not billed.

Compliance and redaction
In-house

PII handling is a pipeline you build, test, and audit yourself.

datamint

Redaction and robots handling sit inside the pipeline before anything is stored or returned.

Who this is for

Built for teams whose data pipeline is the product.

If a broken scraper means a broken sprint, the cost of maintaining one in-house is already higher than the cost of not doing it.

AI agent frameworks & LLM labs

Use case
Real-time agentic research, retrieval-augmented generation, ground-truth verification
Pain
Cloudflare and Turnstile blocks, DOM bloat consuming context windows
Typical volume
1M – 10M pages / mo

Alternative data & hedge funds

Use case
Ticker mentions, consumer sentiment, pricing indices, inventory trackers
Pain
Unannounced schema changes breaking parsers, IP bans mid-crawl
Typical volume
5M – 50M pages / mo

E-commerce intelligence platforms

Use case
Competitor price tracking, stock levels, marketplace catalog indexing
Pain
Aggressive rate-limiting, CAPTCHAs, regional pricing variation
Typical volume
3M – 20M pages / mo

B2B lead enrichment platforms

Use case
Firmographic extraction, team-size monitoring, job board crawling
Pain
JavaScript-heavy SPAs, dynamic hydration, fingerprinting
Typical volume
500k – 5M pages / mo

Request routing

Two engines, three proxy tiers, one decision.

Every request enters through a Rust distributor and escalates only as far as it has to. Most targets never reach the browser fleet, which is what keeps unit cost down.

Lightweight HTTP engine

Direct TLS and JA4 spoofing with HTTP/2 frame fingerprinting that mimics current Chrome client hellos. Handles the majority of targets without the cost of a browser.

Custom TLS network stack · no headless overhead

Headless browser fleet

Sandboxed Chromium over the DevTools Protocol with stealth patches applied. Full DOM execution for JavaScript-heavy targets that hydrate their content after load.

Anti-fingerprinting patches · JS hydration

Tier 180% of volume

Datacenter Backbone

High-throughput backbone pools carrying the bulk of traffic. Sub-second latency with zero browser overhead.

1 credit / page · bandwidth included

Tier 2Smart Fallback

Static ISP & Headless

Clean ASN pools for mid-tier targets that block generic datacenter IP ranges.

5 credits / page · bandwidth included

Tier 3Auto-Escalated

Stealth Armor & Mobile

Rotating residential IPs with Bayesian fingerprint mutation for hard targets (Turnstile, Kasada, DataDome).

15 credits / page · bandwidth included

Escalation is automatic and recorded per target, so a site that once needed residential routing is not re-escalated on the next run.

Data governance

Compliance is architecture here, not a policy page.

Redaction and backoff run inside the pipeline. They are not settings a customer has to remember to enable.

Scope

Public data doctrine

Collection is restricted to publicly accessible pages. No paywalls, no authenticated user sessions, no circumvention of access controls. The legal posture follows hiQ Labs v. LinkedIn and Van Buren v. United States.

Configurable

Robots.txt engine

Compliance is a configurable flag, not a hidden default. Strict compliance is available for every customer; enterprise accounts can route targeted crawling under explicit indemnification terms.

GDPR / CCPA

PII stripping at ingestion

Regex and NLP redaction run before anything is stored or returned, removing SSNs, payment card numbers, personal phone numbers, and private email addresses at the edge of the pipeline.

Zero-DoS

Courtesy backoff

Adaptive backoff keyed to target response codes. When a target returns 429 or 503, we back off immediately, so our fleet never applies denial-of-service pressure to a site.

Enterprise agreements from $1,499 / month include a master services agreement, a bespoke data processing addendum, signed service level commitments, and direct access to on-call engineering.

Pricing

Priced per page, not per seat.

Credits are consumed per request and scale with how hard the target is to reach. A static fetch costs a credit. Residential routing and a schema pass cost more.

Hobby

$0forever

Included credits

1,000

  • Shared datacenter proxies
  • Markdown minting
  • 5 concurrent requests
  • No card required
Start free

Builder

$49per month

Included credits

60,000

$0.82 standard · $12.25 stealth armor / 1k pages

  • Unlimited concurrency
  • Residential proxy fallback
  • JavaScript rendering
  • Structured JSON extraction
  • Schema-validated output
Choose Builder

Scale

Most popular
$299per month

Included credits

450,000

$0.66 standard · $9.97 stealth armor / 1k pages

  • Unlimited concurrency
  • Automated schema drift alerts
  • Priority proxy pools
  • Everything in Builder
Choose Scale

Enterprise

$1,499from, per month

Included credits

Custom, 2M+

Volume tiers below $0.40 / 1k pages

  • Dedicated proxy pools
  • Signed 99.9% success-rate SLA
  • Custom parsers and hotfixes
  • MSA, DPA, IP indemnification
Talk to sales

Credit multipliers

$1 = 1,000 credits

Static fast HTTP fetch
1
credits per page
Dynamic headless render
5
credits per page
Stealth armor bypass
15
credits per page
Structured schema extraction
+2
added to base

Questions

The things engineers ask before signing up.

What happens when a target blocks a request?

The request escalates: a static fetch first, then a headless render, then residential routing with fingerprint mutation. If all of it fails, you get a typed error rather than an empty payload, and no credits are consumed. You are billed for pages that return usable data.

Is this legal?

We collect publicly accessible data only. No paywalls, no authenticated sessions, and no circumvention of access controls: the posture set out in hiQ Labs v. LinkedIn and Van Buren v. United States. Robots.txt compliance is a configurable flag, and enterprise agreements include IP indemnification for public data collection.

Do you store the pages you crawl?

Screenshots, capture timestamps, and MHTML snapshots are retained alongside an extraction so a record can be traced back to the page state that produced it. Payloads are PII-redacted at ingestion: SSNs, card numbers, personal phone numbers, and private emails are stripped before storage or return.

Can I define my own output schema?

Yes. Supply a JSON Schema or Pydantic model to /v1/extract and the engine handles DOM traversal and structural mapping to fit it. There are pre-mapped schemas for common shapes like ecommerce_item if you would rather not define one.

We already have scrapers that mostly work. Why switch?

The word is mostly. Track how much of the last quarter went into keeping them running rather than into the product, and how often a target's markup change reached your customers before it reached your team. That number is the case for switching.

Can AI agents like Cursor or Claude use Datamint directly?

Yes. Datamint operates an official Model Context Protocol (MCP) server at https://mcp.datamint.xyz/sse. Agents in Cursor, Claude Code, Claude Desktop, Windsurf, or LangChain connect via standard SSE or local stdio with zero glue code, gaining native tools to mint pages, extract structured schemas, and run deep crawls directly.

What if we only need a few thousand pages a month?

Start on the free tier: 1,000 credits a month, recurring, no card. Builder covers 60,000 credits at $49 if you outgrow it. There is no minimum commitment on either, and no sales call to get started.

Stop maintaining scrapers. Start shipping.

Point datamint at the targets your team keeps losing. Get typed data back, and stop rebuilding parsers every sprint.

Compare plans

1,000 credits free every month · No card required · Blocked requests cost nothing