MCP serversx402 APIsAI dev toolsFree APIsSearchSubmit a toolGet listed / Advertise
Home › x402 pay-per-call APIs › PennyAPI Robots Permission Check

PennyAPI Robots Permission Check

Check robots.txt before crawling or scraping a page. Decide whether GPTBot, ClaudeBot, Googlebot, or another crawler may fetch the exact URL and return ALLOW/DENY, winning rule, crawl-delay, sitemap URLs, and live provenance.

Visit pennyregwatch.com

Hostpennyregwatch.com
Price$0.001–$0.004 per call
NetworkBase
Paid endpoints listed7
Calls, last 30 days*21
Most unique payers on one endpoint (30 days)*4
Last paid call*2026-10-13
Tagsagent-utility, research, real-time, robots-txt, crawler, scraping, crawl-permission, http-headers

* From the public x402 Bazaar index (snapshot 2026-10-18T04:07Z); reported by the facilitator, not measured by us.

Endpoints

URLMethodPrice
https://pennyregwatch.com/v1/robots
Check robots.txt before crawling or scraping a page. Decide whether GPTBot, ClaudeBot, Googlebot, or another crawler…		GET$0.001
https://pennyregwatch.com/v1/headers
Inspect HTTP response headers before an agent trusts or integrates a URL: content type, cache controls, CORS, HSTS,…		GET$0.001
https://pennyregwatch.com/v1/search
Real-time web search for AI agents. Search the web for current and up-to-date information, news, facts, research,…		GET$0.004
https://pennyregwatch.com/v1/x402/preflight
Preflight an unfamiliar x402 paid API before spending. Validate the live 402 payment challenge, network, USDC asset,…		GET$0.001
https://pennyregwatch.com/v1/redirect
Resolve a URL redirect chain before an agent follows a link. Return the final URL, HTTP status, redirect count, and…		GET$0.001
https://pennyregwatch.com/web/places
Real-time local business and places search for AI agents. Find restaurants, coffee shops, stores, services, and nearby…		GET$0.004
https://pennyregwatch.com/v1/metadata
Extract webpage metadata for agent research and URL triage: title, description, canonical URL, Open Graph, and Twitter…		GET$0.001

Source: x402 Bazaar discovery API (Coinbase CDP) · Licence: Public discovery metadata; descriptions by each provider. Listed in x402 pay-per-call APIs. We list tools; we do not test, rate or endorse them.

Is this your tool? Update the description or add details for free via Submit / claim. Want it at the top of x402 pay-per-call APIs? See Get listed / Advertise (paid placements are labelled Sponsored).

Related

AX1 Console

Send a Base token contract address, get an AI-written verdict: onchain metrics, a plain-language take, and a shareable verdict card. Entertainment,…

blockrun.ai

AI chat completions API - pay per request with USDC on Base

X Pay

Payment receipt with transaction details

x402.sniperx.fun

Get SmartMoney trading activities on token (sorted by time descending)

agents.chain.link

Chainlink For Agents: execute a catalog workflow by workflow name.

Pocket Network

An agent-only messaging network. Agents register with an Ed25519 key and talk in public rooms and end-to-end encrypted private rooms and DMs. POST…

keyring-agent.blockchhub.link

Pay the send-native execution fee (native coin transfer)

StableEnrich

Exa Search - Neural search across the web