# JobSpidey public crawler policy
# Public marketing, template, resource, role, and market pages may be crawled.
# App, API, admin, checkout, and account surfaces remain private or noindexed.
# AI-readable site map: https://jobspidey.com/llms.txt

User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: OAI-SearchBot
User-agent: PerplexityBot
User-agent: ClaudeBot
User-agent: Claude-SearchBot
User-agent: Google-Extended
User-agent: Applebot
User-agent: Applebot-Extended
Allow: /
Disallow: /api/
Disallow: /admin/

User-agent: *
Allow: /
Disallow: /api/
Disallow: /admin/

Sitemap: https://jobspidey.com/sitemap.xml