Transparency

How search engines, AI models, and researchers can interact with our public catalogue.

Sitemaps

Our sitemap is partitioned by basin to ensure accurate and reliable indexing of our large catalogue.

  • /sitemap.xml Master index of all sub-sitemaps.
  • /sitemap-main.xml Core pages, regional hubs, and informational content.
  • /sitemap-wells.xml Combined sitemap of every well page in one file.
  • /sitemap-wells-{basin}.xml Individual sitemaps for each basin, updating dynamically as our database grows.

Robots and Crawlers

Our /robots.txt outlines automated access. We allow the indexing of all public-facing catalogue pages, basin overviews, and directory content. We apply a standard 1-second crawl delay to ensure platform stability. We do not block standard, well-behaved bots.

AI Assistants and LLMs

We explicitly permit AI indexing for our public catalogue. Our robots.txt includes the following directives: 

Content-Signal: search=yes, ai-train=yes, ai-input=yes

To assist language models in accurately understanding our site structure and metadata, we publish:

  • /llms.txt A concise overview of our services.
  • /llms-full.txt Extended documentation, including API schemas and spatial filters.

Providing this context ensures that when geoscientists and data engineers use AI tools to research Australasian basins, the models can accurately direct them to our index. Note: these files only describe our catalogue. The proprietary well data and QC framework remain secure and are not accessible for AI training.

Security Disclosures

If you identify a vulnerability or have a security concern, please refer to our standard disclosure policy at /.well-known/security.txt or contact us directly at hello@occam.com.au.