Free llms.txt & AI Crawler Access Check
See which crawlers can access your content and what they can read.
Start with a studio whose work fits your space and style.
Explore their projects and discuss your renovation.
Visit website ↗Hi, we'd love to discuss our home…
Thanks for getting back to me…
Could you tell me more about…
Tuesday works well for us…
Details captured. Follow-up organised.
Can AI crawlers read your website?
Access rules and JavaScript-heavy pages can change what a crawler can see. A clear access review helps separate a technical obstacle from a content opportunity.
What you’ll receive
A summary of crawler access rules, publicly readable content and any relevant recommendations for your robots.txt and llms.txt files.
- Relevant robots.txt rules
- Training and search crawler distinctions
- Content available without JavaScript
- llms.txt presence and content, where applicable
Use the recommendations yourself or talk to DonutLabs about putting them into action.
Request my crawler access check.
Tell us about your business. We’ll be in touch about your free review.
A simple way to get started.
-
Tell us what you need
Share your website, contact details and the context for your review.
-
We confirm the next step
DonutLabs follows up to clarify the scope and any information needed.
-
Get practical direction
Understand the findings and decide what to tackle next, with help available if you need it.
Before you get started.
Do I need llms.txt to appear in AI answers?
No. It is a proposed convention, not a requirement or guarantee of inclusion. The review considers it alongside access rules and the actual content crawlers can read.
Is the initial review free?
Yes. This offer is for an initial review. Any deeper audit or implementation work would be discussed and quoted separately.
Do I need to commit to further work?
No. The review is a starting point. You decide whether to use the recommendations yourself or ask us to help.
What happens after I enquire?
DonutLabs will contact you about your request and confirm what’s needed. This is a request for a review, not an instant online scan.