Terms of Use

The website provides technical information and tools on an as-available basis. You are responsible for reviewing generated rules, protecting private content with real access controls, and using the service lawfully without abuse.

Last updated: July 15, 2026

Acceptance and scope

These Terms of Use apply when you browse the website or use its generator, checker, crawler references, templates, guides, and related features. If you do not agree with these terms, do not use the service. Mandatory rights that cannot lawfully be excluded remain unaffected.

Technical information, not a guarantee

The service provides general technical information about robots.txt, crawlers, search discovery, AI-related access, and website configuration. It does not provide legal advice, security certification, crawler-provider support, search-engine guarantees, licensing advice, or a promise that any automated client will obey a published rule.

Robots.txt is not authentication or authorization. You are responsible for protecting private, paid, licensed, personal, or confidential material with appropriate server-side controls.

Your responsibility for generated output

Generated robots.txt text and checker results are starting points for review. Before publication, verify the exact crawler tokens, paths, host, sitemap URL, search impact, and compatibility with your infrastructure. You are responsible for testing changes and maintaining backups of your own configuration.

Acceptable use

You may use the public tools for legitimate website administration, research, education, and evaluation. You must not misuse the service to attack, probe, overload, disrupt, bypass access controls, submit secrets or unlawful material, automate abusive request volumes, interfere with other users, or attempt unauthorised access to administration or infrastructure.

The operator may restrict access, rate-limit requests, remove content, or suspend features when reasonably necessary for security, maintenance, abuse prevention, legal compliance, or service integrity.

Third-party names, sources, and websites

Crawler names, company names, documentation links, and product descriptions belong to their respective owners. Their use is descriptive and does not imply affiliation, endorsement, or authority to act for those providers. Third-party documentation and crawler behavior can change without notice.

The checker requests public robots.txt files from domains submitted by users. The operator does not control those websites, their availability, content, security, or response accuracy.

Availability and changes

The service may be changed, interrupted, limited, or discontinued. Features, crawler records, examples, policies, and these terms may be updated as the project develops. Continued use after an updated version is published constitutes acceptance to the extent permitted by applicable law.

Disclaimers and limitation

The service is provided on an “as available” basis without a promise of uninterrupted operation, complete accuracy, fitness for a specific purpose, search ranking, crawler compliance, AI citation, or protection from scraping. To the extent permitted by applicable law, the operator is not responsible for indirect, incidental, special, consequential, or business losses arising from use of, inability to use, or reliance on the service.

Privacy and contact

Use of personal data is described in the Privacy Policy and Cookie Policy. Questions about these terms or reports of misuse should be sent through the active method on the Contact page.

FAQ

Does the generator guarantee that my robots.txt file is legally or technically correct?

No. It produces a technical starting point. You must review the output against your own site, search goals, access controls, contracts, and applicable requirements.

Is the website responsible if a crawler ignores my rules?

No crawler compliance is guaranteed. Robots.txt is a protocol that compliant crawlers are requested to honor, while non-compliant clients may ignore it.

Can I use the checker to test any public domain?

You may use it for legitimate evaluation of public robots.txt files, subject to these terms, reasonable request volumes, and applicable law. Do not use the service for abuse or unauthorised probing.

Are crawler-provider names endorsements or partnerships?

No. Names and documentation references are used descriptively and do not imply affiliation, sponsorship, endorsement, or representation.

Can the tools or terms change?

Yes. Features, crawler records, examples, and terms may be updated as providers, standards, and the project change.

Related tools

Related pages