For website owners

ProfileReader

ProfileReader is the program that reads the public pages of a business's website so that a clear, structured profile of the business (what it sells, its prices, its opening hours, where it is) can be prepared. Its requests identify themselves with the user agent ProfileReader/1.0 (+https://www.askpharo.com/bot), which always links to this page.

What it does

• It reads only public pages of one website, never anything behind a login.
• It picks the pages a customer's questions need (contact, hours, offerings and prices, policies, location), not every page, and at most a few dozen pages per website.
• It waits between requests and follows a Crawl-delay up to ten seconds.
• It never submits a form, never follows an email or telephone link, never logs in and never books anything.
• It does not contact the business: nothing is sent to anyone.

How a website says no

Any one of these is enough. ProfileReader looks for all of them before it reads a page, and a website that opts out is left alone.

robots.txt

Disallow the whole site, or only some paths, for ProfileReader (or for every program with *). A Crawl-delay is kept up to ten seconds; a longer one is treated as a refusal.

User-agent: ProfileReader
Disallow: /

Text and data mining (EU)

The TDM reservation protocol: a file, a response header or a meta tag.

/.well-known/tdmrep.json
[{"location": "/", "tdm-reservation": 1}]

TDM-Reservation: 1
<meta name="tdm-reservation" content="1">

Page and image directives

noai, none or noindex in a robots meta tag or an X-Robots-Tag header keep that page unread. noimageai keeps its images unread.

<meta name="robots" content="noai, noimageai">
X-Robots-Tag: noai
X-Robots-Tag: profilereader: none

AI-usage preferences

A Content-Usage line in robots.txt or a response header that refuses AI use is read as a refusal for the whole website.

User-agent: *
Content-Usage: train-ai=n

A change takes effect the next time ProfileReader visits; nothing it has already prepared from public pages is published, and it is shown only to the people preparing it.