For future reference:
User-agent: GPTBot
Disallow: /
In case your wondering what this is about, see this article by the Washington Post, which explains how generative AIs work. Fun-fact: eay.cc is responsible for 0.000002% of the tokens in Google’s C4 dataset.


