Robots.txt - manually adding a line?

Question
1

I am looking for a way to add a line in sparkle’s created robots.txt. Is there a way to do so?

Reason is: I have, for sentimental reasons, a copy of the historic website in a subfolder on my webserver, and put an external link in the footer. I don’t want the webcrawlers to index that old stuff. Since I am not sure if they just look into what’s defined in the sitemap.xml, I manually edited the robots.txt on the server to disallow that folder. But naturally, every sparkle publish erases that.

2

Suggestion:

Use “Search Engine Control” and disallow Indexing.

Maybe that has to be done for each page. Not sure if you can “hide” a folder.

Mr. F.

3

Thx, Fozzie, but that’s something different (works for pages created within Sitely).

4

Ok. Maybe a .htaccess file could do the Trick when you have this line:

disallow:/oldfolder/

That should keep crawlers away from it.

Mr. F.