Consider a "noindex" key #123
Replies: 4 comments
-
Here's my two cents. As I see it, either the file is willingly incomplete or its contents are wrong. In the first case (fe. placeholder, under construction, test repo) the developer doesn't want the file to be crawled yet, and they should be able to express that, but in the second case (fe. copied from another repo or plain wrong info) the file should just be fixed. I'd KISS and just hint in the documentation to rename the file to something like |
Beta Was this translation helpful? Give feedback.
-
Yeah, I discarded the filename change in the first place since that always implies changing all the places where that file is used (I can think of a CI calling the validator already in use in some repos). |
Beta Was this translation helpful? Give feedback.
-
I didn't know such a tool was in the works, nice! What about using a robots.txt file in the repo's root directory? It's a well known format and we could reuse the same solution in similar circumstances, both present (projects_settings.yml?) and future.
|
Beta Was this translation helpful? Give feedback.
-
Moved to GitHub Discussions |
Beta Was this translation helpful? Give feedback.
-
In the last couple of months we faced situations where the
publiccode.yml
:As such, I wonder if it could be feasible to add a decentralized mechanism to tell the spiders not to index such a file.
I was thinking at something like a boolean "noindex" key which simply states whether the repo should be indexed or not yet.
What do you think about this?
I think that acting in a centralized way from the spider perspective (e.g., using something like a blacklist) is harder since it requires the intervention of the spider's responsibles who are external from the software publishers group, it does not scale, it is very error prone and it requires a communication channel to be established at least twice.
The main risk is of course that someone could leave the value to "true" and such a software would never be indexed.
What do you think?
Beta Was this translation helpful? Give feedback.
All reactions