I'm getting this message way to often now.

Maybe a "robots.txt" file would help to keep search spiders away from results, host and user statistics.

The only database driven area where search spiders probably make sense are the forum contents.

I discover that this is done by some clients that have set their connection to server by every 1 seconds but they are also from different IP addresses and I don't want to block the IPs. I have to figure out how to reduce it but I don't know how to set some timeouts in BOINC server. I didn't find anything useful in documentation and I don't want to ban these hosts.
I have to figure it out somehow.

I discover that this is done by some clients that have set their connection to server by every 1 seconds but they are also from different IP addresses and I don't want to block the IPs. I have to figure out how to reduce it but I don't know how to set some timeouts in BOINC server. I didn't find anything useful in documentation and I don't want to ban these hosts.
I have to figure it out somehow.

I'm crunching now my 11th year and have heard about similar things at another project.
It can be a wanted attack, a faulty pc, an overheated/overclocked pc left alone or a hijacked pc. Anyway, in every possible scenario it makes no sense that a single user (pc) blocks a whole project.
If you're shure about the ip-address, send the user a PM and block the address for a day or so. Maybe the user is not aware about the problem and will not react in any different scenario.

It seems to be good now. But it was many pc's and more IP addresses, but every with the same first two parts of IP.

It seems to be good now. But it was many pc's and more IP addresses, but every with the same first two parts of IP.

Many users too? Or one user with LOTS of pc's? If many users it might not be there fault, after all how could MANY users all have the same problem all at once.

I didn't check it but according to IPs, I suppose it was one user.

I still think that a /robots.txt would make sense anyway, as search spiders crawling the results and hosts pages can produce major database load.

Nearly all BOINC projects have it. There used to be a file "sample_robots.txt" in the CVS- (later SVN-) path "html/user/sample_robots.txt", not sure if it is still there.

In the old version, default had been "allow" and they excluded a few selected directories from crawlers, must have been like this :

User-agent: *
Disallow: /account
Disallow: /edit_
Disallow: /host_
Disallow: /prefs_
Disallow: /result
Disallow: /team
Disallow: /workunit

Afaik. later versions are much harsher defaulting to "disallow everything" and selectively allowing a few directories, something like :

User-agent: *
Allow: /index.php
Allow: /info.php
Allow: /intro.php
Allow: /old_news.php
Allow: /rss_main.php
Disallow: /

The new version has the advantage that people and bots cannot peak into what they are not supposed to see - just in order to explore especially that ;-)

Of course, you would have to add the "/boinc" base directory to all path lines (except for the "/" itself).

edit : You even have the sample installed http://asteroidsathome.net/boinc/sample_robots.txt. If you edit it to add the "/boinc" subdirectory, make sure to let it have *ix style line ends, DOS line ends (x0d x0a) render the file useless.

I didn't check it but according to IPs, I suppose it was one user.

Isn't this why Boinc did the automatic back off on the client side? So someone wasn't hammering the door to get in and blocking everyone else and tying up all the resources? I would bet there is a way to do it on your side, I have no idea how, but Rom or even Dr. A might be able to help if you ask them. If you are friends with another projects Admin they might know too.

Thanks, I will ask them about it. I am currently trying to solve error in scheduler after BOINC server software.

About the robots.txt, I added it there.

 
These people know much about BOINC code (changes, trac, tickets, bugs, ...) and respond promptly:

In 'random' order:

Richard Haselgrove
http://setiathome.berkeley.edu/show_user.php?userid=5509
http://boinc.berkeley.edu/dev/show_user.php?userid=1675

Josef W. Segur
http://setiathome.berkeley.edu/show_user.php?userid=276784

Ageless
http://setiathome.berkeley.edu/show_user.php?userid=41965
http://boinc.berkeley.edu/dev/show_user.php?userid=8

 

---

- ALF - "Find out what you don't do well ..... then don't do it!" :)