Server can't open database


Message boards : Number crunching : Server can't open database

Message board moderation

To post messages, you must log in.
AuthorMessage
Ananas

Send message
Joined: 18 Mar 13
Posts: 32
Credit: 2,506,320
RAC: 0
Message 3320 - Posted: 14 Jul 2014, 20:47:06 UTC
I'm getting this message way to often now.

Maybe a "robots.txt" file would help to keep search spiders away from results, host and user statistics.

The only database driven area where search spiders probably make sense are the forum contents.
ID: 3320 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kyong
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 9 Jun 12
Posts: 584
Credit: 52,667,664
RAC: 0
Message 3352 - Posted: 18 Jul 2014, 9:02:15 UTC
I discover that this is done by some clients that have set their connection to server by every 1 seconds but they are also from different IP addresses and I don't want to block the IPs. I have to figure out how to reduce it but I don't know how to set some timeouts in BOINC server. I didn't find anything useful in documentation and I don't want to ban these hosts.
I have to figure it out somehow.
ID: 3352 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
alexander

Send message
Joined: 28 Apr 13
Posts: 87
Credit: 26,716,176
RAC: 184
Message 3361 - Posted: 18 Jul 2014, 16:22:24 UTC - in response to Message 3352.  
I discover that this is done by some clients that have set their connection to server by every 1 seconds but they are also from different IP addresses and I don't want to block the IPs. I have to figure out how to reduce it but I don't know how to set some timeouts in BOINC server. I didn't find anything useful in documentation and I don't want to ban these hosts.
I have to figure it out somehow.


I'm crunching now my 11th year and have heard about similar things at another project.
It can be a wanted attack, a faulty pc, an overheated/overclocked pc left alone or a hijacked pc. Anyway, in every possible scenario it makes no sense that a single user (pc) blocks a whole project.
If you're shure about the ip-address, send the user a PM and block the address for a day or so. Maybe the user is not aware about the problem and will not react in any different scenario.
ID: 3361 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kyong
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 9 Jun 12
Posts: 584
Credit: 52,667,664
RAC: 0
Message 3365 - Posted: 19 Jul 2014, 6:10:32 UTC
It seems to be good now. But it was many pc's and more IP addresses, but every with the same first two parts of IP.
ID: 3365 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 1 Jan 14
Posts: 300
Credit: 32,043,266
RAC: 15,167
Message 3368 - Posted: 19 Jul 2014, 11:20:29 UTC - in response to Message 3365.  
It seems to be good now. But it was many pc's and more IP addresses, but every with the same first two parts of IP.


Many users too? Or one user with LOTS of pc's? If many users it might not be there fault, after all how could MANY users all have the same problem all at once.
ID: 3368 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kyong
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 9 Jun 12
Posts: 584
Credit: 52,667,664
RAC: 0
Message 3370 - Posted: 19 Jul 2014, 11:39:05 UTC
I didn't check it but according to IPs, I suppose it was one user.
ID: 3370 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Ananas

Send message
Joined: 18 Mar 13
Posts: 32
Credit: 2,506,320
RAC: 0
Message 3371 - Posted: 19 Jul 2014, 20:22:03 UTC

Last modified: 19 Jul 2014, 20:32:29 UTC
I still think that a /robots.txt would make sense anyway, as search spiders crawling the results and hosts pages can produce major database load.

Nearly all BOINC projects have it. There used to be a file "sample_robots.txt" in the CVS- (later SVN-) path "html/user/sample_robots.txt", not sure if it is still there.

In the old version, default had been "allow" and they excluded a few selected directories from crawlers, must have been like this :

User-agent: *
Disallow: /account
Disallow: /edit_
Disallow: /host_
Disallow: /prefs_
Disallow: /result
Disallow: /team
Disallow: /workunit

Afaik. later versions are much harsher defaulting to "disallow everything" and selectively allowing a few directories, something like :

User-agent: *
Allow: /index.php
Allow: /info.php
Allow: /intro.php
Allow: /old_news.php
Allow: /rss_main.php
Disallow: /

The new version has the advantage that people and bots cannot peak into what they are not supposed to see - just in order to explore especially that ;-)

Of course, you would have to add the "/boinc" base directory to all path lines (except for the "/" itself).

edit : You even have the sample installed http://asteroidsathome.net/boinc/sample_robots.txt. If you edit it to add the "/boinc" subdirectory, make sure to let it have *ix style line ends, DOS line ends (x0d x0a) render the file useless.
ID: 3371 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 1 Jan 14
Posts: 300
Credit: 32,043,266
RAC: 15,167
Message 3374 - Posted: 20 Jul 2014, 10:39:16 UTC - in response to Message 3370.  
I didn't check it but according to IPs, I suppose it was one user.


Isn't this why Boinc did the automatic back off on the client side? So someone wasn't hammering the door to get in and blocking everyone else and tying up all the resources? I would bet there is a way to do it on your side, I have no idea how, but Rom or even Dr. A might be able to help if you ask them. If you are friends with another projects Admin they might know too.
ID: 3374 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kyong
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 9 Jun 12
Posts: 584
Credit: 52,667,664
RAC: 0
Message 3377 - Posted: 20 Jul 2014, 12:20:48 UTC

Last modified: 20 Jul 2014, 12:23:40 UTC
Thanks, I will ask them about it. I am currently trying to solve error in scheduler after BOINC server software.

About the robots.txt, I added it there.
ID: 3377 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile BilBg
Avatar

Send message
Joined: 19 Jun 12
Posts: 221
Credit: 623,640
RAC: 0
Message 3378 - Posted: 20 Jul 2014, 16:03:27 UTC - in response to Message 3377.  
 
These people know much about BOINC code (changes, trac, tickets, bugs, ...) and respond promptly:

In 'random' order:

Richard Haselgrove
http://setiathome.berkeley.edu/show_user.php?userid=5509
http://boinc.berkeley.edu/dev/show_user.php?userid=1675

Josef W. Segur
http://setiathome.berkeley.edu/show_user.php?userid=276784

Ageless
http://setiathome.berkeley.edu/show_user.php?userid=41965
http://boinc.berkeley.edu/dev/show_user.php?userid=8

 



- ALF - "Find out what you don't do well ..... then don't do it!" :)
ID: 3378 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Server can't open database