Slower, validation pending??


Message boards : Number crunching : Slower, validation pending??

Message board moderation

To post messages, you must log in.
AuthorMessage
Dagorath

Send message
Joined: 16 Aug 12
Posts: 293
Credit: 1,116,280
RAC: 0
Message 2724 - Posted: 14 Mar 2014, 1:08:28 UTC - in response to Message 2723.  

Last modified: 14 Mar 2014, 1:11:08 UTC
Aww come on now. We don't hold it against you that you talk like a whore. We know the mind controlling boogeyman computer holding your credits ransom made you talk that way.

And they don't depend on luck here, Andy, they use skill and facts. You should try it sometime.
BOINC FAQ Service
Official BOINC wiki
Installing BOINC on Linux
ID: 2724 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dagorath

Send message
Joined: 16 Aug 12
Posts: 293
Credit: 1,116,280
RAC: 0
Message 2726 - Posted: 14 Mar 2014, 2:41:30 UTC - in response to Message 2717.  
The "problem" of crunchers with a big cache of tasks and therefore long waiting times for validation, could easily be solved.
Just limit the number of task to a maximum of 5 per core, and the time to complete them to 5 days.
There are more projects that do that.!


+1
BOINC FAQ Service
Official BOINC wiki
Installing BOINC on Linux
ID: 2726 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 1 Jan 14
Posts: 300
Credit: 32,217,068
RAC: 8,278
Message 2729 - Posted: 14 Mar 2014, 12:14:26 UTC - in response to Message 2719.  
Apparently, these large users have the resources to attach several machines to this project. Unfortunately, these large "new" users have effectively placed all of the other "crunchers" under their control if they are not limited as to how many machines they can have attached to this project. It doesn't make sense for one or a few people to "take" utter control of a public charity without some type of controls already set by the administrators. My point is, something needs to be done or you will possibly find asteroids@home minus several of it's crunchers because of the whims of a few people. It doesn't make sense.


In about 2 weeks the new guys in and outs will balance out, just like ours did when we first started, and everything will be back to normal. It is a temporary thing, the new guys always have a settling in time and this new guy just brought ALOT of resources to the table. That means he sucked up alot of workunits, but as he returns them we will get our credits just fine. There are PLENTY of Asteroids out there to find and map, it's not like the project is going to end tomorrow, this is LONG TERM thing.
ID: 2729 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 1 Jan 14
Posts: 300
Credit: 32,217,068
RAC: 8,278
Message 2730 - Posted: 14 Mar 2014, 12:17:21 UTC - in response to Message 2717.  
The "problem" of crunchers with a big cache of tasks and therefore long waiting times for validation, could easily be solved.
Just limit the number of task to a maximum of 5 per core, and the time to complete them to 5 days.
There are more projects that do that.!


The problem with HUGE users is that they will just go elsewhere with their resources if they can't get what they want here. There are ALSO plenty of projects that do NOT throttle the workunit load. I am guessing most projects would LOVE to have some super user come in with this guys resources and start crunching for them. This ONE guy is like adding 100 of us normal folks.
ID: 2730 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dagorath

Send message
Joined: 16 Aug 12
Posts: 293
Credit: 1,116,280
RAC: 0
Message 2734 - Posted: 14 Mar 2014, 15:12:25 UTC - in response to Message 2730.  

Last modified: 14 Mar 2014, 15:36:49 UTC
The problem with HUGE users is that they will just go elsewhere with their resources if they can't get what they want here. There are ALSO plenty of projects that do NOT throttle the workunit load. I am guessing most projects would LOVE to have some super user come in with this guys resources and start crunching for them. This ONE guy is like adding 100 of us normal folks.


I think Gerard meant 5 cached tasks per core not 5 per core per day. BOINC server has a setting for it.

With a max of 5 in the cache at any given time any user with 24/7 connectivity would have all the tasks he can crunch. Only hosts that have limited connectivity (eg. traveling laptops) would be affected.

Another big benefit of restricting the number of tasks in the cache is that if the project server is offline for a few days and lots of hosts are completely dry and want tasks then fewer hosts will receive the "download errors" some hosts get when many thousands of hosts all want tasks at the same time. Each host gets a few at a time instead of 50 each which means other thirsty hosts can squeeze in and have a quick drink too.

BTW I think this new kid is more like 1,500 of us normal guys. 50 hosts with 32 cores each = ~1,600 cores. Those must be servers so they'll be busy with other work unlike dedicated crunchers so maybe more like 1,000 normal hosts.

P.S.

Correction: Here's his list of hosts. He has 132 hosts, not 50. Each host has 32 cores. You do the math, because I get dizzy when I try to imagine that many cores in one man's control.

@God

We're going to run out of asteroids to crunch soon. Make more, please, but don't put them so close to Earth this time. Close ones are scary.
BOINC FAQ Service
Official BOINC wiki
Installing BOINC on Linux
ID: 2734 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile tom richardson
Avatar

Send message
Joined: 8 Sep 13
Posts: 71
Credit: 15,807,240
RAC: 0
Message 2736 - Posted: 14 Mar 2014, 16:10:28 UTC
quick thought, given the amount of hosts and cores under one mans control, and given that the sheer amount of work units that have to be crunched by those hosts and cores,my concern is, that if this person is running several projects other than A&H, i suspect the chances are, that a lot of the A&H work units might well get timed out.

Result, those tasks will have to be sent out again adding to the validation pending times.

Am i wrong???

regards,

Tom
ID: 2736 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Bryan

Send message
Joined: 11 Dec 12
Posts: 7
Credit: 106,113,480
RAC: 0
Message 2737 - Posted: 14 Mar 2014, 16:43:21 UTC - in response to Message 2736.  

Last modified: 14 Mar 2014, 16:44:45 UTC
quick thought, given the amount of hosts and cores under one mans control, and given that the sheer amount of work units that have to be crunched by those hosts and cores,my concern is, that if this person is running several projects other than A&H, i suspect the chances are, that a lot of the A&H work units might well get timed out.

Result, those tasks will have to be sent out again adding to the validation pending times.

Am i wrong???

regards,

Tom


Look at "top participants" and look at his computers. The number 1 machine is showing this:

In progress (2687) · Validation pending (757) · Validation inconclusive (0) · Valid (667) · Invalid (183)

He is averaging over 9000s /WU w/ 32 cores so he can finish approx. 300 WU per day. That machine has 2687 WU in progress (9.2 day cache) and their due date is 3/19. I don't think he is going to get them finished in 5 days :( Secondly he is running a high rate of invalids on that machine.

The admins would help their "other" crunchers if they would do as suggested and limit the number of WU per core/thread to something reasonable.
ID: 2737 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile tom richardson
Avatar

Send message
Joined: 8 Sep 13
Posts: 71
Credit: 15,807,240
RAC: 0
Message 2738 - Posted: 14 Mar 2014, 17:16:17 UTC - in response to Message 2737.  
You know, i started this thread simply to try and understand why my validation pending tasks had increased to about double, while my RAC was falling, down now by about 3000+.

I`ve sat back and watched the responses, at first, i was not complaining,but the more i read the more concerned i have become, and have now started agreeing with those that want limits set on these large scale hosts.

Its been said above, that a lot of projects would love to have the resources of these (in my opinion) over large hosts. however "IF" these hosts end up "timing out" on a large capacity of W/U`s then where`s the gain to the project, it just means those work units will have to be sent out again and take longer to validate.

I guess time will tell, in the mean time my RAC is still falling, now about 100 W/U still awaiting validation,

regards,

Tom.
ID: 2738 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Davis

Send message
Joined: 8 Aug 13
Posts: 46
Credit: 1,503,000
RAC: 0
Message 2739 - Posted: 14 Mar 2014, 18:36:20 UTC - in response to Message 2724.  

Last modified: 14 Mar 2014, 18:55:58 UTC
Aww come on now. We don't hold it against you that you talk like a whore. We know the mind controlling boogeyman computer holding your credits ransom made you talk that way.

And they don't depend on luck here, Andy, they use skill and facts. You should try it sometime.


I fail to understand how the other users skill, facts and piracy, had anything to do with the problems now existing at A@H?
ID: 2739 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Gerard de Groot

Send message
Joined: 9 Mar 13
Posts: 2
Credit: 2,718,653
RAC: 0
Message 2740 - Posted: 14 Mar 2014, 18:44:18 UTC - in response to Message 2730.  

Last modified: 14 Mar 2014, 18:47:34 UTC
5 cached tasks per core is more then enough, it will also relieve the serverload, and when 1 or 2 tasks are ready and uploaded by boinc, then boinc request immediately 1 or 2 new tasks, so each core will always have enough tasks to crunch, in this case 5.
I do not see the need to have 50 or more tasks waiting to be crunched per core.
And 10.5 days time to complete them is much to long. 5 days is long enough.
With this settings (done by the administrators) the HUGE crunchers will in my opinion NOT leave this project. Why?
B.T.W, I also love HUGE crunchers, the more the better, and I do not care about credits at all.
ID: 2740 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dagorath

Send message
Joined: 16 Aug 12
Posts: 293
Credit: 1,116,280
RAC: 0
Message 2747 - Posted: 15 Mar 2014, 1:16:11 UTC - in response to Message 2739.  
Aww come on now. We don't hold it against you that you talk like a whore. We know the mind controlling boogeyman computer holding your credits ransom made you talk that way.

And they don't depend on luck here, Andy, they use skill and facts. You should try it sometime.


I fail to understand how the other users skill, facts and piracy, had anything to do with the problems now existing at A@H?


I meant the project admins rely on skill and facts not that newbie running the monster machines that are holding all the tasks. And he's no pirate. He's a kind hearted, generous newbie who did a lot of work brining that many machines online to crunch here. Unfortunately he seems to have relied on the default cache settings or perhaps boosted them even higher than the defaults and now he's holding probably over 100,000 tasks in his ~130 caches. I don't see any reason to get pissed about it, it's hilarious, IMHO.

Tell you what Andy, the rest of us took a vote and we decided that if you ever come back then you are appointed to be the one to PM that noob and tell him to shape up. With that many hosts under his thumb he's probably a BOFH and knows some stuff but don't you back down. Rip him a new one. We'll be here listening. Do you know who to PM?

Just remember this... we will all get our credits eventually, we just need to be patient. So don't go telling him he's a pirate because then he'll know you're a noob too and he'll blow you out of the water, OK? Just stick to the facts which are: 1) his cache is too big, 2) our results aren't validating, 3) if he doesn't smarten up he's gonna have to deal with ol' Dag. Off you go now.
BOINC FAQ Service
Official BOINC wiki
Installing BOINC on Linux
ID: 2747 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Dagorath

Send message
Joined: 16 Aug 12
Posts: 293
Credit: 1,116,280
RAC: 0
Message 2748 - Posted: 15 Mar 2014, 1:43:05 UTC - in response to Message 2738.  
Its been said above, that a lot of projects would love to have the resources of these (in my opinion) over large hosts. however "IF" these hosts end up "timing out" on a large capacity of W/U`s then where`s the gain to the project, it just means those work units will have to be sent out again and take longer to validate.


You're right, Tom, but you're thinking short term. After the noob gets his caches tuned properly and stops returning bad results he'll more than make up for the mayhem he's caused in a few weeks if not sooner. With that many hosts under his control you can be sure he's no idiot. He'll get it sorted in short order. And our RACs will be back to normal in plenty of time.

BTW, another thought has crept into my mind. Some of the lads get up to all sorts of shenanigans when there is a challenge going on. If you look at the OS recorded for each of those hosts it says they're Linux virtual hosts. I am beginning to wonder if some joker didn't fake 32 cores on a 4 core virtual machine and then clone that original into 130 bogus hosts. It's an entirely doable scenario and I know because I've done similar myself. Now each of those clones has downloaded a few thousand tasks for a total of over 100,000 but they're all running on just a single 4 core i5 CPU. Hmmmmmm?

Why would anybody do that? Well, maybe they like seeing guys like Andy get all bent out of shape over nothing. Maybe they are on one of the teams in the challenge and they intend to win by hoarding all the tasks so that the other team can't have any, or something like that. Actually I doubt bthat could work but if the guy thinks it would work then he might try it. Another motive might be that he wanted Kyong to set a limit on cacheable tasks but Kyong refused or forgot or wouldn't listen so he's on a mission to teach Kyong a lesson.

Yes, the more I think about it the stronger the aroma of fish becomes. ROFLMAO!! Good one Jamie Kinney, whoever you are. As well, it's a nice change from cheating on credits.
BOINC FAQ Service
Official BOINC wiki
Installing BOINC on Linux
ID: 2748 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Davis

Send message
Joined: 8 Aug 13
Posts: 46
Credit: 1,503,000
RAC: 0
Message 2749 - Posted: 15 Mar 2014, 1:52:20 UTC - in response to Message 2747.  
I don't know who to P.M. Thanks for volunteering me though. LOL
ID: 2749 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Davis

Send message
Joined: 8 Aug 13
Posts: 46
Credit: 1,503,000
RAC: 0
Message 2750 - Posted: 15 Mar 2014, 1:54:42 UTC - in response to Message 2748.  
Possible it was deceit. But, everyone in that challenge has effectively been screwed because the server went down. I don't know all the details, but I would think that grabbing up and controlling all of the w.u.'s possibly helped shut everything down.....
ID: 2750 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Davis

Send message
Joined: 8 Aug 13
Posts: 46
Credit: 1,503,000
RAC: 0
Message 2751 - Posted: 15 Mar 2014, 2:06:24 UTC - in response to Message 2747.  
If it is a deceit, he's probably running under so many pseudonyms that we'll never be able to track down THE individual, or it could be a team as you've implied..... They would never publicly respond
ID: 2751 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Davis

Send message
Joined: 8 Aug 13
Posts: 46
Credit: 1,503,000
RAC: 0
Message 2752 - Posted: 15 Mar 2014, 2:27:01 UTC - in response to Message 2751.  
hmmmmmm
Message boards : Cafe : I feel good comment 1 (banished: ID 49533) 41 23 days ago
I did a search for Jamie Kinney on all of the forums, and the above also came up....
ID: 2752 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Davis

Send message
Joined: 8 Aug 13
Posts: 46
Credit: 1,503,000
RAC: 0
Message 2753 - Posted: 15 Mar 2014, 2:37:26 UTC
Jamie Kinney

CPU type GenuineIntel
Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz [Family 6 Model 62 Stepping 4]
Number of processors 32





Dave Broiles

CPU type GenuineIntel
Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz [Family 6 Model 62 Stepping 4]
Number of processors 32

seem to be confederates......
ID: 2753 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Bryan

Send message
Joined: 11 Dec 12
Posts: 7
Credit: 106,113,480
RAC: 0
Message 2754 - Posted: 15 Mar 2014, 2:48:42 UTC - in response to Message 2748.  


BTW, another thought has crept into my mind. Some of the lads get up to all sorts of shenanigans when there is a challenge going on. If you look at the OS recorded for each of those hosts it says they're Linux virtual hosts. I am beginning to wonder if some joker didn't fake 32 cores on a 4 core virtual machine and then clone that original into 130 bogus hosts. It's an entirely doable scenario and I know because I've done similar myself. Now each of those clones has downloaded a few thousand tasks for a total of over 100,000 but they're all running on just a single 4 core i5 CPU. Hmmmmmm?


Brilliant deduction! First of all that team wasn't in the challenge. Secondly I'm impressed that "fake" machines could throw up 61M credits in a week. I need to check into how that's done.


http://stats.free-dc.org/stats.php?page=userbycpid&cpid=68583442208b71aea7c8eed7bc0f4784
ID: 2754 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Davis

Send message
Joined: 8 Aug 13
Posts: 46
Credit: 1,503,000
RAC: 0
Message 2755 - Posted: 15 Mar 2014, 4:19:54 UTC
Hey guys! Like all of you, I've "donated" my computer, time and resources to science. I am not a whore or a pirate. I'm not in this for any type of self-gratification. This is not the 17th century and I wouldn't have believed in witch-hunts then. I thought there was a problem and I am a newbie to boinc, I'll admit that. *SHAKES HANDS ALL AROUND*
ID: 2755 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Andrew Davis

Send message
Joined: 8 Aug 13
Posts: 46
Credit: 1,503,000
RAC: 0
Message 2756 - Posted: 15 Mar 2014, 4:25:42 UTC
If this was a witch-hunt, I'd probably point my finger at Dagorath. He has already expressed some admiration for the "scheme" which he believes is "doable" because he has done it before! LOL Let's all just get along....
ID: 2756 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Slower, validation pending??