Can you please set smaller deadlines?


Message boards : Number crunching : Can you please set smaller deadlines?

Message board moderation

To post messages, you must log in.
1 · 2 · Next
AuthorMessage
Profile rilian
Avatar

Send message
Joined: 18 Jun 12
Posts: 8
Credit: 83,844
RAC: 0
Message 220 - Posted: 10 Sep 2012, 9:53:01 UTC
3 weeks in much more than really needed

i'd say 1 week is enough.

http://asteroidsathome.net/boinc/workunit.php?wuid=8320
ID: 220 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kyong
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 9 Jun 12
Posts: 584
Credit: 52,667,664
RAC: 0
Message 221 - Posted: 10 Sep 2012, 10:19:11 UTC
The next batch of WUs will have lower deadline.
ID: 221 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile rilian
Avatar

Send message
Joined: 18 Jun 12
Posts: 8
Credit: 83,844
RAC: 0
Message 222 - Posted: 10 Sep 2012, 10:20:48 UTC - in response to Message 221.  
thanks a LOT !!!!!!!!!
ID: 222 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile rilian
Avatar

Send message
Joined: 18 Jun 12
Posts: 8
Credit: 83,844
RAC: 0
Message 223 - Posted: 10 Sep 2012, 10:28:02 UTC - in response to Message 222.  
just curious, is it possible to update deadlines on already sent WUs ?
ID: 223 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kyong
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 9 Jun 12
Posts: 584
Credit: 52,667,664
RAC: 0
Message 224 - Posted: 10 Sep 2012, 15:22:44 UTC
It can be done via database but I don't know how about updating clients because they get the info when they download the WUs.
ID: 224 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Amauri

Send message
Joined: 21 Jun 12
Posts: 13
Credit: 1,440,022
RAC: 1,394
Message 225 - Posted: 11 Sep 2012, 7:36:33 UTC - in response to Message 224.  
AFAIK, you can't change deadlines of already downloaded tasks, you can only abort them, and this you obviously don't want to do...
ID: 225 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Chris Granger

Send message
Joined: 19 Jun 12
Posts: 5
Credit: 310,260
RAC: 0
Message 236 - Posted: 13 Sep 2012, 6:24:17 UTC
It's probably best to just let the WUs with long deadlines finish up as usual, and shorten the deadlines for the next batch of WUs.

Having to wait a couple of weeks for credit to come to you isn't the end of the world. ;)
ID: 236 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile nenym

Send message
Joined: 15 Jun 12
Posts: 6
Credit: 60,404,031
RAC: 10,385
Message 237 - Posted: 13 Sep 2012, 10:39:26 UTC
By my point of view the result of very short DL would be more tasks erased after DL and resent to others; that means longer waiting for granted credit. DL 21 days for 15000s WU is needlessly short, adequate DL is about 14 days. DL 21 days was reasonable to long tasks from the previous batch (credited by CreditRandom, officially called as CreditNew).
ID: 237 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile rilian
Avatar

Send message
Joined: 18 Jun 12
Posts: 8
Credit: 83,844
RAC: 0
Message 238 - Posted: 13 Sep 2012, 10:41:47 UTC - in response to Message 237.  
even WCG that support dinosaur machines have max of 10 days deadline

how do you estimate adequatness of 14 days? :)
ID: 238 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile nenym

Send message
Joined: 15 Jun 12
Posts: 6
Credit: 60,404,031
RAC: 10,385
Message 239 - Posted: 13 Sep 2012, 10:51:41 UTC - in response to Message 238.  
To have no DL problem during PG challenge :).
ID: 239 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile rilian
Avatar

Send message
Joined: 18 Jun 12
Posts: 8
Credit: 83,844
RAC: 0
Message 246 - Posted: 16 Sep 2012, 11:43:29 UTC - in response to Message 239.  

Last modified: 16 Sep 2012, 11:43:56 UTC
Kyong, please abort these 1400 tasks if you do not crunch them

http://asteroidsathome.net/boinc/results.php?hostid=2239
ID: 246 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Kyong
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 9 Jun 12
Posts: 584
Credit: 52,667,664
RAC: 0
Message 247 - Posted: 16 Sep 2012, 12:45:40 UTC
Don't worry, they are crunching. The computer is running 24/7 so it can make it in time.
ID: 247 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Amauri

Send message
Joined: 21 Jun 12
Posts: 13
Credit: 1,440,022
RAC: 1,394
Message 273 - Posted: 20 Sep 2012, 5:37:01 UTC - in response to Message 247.  
Don't worry, they are crunching. The computer is running 24/7 so it can make it in time.

Yes, they are crunching, but that computer takes 10-12 days to run a WU (between download and report), and we must wait them to validate our own WU's where you are the wingman.

With a smaller deadline, the number of tasks anyone could download would be reduced, thus reducing the time to report them (less tasks in cache = less time to wait for them to be scheduled to run). Limiting the number of downloaded tasks would have the same effect.

Maintaining big deadlines, the side effect is an increasing number of pending tasks waiting for validation - like Seti@Home...
ID: 273 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Conan
Avatar

Send message
Joined: 19 Jun 12
Posts: 32
Credit: 5,109,956
RAC: 1,758
Message 274 - Posted: 20 Sep 2012, 7:47:30 UTC
Well Kyong will have a good chance on processing most of those work units.
His computer has 12 cores.
Each WU takes around 18,000 seconds or 5 hours to run.
He does 4.8 WU per day per core = 56.7 WUs per day
He has 1,348 yet to do.
So that will take him 23.7 or 24 days to complete them all.

So he might have to abort a few hundred maybe but not the lot, he should get to all the others.
In 20 days he will process 1,134 work units.

Waiting can be a problem for some but as quite a number of projects have large wait times you learn to be a bit more patient. Rilian knows about OProject and how long we wait over there.

So keep crunching Kyong you're getting through them.

Conan
ID: 274 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Amauri

Send message
Joined: 21 Jun 12
Posts: 13
Credit: 1,440,022
RAC: 1,394
Message 276 - Posted: 20 Sep 2012, 19:54:20 UTC - in response to Message 274.  
I agree he doesn't need to abort the tasks, but in next batch he could set a deadline of 14 days or even less. His machine would crunch the same 1134 WU's in 20 days, as this depends only on the computer capabilities (and server cache / network availability).

With a deadline of 14 days, his computer's cache would have 14 x 56.7 = 794 tasks, instead of 21 x 56.7 = 1191. And the results would return faster, reducing the pending tasks.

His computer is just one example, this applies to all computers on this project. And a smaller deadline would benefit us all, IMHO.
ID: 276 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jujube

Send message
Joined: 5 Sep 12
Posts: 30
Credit: 24,320
RAC: 0
Message 277 - Posted: 20 Sep 2012, 21:36:38 UTC - in response to Message 276.  
And the results would return faster, reducing the pending tasks.


I think that's a misconception. The rate at which any given host crunches a given project's tasks is determined, in the long term, by the resource share allocated to the project, the host's hardware capabilities and host on time. In other words if I assign 20% resource share to this project and my host produces 100 billion (just for example) CPU cycles in a day then over the long term and on average this project will receive 20 billion cycles per day from my host (minus some cycles for my own computing needs and other factors but let's ignore those to keep it simple). BOINC might and often does give more than the allocated resource share to a project that needs extra CPU cycles to complete a task that is in danger of missing deadline (and perhaps in a few other circumstances too) but that's a short term measure. There are certain other "aberrations" that can and do happen but after "the flow" becomes established (in other words after BOINC learns what to expect from your projects regarding deadlines, task durations, etc.) any given host produces only so much work for a project on any given day and the length of the task deadlines has zero effect upon that over the long term, always over the long term. So what we notice wrt pending tasks, over the long term, is that they build to a certain point and then stay at about that level (don't bother quibbling over what about means cuz you'll get nowhere) regardless of the length of the deadlines. I will admit to the fact that anyone can and most likely will (to prove that the system will seem to fail in exactly the way I have said it will seem to fail, thinking they've proven me wrong) micro-manage their cache and other prefs until they have 6 trillion pendings but to them I say in advance RTFM and RMFP.

And a smaller deadline would benefit us all, IMHO.


How does it benefit anyone? Is it because we get credits sooner? Well, that doesn't happen for reasons I explained above. Besides, even if we did get credits sooner the credits are worthless. How does it benefit me to receive something worthless sooner? I mean suppose you want to send me a condom with a hole in it. The condom is worthless. How would I benefit from receiving the worthless condom in 2 days as opposed to 10 days?
ID: 277 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Amauri

Send message
Joined: 21 Jun 12
Posts: 13
Credit: 1,440,022
RAC: 1,394
Message 278 - Posted: 21 Sep 2012, 4:07:08 UTC - in response to Message 277.  
Hi, jujube,
I will not receive a toaster, or "a condom with a hole", but the credits are a representation of my contribution to the project's science. If I donate cycles of my CPU, electricity and my time to a project, I want this work being used in real science, not waiting eons to be used. And a smaller deadline will certainly reduce this gap.
ID: 278 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
jujube

Send message
Joined: 5 Sep 12
Posts: 30
Credit: 24,320
RAC: 0
Message 279 - Posted: 21 Sep 2012, 11:42:40 UTC - in response to Message 278.  
the credits are a representation of my contribution to the project's science


Credits are numbers that mean nothing and represent nothing. How can that possibly be? It's because you, me or anybody can steal as many credits as we want and some projects give out huge amounts of credits for minimal work. If I have 100 million credits you can never know whether I actually earned them or whether I stole them or whether I received them for a mere 10 hours CPU time from some renegade project that uses ridiculously high credit payouts to steal crunchers from other project.

If I donate cycles of my CPU, electricity and my time to a project, I want this work being used in real science, not waiting eons to be used.


Eons is a ridiculous claim. I doubt you'll convince anybody to change anything if you use ridiculous claims and exaggerations. Try sticking with facts instead. The fact is that it takes years for projects to produce any useful conclusions. It doesn't happen overnight, or in 2 weeks or even 2 months. The fact is a certain amount of data needs to be crunched and shortening deadlines doesn't speed up that process. It speeds up with more hosts attaching to the project, faster hardware, science app optimization, greater resource shares assigned to the project by volunteers and probably other factors I've not thought of.

And a smaller deadline will certainly reduce this gap.


Which gap are you referring to? The gap between the time you return your result and the time it gets verified and awarded credits? That has nothing to do with how much time the research takes. The fact is that after your result gets verified and awarded credit the result gets dumped into a database where it might sit for several weeks or months before it gets further analysis and it may be years before any of that analysis yields conclusions that are useful to anyone. It seems like you have little understanding of how real science is accomplished and how much time it requires. You seem to think that your result plus a matching result from your wingman is all the scientists need to call the newspaper and tell them they've discovered something new and fantastic. Nope! That ain't how it works. It takes hundreds of thousands of results plus many months of further analysis of those results and shortening deadlines doesn't speed that up by even 1 second.
ID: 279 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
fractal

Send message
Joined: 12 Jul 12
Posts: 5
Credit: 20,198,079
RAC: 0
Message 282 - Posted: 22 Sep 2012, 21:03:00 UTC
The benefit of a shorter deadline in my opinion has nothing to do with people who actually do the work. It is for those systems that take thousands of work units and never touch them. It helps the project and the wingmen to have those units expire sooner so they can be given to a machine that will process them. It also keeps the volunteers who want to get their credits sooner happier.

I think a 7 day deadline is reasonable for work that takes 3 hours to complete. Crunchers can easily handle a weekend of downtime on either end with no ill affect. It seems to me to be a win/win suggestion.
ID: 282 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Amauri

Send message
Joined: 21 Jun 12
Posts: 13
Credit: 1,440,022
RAC: 1,394
Message 283 - Posted: 24 Sep 2012, 5:47:48 UTC - in response to Message 279.  
Eons is a ridiculous claim. I doubt you'll convince anybody to change anything if you use ridiculous claims and exaggerations. Try sticking with facts instead. The fact is that it takes years for projects to produce any useful conclusions. It doesn't happen overnight, or in 2 weeks or even 2 months. The fact is a certain amount of data needs to be crunched and shortening deadlines doesn't speed up that process. It speeds up with more hosts attaching to the project, faster hardware, science app optimization, greater resource shares assigned to the project by volunteers and probably other factors I've not thought of.

With few hundred computers running this project, certainly the faster the results arrive, the better for scientists. At the beginning of a batch, a smaller deadline does not affect anything, but at the end of a batch is when you feel the effect of smaller deadline. Remember the last batch, when we had to wait over a month, waiting for the results of the few computers running dozens of tasks. With a smaller deadline this waiting time would be much smaller.

Which gap are you referring to? The gap between the time you return your result and the time it gets verified and awarded credits? That has nothing to do with how much time the research takes. The fact is that after your result gets verified and awarded credit the result gets dumped into a database where it might sit for several weeks or months before it gets further analysis and it may be years before any of that analysis yields conclusions that are useful to anyone. It seems like you have little understanding of how real science is accomplished and how much time it requires. You seem to think that your result plus a matching result from your wingman is all the scientists need to call the newspaper and tell them they've discovered something new and fantastic. Nope! That ain't how it works. It takes hundreds of thousands of results plus many months of further analysis of those results and shortening deadlines doesn't speed that up by even 1 second.

You really don't know me to think I'm so naive as to believe that each completed task generates a scientific result immediately. I just hope fill gaps in the mosaic as quickly as possible, after all this is the goal of all projects running on BOINC. I have no way of knowing if the scientists will use the results tomorrow, next week, or next year. The results of scientific research can appear only in a few years, but they certainly need the data quickly to develop research, otherwise they would not need a computational grid like BOINC.
ID: 283 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : Number crunching : Can you please set smaller deadlines?