Tasks running at high priority


Message boards : Number crunching : Tasks running at high priority

Message board moderation

To post messages, you must log in.
AuthorMessage
Dagorath

Send message
Joined: 16 Aug 12
Posts: 293
Credit: 1,116,280
RAC: 0
Message 842 - Posted: 14 Feb 2013, 9:46:00 UTC

Last modified: 14 Feb 2013, 9:51:39 UTC
Now we begin to see the other side of Kyong's mistake. We have been crunching tasks with fpops estimates that were too low and our DCF (Duration Correction Factor) has adjusted upwards in response to that condition. On one of my hosts it is 21.5; usually it is about 1 for Asteroids because usually the fpops estimates are very accurate for Asteroids tasks.

Now we begin to receive tasks with the correct fpops estimates and due to our abnormally high DCF our hosts think those tasks will take longer than the initial estimate derived from the fpops estimate. In fact it appears to one of my hosts that the tasks will run into deadline trouble so it runs them at high priority and will not download tasks from other projects. (not a complaint just an observation)

This condition (abnormally high DCF) that leads to the tasks running at high priority will correct itself in time if you let the tasks run. Don't delete them, they are not really in deadline trouble. If you delete them then the DCF will not correct itself. Remember, DCF rises quickly and decreases slowly so it will be this way for a while. Or you can adjust the DCF manually they say. I've never done that but if I was going to do so I would wait until all Asteroids tasks are complete and reported. I would not recommend doing it while an Asteroids task is in progress.
ID: 842 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile BilBg
Avatar

Send message
Joined: 19 Jun 12
Posts: 221
Credit: 623,640
RAC: 0
Message 843 - Posted: 14 Feb 2013, 18:25:52 UTC - in response to Message 842.  

I have <dcf_debug>1</dcf_debug> in cc_config.xml so using:
find "[Asteroids@home] [dcf] DCF" stdoutdae.* >>Asteroids@home_DCF.txt

... I can see when and how this changed in time:

21-Jan-2013 12:38:00 [Asteroids@home] [dcf] DCF: 1.127323->1.117066, raw_ratio 1.024754, adj_ratio 0.909015
22-Jan-2013 00:32:45 [Asteroids@home] [dcf] DCF: 1.117066->1.115681, raw_ratio 1.103217, adj_ratio 0.987602
...
23-Jan-2013 10:43:18 [Asteroids@home] [dcf] DCF: 1.419931->1.394501, raw_ratio 1.165630, adj_ratio 0.820907
23-Jan-2013 14:32:35 [Asteroids@home] [dcf] DCF: 1.394501->17.759772, raw_ratio 17.759772, adj_ratio 12.735578
23-Jan-2013 14:45:36 [Asteroids@home] [dcf] DCF: 17.759772->17.594134, raw_ratio 1.195946, adj_ratio 0.067340
...
25-Jan-2013 13:25:34 [Asteroids@home] [dcf] DCF: 20.978300->24.228451, raw_ratio 24.228451, adj_ratio 1.154929
27-Jan-2013 05:18:37 [Asteroids@home] [dcf] DCF: 24.404917->24.092082, raw_ratio 21.276569, adj_ratio 0.871815
29-Jan-2013 02:54:24 [Asteroids@home] [dcf] DCF: 24.092082->23.860052, raw_ratio 0.889036, adj_ratio 0.036902
...
08-Feb-2013 16:29:37 [Asteroids@home] [dcf] DCF: 20.330613->19.258935, raw_ratio 9.613838, adj_ratio 0.472875
08-Feb-2013 22:23:08 [Asteroids@home] [dcf] DCF: 19.258935->17.963648, raw_ratio 6.306058, adj_ratio 0.327435
08-Feb-2013 22:48:32 [Asteroids@home] [dcf] DCF: 17.963648->16.854258, raw_ratio 6.869747, adj_ratio 0.382425
...
13-Feb-2013 20:31:57 [Asteroids@home] [dcf] DCF: 11.908548->11.792238, raw_ratio 0.277501, adj_ratio 0.023303
14-Feb-2013 05:36:37 [Asteroids@home] [dcf] DCF: 11.792238->10.879774, raw_ratio 2.667598, adj_ratio 0.226216
14-Feb-2013 05:50:34 [Asteroids@home] [dcf] DCF: 10.879774->10.072983, raw_ratio 2.811866, adj_ratio 0.258449
14-Feb-2013 06:20:56 [Asteroids@home] [dcf] DCF: 10.072983->9.975379, raw_ratio 0.312602, adj_ratio 0.031034
14-Feb-2013 10:29:24 [Asteroids@home] [dcf] DCF: 9.975379->9.878653, raw_ratio 0.302812, adj_ratio 0.030356


For experienced user it's easy to correct DCF manually but as is visible by the above log there is no need - as you said DCF will correct by itself over time as you complete the Asteroids@home tasks.





- ALF - "Find out what you don't do well ..... then don't do it!" :)
ID: 843 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Tasks running at high priority