It's still looks like aborted WUs are not at the front of the queue. Really think you need to stop generating WUs for a while and get things cleared out.

We don't know whether he's generating WUs or not. There are some 19,000 in the queue at this time, they may have been generated days/weeks ago and the generator may not have run since then. It's difficult to tell from our perspective.

Also, the resends might not go to the very front. They might be inserted 1,000 tasks behind the front, maybe 2,000 behind. If we watch some numbers we *might* be able to learn what's going on or we might exist as mushrooms until we die.

From the server status page at this time:

Tasks ready to send 19,685
Tasks in progress 31,899

At this time server status says:

Tasks ready to send 17,753
Tasks in progress 32,228

That's roughly 1,900 tasks sent but the resend for the task I aborted remains unsent. If Kyong merely turned on the "priority resends" option then we can see it obviously does not position the resend at queue head. I'll go out on a limb and predict that resend won't be issued until after the existing queue of 17,753 is drained completely. At that time the resend will go to a host with a fast turn around time and high rate of task verification and it might have a shortened deadline as well.

I'll wager 2,000 credits that my prediction proves true. Anybody accept that wager? If you accept and lose then you will attach your host to my account using my weak account key and you will crunch for me until your host earns 2,000 credits for my account. If I lose then I will attach my host to your account using your weak account key and crunch for you until my host earns 2,000 credits for your account. I'll accept the first counter-wager and only 1 counter wager.

EDIT ADDED...

Actually we may not be able to know whether the host that receives the resend is a fast reliable host or just any old host so I'll eliminate that from the wager and wager the simple and easier to verify condition that the resend won't be issued until after the ~17,000 in the queue now are drained.

It's now down to:

Tasks ready to send 16,033
Tasks in progress 32,385

Have only been running here for a few days but noticed the large number of pendings and read through this thread. I already have about a hundred from a user who aborted a ton (all from steve) which normally isn't a problem. As this thread discusses, none of them have been resent. This seems like another issue of poor default configuration choices in BOINC. Resends should go to the front of the queue by default IMO and not require the project manager to jump through hoops to fix an illogical default in the server settings. Anyway hopefully the queue will get to the point where the configuration changes were made and start issuing resends before new WUs.

I aborted task 368022 from wu 166350 at 13/12, 02:55;33 UTC. The resend was created 4 seconds later. We shall see how long it takes to be sent.

I have about 100 that were all aborted on the 21st. So far all have resends waiting but none have actually been sent.

Edit: Another clue to the current status is that none of the latest WUs (a considerable number) I'm receiving are resends. They're all new.

Tasks ready to send 14,833
Tasks in progress 31,954

The task I aborted hasn't been resent yet.

This seems like another issue of poor default configuration choices in BOINC. Resends should go to the front of the queue by default IMO and not require the project manager to jump through hoops to fix an illogical default in the server settings.

You don't understand. There is no option the admin can select in the stock server code that will put resends at the head of the queue. The "priority resend" option I suspect Kyong turned on does NOT put resends at the head of the queue. All that option does is select a fast, reliable host for the resend to go to and decrease the deadline. That's all it does.

There are a number of different ways a project can structure its database and generate WUs so each and every project requires a slightly different solution for putting resends at the front of the queue. The BOINC devs cannot possibly provide a solution that will work for every project and that is why there is no option an admin can simply turn on that will put resends at the queue head. It would be nice if it could be that way but it isn't that way and it will never be that way.

The only solution is custom code for each project. Sad but true. The good news is I have heard there are code samples floating around that an admin can look at and perhaps adapt to his project's way of doing things.

Anyway hopefully the queue will get to the point where the configuration changes were made and start issuing resends before new WUs.

It will definitely get to that point unless the project folds or everybody quits crunching but if the changes that were made at that point were simply the turning on of the "priority resends" option then you will be disappointed when the new WUs kick in and start generating resends because those will go to the queue tail too same as the resends are doing now. I'll wager a further 2,000 credits that that is precisely what is going to happen. Don't be fooled by the word "priority" in that phrase. It doesn't mean everything you might suspect it does. They found that out the hard way at Sixtack and now they are looking at adding custom high and low water mark feeder code which is possibly the kind of solution this project will need to implement if they want the resends at the head of the queue. Don't expect that to happen overnight. I could be wrong and I hope I am wrong but 10 days just to implement the code and another 10 for it to kick in isn't an outlandish estimate.

Tasks ready to send 13,235
Tasks in progress 31,729

The task I aborted has not been resent.

Tasks ready to send 10,079
Tasks in progress 31,617

The resend from my aborted task remains unsent.

The ready to send number has decreased by 9,606 tasks in the ~18.5 hours that have elapsed since I first recorded the ready to send number. That works out to ~520 tasks per hour. If nothing changes we can expect the tasks currently queued to be finished in ~19.5 hours or ~22:00 UTC on December 24. The resends may be counted in the ready to send figure or they may not. If they are then we can expect a good portion of those to be sent by 22:00. If they are not counted in the ready to send figure then the server should start sending them at ~22:00.. I wager 1,000 credits the resend from my aborted task won't be sent until after 22:00, Dec. 24. Any takers? It's a hi-lo type of bet so you have about a 50-50 chance of winning. Hoo-hah! Imagine what you could do with an extra 1,000 credits!!!

Yipppeee ... good things are happening! My pending is dropping - down 40 WU to 525 to go!!!!

Thanks for the great Christmas present :)

Merry Christmas everyone!

The resend for the task I aborted was issued sooner than I predicted but not much sooner. Someone could have picked up an easy 1,000 credits had they accepted my wager.

The resend from my aborted task went to a host with a turn around time of 1.34 days. Glad to see Kyong did not set the required turn around time for resends to be less than 1 day as that would prevent a lot of hosts from receiving a resend and would possibly slowdown the processing of resends.

Another prediction and another wager...

I doubt Kyong setup a high-water low-water mark system on such notice. The man is fast but not that fast. I am confident that all he has done so far is setup the priority resends option. If that is true and if things go the way they went at Sixtrack then the resends from the next batch of tasks will be inserted at the end of the queue. In other words I am saying nothing has changed except the fact that resends will now go to fast reliable hosts rather than just any host. I'll wager 2,000 credits on that and if I lose I'll attach my i7 to the winner's account to pay the debt ASAP. 2,000 credits just waiting to be snatched up. I'll take the first counter-wager and only 1 counter-wager.

(I think I've finally found a use for credits.)

I see 788 remaining in the queue. Those should be gone in about 1.5 hours. At some point Kyong will dump more tasks into the hopper. If we see only a few thousand go in then I should think that would indicate he's implemented some sort of high-low watermark scheme in which case the conditions have changed and the bet is off.

Tasks ready to send 2
Tasks in progress 30,202

More of the same or something new?

Tasks ready to send 118,154
Tasks in progress 25,625

The hopper (the queue) is full again so I aborted task 392158 from this WU . Task 614093 was generated for the resend. We will see whether 614093 waits until the end of the queue to be resent or is resent sooner. I wager 1,000 credits it will wait until the end of the queue.

Tasks ready to send 238,108
Tasks in progress 28,361

The resend for the most recent task I aborted remains unsent after 10 hours. Unless Kyong takes measures it won't get sent until the first ~118,000 of the queue have been sent. I don't have enough data points to make an accurate prediction but I can predict with some confidence the resend won't be issued until approximately 10 days from now, no sooner than 8 days. I'll wager 2,000 credits on that.

8 days might actually be an improvement over what was before. He may have slipped a high/low watermark mechanism or similar into the works, time will tell.

BobCat13 wrote:

They will get sent, eventually. It seems on a lot of new projects resends are added to the end of the queue instead of being added to the beginning. There is a configuration option for the server about resends.

Here is the page about resends: Project Options
Looks like it is under the section Accelerating retries, option <reliable_priority_on_over>X</reliable_priority_on_over>

Did some more reading, and the feeder has some command-line options also:
Backend Programs

The --priority_order_create_time looks interesting. If I interpret that correctly, then it moves higher priority jobs to the front of the queue in the order that the resends were created, so older resends before newer resends. If no resends are available, then it should distribute work based on task creation order.

Oh well that resend is still there in the queue and has been 5-6 days. I wonder if I get the resend.

The --priority_order_create_time looks interesting. If I interpret that correctly, then it moves higher priority jobs to the front of the queue in the order that the resends were created, so older resends before newer resends. If no resends are available, then it should distribute work based on task creation order.

I think you interpret it correctly. The catch might be that it's a parameter for the stock feeder code and may not be applicable at projects that have tossed out the stock feeder and substituted their own. Not saying that is what this project has done. Hope it's what Kyong needs.

Thanks, I miss this option. There is default feeder so I tried to add the option to it. I can see that previous fix didn't work.