It looks like the uploads problem is now PARTIALLY fixed, at the cost of a major slowdown in the human-readable part of the project. Most, but not all, of my uploads have finished, and I've downloaded a few new tasks.

It is happening again:

04.04.2019 16:21:40 | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space
04.04.2019 16:21:40 | Asteroids@home | Temporarily failed upload of ps_190401_input_47930_1_0_0: transient upload error
04.04.2019 16:21:40 | Asteroids@home | Backing off 00:15:49 on upload of ps_190401_input_47930_1_0_0

Out of curiosity, once The results are returned and validated, what is done with any asteroid models/data obtained? Is there some third party that stores information about various asteroids that it is transferred to? If so, who hosts that? (Im assuming that this is what "DAMIT" is, but I am not clear on who actually hosts that. Is it this project as well, or someone else? Possibly NASA or ESA, etc?)

I know nothing about running this kind of project, and as such I want to be clear that I'm not criticizing anyone about the servers running out of disk space, Im just curious as to how that actually happens. Like, it would make sense if the servers for this project are also being used for the long term storage of the results, but if that is handled by some other organization, wouldn't freeing up server space for the project be as simple as transferring data to whatever server is storing the results long term?

Also, couldn't the total storage capacity of the project servers be increased simply by adding additional hard drives? I know really big, server grade ones are probably expensive, but cant you also use regular consumer grade ones as well? Sure its not ideal, since they were not really meant to handle the same load, and they have lower bandwidth, and if you wanted them cheap you would be getting smaller ones, but how expensive would it be to make an additional RAID array out of cheap, old desktop drives with a capacity of around 250 GB each? The server status page says that there are 2 arrays in use right now, both using RAID 5. One is made from 8 600 GB drives, the other from 3 600 GB drives. If I understand how RAID 5 works, that would give a total storage capacity of 4200 GB, right? Say you get 5 250 GB consumer level drives for cheap, and in RAID 5 that would give you an extra 1000 GB of space. And if you wanted to have greater redundancy in the event of a failure since they are all old, consumer grade drives, just tell it to use 2 drives instead of 1 for redundancy, at the cost of 250 GB of storage space.

If any of that is even remotely viable, (and again, I am by no means an expert on this stuff) would it be better to 'donate' by mailing a couple old hard drives rather than a $20 donation potentially? Again, I don't want to tell anyone how to do their job, or imply that I could do it better, (because I really cant) Im just curious about why things are done the way they are done.

I know nothing about running this kind of project, and as such I want to be clear that I'm not criticizing anyone about the servers running out of disk space, Im just curious as to how that actually happens. Like, it would make sense if the servers for this project are also being used for the long term storage of the results, but if that is handled by some other organization, wouldn't freeing up server space for the project be as simple as transferring data to whatever server is storing the results long term?

Also, couldn't the total storage capacity of the project servers be increased simply by adding additional hard drives? I know really big, server grade ones are probably expensive, but cant you also use regular consumer grade ones as well? Sure its not ideal, since they were not really meant to handle the same load, and they have lower bandwidth, and if you wanted them cheap you would be getting smaller ones, but how expensive would it be to make an additional RAID array out of cheap, old desktop drives with a capacity of around 250 GB each? The server status page says that there are 2 arrays in use right now, both using RAID 5. One is made from 8 600 GB drives, the other from 3 600 GB drives. If I understand how RAID 5 works, that would give a total storage capacity of 4200 GB, right? Say you get 5 250 GB consumer level drives for cheap, and in RAID 5 that would give you an extra 1000 GB of space. And if you wanted to have greater redundancy in the event of a failure since they are all old, consumer grade drives, just tell it to use 2 drives instead of 1 for redundancy, at the cost of 250 GB of storage space.

If any of that is even remotely viable, (and again, I am by no means an expert on this stuff) would it be better to 'donate' by mailing a couple old hard drives rather than a $20 donation potentially? Again, I don't want to tell anyone how to do their job, or imply that I could do it better, (because I really cant) Im just curious about why things are done the way they are done.

+1

what part about that is the +1 in response to?

up
db_purge project1 Не работает (Not Running)

---

Wed 12 Jun 2019 07:37:35 AEST | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space

Isn't the new server supposed to be here by now?

I know nothing about running this kind of project, and as such I want to be clear that I'm not criticizing anyone about the servers running out of disk space, Im just curious as to how that actually happens. Like, it would make sense if the servers for this project are also being used for the long term storage of the results, but if that is handled by some other organization, wouldn't freeing up server space for the project be as simple as transferring data to whatever server is storing the results long term?

Also, couldn't the total storage capacity of the project servers be increased simply by adding additional hard drives? I know really big, server grade ones are probably expensive, but cant you also use regular consumer grade ones as well? Sure its not ideal, since they were not really meant to handle the same load, and they have lower bandwidth, and if you wanted them cheap you would be getting smaller ones, but how expensive would it be to make an additional RAID array out of cheap, old desktop drives with a capacity of around 250 GB each? The server status page says that there are 2 arrays in use right now, both using RAID 5. One is made from 8 600 GB drives, the other from 3 600 GB drives. If I understand how RAID 5 works, that would give a total storage capacity of 4200 GB, right? Say you get 5 250 GB consumer level drives for cheap, and in RAID 5 that would give you an extra 1000 GB of space. And if you wanted to have greater redundancy in the event of a failure since they are all old, consumer grade drives, just tell it to use 2 drives instead of 1 for redundancy, at the cost of 250 GB of storage space.

If any of that is even remotely viable, (and again, I am by no means an expert on this stuff) would it be better to 'donate' by mailing a couple old hard drives rather than a $20 donation potentially? Again, I don't want to tell anyone how to do their job, or imply that I could do it better, (because I really cant) Im just curious about why things are done the way they are done.

Unfortunately, running a database isn't as simple as adding more drives for space. Imagine you have a 4GB mp4 file, and two 2GB thumb drives. Yes, you can split an mp4. Yes, you can split a database, but that doesn't necessarily make it wise to do so. Mixing drives is a terribly bad idea, though, especially used ones. One also has to consider that perhaps there are no more HDD slots on their server. Connecting them via USB just isn't an option. There's no way the disks could handle the I/O over that channel to a database facing the internet with potentially 250k+ computers attempting access. It's just one of those things where you don't want to use 128 shot glasses when you really need a gallon jug... Hope that helps.

"Unfortunately, running a database isn't as simple as adding more drives for space."

Actually, it is

You add the physical drives to the DB server, then you extend the size of the database

The question is:

Why is there no scheduled task to purge the database?

"Unfortunately, running a database isn't as simple as adding more drives for space."

Actually, it is

You add the physical drives to the DB server, then you extend the size of the database

That might work if the DB server still has room for more physical drives. Does it?

"Unfortunately, running a database isn't as simple as adding more drives for space."

Actually, it is

You add the physical drives to the DB server, then you extend the size of the database

Over simplification. There's a few things to consider: (1) Is there room on the server for another drive? (2) A new disk would need to be added to the RAID array (3) the current logical volume is SAS. While you technically could add SATA to the SAS array, I don't know a single person that would ever consider it, much less recommend it (4) you're going to take a performance hit if you're using a non-matching drive (10k v 7200--or Bob forbid 5400, SAS v SATA, 64MB v 128MB or 256MB cache, etc.). There are too many unknowns about the server being discussed to just say 'add a drive and extend the database', as per my actual statement, but thanks for cherry-picking the comment.

2/14/2020 10:50:46 PM | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space
2/14/2020 10:50:46 PM | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space
2/14/2020 10:50:47 PM | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space

I get the same message, and looking at the Server status page, you can see that the db_purge service is not running, maybe thats is the problem.

I hope he is reading this, and will get it up and running again soon.

I get the same message, and looking at the Server status page, you can see that the db_purge service is not running, maybe thats is the problem.

I hope he is reading this, and will get it up and running again soon.

How much information do you have on how many workunits have files that are ready to delete and therefore how much space restarting db_purge will save?

The project is currently sending out more tasks, even when it cannot accept uploads of results of completed tasks. Is this suitable for freeing up disk space on the server, or is it increasing the amount of disk space in use instead?

Should the project stop sending out any more tasks if the amount of free disk space drops below a certain level?

"Composer" you said it all,I agree 100%,I'm on a fixed income,& would gladly give my 'Virus'-ridden old computers & harddrives "Windows 98" "95" "Vista" & "Windows 7" in place of cash to help,but all 3 computers are gone now."HP" & a 14-16 inch Monitor.

---