sever out of disk space
Message boards :
Problems and bug reports :
sever out of disk space
Message board moderation
Previous · 1 · 2
Author | Message |
---|---|
Send message Joined: 17 Aug 14 Posts: 49 Credit: 5,225,280 RAC: 0 |
|
Send message Joined: 19 Jun 12 Posts: 1 Credit: 591,840 RAC: 0 |
It is happening again: 04.04.2019 16:21:40 | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space 04.04.2019 16:21:40 | Asteroids@home | Temporarily failed upload of ps_190401_input_47930_1_0_0: transient upload error 04.04.2019 16:21:40 | Asteroids@home | Backing off 00:15:49 on upload of ps_190401_input_47930_1_0_0 |
Send message Joined: 20 Aug 14 Posts: 20 Credit: 753,120 RAC: 0 |
Out of curiosity, once The results are returned and validated, what is done with any asteroid models/data obtained? Is there some third party that stores information about various asteroids that it is transferred to? If so, who hosts that? (Im assuming that this is what "DAMIT" is, but I am not clear on who actually hosts that. Is it this project as well, or someone else? Possibly NASA or ESA, etc?) I know nothing about running this kind of project, and as such I want to be clear that I'm not criticizing anyone about the servers running out of disk space, Im just curious as to how that actually happens. Like, it would make sense if the servers for this project are also being used for the long term storage of the results, but if that is handled by some other organization, wouldn't freeing up server space for the project be as simple as transferring data to whatever server is storing the results long term? Also, couldn't the total storage capacity of the project servers be increased simply by adding additional hard drives? I know really big, server grade ones are probably expensive, but cant you also use regular consumer grade ones as well? Sure its not ideal, since they were not really meant to handle the same load, and they have lower bandwidth, and if you wanted them cheap you would be getting smaller ones, but how expensive would it be to make an additional RAID array out of cheap, old desktop drives with a capacity of around 250 GB each? The server status page says that there are 2 arrays in use right now, both using RAID 5. One is made from 8 600 GB drives, the other from 3 600 GB drives. If I understand how RAID 5 works, that would give a total storage capacity of 4200 GB, right? Say you get 5 250 GB consumer level drives for cheap, and in RAID 5 that would give you an extra 1000 GB of space. And if you wanted to have greater redundancy in the event of a failure since they are all old, consumer grade drives, just tell it to use 2 drives instead of 1 for redundancy, at the cost of 250 GB of storage space. If any of that is even remotely viable, (and again, I am by no means an expert on this stuff) would it be better to 'donate' by mailing a couple old hard drives rather than a $20 donation potentially? Again, I don't want to tell anyone how to do their job, or imply that I could do it better, (because I really cant) Im just curious about why things are done the way they are done. |
Send message Joined: 11 Aug 12 Posts: 8 Credit: 54,899,677 RAC: 0 |
+1 |
Send message Joined: 20 Aug 14 Posts: 20 Credit: 753,120 RAC: 0 |
|
Send message Joined: 19 Sep 14 Posts: 13 Credit: 10,026,859 RAC: 0 |
|
Send message Joined: 11 Apr 18 Posts: 20 Credit: 11,869,433 RAC: 2,729 |
|
Send message Joined: 15 Jun 15 Posts: 16 Credit: 123,015,840 RAC: 0 |
Unfortunately, running a database isn't as simple as adding more drives for space. Imagine you have a 4GB mp4 file, and two 2GB thumb drives. Yes, you can split an mp4. Yes, you can split a database, but that doesn't necessarily make it wise to do so. Mixing drives is a terribly bad idea, though, especially used ones. One also has to consider that perhaps there are no more HDD slots on their server. Connecting them via USB just isn't an option. There's no way the disks could handle the I/O over that channel to a database facing the internet with potentially 250k+ computers attempting access. It's just one of those things where you don't want to use 128 shot glasses when you really need a gallon jug... Hope that helps. |
Send message Joined: 24 Jun 14 Posts: 23 Credit: 19,445,190 RAC: 3,166 |
"Unfortunately, running a database isn't as simple as adding more drives for space." Actually, it is You add the physical drives to the DB server, then you extend the size of the database The question is: Why is there no scheduled task to purge the database? |
Send message Joined: 17 Aug 14 Posts: 49 Credit: 5,225,280 RAC: 0 |
"Unfortunately, running a database isn't as simple as adding more drives for space." That might work if the DB server still has room for more physical drives. Does it? |
Send message Joined: 15 Jun 15 Posts: 16 Credit: 123,015,840 RAC: 0 |
"Unfortunately, running a database isn't as simple as adding more drives for space." Over simplification. There's a few things to consider: (1) Is there room on the server for another drive? (2) A new disk would need to be added to the RAID array (3) the current logical volume is SAS. While you technically could add SATA to the SAS array, I don't know a single person that would ever consider it, much less recommend it (4) you're going to take a performance hit if you're using a non-matching drive (10k v 7200--or Bob forbid 5400, SAS v SATA, 64MB v 128MB or 256MB cache, etc.). There are too many unknowns about the server being discussed to just say 'add a drive and extend the database', as per my actual statement, but thanks for cherry-picking the comment. |
Send message Joined: 17 Aug 14 Posts: 49 Credit: 5,225,280 RAC: 0 |
2/14/2020 10:50:46 PM | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space 2/14/2020 10:50:46 PM | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space 2/14/2020 10:50:47 PM | Asteroids@home | [error] Error reported by file upload server: Server is out of disk space |
Send message Joined: 18 Jan 20 Posts: 2 Credit: 6,759,160 RAC: 0 |
|
Send message Joined: 17 Aug 14 Posts: 49 Credit: 5,225,280 RAC: 0 |
I get the same message, and looking at the Server status page, you can see that the db_purge service is not running, maybe thats is the problem. How much information do you have on how many workunits have files that are ready to delete and therefore how much space restarting db_purge will save? |
Send message Joined: 17 Aug 14 Posts: 49 Credit: 5,225,280 RAC: 0 |
The project is currently sending out more tasks, even when it cannot accept uploads of results of completed tasks. Is this suitable for freeing up disk space on the server, or is it increasing the amount of disk space in use instead? Should the project stop sending out any more tasks if the amount of free disk space drops below a certain level? |
Send message Joined: 24 Feb 20 Posts: 3 Credit: 27,840 RAC: 0 |
|
Previous · 1 · 2
Message boards :
Problems and bug reports :
sever out of disk space