The going theory over in this thread: http://asteroidsathome.net/boinc/forum_thread.php?id=333 is that the server may have run out of disk space some time prior to 7:32 UTC on Aug 12th.

I'm posting here in case the Admin(s?) check or are alerted to activity in this category of forum message more often.

"Feeder" hasn't been running for a couple of days now. No new work. Also, db_purge" server not running. I have 4 units completed and another 20 in uploading status -- for 2 days. Frozen.

I restarted BOINC manager & see in the event log: "File upload server: Server is out of disk space".

No notice of planned maintenance -- we need the Admin to fix this.

"Feeder" hasn't been running for a couple of days now. No new work. Also, db_purge" server not running. I have 4 units completed and another 20 in uploading status -- for 2 days. Frozen.

I restarted BOINC manager & see in the event log: "File upload server: Server is out of disk space".

No notice of planned maintenance -- we need the Admin to fix this.

Oh, I remember now. Without the feeder, nobody gets new tasks. Without the purge, old tasks never get deleted. These 2 server functions are essentially the bottleneck, and now they're not working. :/

Feeder & db_purge servers are still down - this is day #3. Can't get new work and can't upload completed tasks.

We need the Admin to fix this.

Feeder & db_purge servers are still down - this is day #3. Can't get new work and can't upload completed tasks.

We need the Admin to fix this.

He is out of town until Sunday, hopefully 'fixing' his family matters.

19.8.2014 12:53 "Error reported by file upload server: Server is out of disk space" problem again.

19.8.2014 12:53 "Error reported by file upload server: Server is out of disk space" problem again.

19.8.2014 12:53 "Error reported by file upload server: Server is out of disk space" problem again.

A suggestion from the News Forum: some disk space could be freed up by deleting some the older task records. E.G., I have almost 5 months-worth, some 1600+ task records, most of which could be deleted. I'd be happy with just the past couple of weeks of task records.

People with millions of credits, do you have thousands of old task records?

People with millions of credits, do you have thousands of old task records?

My record says 3040 valids. Did not check all of them, as it gets slower and slower moving back in 'time'.
But for shure, I don't need them!

My own valid record shows as 24334. I agree, we don't probably don't need to know that far back.

The only tasks I really look at are the ones with errors to make sure I don't have a machine with a hardware/software issue.

 
The server have 7 x 600 GB IBM SAS 10000 rpm (RAID5)
http://asteroidsathome.net/boinc/server_status.php

7 x 600 GB = 4.2 TB but RAID5 uses some of it for parity information:
http://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5

According the table the usable (for data) capacity is 1 − 1/n = 1 − 1/7 = 6/7
4.2 TB * (6/7) = 3.6 TB
http://en.wikipedia.org/wiki/Standard_RAID_levels#Comparison


So the goal of this server upgrade (~year ago) was fast disk subsystem using expensive but 'small' disks
http://www.amazon.com/IBM-2-5-Inch-Internal-Drives-49Y2048/dp/B004K6QZLI

This link shows (today): List Price: $520.00 , Price: $325.00


I don't think the Download/Upload need fast disk subsystem (only database accesses need it)
A few consumer-grade disks can do for Download/Upload server

For the price of one 600 GB IBM SAS 10000 rpm you can get 3 x 3 TB consumer-grade disks
(they may not be hot-swappable and in case of failure may need server shutdown to be replaced)
http://www.amazon.com/SEAGATE-ST3000DM001-Barracuda-cache-internal/dp/B0067G0YR0


If those 3 x 3 TB disks are in RAID5 the (additional for the server) usable capacity will be 6 TB (for < $400)

Someone willing to make a donation ;)

 

---

- ALF - "Find out what you don't do well ..... then don't do it!" :)

 
The server have 7 x 600 GB IBM SAS 10000 rpm (RAID5)
http://asteroidsathome.net/boinc/server_status.php

7 x 600 GB = 4.2 TB but RAID5 uses some of it for parity information:
http://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5

According the table the usable (for data) capacity is 1 − 1/n = 1 − 1/7 = 6/7
4.2 TB * (6/7) = 3.6 TB
http://en.wikipedia.org/wiki/Standard_RAID_levels#Comparison


So the goal of this server upgrade (~year ago) was fast disk subsystem using expensive but 'small' disks
http://www.amazon.com/IBM-2-5-Inch-Internal-Drives-49Y2048/dp/B004K6QZLI

This link shows (today): List Price: $520.00 , Price: $325.00


I don't think the Download/Upload need fast disk subsystem (only database accesses need it)
A few consumer-grade disks can do for Download/Upload server

For the price of one 600 GB IBM SAS 10000 rpm you can get 3 x 3 TB consumer-grade disks
(they may not be hot-swappable and in case of failure may need server shutdown to be replaced)
http://www.amazon.com/SEAGATE-ST3000DM001-Barracuda-cache-internal/dp/B0067G0YR0


If those 3 x 3 TB disks are in RAID5 the (additional for the server) usable capacity will be 6 TB (for < $400)

Someone willing to make a donation ;)  

I just bought two 3tb drives for my own NAS, on sale they were about $250 US together. The problem with buying cheap on sale drives is that when put them in a small low airflow rack server or even a NAS box they can quickly overheat. The drives that are recommended to work in a NAS environment are designed to produce less heat and thus last longer. I am NOT saying your idea won't work, I am just saying the environment they will be in is key.

I have been already thinking about adding secondary server mounting folder for data via sshfs. db_purge is set to archive older records than 90 days. Problem is that there are too many results now.
Also prepared workunits and finished results (post-processed) have more than 1 TB. I am just deleting them and storing in my server home where I have also 5 TB RAID5. But extending capacity for it also cost something, I have discussed it but I don't know when the university buy new disks.

I have been already thinking about adding secondary server mounting folder for data via sshfs. db_purge is set to archive older records than 90 days. Problem is that there are too many results now.
Also prepared workunits and finished results (post-processed) have more than 1 TB. I am just deleting them and storing in my server home where I have also 5 TB RAID5. But extending capacity for it also cost something, I have discussed it but I don't know when the university buy new disks.

Would a Drobo be a better idea? It gives storage, handles the data redundancy and you can just attach to it like a network drive. You just put some drives in and connect it to power and a network port. It does the rest. Not too sure if it could handle the speed though.

---

BOINC blog

Feeder and db_purge are down again.

Kyong, could you please post here what new or additional hardware could help making this project stable with some headroom for increasing load?

BilBG started to post the word 'Donation', Mikey responded direct to that. So maybe the volunteers can help where bureaucracy fails to respond adequate.

Alexander

Hallo all,

The IBM System x3550 M4 has 8 disk slots (2.5") . The Server Status page says there are 7 disks in a RAID5 configuration, so I assume there is 1 boot disk. (which is already a bad idea: you should have the system disk in RAID1 (mirror) configuration). Currently there are 7x 600GB disks in RAID5. You can replace those with higher capacity disks, but (again I assume) in the current configuration that will be a major job. There is a solution though: zfs. Invented by Sun; originally a abbreviation for Zeta Byte File System.
It is supported in Linux. You should really read yourself into this! It is (simply said) a combination of a volume manager and (software) RAID (and a lot more).
You can (on the fly) replace your disks with higher capacity ones, and in the end have a higher capacity RAID5.
Of course the first time you implement this is also a major job.

For the 'bigger' disk, as the previous poster mentioned: Donation?

Regards,
Ruud

I think it's one raid5 volume with 7 disks.

There is only one volume with RAID5. The last new bay will be filled soon and we are thinking about new solution. When I am home I will check my backup and if there is one folder then I am going to free up almost 1 TB so the problem with running out of space should be solved in the evening. This should be enough before we extended capacity with some new storage server.