Server outage resolved


Message boards : News : Server outage resolved

Message board moderation

To post messages, you must log in.
1 · 2 · 3 · 4 . . . 5 · Next
AuthorMessage
Profile Kyong
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 9 Jun 12
Posts: 584
Credit: 52,667,664
RAC: 0
Message 6857 - Posted: 8 Jul 2020, 11:48:21 UTC

Last modified: 8 Jul 2020, 11:49:14 UTC
I am sorry for server outages which was happening recently. There were several factors which often caused lack of free space and some services crashed and I had to do an emergency shutdown on Saturday in order to fix everything so I couldn't inform you about it except for Twitter. There is still a hardware problem with one of servers which I have to fix but the server is not being used now so project should be stable now.

Radim Vančo (FoxKyong)
ID: 6857 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Nflight
Avatar

Send message
Joined: 10 Jul 12
Posts: 18
Credit: 6,214,080
RAC: 0
Message 6859 - Posted: 8 Jul 2020, 13:51:36 UTC - in response to Message 6857.  
I am still showing nearly >120 work units trying to upload ?
User ID #1865
and Computer ID #449480
ID: 6859 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Clint

Send message
Joined: 26 Sep 13
Posts: 5
Credit: 13,443,068
RAC: 871
Message 6860 - Posted: 8 Jul 2020, 14:11:43 UTC
I have over 20 completed work units that won't upload, "error reported by file upload server,: can't open log file /log_project1/file_upload_handler.log errno: 9
ID: 6860 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
afl

Send message
Joined: 17 Mar 20
Posts: 1
Credit: 115,509
RAC: 0
Message 6862 - Posted: 8 Jul 2020, 14:28:14 UTC

Last modified: 8 Jul 2020, 14:29:39 UTC
I also have 3 WUs to upload and 5 WUs to report but nothing works...
(this effect is since sunday (July 5.)
ID: 6862 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
sachse

Send message
Joined: 25 Jan 13
Posts: 1
Credit: 10,427,685
RAC: 0
Message 6863 - Posted: 8 Jul 2020, 15:08:02 UTC
I have 178 WUs that cannot uploaded because of "Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)". I'm not amused.
ID: 6863 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Saenger
Avatar

Send message
Joined: 18 Jun 12
Posts: 23
Credit: 6,190,123
RAC: 3,156
Message 6864 - Posted: 8 Jul 2020, 16:09:20 UTC
Same here, just on another scale (that's far less relevant with only 3 WUs on my main machine):

Mi 08 Jul 2020 18:04:12 CEST | Asteroids@home | [fxd] starting upload, upload_offset -1
Mi 08 Jul 2020 18:04:12 CEST | Asteroids@home | Started upload of ps_200630_input_116551_11_0_0
Mi 08 Jul 2020 18:04:12 CEST | Asteroids@home | [file_xfer] URL: http://asteroidsathome.net/boinc_cgi/file_upload_handler
Mi 08 Jul 2020 18:04:14 CEST | Asteroids@home | [file_xfer] http op done; retval 0 (Success)
Mi 08 Jul 2020 18:04:14 CEST | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
Mi 08 Jul 2020 18:04:14 CEST | Asteroids@home | [file_xfer] parsing upload response: <data_server_reply>    <status>1</status>    <message>can't open log file '../log_project1/file_upload_handler.log' (errno: 9)</message></data_server_reply>
Mi 08 Jul 2020 18:04:14 CEST | Asteroids@home | [file_xfer] parsing status: -127
Mi 08 Jul 2020 18:04:14 CEST | Asteroids@home | [file_xfer] file transfer status -127 (transient upload error)
Mi 08 Jul 2020 18:04:14 CEST | Asteroids@home | Temporarily failed upload of ps_200630_input_116551_11_0_0: transient upload error
Mi 08 Jul 2020 18:04:14 CEST | Asteroids@home | Backing off 04:24:19 on upload of ps_200630_input_116551_11_0_0

So I'm just a wee bit not amused, the reporting of my already uploaded WUs has happened quite fine, obviously.
Grüße vom Sänger
ID: 6864 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Steffen

Send message
Joined: 25 May 13
Posts: 1
Credit: 20,912,565
RAC: 2,581
Message 6865 - Posted: 8 Jul 2020, 16:16:37 UTC - in response to Message 6857.  
It's imposible for me to upload my completed Workunits.
ID: 6865 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
tvcunningham77

Send message
Joined: 9 May 20
Posts: 1
Credit: 6,518,822
RAC: 0
Message 6866 - Posted: 8 Jul 2020, 16:30:37 UTC
Still unable to upload my completed work units which are about to hit their deadline.
ID: 6866 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Alan Barnes

Send message
Joined: 11 Nov 13
Posts: 2
Credit: 2,071,827
RAC: 0
Message 6868 - Posted: 8 Jul 2020, 17:16:33 UTC - in response to Message 6857.  
Will work-units completed within the deadline but which failed to upload and which have now timed out due to the server outages eventually be uploaded and attract credit?

For example
Name ps_200624_input_82381_5_0
Workunit 130936318
Created 24 Jun 2020, 19:35:26 UTC
Sent 25 Jun 2020, 13:05:54 UTC

Alan Barnes
ID: 6868 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Lars Vindal

Send message
Joined: 7 Jun 16
Posts: 3
Credit: 2,336,716
RAC: 0
Message 6869 - Posted: 8 Jul 2020, 17:54:44 UTC
I no longer get the "out of disk space" messages, and most of my previos pending uploads have gone through.

BUT... Now I can't upload anything because the server can't read its log files:

08/07/2020 17:28:12 | Asteroids@home | Started upload of ps_200630_input_110498_1_1_0
08/07/2020 17:28:12 | Asteroids@home | Started upload of ps_200630_input_110498_3_1_0
08/07/2020 17:28:12 | Asteroids@home | Started upload of ps_200624_input_88153_14_0_0
08/07/2020 17:28:14 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
08/07/2020 17:28:14 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
08/07/2020 17:28:14 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
08/07/2020 17:28:14 | Asteroids@home | Temporarily failed upload of ps_200630_input_110498_1_1_0: transient upload error
08/07/2020 17:28:14 | Asteroids@home | Backing off 04:52:58 on upload of ps_200630_input_110498_1_1_0
08/07/2020 17:28:14 | Asteroids@home | Temporarily failed upload of ps_200630_input_110498_3_1_0: transient upload error
08/07/2020 17:28:14 | Asteroids@home | Backing off 05:58:53 on upload of ps_200630_input_110498_3_1_0
08/07/2020 17:28:14 | Asteroids@home | Temporarily failed upload of ps_200624_input_88153_14_0_0: transient upload error
08/07/2020 17:28:14 | Asteroids@home | Backing off 00:17:47 on upload of ps_200624_input_88153_14_0_0



Is this related to the low disk space error we had earlier, or is it a misconfiguration somewhere?
ID: 6869 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Lars Vindal

Send message
Joined: 7 Jun 16
Posts: 3
Credit: 2,336,716
RAC: 0
Message 6870 - Posted: 8 Jul 2020, 18:00:02 UTC
Prior to these messages, there also was a period with additional log read failures, but related to the scheduler instead.
ID: 6870 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Steve Dodd

Send message
Joined: 23 Aug 12
Posts: 8
Credit: 11,880,952
RAC: 8,841
Message 6871 - Posted: 8 Jul 2020, 18:05:56 UTC
Maybe that server that isn't working still happened to house the log files?
ID: 6871 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Manxscouse

Send message
Joined: 2 Feb 14
Posts: 11
Credit: 17,239,871
RAC: 0
Message 6874 - Posted: 8 Jul 2020, 21:21:52 UTC - in response to Message 6857.  
Malwarebytes has started saying that that asteroidsathome.net has a trojan. Maybe you can check this as well.
ID: 6874 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
FreeQuest

Send message
Joined: 31 Jul 18
Posts: 1
Credit: 732,538
RAC: 0
Message 6878 - Posted: 9 Jul 2020, 4:28:23 UTC
Malwarebytes classes your application as a trojan.

see for yourself. in order to post this I had to allow your website.

Malwarebytes
www.malwarebytes.com

-Log Details-
Protection Event Date: 7/9/20
Protection Event Time: 2:10 PM
Log File: 0fed5ace-c19a-11ea-89a6-88ae1d7c2914.json

-Software Information-
Version: 4.1.0.56
Components Version: 1.0.955
Update Package Version: 1.0.26585
License: Premium

-System Information-
OS: Windows 10 (Build 18362.900)
CPU: x64
File System: NTFS
User: System

-Blocked Website Details-
Malicious Website: 1
, C:\Program Files\BOINC\boinc.exe, Blocked, -1, -1, 0.0.0

-Website Data-
Category: Trojan
Domain: asteroidsathome.net
IP Address: 89.29.55.30
Port: 80
Type: Outbound
File: C:\Program Files\BOINC\boinc.exe



(end)
ID: 6878 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Thyme Lawn

Send message
Joined: 15 Oct 17
Posts: 9
Credit: 2,544,804
RAC: 0
Message 6879 - Posted: 9 Jul 2020, 6:17:36 UTC - in response to Message 6868.  
Will work-units completed within the deadline but which failed to upload and which have now timed out due to the server outages eventually be uploaded and attract credit?

For example
Name ps_200624_input_82381_5_0
Workunit 130936318
Created 24 Jun 2020, 19:35:26 UTC
Sent 25 Jun 2020, 13:05:54 UTC

Alan Barnes

Possibly. That WU has issued a new task to replace your timed out one. If that task is uploaded, reported and validated first the status for your one will change to "Too late" and you won't get any credit. If your task is validated first both will be credited if the replacement starts running before that computer makes its next scheduler request (if it hasn't the server will abort it).
"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer
ID: 6879 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Maike-Sophy

Send message
Joined: 13 Jan 17
Posts: 1
Credit: 2,620,278
RAC: 0
Message 6880 - Posted: 9 Jul 2020, 7:28:07 UTC
09.07.2020 06:35:52 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
09.07.2020 06:35:52 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
09.07.2020 06:35:52 | Asteroids@home | Backing off 00:29:00 on upload of ps_200630_input_113518_2_0_0
09.07.2020 06:35:52 | Asteroids@home | Backing off 00:31:05 on upload of ps_200630_input_113611_7_0_0
09.07.2020 08:25:15 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
09.07.2020 08:25:15 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
09.07.2020 08:25:15 | Asteroids@home | Backing off 00:54:55 on upload of ps_200630_input_113518_2_0_0
09.07.2020 08:25:15 | Asteroids@home | Backing off 00:43:51 on upload of ps_200630_input_113611_7_0_0

These are the log reports i get
ID: 6880 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Glenn Marsden

Send message
Joined: 15 Apr 16
Posts: 1
Credit: 3,664,320
RAC: 0
Message 6881 - Posted: 9 Jul 2020, 7:46:52 UTC
09/07/2020 08:32:00 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
09/07/2020 08:32:00 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
ID: 6881 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
fbac1492

Send message
Joined: 20 May 20
Posts: 1
Credit: 600,480
RAC: 0
Message 6882 - Posted: 9 Jul 2020, 7:58:48 UTC
i hope it help's !

09.07.2020 08:57:05 | Asteroids@home | Started upload of ps_200624_input_73675_24_2_0
09.07.2020 08:57:05 | Asteroids@home | Started upload of ps_200624_input_73649_6_2_0
09.07.2020 08:57:06 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
09.07.2020 08:57:06 | Asteroids@home | [error] Error reported by file upload server: can't open log file '../log_project1/file_upload_handler.log' (errno: 9)
09.07.2020 08:57:06 | Asteroids@home | Temporarily failed upload of ps_200624_input_73675_24_2_0: transient upload error
09.07.2020 08:57:06 | Asteroids@home | Backing off 00:38:05 on upload of ps_200624_input_73675_24_2_0
09.07.2020 08:57:06 | Asteroids@home | Temporarily failed upload of ps_200624_input_73649_6_2_0: transient upload error
09.07.2020 08:57:06 | Asteroids@home | Backing off 00:25:07 on upload of ps_200624_input_73649_6_2_0
ID: 6882 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Brummig
Avatar

Send message
Joined: 30 Jan 18
Posts: 10
Credit: 1,415,740
RAC: 129
Message 6884 - Posted: 9 Jul 2020, 8:30:51 UTC
The admin hasn't done any BOINC crunching since 2013 (http://asteroidsathome.net/boinc/show_user.php?userid=1), so he wont be experiencing any upload problems himself, and clearly he doesn't monitor this forum. I've sent him a PM in the hope of waking him up. Does anyone know how to get his attention using the unnamed Twitter account to which he referred?
ID: 6884 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Zarck

Send message
Joined: 21 Jun 12
Posts: 24
Credit: 20,009,642
RAC: 1,609
Message 6885 - Posted: 9 Jul 2020, 8:32:22 UTC - in response to Message 6884.  
Unable to return calculated units...
ID: 6885 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · 3 · 4 . . . 5 · Next

Message boards : News : Server outage resolved