[SOLVED] Login webadmin zentyal 7.x

[cingo_5]

Hi

I manage about 70 zentyal systems and from today, i can't login in every zentyal 7.x dashboard.
it return 504 Gateway Time-out from nginx

zentyal.webadmin-nginx and zentyal.webadmin-uwsgi are up and running

no problem with zentyal 8.x

the problem started for all of them this morning, and the only thing in common is version 7.x

I tried to investigate and I see that during the login phase, requests are made to a public IP 3.121.107.3 so I tried to disable any traffic other than the LAN from the IP of the Zentyal machine but I still received the nginx timeout.
I finally disconnected the network cable from the machine and the console now opens...

is it possible there is something related to the crowdsrike problem?

anyone encountering the same problem?

Thank you

[juliocp1976]

hello
I have the same problem.

I already had this error 1 month ago on my servers, and strangely after 3 days they returned to normal access without the time-out error, I also didn't find anything related, I'm also waiting for a solution or answer that can identify this problem to resolve the situation delicate, unfortunately, if necessary, try accessing via RSAT if you need something in Zentyal.

[jrtaylor]

I can confirm that this happening on zentyal 8 also.

[rubicon]

I have the same problem, it hangs after logging in to the web. In the logs *84 no live upstreams while connecting to upstream, client

[pcready.cl]

Same problem here, several servers v6, 7, 8.

[pcready.cl]

/var/log/zentyal/error.log

Code: [Select]

2024/07/19 13:16:05 [error] 2658#2658: *23 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.x0.x0.x, server: , request: "GET / HTTP/1.1", upstream: "uwsgi://unix:///run/zentyal-webadmin/webadmin.sock", host: "x.x0.x.x3:xxxxx", referrer: "https://x.x0.x.x3:xxxxx/Login/Index"
2024/07/19 13:23:29 [error] 2658#2658: *41 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.x0.x0.x, server: , request: "GET / HTTP/1.1", upstream: "uwsgi://unix:///run/zentyal-webadmin/webadmin.sock", host: "x.x0.x.x3:xxxxx", referrer: "https://x.x0.x.x3:xxxxx/Login/Index"
2024/07/19 13:32:36 [error] 2658#2658: *45 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.x0.x0.x, server: , request: "GET / HTTP/1.1", upstream: "uwsgi://unix:///run/zentyal-webadmin/webadmin.sock", host: "x.x0.x.x3:xxxxx", referrer: "https://x.x0.x.x3:xxxxx/Login/Index"


[pcready.cl]

v4.2.x servers still running in labs, 0 issues, looks like a core nginx issue.

In the lab I have ISOLATED Zentyal v6 and v7 from Internet servers to test, and these servers had the same error.... ¬¬

[pcready.cl]

Please zentyal support or moderators, any trackbug to confirm the fail? thanks

[juliocp1976]

I did some investigation about the information in this issue, and the assumption appears to be true, the inaccessibility of update.zentyal.org causes the webadmin hanging and getting error 504 from nginx.

If I comment out the new version checking in the sources, then the webadmin Dashboard going to work again, without problems.



CREDITS: https://github.com/zentyal/zentyal/issues/2112


The file, which contains this version check is in /usr/share/perl5/EBox/Dashboard/CGI/Index.pm
In this file (as of Zentyal 7.1.2), starting at line 267, there is a function sub _getNewVersionFromCloud, which downloads a file from update.zentyal.org with wget (system call). wget normally timeouts in 900 secs, this is far too more than nginx waits for uWSGI answer. Unfortunately, if I set a much sorter timeout for wget (-T 15), it does not help. But if I comment out the relevant lines completely (273-278 for Zentyal 7.1.2), then this check it will be breasted, and the webadmin start working.

Of course, after modifying the file, the webadmin uwsgi process need to be killed (it is likely in a stuck state already), then a zs webadmin restart needed, too.

I wrote this result in #2170, too, for reference.

[Gbn]

Thank you @juliocp1976, that fixed my issue.

I can confirm that the fix by commenting out these lines, the webinterface works again, and that www.zentyal.org and update.zentyal.org are not reachable from my 7.x version.

I have a 2nd Zentyal 8, check the file there, there is a timeout build in the 8 version

Zentyal 7 line:
system("wget --quiet -O /tmp/new-release - http://update.zentyal.org/update-from-'$version'.txt");

Zentyal 8 line:
 system("timeout 15 wget --timeout 15 --quiet -O /tmp/new-release - http://update.zentyal.org/update-from-'$version'.txt");

So in 8, there is a timeout and my 8 version shows the desktop.

Gerard

[Gbn]

Confirmed, after making the line in 7.1.2 identical to the line in 8.x, the Dashboard keeps working normal, even when update.zentyal.org is not reachable.

Gerard

(mumbles something about CrowdStrike Falcon update, since outage started on 19th and update.zentyal.org not reachable since that time)

[umeshhrr]

All,

We have few Zentyal 6.2 server's & Issue facing since 19.7.24. Please support..

Regard's
Umesh

[blorente]

Hi everyone, we are working on this right now.

I'll reply on this thread as soon as we have the issue solved.

Sorry for the inconvenience

[blorente]

Systems have recovered and are up and running.

The problem was caused by a system service that could not terminate correctly and consumed resources quickly, not giving the monitoring system time to solve the problem.

One more time, sorry for the inconvenience.

Have a nice day community!

[cingo_5]

I confirm that the problem is solved.
Thank you all