503 Error and high server loads
-
-
- shahriyar Barani
- Newbie
- Posts: 19
- Thanks: 0
-
Hi,
Sometimes specially when we have high number of visitors, our server load will increase and 503 error is occured.
It needs to mention our server is very powerful compare to our visitors and mainly our server load is under 1-2.
We have two evidence that is there problem with our RocketTheme hydrogen based template and your framework:
1- If we make another template default, the problem will be solved immediately like Atomic,etc. But even If we make the original hydrogen template as default for website frontend, the problem will be existed. So It's completely related to our RocketTheme template or Gantry.
2- We observe that when we strace the processes we note that there sometimes over 200,000 lstat system calls. sometimes over 1.3 millions in just 3 minutes! in strace we can observe that most of these system calls are related to gantry/hydrogen folders, all of them almost is template related folders.
When this problem occures, We usually can access /administrator panel or our forums that is hosted on this server despite the server load without any problem but when we want to browse a page in frontend we get 503 error.
Thanks,
Shahryar
-
-
-
- Damir
- Preeminent Rocketeer
- Posts: 22450
- Thanks: 2679
- Web Developer
- Do you have the path of the file that is calling them?
- Available for CUSTOM WORK with Joomla, WordPress and Grav Templates and Gantry Framework!
-
-
-
- shahriyar Barani
- Newbie
- Posts: 19
- Thanks: 0
-
No, we use strace for see the files that is been lstat. Here is the latest one:
144636 lstat("/home/shahrsak/public_html",
144636 lstat("/home/shahrsak",
144540 lstat("/home/shahrsak/public_html/fa",
45627 lstat("/home/shahrsak/public_html/fa/media",
30987 lstat("/home/shahrsak/public_html/fa/templates",
30243 lstat("/home/shahrsak/public_html/fa/templates/g5_hydrogen",
27445 lstat("/home/shahrsak/public_html/fa/libraries",
21804 lstat("/home/shahrsak/public_html/fa/media/gantry5",
21081 lstat("/home/shahrsak/public_html/fa/media/gantry5/engines",
20997 lstat("/home/shahrsak/public_html/fa/media/gantry5/engines/nucleus",
19422 lstat("/home/shahrsak/public_html/fa/media/widgetkit",
18612 lstat("/home/shahrsak/public_html/fa/media/widgetkit/widgets",
14886 lstat("/home/shahrsak/public_html/fa/media/gantry5/engines/nucleus/scss",
10035 lstat("/home/shahrsak/public_html/fa/media/gantry5/engines/nucleus/scss/vendor",
9734 lstat("/home/shahrsak/public_html/fa/components",
9462 lstat("/home/shahrsak/public_html/fa/libraries/gantry5",
9189 lstat("/home/shahrsak/public_html/fa/media/gantry5/engines/nucleus/scss/vendor/bourbon",
7398 lstat("/home/shahrsak/public_html/fa/media/widgetkit/widgets/slideset",
7341 lstat("/home/shahrsak/public_html/fa/libraries/gantry5/vendor",
6795 lstat("/home/shahrsak/public_html/fa/templates/g5_hydrogen/scss",
6783 lstat("/home/shahrsak/public_html/fa/plugins",
6717 lstat("/home/shahrsak/public_html/fa/administrator",
6568 lstat("/home/shahrsak/public_html/fa/components/com_komento",
6532 lstat("/home/shahrsak/public_html/fa/libraries/joomla",
6216 lstat("/home/shahrsak/public_html/fa/modules",
6108 lstat("/home/shahrsak/public_html/fa/templates/g5_hydrogen/custom",
5997 lstat("/home/shahrsak/public_html/fa/cache",
5961 lstat("/home/shahrsak/public_html/fa/plugins/system",
5772 lstat("/home/shahrsak/public_html/fa/libraries/cms",
5013 lstat("/home/shahrsak/public_html/fa/media/widgetkit/widgets/slideset/styles",
4822 lstat("/home/shahrsak/public_html/fa/modules/mod_menu",
4728 lstat("/home/shahrsak/public_html/fa/administrator/components",
4698 lstat("/home/shahrsak/public_html/fa/media/widgetkit/widgets/slideshow",
4644 lstat("/home/shahrsak/public_html/fa/modules/mod_menu/tmpl",
4563 lstat("/home/shahrsak/public_html/fa/libraries/gantry5/vendor/composer",
4472 lstat("/home/shahrsak/public_html/fa/components/com_komento/themes",
4436 lstat("/home/shahrsak/public_html/fa/components/com_komento/themes/kuro",
The numbers you see is the number of times lstat is done for that directory. These system calls are just for some minutes. as you can observe most of them are related to template.
Thanks.
-
-
-
- shahriyar Barani
- Newbie
- Posts: 19
- Thanks: 0
-
It seems the template create processes for every visit that is not kill at the right time and continue even after 90 seconds!
We're looking forward for your help and support.
-
-
-
- shahriyar Barani
- Newbie
- Posts: 19
- Thanks: 0
-
-
-
- Damir
- Preeminent Rocketeer
- Posts: 22450
- Thanks: 2679
- Web Developer
- I notified thee dev about this. Will let you knwo what he say.
- Last Edit: 5 years 4 months ago by Damir.
-
The following users have thanked you: shahriyar Barani
- Available for CUSTOM WORK with Joomla, WordPress and Grav Templates and Gantry Framework!
-
-
-
- Matias Griese
- Sr. Rocketeer
- Posts: 249
- Thanks: 104
- Lead Developer
-
shahriyar Barani wrote:
14886 lstat("/home/shahrsak/public_html/fa/media/gantry5/engines/nucleus/scss",
10035 lstat("/home/shahrsak/public_html/fa/media/gantry5/engines/nucleus/scss/vendor",
9189 lstat("/home/shahrsak/public_html/fa/media/gantry5/engines/nucleus/scss/vendor/bourbon",
6795 lstat("/home/shahrsak/public_html/fa/templates/g5_hydrogen/scss",
The numbers you see is the number of times lstat is done for that directory. These system calls are just for some minutes. as you can observe most of them are related to template.
You are on development mode which adds a huge amount of extra file checks to make sure that every file change causes update on the site. If you turn on production mode, your site should speed up a lot as it does not do checks for non-existing files. - Gantry 5, enjoy!
-
-
-
- shahriyar Barani
- Newbie
- Posts: 19
- Thanks: 0
-
If you mean the option that is under extra menu that production has green icon and development has red icon, we tested both, nothing has been changed in term of lstats and also the server load and 503 error. Maybe It's a bug that don't turn of the development mode? The speed is better on production mode but the lstats and server load don't change.
Also, need to mention the only temporary solution we could reach was making website offline and make it online some minutes later. It seems the template create processes that doesn't end and continue to running for a very unsual long time. - Last Edit: 5 years 4 months ago by shahriyar Barani.
-
-
-
- Matias Griese
- Sr. Rocketeer
- Posts: 249
- Thanks: 104
- Lead Developer
-
Having a lot of file requests shouldn't really matter by itself as PHP is usually caching them and there's almost no performance penalty of accessing existing files. Non-existing files aren't cached, but production mode should fix that. Having production mode on should reduce lstat checks into at least half, though like I said, it should not be an issue as Gantry usually runs slightly faster than other Joomla templates. Typical page load times depend on the server (being usually in the range of 100ms-1000ms).
This said I have heard about this problem before: in some rare cases, Gantry performs really badly. What people have done in that case was to switch into another hosting provider which fixed the issue for everyone this far. Unfortunately, that has also prevented me from finding a reason why the performance isn't good in those rare cases. I also know that Windows servers are particularly bad due to architectural differences, especially when it comes to file access.
On the other hand, processes running forever may have two reasons: either server load is too high (oversaturated) or something is triggering Gantry to recompile CSS which takes a very long time.
I would love to look into this issue; I have also a debugging tool which allows me to locate if there's something slow running in your site. - Gantry 5, enjoy!
-
-
-
- shahriyar Barani
- Newbie
- Posts: 19
- Thanks: 0
-
We have changed three servers in the past months with putting more and more hardware at it to solve the problem. But It didn't solve the issue.
The problem cause is not server load, server load is effect of gantry/template related issue. Because when we change the template or make the frontend offline for some minutes we don't get 503 anymore, So It shows the frontend create those processes that does not end at the right time and make the server load.
We definitely need your help regarding this issue because we have a high visiting media with more than 40K daily sessions and these days this problem make a very image for us.
Please let me know when login data you need to send you in PM.
Thanks.
-
Time to create page: 0.060 seconds