Requesting Help (SOS)

Hello People,

We have the ERPNEXT deployed as under:

  • AWS Classic Load Balancer
  • eC2 t3.2xlarge
  • RDS t3 large

We migrated the backup from our staging to our production and launched it within the company. We have about 200 technicians creating 1000 invoices every single day! However, we are facing issues as under:

  • Suddenly the site gives a pop up Internal Server Error
  • Then, within a few seconds we get Request Timed out
  • Then, the site throws 504 Gateway-Timed Out
  • Then, the site throws 502 Bad Gateway

And then, in a few minutes, the site COMES ONLINE AGAIN and the above is repeated.

Hello,
Ideally, I would recommend you to move to FrappeCloud.

If that’s not feasible try to find out where the bottleneck is. Ask the DevOps team to identify where the stress is. The issue could be with RDS or EC2 or both. Looking at resource usage can give you some hint. Frappe logs can be helpful too, it may have clue on what failed.

Can Frappe Cloud take up to 700 concurrent users?

This question should be directed towards frappecloud support.

However, looking at the plans FrappeCloud provides it looks like they do support very large use-cases. You can try reaching out to support either from their portal or via this telegram group.

In your case, as you stated you have 200 technicians creating around 1000 invoices. It should only be a problem if a lot of these are created/generated concurrently. If it is spread across the day, it should not be an issue.

If they are concurrent and if your invoice includes many items or validations you could use queue in invoice when it is submitted via a hook. This could help you manage scale without having to vertically scale up your servers.

This could be coming from the actions of fail2ban

7 Likes

I think you have to increase the number of gunicorns workers

Please check this: