Instance refuses IP address at intervals

ERPNext
1

Hello Team

I am experiencing issues with my instance hosted on AWS. The ERPNext sites at intervals goes unreachable and other sites opens conveniently at this period.

More like the instance restricts the IP address for few minutes and then releases it after a while…Any idea on what possible issues and resolution could be will be appreciated

regards

2

Its mostly because of fail2ban.

This might help you…

1 Like
3

I had faced the same issue. Solved by adding my IP address to fail2ban’s whitelist referring below article.

https://blog.rimuhosting.com/2017/02/24/whitelist-your-own-computer-in-fail2ban/

4 Likes
4

thank your @VamYip @Basawaraj_Savalagi

i have tried both solutions and would wait couple of days to confirm this solves my issues

Best regards

5

Hello all

Apart from white listing IP Address is there any other solution? This is because we are not using a dedicated public IP so the IP changes almost daily.

Any help will be appreciated because this is becoming frustrating.

Regards

6

It might be worth investing in a static IP if possible. If you feel adventurous and don’t mind doing some acrobatics, can try below solution. I didn’t get a chance to implement this service but suggesting if you are able to get someone configure this for you.

Option 1: Dynamic DNS

  1. Subscribe to a Dynamic Dns service (there are plenty of them). The service will mostly like have you install a client on the machine whose IP keeps changing.
  2. Setup a script on ERPNext server which regularly probes DDNS for new IP Address and updates fail2ban configuration accordingly

Option 2: VPN tunneling

  1. You’ll have to host a VPN Server or subscribe to a service.
  2. Connect your Client as well as ERPNext server to VPN (make sure that both get the same VPN IP address each time)
  3. Whitelist your client’s VPN IP in Fail2Ban
  4. Access your ERPNext site using Server’s VPN IP address
  5. If the server is hosting multiple domains, you’ll have to host a local DNS server (risky and overkill for the purpose) and map your ERPNext domains to the server’s VPN IP address.

Regards,
Vamyip

3 Likes
7

Adding to the solution above, you could also use some shell scripts to add/edit your /etc/hosts file - effectively a mimic of the dyndns client option, but without having to use actual DDNS

8

Hi all,

So I’ve found that the blocking is indeed from fail2ban and the particular rule causing this seems to be the following:

#Block IPs trying to use server as proxy.
[Definition]
failregex = <HOST>.*\" 400

This effectively blocks the ip address/network for 10minutes whenever there are up to 6 “http 400 bad request errors” within a space of 10minutes

From the Access logs, I find that requests such as the ones shown below are the cause of the issue:

41.xxx.xxx.x8 - - [24/Oct/2018:07:54:07 +0000] "POST / HTTP/1.1" 400 103 "http://erp.abc.com/desk" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36" "-"

41.xxx.xxx.x8 - - [24/Oct/2018:07:55:08 +0000] "POST /socket.io/?EIO=3&transport=polling&t=MQb4TN_&sid=FH3UE9PiDLiUauc6AAjM HTTP/1.1" 400 52 "http://erp.abc.com/desk" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36" "-"

  1. Is it normal to have up to 6 “400 bad request errors” within 10minutes? If so, what would be a more acceptable limit?

  2. If the scenario above is abnormal and indicates a real issue, are there any pointers as to what may be the cause and how we can easily identify the offending client (most likely a web browser as indicated in the logs)?

Hoping someone can help shed more light on this

Thanks

1 Like
9

I’m facing this same issue and yet I don’t seem to have fail2ban installed on my ERPNext server

10

Fail2ban is auto installed with easy script or Linux. Not sure which one. I have done many installation with easy script. Never installed fail2ban independently but it’s there in all my installation.

11

True. Had the same issue and battled it for weeks thinking my database was crashing or my server specs was too low. Tripled my server config, tweaked mysql settings, increased gunicorn workers, optimized nginx but continued to have the same issue until i stumbled on a post about fail2ban blacklisting IPs intermittently. I checked the jail but couldn’t find my IP so i initially dismissed it as the problem. After a few more days and near sleepless nights i thought to myself “what have i got to lose?”. Whitelisted my IPs and wowzer!!! the problem went away. I almost kicked myself for it.

1 Like