Is my Firewall blocking make_post_request in my production server?

Frappe Framework Customization
1

Hi,

I am trying to pull data via api access and I just tested every thing on my local server and it works perfectly fine but the moment I try to get the data from my live server I am getting a SSL validation error which I never got at my local domain, I am presuming that my Firewall at the production server is blocking some port of which I am unaware can any one help me out with this error:

Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/app.py", line 56, in application
    response = frappe.handler.handle()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/handler.py", line 21, in handle
    data = execute_cmd(cmd)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/handler.py", line 52, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 922, in call
    return fn(*args, **newargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/handler.py", line 80, in runserverobj
    frappe.desk.form.run_method.runserverobj(method, docs=docs, dt=dt, dn=dn, arg=arg, args=args)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/desk/form/run_method.py", line 35, in runserverobj
    r = doc.run_method(method)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 666, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 887, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 870, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 660, in <lambda>
    fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
  File "/home/frappe/frappe-bench/apps/rigpl_erpnext/rigpl_erpnext/rigpl_erpnext/doctype/shipway_settings/shipway_settings.py", line 45, in get_carriers
    carriers = make_post_request(url=url, auth=None, headers=None, data=None)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/integrations/utils.py", line 48, in make_post_request
    raise exc
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)

The code that I am using is at here:

The problem is that its working flawlessly in my local setup but not in production

2
  1. Are you using a trusted SSL certificate on your production server?
  2. If answer to the first question is yes, are you doing any SSL inspection in the firewall? If yes, is that a trusted cert?
3
  1. I am using letsencrypt on my production server which I believe is a trusted SSL certificate.
  2. I have no idea as what a SSL inspection is on a firewall.

The website I am trying to connect to has a COMODO SSL installed.

4

Lets Encrypt doesn’t issue Comodo SSL certificates so theres something else going on. What is the link to your install? If you don’t want to put the link in public, you can also DM it.

5

I think you got me wrong I am trying to get data from https://shipway.in/api/carriers this is the link which even on browser gives json output since no authentication is needed. This website is what has a COMODO SSL certificate (not letsencrypt)

I am trying to send the request from https://www.rigpl.com which is my production server and has letsencrypt certificate.

6

In case you are using Ubuntu Linux as your server, you can check iptables to see the rules configured

7

First I am using CentOS and I am unaware what am I a supposed to check in the iptables.

[frappe@rigpl frappe-bench]$ cat /etc/*release
CentOS Linux release 7.0.1406 (Core) 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CentOS Linux release 7.0.1406 (Core) 
CentOS Linux release 7.0.1406 (Core)
8

This seems to be a common problem because certain versions of various packages like requests and certifi sometimes dont work well together. In a nutshell, your server isn’t trusting the certificate shipway is providing. Why? …some people said directly add the shipway cert to your server, some said create an unverified SSL context. There are many possibilities…

In general, the following quick/bad hack should work

import requests
carriers = requests.get(url, verify = FALSE)
blah blah blah

9

So you are saying that I use the below code:

	def get_carriers(self):
		url = requests.get(get_shipway_url() + "carriers", verify=FALSE)
		carriers = make_post_request(url=url, auth=None, headers=None, data=None)
		text = "Courier Name\t\t\t\tCourier ID\n"
		courier_list = carriers.get("couriers")
		for entry in courier_list:
			courier_name = entry.get("courier_name")
			courier_id = entry.get("id")
			text += str(courier_name) + "\t\t\t\t" + str(courier_id) + "\n"
		self.carrier_list = text
		self.save()

But isn’t this is a less secure method to do altogether forfeit the verifying of SSL certificate.

Isn’t there a way I could make the CentOS verify COMODO SSL certificate since I tried

yum install ca-certificates

But the ca-certificates were already installed and in their newest form

10

Yes, it is absolutely less secure. But in this particular instance, you’re not dealing with sensitive data, so in my opinion, it shouldn’t be an issue.

As to a real fix, there were 8-10 possible solutions I found in various stack overflow threads. Unfortunately, you’d have to try them all to see which one is the cause in your particular scenario.

11

Sorry for bothering I am still unable to understand how to bypass the verify since I have very little knowledge about how it is going to work.

Would like to know what is wrong that I am doing here:

		url = requests.get(concat_url, verify=FALSE)
		carriers = make_post_request(url=url, auth=None, headers=None, data=None)

But when I run the code I am getting the below error:

Traceback (most recent call last):
  File "/home/aditya/frappe-bench/apps/frappe/frappe/app.py", line 56, in application
    response = frappe.handler.handle()
  File "/home/aditya/frappe-bench/apps/frappe/frappe/handler.py", line 21, in handle
    data = execute_cmd(cmd)
  File "/home/aditya/frappe-bench/apps/frappe/frappe/handler.py", line 52, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "/home/aditya/frappe-bench/apps/frappe/frappe/__init__.py", line 922, in call
    return fn(*args, **newargs)
  File "/home/aditya/frappe-bench/apps/frappe/frappe/handler.py", line 80, in runserverobj
    frappe.desk.form.run_method.runserverobj(method, docs=docs, dt=dt, dn=dn, arg=arg, args=args)
  File "/home/aditya/frappe-bench/apps/frappe/frappe/desk/form/run_method.py", line 35, in runserverobj
    r = doc.run_method(method)
  File "/home/aditya/frappe-bench/apps/frappe/frappe/model/document.py", line 666, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/aditya/frappe-bench/apps/frappe/frappe/model/document.py", line 887, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/aditya/frappe-bench/apps/frappe/frappe/model/document.py", line 870, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "/home/aditya/frappe-bench/apps/frappe/frappe/model/document.py", line 660, in <lambda>
    fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
  File "/home/aditya/frappe-bench/apps/rigpl_erpnext/rigpl_erpnext/rigpl_erpnext/doctype/shipway_settings/shipway_settings.py", line 46, in get_carriers
    url = requests.get(concat_url, verify=FALSE)
NameError: global name 'FALSE' is not defined
12

Code is off.

url = requests.get(get_shipway_url() + "carriers", verify=FALSE)
carriers = make_post_request(url=url, auth=None, headers=None, data=None)

You should no longer use the make_post_request function.

url = get_shipway_url() + "carriers"
carriers = requests.get(url, verify=FALSE)