Where do I put Whitelisted Method?

1

For some reason, I’m not finding documentation explaining where you can put custom Whitelisted Method Python code.

I wanted to put my code directly in hooks.py, but that doesn’t seem to work.
The examples say when I create my method in the right location, it will be available here:
/api/method/[path.to.method]

Unfortunately there’s no explanation of what “path.to.method” can actually be.
Does the method have to be put into a doctype python file? And if yes, how do I do that properly?

My method is meant for fetching Expense Claims based on a very particular query.

I’d really like to put my method somewhere more “high-level” in my app folder. I don’t want to bury it in a doctype.py file.

2

Great question!

The 'path.to.method' is the path to any Python function, relative to its App’s root module.

So, let’s say we’re using the ‘ERPNext’ App. And you want to call the function named 'get_uom_conv_factor()' found in the Item DocType here, at line number 1196.

To call that function directly with an API:

{{ base_url }}/api/method/erpnext.stock.doctype.item.item.get_uom_conv_factor

You can absolutely call any function, anywhere. Check out the function here named 'get_default_company()'. Now, I know it is not whitelisted. But pretending that it was:

{{ base_url }}/api/method/erpnext.get_default_company

So feel free to write your code wherever it makes the most sense. For custom Apps, I sometimes just create a single Python module file named 'api.py', and put all my custom API functions in there. Makes it easy to find them later.

1 Like
3

Thank you for the thorough explanation!!!

Apparently I was doing it right, though I felt like I was stumbling blindly. I’m getting this error:

AttributeError: module ‘my_app’ has no attribute ‘my_method’

The method is in my_app/hooks.py and its got the @frappe.whitelist() decorator before it.

Any idea on common mistake I could be making?

4

Should work fine. To test this, I just added a function to my custom App’s hooks.py

image
image759×436 35.3 KB

To call this API method, here’s the curl I used.

curl --request GET \
  --url http://127.0.0.1:8000/api/method/btu.hooks.foo_bar \
  --header 'Authorization: token abcdef:123456' \
  --header 'Content-Type: application/json' \
  --cookie 'sid=Guest; system_user=yes; full_name=Guest; user_id=Guest; user_image='

And I receive this response:
image

(Note, I’m using actually not using curl, I’m using Insomnia. But you can use curl from a shell terminal, or Postman, or whatever REST client you like.)

Digress

Now, I don’t want to confuse matters. However…since we’re already talking about ‘hooks.py’. Did you notice how I imported the frappe module in a really weird way? I wrote this:

import frappe as _frappe

instead of this:

import frappe

This has nothing to do with API calls. But it’s an important “pitfall” when working with hooks.py. Whenever doing imports into ‘hooks.py’ specifically? Alias them. The reasons “why” are complex. It has to do with how Frappe framework automatically executes ‘hooks.py’ code in the background, and Python pickling.

If you import in the usual way, with 'import frappe'. Then your code won’t run, and you’ll get an obscure message about pickling. Here’s the article where I learned about this little trick:

(for better or worse, I’ve had to interact with Python pickling a lot more, since then)

4 Likes
5

Thank you for the details on the import frappe gotcha!! Very useful information!

With your answers I was able to figure out my goofy mistake. I didn’t realize I had to specify the filename containing the method. Once I used ‘app_name.hooks.method’ instead of simply ‘app_name.method’, it worked!

1 Like
6

Actually… appname.hooks.method works on the command line with “bench --site erpnext execute”

But when I try to curl or visit web api url in browser, I’m getting Module ‘appname.hooks’ has no attribute ‘method’.

:frowning:

I ran “bench --site erpnext clear-cache”
which ran fine without errors, but it had no impact on the situation.

I also tried “bench --site erpnext migrate”
which had no impact.

Restarting the server, however, fixed the problem.
What is the correct bench command I should be running to get the whitelisted methods to become available without restarting services?

7

If you’re making hotfixes on a production server, you’ll probably need to restart gunicorn before new endpoints are recognized. You can do that directly via supervisorctl (supervisorctl restart PROCESS_NAME) or bench (bench restart).

8

^ Agree with what @peterg said. :+1:

ERPNext behaves a bit differently in development mode, versus production.

In development mode, you can make changes to the code, and see results (usually) immediately. The reasons “why” are a bit complex. For one, development mode doesn’t use gunicorn. Also, there’s a little “inotify” tool that watches the code files, and reacts when they change. You may still need to flush your cache, or even do a full page reload in your browser. But it’s pretty snappy.

In production mode, Peter is correct: usually you must restart gunicorn. If I’m deploying a ton of changes, I’ll delete all the bytecode before restarting too, just to play it safe.

9

Thank you for all of this information!

Some useful notes from my side:

  1. My site is in production mode.
  2. “bench restart” also does not fix the problem. This advice may be erroneous.
  3. Only restarting the entire service stack fixes it.

I didn’t even know about gunicorn. This is immensely useful and I’m reading about it presently to understand the architecture.

I’m somewhat dismayed by the complexity that the python world seems to have in the webspace due to dependence on gunicorn. In the case of frappe, it sounds like there are two independent service stacks (one with gunicorn and one without) depending on how you’ve configured frappe (or your site???).

This is immensely confusing. My understanding so far is that gunicorn is a service worker that sits between http server (apache for instance) and python. If this understanding is correct, then what’s happening in development mode when gunicorn is taken out of the equation?

Thank you in advance! Very very helpful technical information so far!!

10

Resetting gunicorn is sufficient on my system, but there are several other layers of caching that might need to be manually refreshed in your installation (some inside Frappe and some outside). Other culprits might be bytecode (mentioned by Brian), nginx, redis, and proxies. If resetting gunicorn isn’t enough, you’ll have to trouble shoot your system to identify which layer isn’t updating appropriately.

Frappe uses Gunicorn in production because it is highly performant. The trade off is that it assumes stable code. In development, frappe uses werkzeug, which services http requests similarly but watches for code changes proactively.

11

The documentation for frappe says development mode is set in the site config.

My understanding is that gunicorn sits behind apache and handles requests and passes them to python. Apparently I either misunderstand or something else is happening, because the “development” setting being per-site means that Frappe would need to be responsible for handling the initial request to determine which service worker engine to use (gunicorn/werkzeug). So I’m left to presume the stack looks like the following:

gunicorn or werkzeug (invoked by frappe)
frappe
python
<something??> another gunicorn instance here??
apache

Or perhaps there’s only one gunicorn instance and one werkzeug instance running at all times and the flow works like this:

request to apache > gunicorn > python > frappe > hand request back to gunicorn (Or to werkzeug)

Can someone please clarify?

12

Hi:

A little help here … i hope this be useful.

“development mode” (that you can activate on site_config.json or with bench set-config developer_mode 1) is a state of “framework” like “design mode”. Is not related to environment.

Production mode (that you set with bench setup production youruser) is related to environment: uses fail2ban, nginx, etc … . You can deactivate production mode with bench disable-production.

Your bench in “non-production mode” only works with “bench start” command, that runs werkzeug server, etc … By the way, you can’t “bench stop” except Ctrl+C on terminal or killing some procceses (there are a script somewhere that helps …)

Production mode and development mode are not antagonist terms … in fact, you can set development mode in both production and non-production mode.

I think this is the confussion here … another “semantic” matter.

Sorry if I missunderstanding something . And sorry, i am not an expert, just 1 year digging here, and my english is not really fluent neither.

1 Like
13

If you’re using Apache, you’ve got an unconventional setup. It is hard to guess how exactly everything works on your system.

Generally, a production setup would use nginx as a reverse proxy, directing external traffic to a gunicorn process controlled by supervisor. A development setup, in contrast, has a user controlled werkzeug process (initiated by bench start), listening for web traffic at an arbitrary port. All of this is bench-wide. It’s either gunicorn or werkzeug, never both (and only one of each) for a given bench.

This question of server configuration is distinct from the developer_mode setting activated via site_config.json. As avc said, developer_mode is purely application-level. It changes how site customizations are stored to disk, but it has no impact whatsoever on how your server works. They’re two separate parts of your installation.

1 Like
14

Well… that explains a lot! Thank you guys for clearing that up!!