How to add non_profit app to docker setup

[nikkilocke]

I am running erpnext in docker, using a compose file based on pdw.yml.

I want to install non_profit app (which also involves installing payments app first).

How do I modify the docker file to do that automatically?

I have tried altering create-site thus:

  create-site:
    image: frappe/erpnext:v14.15.1
    deploy:
      restart_policy:
        condition: none
    volumes:
      - ./volumes/sites:/home/frappe/frappe-bench/sites
      - ./volumes/logs:/home/frappe/frappe-bench/logs
    entrypoint:
      - bash
      - -c
    command:
      - >
        wait-for-it -t 120 db:3306;
        wait-for-it -t 120 redis-cache:6379;
        wait-for-it -t 120 redis-queue:6379;
        wait-for-it -t 120 redis-socketio:6379;
        export start=`date +%s`;
        until [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".db_host // empty"` ]] && \
          [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_cache // empty"` ]] && \
          [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_queue // empty"` ]];
        do
          echo "Waiting for sites/common_site_config.json to be created";
          sleep 5;
          if (( `date +%s`-start > 120 )); then
            echo "could not find sites/common_site_config.json with required keys";
            exit 1
          fi
        done;
        echo "sites/common_site_config.json found";
        echo "Getting payments";
        bench get-app payments;
        echo "Getting non_profit";
        bench get-app non_profit;
        echo "Creating site $FRAPPE_SITE_NAME_HEADER and installing erpnext";
        bench new-site $FRAPPE_SITE_NAME_HEADER --no-mariadb-socket --admin-password=admin --db-root-password=admin --install-app erpnext --set-default;
        echo "Installing payments";
        bench --site $FRAPPE_SITE_NAME_HEADER install-app payments;
        echo "Installing non_profit";
        bench --site $FRAPPE_SITE_NAME_HEADER install-app non_profit;

However, downing and then upping the compose now results in Internal Server Error in the browser, and the following logs from backend:

[2023-02-14 13:50:27 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2023-02-14 13:50:27 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2023-02-14 13:50:27 +0000] [1] [INFO] Using worker: gthread
[2023-02-14 13:50:27 +0000] [7] [INFO] Booting worker with pid: 7
[2023-02-14 13:50:27 +0000] [8] [INFO] Booting worker with pid: 8
/usr/local/lib/python3.10/functools.py:981: UserWarning: The 'filters_config' hook used to ad
d custom operators is not yet implemented in frappe.db.query engine. Use db_query (frappe.get_list) instead.
  val = self.func(instance)
[2023-02-14 15:38:09 +0000] [7] [ERROR] Error handling request /
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/website/serve.py", line 17, in get_respo
nse
    endpoint, renderer_instance = path_resolver.resolve()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/website/path_resolver.py", line 58, in r
esolve
    renderer_instance = renderer(endpoint, 200)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/website/page_renderers/static_page.py",
line 19, in __init__
    self.set_file_path()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/website/page_renderers/static_page.py",
line 26, in set_file_path
    file_path = frappe.get_app_path(app, "www") + "/" + self.path
  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 1356, in get_app_path
    return get_pymodule_path(app_name, *joins)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 1373, in get_pymodule
_path
    return os.path.join(os.path.dirname(get_module(scrub(modulename)).__file__ or ""), *joins
)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 1327, in get_module
    return importlib.import_module(modulename)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'payments'

If I exec all those bench commands inside the backend container, and then restart it, it starts working.

Not sure if I have to do them again in all the other containers (scheduler, the queues, etc.)? Or how to make this work without manual intervention if/when I rebuild the containers.

[nikkilocke]

I suspect I have a fundamental misunderstanding about how all this works.

Presumably installing an “app” involves installing some extra code somewhere. I can’t see where, in the persistent volumes, any new code is stored. I can see in apps.txt and apps.json there is reference to the new apps. But the code doesn’t seem to be anywhere.

As this is running in 9 separate copies of the frappe/erpnext:v14.15.1 container, they will all their code stored in separate places. How can the build-site container install code in any of the other containers, unless the code is stored in volumes?

Or does build-site just modify the sites volume to tell the other containers (all of which are connected to the volume) to individually load the code? That seems unlikely, as the add and install processes say they are doing something. Unless, I suppose, if they are just doing something to the database (which is also on a shared volume)?

[Later]
It seems that bench get-app installs the app code into ~/frappe-bench/apps. So I guess that bit needs doing in every container (or, perhaps, that directory needs to be a volume mount).

I have been looking at images/production/Containerfile in github, and it explains a lot of the issues I have been having.

1. The Containerfile mounts 3 volumes (sites, assets and logs). These must be created as docker volumes. (I am not clear what happens when you pull the container - does it pull the volume contents as well, or does it just pull the Dockerfile, and then do everything it says?)
2. The Containerfile has commands to get the erpnext app, and then delete the git repository.

It seems as if my only option to have a reproducible install with all the apps I need is to make my own copy of the Containerfile, or to add commands to install the other apps to my docker compose file.

If I want to use bind mounts, I need to do some of the stuff that is in your Containerfile in the docker-compose.

Help and advice gratefully received.

[revant_one]

Frequently Asked Questions

Frequently Asked Questions

[abdussamadbello]

Thanks @revant_one , Please the frappe_docker/custom-apps is a bit confusing.

Can you please number the steps, or can some do more with explanations shots, since it’s a common request?

[revant_one]

if someone has successfully built images they can explain.

or you can send a PR.

i have 2 posts on this forum explaining basics

Container Basics

Why? People who don’t know containers land up successfully installing ERPNext with containers and get stuck later. Read for container basics. Prerequisites Unix/Linux basics, init system, processes, mount points, bash shell. podman or docker, podman-compose or docker compose is installed. Go for rootless podman if you can. Standard docker will work. Make sure you’re NOT using snap to install docker, it’ll not work as expected through snap. Install, docker docker compose Or alternatively …

Container Builds

Read Container Basics before you begin. Let’s build a simple frappe py server that returns 404 Not Found Create a Dockerfile in current working directory. FROM python:3-alpine # Install required os packages to install frappe RUN apk add -U git build-base linux-headers && \ # Create directory for apps and sites mkdir -p /opt/simple-server/apps /opt/simple-server/sites && \ # Create python env python3 -m venv /opt/simple-server/env && \ # install frappe version-14 /opt/simple-serve…

second post has more details about automating builds and ci examples

I’m out of ideas. I will anyway land up answering individual questions even after writing and explaining a lot.

[abdussamadbello]

Thanks, a lot, I have been able to build the image with the instructions, but I can’t find the new app (hrms) when I run this command, docker compose --project-name erpnext-one exec backend bench version or docker compose --project-name erpnext-one exec backend bench list-apps

When I run this command, I get this.

docker compose --project-name erpnext-one exec backend bench --site erpone.zulaiy.com install-app hrms

App erpnext already installed
An error occurred while installing hrms: App hrms not in apps.txt
Traceback (most recent call last):
File “apps/frappe/frappe/commands/site.py”, line 416, in install_app
_install_app(app, verbose=context.verbose, force=force)
File “apps/frappe/frappe/installer.py”, line 275, in install_app
raise Exception(f"App {name} not in apps.txt")
Exception: App hrms not in apps.txt

I am using the Single Server Example

[revant_one]

to update apps.txt execute from running container

ls -1 apps > sites/apps.txt

it will recreate apps.txt with all apps available

Edit: this is part of configurator service now. fix: create apps.txt during configuration by revant · Pull Request #1081 · frappe/frappe_docker · GitHub, if anyone adds or removes app from image, just re-run the configurator.

[abdussamadbello]

Thanks, this solved my problem!.

YOURE AWESOME!

Thanks for all you do?

[abdussamadbello]

@revant_one ow I’m getting a 404 on my assests.

[abdussamadbello]

Update it worked by restarting the docker containers!!.

[nikkilocke]

Is the custom container available on docker Hub? I can’t see one with that name.

If not, I assume I have to build it. If I have to do that, why do you pass the apps.json file in through a base 64 encoded environment variable, rather than just ADD it in as a plain file?

Just trying to understand the logic.

[revant_one]

• ADD / COPY needs the full context to be available. That means you need that repo locally and the file to be ADD/COPY need to be in the repo.
• If we add such file in repo then it will be static, If I need to add few apps to this file, I’ll need to fork the repo and maintain it.
• if you directly pass stringified json as env var there are high chances of it not getting converted back into json because of shell escape characters, base64 ensures the data remains shell env var friendly

[revant_one]

@nikkilocke

Is the custom container available on docker Hub? I can’t see one with that name.

This repository only publishes frappe/erpnext image with no additional app.

source: https://github.com/frappe/frappe_docker/wiki/Frequently-Asked-Questions#how-to-install-official-or-custom-apps

[nikkilocke]

Thanks again @revant_one for your usual rapid and helpful response.

But I don’t understand your answer. The apps.json file only needs to contain:

[
  {
    "url": "https://github.com/frappe/erpnext",
    "branch": "version-14"
  },
  {
    "url": "https://github.com/frappe/payments",
    "branch": "develop"
  },
  {
    "url": "https://github.com/frappe/non_profit.git",
    "branch": "develop"
  }
]

according to frappe_docker/custom-apps.md at main · frappe/frappe_docker · GitHub

Why would that need the repo locally?

Also, I wasn’t suggesting you add apps.json to the repo - like the .env file, I was thinking you could have an example one to copy from. It just seems easier for a new person wanting to fire up a container to copy an example file and edit it, than to create an obscure BASE65 string environment variable from a file.

[revant_one]

@nikkilocke

Why would that need the repo locally?

Not needed. You can skip it. In that case you need to keep track of apps.json somewhere else.

@nikkilocke

I was thinking you could have an example one to copy from.

Example repo described it in a post. custom_containers repo has ci directory with working apps.json and other variables. custom_containers/ci at main · castlecraft/custom_containers · GitHub

[nikkilocke]

I am suggesting, for clarity and ease of use, that in images/custom/Containerfile that

RUN if [ -n "${APPS_JSON_BASE64}" ]; then \
    mkdir /opt/frappe && echo "${APPS_JSON_BASE64}" | base64 -d > /opt/frappe/apps.json; \
  fi

is replaced by

ADD resources/apps.json /opt/frappe/

and

export APP_INSTALL_ARGS="" && \
  if [ -n "${APPS_JSON_BASE64}" ]; then \
    export APP_INSTALL_ARGS="--apps_path=/opt/frappe/apps.json"; \
  fi && \
  bench init ${APP_INSTALL_ARGS}\

is replaced by

bench init --apps_path=/opt/frappe/apps.json \

Don’t you think that is easier, and clearer?

[revant_one]

Fork the repo and do that!

ADD documentation Dockerfile reference | Docker Documentation

Also have a apps.json in your repo. It is just a dockerfile, if you know and use docker you can use it, modify it. In some cases I’m copying the cloned apps in the image and doing bench setup requirements you can do anything you wish.

If you don’t have a repo, have a local directory somewhere and make changes there.

[nikkilocke]

I’m not suggesting this for me (now I have a better idea how it works, I can do that for myself). I’m suggesting it to improve the code, and make it easier to understand and use for the next person.

[revant_one]

will it be as simple as making a restapi call and getting an image ready? I wish to use it part of ci only. I don’t do manual builds.

if you are willing, send a PR and after it’s merged answer the questions related to image building. At least for now there are many successful builds without the changes you prefer.

If I’m the only person who is going to answer docker related questions and maintain the repository then you’re stuck with this. I’m not going to change anything.

[nikkilocke]

What’s a restapi call, please? And what has it to do with loading a docker image?

My logic is:
The custom Container is not available on DockerHub (as far as I can see). The only way I know of (in my ignorance) to build it is to:

• Make a copy of it in a directory, renaming it Dockerfile
• Make a copy of the resources directory as a subdirectory
• Add build: <the directory name> to your docker-compose.yml file

If you do that, then why jump through the hoops of creating an apps.json, base64 encoding it, and adding the result to an environment variable. Why not just put the apps.json in the resources folder, and ADD or COPY it into the container?

However, I imagine there is some clever alternative way of building your images, which requires them to be in the directory structure in which you supply them. Don’t know what it is, or how it works, though