ERPNext installation issues [docker]

EI-IT · September 13, 2024, 9:12pm

Good afternoon Everyone.
I’d like to share my experience for erpnext on docker and hopes to get some help before I scrap it entirely.

ErpNext docker install works…once
When running the docker installer from the github page using the file pwd.yml works but if you install lets say raven like I did and it doesn’t load and you attempt to docker prune everything even deleting the folder and restart from scratch some reason your information is still in there. I have tried this on both windows and ubuntu. It is an absolute pain trying to get it to how it was when its initially installed. The main error i get is module not found when you go to uninstall it raven app not found when you try and download it from git timeout.

The main issue with other apps using the default pwd.yml is none of them work they install but don’t work. If I am trying to developed a proper erp to test with other apps they could at least load.

Trying to create a new site in docker doesn’t load I assume its because docker doesn’t have access to /etc/host file [ubuntu 22.04]

If i can get some direction on how to set this up to test other apps in I would greatly appreciate it .
Thanks

revant_one · September 14, 2024, 9:44pm

I’ve given up on telling every new comer that you need to build image with all your custom apps instead of starting the erpnext image and installing apps.

I had this pinned faq that mostly is missed.

I only use kubernetes, I’ve more than frappe sites hosted there. It only makes sense when I was already using containers in my infrastructure to use it for Frappe as well.

Edit: even this post will be missed and someone will ask, “I started docker compose and did bench get-app, now app is not working”

Peer · September 15, 2024, 7:18pm

What do you mean by “is missed”:

“is ignored”
“is not found when needed”
“disappeared from being pinned so people start to miss it” (long for it)?

Your help generally is extremely spot on, as it’s here, too, albeit with a strong melancholic subsound.

Could it be a rather shallow documentation bug?
Certain rather longish instructions along the lines of “Instructions how to install v15 on Ubuntu” seem to be well known and found. Which doesn’t solve everything, but they even evolve if new stuff happens to the software and thus the knowledge.

Maybe a chart like a flow diagram could help?
E.g. a common starting point for the docker / kubernetes / press newcomers, with some easy 1 … 2 … 3 steps to have the inexperienced “I need to know now, quick, and everything” understand where they stand, what won’t work, what they need to know to proceed, and which (the chart) is kept versioned and current and where people can be directed to?
It might even have a link on “discuss”, and also suggest FC if the goal is evaluation, because the OP doesn’t seem to be aware of how useful FC is for quickly installing some working apps with FC.

Other possibility: bench in docker could disallow installing apps and show an explanation where your missed but needed hints are to be found.

That’s what I meant by “shallow”, as in “just a missing link to direct people to the help or kb they need”.

EI-IT · September 15, 2024, 11:38pm

Thanks for the response, I forgot to mention that i did see this post and tried to build an image with raven, erpnext and frappe but it never completes the build process. It looks like it does install the apps but it errors at the end never completing the process. I’ll post the errors when i get back to the office.

revant_one · September 16, 2024, 3:21am

Generally erpnext and Frappe have version tags.
All other apps may or may not have version maintained.

Recently I was trying wiki on v15, I had to install master branch, the version released is old and causes error during installation.

revant_one · September 16, 2024, 6:21am

My machine:

uname -a
Linux revant-laptop 6.10.9-arch1-2 #1 SMP PREEMPT_DYNAMIC Tue, 10 Sep 2024 14:37:32 +0000 x86_64 GNU/Linux

I am NOT using Mac or Windows.

My ./apps.json:

[
    {
        "url": "https://github.com/frappe/erpnext",
        "branch": "v15.35.1"
    },
    {
        "url": "https://github.com/The-Commit-Company/Raven",
        "branch": "v1.7.0"
    }
]

Build commands:

git clone https://github.com/frappe/frappe_docker

cd frappe_docker

# Add apps to apps.json
nano apps.json

export APPS_JSON_BASE64=$(base64 -w 0 ./apps.json)

docker build \
  --build-arg=FRAPPE_PATH=https://github.com/frappe/frappe \
  --build-arg=FRAPPE_BRANCH=v15.40.6 \
  --build-arg=PYTHON_VERSION=3.11.9 \
  --build-arg=NODE_VERSION=20.17.0 \
  --build-arg=APPS_JSON_BASE64=$APPS_JSON_BASE64 \
  --tag=registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0 \
  --file=images/custom/Containerfile .

I pushed the image here: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0

I started containers with compose file, I was able to login to ERPNext as well as Raven.

docker compose -p erpnext-raven -f compose.yml up -d

The compose.yml I used to test if the above image is working. It was copied from pwd.yml.

services:
  backend:
    image: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  configurator:
    image: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0
    deploy:
      restart_policy:
        condition: none
    entrypoint:
      - bash
      - -c
    # add redis_socketio for backward compatibility
    command:
      - >
        ls -1 apps > sites/apps.txt;
        bench set-config -g db_host $$DB_HOST;
        bench set-config -gp db_port $$DB_PORT;
        bench set-config -g redis_cache "redis://$$REDIS_CACHE";
        bench set-config -g redis_queue "redis://$$REDIS_QUEUE";
        bench set-config -g redis_socketio "redis://$$REDIS_QUEUE";
        bench set-config -gp socketio_port $$SOCKETIO_PORT;
    environment:
      DB_HOST: db
      DB_PORT: "3306"
      REDIS_CACHE: redis-cache:6379
      REDIS_QUEUE: redis-queue:6379
      SOCKETIO_PORT: "9000"
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  create-site:
    image: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0
    deploy:
      restart_policy:
        condition: none
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs
    entrypoint:
      - bash
      - -c
    command:
      - >
        wait-for-it -t 120 db:3306;
        wait-for-it -t 120 redis-cache:6379;
        wait-for-it -t 120 redis-queue:6379;
        export start=`date +%s`;
        until [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".db_host // empty"` ]] && \
          [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_cache // empty"` ]] && \
          [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_queue // empty"` ]];
        do
          echo "Waiting for sites/common_site_config.json to be created";
          sleep 5;
          if (( `date +%s`-start > 120 )); then
            echo "could not find sites/common_site_config.json with required keys";
            exit 1
          fi
        done;
        echo "sites/common_site_config.json found";
        bench new-site --no-mariadb-socket --admin-password=admin --db-root-password=admin --install-app erpnext --install-app raven --set-default frontend;

  db:
    image: mariadb:10.6
    healthcheck:
      test: mysqladmin ping -h localhost --password=admin
      interval: 1s
      retries: 15
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - --character-set-server=utf8mb4
      - --collation-server=utf8mb4_unicode_ci
      - --skip-character-set-client-handshake
      - --skip-innodb-read-only-compressed # Temporary fix for MariaDB 10.6
    environment:
      MYSQL_ROOT_PASSWORD: admin
    volumes:
      - db-data:/var/lib/mysql

  frontend:
    image: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0
    depends_on:
      - websocket
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - nginx-entrypoint.sh
    environment:
      BACKEND: backend:8000
      FRAPPE_SITE_NAME_HEADER: frontend
      SOCKETIO: websocket:9000
      UPSTREAM_REAL_IP_ADDRESS: 127.0.0.1
      UPSTREAM_REAL_IP_HEADER: X-Forwarded-For
      UPSTREAM_REAL_IP_RECURSIVE: "off"
      PROXY_READ_TIMEOUT: 120
      CLIENT_MAX_BODY_SIZE: 50m
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs
    ports:
      - "8080:8080"

  queue-long:
    image: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - worker
      - --queue
      - long,default,short
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  queue-short:
    image: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - worker
      - --queue
      - short,default
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  redis-queue:
    image: redis:6.2-alpine
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - redis-queue-data:/data

  redis-cache:
    image: redis:6.2-alpine
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - redis-cache-data:/data

  scheduler:
    image: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - schedule
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  websocket:
    image: registry.gitlab.com/castlecraft/cepl-erpnext-images/erpnext-raven:v1.7.0
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - node
      - /home/frappe/frappe-bench/apps/frappe/socketio.js
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

volumes:
  db-data:
  redis-queue-data:
  redis-cache-data:
  sites:
  logs:

Using the same build steps I’ve built pipelines for lot of teams on lot of different CI runners. There must be 100+ builds and deploys happening daily. No failure alerts from any of them.

Automated tests pass, I can re-create what was documented. I did exactly what I had written and said in videos and it worked.

Peer · September 16, 2024, 9:37am

I know you master this stuff. This takes time and practice.
Integration of the agilely moving parts are another area, some issues are unwittingly created where the apps intersect, or should intersect. That’s why you got that error trying wiki on v15.
Is there a public table showing which versions are compatible or are not? Otherwise the whole ecosystem would be reduced to trial and error for the incompatibilites already found.

If I understand the build process and this thread correctly, the OP was not aware that the docker image (or the relevant software parts in it) is immutable, so using “bench install-app” in the running system won’t work (or needs to be modified somehow: pushed into the build step).

Thank you very much for showcasing your working procedure demonstrating once more how the moving parts interact, and for the more general hints!

revant_one · September 16, 2024, 1:52pm

Check for how frequently the git tags are created and how frequently the branches are updated. If tags are frequent use tags, or stick to frequent branch.

Wiki has tag. It is not frequently bumped up. master branch has frequent changes. So I used master.

EI-IT · September 16, 2024, 1:52pm

I’ve watched the videos and read the documentation. And I did see this post/ documentation. It fails at docker build.

I’m using ubuntu 22.04

Linux ei-dev 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

After running commands of the docker build it fails to change the permissions to the templates file

ERROR: failed to solve: process "/bin/sh -c useradd -ms /bin/bash frappe     && apt-get update     && apt-get install --no-install-recommends -y     curl     git     vim     nginx     gettext-base     file     libpango-1.0-0     libharfbuzz0b     libpangoft2-1.0-0     libpangocairo-1.0-0     restic     gpg     mariadb-client     less     libpq-dev     postgresql-client     wait-for-it     jq     && mkdir -p ${NVM_DIR}     && curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.5/install.sh | bash     && . ${NVM_DIR}/nvm.sh     && nvm install ${NODE_VERSION}     && nvm use v${NODE_VERSION}     && npm install -g yarn     && nvm alias default v${NODE_VERSION}     && rm -rf ${NVM_DIR}/.cache     && echo 'export NVM_DIR=\"/home/frappe/.nvm\"' >>/home/frappe/.bashrc     && echo '[ -s \"$NVM_DIR/nvm.sh\" ] && \\. \"$NVM_DIR/nvm.sh\"  # This loads nvm' >>/home/frappe/.bashrc     && echo '[ -s \"$NVM_DIR/bash_completion\" ] && \\. \"$NVM_DIR/bash_completion\"  # This loads nvm bash_completion' >>/home/frappe/.bashrc     && if [ \"$(uname -m)\" = \"aarch64\" ]; then export ARCH=arm64; fi     && if [ \"$(uname -m)\" = \"x86_64\" ]; then export ARCH=amd64; fi     && downloaded_file=wkhtmltox_${WKHTMLTOPDF_VERSION}.${WKHTMLTOPDF_DISTRO}_${ARCH}.deb     && curl -sLO https://github.com/wkhtmltopdf/packaging/releases/download/$WKHTMLTOPDF_VERSION/$downloaded_file     && apt-get install -y ./$downloaded_file     && rm $downloaded_file     && rm -rf /var/lib/apt/lists/*     && rm -fr /etc/nginx/sites-enabled/default     && pip3 install frappe-bench     && sed -i '/user www-data/d' /etc/nginx/nginx.conf     && ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log     && touch /run/nginx.pid     && chown -R frappe:frappe /etc/nginx/conf.d     && chown -R frappe:frappe /etc/nginx/nginx.conf     && chown -R frappe:frappe /var/log/nginx     && chown -R frappe:frappe /var/lib/nginx     && chown -R frappe:frappe /run/nginx.pid     && chmod 755 /usr/local/bin/nginx-entrypoint.sh     && chmod 644 /templates/nginx/frappe.conf.template" did not complete successfully: exit code: 100

Not sure why it would build on arch and not debian. I will try it on windows her in a bit

revant_one · September 16, 2024, 2:03pm

I build on ubuntu 22.04 here build (#7777442615) · Jobs · castlecraft / CEPL ERPNext Images · GitLab, It works.

I just manually tried on ubuntu VM and it worked.

cat /etc/os-release 
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

EI-IT · September 16, 2024, 2:06pm

Which docker are you using from snap or apt get docker?
I’m using the one on snap

revant_one · September 16, 2024, 2:11pm

I’m not using snap. I installed using apt-get. (using the bash script from https://get.docker.com)

Also Dockerfile and shell scripts in resources need to be confirmed. LF and not CRLF.

EI-IT · September 16, 2024, 2:22pm

I guess snap is the culprit
it is also building just fine on windows.

Incase anyone else is building on windows. The command to export apps.json to base64 in PowerShell is below (if frappe_docker is in documents director)

$APPS_JSON_BASE64 = [Convert]::ToBase64String([IO.File]::ReadAllBytes("Documents/frappe_docker/apps.json"))```

Peer · September 17, 2024, 10:26am

I also tried it.
Works perfectly.

Also, first time I get to see raven in action, and integrated with erpnext.

Installed Apps

ERPNext: v15.35.1
Frappe Framework: v15.40.6
Raven: v1.7.0

EI-IT · September 19, 2024, 2:01am

So I am trying a large amount of apps and it shows that all of them are installed but doesn’t load some of them. Like drive, payments, and shipping doesn’t load or show in desk

apps.json

[

  {
    "url": "https://github.com/frappe/erpnext",
    "branch": "v15.35.1"
  },
  {
      "url": "https://github.com/The-Commit-Company/Raven",
      "branch": "v1.7.0"
  },
  {
    "url": "https://github.com/frappe/payments",
    "branch": "version-15"
  },
  {
    "url": "https://github.com/frappe/drive",
    "branch": "v0.0.3"
  },
  {
    "url": "https://github.com/frappe/lms",
    "branch": "v2.5.0"
  },
  {
    "url": "https://github.com/frappe/insights",
    "branch": "v2.2.6"
  },
  {
    "url": "https://github.com/frappe/hrms",
    "branch": "v15.29.0"
  },
  {
    "url": "https://github.com/phamos-eu/it_management",
    "branch": "version-15"
  },
  {
    "url": "https://github.com/frappe/erpnext-shipping.git",
    "branch": "v15.0.2"
  },
  {
    "url": "https://github.com/frappe/print_designer",
    "branch": "v1.4.2"
  }
]

revant_one · September 19, 2024, 10:52am

Make sure you install apps on site.

Check Help > About from desk to check apps installed on site.

EI-IT · September 19, 2024, 1:44pm

Yes the docker compose has the command for each app
but its still inaccessible.

services:
  backend:
    image: eic.erp:staging
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  configurator:
    image: eic.erp:staging
    deploy:
      restart_policy:
        condition: none
    entrypoint:
      - bash
      - -c
    # add redis_socketio for backward compatibility
    command:
      - >
        ls -1 apps > sites/apps.txt;
        bench set-config -g db_host $$DB_HOST;
        bench set-config -gp db_port $$DB_PORT;
        bench set-config -g redis_cache "redis://$$REDIS_CACHE";
        bench set-config -g redis_queue "redis://$$REDIS_QUEUE";
        bench set-config -g redis_socketio "redis://$$REDIS_QUEUE";
        bench set-config -gp socketio_port $$SOCKETIO_PORT;
    environment:
      DB_HOST: db
      DB_PORT: "3306"
      REDIS_CACHE: redis-cache:6379
      REDIS_QUEUE: redis-queue:6379
      SOCKETIO_PORT: "9000"
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  create-site:
    image: eic.erp:staging
    deploy:
      restart_policy:
        condition: none
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs
    entrypoint:
      - bash
      - -c
    command:
      - >
        wait-for-it -t 120 db:3306;
        wait-for-it -t 120 redis-cache:6379;
        wait-for-it -t 120 redis-queue:6379;
        export start=`date +%s`;
        until [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".db_host // empty"` ]] && \
          [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_cache // empty"` ]] && \
          [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_queue // empty"` ]];
        do
          echo "Waiting for sites/common_site_config.json to be created";
          sleep 5;
          if (( `date +%s`-start > 120 )); then
            echo "could not find sites/common_site_config.json with required keys";
            exit 1
          fi
        done;
        echo "sites/common_site_config.json found";
        bench new-site --no-mariadb-socket --admin-password=admin --db-root-password=admin --install-app erpnext --install-app raven --install-app payments --install-app drive --install-app it_management --install-app hrms --install-app insights --install-app lms --install-app erpnext_shipping --install-app print_designer --set-default frontend ;

  db:
    image: mariadb:10.6
    healthcheck:
      test: mysqladmin ping -h localhost --password=admin
      interval: 1s
      retries: 15
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - --character-set-server=utf8mb4
      - --collation-server=utf8mb4_unicode_ci
      - --skip-character-set-client-handshake
      - --skip-innodb-read-only-compressed # Temporary fix for MariaDB 10.6
    environment:
      MYSQL_ROOT_PASSWORD: admin
    volumes:
      - db-data:/var/lib/mysql

  frontend:
    image: eic.erp:staging
    depends_on:
      - websocket
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - nginx-entrypoint.sh
    environment:
      BACKEND: backend:8000
      FRAPPE_SITE_NAME_HEADER: frontend
      SOCKETIO: websocket:9000
      UPSTREAM_REAL_IP_ADDRESS: 127.0.0.1
      UPSTREAM_REAL_IP_HEADER: X-Forwarded-For
      UPSTREAM_REAL_IP_RECURSIVE: "off"
      PROXY_READ_TIMEOUT: 120
      CLIENT_MAX_BODY_SIZE: 50m
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs
    ports:
      - "8080:8080"

  queue-long:
    image: eic.erp:staging
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - worker
      - --queue
      - long,default,short
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  queue-short:
    image: eic.erp:staging
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - worker
      - --queue
      - short,default
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  redis-queue:
    image: redis:6.2-alpine
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - redis-queue-data:/data

  redis-cache:
    image: redis:6.2-alpine
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - redis-cache-data:/data

  scheduler:
    image: eic.erp:staging
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - schedule
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  websocket:
    image: eic.erp:staging
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - node
      - /home/frappe/frappe-bench/apps/frappe/socketio.js
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

volumes:
  db-data:
  redis-queue-data:
  redis-cache-data:
  sites:
  logs:

revant_one · September 20, 2024, 4:49am

As per the about dialog apps are installed.

Enable them via domain settings, workspaces, permissions. Check docs about configuration. Apps may add new domains that can be enabled. Enable domains.

You’re now out of scope of setup. If you manage to get those apps running on vm you’ll land up in same place.

Check for the non-setup docs now.