Help Moving a site to Docker

Hi,

I’m trying to move a site to a Docker setup.

Editing the pwd.yml and changing bench new-site and FRAPPE_SITE_NAME_HEADER:to the name of the site being transferred. Then docker compose -f pwd.yml up -d. The command completes but the site renders Internal Server Error. I use bench restore with --force in the backend container.

With respect to this post, are there other steps that I need to take using the above? Would adding -no-mariadb-socket be advisable?

The logs for the backend say: AssertionError: site must be fully initialized, db_user missing

It is must! It allows db_user@% (wildcard host).

This is aarch64 , the pwd.yml has image: frappe/erpnext:latest and
platform: linux/arm64

I added --no-mariadb-socket to the standard new-site stanza in pwd.yml and composed it. Then:

docker compose exec backend bench drop-site frontend
docker compose exec backend bench new-site -no-mariadb-socket (siteToMigrate)
docker compose exec backend bench --site siteToMigrate --force restore siteToMigraate-database.sql.gz
docker compose exec backend bench --site siteToMigrate migrate

But I got a site frontend no found in the browser

Edited the pwd. commented out the bench-new site stanza, changed FRAPPE_SITE_NAME_HEADER: to siteToMigrate. recomposed and the ip.of.the.server:8080/login worked, ip of the server:8080 by itself not,

Also needed to docker compose exec backend bench --site siteToMigrate enable-scheduler.

docker compose exec backend bench version
erpnext 16.0.0-dev
frappe 16.0.0-dev

Having issues with email, replacing the encryption key with the one from the source didn’t help. That’s not a big deal.

Is this in develop or production mode?

docker compose exec backend/frontend show-config says
default_site | frontend
instead of the migrated site name. Something I missed in the pwd.yml ?

I didn’t try it on aarch64. Hope someone who uses it helps.

Everything I mentioned works for me on amd64.

Removed default site from common_site_config.json

wkhtmltopdf wasn’t happy and emails were not sent:

Error log had:

builtins.OSError: wkhtmltopdf reported an error:
Exit with code 1 due to network error: ConnectionRefusedError

Tried adding: “host_name”: “http://127.0.0.1:80” to site config, which didn’t work so tried setting it to the ip reported in the backend , and that worked.

I think a better knowledge of tcompose pwd.yml options would have made it easier to orchestrate, but the migration looks like is succeeded.

1 Like

That’s the kind of “side”-effects which makes it difficult to learn all the mechanics.
Such are all over the place.
One might think about that “option” that “it’s for not going via network” (e.g. slower) but it also and importantly is about permissions.

So you need to know and learn/memorize all of these unspoken “side”-effects.

On my imagined “Big Frappe DevOps Poster” these would all be explicited with arrows and short explanations or references to the same.

I didn’t start such a poster yet, but maybe one day I will. It could become a nice giveaway, and a help for newbies needing to DevOps. On the other hand, once really through with all this, your own itch-to-be-scratched disappears and thus part of the motivation to posterize the changing system.

The place of memorizing stuff can be quite different between cultures, and so can be the memorizing capabilities.
So these “convention over configuration”, “explicit over implicit”, “inheritance over restating” (aka: DRY) etc. memes/ITcultures have different trade-offs regarding their understandability/learnability by DevOps tasked newbies.

I copied the new-site stanza out of yml and tried it on the host, after dropping the site.

docker compose exec backend bench new-site --no-mariadb-socket --mariadb-user-host-login-scope=‘%’ --admin-password=admin --db-root-username=root --db-root-password=admin --install-app erpnext --set-default frontend

which returned:

–no-mariadb-socket is DEPRECATED; use --mariadb-user-host-login-scope=‘%’ (wildcard) or --mariadb-user-host-login-scope=, instead. The name of this option was misleading: it had not> hing to do with sockets.

Is the --no-mariadb-socket needed in this case?

If you are using --mariadb-user-host-login-scope='%' no need for --no-mariadb-socket

There was some issue with --mariadb-user-host-login-scope='%' which was fixed in this commit fix: don't use `frappe.flags` for `mariadb_user_host_login_scope` · frappe/frappe@0d3b249 · GitHub

I’ve been facing the same issue with wkhtmltopdf for print formatting when using Docker—specifically with the pwd.yml compose file only. Despite trying everything over the past 10 months, I haven’t been able to resolve it. As a result, I can’t use print formats created with the Print Designer app or send emails with these formats.

If someone could create a video tutorial or update the documentation to help fix this issue in Frappe, it would be greatly appreciated.

@revant_one :saluting_face:

Hi

Try: docker network ls and note the ip of the frontend.

Then try: docker compose exec backend bench --site frontend set-config hostname (ip.address.of.the.frontend) or enter the backend container and manually edit the sites site_config.json and add “hostname: ip.of.the.backend”

That method seems to be working for me, at some point I’ll try and incorporate it into a pwd.

1 Like

@edardev you’re already part of that thread.

In case of pwd, did you try changing FRAPPE_SITE_NAME_HEADER to $host?

@revant_one When I set FRAPPE_SITE_NAME_HEADER to $host, I get a “Not Found” error in the browser. To properly serve the frontend, FRAPPE_SITE_NAME_HEADER must match the site name. If it doesn’t, the “Not Found” error appears.

@smino I added both the frontend’s and backend’s internal IPs, but it still doesn’t work. Below is the Docker Compose configuration:


services:
  backend:
    image: frappe/erpnext:v15.48.2
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs
    environment:
      DB_HOST: db
      DB_PORT: "3306"
      MYSQL_ROOT_PASSWORD: admin
      MARIADB_ROOT_PASSWORD: admin

  configurator:
    image: frappe/erpnext:v15.48.2
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: none
    entrypoint:
      - bash
      - -c
    command:
      - >
        ls -1 apps > sites/apps.txt;
        bench set-config -g db_host $$DB_HOST;
        bench set-config -gp db_port $$DB_PORT;
        bench set-config -g redis_cache "redis://$$REDIS_CACHE";
        bench set-config -g redis_queue "redis://$$REDIS_QUEUE";
        bench set-config -g redis_socketio "redis://$$REDIS_QUEUE";
        bench set-config -gp socketio_port $$SOCKETIO_PORT;
        bench --site frontend set-config hostname 172.21.0.9
    environment:
      DB_HOST: db
      DB_PORT: "3306"
      REDIS_CACHE: redis-cache:6379
      REDIS_QUEUE: redis-queue:6379
      SOCKETIO_PORT: "9000"
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  create-site:
    image: frappe/erpnext:v15.48.2
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: none
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs
    entrypoint:
      - bash
      - -c
    command:
      - >
        wait-for-it -t 120 db:3306;
        wait-for-it -t 120 redis-cache:6379;
        wait-for-it -t 120 redis-queue:6379;
        export start=`date +%s`;
        until [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".db_host // empty"` ]] && \
          [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_cache // empty"` ]] && \
          [[ -n `grep -hs ^ sites/common_site_config.json | jq -r ".redis_queue // empty"` ]];
        do
          echo "Waiting for sites/common_site_config.json to be created";
          sleep 5;
          if (( `date +%s`-start > 120 )); then
            echo "could not find sites/common_site_config.json with required keys";
            exit 1
          fi
        done;
        echo "sites/common_site_config.json found";
        bench new-site --mariadb-user-host-login-scope='%' --admin-password=admin --db-root-username=root --db-root-password=admin --install-app erpnext --set-default frontend;

  db:
    image: mariadb:10.6
    networks:
      - frappe_network
    healthcheck:
      test: mysqladmin ping -h localhost --password=admin
      interval: 1s
      retries: 20
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - --character-set-server=utf8mb4
      - --collation-server=utf8mb4_unicode_ci
      - --skip-character-set-client-handshake
      - --skip-innodb-read-only-compressed # Temporary fix for MariaDB 10.6
    environment:
      MYSQL_ROOT_PASSWORD: admin
      MARIADB_ROOT_PASSWORD: admin
    volumes:
      - db-data:/var/lib/mysql

  frontend:
    image: frappe/erpnext:v15.48.2
    networks:
      - frappe_network
    depends_on:
      - websocket
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - nginx-entrypoint.sh
    environment:
      BACKEND: backend:8000
      FRAPPE_SITE_NAME_HEADER: $host
      SOCKETIO: websocket:9000
      UPSTREAM_REAL_IP_ADDRESS: 127.0.0.1
      UPSTREAM_REAL_IP_HEADER: X-Forwarded-For
      UPSTREAM_REAL_IP_RECURSIVE: "off"
      PROXY_READ_TIMEOUT: 120
      CLIENT_MAX_BODY_SIZE: 50m
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs
    ports:
      - "8080:8080"

  queue-long:
    image: frappe/erpnext:v15.48.2
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - worker
      - --queue
      - long,default,short
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  queue-short:
    image: frappe/erpnext:v15.48.2
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - worker
      - --queue
      - short,default
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  redis-queue:
    image: redis:6.2-alpine
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - redis-queue-data:/data

  redis-cache:
    image: redis:6.2-alpine
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: on-failure
    volumes:
      - redis-cache-data:/data

  scheduler:
    image: frappe/erpnext:v15.48.2
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - bench
      - schedule
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

  websocket:
    image: frappe/erpnext:v15.48.2
    networks:
      - frappe_network
    deploy:
      restart_policy:
        condition: on-failure
    command:
      - node
      - /home/frappe/frappe-bench/apps/frappe/socketio.js
    volumes:
      - sites:/home/frappe/frappe-bench/sites
      - logs:/home/frappe/frappe-bench/logs

volumes:
  db-data:
  redis-queue-data:
  redis-cache-data:
  sites:
  logs:

networks:
  frappe_network:
    driver: bridge
Server Error
Traceback (most recent call last):
  File "apps/frappe/frappe/app.py", line 114, in application
    response = frappe.api.handle(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/api/__init__.py", line 49, in handle
    data = endpoint(**arguments)
           ^^^^^^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/api/v1.py", line 36, in handle_rpc_call
    return frappe.handler.handle()
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/handler.py", line 50, in handle
    data = execute_cmd(cmd)
           ^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/handler.py", line 86, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/__init__.py", line 1726, in call
    return fn(*args, **newargs)
           ^^^^^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/utils/typing_validations.py", line 31, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/utils/print_format.py", line 236, in download_pdf
    pdf_file = frappe.get_print(
               ^^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/__init__.py", line 2142, in get_print
    return get_pdf(html, options=pdf_options, output=output) if as_pdf else html
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "apps/frappe/frappe/utils/pdf.py", line 90, in get_pdf
    filedata = pdfkit.from_string(html, options=options or {}, verbose=True)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "env/lib/python3.11/site-packages/pdfkit/api.py", line 75, in from_string
    return r.to_pdf(output_path)
           ^^^^^^^^^^^^^^^^^^^^^
  File "env/lib/python3.11/site-packages/pdfkit/pdfkit.py", line 201, in to_pdf
    self.handle_error(exit_code, stderr)
  File "env/lib/python3.11/site-packages/pdfkit/pdfkit.py", line 155, in handle_error
    raise IOError('wkhtmltopdf reported an error:\n' + stderr)
OSError: wkhtmltopdf reported an error:
Exit with code 1 due to network error: ConnectionRefusedError

Your bench new site does not have the -no-mariadb-socket, may or may not matter.

I also see the ip in my site_config is “host_name”: “http://172.18.0.7:8080”, note the port! Try adding the port?

1 Like

Try to escape the $ symbol with $$, the final string will become $$host.

rename the frontend directory in /home/frappe/frappe-bench/sites/frontend to /home/frappe/frappe-bench/sites/your.site.com.

make sure frontend is not set as default site in common_site_config.json.

2 Likes

Wow, it seems to be working after adding port 8080! I’ll proceed to install the print designer to test it further. :raised_hands:
image

1 Like

Hi @revant_one! I had tried $ and $$ before without success, but it’s finally working now after adding port 8080 to the hostname :tada:. Thanks so much for your help—this community is truly the best!

image

What is the difference between hostname and host_name inside the site config file? While they appear to work similarly, they do not function together and seem duplicated.

Here’s the issue I’m facing now:

  1. The print format works correctly when either setting the Frontend IP with the port or setting the hostname to Frontend:8080 in the site_config.json file.

  2. However, when emails are sent, the link generated to view documents points to http://Frontend:8080/etc instead of the actual real domain, such as https://example.com/etc.

Question:

How can I modify this behavior to ensure both functionalities work seamlessly without breaking other features in Frappe?

@revant_one @smino @Peer

That was half part, other part was renaming the site directory from frontend to example.com

remove the hostname and try after that. You can add host_name to the actual site name and not frontend.

Yes, the site was initially created using the actual site name, e.g., sub.example.com.

The current issue is that setting host_name breaks the print format when replacing for the actual site URL (https://sub.example.com) instead of Frontend:8080 or Frontend ip.

@revant_one, by “Frontend:8080,” I am referring to the service inside the Docker Compose setup. Instead of using the dynamic IP of the service assigned by docker , you can use the service’s name as defined in the docker-compose.yml file.

I attempted to modify the header to $host and $$host, but it didn’t seem to have any effect.

Question:

How can I resolve this issue so that the site consistently uses the actual domain URL (https://sub.example.com) without breaking other functionalities?



I spent another day trying to resolve the host_name issues. While I’m glad we managed to get the print formats working, it’s frustrating that this breaks the email links for the site—and possibly in other places too. Something still feels incomplete.

I’ve created an issue in the Frappe Docker repository, hoping someone might figure out a solution before I do. Here’s the link: frappe_docker#1547.

1 Like