Custom Frappe Docker with HRMS App | Tables tabdepartment and tabtask is marked as crashed and should be repaired

gsarunk · September 4, 2022, 10:31am

Team,
I have been trying out frappe_docker official for the past two weeks.
With the documentations i was able to setup multi bench in single server.

From V14 HRMS is no more part of standard ERPNext modules. I was trying to find if there is a standard image with HRMS. I could not find one.
Then I started to follow the custom app documentation to build an image with HRMS.

I was successful creating a bench and site with erpnext
When i install HRMS app, i constantly get the same error

Installing hrms...
Updating DocTypes for hrms          : [========================================] 100%
Traceback (most recent call last):
  File "/usr/local/bin/patched_bench_helper.py", line 48, in <module>
    raise SystemExit(main())
  File "/usr/local/bin/patched_bench_helper.py", line 43, in main
    frappe.utils.bench_helper.main()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/utils/bench_helper.py", line 18, in main
    click.Group(commands=commands)(prog_name="bench")
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/__init__.py", line 29, in _func
    ret = f(frappe._dict(ctx.obj), *args, **kwargs)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/site.py", line 357, in reinstall
    _reinstall(site, admin_password, db_root_username, db_root_password, yes, verbose=context.verbose)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/commands/site.py", line 381, in _reinstall
    _new_site(
  File "/home/frappe/frappe-bench/apps/frappe/frappe/installer.py", line 101, in _new_site
    install_app(app, verbose=verbose, set_as_patched=not source_sql, force=False)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/installer.py", line 299, in install_app
    frappe.get_attr(after_install)()
  File "/home/frappe/frappe-bench/apps/hrms/hrms/setup.py", line 12, in after_install
    create_custom_fields(get_custom_fields())
  File "/home/frappe/frappe-bench/apps/frappe/frappe/custom/doctype/custom_field/custom_field.py", line 208, in create_custom_fields
    frappe.db.updatedb(doctype)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/database.py", line 365, in updatedb
    db_table.sync()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/schema.py", line 44, in sync
    self.alter()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/schema.py", line 148, in alter
    raise e
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/mariadb/schema.py", line 132, in alter
    frappe.db.sql(query)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/database/database.py", line 209, in sql
    self._cursor.execute(query, values)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/pymysql/cursors.py", line 148, in execute
    result = self._query(query)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/pymysql/cursors.py", line 310, in _query
    conn.query(q)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/pymysql/connections.py", line 548, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/pymysql/connections.py", line 775, in _read_query_result
    result.read()
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/pymysql/connections.py", line 1156, in read
    first_packet = self.connection._read_packet()
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/pymysql/connections.py", line 725, in _read_packet
    packet.raise_for_error()
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/home/frappe/frappe-bench/env/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.OperationalError: (1194, "Table 'tabtask' is marked as crashed and should be repaired")

Once logged into desk, company initially setup is missing.
when tried to create a new company getting the below error.

gsarunk · September 6, 2022, 6:41pm

Restarted the Mariadb container and it did the magic. The tables were repaired.

Mostly if you install a new site only install erpnext at first. Complete the setup and then install the hrms app.

If you face the tab is marked as crashed error restart the mariadb container.

nuttcorp · September 30, 2022, 7:18pm

How do you install HRMS once you have ERPNext installed? I have tried
bench --site erp.sitename.com install-app hrms
but it fails with “Could not find app “hrms””

kevingee · October 1, 2022, 4:27pm

You have to run this command first
bench get-app https://github.com/frappe/hrms

After this, then run
bench --site erp.sitename.com install-app hrms

nuttcorp · October 2, 2022, 1:26am

When I run bench get-app it I just get “Error: Only Frappe framework bench commands are available in container setup.”
I am trying to run within the erpnext backend container. Was hoping to do it without making an custom app container but seems not possible. seeing as this is a standard frappe app I don’t know why this cant be directly available in frappe_docker.

gsarunk · October 2, 2022, 3:36am

The official frappe Docker only ship Frappe and ERPNext in its image.

If you need HRMS you need to build your own custom image using the steps in the official frappe docker documentation.

I have built a custom image with HRMS app. You can refer to this repo for reference

You can refer to the post for more information frappe docker with custom app error

Ismail_Tabtabai · December 1, 2022, 7:33am

This procedure you presented is for loading the hrms module with the existing erpnext docker installation, or is this a whole new instance that will have hrms as well?

revant_one · December 1, 2022, 7:35am

for existing instance you can follow these steps:

replace the image: to new one with new app(s) and redeploy.
run the command ls -1 ../apps > apps.txt with sites volume attached to worker image/container. This will re-create apps.txt file with all the apps now included with new image.
after that you proceed to use bench helper Frappe Commands like install-app, uninstall-app.

For re-creating assets refer frappe_docker/setup-options.md at main · frappe/frappe_docker · GitHub

or add populate-assets service like this that will run once after each re-deploy ERPNext on Docker Swarm · GitHub

gsarunk · December 1, 2022, 8:48am

The above procedure is to create a new Docker image with HRMS Module.

adamjgale · January 9, 2023, 12:31pm

Is there a guru out there that would be kind enough to docker compose a package with ERPNext and HRMS? Seems like it would be useful rather than trying to package them all. I’ve attempted a few times and haven’t had much luck.

gsarunk · January 9, 2023, 2:28pm

I already have a repo and a image pre built with HRMS
Please find the same here in our repo

update your frappe-docker/custom_app/ compose.override.yaml like below

services:
  configurator:
    image: ghcr.io/lmnaslimited/lensdocker/worker:${VERSION}

  backend:
    image: ghcr.io/lmnaslimited/lensdocker/worker:${VERSION}

  frontend:
    image: ghcr.io/lmnaslimited/lensdocker/nginx:${VERSION}

  queue-short:
    image: ghcr.io/lmnaslimited/lensdocker/worker:${VERSION}

  queue-default:
    image: ghcr.io/lmnaslimited/lensdocker/worker:${VERSION}

  queue-long:
    image: ghcr.io/lmnaslimited/lensdocker/worker:${VERSION}

  scheduler:
    image: ghcr.io/lmnaslimited/lensdocker/worker:${VERSION}

Create a new Variable “VERSION” in your Environment file
after generating your project environment file from the below step

cp example.env ~/gitops/erpnext-one.env
sed -i 's/DB_PASSWORD=123/DB_PASSWORD=changeit/g' ~/gitops/erpnext-one.env
sed -i 's/DB_HOST=/DB_HOST=mariadb-database/g' ~/gitops/erpnext-one.env
sed -i 's/DB_PORT=/DB_PORT=3306/g' ~/gitops/erpnext-one.env
echo 'ROUTER=erpnext-one' >> ~/gitops/erpnext-one.env
echo "SITES=\`one.example.com\`,\`two.example.com\`" >> ~/gitops/erpnext-one.env
echo "BENCH_NETWORK=erpnext-one" >> ~/gitops/erpnext-one.env

INTRODUCE A VERSION VARIABLE MANUALLY

# Reference: https://github.com/frappe/frappe_docker/blob/main/docs/images-and-compose-files.md

FRAPPE_VERSION=v14.21.0

# Only with ERPNext override
ERPNEXT_VERSION=v14.11.1
#For Custom Frappe app
VERSION=latest
DB_PASSWORD=123

Docker compose with custom override to generate the new yaml. Replace the project name

docker compose --project-name $project_name\
            --env-file ~/gitops/$project_name.env \
            -f compose.yaml \
            -f overrides/compose.erpnext.yaml \
            -f overrides/compose.redis.yaml \
            -f docs/compose/compose.multi-bench.yaml \
            -f docs/compose/compose.multi-bench-ssl.yaml \
            -f custom_app/compose.override.yaml config > ~/gitops/$project_name.yaml

Compose project up -d
Then it should go through.
Note if your are updating existing project ie from V13 to V14, You must populate the apps.txt manually with hrms.
Alternatively you can drop all the volumes including assets. You must take a copy of all the sites into your local directory and put back into sites folder again.
On a fresh container this will automatically populate

adamjgale · January 9, 2023, 2:35pm

Thanks gsarunk. I’ll give this one a run and see how I make out. Appreciate it.
I found myself running around in circles looking.

tareks · January 9, 2023, 7:38pm

are there plans to add the bench get-app command into the official docker containers? We will start with ERPNEXT 13.xx because of the missing HR Module.

revant_one · January 9, 2023, 10:32pm

don’t use containers, they are immutable, you’ve to build new images to update or add apps.

you can use vm, native install, lxd or commonly discussed methods on forum.

conclusion: DO NOT use containers if you’re not already using them.

adamjgale · January 12, 2023, 2:35pm

Revant are you suggesting to avoid docker containers completely and just install direct? I find that’s more tangly when you’re already running multiple containers. My biggest issue right now is having multiple containers and the frappe_docker doesn’t seem to play well. The only version I’ve got up and running is the unprotected PWD test.

revant_one · January 12, 2023, 3:30pm

yes.

I think direct install is also a challenge. Check forum posts, everyday there is something related to installation bumped somewhere on posts.

Personally I don’t use anything from frappe_docker/docs.

For simple setups I use dockerswarm.rocks guide : ERPNext on Docker Swarm · GitHub Don’t use it! If frappe_docker didn’t work then the gist is not going to work.

For complex setups I use helm chart GitHub - frappe/helm: Helm Chart Repository for Frappe/ERPNext

For development I use vscode devcontainers. (frappe_docker/development)

That means I don’t use native direct installs in any case. I always prefer containers over native install.

Yet recently I installed on native on RHEL8 without internet connection: Frappe/ERPNext version 14 on RHEL 8 · GitHub

adamjgale · January 13, 2023, 12:06pm

Interested on your RedHat install. We’re running a large server here which is why docker containers was good for us, keeps everything seperate. I have a small mini server which I could run separate and use specificially for this function. We don’t expose ourselves to the web here, everything is vpn so this version interests me as well.

Really, I’ve been up and running, I’m just having trouble integrating HRMS which without it, the program wouldn’t be useful.

Thanks for that information, guess weekend project for me.

GeorgBNM · January 26, 2023, 7:36am

@revant_one

Don’t get it. On the one hand you say:

don’t use containers, they are immutable, you’ve to build new images to update or add apps. you can use vm, native install, lxd or commonly discussed methods on forum.
conclusion: DO NOT use containers if you’re not already using them.

And on the other hand you say:

That means I don’t use native direct installs in any case. I always prefer containers over native install.

What now? What is your recommendation for future-proof production install?

Thanks,
Georg

Patrick.St · January 26, 2023, 7:55am

Hi @GeorgBNM ,
I think this is a misunderstanding. What @revant_one was saying is that he is not using frappe_docker/docs at main · frappe/frappe_docker · GitHub for his/their installs, but instead based on the scenario Docker-Swarm or helm charts.

Maybe also see Container Basics

Docker Swarm is a native container orchestration tool for Docker. It allows you to create and manage a cluster of Docker nodes, and deploy and scale containerized applications across the cluster. It provides features such as automatic service discovery, load balancing, and service scaling. It is easy to set up and manage, and is well integrated with the Docker ecosystem.

Helm, on the other hand, is a package manager for Kubernetes. It provides a way to define, install, and upgrade complex Kubernetes application resources, such as pods, services, and ingresses, using a single configuration file called a Chart. Helm charts can be versioned, shared, and reused across teams and projects. It also provides a searchable repository of pre-built charts called Helm Hub. Helm is a more powerful tool than Docker Swarm, but it is also more complex to use, especially for teams that are new to Kubernetes.

In summary, Docker Swarm is a good choice for teams that are already using Docker and want a simple and easy-to-use orchestration tool, while Helm is a good choice for teams that are using or planning to use Kubernetes and want a powerful and flexible way to manage complex application resources.

GeorgBNM · January 26, 2023, 8:07am

Hi @Patrick.St,

Thanks for the clarification.

Fighting for getting ERPnext up with docker on a Synology NAS.

Best,
Georg