How I installed ERPNext on a Kubernetes cluster on GKE

I probably spent 30 hours to figure this out, playing with k3s on EC2, EKS and finally GKE. I’m not an expert with K8s, beginner level I would consider myself.

A great deal of help from @revant_one helped me to get here, so special thanks to him.

Here are the exact steps I did.

  1. Create a K8s cluster. In this case I used GKE, but other K8s clusters should work too. This step is irrelevant to ERPNext installation.

  2. Install an in-cluster NFS service

The special thing here is to make the storage larger than 8GiB, otherwise ERPNext will complain when claiming PV.

# prepare NFS, this NFS has to be slightly larger than 8GiB, or ERPNext will fail to bind the storage
kubectl create namespace nfs
helm repo add nfs-ganesha-server-and-external-provisioner https://kubernetes-sigs.github.io/nfs-ganesha-server-and-external-provisioner
helm upgrade --install -n nfs in-cluster nfs-ganesha-server-and-external-provisioner/nfs-server-provisioner --set 'storageClass.mountOptions={vers=4.1}' --set persistence.enabled=true --set persistence.size=9Gi
  1. Install ERPNext

Nothing special here, these are just copy and paste from the helm chart.

# install ERPNext, don't use custom values
kubectl create namespace erpnext
helm repo add frappe https://helm.erpnext.com
helm upgrade --install frappe-bench --namespace erpnext frappe/erpnext --set persistence.worker.storageClass=nfs
  1. Create a site, modify the values.yaml file which you can find in the helm chart. This is just following up the guide.
jobs:
  createSite:
    enabled: true
    siteName: "erp.example.com"
    adminPassword: "secret"

Run the following commands:

helm template frappe-bench -n erpnext frappe/erpnext -f custom-values.yaml -s templates/job-create-site.yaml > create-new-site-job.yaml
kubectl apply -f create-new-site-job.yaml -n erpnext
  1. Verify the site is working. You can certainly check the pods in the K8s cluster too.
kubectl port-forward -n erpnext svc/frappe-bench-erpnext 8080:8080
curl -H "Host: erp.example.com" http://0.0.0.0:8080/api/method/ping

It should return:

{"message":"pong"}
  1. Install ingress-nginx
helm install nginx-ingress ingress-nginx/ingress-nginx
  1. Create a new ingress to route the traffic from internet

Create a yaml file called ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: erp-resource
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - host: "erp.example.com"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: frappe-bench-erpnext
            port:
              number: 8080
kubectl apply -f ingress.yaml -n erpnext

Then you should be able to visit it from erp.example.com.

This is only http, so you will get insecure access warning from the browser.

2 Likes

if budget is not the limit then go for https://cloud.google.com/filestore/docs/service-tiers
instead of in cluster nfs service

CSI installation: Access Filestore instances with the Filestore CSI driver  |  Google Kubernetes Engine (GKE)  |  Google Cloud

also instead of In cluster MariaDB, use Google SkySQL

Hi Revant,

Thanks for your inputs. These are great suggestions for taking the basic k8s setup to another level with fully managed persistent storage both for volumes and database.

GKE setup is my test bench for now. I’m also considering AWS options, e.g., EKS + EFS + RDS.

I’m learning managing the letsencrypt cert manager within the cluster for now.

Here are steps for it. castlecraft / aws-eks-erpnext · GitLab

1 Like

Hi Revant,

This is most helpful, I’m spoiled.

thank you for posting this

I don’t know if this needs separate posts.

here’s kubernetes operator that can be used to automate ingress creation on site creation.

it is multiple things:

  • python app
  • fastapi app serving rest api
  • frappe app (bench get-app)
  • kopf k8s operator that watches jobs and manages ingress

source: castlecraft / k8s_bench · GitLab
docs and helm chart: https://k8s-bench.castlecraft.in/

here’s another app that creates benches and site, it is obsolete and uses old Fluxcd API. It was used to host feature board that created sites based on GitHub pull request. I’m sharing it for code reference.

Thank you for this post. Have you succeeded getting it working with OpenEBS or Longhorn? For some reason, mounting the volumes does not work, although I installed the required packages to get NFS functionality working (theoritically).

I’ve not tried it on OpenEBS or Longhorn.

I was able to get it running using rook/ceph Rook - Rook Ceph Documentation

make sure you’ve a working storage class with RWX (ReadWriteMany) access mode.

@revant_one thank you for your answer. Actually I have just seen that it even does not work with the suggested NFS provisioner.

Have you tried the latest available helm chart?

At the moment using the NFS I even get the following issue with MariaDB:

kubectl describe pod -n erpnext frappe-bench-mariadb-0
kubectl describe pod -n erpnext frappe-bench-mariadb-0

  Warning  FailedScheduling  2m19s  default-scheduler  0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.

With longhorn I had similar issue but with erpnext pods.

Could you give it a try please and let me know? It is gonna be great if you can try Longhorn, but I understand if you don’t have time for that.

Thank you!

make sure you’ve a working storage class with RWX (ReadWriteMany) access mode.

Actually Longhorn supports that almost out of the box when nfs-common is installed on all nodes.

share your values file.

you’re facing issue with MariaDB pod. it doesn’t use RWX storage. check official docs charts/README.md at main · bitnami/charts · GitHub

I am using no custom values, just the default values used in the helm chart page. So basically just copy and paste to test if it gonna work at all.

The only thing I tried to change was the storageClass from NFS to longhorn (but both did not work, longhorn even did a bit better job, MariaDB was not an issue with it). I am gonna dig a little further and let you know if I have any news.

Hi @revant_one @madwyn, I run the following helm command to install in-cluster NFS on my k8s cluster however, the nfs pod is not able to pull image, when I described the in-cluster-nfs-server-provisioner-0 pod, I can see the error "Back-off pulling image “k8s.gcr.io/sig-storage/nfs-provisioner:v4.0.8"” and this also affect the frappe pods from running with error. Please how can I resolve this?

It seems image k8s.gcr.io/sig-storage/nfs-provisioner:v4.0.8 is no longer available?

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.