Frappe/ERPNext with Dragonfly instead of redis

Frappe Framework App Development
, ,
1

Q: Has anyone used Dragonfly with frappe in production or has any experience?

I recently had a use-case around replacing Redis with Dragonfly (a multithreaded drop-in replacement for Redis).

Why consider Dragonfly?

  1. Scalability: At a higher scale, Redis requires clustering, which can be significantly more expensive (~5x) than Dragonfly.
  2. Performance: Redis is single-threaded, whereas Dragonfly is multithreaded, potentially offering better performance.

Does it work?

I tested with both Frappe v14 and v15, and it seems to work out of the box with the latest Dragonfly (1.20.1).

So far, I’ve tested Background Jobs and Redis cache, and both appear to function as expected.

RQ_JOBS_STARTED
RQ_JOBS_STARTED1845×998 151 KB

RQ_JOBS_COMPLETED
RQ_JOBS_COMPLETED1845×998 153 KB

How to setup?

I used frappe docker with a packed image for Dragonfly 1.20.1 from dragonflydb.io docker.dragonflydb.io/dragonflydb/dragonfly

Here’s my docker-compose configuration if you want to try it:

version: "3.7"
services:
  mariadb:
    image: docker.io/mariadb:10.6
    command:
      - --character-set-server=utf8mb4
      - --collation-server=utf8mb4_unicode_ci
      - --skip-character-set-client-handshake
      - --skip-innodb-read-only-compressed
    environment:
      MYSQL_ROOT_PASSWORD: CHANGE_IT
    volumes:
      - mariadb-data:/var/lib/mysql

  redis-cache:
    image: docker.dragonflydb.io/dragonflydb/dragonfly
    ulimits:
      memlock: -1
    ports:
      - 6379:6379
    volumes:
        - redisQueueData:/data

  redis-queue:
    image: docker.dragonflydb.io/dragonflydb/dragonfly
    ulimits:
      memlock: -1
    ports:
      - 6380:6379
    volumes:
        - redisQueueData:/data

  frappe:
    image: docker.io/frappe/bench:latest
    command: sleep infinity
    environment:
      - SHELL=/bin/bash
    volumes:
      - ..:/workspace:cached
    working_dir: /workspace/development
    ports:
      - 8000-8005:8000-8005
      - 9000-9005:9000-9005

volumes:
  mariadb-data:
  redisQueueData:
  redisCacheData:

Note: This setup is for development purposes. For production use, ensure proper security measures and performance tuning.

Next steps:

  • Conduct thorough performance testing comparing Redis and Dragonfly in a Frappe environment.
  • Test all Frappe features that rely on Redis to ensure full compatibility.
  • Share findings with the Frappe community for feedback and further testing.

Has anyone else experimented with Dragonfly in a Frappe setup? I’d be interested in hearing about your experiences or any potential issues you’ve encountered.

3 Likes