ERPNext, HRMS & Frappe Framework v16 Release Dates

Announcements
, , ,
7

You’ll see on average a 2x speed up on most typical requests. It won’t address very specific things like a datatable architecture or some slowness attributable to a particular app’s design.

This was achieved by reducing overheads and speeding up all core utilities. E.g.

  1. Frequent Redis accesses are completely eliminated - Beating Redis with a Dictionary and Redis | Frappe Blog
  2. We use C library for the database connector instead of Python. This gives 3-5x speedup in DB query result parsing, especially on large data. Contributed by @snv feat: re-introduce `mysqlclient` 🚀 by sagarvora · Pull Request #31719 · frappe/frappe · GitHub
  3. Page rendering is faster by caching compiled templates.
  4. There’s an experimental no-fork worker type that gives ~5x-9x speed up when you have high-frequency background jobs. perf: Support running RQ worker without forking by ankush · Pull Request #31283 · frappe/frappe · GitHub
  5. We made threaded workers the default and found most multithreading bugs and addressed them. This allows Frappe deployments to be way more memory efficient by achieving concurrency using threads instead of processes. We run a forked version of gunicorn with multi-threading fixes specific to Frappe. GitHub - frappe/gunicorn: gunicorn 'Green Unicorn' is a WSGI HTTP Server for UNIX, fast clients and sleepy applications.
  6. There is a long, long tail of microoptimizations that fixed slowness in utility functions like…
    1. DB ORM functions - get_value/get_all (1.5x-3x), query builder (2x-5x), get_doc (7x), get_meta (180x).
    2. Page rendering (3x-5x)
    3. doc.save (2x)
    4. cache utils like site_cache (4x), redis_cache (30x)
    5. large SQL result sets (5x)
    6. date, data parsing utils now use C versions (2x-10x)
    7. Server script overheads (10x)
    8. Global search (O(n) → FTS)
    9. get_lazy_doc in internal functions, to not load entire document but load child tables on demand.
    10. background scheduler (2x)

Overall, there were ~200 PRs in Frappe + ERPNext + FC and nearly 100% of my time for ~4 months on this project :slight_smile:. The number 2x is validated with several benchmarks and load tests. You can find the code here: GitHub - frappe/caffeine: Performance Engineering @ Frappe

43 Likes