Frappix (preview)

Announcements
1

Introduction

Frappix bridges the gap between system depedencies and such that are already available in Python.

It brings the power of the entire software ecosystem to Frappe, not only the Python ecosystem.

It achieves this by plugging into one of the biggest software repositories to date.

System dependencies are declared by their identifier in pyproject.toml and are then mapped to their available implementation from the backing software repository.

When a user wants to start working on a project wired with Frappix, it is guaranteed for him to be productive at exactly on command away. Furthermore, the exact same environment used for development is recycled to produce a variety of production grade runtime artifacts, such as:

  • MicroVMs
  • VMs
  • Entire Host Operating System (fully configured)
  • OCI Containers

Whatever deployment scenario can be supported by this infrastructure.

The purpose, hence, of this work is to expand the unified expierience of the frappe ecosystem beyond its current boundary which it inherits from its language ecosystems.

Project ToDos

Before this is ready for general use, thereā€™s still some work to do, among it:

  • Quality assured and continually maintained onboarding story into the frappe framework
  • A comprehensive development environment for Linux & MacOS (Windows on WSL2, only)
  • Fully supported deployment targets: VM, MicroVM, Host OS, OCI Container
  • Appropriate documentation at its layer of the stack with clear interlinking to upstream documentation

Regardless, Iā€™m working on this in my freetime, hence it is a bootstrapped project. Please adjust your expectations, accordingly. And with that out of the way: I just wanted to gather early feedback from interested users.

Its predecessor has served me very well in production over the last year or so, so I guess itā€™s about time to spread the word just a little bit.

Iā€™m always looking for interested parties willing to contribute!! After all, thatā€™s the whole point of open sourcing it in the first place, right? So please feel free to do so ā€” and above all: not overly intimidated by Nix :pray:

15 Likes
2

A tiny little update:

  • Iā€™m in the process of porting our production deployment over to using upstream Frappix
  • In the process, we needed a better tool to pin inputs, based on nvchecker with a small patch
  • Today, a copy of our production database started running for the first time on a completely new pin of nixpkgs-23.11 which containes lots of new frappe dependencies that we had upstreamed over the last year or so
  • The amount of custom python packaging required has significantly molten down
  • Our puppeteering matrix chat integration also started working today again on a new version of the matrix server and after some refactoring on the side of its NixOS upstream module

So far so good. :rocket:

A couple of admirably audacious people in the Frappix chat room are already trying to run it for their own purposes :smile: . Hence, Iā€™m trying to keep up with the docs as much as possible. But beware, for now, there are still dragons ahead.

3 Likes
3

I added a little script to answer the following question:

Which of my merged PRs already landed in a V15 release?

frappe on ī‚  version-15-hotfix [$] via īœ˜ v18.19.1 via šŸ v3.11.8 (pyenv)
āÆ analyze-prs.py frappe/frappe blaggacao
5f9dc1f52949cd13b86c37c9526c94b95c1ad626  PR-25347                                              fix: socketio port
c66d35fe5b0e41267f2b01a9bf1b7af1ac17df40  PR-24814                                              fix(notification): ensure outgoing mail is set on comms
eadce6ccb88cce3085c818f97ca845ffb4b5bb54  PR-24704                                              feat: add more payment lifecycle events to server scripts
ea65d86d7201983260eaac0235cb00a1db834dd9  PR-24625                                              fix: shell escaping on external commands
c09f48f5d7421f71149321487aac14ee4a2d2085  PR-24521                                              feat: comply trackers with utm and add utm_content for a/b testing
5e904d407ef841c5d7e6f9fc285cccf7f1e5bfe4  PR-24474                                              fix: run-tests --module-def as not all doctypes are guaranteed to have test_*.py files
70cf76cc44cda527aa6fda75cecbbe5bdfa0b163  PR-24434                                              feat: add defaults to db superuser prompt
f19c25ec42691e556ebf5da8e891b6856291af29  PR-24432                                              chore: cleanup frappe connect II/II
483fcf89c31cb0d5ef8dcd0b3d9e5e8e87d31291  PR-24428                                              chore: cleanup frappe connect (I/II)
608f2ed20b55fe0a6d6cd078b32ed21a153ad35a  PR-24367                                              test: make phone number unique bis
1640aaad046e24fcb98ac07509b8d306b64f7b01  PR-24332                                              chore: cleanup low level db interface
78463031b5b33b3f9f30c498e21e594258956106  PR-24329                                              fix: undue error printing
7f64abc9bd0db64b4daa6a5192d0a6ae1bad7437  PR-24328                                              chore: use env variables in esbuilds
12673c1a2b2dba491b87d4342d29f4ec0e0f7bca  PR-24327  version-15-hotfix: backported via PR-24331  test: make phone number unique
431b2eb89cf7f83e1c086fa362a12ecc2f066625  PR-24318                                              chore: name current db handle properly
6cf8f4ecae55e49d9078ed1cb206a37c694ed19e  PR-24317                                              chore: remove confusing indirections from esbuild path resolution
df8d00c9e53d95592bfe956644f68ff52aaefccb  PR-24109                                              feat: add link preview capability to tree view
515c4410d5530f7ab0341778f3cb8075c77038c9  PR-23337                                              fix(naming_rule): polish list view
c312f66e96df2879037a73b966432064a50aeeba  PR-23272                                              fix(list_settings): don't count tags to total fields
0ff08e4e0f61957890eb0f2c03e587c67d40490d  PR-23229                                              build(python): restricted python goes up to 3.11 only
4f90d24f020157e5e4748c7cf9f3f02b430081e4  PR-23170                                              feat(db): boostrap only option if database and user is managed externally to bench
ae9ae40bad9d6a5665bcab435e8d7d16afda750b  PR-23168                                              refactor: give cli invocations of dbtools a dedicated interface
cfbe6da95943a6d8ced5648d402d4ca9a8418c06  PR-23167                                              chore: compose postgres conn settings in the same way as with mariadb
e91b72e8c482ca06c5012dbfe3c389a1fa536a63  PR-23144                                              fix(integration): preserver data json formatting on update
8f2db21f6481c1e2afbfc2318e40f2a3dd8c57b7  PR-23143  version-15-hotfix: backported via PR-23778  fix(float): input formatting
71a1712e35af922cee581ad25eb5919ddc1cee2f  PR-22797                                              fix(geolocation): modal and state flow
a14966c98094acf3fb9a854314ae16734375f908  PR-22595                                              fix: non-html notifications from files
a7171918ee5f764df0a1ebb28340771857ead563  PR-22592                                              feat(model): implement into-child table mapper
68d586e206655ff81f10f4184e678bcbcf8d8787  PR-22519                                              feat: import scss from node_modules in apps also
e57dc10631620330999831bcd3652211d7090431  PR-22511  version-15-hotfix: included                 fix: rss rendering of blog entries
086435c0041ad622e2170d741844b77b124dde48  PR-22462  version-15-hotfix: included                 feat: add upstream json argument to integrations make_request
091c6ca49f86817bee99c36c8bb2b44ab76a950c  PR-22441  version-15-hotfix: included                 fix: `flt` with trailing zeros
eee418a4c69148e76621f52dcf6ee9543199e334  PR-22330  version-15-hotfix: included                 feat: improve openid connect devx
9309f9743f2da00a772cbfe9bb5d34bdd99d7fb1  PR-22309  version-15-hotfix: included                 refactor!: monkey patching {imap,pop}lib is no more necessary since 4 years
2001bc278f34e7bb905ed0296978daf86a97ad4c  PR-22308  version-15-hotfix: included                 refactor!: merge get_site_url into get_url
2d73b8f314f5d4c40d5cd54aecbdf61cfc16edac  PR-22298  version-15-hotfix: included                 fix: align base template to socketio port default 9000
ed5da35b1a6a69b5386a5312cc20fbc1d0d7a6ae  PR-22277  version-15-hotfix: included                 fix: align  to socketio port default 9000
1a061d4590f5b7734db53ed736e30d2a40c2f359  PR-21975  version-15-hotfix: included                 fix: rely on pyenv to collect bench commands, however that may be set up
fddd3b24f7ab56bd9e2309cf651f2f02a164fa60  PR-21878  version-15-hotfix: included                 test: skip tests on RO app paths which require write access
1dcf1d1f4be7f741cab520ffc432b65df47cc68d  PR-21874  version-15-hotfix: included                 fix(test): use app path directly without looping over bench path
8f7a4f6697bd732a712cf582c2f321dc67c73629  PR-21873  version-15-hotfix: included                 refactor: make the app source accessor explicit
3471685eaa046b3db805984f425be13fd9022f92  PR-21872  version-15-hotfix: included                 feat(ops): add environment variable to set bench root / workdir
011e44656b16d2fe0b06525f0e1b1c7929c90b30  PR-21870  version-15-hotfix: included                 fix(test): ensure tests use proper url
7780670ae44c5359e6e404075d52e65600e960bd  PR-21797  version-15-hotfix: included                 build(deps): update node redis client to v4
cf8a603dac56b1cf3d93fcb015aebb60fdd26a5e  PR-21783  version-15-hotfix: included                 build(deps): update premailer 3.8.0 -> 3.10.0
6c509f6dd12796ff11e8e75b049fefe915f5fc3c  PR-21782  version-15-hotfix: included                 build(deps): update bleach 3.3.0 -> 6.0.0
a926e64ec9956e2f6b9a32c5e29d9c36d230d338  PR-21578  version-15-hotfix: included                 fix: procure db config from single authority
6b2bb9a2abcc12896982985e5f4853b513880570  PR-21577  version-15-hotfix: included                 fix: add env overrides for service orchestration
2ccab0d6251eab6ac5836ab4d3715a985a954d82  PR-21576  version-15-hotfix: included                 fix: checkpoint the supported schemes for connectivity
98d38e733390f1c62ad37fdc8776ac56cf1458a2  PR-21575  version-15-hotfix: included                 fix: gently log the port on which the realtime service listens
f5b93cc2a02a9298b7e025a8a107bd204609d95d  PR-21558  version-15-hotfix: included                 chore: urlparse already does the parsing for us, no need to diy
369b185c50e2f2b9e68b8e201715a70d73be1843  PR-21478  version-15-hotfix: included                 fix: don't break link list in middle of link in table multiselect
3549fe78eff092593623b10eafb1fdb0f934ed90  PR-21419  version-15-hotfix: included                 fix: field replace
e500dcde3be3144299100ecbd18f554bd6612c4e  PR-21418  version-15-hotfix: included                 fix: section fields_dict
5d074f3237ac24c09366c66e7ef31fc554e0b5a3  PR-21363  version-15-hotfix: included                 ci: fix patch workflow
4da6796b2db5c56d7b3ea586cf4cc38305bb422c  PR-21322  version-15-hotfix: included                 feat: add style & onEachFeature function for geojson in geolocation controller
23dd8e92482c24ef0351462249e3ffb464e01e3e  PR-21309  version-15-hotfix: included                 refactor: normalize redis invocation and unblock unix domain socket conn

Hope that may be useful to someā€¦

Unfortunately, it doesnā€™t (yet?) detect backportsā€¦ Edit: now it does, I updated the listing.

1 Like
4

Hey @blaggacao great work here! This thread should be blowing up, but I suppose that the world of Nix is so massive and different this paradigm shift is a lot to take in, especially for those of us that have been developing Frappe apps almost exclusively on Ubuntu.

Iā€™m especially interested in your OCI Container target, and curious how this may relate to the various paradigms that exist for containerizing Frappe.

Iā€™ve also followed your PR for unix domain sockets support. This feature was long overdue, IMO (and I hope it gets merged soon!)

Iā€™m super happy for the work youā€™re doing and excited to see how this will play out once these ideas gain more traction.

3 Likes
5

Thanks for the little encouragement! :smile:

I havenā€™t implemented it yet in Frappix (but I did in other projects), and I imagine something like:

let
  inherit (inputs) std;
  inherit (apps.pkgs) frappix;
in {
  web-entrypoint = std.lib.ops.mkOperable {
     package = frappix.frappe.pythonModule.buildEnv.override {extraLibs = [
       frappix.erpnext
       # other apps
     ];};
     runtimeScript = ''
       # todo figure out the propper entrypoint script
     '';
   };
  web = std.lib.ops.mkStandardOCI {
    name = "ghcr.io/myorg/myproject";
    operable = cell.web-entrypoint;
    meta.description = "My Frappe Project bundled OCI";
    config.Env = [
      # figure out which container-only environment to set
    ];
  };
  # and so on for mariadb, redis (which are only pinned and proxied from upstream)
  # and then also different workers, scheduler, socketio
}

This then would be built with your respective bespoke package set from the apps.pkgs.frappix namespace, whether they are plain upstream or heavily patched.

If you have a stable base image (pre apps / pre frappe but post dependencies) then the CI should also run a couple of minutes faster, because GHCR is known to be faster than cache restore on Github, and definitely significantly faster than the current apt get install.

1 Like