I agree with the thought of progressing towards PostgreSQL as the DB fo ERPNextr
i like to see PostgreSQL support in the future.
This might be relevant to the conversation:
1 Like
Postgres support is not updated anymore. If you need postgres you’ll need to send pr to support the latest lts.
Any database that’ll be added by copying postgres code will be exact maintenance overhead like postgres.
Every hacker in the community has a setup that was partly converted to oracle. It’s given up on after realising it is just copy-pasting code and renaming imports. It’ll only add code that’ll never be maintained in future.
No official app is made using postgres.
I disagree with this outlook and am disappointed by it. Oracle wants to charge people for MariaDB in the same sort of clawback of licenses that they’ve done with Java. MariaDB isn’t getting feature or improvements at anything close to the pace that Postgres is. Postgres is about ten years ahead at this point.
Many of these features won’t be used by the Frappe framework because of modularity and multitenancy concerns and general dismissiveness of databases as anything other than a “place for your data”, but would be a major boon to performance if appropriately used. Frappe and ERPNext has more to lose by sticking with MariaDB than by dropping it entirely and moving to Postgres wholesale.
I am prepared to provide examples.
3 Likes
I also disagree and am disappointed. But I am not surprised.
Postgres was discussed on these forums back in 2014, and again in 2018. In year 2021 an official GitHub Issue was created (which I guess should be marked Closed now). A few months ago, I concluded that if Postgres hadn’t happened by now, it probably wasn’t going to. Or at least not for many years.
I could also write volumes about why Postgres was/is the best direction. However, my opinions certainly won’t sway any maintainers’ minds. So instead, I think I’ll spare my hands and fingers a lot of pointless keystrokes.
@tmatteson Anticipating this outcome last year, I edited my own forks, and made them compatible with Postgres 17. Believe it or not, that’s actually scheduled to go into Production -tonight- in about 12 hours. (the timing of these past few replies is crazy). After my newly-cooked database has a chance to cool, if you want to know how it all turned out, challenges, etc? You know how to reach me.
5 Likes
I just expressed my observation. I can’t do much about it.
Contributor team and frappe developers agreed to add postgres support, it was assumed that the team will keep using frappe and keep supporting postgres. I don’t think that happened.
If at all someone takes frappe psql to lts, it will be a community contributor who is actively using postgres. That contributor will be expected to maintain the code as well. Who’s up for it?
I feel orm should be pluggable. Right now postgres code feels like it’s an after thought. If I install frappe that supports 25 db engines, pyproject.toml will install 24 dependencies for unused engines.
I don’t think there is any official app right now or even planned in future that’ll be based on postgres first. Everything is built for and is tested on mariadb.
Every official communication is pushing towards Frappecloud. Adding postgres to frappecloud will be another challenge. I guess if it doesn’t fit Frappecloud it doesn’t really make money (motivate)
“Frappe supports Postgres” is misleading to new comers in general.
I’m not asking for PR. Is your fork public? If someone needs they can refer?
2 Likes
We’re just expressing our opinions. And we can’t do much about it.
I feel orm should be pluggable. Right now postgres code feels like it’s an after thought. If I install frappe that supports 25 db engines, pyproject.toml will install 24 dependencies for unused engines.
I thought one of the primary reasons for adding PyPika was to support this exact case: multiple backend databases. If not, what was the point of PyPika?
As far as dependencies go, just make them optional. Plenty of other similar-sized Python applications do this. You all are smart: I’m sure you could figure out a way.
“Frappe supports Postgres” is misleading to new comers in general.
On this point, we are in full agreement.
I’m not asking for PR. Is your fork public? If someone needs they can refer?
PR’s are impractical for changes of this magnitude imo; so that’s good.
Yes, my forks of Frappe and ERPNext forks are always kept public.
Caveat 1 : I suggest not paying attention to them for at least a few weeks, until I finish the launch + bug fixes.
Caveat 2 : I only had 2-3 weeks to accomplish this. And once I move to Postgres, I’m never looking back. So I took plenty of shortcuts. Altering various code to make it Postgres-compatible, but not compatible with MariaDB anymore. Certainly the correct and proper thing would have been to replace them all with Pypika queries, or use frappe.db.multi_sql. But I had neither the time, nor motivation. So it’s not pretty. But it does work.
1 Like
Now I wonder what the big benefits are for you to take the trouble of maintaining a fork. 
I can’t remember a time in the past 6 years when I ran into database bottlenecks, even with a fairly primitive setup.
Raffael, I’ve been maintaining forks of Frappe and ERPNext for as long as I can remember. Probably all the way back to early 2017. There have been many, many benefits to doing so. A lot of which I’ve mentioned on the forums over the years. Without the ability to maintain forks and and solve my own problems, I would have abandoned ERPNext a long time ago.
Postgres is just the latest task or project that required altering the framework and ERP repositories directly.
I’m glad most people have no problems with MariaDB.
But for myself, it’s been the root cause of many headaches the past 4 years, across multiple clients and installations. The biggest challenges have been I/O locks that require restarting the service to undo. Poor query planning. Repeated occurrence of corrupt indexes. A completely unreliable Replication system. A tendency to deadlock. And although not a deal breaker, their physical backups are awkward and a pita.
And that’s not even mentioning all the benefits of Postgres over MariaDB in general.
Maybe Postgres will help me. Maybe it won’t. There’s only 1 way to find out.
This is what we keep saying is so great about open-source ERP, right? Take it, make it your own, do what you want with it.
The alternative is to sit here, and wait for other people to solve these problems for me. Not a good strategy.
9 Likes
I also would love to see Frappe adopt postgres. There have been some important incremental steps towards making that possible, such as the introduction of query builder, but it’s a massive, high-stakes undertaking. I can’t really imagine anyone putting forward the kind of time and resources involved anytime in the near future.
The mood lately also seems to be to prioritize stability over new features for ERPNext. That’s a positive choice, in my opinion, but it makes it even harder to justify devoting resources a massive rework like this.
1 Like
As a long time (~25 years) Postgres user I am glad that you managed to hack it together in a short time. 
I agree that moving away from frappe.db.sql and frappe.db.multi_sql to pypika queries would be the right way to move forward - not just for people using Postgres, but also for those who are using MariaDB as well.
1 Like
I am confused as why Frappe is going in the opposite direction and actively trying to add SQLite support. There’s no public ticket that describes why the requirements or use case for this would be. SQLite is single threaded but fast, so it should be able to support more than one concurrent user, but who knows how many. There are no load tests in the existing test suite.
1 Like
The intersection of businesses large enough to need an online/hosted ERP but small enough to be satisfied with a single threaded database is for practical purposes, zero.
Definitely, I can’t imagine wanting to run a full ERP on SQLite. But, as a framework, Frappe is quite useful for much more modest applications.
1 Like
Yes and I think that it could be a meaningful and adequate replacement for Redis and that licensing debacle, but still has less-than-ideal locking issues in that context.
General comment on influencing direction in open source projects (not limited to postgres)
Its been more than a decade since we have been running this project and we often see people complaining that they are unable to influence the direction of the project. Here is what they don’t get - influencing is a longer path. The goal should be to become a maintainer. They first have to make regular small contributions and help the existing maintainers. Then become part of the team that maintains. Be a good citizen, don’t break stuff, maitain the CI, help users. Then you get to have a say in the direction of the project
Yes you can say maintainers / Frappe should do this or that, but you don’t really know what every maintainer is doing. We have clearly said that it is really upto the maintainers to do what they think best. There are also several non-Frappe folks with maintainer rights - @snv, @revant_one @rmeyer
Then I see folks like @brian_pond @tmatteson who are very active, but choose not to help the maintainers and expect some kind of obligatory attention to their priorities. True, its open source and forking is always a good choice. Its sad though. Would have been great if they had chosen a different path.
3 Likes
Just a few corrections.
Well, I cannot speak for every person who has ever complained on these forums.
But I can speak for myself and @tmatteson. We definitely do “get it”. This is not the first time you’ve shared your requirements and thoughts about contributions. We are -definitely- aware of your positions on this matter, Rushabh. They have been crystal clear.
… and expect some kind of obligatory attention to their priorities
This is untrue. We certainly don’t -expect- anything from you or the maintainers. Believe me, if there’s one thing I’ve learned? It’s to not expect anything from the project and its stewards.
Certainly I have frequently expressed disagreement, disappointment, and dissent. However, I definitely do not expect my words to have any influence or impact. That is not one of my goals when writing them.
Its sad though. Would have been great if they had chosen a different path.
I do agree, this is very sad indeed. I wonder if they have reasons? 
3 Likes
Some of the problems with MariaDB actually stem from the pure-Python client we’ve been using.
With help from @ankush, I recently re-introduced mysqlclient. It is upto 4x faster in read queries.
We are expecting to make the switch in v16.
8 Likes