I’m new to ERPNext and I’m liking it at the moment. I have also contributed a couple of PRs to the Frappe repo.
Frappe makes use of mysql-python at the moment but the development of mysql-python seems to have stalled since 2014. The maintainer of the repo even mentioned that the repo needs a new maintainer (view comment on github ). Hence I think it makes sense for ERPNext to move forward and choose another adapter.
I believe we need to consider python 3 support as I can see there are already efforts by some developers (including me) to make the Frappe ecosystem python 3 compliant. We also need to make sure it is actively being developed and mature.
What does the community think about this? If we should change the database adapter, can we have suggestions?
tried replacing pymysql in database.py resulted in error.
We can always try out on forks. If it works there we’ll send a PR.
File "/home/revant/frappe-bench/apps/frappe/frappe/middlewares.py", line 15, in __call__
return super(StaticDataMiddleware, self).__call__(environ, start_response)
File "/home/revant/frappe-bench/env/lib/python2.7/site-packages/werkzeug/wsgi.py", line 599, in __call__
return self.app(environ, start_response)
File "/home/revant/frappe-bench/env/lib/python2.7/site-packages/werkzeug/wsgi.py", line 599, in __call__
return self.app(environ, start_response)
File "/home/revant/frappe-bench/env/lib/python2.7/site-packages/werkzeug/local.py", line 228, in application
return ClosingIterator(app(environ, start_response), self.cleanup)
File "/home/revant/frappe-bench/env/lib/python2.7/site-packages/werkzeug/wrappers.py", line 291, in application
return f(*args[:-2] + (request,))(*args[-2:])
File "/home/revant/frappe-bench/apps/frappe/frappe/app.py", line 95, in application
frappe.destroy()
File "/home/revant/frappe-bench/apps/frappe/frappe/__init__.py", line 218, in destroy
db.close()
File "/home/revant/frappe-bench/apps/frappe/frappe/database.py", line 849, in close
self._cursor.close()
We did replace MySQLDb with pymysql when we were experimenting with gevent. It takes a few mins to migrate the current connection settings. It worked alright, but we did not do extensive tests.
I too like pymysql. Lets move!
Thanks @tundebabzy and @revant_one for taking the lead for Python 3, lets keep fixing things part by part!
@revant_one, I changed all MySQLdb to pymysql and quite alright there were errors. After investigating, I discovered that PyMySQL does not use the _mysql level APIs that mysqldb used as its a pure python implementation.
Presently, the changes work on my computer but fails on Travis
Request Error
Traceback (most recent call last):
File "/home/revant/frappe-bench/apps/frappe/frappe/app.py", line 52, in application
init_request(request)
File "/home/revant/frappe-bench/apps/frappe/frappe/app.py", line 114, in init_request
frappe.local.http_request = frappe.auth.HTTPRequest()
File "/home/revant/frappe-bench/apps/frappe/frappe/auth.py", line 39, in __init__
self.set_lang()
File "/home/revant/frappe-bench/apps/frappe/frappe/auth.py", line 82, in set_lang
frappe.local.lang = guess_language()
File "/home/revant/frappe-bench/apps/frappe/frappe/translate.py", line 27, in guess_language
lang_list = get_all_languages() or []
File "/home/revant/frappe-bench/apps/frappe/frappe/translate.py", line 82, in get_all_languages
return frappe.cache().get_value('languages', _get)
File "/home/revant/frappe-bench/apps/frappe/frappe/utils/redis_wrapper.py", line 70, in get_value
val = generator()
File "/home/revant/frappe-bench/apps/frappe/frappe/translate.py", line 81, in _get
return frappe.db.sql_list('select name from tabLanguage')
File "/home/revant/frappe-bench/apps/frappe/frappe/database.py", line 211, in sql_list
return [r[0] for r in self.sql(query, values, debug=debug)]
File "/home/revant/frappe-bench/apps/frappe/frappe/database.py", line 154, in sql
self._cursor.execute(query)
File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 166, in execute
result = self._query(query)
File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/cursors.py", line 322, in _query
conn.query(q)
File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 856, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1057, in _read_query_result
result.read()
File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1340, in read
first_packet = self.connection._read_packet()
File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 1014, in _read_packet
packet.check_error()
File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error
err.raise_mysql_exception(self._data)
File "/home/revant/frappe-bench/env/local/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception
raise errorclass(errno, errval)
I dont known how complex is erpnext in this abstraction, but could be a good moment to implement a ‘standard library’ that provide support to other databases, for example sqlalchemy without orm only core (it uses pymysql among others …)
I understand that this could be a lot of work but i think that it was be a great improvement for future of the erp.
hello @meigallodixital, I haven’t gotten a go ahead from any of the core developers but I do have this in mind as something to contribute (with the help of others). However, I think it will be better to first of all finish up python 3 compatibility before adopting the sqlalchemy layer. Sqlalchemy doesn’t come up database adapters.
The question is that for example in this company (and I know more cases), the
support of python 3 and postgresql is a limitation when deciding by
erpnext. But we understand that it was not the initial focus of the project and the enormous work that it would take.
So I ran bench update and reinstalled my bench because it was acting up. I tried to restore my database and it gave me AttributeError instead. I’ve submitted an issue on github.com but in the meantime, I’m going to revert the commit that seems to have brought in the problem.
I suspect pymysql is not able to connect to the database.