hello,
how can I improve database performance for erpnext, when I do import data from csv for 20000 items it take more time 20 min , I have total memory available 9G , I try to edit mariadb conf file and make innodb_buffer_pool_size to 7G and innodb_buffer_pool_chunk_size to 3G but no change, import take more time the same thing for postgres , any help please how can I import data faster than this
Data import is a largely a sequential process, so it’s unlikely that it can be improved much by increasing buffer pool size. Bufferpool roughly acts as “cache” for storing frequently accessed parts of DB in memory.
If I had to guess the bottleneck isn’t DB at all. Try profiling the code with python’s inbuilt CProfile (using ipython %prun magic command)
(unrelated but maybe useful self-promotion: I gave a talk on performance profiling last year - How Frappe v14 is blAziNgLy fast - Ankush Menat | ERPNext Conference 2022 - YouTube )
Hi,
This thread may have some tools that will help: https://discuss.frappe.io/t/erpnext-runs-very-slow/68086/5