High storage consumption in ERPNext, is this normal?

ERPNext
1

ERPnext v13

Hi, I have my ERPnext installation on a server on AWS, at the beginning I used a 10gb disk and on it I was able to upload 500 products in the warehouse.

Then the storage full sound went off and I expanded the disk to 20gb, so I was able to save 10 of product transfers between warehouses.

Now, I have expanded the storage 2 more times, up to 40gb, and yesterday it was full again. I consider that I don’t have much information uploaded here, the 500 products don’t even have an image.

how can a database with 500 products, 10 workers and 10 transfers occupy 40gb? is this normal? does anyone know what could be occupying so much space?

1 Like
2

Many of ERPNext’s logging and auditing tables will grow forever …
…unless you write your own maintenance scripts, to delete unwanted records.

Below is a SQL query that will list your tables (by database) from largest to smallest.
This should help identify potential SQL growth problems.

SELECT
  TABLE_SCHEMA,
  TABLE_NAME AS `Table`,
  ROUND((DATA_LENGTH + INDEX_LENGTH) / 1024 / 1024) AS `Size (MB)`
FROM
  information_schema.TABLES
ORDER BY
	TABLE_SCHEMA,
  	(DATA_LENGTH + INDEX_LENGTH)
DESC;
2 Likes
3

High and unexpected disk usage can be caused by error logs. See if the error log list is large. If you are able to log into the server console, the linux du command can be used to analyze disk usage.

4

Scary proposal. So now, in order actually use the ERPNext app, we have to figure out how to stop the growth of over active tables in the database just so we can do something beyond a minimal test size database?!?

This does not sound promising for the new version releases here.

BKM

5

You know I don’t usually pull my punches on the forums.

But to be fair to ERPNext…plenty of other ERP software has placed this responsibility in the user’s hands too.

It’s rather difficult to get a universal consensus on:

  • Which tables are okay to Delete, vs. Archive, vs. Never Touch.
  • How old records should be, before it’s okay to delete them.
  • Be safe, and not break referential integrity.

With that said, certainly ERPNext could be doing a better job.

And though I often scoff at the “just submit a PR” copouts…
…this may actually be a rather good candidate for just such a thing. Some kind of “Data Cleaning” module.

The risk is adding such a tool. And then a few ERPNext users take it too far, destroy half their database, and point blame at the Data Cleaning tool.

Which might explain why no brave soul has invented it yet. :wink:

3 Likes
6

There is a Log Settings page that allows frequency of log cleanup to be set. The Linux logrotate feature may also be helpful.

The more important question is what is filling the disk? It may not be frappe/ERPNext, rather something in the operating environment.

6 Likes
7

OMG!, thanks!, i use du -h folder by folder and i found the logs folder with 41gb size

i wanna use logrotate to clean all.

Thank you so much

1 Like
8

instead of doing du -h for each folder, I recommend to install ncdu then run ncdu on root folder.

2 Likes
9

cool! thanks!