Export data from data import tool - timeout from large dataset

ERPNext
#1

I am trying to download my item master with about 20,000 items and it is failing with a 504 timeout even when i try to export it as csv. please help

#2

Do you still get a timeout when you download just one item?

#3

For such challanges, tool must have date range to download the bata between those dates.

It’ll help for those who has large databases

#4

@gurshish your point is valid. It will not export item master with such large data. Erpnext team needs to look into data export feature.

However u can use following workaround
Instead of exporting item master from import export tool you can use item report builder and pick columns to add all the relevent data u need and then export that report

1 Like
#5

that’s a good idea, thank you.

#6

how do you do this

#7

Pick a DocType that has no elements ie an empty list.

Now a create a single new instance then export it.

#8

that works smaller doctypes get exported properly its just that when there is a lot of data it times out

#9

So you already know what causes the timeout…

#10

yes i do the data import tool is unable to make large sheets

#11

Maybe try this

frappe@ubuntu:~/frappe-bench$ bench export-csv
Usage: bench export-csv [OPTIONS] DOCTYPE PATH

Error: Missing argument “doctype”.

Also try ‘bench worker’ to monitor progress?

1 Like
#12

i ran bench export-csv Item item.csv this worked for a while and then stopped but there was no output

#13

If you would condense and report this that would help here…