I’m new to erpnext and frappe, have started using it recently. I’m actually trying to import a csv file i.e item into erpnext, and the command I’m trying to use is bench import-csv . As I’m importing aroung 30k records, I found that importing it through bench import-csv command should be possible as it is used to do a bulk import. But, I’ve encountered a problem here which is, only 3001 records are getting added in the db even though the logs says that the document has been saved successfully.
did you know that ERPNext actually provides an import tool in it’s interface? You can try uploading your csv there and then start importing. It has to be of a certain format though. The interface will also notify you of any false entries that can’t be imported.
Yes, I’ve seen that but I’ve read that there is a limitation that only 5000 records can be uploaded at once through UI. To import a large CSV there’s a documentation which says to use bench import-csv. Below is the link to it- https://frappe.io/docs/user/en/guides/data/import-large-csv-file
Oh yeah, you’re right about that. I did import more than 5000 records before, so I forgot about the limitation.
Perhaps you can upload your csv here? I can’t really point my finger on what could be wrong with the given info
Sorry for coming back late, was busy with other things. I will not be able to send the original data which I’m trying to import, but I would be able to give you csv file with dummy data which you can give a try.
The other thing I started noticing is that the dummy csv file which I created is getting imported successfully which has 10k records. The only difference in this file is that I’ve changed the item-code to a numeric value. When I added alphanumeric values to the item-code, I wasn’t able to import the data completely i.e only 3k records are getting imported.
I tried to attach the files but couldn’t, please download the files using the below link-
So, I’ve played around with the files a bit. I tried importing parts only, and the only error I’ve gotten so far was one where the item_code of one item is identical to another item’s item_code.
I assume that some ID’s should be for example 000021 as some have no zeros. However, if the item_code section isn’t properly formatted, it will omit all zeros which may cause duplicate item codes.
See if this helps. However there is probably something else, considering I got only 1 error, while you couldn’t import a majority of them.
Thank you for pointing on the right direction. Yes there was a problem with duplicate item-code being present in the csv file. There was one more problem i.e some item-codes were having the same name as the item-group.
I would really expect the import to stop and throw an detailed error when these kind of scenarios happen. Because it’s a big headache to find out what the actual problem is when you are doing a very big data import (in my case I was doing 40K records import). Just like how you do an import through the UI where if there is an error it makes the import as failure and then shows you a error detail.
But anyways I will raise this issue in the ERPNext GitHub portal. Thank you all for the help.
