(1)
Documentation:
https://frappeframework.com/docs/user/en/guides/data/import-large-csv-file
doesn’t match tool:
$ bench import-csv
Usage: bench [OPTIONS] COMMAND [ARGS]...
Try 'bench --help' for help.
Error: No such command 'import-csv'.
(2)
Also, this page seems to be completely outdated, as the relevant doctypes cannot be found with awesomebar:
https://frappeframework.com/docs/user/en/guides/data/using-data-migration-tool
(3)
The reciprocal command exists, though:
$ bench export-csv Airport test.csv
but seems to silently fail.
$ find . -name "test.csv"
Well, no, in fact,:
$ find ~/frappe/training -name "test.csv"
.../frappe/training/sites/test.csv
Compare with the help:
$ bench export-csv 2 ⨯
Usage: bench export-csv [OPTIONS] DOCTYPE PATH
Try 'bench export-csv --help' for help.
Error: Missing argument 'DOCTYPE'.
So the export command expects a path, but doesn’t honor it and puts the file elsewhere, without telling the user where it can be found.
(4)
The exported file looks like this:
"Data Import Template"
"Table:","Airport"
""
""
"Notes:"
"Please do not change the template headings."
"First data column must be blank."
"If you are uploading new records, leave the ""name"" (ID) column blank."
"If you are uploading new records, ""Naming Series"" becomes mandatory, if present."
"Only mandatory fields are necessary for new records. You can delete non-mandatory columns if you wish."
"For updating, you can update only selective columns."
"You can only upload upto 5000 records in one go. (may be less in some cases)"
""
"DocType:","Airport","","",""
"Column Labels:","ID","Code","City","Country"
"Column Name:","name","code","city","country"
"Mandatory:","Yes","Yes","Yes","Yes"
"Type:","Data","Data","Data","Data"
"Info:","","","",""
"Start entering data below this line"
"","""Dortmund Airport 21""","DTM","Dortmund","Deutschland"
"","""Flughafen Köln/Bonn „Konrad Adenauer“""","CGN","Hamburg","Deutschland"
"","""Flughafen München „Franz Josef Strauß“""","MUC","München","Deutschland"
"","""Frankfurt Airport""","FRA","Frankfurt","Deutschland"
"","""Hamburg Airport Helmut Schmidt""","HAM","Hamburg","Deutschland"
Note: This looks similar (or is identical) to the template which can be exported for preparing an import with it (for adding or updating docs).
The problem is that this file explicitly states
"Please do not change the template headings."
But with all these headings it cannot be imported.
Most of them MUST be stripped in order for any import to work.
So this note is directly misleading.
Note that it also doesn’t state a word to tell which headings to keep and which ones to strip.
Can imports be imported in a more streamlined if the DocType is left, so it doesn’t have to be entered? So far I don’t know.
The solution for importing using this template system seems to be to keep only the one line starting with “Column Name:” or maybe also “Column Labels:”.
Only then the import dialogue (Tools > Import Data, or via “Data Import” in awesomebar) allows to check the mapping and then the import to proceed.
Fine, finally!
All the other lines produce “Errors and Warnings” but without telling the user which one is which, stating line numbers (and not the content, which could also be displayed, if in some sanitized form).
So with trial and error one finally arrives at success and learns to ignore false statements in the template produced by the framework.
Alternative: The template could state correct and more exhaustive instructions in the first place.
I almost switched to using the API, but the template system looked useful in theory, if not easily in first practice. If you have some small tables from a spreadsheet it could be quick to use. The Mapping system also looks nice, it seems to enable using any spreadsheet, attach a header with the columns of the table where the column’s fields are supposed to land, and that could make for a quick and easy use.
Nice!
If only it were documented in template-producing code and doc.
Also, the info about the Label vs. names and mandatoryness etc. looks useful. Just strip it before importing.
Or why does the framework not put “#” signs in the template and ignore all these documentation lines, except the column names of course, when importing? Then the system would work easily and more clearly.
Maybe it worked like this in the past and somehow functionality got lost? Maybe some tests could be useful? Also, maybe importing a fresh export should “just work” (at least for update mode).
BTW, also the line “enter your date below this line” seems to suggest that you just put your date there and you are fine. But no, that line has to be stripped, which is counter-intuitive alltogether.
So, this is my secret knowledge acquired today, which hopefully helps others equally stumped when trying the import function.
This was with a fully updated (5 min before trying) developer frappe framework.