Bulk img file direct import (git>/Files)

Upon finding the 10mb max file upload and people having issues trying to upload anything more than 50mb after manually changing the limit, I saw somewhere to just do it via ftp/etc through /Files. Fair enough.

I uploaded ~3000 images (~1gb) of files via a git repo to site/public/files/imgrepo/. All good.

I cannot get file manager to recognize that they are there.

I have tried “update” through Data Import, I have tried “Insert New Records” through Data Import, I have tried moving them out of the /files/imgrepo directory and just dropping them all in /files (and changing the fileURL’s in the Import file). I can’t think of anything else. Below is a sample row from the Data Import file along with it’s associated Error File

*note- bool’s have been tried as both txt and int, and I just retried with filesize as int rather than txt after taking this grab. Error code below is from this latest retry.

Traceback (most recent call last):
  File "/home/frappe/bench/apps/frappe/frappe/core/doctype/data_import/importer.py", line 403, in upload
  File "/home/frappe/bench/apps/frappe/frappe/model/document.py", line 222, in insert
  File "/home/frappe/bench/apps/frappe/frappe/model/document.py", line 876, in run_before_save_methods
  File "/home/frappe/bench/apps/frappe/frappe/model/document.py", line 772, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "/home/frappe/bench/apps/frappe/frappe/model/document.py", line 1048, in composer
    return composed(self, method, *args, **kwargs)
  File "/home/frappe/bench/apps/frappe/frappe/model/document.py", line 1031, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "/home/frappe/bench/apps/frappe/frappe/model/document.py", line 766, in <lambda>
    fn = lambda self, *args, **kwargs: getattr(self, method)(*args, **kwargs)
  File "/home/frappe/bench/apps/frappe/frappe/core/doctype/file/file.py", line 70, in validate
  File "/home/frappe/bench/apps/frappe/frappe/core/doctype/file/file.py", line 143, in validate_folder
    frappe.throw(_("Folder is mandatory"))
  File "/home/frappe/bench/apps/frappe/frappe/__init__.py", line 353, in throw
    msgprint(msg, raise_exception=exc, title=title, indicator='red')
  File "/home/frappe/bench/apps/frappe/frappe/__init__.py", line 339, in msgprint
  File "/home/frappe/bench/apps/frappe/frappe/__init__.py", line 312, in _raise_exception
    raise raise_exception(msg)
frappe.exceptions.ValidationError: Folder is mandatory

Having checked all of the ‘Suggested Topics’ to make sure I didn’t miss something easy…

There is that, which sounds like exactly what I need, but specifically makes use of the ‘image’ field.

From what I am gathering, this is implying simply to put the image path in the (ie, Item) import image fields and it will work? This, of course, basically leaves File Manager completely out of it, which I am totally fine with.

If this isn’t correct, where do I need to add an ‘image’ field to the File Data Import template?

For anyone else that finds this>

Yes, the bulk import images post was pretty much it… with some caveats.

Once files are added to the backend via FTP/git (recommended for easy change-syncing)/etc,

  • Open an item
  • Go to Customize
  • Find ‘image’
  • Open ‘image’ field details, uncheck ‘hidden’
  • Save and go to Data Import

Make sure to ‘reload’, and start a new import for ‘Item List’. Select ‘Update’ and download a template (CSV seems to work best). In ‘image’ field for item, put your file path (**/files/**path/to/file) with the image name and extension. As an example, for my item ‘800100’, my ‘image’ field contains ‘/files/imgrepo/orig/800100.png’