Custom App Installation breaks on Fixture Import due to wrong Order of Imports

Hey everybody :slightly_smiling_face:

I have a weird issue regarding my custom app and fixtures and I hope someone might be able to help me.

The Setting

Reading through the documentation and a lot of posts in this forum, I came to the conclusion that create a Custom App is the way to go for me. The basics are quite simple:

  • there are a couple of Custom Field I need to add to some DocTypes belonging to the Stock and Buying Module of ERPNext
  • I need to create a couple of Suppliers, Item Groups, Item Variants and Items
  • I want to make those changes portable and need to ability to install them in testing- and production-environments

So, working with bench export-fixtures, I found out I probably need to filter my fixtures inside hooks.py. Because if I don’t do this, my exported json files contain a lot of stuff from other apps and module. Ok, here is the important part of my hooks.py

fixtures = [
    "Address",
    {
        "doctype": "Supplier Group",
        "filters": [["name", "in", [
            "Primary Processor",
        ]]]
    },
    "Supplier",
    {
        "doctype": "Item Group",
        "filters": [["name", "in", [
            "Packaging",
            "Plastics"
        ]]]
    },
    {
        "doctype": "Item Attribute",
        "filters": [["name", "in", [
            "Material Form",
            "Plastic Type"
        ]]]
    },
    "Item",
    {
        "doctype": "Custom Field",
        "filters": {
            "module": ["=", "Bestandsmeldung EBH"]
        }
    },
]

So far so good. Executing bench export-fixtures created a couple of json files inside the fixtures subfolder and they look good.

The Problem

However, when I try to install the app in another environment, I get strange error:

builtins.AttributeError: 'NoneType' object has no attribute 'lft'

So, what I have found out is, that his happens here:

File "/home/frappe/frappe-bench/apps/frappe/frappe/model/document.py", line 912, in fn
    return method_object(*args, **kwargs)
      self = <Item: MAT>
      args = ()
      kwargs = {}
      method_object = <bound method Item.on_update of <Item: MAT>>
      method = 'on_update'
  File "/home/frappe/frappe-bench/apps/erpnext/erpnext/stock/doctype/item/item.py", line 124, in on_update
    invalidate_cache_for_item(self)
      self = <Item: MAT>
  File "/home/frappe/frappe-bench/apps/erpnext/erpnext/stock/doctype/item/item.py", line 1134, in invalidate_cache_for_item
    invalidate_cache_for(doc, doc.item_group)
      doc = <Item: MAT>
  File "/home/frappe/frappe-bench/apps/erpnext/erpnext/setup/doctype/item_group/item_group.py", line 203, in invalidate_cache_for
    for d in get_parent_item_groups(item_group):
      doc = <Item: MAT>
      item_group = 'Plastics'
  File "/home/frappe/frappe-bench/apps/erpnext/erpnext/setup/doctype/item_group/item_group.py", line 192, in get_parent_item_groups
    (item_group.lft, item_group.rgt),
      item_group_name = 'Plastics'
      from_item = False
      settings = <ECommerceSettings: E Commerce Settings>
      base_nav_page = {'name': 'All Products', 'route': '/all-products'}
      base_parents = [{'name': 'Home', 'route': '/'}, {'name': 'All Products', 'route': '/all-products'}]
      item_group = None
builtins.AttributeError: 'NoneType' object has no attribute 'lft'

Apparently, bench is importing my Item Template MAT and while doing this, it tries to “invalidate a cache”. For some reason, they need to query the Item Group of my Item MAT. But, its Item Group Plastics hasn’t been imported yet. This import would be part of importing the file: item_group.json.

So, the installation of my app broke, but somehow, if I try it again, bench tells me the app has been installed already :woman_shrugging:

Subsequently executing bench migrate created the same error.

Workaround

But, what I was able to find out is, that things work out, if I manually manipulate the “order of imports” by moving json files out and in the fixtures folder.

The Big Question

So, my question would be, is it possible to enable bench to export and import everything in the correct order to prevent such errors?

Could you try placing your item_group fixture after the item fixture in the hooks file and try again?

Hey @Void_Moon , thank you for your feedback.

I did change the order of entries in hooks.py:

fixtures = [
    "Address",
    {
        "doctype": "Supplier Group",
        "filters": [["name", "in", [
            "Primary Processor",
        ]]]
    },
    "Supplier",
    {
        "doctype": "Item Attribute",
        "filters": [["name", "in", [
            "Material",
            "Type"
        ]]]
    },
    "Item",
    {
        "doctype": "Item Group",
        "filters": [["name", "in", [
            "Packaging",
            "Plastics"
        ]]]
    },
    {
        "doctype": "Custom Field",
        "filters": {
            "module": ["=", "EBH"]
        }
    },
]

When executing bench export-fixtures I noticed, nothing changed in the exported json files. But I also did not know if I should have expected things to change.

However, things also did not change during app installation. I still get the same error:

  File "apps/erpnext/erpnext/stock/doctype/item/item.py", line 124, in on_update
    invalidate_cache_for_item(self)
  File "apps/erpnext/erpnext/stock/doctype/item/item.py", line 1134, in invalidate_cache_for_item
    invalidate_cache_for(doc, doc.item_group)
  File "apps/erpnext/erpnext/setup/doctype/item_group/item_group.py", line 203, in invalidate_cache_for
    for d in get_parent_item_groups(item_group):
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "apps/erpnext/erpnext/setup/doctype/item_group/item_group.py", line 192, in get_parent_item_groups
    (item_group.lft, item_group.rgt),
     ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'lft'

However, the execution path to the error does go through invalidate_cache_for(doc, doc.item_group) anymore.

Does anybody have another idea maybe? :thinking:

While researching my issue I have found a lot of incidents where people all ofer over had similar issues.

There even was a PR merged on github introducing a deterministic import mechanism for fixtures:

Unfortunately, this fix hasn’t made it into v14 oder even v15 of the frappe framework yet :confounded:

After talking to the person who provided a fix and validating together, that this code never made it into v14 or v15. I opened a new issue on GitHub: