I somehow thought custom fields were linked to a particular module or an app, but I can’t find any fields in the table ‘tabCustom Field’ that would make this link.
So I guess using filters to specify each one of my own custom fields is the way to go.