Upgrade v11 to v12 results in multiple errors

Hi all

After running bench update (Bench Start Error in Development - Error: No such commands watch serve worker schedule, frappe commands!? - #4 by trentmu) all the worker apps went back to working state.

So the only thing left is the ASCII problem.

According to the log it’s 0xd0 what’s upsetting python from working. According to the ASCII table, this is the character ð . The web.log shows

web.log
Traceback (most recent call last):
  File "/home/frappe/frappe-bench/apps/frappe/frappe/www/desk.py", line 21, in get_context
    boot = frappe.sessions.get()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/sessions.py", line 132, in get
    bootinfo = get_bootinfo()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/boot.py", line 54, in get_bootinfo
    load_translations(bootinfo)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/boot.py", line 188, in load_translations
    messages = frappe.get_lang_dict("boot")
  File "/home/frappe/frappe-bench/apps/frappe/frappe/__init__.py", line 90, in get_lang_dict
    return get_dict(fortype, name)
  File "/home/frappe/frappe-bench/apps/frappe/frappe/translate.py", line 114, in get_dict
    messages = get_messages_from_include_files()
  File "/home/frappe/frappe-bench/apps/frappe/frappe/translate.py", line 470, in get_messages_from_include_files
    messages.extend(get_messages_from_file(os.path.join(frappe.local.sites_path, file)))
  File "/home/frappe/frappe-bench/apps/frappe/frappe/translate.py", line 498, in get_messages_from_file
    message) for message in  extract_messages_from_code(sourcefile.read(), path.endswith(".py"))]
  File "/home/frappe/frappe-bench/env/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 1801299: ordinal not in range(128)

So I did a search for that character in all *.py files since I don’t know what python files are included for translation: grep -nr --include \*.py 'ð'

grep result
env/lib/python3.5/site-packages/IPython/core/latex_symbols.py:105:    "\\eth" : "ð",
env/lib/python3.5/site-packages/IPython/core/latex_symbols.py:106:    "\\dh" : "ð",
env/lib/python3.5/site-packages/IPython/utils/tests/test_pycolorize.py:42:    print("wěird ünicoðe")
env/lib/python3.5/site-packages/stdnum/is_/vsk.py:21:"""VSK number (Virðisaukaskattsnúmer, Icelandic VAT number).
env/lib/python3.5/site-packages/bs4/dammit.py:762:        0xf0 : b'\xc3\xb0',     # ð
env/lib/python3.5/site-packages/pip/_vendor/idna/uts46data.py:226:    (0xD0, 'M', u'ð'),
env/lib/python3.5/site-packages/pip/_vendor/idna/uts46data.py:1669:    (0x1D9E, 'M', u'ð'),
env/lib/python3.5/site-packages/idna/uts46data.py:226:    (0xD0, 'M', u'ð'),
env/lib/python3.5/site-packages/idna/uts46data.py:1669:    (0x1D9E, 'M', u'ð'),
env/lib/python3.5/site-packages/docutils/utils/math/tex2unichar.py:61:    'eth': '\xf0', # ð LATIN SMALL LETTER ETH
env/lib/python3.5/site-packages/docutils/utils/math/math2html.py:332:      '\\delta':'δ', '\\dh':'ð', '\\digamma':'ϝ', '\\epsilon':'ϵ',
env/lib/python3.5/site-packages/docutils/utils/math/math2html.py:333:      '\\eta':'η', '\\eth':'ð', '\\gamma':'γ', '\\i':'ı',

I’ll further investigate which files are included in translate.py.

I’ll keep you updated.

Best regards
Simon