Seems like a unicode issues… we believe we had fix this in most
places… surprised that it is still there
did you setup mysql (my.cnf) to utf-8 encoding… (i think default
mysql 5 is still ansi)
Rushabh
On Sep 22, 2011, at 5:08 PM, Ali wrote:
I need to store some information such as customer name another
language. However, saving fails.
Also, e-mail sending and even receiving doesn’t support other
characters.
Thoughts?
–
You received this message because you are subscribed to the Google
Groups “Web Notes Framework Forum” group.
To post to this group, send email to wn...@googlegroups.com.
To unsubscribe from this group, send email to wnframework +un...@googlegroups.com.
For more options, visit this group athttp://groups.google.com/
group/wnframework?hl=en.
ERPNext - World’s most affordable ERP - (Rs 299 / $7 per user per month)
I’ve installed the first version of installer (which has an old version of framework I guess) and all tables in database has “latin1” charset. So system does not allow to work with unicode symbols. After entering and saving of document they transforming to “???..”. How to fix it? Do I need to alter all tables manually for this? Will this be fixed in next install friendly releases?
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
Its best if you install the latest update - all tables are ut8 in there - also the code is much more utf8 friendly. (see install instructions on this list a few days ago)
You can also run the patch vim erpnext/patches/jan_mar_2012/convert_tables_to_utf8.py
also make sure your settings in my.cnf default to utf8
best,
Rushabh
----------------------------------------------------- ERPNext - Business Operating System
I've installed the first version of installer (which has an old version of framework I guess) and all tables in database has "latin1" charset. So system does not allow to work with unicode symbols. After entering and saving of document they transforming to "???????...". How to fix it? Do I need to alter all tables manually for this? Will this be fixed in next install friendly releases?
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
I’ve checked latest version but there is also “DEFAULT CHARSET=latin1”
in “CREATE TABLE” statement for all tables in master.sql. Is this
normal? Or somewhere must be setting with higher priority? Also I’ve
checked my.cnf - there is no settings about charset currently.
Its best if you install the latest update - all tables are ut8 in there - also the code is much more utf8 friendly. (see install instructions on this list a few days ago)
I’ve checked latest version but there is also “DEFAULT CHARSET=latin1”
in “CREATE TABLE” statement for all tables in master.sql. Is this
normal? Or somewhere must be setting with higher priority? Also I’ve
checked my.cnf - there is no settings about charset currently.
Its best if you install the latest update - all tables are ut8 in there - also the code is much more utf8 friendly. (see install instructions on this list a few days ago)
I have another issue with language - I have items in stock with non ascii symbols (cyrillic) in Item Code, when I use Stock Reconciliation I get an error: “UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xd0 in position 0: ordinal not in range(128)” which is rased by stock_reconciliation.py in row 63 (self.data.append([d.encode(“ascii”) for d in s])). I’ve tried to omit this decoding but got another error: “File “erpnext/stock/doctype/warehouse/warehouse.py”, line 42, in get_bin
bin = sql(“select name from tabBin where item_code = ‘%s’ and warehouse = ‘%s’” % (item_code, self.doc.name))
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xd0 in position 43: ordinal not in range(128)”. Is it possible to use your system with non ascii characters?
On Thursday, March 15, 2012 9:49:34 PM UTC+4, rushabh wrote:
Are you sure you are looking at the latest master from GitHub? Let me check this tomorrow.
As a quick fix you can just do a mass search and replace in the SQL file and it should work.
Sent from mobile
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
Hi Anton, it works for the most part. There may be some places that might be using some old encoding code. Will have to look at it next week as I am travelling.
Make sure the file you are uploading is encoded as utt8. If you are on windows you might have to do it via notepad. And remove all the encoding code.
I have another issue with language - I have items in stock with non ascii symbols (cyrillic) in Item Code, when I use Stock Reconciliation I get an error: "UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)" which is rased by stock_reconciliation.py in row 63 (self.data.append([d.encode("ascii") for d in s])). I've tried to omit this decoding but got another error: "File "erpnext/stock/doctype/warehouse/warehouse.py", line 42, in get_bin
bin = sql("select name from tabBin where item_code = '%s' and warehouse = '%s'" % (item_code, self.doc.name))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 43: ordinal not in range(128)". Is it possible to use your system with non ascii characters?
On Thursday, March 15, 2012 9:49:34 PM UTC+4, rushabh wrote:
Are you sure you are looking at the latest master from GitHub? Let me check this tomorrow.
As a quick fix you can just do a mass search and replace in the SQL file and it should work.
Sent from mobile
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
Rushabh, I’ve checked - csv file is in utf8. I also tried to change encode(‘ascii’) to encode(‘cp1251’) but no luck - it somehow uses ascii all the same according to error message: " File “/usr/lib/python2.7/encodings/cp1251.py”, line 12, in encode return codecs.charmap_encode(input,errors,encoding_table) UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xd0 in position 0: ordinal not in range(128) "
On Saturday, March 24, 2012 11:33:34 AM UTC+4, rushabh wrote:
Hi Anton, it works for the most part. There may be some places that might be using some old encoding code. Will have to look at it next week as I am travelling.
Make sure the file you are uploading is encoded as utt8. If you are on windows you might have to do it via notepad. And remove all the encoding code.
Sent from mobile
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
Rushabh, I’ve checked - csv file is in utf8. But there is no BOM, should it be there?. I also tried to change
encode(‘ascii’) to encode(‘utf-8’) but no luck - it somehow uses ascii
all the same according to error message: " File “erpnext/stock/doctype/stock_reconciliation/stock_reconciliation.py”, line 63, in convert_into_list
self.data.append([d.encode(“utf-8”) for d in s])
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xd0 in position 0: ordinal not in range(128) "
On Saturday, March 24, 2012 11:33:34 AM UTC+4, rushabh wrote:
Hi Anton, it works for the most part. There may be some places that might be using some old encoding code. Will have to look at it next week as I am travelling.
Make sure the file you are uploading is encoded as utt8. If you are on windows you might have to do it via notepad. And remove all the encoding code.
Sent from mobile
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
The encode function converts a Unicode string to an ASCII byte string by encoding special characters based on a character set, which is utf8 in this case.
Recently we faced an issue in import because there was a hyphen like character which was not getting encoded properly while saving the csv file as utf8. We were using OpenOffice. You will need to find which particular cell data is causing the decoding issue. There might be a character which might be causing the issue.
I would suggest you to try importing only one record and see if you are getting the same error.
Rushabh,
I’ve checked - csv file is in utf8. But there is no BOM, should it be
there?. I also tried to change encode(‘ascii’) to encode(‘utf-8’) but no
luck - it somehow uses ascii all the same according to error message:
" File
“erpnext/stock/doctype/stock_reconciliation/stock_reconciliation.py”, line
63, in convert_into_list self.data.append([d.encode(“utf-8”) for d in s])
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xd0 in position 0:
ordinal not in range(128)
"
On Saturday, March 24, 2012 11:33:34 AM UTC+4, rushabh wrote:
Hi Anton, it works for the most part. There may be some places that might
be using some old encoding code. Will have to look at it next week as I am
travelling.
Make sure the file you are uploading is encoded as utt8. If you are on
windows you might have to do it via notepad. And remove all the encoding
code.
Hi Anton it's difficult to say unless I have a look at the code. If every string is encoded UTF8 then there should be no issues and all forced .encode() methods must be removed.
Rushabh, I've checked - csv file is in utf8. But there is no BOM, should it be there?. I also tried to change
encode('ascii') to encode('utf-8') but no luck - it somehow uses ascii
all the same according to error message: " File "erpnext/stock/doctype/stock_reconciliation/stock_reconciliation.py", line 63, in convert_into_list
self.data.append([d.encode("utf-8") for d in s])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128) "
On Saturday, March 24, 2012 11:33:34 AM UTC+4, rushabh wrote:
Hi Anton, it works for the most part. There may be some places that might be using some old encoding code. Will have to look at it next week as I am travelling.
Make sure the file you are uploading is encoded as utt8. If you are on windows you might have to do it via notepad. And remove all the encoding code.
Sent from mobile
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
Anand, It is russian letter causing error. Cell data is “П001”. That is the correct Item Code stored in erpnext database. CSV file have only one test string, please find it attached.
On Saturday, March 24, 2012 7:00:45 PM UTC+4, Anand Doshi wrote:
Hi Anton,
The encode function converts a Unicode string to an ASCII byte string by encoding special characters based on a character set, which is utf8 in this case.
Recently we faced an issue in import because there was a hyphen like character which was not getting encoded properly while saving the csv file as utf8. We were using OpenOffice. You will need to find which particular cell data is causing the decoding issue. There might be a character which might be causing the issue.
I would suggest you to try importing only one record and see if you are getting the same error.
Thanks Anand
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
Anand,
It is russian letter causing error. Cell data is “П001”. That is the
correct Item Code stored in erpnext database. CSV file have only one test
string, please find it attached.
On Saturday, March 24, 2012 7:00:45 PM UTC+4, Anand Doshi wrote:
Hi Anton,
The encode function converts a Unicode string to an ASCII byte string by
encoding special characters based on a character set, which is utf8 in this
case.
Recently we faced an issue in import because there was a hyphen like
character which was not getting encoded properly while saving the csv file
as utf8. We were using OpenOffice. You will need to find which particular
cell data is causing the decoding issue. There might be a character which
might be causing the issue.
I would suggest you to try importing only one record and see if you are
getting the same error.
>Anand, >It is russian letter causing error. Cell data is “П001”. That is the >correct Item Code stored in erpnext database. CSV file have only one test >string, please find it attached. > >On Saturday, March 24, 2012 7:00:45 PM UTC+4, Anand Doshi wrote: >> >> Hi Anton, >> >> The encode function converts a Unicode string to an ASCII byte string by >> encoding special characters based on a character set, which is utf8 in this >> case. >> >> Recently we faced an issue in import because there was a hyphen like >> character which was not getting encoded properly while saving the csv file >> as utf8. We were using OpenOffice. You will need to find which particular >> cell data is causing the decoding issue. There might be a character which >> might be causing the issue. >> >> I would suggest you to try importing only one record and see if you are >> getting the same error. >> >> Thanks >> Anand >> >> > >– >You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group. >To view this discussion on the web visit https://groups.google.com/d/ msg/wnframework/-/UmYahdq_lP0J . >To post to this group, send email to wn…@googlegroups.com. >To unsubscribe from this group, send email to wnframework+unsubscribe@ googlegroups.com. >For more options, visit this group at http://groups.google.com/ group/wnframework?hl=en. >
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.
Anand, all tables in my database already have CHARSET=utf8 (I’ve run the patch though - all the same). I think issue is somewhere in csv file reading part. Maybe charset must be explicitly specified in file_manager.py?
On Saturday, March 24, 2012 11:31:51 PM UTC+4, Anand Doshi wrote:
Hi Anton,
I think you should. Use python wnf.py -d yourdbname -p patches.jan_mar_2012.convert_ to_utf8
*pls confirm the patch name… In erpnext/patches/Jan_mar_2012 folder
I think thats the patch.
It converts your database tables to support utf8
Thanks, Anand.
–
You received this message because you are subscribed to the Google Groups “ERPNext Technical Forum” group.