Thread subject: sK1 Project :: Drop usage of external iconv tool

Posted by andriy on 20-09-2009 16:56
#1

First, you have done total mess up with bug tracking:
- site refers to launchpad, but last doesn't recognize itself is a BTS for uniconvertor
- all old bugs are lost with old forum (is there some way to recover them?)

Now, time for bug report.

The locale_utils.py module uses external iconv command. However, python (starting from very 2.0) supports conversion natively.
The patch fixes behaviour and thus performance will be increased.

Posted by andriy on 20-09-2009 16:59
#2

WTF? IS it possible to attach patches as *.diff or *.patch?

Code

diff -up UniConvertor-1.1.4/src/app/utils/locale_utils.py.orig UniConvertor-1.1.4/src/app/utils/locale_utils.py
--- UniConvertor-1.1.4/src/app/utils/locale_utils.py.orig   2008-09-06 16:11:20.000000000 +0300
+++ UniConvertor-1.1.4/src/app/utils/locale_utils.py   2009-09-20 19:31:04.000000000 +0300
@@ -6,47 +6,36 @@
 # For more info see COPYRIGHTS file in sK1 root directory.
 
 import os
-from popen2 import popen2
 from app import config
 
 UTF_CODEC='utf-8'
+POLICY = 'replace'
 
 def locale_to_utf(string=''):
    if string=='' :
       return string
    locale=get_locale()
-   if locale==UTF_CODEC:
+   if locale == '' or locale == UTF_CODEC:
       return string
-   return strip_line(cmd_iconv(locale, UTF_CODEC, string))
+   return strip_line(string.decode(locale).encode(UTF_CODEC, POLICY))
 
 def utf_to_locale(string=''):
    if string=='' :
       return string   
    locale=get_locale()
-   if locale==UTF_CODEC:
+   if locale == '' or locale == UTF_CODEC:
       return string
-   return strip_line(cmd_iconv(UTF_CODEC, locale, string))
+   return strip_line(string.decode(UTF_CODEC).encode(locale, POLICY))
       
 def get_locale():   
    return config.preferences.system_encoding
-   
-def cmd_iconv(from_codec='', to_codec='', string=''):
-   if from_codec=='' or to_codec=='' or string=='' :
-      return string
-   from_bash = os.popen('echo "'+string+'" |iconv -f '+from_codec+' -t '+to_codec)
-#    to_bash.write(string)
-   result=from_bash.read()
-#    to_bash.close()
-   from_bash.close()
-   return result
-   
+
 def strip_line(string=''):
    if string=='' :
       return string
-   return string[0:len(string)-1]
+   return string[:-1]
    
 def getshell_var(s):
    if os.confstr(s):
       return os.environ[s]
    return None
-   
\ No newline at end of file




Posted by igor on 20-09-2009 19:34
#3

Hi,

Thank you for reported issue.

Actually it's very old code and of course in other modules we have used Python built-in conversion. I have checked and found that these locale procedures are not used in runtime anywhere. It seems it's just forgotten code after refactoring.

According to our plans a lot of common modules will be moved into sk1libs package, i.e. common package as for sK1 and for UniConvertor. During refactoring we will fix this issue and external tool usage will be dropped in next version.

Also I have fixed allowed attachment extension issue so you can use *.patch or *.diff files as attachment.

Posted by andriy on 20-09-2009 21:09
#4

igor wrote:Also I have fixed allowed attachment extension issue so you can use *.patch or *.diff files as attachment.

Thank you!