svghmi/i18n.py
author Edouard Tisserant <edouard.tisserant@gmail.com>
Wed, 05 Jun 2024 15:05:54 +0200
changeset 3954 5744391252ec
parent 3919 0ce4b50d1182
permissions -rw-r--r--
merge
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
     1
#!/usr/bin/env python
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
     2
# -*- coding: utf-8 -*-
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
     3
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
     4
# This file is part of Beremiz
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
     5
# Copyright (C) 2021: Edouard TISSERANT
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
     6
#
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
     7
# See COPYING file for copyrights details.
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
     8
3750
f62625418bff automated conversion using 2to3-3.9 tool
GP Orcullo <kinsamanka@gmail.com>
parents: 3573
diff changeset
     9
3115
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
    10
from lxml import etree
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    11
import os
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    12
import sys
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    13
import subprocess
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
    14
import time
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    15
import ast
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    16
import wx
3157
279e08f238ca SVGHMI: Fix launch of POEdit on widows. Parsing oommand in registry based on heuristic was bad, now using regex should be safer. TODO : replicate for Inkscape launch.
Edouard Tisserant
parents: 3144
diff changeset
    17
import re
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
    18
from email.parser import HeaderParser
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    19
3133
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
    20
import pycountry
3573
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    21
from dialogs import MessageBoxOnce
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
    22
from POULibrary import UserAddressedException
3133
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
    23
3157
279e08f238ca SVGHMI: Fix launch of POEdit on widows. Parsing oommand in registry based on heuristic was bad, now using regex should be safer. TODO : replicate for Inkscape launch.
Edouard Tisserant
parents: 3144
diff changeset
    24
cmd_parser = re.compile(r'(?:"([^"]+)"\s*|([^\s]+)\s*)?')
279e08f238ca SVGHMI: Fix launch of POEdit on widows. Parsing oommand in registry based on heuristic was bad, now using regex should be safer. TODO : replicate for Inkscape launch.
Edouard Tisserant
parents: 3144
diff changeset
    25
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    26
def open_pofile(pofile):
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    27
    """ Opens PO file with POEdit """
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    28
    
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    29
    if sys.platform.startswith('win'):
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
    30
        import winreg
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    31
        poedit_cmd = None
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    32
        try:
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    33
            poedit_cmd = winreg.QueryValue(winreg.HKEY_LOCAL_MACHINE,
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    34
                                           'SOFTWARE\\Classes\\poedit\\shell\\open\\command')
3157
279e08f238ca SVGHMI: Fix launch of POEdit on widows. Parsing oommand in registry based on heuristic was bad, now using regex should be safer. TODO : replicate for Inkscape launch.
Edouard Tisserant
parents: 3144
diff changeset
    35
            cmd = re.findall(cmd_parser, poedit_cmd)
279e08f238ca SVGHMI: Fix launch of POEdit on widows. Parsing oommand in registry based on heuristic was bad, now using regex should be safer. TODO : replicate for Inkscape launch.
Edouard Tisserant
parents: 3144
diff changeset
    36
            dblquote_value,smpl_value = cmd[0]
279e08f238ca SVGHMI: Fix launch of POEdit on widows. Parsing oommand in registry based on heuristic was bad, now using regex should be safer. TODO : replicate for Inkscape launch.
Edouard Tisserant
parents: 3144
diff changeset
    37
            poedit_path = dblquote_value+smpl_value
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    38
        except OSError:
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    39
            poedit_path = None
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    40
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    41
    else:
3750
f62625418bff automated conversion using 2to3-3.9 tool
GP Orcullo <kinsamanka@gmail.com>
parents: 3573
diff changeset
    42
        if "SNAP" in os.environ:
3573
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    43
            MessageBoxOnce("Launching POEdit with xdg-open",
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    44
                    "Confined app can't launch POEdit directly.\n"+
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    45
                        "Instead, PO/POT file is passed to xdg-open.\n"+
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    46
                        "Please select POEdit when proposed.\n\n"+
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    47
                    "Notes: \n"+
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    48
                    " - POEdit must be installed on you system.\n"+
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    49
                    " - If no choice is proposed, use file manager to change POT/PO file properties.\n",
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    50
                    "SVGHMII18SnapWarning")
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    51
            poedit_path = "xdg-open"
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    52
        else:
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    53
            try:
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    54
                poedit_path = subprocess.check_output("command -v poedit", shell=True).strip()
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    55
            except subprocess.CalledProcessError:
1ee56fb544fc IDE, SVGHMI: Workaround Snap package not launching Inskape, POEdit or Chromium directly, producing lot of output in Incskape CLI, and messing with TMPDIR
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3218
diff changeset
    56
                poedit_path = None
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    57
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    58
    if poedit_path is None:
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    59
        wx.MessageBox("POEdit is not found or installed !")
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    60
    else:
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
    61
        subprocess.Popen([poedit_path,pofile])
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
    62
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    63
def EtreeToMessages(msgs):
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    64
    """ Converts XML tree from 'extract_i18n' templates into a list of tuples """
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    65
    messages = []
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    66
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    67
    for msg in msgs:
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    68
        messages.append((
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
    69
            b"\n".join([line.text.encode() for line in msg]),
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
    70
            msg.get("label").encode(), msg.get("id").encode()))
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    71
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    72
    return messages
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    73
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    74
def SaveCatalog(fname, messages):
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    75
    """ Save messages given as list of tupple (msg,label,id) in POT file """
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    76
    w = POTWriter()
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    77
    w.ImportMessages(messages)
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    78
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
    79
    with open(fname, 'wb') as POT_file:
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    80
        w.write(POT_file)
3114
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
    81
3214
bb314cdfc656 SVGHMI: force svghmi build if any change in fonts or translation (.PO) files
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3198
diff changeset
    82
def GetPoFiles(dirpath):
bb314cdfc656 SVGHMI: force svghmi build if any change in fonts or translation (.PO) files
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3198
diff changeset
    83
    po_files = [fname for fname in os.listdir(dirpath) if fname.endswith(".po")]
bb314cdfc656 SVGHMI: force svghmi build if any change in fonts or translation (.PO) files
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3198
diff changeset
    84
    po_files.sort()
3218
76f1cd1291f4 SVGHMI: Fixed last i18n changes meant to allow checking changes in PO files. Was making build exception when no PO files available.
Edouard Tisserant
parents: 3214
diff changeset
    85
    return [(po_fname[:-3],os.path.join(dirpath, po_fname)) for po_fname in po_files]
3214
bb314cdfc656 SVGHMI: force svghmi build if any change in fonts or translation (.PO) files
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3198
diff changeset
    86
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    87
def ReadTranslations(dirpath):
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
    88
    """ Read all PO files from a directory and return a list of (langcode, translation_dict) tuples """
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    89
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    90
    translations = []
3218
76f1cd1291f4 SVGHMI: Fixed last i18n changes meant to allow checking changes in PO files. Was making build exception when no PO files available.
Edouard Tisserant
parents: 3214
diff changeset
    91
    for translation_name, po_path in GetPoFiles(dirpath):
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
    92
        messages = POReader().read(po_path)
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
    93
        translations.append((translation_name, messages))
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    94
    return translations
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    95
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
    96
def MatchTranslations(translations, messages, errcallback):
3114
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
    97
    """
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
    98
    Matches translations against original message catalog,
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
    99
    warn about inconsistancies,
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
   100
    returns list of langs, and a list of (msgid, [translations]) tuples
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   101
    """
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   102
    translated_messages = []
3114
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
   103
    broken_lang = set()
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   104
    incomplete_lang = set()
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   105
    for msgid,label,svgid in messages:
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   106
        translated_message = []
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   107
        for langcode,translation in translations:
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   108
            msg = translation.pop(msgid, None)
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   109
            if msg is None:
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   110
                # Missing translation (msgid not in .po)
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   111
                broken_lang.add(langcode)
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   112
                msg = msgid
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   113
            elif msg == b"":
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   114
                # Empty translation (msgid is in .po)
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   115
                incomplete_lang.add(langcode)
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   116
                msg = msgid
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   117
            
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   118
            translated_message.append(msg)
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   119
        translated_messages.append((msgid,translated_message))
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   120
    langs = []
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   121
    for langcode,translation in translations:
3133
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
   122
        try:
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   123
            l,c = langcode.split("_")
3133
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
   124
            language_name = pycountry.languages.get(alpha_2 = l).name
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
   125
            country_name = pycountry.countries.get(alpha_2 = c).name
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   126
            langname = "{} ({})".format(language_name, country_name)
3133
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
   127
        except:
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
   128
            try:
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   129
                langname = pycountry.languages.get(alpha_2 = langcode).name
3133
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
   130
            except:
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   131
                langname = langcode
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   132
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   133
        langs.append((langname,langcode))
3133
450cd01324ad SVGHMI: HMI:DropDown:#langs@lang now automatically shows human readable list of languages. tests/svghmi_i18n updated accordingly.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3116
diff changeset
   134
3114
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
   135
        broken = False
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   136
        if translation:
3115
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   137
            broken = True
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   138
        if broken or langcode in broken_lang:
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   139
            errcallback((
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   140
                    _('Translation for {} is outdated and incomplete.') 
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   141
                    if langcode in incomplete_lang else
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   142
                    _('Translation for {} is outdated.')).format(langcode) + 
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   143
                " "+_('Edit {}.po, click "Catalog -> Update from POT File..." and select messages.pot.\n').format(langcode))
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   144
        elif langcode in incomplete_lang:
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   145
            errcallback(_('Translation for {} is incomplete. Edit {}.po to complete it.\n').format(langcode,langcode))
3114
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
   146
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   147
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   148
    return langs,translated_messages
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   149
3114
fb1e320836e8 SVGHMI: i18n: better warning messages, more explicit.
Edouard Tisserant
parents: 3113
diff changeset
   150
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   151
def TranslationToEtree(langs,translated_messages):
3115
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   152
3116
6da94ec04325 SVGHMI: i18n: finished passing back translated messaged to xslt proc, finished template to generate corresponding javascript literals.
Edouard Tisserant
parents: 3115
diff changeset
   153
    result = etree.Element("translations")
6da94ec04325 SVGHMI: i18n: finished passing back translated messaged to xslt proc, finished template to generate corresponding javascript literals.
Edouard Tisserant
parents: 3115
diff changeset
   154
6da94ec04325 SVGHMI: i18n: finished passing back translated messaged to xslt proc, finished template to generate corresponding javascript literals.
Edouard Tisserant
parents: 3115
diff changeset
   155
    langsroot = etree.SubElement(result, "langs")
3144
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   156
    for name, code in langs:
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   157
        langel = etree.SubElement(langsroot, "lang", {"code":code})
2af6afaccaf2 SVGHMI: i18n: ensure langs always appea in same order, and add two variables that are updated automatically when selecting a new language : lang_name and lang_code. Also fixed i18n startup, prevent wrong lang numer to crash loading.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3133
diff changeset
   158
        langel.text = name
3115
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   159
3116
6da94ec04325 SVGHMI: i18n: finished passing back translated messaged to xslt proc, finished template to generate corresponding javascript literals.
Edouard Tisserant
parents: 3115
diff changeset
   160
    msgsroot = etree.SubElement(result, "messages")
3115
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   161
    for msgid, msgs in translated_messages:
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   162
        msgidel = etree.SubElement(msgsroot, "msgid")
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   163
        for msg in msgs:
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   164
            msgel = etree.SubElement(msgidel, "msg")
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   165
            for line in msg.split(b"\n"):
3116
6da94ec04325 SVGHMI: i18n: finished passing back translated messaged to xslt proc, finished template to generate corresponding javascript literals.
Edouard Tisserant
parents: 3115
diff changeset
   166
                lineel = etree.SubElement(msgel, "line")
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   167
                lineel.text = escape(line).decode()
3116
6da94ec04325 SVGHMI: i18n: finished passing back translated messaged to xslt proc, finished template to generate corresponding javascript literals.
Edouard Tisserant
parents: 3115
diff changeset
   168
6da94ec04325 SVGHMI: i18n: finished passing back translated messaged to xslt proc, finished template to generate corresponding javascript literals.
Edouard Tisserant
parents: 3115
diff changeset
   169
    return result
3115
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   170
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   171
# Code below is based on :
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   172
#  cpython/Tools/i18n/pygettext.py
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   173
#  cpython/Tools/i18n/msgfmt.py
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   174
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   175
locpfx = b'#:svghmi.svg:'
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   176
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   177
pot_header = '''\
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   178
# SOME DESCRIPTIVE TITLE.
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   179
# Copyright (C) YEAR ORGANIZATION
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   180
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   181
#
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   182
msgid ""
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   183
msgstr ""
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   184
"Project-Id-Version: PACKAGE VERSION\\n"
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   185
"POT-Creation-Date: %(time)s\\n"
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   186
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\\n"
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   187
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\\n"
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   188
"Language-Team: LANGUAGE <LL@li.org>\\n"
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   189
"MIME-Version: 1.0\\n"
3113
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   190
"Content-Type: text/plain; charset=UTF-8\\n"
18133b90196e SVGHMI: i18n: now loads PO filesand match translation against catalog. Refactored a bit to move i18n related code in i18n.py
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3112
diff changeset
   191
"Content-Transfer-Encoding: 8bit\\n"
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   192
"Generated-By: SVGHMI 1.0\\n"
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   193
3918
9f0ef23569cb SVGHMI: i18n: fix .POT generation + other cosmetic fixes.
Edouard Tisserant <edouard@beremiz.fr>
parents: 3915
diff changeset
   194
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   195
'''
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   196
escapes = []
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   197
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   198
def make_escapes():
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   199
    global escapes
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   200
    escapes = [b"\%03o" % i for i in range(128)]
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   201
    for i in range(32, 127):
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   202
        escapes[i] = bytes([i])
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   203
    escapes[ord('\\')] = b'\\\\'
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   204
    escapes[ord('\t')] = b'\\t'
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   205
    escapes[ord('\r')] = b'\\r'
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   206
    escapes[ord('\n')] = b'\\n'
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   207
    escapes[ord('\"')] = b'\\"'
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   208
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   209
make_escapes()
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   210
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   211
def escape(s):
3918
9f0ef23569cb SVGHMI: i18n: fix .POT generation + other cosmetic fixes.
Edouard Tisserant <edouard@beremiz.fr>
parents: 3915
diff changeset
   212
    return b''.join([escapes[c] if c < 128 else bytes([c]) for c in s])
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   213
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   214
def normalize(s):
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   215
    # This converts the various Python string types into a format that is
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   216
    # appropriate for .po files, namely much closer to C style.
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   217
    lines = s.split(b'\n')
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   218
    if len(lines) == 1:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   219
        s = b'"' + escape(s) + b'"'
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   220
    else:
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   221
        if not lines[-1]:
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   222
            del lines[-1]
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   223
            lines[-1] = lines[-1] + b'\n'
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   224
        for i in range(len(lines)):
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   225
            lines[i] = escape(lines[i])
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   226
        lineterm = b'\\n"\n"'
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   227
        s = b'""\n"' + lineterm.join(lines) + b'"'
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   228
    return s
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   229
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   230
class POTWriter:
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   231
    def __init__(self):
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   232
        self.__messages = {}
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   233
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   234
    def ImportMessages(self, msgs):
3115
77cfbf1aacf0 SVGHMI: i18n: various fixes about unicode encoding and converting translation result back to XML to feed xslt processor
Edouard Tisserant
parents: 3114
diff changeset
   235
        for  msg, label, svgid in msgs:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   236
            self.addentry(msg, label, svgid)
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   237
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   238
    def addentry(self, msg, label, svgid):
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   239
        entry = (label, svgid)
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   240
        self.__messages.setdefault(msg, set()).add(entry)
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   241
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   242
    def write(self, fp):
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   243
        timestamp = time.strftime('%Y-%m-%d %H:%M%z')
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   244
        header = pot_header % {'time': timestamp}
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   245
        fp.write(header.encode())
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   246
        reverse = {}
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   247
        for k, v in self.__messages.items():
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   248
            keys = list(v)
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   249
            keys.sort()
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   250
            reverse.setdefault(tuple(keys), []).append((k, v))
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   251
        rkeys = sorted(reverse.keys())
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   252
        for rkey in rkeys:
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   253
            rentries = reverse[rkey]
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   254
            rentries.sort()
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   255
            for k, v in rentries:
3112
bd20f9112014 SVGHMI: still WIP, now POT file is properly generated with utf-8 encoding and POEdit is launched when pressing button.
Edouard Tisserant
parents: 3108
diff changeset
   256
                v = list(v)
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   257
                v.sort()
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   258
                locline = locpfx
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   259
                for label, svgid in v:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   260
                    d = {b'label': label, b'svgid': svgid}
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   261
                    s = b' %(label)s:%(svgid)s' % d
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   262
                    if len(locline) + len(s) <= 78:
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   263
                        locline = locline + s
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   264
                    else:
3918
9f0ef23569cb SVGHMI: i18n: fix .POT generation + other cosmetic fixes.
Edouard Tisserant <edouard@beremiz.fr>
parents: 3915
diff changeset
   265
                        fp.write(locline + b'\n')
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   266
                        locline = locpfx + s
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   267
                if len(locline) > len(locpfx):
3918
9f0ef23569cb SVGHMI: i18n: fix .POT generation + other cosmetic fixes.
Edouard Tisserant <edouard@beremiz.fr>
parents: 3915
diff changeset
   268
                    fp.write(locline + b'\n')
9f0ef23569cb SVGHMI: i18n: fix .POT generation + other cosmetic fixes.
Edouard Tisserant <edouard@beremiz.fr>
parents: 3915
diff changeset
   269
                fp.write(b'msgid ' + normalize(k) + b'\n')
9f0ef23569cb SVGHMI: i18n: fix .POT generation + other cosmetic fixes.
Edouard Tisserant <edouard@beremiz.fr>
parents: 3915
diff changeset
   270
                fp.write(b'msgstr ""\n\n')
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   271
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   272
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   273
class POReader:
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   274
    def __init__(self):
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   275
        self.__messages = {}
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   276
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   277
    def add(self, ctxt, msgid, msgstr, fuzzy):
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   278
        "Add eventually empty translation to the dictionary."
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   279
        if msgid:
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   280
            self.__messages[msgid if ctxt is None else (b"%b\x04%b" % (ctxt, id))] = msgstr
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   281
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   282
    def read(self, infile):
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   283
        ID = 1
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   284
        STR = 2
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   285
        CTXT = 3
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   286
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   287
        self.__messages = {}
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   288
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   289
        try:
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   290
            with open(infile, 'rb') as f:
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   291
                lines = f.readlines()
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   292
        except Exception as e:
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   293
            raise UserAddressedException(
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   294
                        'Cannot open PO translation file :%s' % str(e))
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   295
            
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   296
        section = msgctxt = None
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   297
        fuzzy = 0
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   298
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   299
        # Start off assuming Latin-1, so everything decodes without failure,
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   300
        # until we know the exact encoding
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   301
        encoding = 'latin-1'
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   302
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   303
        # Parse the catalog
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   304
        lno = 0
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   305
        for l in lines:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   306
            l = l.decode(encoding)
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   307
            lno += 1
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   308
            # If we get a comment line after a msgstr, this is a new entry
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   309
            if l[0] == '#' and section == STR:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   310
                self.add(msgctxt, msgid, msgstr, fuzzy)
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   311
                section = msgctxt = None
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   312
                fuzzy = 0
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   313
            # Record a fuzzy mark
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   314
            if l[:2] == '#,' and 'fuzzy' in l:
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   315
                fuzzy = 1
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   316
            # Skip comments
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   317
            if l[0] == '#':
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   318
                continue
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   319
            # Now we are in a msgid or msgctxt section, output previous section
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   320
            if l.startswith('msgctxt'):
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   321
                if section == STR:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   322
                    self.add(msgctxt, msgid, msgstr, fuzzy)
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   323
                    fuzzy = 0
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   324
                section = CTXT
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   325
                l = l[7:]
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   326
                msgctxt = b''
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   327
            elif l.startswith('msgid') and not l.startswith('msgid_plural'):
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   328
                if section == STR:
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   329
                    self.add(msgctxt, msgid, msgstr, fuzzy)
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   330
                    fuzzy = 0
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   331
                    if not msgid:
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   332
                        # See whether there is an encoding declaration
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   333
                        p = HeaderParser()
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   334
                        charset = p.parsestr(msgstr.decode(encoding)).get_content_charset()
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   335
                        if charset:
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   336
                            encoding = charset
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   337
                section = ID
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   338
                l = l[5:]
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   339
                msgid = msgstr = b''
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   340
                is_plural = False
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   341
            # This is a message with plural forms
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   342
            elif l.startswith('msgid_plural'):
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   343
                if section != ID:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   344
                    raise UserAddressedException(
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   345
                        'msgid_plural not preceded by msgid on %s:%d' % (infile, lno))
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   346
                l = l[12:]
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   347
                msgid += b'\0' # separator of singular and plural
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   348
                is_plural = True
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   349
            # Now we are in a msgstr section
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   350
            elif l.startswith('msgstr'):
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   351
                section = STR
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   352
                if l.startswith('msgstr['):
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   353
                    if not is_plural:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   354
                        raise UserAddressedException(
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   355
                            'plural without msgid_plural on %s:%d' % (infile, lno))
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   356
                    l = l.split(']', 1)[1]
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   357
                    if msgstr:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   358
                        msgstr += b'\0' # Separator of the various plural forms
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   359
                else:
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   360
                    if is_plural:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   361
                        raise UserAddressedException(
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   362
                            'indexed msgstr required for plural on  %s:%d' % (infile, lno))
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   363
                    l = l[6:]
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   364
            # Skip empty lines
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   365
            l = l.strip()
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   366
            if not l:
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   367
                continue
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   368
            l = ast.literal_eval(l)
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   369
            if section == CTXT:
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   370
                msgctxt += l.encode(encoding)
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   371
            elif section == ID:
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   372
                msgid += l.encode(encoding)
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   373
            elif section == STR:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   374
                msgstr += l.encode(encoding)
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   375
            else:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   376
                raise UserAddressedException(
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   377
                    'Syntax error on %s:%d' % (infile, lno) + 'before:\n %s'%l)
3108
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   378
        # Add last entry
079419e7228d SVGHMI: Intermediate commit while implementing i18n. WIP.
Edouard Tisserant <edouard.tisserant@gmail.com>
parents:
diff changeset
   379
        if section == STR:
3915
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   380
            self.add(msgctxt, msgid, msgstr, fuzzy)
b5017dd5c049 SVGHMI: fix py3 consequences on i18n
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3750
diff changeset
   381
3919
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   382
        return self.__messages
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   383
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   384
0ce4b50d1182 SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations
Edouard Tisserant <edouard.tisserant@gmail.com>
parents: 3918
diff changeset
   385