Merged in masterschlumpf/beremiz/generate-button (pull request #38)

"Generate Prgram" button

Approved-by: Andrey Skvortsov <andrej.skvortzov@gmail.com>

Create a "Generate Program As..." menu to make the "Generate Program" toolbutton even more useful.

The behaviour is the same as "Save" and "Save As...". Now "Generate Program" uses the last filename
if known and "Generate Progam As..." asks every time.

Add "Generate Program" to toolbar in PLCOpenEditor standalone mode.

Currently you can find the "Generate Program" action only in the file menu,
but this action is very often used in standalone mode, so I added toolbuffon.

Merged search in CTN branch

Finished adding search in python and C files. Stripped debug code and useless comments. Close branch

WIP adding searching capabilities in python files. was done :
- added search in body of Code File Tree Nodes (moved editor code so that we CTN search can have the same sections text layout as editor to search in)

WIP adding searching capabilities in python files. was done :
- added search in global variables used by Code File Tree Nodes

WIP adding searching capabilities in python files. was done :
- fixed highlighting search result in text body of CTN code editor.

WIP adding searching capabilities in python files. was done :
- change stub search results to trigger highlighting un variable panel
- trying to highlight in CTN code file editors -> problem with EditorPanel not having self.VariableEditor set...
- added highlight capability to CTN's CodeFileEditor's VariablePanel

WIP adding searching capabilities in python files. was done :
- fixed stub CTN search results so that they can be accepted by search result panel
- worked on search result panel so that it can display those CTN results
- made search result panel display correctly stub search result, for variables and body matches
- EditProjectElement now opens CTN node as well as PlcOpen project nodes

WIP adding searching capabilities in python files:
- Added recursive Search capability to config tree nodes
- Added stub search to code editors, for test

Fixed more pylint reports :
connectors/__init__.py:31: [W1652(deprecated-types-field), ] Accessing a deprecated fields on the types module
in fact, it should have been unused import.

Long due merge

Fix backtrace on printing localized (non-ascii) error message in PLCGenerator.

traceback:
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\BeremizIDE.py", line 811, in OnMenu
getattr(self.CTR, method)()
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\ProjectController.py", line 1134, in _Build
IECGenRes = self._Generate_SoftPLC()
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\ProjectController.py", line 746, in _Generate_SoftPLC
if self._Generate_PLC_ST():
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\ProjectController.py", line 763, in _Generate_PLC_ST
self._getIECgeneratedcodepath())
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\PLCControler.py", line 466, in GenerateProgram
errors.append(str(ex))
<type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode characters in position 0-9: ordinal not in range(128)
Fix backtrace on printing localized (non-ascii) error message in PLCGenerator.

traceback:
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\BeremizIDE.py", line 811, in OnMenu
getattr(self.CTR, method)()
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\ProjectController.py", line 1134, in _Build
IECGenRes = self._Generate_SoftPLC()
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\ProjectController.py", line 746, in _Generate_SoftPLC
if self._Generate_PLC_ST():
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\ProjectController.py", line 763, in _Generate_PLC_ST
self._getIECgeneratedcodepath())
File "C:\Beremiz\targets\st\beremiz\..\..\..\beremiz\beremiz\PLCControler.py", line 466, in GenerateProgram
errors.append(str(ex))
<type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode characters in position 0-9: ordinal not in range(128)


Close #67

Fix crash if LD input contact isn't connected

Traceback (most recent call last):
File "/home/developer/WorkData/PLC/beremiz/beremiz/BeremizIDE.py", line 339, in OnMethod
obj.CTR.CallMethod('_'+meth)
File "/home/developer/WorkData/PLC/beremiz/beremiz/ProjectController.py", line 1996, in CallMethod
getattr(self, method)()
File "/home/developer/WorkData/PLC/beremiz/beremiz/ProjectController.py", line 1134, in _Build
IECGenRes = self._Generate_SoftPLC()
File "/home/developer/WorkData/PLC/beremiz/beremiz/ProjectController.py", line 746, in _Generate_SoftPLC
if self._Generate_PLC_ST():
File "/home/developer/WorkData/PLC/beremiz/beremiz/ProjectController.py", line 763, in _Generate_PLC_ST
self._getIECgeneratedcodepath())
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCControler.py", line 456, in GenerateProgram
self.ProgramChunks = GenerateCurrentProgram(self, self.Project, errors, warnings)
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCGenerator.py", line 1762, in GenerateCurrentProgram
generator.GenerateProgram()
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCGenerator.py", line 478, in GenerateProgram
self.GeneratePouProgram(pou_name)
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCGenerator.py", line 260, in GeneratePouProgram
program = pou_program.GenerateProgram(pou)
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCGenerator.py", line 1718, in GenerateProgram
self.ComputeProgram(pou)
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCGenerator.py", line 1028, in ComputeProgram
self.GenerateBlock(instance, block_infos, body, None)
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCGenerator.py", line 1144, in GenerateBlock
expression = self.ComputeExpression(body, connections, executionOrderId > 0)
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCGenerator.py", line 1373, in ComputeExpression
paths = self.GeneratePaths(connections, body, order, to_inout)
File "/home/developer/WorkData/PLC/beremiz/beremiz/PLCGenerator.py", line 1344, in GeneratePaths
elif isinstance(result[0], list):
IndexError: list index out of range


Close #70

Fix section styling in code editor (python and c extensions) if non-latin symbols are used in the code

Problem exists even if non-latin characters are placed in comments.
Close #76

update localization template

merge

updated list of translators

update translations from https://www.transifex.com/beremiz/beremiz-ide

added Basque (eu) and Vietnamese (Viet Nam) (vi_VN) translations

Add SpinButtons for integer and floating point variables in ForceVariableDialog

it allows to change variable value using only mouse.

Fix non-marking as manually forced floating point variable if the value isn't integer

For example, if user in debug variable panel set any floating point
variable to 34.3, then it's not shown as forced (blue color) and user
can't release this enforcement.

If user changes the value to 34.0, then enforcement shows
correctly. This is done because binary representation of floating point
numbers in IDE and runtime can be slightly different (double vs float)
and as a result values aren't equal.

Don't close ForceVariableDialog if user entered wrong new value

return back to edit field in this case

Additional fix for rewrite ForceVariableDialog commit (9076249f)

fix following backtrace:

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk3/wx/_core.py", line 16765, in <lambda>
lambda event: event.callable(*event.args, **event.kw) )
File "/home/developer/WorkData/PLC/beremiz/beremiz/controls/DebugVariablePanel/DebugVariableViewer.py", line 232, in HandleButton
button.ProcessCallback()
File "/home/developer/WorkData/PLC/beremiz/beremiz/controls/DebugVariablePanel/GraphButton.py", line 156, in ProcessCallback
self.Callback()
File "/home/developer/WorkData/PLC/beremiz/beremiz/controls/DebugVariablePanel/DebugVariableViewer.py", line 350, in OnForceButton
self.ForceValue(self.ItemsDict.values()[0])
File "/home/developer/WorkData/PLC/beremiz/beremiz/controls/DebugVariablePanel/DebugVariableViewer.py", line 428, in ForceValue
dialog.GetValue())
File "/home/developer/WorkData/PLC/beremiz/beremiz/dialogs/ForceVariableDialog.py", line 232, in GetValue
return GetTypeValue[self.IEC_Type](wx.TextEntryDialog.GetValue(self))
TypeError: unbound method GetValue() must be called with TextEntryDialog instance as first argument (got ForceVariableDialog instance instead)

Fix redraw problems in preview frames and debug panel

The problem was that direct call of RefreshView (now DrawView) doesn't
generate EVT_PAINT (at least on GNU/Linux with Wayland and wxgtk build
with gtk3+ support).
Now RefreshView simply triggers EVT_PAINT and EVT_PAINT handler does actual work.

Rewrite ForceVariableDialog. It's not based on wx.TextEntryDialog

Don't customize standard wx.TextEntryDialog on the fly, implement your
own dialog for that with expected behavior and without any hacks.

merge updates to modbus/mb_runtime.c

fix 'SaveAs' in case of overwriting existing project

Traceback (most recent call last):
File "/home/developer/WorkData/PLC/beremiz/beremiz/BeremizIDE.py", line 959, in OnSaveProjectAsMenu
self.CTR.SaveProjectAs()
File "/home/developer/WorkData/PLC/beremiz/beremiz/ProjectController.py", line 600, in SaveProjectAs
self.SaveProject(old_project_path)
File "/home/developer/WorkData/PLC/beremiz/beremiz/ProjectController.py", line 579, in SaveProject
self._getProjectFilesPath(self.ProjectPath))
File "/usr/lib/python2.7/shutil.py", line 200, in copytree
os.makedirs(dst)
File "/usr/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 17] File exists: '/tmp/tests/project_files'

fix 'SaveAs' to non-empty directory and directory without write permissions

fix problems with recursive beremiz.h inclusion

remove LogMessage prototype from plc_main_head.c

it's not needed, because prototype is included in beremiz.h
But it caused problems with TARGET_LOGGING_DISABLE.

merge

Make online debug optional

It could be useful for very small targets like Atmega (Arduino) and
for target bring-up there developer want to have running PLC program,
but has not implemented runtime communication yet.


TARGET_DEBUG_AND_RETAIN_DISABLE - completely disable debug and retain
functionality. Previously named TARGET_DEBUG_DISABLE.

TARGET_ONLINE_DEBUG_DISABLE - can be used to enable retain
functionality (no define TARGET_DEBUG_AND_RETAIN_DISABLE is used), but disable
online debug with corresponding RAM/FLASH overhead.

TARGET_LOGGING_DISABLE - disables logging functionality from runtime and PLC program

TARGET_EXT_SYNC_DISABLE - disables PLC program synchronization with
external events. For example, it could be used to synchronize several
PLCs that control motors for different axes.

By default all these options are off.

To test generate program for Generic target, put following files in
project files directory and run build.sh after generating PLC program.
This is very easy to integrate into makefile (Generic target).

[------------- build.sh --------------------------]
files=$(find $PWD/../build -iname '*.c' | grep -v POUS.c)
arm-none-eabi-gcc \
-DTARGET_DEBUG_AND_RETAIN_DISABLE \
-DTARGET_ONLINE_DEBUG_DISABLE \
-DTARGET_LOGGING_DISABLE \
-DTARGET_EXT_SYNC_DISABLE \
-flto -ffunction-sections -fdata-sections -I../../../../matiec/lib/C \
$files \
main.c \
-Wl,--Map=./program.map,--cref \
-nodefaultlibs --specs=nano.specs -Wl,--static -Wl,--gc-section -Wl,--start-group -lc -lm -lnosys -lgcc -Wl,--end-group
[------------------------------------------------]

[------------- main.c --------------------------]
#ifndef TARGET_DEBUG_AND_RETAIN_DISABLE
void Retain(void){}
void InValidateRetainBuffer(void){}
void ValidateRetainBuffer(void){}
#endif

extern void __run(void);
int main(void)
{
for(;;) {
__run();
// sleep common_ticktime__ ns
// add common_ticktime__ ns to __CURRENT_TIME
}
return 0;
}
[------------------------------------------------]

Disable Python extension in First steps example

use pregenerated CRC32 lookup tables for retain on Win32 and GNU/Linux

This code could be possible reused on low-end targets with limited RAM.

code to generate lookup table:
[---------------------------------------------------------------------]

/* CRC lookup table and initial state. */
uint32_t crc32_table[256];

/* Generate CRC32 lookup table. */
void GenerateCRC32Table(void)
{
unsigned int i, j;
/* Use CRC-32-IEEE 802.3 polynomial 0x04C11DB7 (bit reflected). */
uint32_t poly = 0xEDB88320;

for (i = 0; i <= 0xFF; i++)
{
uint32_t c = i;
for (j = 0 ; j < 8 ; j++)
c = (c & 1) ? (c >> 1 ) ^ poly : (c >> 1);
crc32_table[i] = c;
}
}

void main(void)
{
GenerateCRC32Table();
int j=0;
for(int i=0; i<256; i++) {
printf("0x%08X, ", crc32_table[i]);
if (++j >= 8) {
j = 0;
printf("\n");
}
}
}
[---------------------------------------------------------------------]

Merged in masterschlumpf/beremiz/wx3-fix (pull request #37)

Fix an exception with wx-3.0-gtk3 in PLCOpenEditor when generating ST files

Approved-by: Andrey Skvortsov <andrej.skvortzov@gmail.com>

Fix an exception on wx-3.0-gtk3 in PLCOpenEditor when generating ST files.

On Python2.7 with WX3.0 and GTK3, an assertionError rises on generating a ST file if the name is already set. The first generation works without problems, if you generate the file a second one, PLCOpenEditor tries to open the file
save dialog with the pre entered name from last run. Then the following assertion pops up:

PyAssertionError: C++ assertion "volDummy.empty() && pathDummy.empty()" failed at ./src/common/filename.cpp(568) in Assign(): the file name shouldn't contain the path

This fix reduces the filepath of the ST file the to the filename. Now it works fine.

Fix typos in function names

update links to matiec and beremiz repos in README

fix zoom-in/zoom-out using mouse wheel

regression introduced by 2393:235f847be37e "python3 support: pylint, W1619 #(old-division) division w/o __future__ statement"

better pylint complain workaround

check_sources.sh makes me become even less productive

Added stub axis_s type in EtherLAB CiA402 support code, until Motion Contol Library comes back

Allow using one XSD based target description for many targets. Target description XSD can now use %(target_name)s to refer to target name.

Fix previous commit : Using wx.CallAfter shouldn't have been necessary, and was a symptom of failing evaluator's deadlock detection.

Fix startup of wxGlade base HMI. Wx calls were issued from non-wx thread (main).

Reworked chunk based transfer to support duplicated files (i.e. files with same content, but different names)

Fixed deadlock in runtime's Worker. Was discovered while using WAMP and PYRO simultaneously.

Add a debugger token to SetTraceVariables and GetTraceVariables to prevent crash an inconsistant data in case of multiple connections. Last connection now takes over existing connections's debug, and first connected IDE gets a wrning.

Fixed deadlock when using Wx together with Twisted in runtime.

Split IDE's frame OnCloseFrame() into TryCloseFrame() and OnCloseFrame(), to help further customization

fixed exception happening randomly just after showing discovery panel.

fixed exception happening when connecting to empty URI, and set URI through URI Editor Dialog popping up becaue empty.

Merged Mario's modbus fix : Modbus plugin: reset periodic activation timer when overrun occurs (delays due to communication errors, ...)

Modbus plugin: reset periodic activation timer when overrun occurs (delays due to communication errors, ...)

Disable MDNS:// scheme support. Also prevent discovery pannel to produce some. For now it seems that that feature is unused, and could be a security issue.

Added support to PYROS in Zeroconf discovery and publish. For now, PYRO://mdnsname._tcp.local. URI are broken, and discovery panel generate IP based URI by default.

PyroServer: fixed naming of class, and reorganized methods in order to ease subclassing, in case of customization of service publication on the local network (i.e. zeroconf)

Fix exception dialog with non recognized URI schemes

Fixed runtime's reconnecting loop not stopping when re-configuring WAMP.
In case WAMP was already trying to connect without success and user change WAMP configuration through web interface, previous ReconnectingClient wasn't stopped.

Removed nonsense code, that I should have check more in details before merging m(

Allow easier monkey patching of WAMP connector Session class

Expose new PLCObject API members to WAMP interface.

Use another way to ensure compatibility with Py3 about text type.

Make easier to read console error message in case of connection problem.

PYRO(S): Added timeout on client side to allow unfreezing IDE after some time when communication with target dropped

Enure that autostart blocks other @RunInMain PLCObject methods, and _unblocks_ them once autostart is done...

Fixed Discovery panel in URI Editor dialog : double click works back again, and avoid setting URI to unicode type.

Bigger chunks, from 16k to 1M

Empirically reverted calls done just after NewPLC back to the order as it was before chunked transfer. Apparently I broke something... To be continued

Chunk based transfer for PLC binary and extra files, and some collateral code refactoring.

Fix failing PLC build when building freshly created project that use default libraries blocks. When controller loads libraries according to attibutes in project files (BeremizRoot/Enable_{libname}_Library), use known default if attribute is missing.

Cosmetic change in PSKManagement.py

Fixed ID,last URI, or last connection date not being updated when connnecting to target.

Merged. Some changes that should already have been incuded during previous merge (mostly about PlcStatus) have been included this time.

IDManager : small cosmetic fixes and cleanup.

python3 support: pylint, W1633 # (round-builtin) round built-in referenced

because round behavior is changed to default behavior in Python3
(Banker round). It can cause slight differences in some cases.
Mostly graphical editors are affected, there could be single pixel differences.
Now I couldn't locate any visual differences in test programs.

python3 support: pylint,W1610 # (reduce-builtin) reduce built-in referenced

python3 support: pylint, W1653 # (next-method-defined) next method defined

python3 support: pylint, W1622 # (next-method-called) Called a next() method on an object

python3 support: pylint, W1625 # (raising-string) Raising a string exception

python3 support: pylint, W1624 # (indexing-exception) Indexing exceptions will not work on Python 3

python3 support: pylint, W1611 # (standarderror-builtin) StandardError built-in referenced

python3 support: pylint, W1652 # (deprecated-types-field) Accessing a deprecated fields on the types module

python3 support: pylint, W1651 (deprecated-itertools-function) Accessing a deprecated function on the itertools module

It's even 20% faster, than previous code. On python3 zip is even much more faster.

python3 support: pylint, W1649 # (deprecated-string-function) Accessing a deprecated function on the string module

python3 support: pylint, W1645 # (exception-message-attribute) Exception.message removed in Python 3

python3 support: pylint, W1646 # (invalid-str-codec) non-text encoding used in str.decode

python3 support: pylint,W1602 # (basestring-builtin) basestring built-in referenced

python3 support: pylint, W1629 # (nonzero-method) __nonzero__ method defined

python3 support: pylint, W1606 # (execfile-builtin) execfile built-in referenced

python3 support: pylint, W1607 # (file-builtin) file built-in referenced

python3 support: pylint, W1659 # (xreadlines-attribute) Accessing a removed xreadlines attribute

python3 support: pylint, W1601 # (apply-builtin) apply built-in referenced

Fix pep8 warning: W605 invalid escape sequence ?x?

pep8, ignore W606 'async' and 'await' are reserved keywords starting with Python 3.7

This error exists in pyjs extension, but pyjs isn't going to Python3.
'noqa' isn't working for W606, so this warning is disabled completely.
'async' and 'await' are still checked by pylint.

python3 support: pylint, W1619 #(old-division) division w/o __future__ statement

Make easier to run pytest on local X-server

cleanup pyc files between running pytest

this is need to avoid problem sometimes tests are run inside docker
container and sometimes local.

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/_pytest/config/__init__.py", line 409, in _importconftest
mod = conftestpath.pyimport()
File "/usr/lib/python2.7/dist-packages/py/_path/local.py", line 686, in pyimport
raise self.ImportMismatchError(modname, modfile, self)
ImportMismatchError: ('conftest', '/beremiz/tests/tools/conftest.py', local('/home/developer/beremiz/tests/tools/conftest.py'))
ERROR: could not load /home/developer/beremiz/tests/tools/conftest.py

python3 support: pylint, W1612 # (unicode-builtin) unicode built-in referenced

Use multiple processes to speed up pylint

0 will auto-detect the number of processors available to use.
version 1.6.5 (Debian 9.5) hangs for some reason in multi-process mode.

python3 support: pylint, W1613 # (xrange-builtin) xrange built-in referenced

python3 support: pylint, W1648 # (bad-python3-import) Module moved in Python 3

Fixed regression in connecting to LOCAL:// targets

Merge, with surprizingly little conflicts

IDManager : finished Import/Export. Added merging capability to import (asks if particular ID is replaced during import). Added ESC as closing shortcut to IDManager dialog, and adjusted its size.

merge

Update localization template

Fix 'msgid' format string with unnamed arguments cannot be properly localized

Add automatic check for localization strings with multiple unnamed arguments

Improve localization support in etherlab (mostly)

cleanup etherlab: pylint, W0602 (global-variable-not-assigned) Using global for 'X' but no assignment is done

cleanup: pylint, W0109 # (duplicate-key) Duplicate key 'X' in dictionary

Add python3 syntax checking to CI

Fix old style exception raise and exec syntax

Use python 3 compatible exception syntax everywhere

cleanup: pylint, R0123 # (literal-comparison) Comparison to literal

Use predefined constants for PlcStatus instead of string literals

cleanup: pylint, R1704 # (redefined-argument-from-local) Redefining argument with the local name 'Y'

cleanup: pylint, R1701 # (consider-merging-isinstance) Consider merging these isinstance calls to isinstance(CTNLDFLAGS, (str, unicode))

cleanup: pylint, E1304 # (missing-format-string-key) Missing key 'X_name' in format string dictionary

cleanup: pylint, E1300 # (bad-format-character) Unsupported format character '"' (0x22) at index 17

cleanup: pylint, W0111 # (assign-to-new-keyword) Name async will become a keyword in Python 3.7

cleanup: pylint, E1310 (bad-str-strip-call) Suspicious argument in str.strip call

Fix segmentation fault when last lines in text POUs should be highlighted

enable etherlab checks

resort enabled pylint check to move likely-most invasive changes to the end

clean etherlab: pylint, C0330 # (bad-continuation) Wrong hanging indentation before block

clean etherlab: pylint, W0612 # (unused-variable) Unused variable 'X'