Python3 fixes for Modbus Web Settings

Fix template conflict in XSLT with lxml>=4.9.0 again

Earlier attempt was fixing conflict with -1.0 priority attribute,
as a side effect of such low priority, SFC Actions were not
editable anymore.

This time move template around instead of using explicit priority.

Revert "IDE: fix exception when adding Action"

This reverts commit 94e42cda0a8e093aac4830b5c40656c240ec8cf9.

IDE: fix exception when adding Action

IDE: Fix status bar update when conection lost

SVGHMI: Fix watchdog exception on windows

Typo

Upgrade wxPython to 4.2.1, drop 4.2.0 workaroud

Runtime: add GetVersions() call to PLCObject, use it in web settings and expose it in WAMP and Pyro.

Note: WAMP changes are untested

Runtime: Add stderr support to local posix spawn based subprocess replacement

Modbus: More verbose error message in case of thread creation problem

Runtime web settings: fix punctuation and update CSS again

Update default CSS for Runtime WebSettings.

Runtime WebSetings for extensions are now served on individual URLs to prevent cluttering.

Runtime web settings: allow customization of version string by extensions

Merge

Tests: more OCR tuning

Tests: better font for OCR in svghmi_basic

Runtime: fix non-interleaved worker loop

Win32 rutime: fix PLC Stop freezing

Fix untested code in worker.py...

Addapt to Py3.11.5 in Stunnel.py
b2a_hqx is now deprecated, base64 should bring comparable results

IDE: connect local runtime with '127.0.0.1' instead of 'localhost'
Workaround localhost possibly resolving as ::1. leading to problems.

Runtime/MainWorker: fix blocking race condition

syntax error

Update version.py

Runtime/win32: no sys.stdout with pythonw.exe

Tests: revert Teseract to less perforant legacy mode, but more repeatable. Compensate with zoom in edit_project and filter noise out of OCR text.

Tests: align on requirements.txt

Tests: Add minimal SVGHMI test

Tests: use SID+KILL for sikuli subprocesses

Strip now useless file.

SVGHMI: add support of all IEC61131 interger types.

- add HMI_*INT types
- fix sprintf.js with big int (64b)
- extend svghmi_real test project

Add testing status badge

Use requirements.txt and update doc accordingly.

Tests: pytests: only one wxpython app per process lifetime... Strip test_application.py

IDE: fix py3/wxpy4 related deprecation warnings

Runtime: remove now useless typechecking/convertion

Github CI: do not save docker image if just restored from cache

Runtime: fix unpacking of variable trace data

Github CI: update matiec revision

Github CI: update open62541 version, try avoid useles doker image rebuild

Update run_tests_in_docker.yml

Update .gitignore

SVGHMI Py3 related fix in test project.

Tests: have to terminate language test with KILL, side effect of BEREMIZ_TEST_CYCLES

Runtime: py3 fix forcing of STRING

Tests: fix systematically rebuilding matiec

Runtime: spawn_subprocess: handle non-bytes args

OPC-UA client : python3 + AsyncUA fixes

update .gitignore

Py3 related fixes for SVGHMI UI.

Fix Py3 problems with basic SVGHMI build and run.

Py3 usual encoding, ints and exceptions fixes.

Tests: Fix dockerfile for python3

Examples: use print() function in "python" example

Runtime: Fix IEC/ctypes/python3 type mapping

CLI: simpler status polling, fixing pyro5 thread

Fix bytes/string confusion with Logging

Fix wrong default target being picked in CLI build

Fake_wx objects now support comparison and hashing

Use msgpack and remove serpent's bytes workaround

Updaye .gitignore

IDE: Fix py_ext build with python3.10

Pyro5: fix runtime typo + IDE exception handling

Runtime: Fix web settings file upload.

Runtime: drop Athena. Nevow now only for settings.

Fix template conflict in XSLT with lxml>=4.9.0. Thanks to GP Orcullo !

Update generated XSLT headers (updated YSLT defs)

WIP python3 support for runtime

Better LDFLAGS and CFLAGS for win32 an MACOS

Fix private n matplotib function import

Update version.py

update .gitignore

IDE: more fixes for wxPython 4.0.7 and pyhon 3.10

IDE: python-3.10: wxPython calls don't accept float anymore, continued

IDE: fixed/remove checks wxPython versions

IDE: python-3.10: wxPython calls don't accept float anymore when int is expected

Fixes for wxPython-4.0.7 (ubuntu22) compatible with wxPython-4.1.x

fake_wx: handle missing classname

fix wxPyDeprecationWarnings

show underscore as space

CLIController: use absolute project path

allow passing environment variables to ProcessLogger

Generate warning instead of Exception if CTNMaxCount is exceeded for a confnode

fix name clash

fix str encoding issue

OSX: use wall time instead of system clock

replaced depreciated functions

fix platform identification when running CLI under Win32

silence compile warnings

Preliminary OSX support

pythonw is used on OSX

switched to Pyro5

TODO: add ssl support, WAMP

Add PLC stop command

wx: replaced deprecated class

wx fixes

str encoding fixes

ctypes fixes

fixed missing import

set executionOrderId default value to 0

replaced depreciated function

cmp is not used for sorting

upgrade fake_wx

Update to wx.NewIdRef()

fix for gc errors on shutdown

wx fixes for controls

Fix: wx._core.wxAssertionError: C++ assertion "CanBeToggled()" failed

fix - TypeError: unhashable type: 'Colour'

xmlclass.py: fix AttributeError

AbsDir fix

convert sort and cmp functions to Python3

switch to pickle from cPickle

fix dynamic module loading

fix ctypes encoding

fix string encoding

updated click api

remove six.reraise

removed Python2 modules

fixed plcopen imports

automated conversion using 2to3-3.9 tool

cmd used: 2to3-3.9 -w <file>

Tests: Add preliminary IEC-61131 language test, to be continued.

Test sets BEREMIZ_TEST_CYCLES in order to speed-up TON duration test.

Tests: Add time emulation feature for tests with BEREMIZ_TEST_CYCLES CFLAG.

Adding BEREMIZ_TEST_CYCLES=1000 in a project's CFLAGS will:
- run 1000 cycles with no pause
- emulate time flowing normaly for PLC code
- exit PLC thread

This allows:
- testing standard library blocks that deal with time without having to wait
- unit testing and code coverage with POUs that uses time

IDE: fix exception when re-opening previously closed tab

Same as commit with same message, this time for tabs that are not POUs editors.

IDE: fix again ruberband with gtk3.

DC logical functions are now disabled when using GTK3.
Apparently using XOR was still having an effect.
Use regular black pen with no logical funciton instead.

Merged from default

merged

Modbus web configuration: add check for valid delay and period parameters

IDE: Fix missing highlighted element in diagrams when using GTK3

- wx's logical function does not work on gtk3, and highlighting relies on it
- skipping logical funciion at all "works" but this now makes highlight opaque
- no easy fix for highlight transparency since emulated with AND Wx logical function
+ added other highlight related cosmetic fix.

Merged default in wxPython4 branch

Linux runtime: overrun detection for real-time timers and for plc execution.

If real-time timer wakes-up PLC thread too late (10% over period), then
warning is logged.

If PLC code (IO retreive, execution, IO publish) takes longer than requested
PLC execution cycle, then warning is logged, and CPU hoogging is mitigated
by delaying next PLC execution a few cylces more until having at least
1ms minimal idle time.

Fix typo in toolchain_gcc.py : LDLAGS -> LDFLAGS

Merged from default

Merge.

SVGHMI: add two more simple widgets to default widget library

Cosmetic fix in plc_debug.c template

Merge.

Modbus plugin: add optional delay between two consecutive modbus requests sent by master/client

Runtime: Add RealTime checkbox in Linux target options to enable PREEMPT_RT scheduling.

Runtime: move LC_NUMERIC locale setting to Beremiz_service.py.

Test: Customize Fluxbox launch to get rid of toolbar in IDE tests.

Tests: Use TESSERACT_LSTM_COMBINED, LSTM + Legacy OCR mode in Sikuli, more stable results.

Merged default in wxPython4 branch

Runtime: Fix bugs in Linux target.

- next occurence absolute time computation failing on 32-bit because of casting
- PLC thread was started before period was set, potentially passing uninitialized timespec to clock_nanosleep()

runtime: Change Linux target to use clock_nanosleep instead of timer (better rt-preempt perf).

Linux runtime: fix thread waiting on wait_RT_to_nRT_signal not being waken-up when delete_RT_to_nRT_signal is invoked.

Tests: docker image needs tesseract4, but opencv lib since sikuli.jar already contains it.

More explicit default value for generic makefile target command. Assume Makfile is located in project_files

IDE: prevent IDE/CLI process surviving to sys.exit() by using deamon threads to monitor external processes output streams.

Tests: allow overriding local host's hostname with BEREIS_LOCAL_HOST env variable.

Tests: IDE: explicitely wait for build success rather than just stdout to become silent.

Tests: Fixed wrong path (relative to $HOME) in CLI tests.

Tests: OPCUA tests can now use OPCUA_DEFAULT_HOST environment variable to force host to something else than "127.0.0.1" or "localhost"

IDE: fix segfault when unselecting network discovery in URI editor

IDE: fix exception when showing project's context menu in case of empty clipboard (wxpython4).

Tests: Fix dockerfile not finding locales.

cosmetic fix

Tests: use expedited termination of IDE with SIGTERM instead of normal close

Work around test not finishing when IDE ask for confirmation on exit when
project is modified.

IDE: When receiving SIGTERM signal, kill runtime befor bailing out without saving.

Purpose is to allow expedited test termination.

Merged default in wxPython4 branch

tests: IDE: own text matching logic for OCR reliability, enhanced ihtml report

SVGHMI: update generated XSLT files

examples: add "csv_read" project to demo CSV access from PLC

py_ext: add CSV file reading POUs in python extension library

SVGHMI: fix input widget sometime messing with font style

Merged default in wxPython4 branch

ST code generator: when including PLCopen TC6 XML libraries (for example py_ext/pous.xml or NativeLib.xml), also includes libraries globals in result

Runtime: add arbitrary file upload to working dir in web interface

SVGHMI: fix double-dispatch happening when using low widget update frequency on variable already being updated with high frequency by other widgets.

Merge

Backed out changeset ddadbdf20e70, no effect on current problem.

GH CI: try reverting to previous version of ubuntu

IDE tests: ensure that file browser idled before entering path in new_project

tests/tools: move comment and remove dead code in Dockerfile

tests: add tesseract training data "tessdata", still trying to enhance OCR reliability in UI tests

tests: attemps to make OCR based hits more stable

Tests: fix path logic in Docker creation script, consequence of previous commit.

Tests: cleanup and tidy docker definitions and scripts

Merged changes from default in wxPython4 branch

SVGHMI: fix XY graph init (consequence of earlier change) + reindent

Earlier change did modify the variables attributes (options) are stored in widget objects. XY graph uses variables' min and max to determine range.

SVGHMI: move declaration of "xmlns" javascript const so that widgets can use it.

This was broken in earlier commit, when changing initial execution order because of implementing websocket reconnect.

SVGHMI: add Page assignment use case in svghmi test project.

Marged default in wxPython4

SVGHMI: update generated xslt

SVGHMI: avoid Widget's member name colision in between enable expressions and assignments.

SVGHMI: Add arbitrary variable assignment when entering Pages

Usage similar to Assign widget.
Required instanciating PageWidgets for each page, so that assigned variables can be subsribed.

SVGHMI: update generated XSLT

SVGHMI: fix periodic reconnect triggering watchdog

IDE: Optimization of modification events processing in text editors.

Too many modifications types where registered, and then too many events were fired.
Also, in case of uninterrupted sequence of events, updates to the model is deferred to the end of that sequence (wx.Callafter).

SVGHMI: cosmetic change

SVGHMI: do not try to reconnect websocket if closed properly by server.

Tests: force bigger fonts and flat theme in GTK to make OCR more reliable.

OPC-UA: tolerate better pre-encryption formated client configuration file.

OPC-UA: only support the encryption policy selected in config.

By default open62541 client accepts all supported policies, but in makes problem
when negociating with some servers while most clients seems to only support
one policy at a time.

Tests: add encrypted OPCUA browsing test.

Tests: IDE: add stdout monitoring capabilities to auxiliary process

OPC-UA: Update CTN "ChangesToSave" status when variable selection change.

- fixes changes not being saved when no other changes happening to configuration
- added CTNMarkModified to ConfigTreeNode in order to generalize

Tests: Add project for OPC-UA server browsing and variable picking UI test

Tests: Add OPC-UA server browsing and veriable picking UI test.

Tests: Add 'dragNdrop' for UI tests with slower begining than sikuli's dragDrop

Tests: UI tests now avoid polling stdout if process didn't start.

Tests: UI tests can now load projects from tests/projects

Tests: add command for UI tests to launch and close Auxiliary Process

OPC-UA: better handling and reporting of server browser's connection errors.

OPC-UA: Extend Drag'N'Drop target to whole input or output panel not only grid.

Merge default in wxPython4

SVGHMI: fix HMI freeze at load when no screen saver is defined

SVGHMI: update generated XSLT

SVGHMI: watchdog now calls OnStart and OnStop commands just after calling OnWatchdog. Therefore, OnWatchdog command should not restart browser by itself.

SVGHMI: server systematically resets session on connect, rather than waiting for client to ask for it

IDE: Fix DropTarget() methods that must return a bool with wxPython4.

IDE: Fix visibility of text in Debug panel when using dark theme.

IDE: Fix use of deprecated wx.NamedColour

Merged changes from default in wxPython4 branch

SVGHMI: update generated XSLT

SVGHMI: use messy test/projects/svghmi to give a try to ScreenSaver feature.

SVGHMI: cosmetic fix in Back widget

SVGHMI: add ScreenSaver feature. Automatically jumps to a page named "ScreenSaver" after timeout defined in page arguments.

Backed out changeset c6068b674b30

SVGHMI: update generated XSLT

SVGHMI: enable periodical reconnect when browser is QtWebengine, working around memory leak with websocket on Qt's embedded Chromium.

IDE: Added filtering in POU instance variables panel.

SVGHMI: add automatic reconnection of websocket with 1s reconnection delay after first attempt.

SVGHMI: prevent browser and watchdog commands to become zombie once finished.

SVGHMI: svghmi_references exemple: add missing PLC part to demonstrate assigning PLC/HMI variables when validating a modal dialog.