py_ext: fix CSV Writer

fix POU logic :
- SAVE is a BOOL
- invocation of py_eval on rising edge of SAVE
- remove save python argument

fix python:
- use no encoding for file open (python2)
- re-use detected dialect if any
- use no "rt+" and truncate since no need to re-sniff dialect for output file
- return "OK" instead of "#SUCCESS", preventing POU logic to ACK result
- support creating new line if writing just after last line
- support appending data on short rows

fix example:
- use a HMI:Button to trigger CSV write instead of HMI:Input +1
- reload CSVs on on each new CSV opened in file browser
- add display of CSV write output

IDE: Fix wrong red wire coloring in FBD

C runtime: move generated erpc interface in separate "service" directory

IDE: Fix PLCOpenEditor (wxPython4 menus) and drop broken PDF doc support.

Extend HMI:JsonTable, create Edit CSV POU, create example, add some doc (#41)

* Extend Jsontable widget, create Edit CSV POU, create example for both

* Add doc for HMI:Image and HMI:JsonTable

* Expand example to include HMI:Image

Tests: matiec version bump for GH workflow

Merge remote-tracking branch 'hggit/python3' into python3

IDE: fix exception when moving FBD elements with keyboard.

Tests: matiec version bum for GH workflow

merge

Tests: extend FOR loop tests in IEC-61131 language test

Modbus: allows modbus thread stack size customization with MODBUS_STACK_SIZE macro.

For example, add "-DMODBUS_STACK_SIZE=0x2000" to target CFLAGS to limit stack size to 128kB

SVGHMI: swap position and range of ForEach widget so that both range and size can be optional.

Tests: add ST FOR loop tests to IEC-61131 language tests

requirements: update eRPC dependency to include subdirectory for Python

C runtime: fix ExtendedCall stub

eRPC: switch to beremiz/erpc fork with no CRC on TCP

Merge remote-tracking branch 'hggit/python3' into python3

IDE: Allow environment variable override in ThirdPartyPath, and use it also for matiec.

IDE: re-use eRPC URI definitions in connector factory.

Preparing for serial and US eRPC connection.

Cosmetic changes

IDE: refactor discovery panel / zeroconf

Bundle ZeroConf listener and interfaces changes monitor into separate module.
Prepare to extend with USB device browsing.

IDE: refactor ERPC URI editor, add serial and USB URI types

python runtime: update ServicePublisher to current Zeroconf API

eRPC: Backward compatibility with runtime versions missing GetVersions extended call.

eRPC: Version bump

python_runtime: fix ExtendedCall register/unregistering.

eRPC: Server closes connection on exception to prevent client to block until timeout when it happens.

Add ExtendedCall to PLCObject as a replacement of RemoteExec and reflect it in eRPC interface.

ExtendedCall is a generic remote procedure call that runtime code can register to and that IDE extensions can call.
For example a fieldbus extension can use ExtendedCall to browse the fieldbus devices connected to the runtime.

Removed dependency in requirements.txt.

TESTS: Update Matiec revision in GH workflow

MQTT: update test project to reflect latest changes

SVGHMI update generated XSLT

SVGHMI: remove widgets deprecated since long

SVGHMI: add ForEach widget example

SVGHMI: Extend ForEach widget to expose range, position and size in a way comparable with ScrollBar widget.

SVGHMI: code cleanup

MQTT: Add status global variable MQTT_STATUS_n, one per MQTT client

MQTT: strip unsupported types from type choice dropdown, better wrong type detection in build.

MQTT: remove debug prints

MQTT: add JSON fixed size 1D array support. WIP. Parsing array still broken.

MQTT: do not convert to UPPERCASE json keys.

MQTT: added support for strings in JSON payload only.

MQTT: remove accidentally committed debug printfs + cosmetic

MQTT: fix invocation of json_scanf (use pointers, not values...)

MQTT: fix bisection failing when topics start with same substring.

MQTT: Update available type choices when data types are changed in the IDE

MQTT: remove standalone MQTT clien generator code.

Code was inherited from OPC-UA extension, but not properly maintained.

MQTT: add JSON payload corresponding to IEC Structured types

- uses https://github.com/cesanta/frozen
- frozen.c is added as a resulting c file in generat_C (fix toolchain_gcc.py as it was breaking dependency checking)
- structured types are converted to C #define, in turn generating json_scanf and json_printf statements

MQTT: WIP supporting structured types : do not crash when generating C code if using location with no size

MQTT: add dropdown to select type assigned to a topic

MQTT+IDE: Allow user to associate IEC structure datatype to a topic.

IDE: allow structures to be located.

merge MQTT grafted from python2 branch - untested

MQTT: add test projects, to be later included in automated testing.

MQTT: add SSL support

SVGHMI: cosmetic

SVGHMI: Fix watchdog not cleanly restarting browser because not waiting for stop operation to finish before restarting.

MQTT: WIP, re-organized disconnection handling, now keep track of connection status and reconnect accordingly.

This fixes bug with non-reconnecting to broker when having no published variables.

MQTT: link to SSL only when needed.

IDE: Fix "Save" toolbar button and file menu entry not appearing when extension makes changes

MQTT: WIP fix modified status not being set when adding and modifying topics or attributes.

Fix loading of CSV that was not applying model types, and that not checking conformance either.

MQTT: WIP, PLC starts even if MQTT client can't connect broker, and publish+subscribe again when reconnecting.

MQTT: WIP, destroys paho client during failure handling at init and at cleanup. Use mutex to protect retrieve.

MQTT: Implements reconnecting in publish thread in case thread is waken-up but client is disconnected.

Note: paho's lostConnection callback got already disabled in previous commit.

MQTT: take C part away from python code for readability.

C lines change a bit because of unnecessary escaping for '\'.

MQTT: WIP, prevent crash after disconnect. Disable auto-reconnect on connection lost callback.

MQTT: Stop publish thread properly on PLC stop

MQTT: Fix overkill use of wx sizer

MQTT: Fix crash when browsing locations

MQTT: WIP. Added handling of received messages (subscriptions)

Dropped perfect hash in favor of bisection
Also fixed indentation (tabs)

MQTT: WIP, added publish thread waking-up when published variables change.

MQTT: WIP, subscibed topics have no "Retained" attribute.

Allow subscribed and published data models to be different, by removing "Retained" column.

MQTT: WIP, now publishes initial values at init.

MQTT: WIP, now connects to broker. Added MQTT 3 support and protocol version selection in IDE.

MQTT: (WIP) Fix OpenSSL linking.

MQTT: still WIP, generated C code builds and link.

Extensions: add optional arguments to ThirdPartyPath

ThirdPartyPath("name", "path1", "path2") will resolve by default into "../name/path1/path2"

GCC BUILD: Prevent Memory Error when compiling huge projects.

Also deleted dead code.

MQTT WIP: intermediate state, added perfect hash to map topic to variables

MQTT WIP: intermediate state, implementing MQTT client C code generator.

WIP: Add skeleton for MQTT extension based on part of OPC-UA extension.

For now generated code makes no sense but persistently-configurable-C-generating-extension infrastructure is there.

Merge remote-tracking branch 'origin/python3' into python3

C runtime: Fix PLCObject.hpp preventing build.

IDE: generalize EVT_GRID_CELL_CHANG{ING->ED} (WxPy4)

IDE: hopefully last fix about float type being passed to WxPy4.

IDE: enhance variable forcing contextual menu

IDE: fix completion

uses ctrl key on Mac
update internal variable list before listing them
avoid exception if empty list
apply selection when pressing return or enter

IDE: Workaround some reported WxPython/MacOS problems.

SetSelection should be used instead of SetStringSelection , and FindString must use case_sensitive=True

IDE: Fix search result not showing single occurrences

IDE: remove warning if editing location leads to no change

IDE: Use 8x8 grid by default

IDE: Fix exception when connecting complex types

LocalRuntime: no crash if zombie PLC prevents erasing workdir

IDE: Fix printing

More side effects of WxPy4 et Py3 + workaround WxWidget bug.

IDE: failsafe escaping for input not matching utf-8 encoding

Update Nevow revision in requirements.txt

Merge commit '2d061a66ddc737f5dffae82413b4f8573b7eee54' into HEAD

Add "CONTINUE" ST statement.

Matches "Add CONTINUE statement, fix FOR loop accordingly" 8a06b3bb92967b389466b8afa977a995ec605e2d commit in Matiec.

C runtime: cosmetic changes

merge again

Revert unwanted changes potentially breaking wxGlade support.
Partly revert 34d4c77ccb1bcb14601ab347ec6080466b7e143d. Skips evaluator related changes that was included by mistake.

Tests: add erpc source to GitHub tests

C runtime: add autoload

C runtime: fix logging, makes test pass

Tests: Add minimal C runtime test

merge

Xenomai runtime: report and mitigate when PLC cycle overruns.

Side effect :
_Log* macro was changed in plc_Xenomai_main.c,
and fixed in plc_Linux_main.c

SVGHMI: Do not generate Popen call when comamn is empty.

Fix python path in readme.

Python runtime: ensure that python thread finished before returning from StopPLC.

C runtime: initial fixes. Now runs and traces first_steps example.

beremiz_runtime -v -t tcp -p 61131 -h localhost

IDE: fix eRPC exception handling

All Runtimes: Now sizeof(tick)=4 on all platforms (unsigned int).

eRPC: update erpc version, add readme

C++ runtime: WIP. Continue PLCObject implementation.

Tests: fix wx_widgets.pytest/test_CustomIntCtrl.py

Merge commit 'e2df70b60b84031d3ccd3ae799a4e3aa8112f3ff' into python3

Doc: Add about section (PR #33)

* added about section

* updated .gitignore to exclude doc/_build/

ERPC: fix SetTraceVariablesList interface

This fix debug not suspending when having no variables, and other debug error being not reported.

C++ runtime: WIP: untested PLCObject implementation. Still missing tracing.

Update revisions used in tests running on GitHub.

merge

C++ runtime: add eRPC server, minimal CLI and Makefile. WIP.

More documentation. Work in progress.

IDE: Fix crashing non-thread safe code using Zeroconf

BACnet: Fixed grid cell choice editor. Consequence of wxPython4.

SVGHMI: fix font embedding with py3

IDE: Fix Variable grid ignoring edits. WxPython4 consequence.

IDE: Remove all occurrences of deprecated wx.grid.PyGrid

Merge commit '80e123b34e488ffea851dc2168b366ba6124c09b' into python3

Doc: start converting Standard Library documentation from old MC8 manual.

merge

Modbus: fix test: add missing filles and add Modbus source+build in GH CI

Modbus: add CI test + test project

Doc: re-organize existing, sketch outline, add SVGHMI intro.

Examples: add svghmi_json_table

SVGHMI: add "DISCARD:" prefix to force discarding an SVG element

SVG: fix HMI:Assign widget to properly handle single-letter variable names.

SVGHMI: Fix wrong documentation copy-pasted in HMI:Page.

SVGHMI: add FlatButton with no active or inactive elements.

SVGHMI: i18n: minimize warnings for incomplete/fuzzy/outdated translations

SVGHMI: i18n: fix .POT generation + other cosmetic fixes.

merge

Removed forgotten debug print.

SVGHMI: fix py3 consequences on i18n

merge

SVGHMI: Fix thumbnail generation.

Tests: update svghmi test project to py3 and erpc

IDE: Fix local runtime not starting on windows.
Consequence of updating ProcessLogger's argument handling.

Add credits for Czech i18n

Shows AboutDialog even if revisions.tx is missing

IDE: Fix encrypted ERPC: newer TLS, better error handling, re-add explicit ERPCS scheme, use sslpsk better.

merge

IDE: display contents of revisions.txt (put there by installer builder script) in About Dialog.

eRPC Server: fix exception at disconnect and reconnecting

Merge pull request #31 from martinpihrt/python3

Czech translation from Martin Pihrt

Create Beremiz_cs_CZ.po

Create Beremiz.mo

Fix SVGHMI build exception on windows: return bytes for inkscape path in docsvg.

SVGHMI: fix consequence of last change in ProcessLogger

Update requirements.txt

Remove forgotten debug prints

merge

Updates requirements.txt about forks of Nevow and sslpsk

Fix deprecated WX call producing warning.

Fix exception on right click, showing context menu.

GCC toolchain: Add {SYSROOT} substitution with support for whitespaces

Any instance of {SYROOT} string in config's CFLAGS and LDFLAGS is replaced by sysroot path obtained from "gcc -print-sysroot"

Fix argument lexer in ProcessLogger, replace it with shlex standard python module.

SVGHMI: Fix warning with WxPython > 4.x

docsvg: safer version checking of Inkscape

IDE. update genicons.sh to inkscape 1.1 command line

IDE: adjust Build and Clean icons colors.

Fix variable forcing. Now works with eRPC. PLCObject API changed.

Fix strings displaying as repr(bytes) in debug view.

Fix variable tracing in new eRPC interface.

Replace PYRO with ERPC. Work In Progress.

Fix blind replace ('text' into 'str') from py3 port

Fix settings URL: now '/' instead of '/settings'

Merge

SVGHMI: Fixes UI lifecycle problems

Was sometime showing HMI tree of previously closed project : no more module globals to store hmi tree.
C++ dead object exceptions happened when re-opening SVGHMI UI or building after close of UI.

IDE: Libraries in OrderedDict for access by name

SVGHMI: fix exception when stopping

SVGHMI: fix JS error loading empty single page

svg:svg tag should not be considered as a detachable group,
so filter only svg:g elements

IDE: fix exception showing network discovery

SVGHMI: Fix creation of SVG file if none provided

IDE: Auto-select required libraries when adding extension

IDE: Disable not-yet-working extensions

Fix typo in README.md

IDE: py3.10 related float/int fixes in debug plots

Move tests/project/modbus in exemples.

Python3 fixes for Modbus Web Settings

Fix template conflict in XSLT with lxml>=4.9.0 again

Earlier attempt was fixing conflict with -1.0 priority attribute,
as a side effect of such low priority, SFC Actions were not
editable anymore.

This time move template around instead of using explicit priority.

Revert "IDE: fix exception when adding Action"

This reverts commit 94e42cda0a8e093aac4830b5c40656c240ec8cf9.

IDE: fix exception when adding Action

IDE: Fix status bar update when conection lost

SVGHMI: Fix watchdog exception on windows

Typo

Upgrade wxPython to 4.2.1, drop 4.2.0 workaroud

Runtime: add GetVersions() call to PLCObject, use it in web settings and expose it in WAMP and Pyro.

Note: WAMP changes are untested

Runtime: Add stderr support to local posix spawn based subprocess replacement

Modbus: More verbose error message in case of thread creation problem

Runtime web settings: fix punctuation and update CSS again

Update default CSS for Runtime WebSettings.

Runtime WebSetings for extensions are now served on individual URLs to prevent cluttering.

Runtime web settings: allow customization of version string by extensions

Merge

Tests: more OCR tuning

Tests: better font for OCR in svghmi_basic

Runtime: fix non-interleaved worker loop

Win32 rutime: fix PLC Stop freezing

Fix untested code in worker.py...

Addapt to Py3.11.5 in Stunnel.py
b2a_hqx is now deprecated, base64 should bring comparable results

IDE: connect local runtime with '127.0.0.1' instead of 'localhost'
Workaround localhost possibly resolving as ::1. leading to problems.

Runtime/MainWorker: fix blocking race condition

syntax error

Update version.py

Runtime/win32: no sys.stdout with pythonw.exe

Tests: revert Teseract to less perforant legacy mode, but more repeatable. Compensate with zoom in edit_project and filter noise out of OCR text.

Tests: align on requirements.txt

Tests: Add minimal SVGHMI test

Tests: use SID+KILL for sikuli subprocesses

Strip now useless file.

SVGHMI: add support of all IEC61131 interger types.

- add HMI_*INT types
- fix sprintf.js with big int (64b)
- extend svghmi_real test project

Add testing status badge

Use requirements.txt and update doc accordingly.

Tests: pytests: only one wxpython app per process lifetime... Strip test_application.py

IDE: fix py3/wxpy4 related deprecation warnings

Runtime: remove now useless typechecking/convertion

Github CI: do not save docker image if just restored from cache

Runtime: fix unpacking of variable trace data

Github CI: update matiec revision

Github CI: update open62541 version, try avoid useles doker image rebuild

Update run_tests_in_docker.yml

Update .gitignore

SVGHMI Py3 related fix in test project.

Tests: have to terminate language test with KILL, side effect of BEREMIZ_TEST_CYCLES

Runtime: py3 fix forcing of STRING

Tests: fix systematically rebuilding matiec

Runtime: spawn_subprocess: handle non-bytes args