Merge with Donggu modifications

Merge with Donggu modifications

Adding support for drag'n dropping located variables from topology panel to configurations and resources variable panel for declaring global located variables

Master relase code added

Master release code added

Adding support for not configuring pdos when not mandatory and not needed by locations defined in PLC program. Adding support for displaying locations tree in Topology panel and BrowseLocationsDialog. Merging pdos grid and variables grid into one single grid in slave infos panel.

First working implementation of Beremiz plugin for etherlab

merge

Add BACnet extension from Mario de Sousa <msousa@fe.up.pt>

Make Modbus extension license compatible with the rest of Beremiz IDE python code : GPLv2.

Commented out safety check to prevent deadlock in wxEvaluator, since wx Mainloop is not anymore in main thread. That safety check actually leads to having wx code running outside of MainLoop

Merge

Print version information about software used for tests

Fix codestyle

Fix typos introduced by 9fa2f8e "Fixed random segfault happening when loading new PLC in runtime, when using Xenonai."

merged changes in README.md

add python-cycler dependence in readme

Make Modbus mutexes non blocking for PLC. If data related to some modbus request is alredy locked when PLC wants to access it, then data isn't copied to/from PLC

Merged #2476, URI dialog fixes

close branch #2476 - URI dialog fixed

#2476 Changes to the comment.

#2476 Connectors dialog module added.

#2476 Separate connectors dialogs from connectors module.

#2476 Additional check added if we can import wx.

Allow deleting configuration tree node with no corresponding directory in project tree

Support unicode in LDFLAGS returned by extensions

Fixed typos

#2476 Added uri location editor, getting connectors from list in Beremiz.

Fixed random segfault happening when loading new PLC in runtime, when using Xenonai.

PEP-8 and PyLint

merged Denis change to apply input as we type in CTN configuration forms text fieds

PEP-8 and PyLint

Add safety check to prevent exception when disconnecting while debugging

Removed DebugThread. Take advantage of the fact that buffering is done in runtime. No need to poll for traces more than ten per second, then use simple wxTimer for that. Also removed Debug Lock since now everything happens in wx' mainloop.

Rework of runtime non-real-time threading, and shared object dynamic loading :
- All exposed operations on PLCObject are now serialized through main thread
(@RunInMain). Only one exception : python trace thread calling _GetDebugData
- Re-loading of same shared object is prevented. dlclose/delte/rewrite/dlopen
don't happen in that case. This is a workaround for some xenomai copperplate
bug triggering segfault, inbetween delete and rewrite.
- Trace thread now waits for data, dies if no data, and dies instead of
suspending as before when no data consumed after 3 seconds.
- Disabled threading in Pyro client
- Enforced auxiliary threads starting sequence : Pyro -> [UI] ->
PLCWorker.RunLoop

Stripped forgotten debug code

close branch

#2476 Fix for saving text in URI edit text box. No need to click out of URI edit text box to save edited text.

Fixed Xenomai 3 PLC stop freeze. Now use explicit finish command with pipes. Closing both ends of pipes doesn't abort blocking read anymore.

More canonical test.
There is an exception happening around here in some cases when connection is dropped while waiting for answer. Not sure that this fixes it but this is anyhow better.

PLCObject got more methods serialized through worker : Start, Stop, NewPLC.
This particularly helps with initialization/startup/re-load problems
when targeting Xenomai. It should help in general for runtime robustness,
since those operations aren't meant to be executed concurrently.

Fixed implementation of runtime worker that ensure that PLCObject Load and Unload methods always run main thread.

merged

merged

Intermediate state while implementing runtime worker to ensure that PLCObject Load and Unload methods always run main thread.

Stripped more dead code.

Cleanup of some dead code in PYRO connector.

Added logging in Xenomai init phase.

Apply new RT Pipes API from Xenomai 3

Removed useless try/except in Canfestival.py

#2579 On showing search result, text color of tree item is set to white. We need to change back color to show the result.

PEP-8...

Fixed bug that was creating many frames at starup on Windows

re-merge

Xenomai 3

merged Sergeys changes

fix bug with indelible values in some 'optional' fields in project properties panel

only 'required' fields should be indelible

make the URI model compatible with https

add address validation in Company URL field

remove duplicating code for variables name generate

add fields autocomplete when adding task or instance in resources

values will be copied from previous row, first row fills with the default values

fix search in TextViewer objects, search returns only first entry

side effect of a63bb40

fix import order according to PEP8

fix default time range choice for debug variable panel

only localized versions were affected

enlarge canvas border for debug variable graphic viewer

fix rounding milliseconds from nanoseconds

example:rounding 299999 ns to 300ms, if division result is an
integer, remainder will be lost, and result will be 299.

fix display TIME type variables in debug variable panel

fix for test app from skv. thanks

Prevent catching exception if exception handling was not enabled (or disabled by test test app).

pep8 and pylint hate me

Fixed typo introduced in 5736d25bb393

GetEditedElementInstancesInfos now optimized as well. Forgotten because not using [R.I.P]LibraryResolver, but still XSLT compiling was hapenning on each call.

Another attempt to make BitBuket's version of pylint happy.

Attempt to make BitBuket's version of pep8 and pylint happy.

Removed dead slow code

PEP8 and PyLint conformance: whitespaces and stuff

Copy pasted, no time wasted !

GetVariableDictionary and GetPouInterfaceReturnType optimized with new VariableInfoCollector based on XSLTModelQuery. Moved corresponding definitions out of PLCControler.py.

GetPouInstanceTagName optimized with XSLTModelQuery.

Better writing of some xpath added in previous commits about XSLT optimization.

Moved some definitions away from controller class, and adaped references them through all code.

Hook Beremiz Launcher AppStart to AdvancedSplash OnPaint event in order to ensure image is beeing displayed before loading.

Avoid endline escaping in multiline xpath epressions in xslt. '&#10;' Is apparently ignored but never know...

Fixed earlier XSLT optimizations. Some/most results were missing.

Moved some code from PLCController.py to other modules. Added necessary imports.

GetPouVariables optimized with XSLTModelQuery

Added /plcopen/InstancesPathCollector.py, to move more model xslt querying code away from PLCController.py.

Fixed Exception dialog disapearing after a view second when exception occurs during app startup.
Problem was caused by splashscreen timeout, wx closing averything else if there is no more main frame.
Changes:
- no more timeout for splashscreen
- use wx.App OnInit method to give first operation to mainloop, object are then now created in mainloop
- main loop is then created _before_ showing splash screen
- no more wxyield or wx processevent tricks needed to display splash screen
- exception handler not blocking anymore on dialog (callafter)
- because of mainloop being there before everything, exit must be called explicitely if exception caught during startup -> exit parameter in handle_exception + try/except around startup calls

UNTESTED ON WINDOWS

Moved XSLTModelQuery class to own module in plcopen package.

Added a class to speedup all XSLT queries. WIP, now only for instances_path.xslt / GetInstanceList()

Stripped useless node creation in instances_path stylesheet.

This should really help GetInstanceList to perform better: resolver removed, no more deepcopy, no more useless xml serialization, no more node-set() calls.

YSLT auto indent trick is useless here, added yslt_noindent.yml2, regenerated XSLT files

Prevent compiling XSLT on each call of GetInstanceList. To be continued. More optimization needed here. 100x would be fine...

Changed meaning of -c commandline argument for Beremiz service : only gives DEFAULT wamp config.
If existing project already has a wamp config, it is taken instead.

Factorized duplicated code

Better variable naming, and small optimization.

Unclutter graphics package scope.
Moved some import statements from "graphics" package to "viewer" module.
It also avoid systematic import of everything in graphics when we only need one submodule.

Removed dead code

Fixed pyro timeout for runtime, was set too short and runtime was disconnecting when building PLC

Ask to stop PLC before transfer if it is running

Factorized some repeated refresh code into a method, to allow customisation.

Launcher : add a debug attribute, instead of BMZ_DBG, so that customization can use it.

Stripped old dead code.

More changes in Launcher. Modules to be loaded in the background are given in a list, and assigned as object attributes. Also splitted CreateUI into CloseSplashScreen and CreatueUI.

ClassImporter now accepts string, callable or class, for compatibility with some old extensions.

Launcher: ShowUI call moves from preStart to start. ShowUI() split into CreateUI() and ShowUI(). Start()'s arguments are passed to CreateUI()
This is meant to allow customizations to pass own pre-instanciated controller to UI at startup.

Launcher: allows customizations to specify what globals passed to extensions

Documentation

Another set of meaningless changes to satisfy PEP8 and PyLint.

PEP8 conformity for modbus. Most of it done by autopep8, plus some easy refactoring.

Add some class factory function to ease declaration of simple POU libraries in customizations (i.e. TC6 XML file only, no special C code generation)

Small pep8 conformity enforcement.

Better have globals passed to python extensions (-e) than locals.

launcher : move call to ShowSplashScreen and BackgroundInitialization to PreStart()
It doesn't change behaviour of launcher in beremiz, but may have some impact on some customizations.
Some customizations need to squeeze some bitmap folder init in between app creation and ShowSplashScreen

modbus extension code cleanup, from smarteh's patch

Merged Mario's modbus branch. Fixed line endings.

merge

Ignore bug report files.

Add support for Modbus (TCP and RTU) working as master & slave

merge

fixed erro introduced 2 commits ago (-e) : tuples have no reverse() method

Trying to get better logging/display of exceptions at runtime's startup

Changed way to deal with extensions arguments (-e) so that path and filename are stored independently and accessible for extensions themselves.

Textviewer : moved call to Colourise in a better place, so that ST highlighting gets also updated when typing in some case, not only IL

Main runtime template C code : Added HAVE_RETAIN preprocessor definition for customized build to signal it provides IEC-61131 Retain memory handling function. Removed targets/Xenomai/plc_Xenomai_noretain.c, now useless.

Clarify licensing, and packaging of runtime only files :
- moved and split targets/typemapping.py into runtime/typemapping.py and runtime/loglevels.py
- added runtime_files.list pointing to files necessary for runtime, and referring to runtime license

Enforced check_source.sh pep8 and pylint rules.

runtime/WAMP: Fixed secret loading into configuration. Fixed secret commandline switch handling.

runtime/WAMP : was 'Added time print on session left.' from dporopat. Purpose is to log (dis/re)connection, to help debugging connection problem

runtime/Wamp : adapt to unicode being enforced in latest crossbar API. Updated some documentation, to explain why there is unused definitions like SubscribedEvents or DoOnJoin.

Extend python<->PLC global variable description "GlobalDesc" to support Initial value setting.

Rebased from dporopat change "add initial value to runtime.py file", which means :
- GlobalDesc is used by some customized beremiz python runtime
- Those runtimes need to know what is initial value of these special variables

update tests/wamp README file, based on changes from dporopat.

Fixed WAMP test (crossbar confile upgrade and use same ID on both PLC and IDE side)

Fixed changes from dporopat and agregorcic about WAMP CRA and config.

WAMP runtime : changes from dporopat and agregorcic, adding CRA (-s) and changed way to load WAMP conf (-c)
- added Challenge Response Authentication,
- use -s commandline switch to give the shared secret.
- WAMP configuration now either given as -c commandline switch
or taken as default wampconf.json in project files.

Set WAMP registration option so that reconnecting while earlier session still didn't timeout is possible (in case of connection lost, for example).

More canonical overload of ReconnectingClientFactory (according twisted doc)

non significant print syntax change. ready for py3.

whitespace cosmetic for pep8

Added arbitrary timeout on Pyro, to force it to free memory after disconnect.

Moved code from to Beremiz_service.py Server class Start method to Loop method to allow customisation by hot patching. Also renamed Stop method as _stop and use Restart instead of Stop from everywhere else to make it clear that _stop actually does restart...

enforce PEP8 in previous commit

replaced normal panel with scrolled panel under project properties

fix missing root in empty search results panel

add possibility to specify subprocess stdout encoding

fix deadlock if method kill() is called after process is already finished

fix pylint py3k conversion warning: "(no-absolute-import) import missing `from __future__ import absolute_import`"

make CanFestival to be at the expected place in bitbucket pipelines container

fix pylint error '(unsubscriptable-object) Value 'X' is unsubscriptable'

fix pylint warning '(bad-continuation) Wrong hanging indentation before block'

force all python source code to have LF line ending

fix pylint warning '(expression-not-assigned) Expression "X" is assigned to nothing'

fix pylint warning '(redefine-in-handler) Redefining name 'X' from outer scope (line Y) in exception handler'

fix displaying result of update check during Beremiz startup

fix pylint warning '(no-init) Class has no __init__ method'

fix pylint error "(undefined-variable) Undefined variable 'X'"

fix problem happend when tests are running under python-coverage

$ python-coverage run ./tests/tools/test_application.py

Exception while connecting LOCAL://!
Traceback (most recent call last):
File "/home/developer/WorkData/PLC/beremiz/beremiz/ProjectController.py", line 1753, in _Connect
self._SetConnector(connectors.ConnectorFactory(uri, self)
File "/home/developer/WorkData/PLC/beremiz/beremiz/ProjectController.py", line 1702, in _SetConnector
if connector is not None:
PyAssertionError: C++ assertion "m_window" failed at ../src/gtk/dcclient.cpp(2043) in DoGetSize(): GetSize() doesn't work without window

fix pylint error '(too-many-function-args) Too many positional arguments for function call'

fix pylint error '(import-error) Unable to import X'

fix pylint error '(no-self-argument) Method should have "self" as first argument'

fix pylint error "(no-name-in-module) No name 'X' in module 'Y'"

fix pylint error '(bad-except-order) Bad except clauses order (X is an ancestor class of Y)'

fix pylint warning "(no-value-for-parameter) No value for argument 'X' in function call"

fix problems with unconnected input of InOut function variables and
enfoce InOut variables to be connected for functions and function blocks

fix error for functions with VAR_IN_OUT

The problem was found by pylint '(used-before-assignment) Using variable "X" before assignment'

fix pylint warning '(unidiomatic-typecheck) Using type() instead of isinstance() for a typecheck'

fix pylint warning '(misplaced-bare-raise) The raise statement is not inside an except clause'

fix format string argument error found by pylint '(too-many-format-args) Too many arguments for format string'

fix pylint warning '(wrong-import-position) Import "import X" should be placed at the top of the module'

fix pylint warning '(import-self) Module import itself'

remove old code useless code found by pylint "(unnecessary-pass) Unnecessary pass statement"

fix error found by pylint 'W0104(pointless-statement) Statement seems to have no effect'

fix pylint warning "(undefined-loop-variable) Using possibly undefined loop variable 'X'"

fix pylint warning 'C0112(empty-docstring) Empty function docstring'

fix pylint warning "(relative-import) Relative import 'Y', should be 'X.Y'"

fix pylint warning '(dangerous-default-value) Dangerous default value {} as argument'

fix pylint warning '(old-style-class) Old-style class defined.'

This warning is ignored for classes from PyJSLib, because library
overrides base object class.

fix pylint warning '(unused-import), Unused import connectors'

fix typos

comment out unused code SFC_Step.RefreshBoundingBox()

explicitly mark unused variables found by pylint with _ or dummy

pylint don't ignores unused variables with '_' in the name, even when
they are started with '_'. This bug is already fixed upstream, to fix
this in all versions of pylint custom dummy-variables-rgx is used
'_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy'

remove unused variables found by pylint

update .PHONE target in Makefile for mki18n.py

make mki18n return 0 on success

update list with files for internatialization, app.fil

move README content to Makefile

It's not necessary to copy commands from README anymore. Just use
'make source_list', 'make template' or 'make locales'.

fix pylint warning '(global-variable-not-assigned) Using global for 'X' but no assignment is done'

fix pylint error '(function-redefined) method already defined'

fix pylint warning '(unreachable) Unreachable code'

fix pylint warning (bad-indentation) Bad indentation. Found X spaces, expected Y

fix pylint warning '(pointless-string-statement) String statement has no effect'

Python doesn't support docstring for variables directly. Where
appropriate Sphinx style for docstrings for variables is used
'#: some-docstring'