1

     2

     3

     4

  MATIEC - IEC 61131-3 compiler

     5

     6

     7

  The following compiler has been based on the

     8

  FINAL DRAFT - IEC 61131-3, 2nd Ed. (2001-12-10)

     9

    10

    11

  Copyright (C) 2003-2012  Mario de Sousa (msousa@fe.up.pt)

    12

    13

    14

****************************************************************

    15

****************************************************************

    16

****************************************************************

    17

*********                                              *********

    18

*********                                              *********

    19

*********         O V E R A L L    G O A L S           *********

    20

*********                                              *********

    21

*********                                              *********

    22

****************************************************************

    23

****************************************************************

    24

****************************************************************

    25

    26

    27

    28

 This project has the goal of producing an open source compiler for the programming languages defined

    29

in the IEC 61131-3 standard. These programming languages are mostly used in the industrial automation 

    30

domain, to program PLCs (Programmable Logic Controllers).

    31

    32

 This standard defines 5 programming languages:

    33

     - IL : Instructtion List

    34

            A textual programming language, somewhat similar to assembly.

    35

     - ST : Structured Text

    36

            A textual programming language, somewhat similar to Pascal.

    37

     - FBD: Function Block Diagram

    38

            A graphical programming language, somewhat similar to an electrical circuit diagram based on small 

    39

             scale integration ICs (Integrated Circuits) (counters, AND/OR/XOR/... logic gates, timers, ...).

    40

     - LD : Ladder Diagram

    41

            A graphical programming language, somewhat similar to an electrical circuit diagram based on 

    42

             relays (used for basic cabled logic controllers).

    43

     - SFC: Sequential Function Chart

    44

            A graphical programming language, that defines a state machine, based largely on Grafcet.

    45

            (may also be expressed in textual format).

    46

    47

 Of the above 5 languages, the standard defines textual representations for IL, ST and SFC. 

    48

It is these 3 languages that we target, and we currently support all three, as long as they are

    49

expressed in the textual format as defined in the standard.

    50

    51

 Currently the matiec project generates two compilers (more correctly, code translaters, but we like

    52

to call them compilers :-O ): iec2c, and iec2iec

    53

    54

 Both compilers accept the same input: a text file with ST, IL and/or SFC code.

    55

    56

 The iec2c compiler generates ANSI C code which is equivalent to the IEC 61131-3 code expressed in the input file.

    57

    58

 The iec2iec compiler generates IEC61131-3 code which is equivalent to the IEC 61131-3 code expressed in the input file.

    59

This last compiler should generate and output file which should be almost identical to the input file (some formating 

    60

may change, as well as the case of letters, etc.). This 'compiler' is mostly used by the matiec project contributors

    61

to help debug the lexical and syntax portions of the compilers.

    62

    63

    64

    65

 To compile/build these compilers, just

    66

$./configure; make

    67

    68

    69

    70

    71

    72

****************************************************************

    73

****************************************************************

    74

****************************************************************

    75

*********                                              *********

    76

*********                                              *********

    77

*********   O V E R A L L    A R C H I T E C T U R E   *********

    78

*********                                              *********

    79

*********                                              *********

    80

****************************************************************

    81

****************************************************************

    82

****************************************************************

    83

    84

 The compiler works in 4(+1) stages:

    85

 ==================================

    86

 Stage 1    - Lexical analyser       - implemented with flex (stage1_2/iec_flex.ll)

    87

 Stage 2    - Syntax parser          - implemented with bison (stage1_2/iec_bison.yy)

    88

 Stage pre3 - Populate symbol tables - Symbol tables that will ease searching for symbols in the abstract symbol tree.

    89

 Stage 3    - Semantics analyser     - currently does type checking only

    90

 Stage 4    - Code generator         - generates ANSI C code

    91

    92

 Stage 5    - Binary code generator - gcc, javac, etc... (Not integrated into matiec compiler. Must be called explicitly by the user.)

    93

    94

    95

    96

 Data structures passed between stages, in global variables:

    97

 ==========================================================

    98

   1->2   : tokens (int), and token values (char *) (defined in stage1_2/stage1_2_priv.hh)

    99

   2->1   : symbol tables (implemented in util/symtable.[hh|cc], and defined in stage1_2/stage1_2_priv.hh)

   100

   2->3   : abstract syntax tree (tree of C++ objects, whose classes are defined in absyntax/absyntax.hh)

   101

pre3->3,4 : global symbol tables (defined in util/[d]symtable.[hh|cc] and declared in absyntax_utils/absyntax_utils.hh)

   102

   3->4   : abstract syntax tree (same as 2->3), but now annotated (i.e. some extra data inserted into the absyntax tree)

   103

   104

   4->5   : file with program in c, java, etc...

   105

   106

   107

   108

   109

 The compiler works in several passes:

   110

 ====================================

   111

   112

Stage 1 and Stage 2

   113

-------------------

   114

  Executed in one single pass. This pass will:

   115

  - Do lexical analysis

   116

  - Do syntax analysis

   117

  - Execute the absyntax_utils/add_en_eno_param_decl_c visitor class

   118

    This class will add the EN and ENO parameter declarations to all

   119

    functions that do not have them already explicitly declared by the user.

   120

    This will let us handle these parameters in the remaining compiler just as if

   121

    they were standard input/output parameters.

   122

   123

   124

Stage Pre3

   125

----------

   126

  Executed in one single pass. This pass will populate the following symbol tables:

   127

   - function_symtable;            /* A symbol table with all globally declared functions POUs. */

   128

   - function_block_type_symtable; /* A symbol table with all globally declared functions block POUs. */

   129

   - program_type_symtable;        /* A symbol table with all globally declared program POUs. */

   130

   - type_symtable;                /* A symbol table with all user declared (non elementary) datat type definitions. */

   131

   - enumerated_value_symtable;    /* A symbol table with all identifiers (values) declared for enumerated types. */

   132

   133

   134

Stage 3

   135

-------

   136

  Executes two algorithms (flow control analysis, and data type analysis) in several passes.

   137

   138

  Flow control:

   139

     Pass 1: Does flow control analysis (for now only of IL code) 

   140

             Implemented in -> stage3/flow_control_analysis_c

   141

             This will anotate the abstract syntax tree 

   142

             (Every object of the class il_instruction_c that is in the abstract syntax tree will have the variable 'prev_il_instruction' correctly filled in.)

   143

   144

  Data Type Analysis

   145

     Pass 1: Analyses the possible data types each expression/literal/IL instruction/etc. may take 

   146

             Implemented in -> stage3/fill_candidate_datatypes_c

   147

             This will anotate the abstract syntax tree 

   148

             (Every object of in the abstract syntax tree that may have a data type, will have the variable 'candidate_datatypes' correctly filled in.)

   149

     Pass 2: Narrows all the possible data types each expression/literal/IL instruction/etc. may take down to a single data type 

   150

             Implemented in -> stage3/narrow_candidate_datatypes_c

   151

             This will anotate the abstract syntax tree 

   152

             (Every object of in the abstract syntax tree that may have a data type, will have the variable 'datatype' correctly filled in.

   153

              Additionally, objects in the abstract syntax tree that represen function invocations will have the variables 

   154

              'called_function_declaration', 'extensible_param_count' and 'candidate_functions' correctly filled in.

   155

              Additionally, objects in the abstract syntax tree that represen function block (FB) invocations will have the variable

   156

              'called_fb_declaration' correctly filled in.)

   157

     Pass 2: Prints error messages in the event of the IEC 61131-3 source code being analysed contains semantic data type incompatibility errors.

   158

             Implemented in -> stage3/print_datatype_errors_c

   159

   160

   161

Stage 4

   162

-------

   163

  Has 2 possible implementations.

   164

   165

  iec2c  :  Generates C source code in a single pass (stage4/generate_c).

   166

  iec2iec:  Generates IEC61131 source code in a single pass (stage4/generate_iec).

   167

   168

   169

   170

   171

   172

   173

****************************************************************

   174

****************************************************************

   175

****************************************************************

   176

*********                                              *********

   177

*********                                              *********

   178

*********               N O T E S                      *********

   179

*********                                              *********

   180

*********                                              *********

   181

****************************************************************

   182

****************************************************************

   183

****************************************************************

   184

   185

   186

   187

   188

   189

   190

 NOTE 1

   191

 ======

   192

 Note that stage 2 passes data back to stage 1. This is only

   193

possible because both stages are executed in the same pass.

   194

   195

   196

   197

 NOTE 2

   198

 ======

   199

 It would be nice to get this parser integrated into the gcc

   200

group of compilers. We would then be able to compile our st/il

   201

programs directly into executable binaries, for all the processor

   202

architectures gcc currently supports.

   203

 The gcc compilers are divided into a frontend and backend. The

   204

data structure between these two stages is called the syntax

   205

tree. In essence, we would need to create a new frontend that

   206

would parse the st/il program and build the syntax tree.

   207

Unfortunately the gcc syntax tree is not very well documented,

   208

and doing semantic checking on this tree would probably be a

   209

nightmare.

   210

 We therefore chose to follow the same route as the gnat (ada 95)

   211

and cobol compilers, i.e. generate our own abstract syntax tree,

   212

do semantic checking on our tree, do whatever optimisation

   213

we can at this level on our own tree, and only then build

   214

the gcc syntax tree from our abstract syntax tree.

   215

 All this may still be integrated with the gcc backend to generate

   216

a new gnu compiler for the st and il programming languages.

   217

Since generating the gcc syntax tree will probably envolve some

   218

trial and error effort due to the sparseness of documentation,

   219

we chose to start off by coding a C++ code generator for

   220

our stage 4. We may later implement a gcc syntax tree generator

   221

as an alternative stage 4 process, and then integrate it with

   222

the gcc toplevel.c file (command line parsing, etc...).

   223

   224

   225

   226

****************************************************************

   227

****************************************************************

   228

****************************************************************

   229

*********                                              *********

   230

*********                                              *********

   231

*********               S T A G E      1               *********

   232

*********                                              *********

   233

*********                                              *********

   234

****************************************************************

   235

****************************************************************

   236

****************************************************************

   237

   238

   239

   240

Issue 1

   241

=======

   242

   243

 The syntax defines the common_character_representation as:

   244

<any printable character except '$', '"' or "'"> | <escape sequences>

   245

   246

 Flex includes the function print_char() that defines

   247

all printable characters portably (i.e. whatever character

   248

encoding is currently being used , ASCII, EBCDIC, etc...)

   249

Unfortunately, we cannot generate the definition of

   250

common_character_representation portably, since flex

   251

does not allow definition of sets by subtracting

   252

elements in one set from another set (Note how

   253

common_character_representation could be defined by

   254

subtracting '$' '"' and "'" from print_char() ).

   255

This means we must build up the defintion of

   256

common_character_representation using only set addition,

   257

which leaves us with the only choice of defining the

   258

characters non-portably...

   259

   260

 In short, the definition we use for common_character_representation

   261

only works for ASCII character encoding!

   262

   263

   264

   265

   266

Issue 2

   267

=======

   268

   269

We extend the IEC 61131-3 standard syntax to allow inclusion of 

   270

other files. The accepted syntax is:

   271

   272

   {#include "<filename>" }

   273

   274

We use a pragma directive for this (allowed by the standard itself),

   275

 since it is an extension of the standard. In principle, this would 

   276

be ignored by other standard complient compilers!

   277

   278

   279

   280

****************************************************************

   281

****************************************************************

   282

****************************************************************

   283

*********                                              *********

   284

*********                                              *********

   285

*********               S T A G E      2               *********

   286

*********                                              *********

   287

*********                                              *********

   288

****************************************************************

   289

****************************************************************

   290

****************************************************************

   291

   292

 Overall Comments

   293

 ================

   294

   295

   296

 Comment 1

   297

 ---------

   298

 We have augmented the syntax the specification defines to include

   299

restrictions defined in the semantics of the languages.

   300

   301

 This is required because the syntax cannot be parsed by a LALR(1)

   302

parser as it is presented in the specification. Many reduce/reduce

   303

and shift/reduce conflicts arise. This is mainly because the parser

   304

cannot discern how to reduce an identifier. Identifiers show up in

   305

many places in the syntax, and it is not entirely possible to

   306

figure out if the identifier is a variable_name, enumeration

   307

value, function block name, etc... only from the context in

   308

which it appears.

   309

   310

 A more detailed example of why we need symbol tables are

   311

the type definitions...  In definition of new types

   312

(section B 1.3.3) the parser needs to figure out the class of

   313

the new type being defined (enumerated, structure, array, etc...).

   314

This works well when the base classes are elementary types

   315

(or structures, enumeration, arrays, etc. thereof). It becomes

   316

confusing to the parser when the new_type is based on a previously

   317

user defined type.

   318

   319

TYPE

   320

  new_type_1 : INT := 99;

   321

  new_type_2 : new_type_1 := 100;

   322

END_TYPE

   323

   324

 When parsing new_type_1, the parser can figure out that the

   325

identifier new_type_1 is a simple_type_name, because it is

   326

based on a elementary type without structure, arrays, etc...

   327

 While parsing new_type_2, it becomes confused how to reduce

   328

the new_type_2 identifier, as it is based on the identifier

   329

new_type_1, of which it does not know the class (remember, at this

   330

stage new_type_1 is a simple identifier!).

   331

 We therefore need to keep track of the class of the user

   332

defined types as they are declared, so that the lexical analyser

   333

can tell the syntax parser what class the type belongs to. We

   334

cannot use the abstract syntax tree itself to search for the

   335

declaration of new_type_1 as we only get a handle to the root

   336

of the tree at the end of the parsing.

   337

   338

 We therefore maintain an independent and parallel table of symbols,

   339

that is filled as we come across the type delcarations in the code.

   340

Actually, we ended up also needing to store variable names in

   341

the symbol table. Since variable names come and go out of scope

   342

depending on what portion of code we are parsing, we sometimes

   343

need to remove the variable names from the symbol table.

   344

Since the ST and IL languages only have a single level of scope,

   345

I (Mario) found it easier to simply use a second symbol table for

   346

the variable names that is completely cleared when the parser

   347

reaches the end of a function (function block or program).

   348

   349

What I mean when I say that these languages have a single level

   350

of scope is that all variables used in a function (function block

   351

or program) must be declared inside that function (function block

   352

or program). Even global variables must be re-declared as EXTERN

   353

before a function may access them! This means that it is easy

   354

to simply load up the variable name symbol table when we start

   355

parsing a function (function block or program), and to clear it

   356

when we reach the end. Checking whether variables declared

   357

as EXTERN really exist inside a RESOURCE or a CONFIGURATION

   358

is left to stage 3 (semantic checking) where we can use the

   359

abstract tree itself to search for the variables (NOTE: semantic

   360

cheching at stage 3 has not yet been implemented, so we may yet

   361

end up using a symbol table too at that stage!).

   362

   363

 Due to the use of the symbol tables, and special identifier

   364

tokens depending on the type of identifier it had previously

   365

been declared in the code being parsed, the syntax was slightly

   366

changed regarding the definition of variable names, derived

   367

function names, etc... FROM for e.g.:

   368

variable_name: identifier;

   369

TO

   370

variable_name: variable_name_token;

   371

   372

 Flex first looks at the symbol tables when it finds an identifier,

   373

and returns the correct token corresponding to the identifier

   374

type in question. Only if the identifier is not currently stored

   375

in any symbol table, does flex return a simple identifier_token.

   376

   377

 This means that the declarations of variables, functions etc...

   378

were changed FROM:

   379

function_declaration: FUNCTION derived_function_name ...

   380

TO

   381

function_declaration: FUNCTION identifier ...

   382

since the initial definition of derived_function_name had been

   383

changed FROM

   384

derived_function_name: identifier;

   385

TO

   386

derived_function_name: derived_function_name_token;

   387

   388

   389

   390

   391

 Comment 2

   392

 ---------

   393

 Since the ST and IL languages share a lot of common syntax,

   394

I have decided to write a single parser to handle both languages

   395

simultaneously. This approach has the advantage that the user

   396

may mix the language used in the same file, as long as each function

   397

is written in a single lanuage.

   398

   399

 This approach also assumes that all the IL language operators are

   400

keywords, which means that it is not possible to define variables

   401

using names such as "LD", "ST", etc...

   402

Note that the spec does not consider these operators to be keywords,

   403

so it means that they should be available for variable names! On the

   404

other hand, all implementations of the ST and IL languages seems to

   405

treat them as keywords, so there is not much harm in doing the same.

   406

   407

 If it ever becomes necessary to allow variables with names of IL 

   408

operators, either the syntax will have to be augmented, or we can 

   409

brake up the parser in two: one for ST and another for IL.

   410

   411

   412

   413

/********************************/

   414

/* B 1.3.3 - Derived data types */

   415

/********************************/

   416

   417

Issue 1

   418

=======

   419

   420

According to the spec, the valid construct

   421

TYPE new_str_type : STRING := "hello!"; END_TYPE

   422

has two possible routes to type_declaration...

   423

   424

Route 1:

   425

type_declaration: single_element_type_declaration

   426

single_element_type_declaration: simple_type_declaration

   427

simple_type_declaration: identifier ':' simple_spec_init

   428

simple_spec_init: simple_specification ASSIGN constant

   429

(shift:  identifier <- 'new_str_type')

   430

simple_specification: elementary_type_name

   431

elementary_type_name: STRING

   432

(shift: elementary_type_name <- STRING)

   433

(reduce: simple_specification <- elementary_type_name)

   434

(shift: constant <- "hello!")

   435

(reduce: simple_spec_init: simple_specification ASSIGN constant)

   436

(reduce: ...)

   437

   438

   439

Route 2:

   440

type_declaration: string_type_declaration

   441

string_type_declaration: identifier ':' elementary_string_type_name string_type_declaration_size string_type_declaration_init

   442

(shift:  identifier <- 'new_str_type')

   443

elementary_string_type_name: STRING

   444

(shift: elementary_string_type_name <- STRING)

   445

(shift: string_type_declaration_size <- /* empty */)

   446

string_type_declaration_init: ASSIGN character_string

   447

(shift: character_string <- "hello!")

   448

(reduce: string_type_declaration_init <- ASSIGN character_string)

   449

(reduce: string_type_declaration <- identifier ':' elementary_string_type_name string_type_declaration_size string_type_declaration_init )

   450

(reduce: type_declaration <- string_type_declaration)

   451

   452

   453

 At first glance it seems that removing route 1 would make

   454

the most sense. Unfortunately the construct 'simple_spec_init'

   455

shows up multiple times in other rules, so changing this construct

   456

would mean changing all the rules in which it appears.

   457

I (Mario) therefore chose to remove route 2 instead. This means

   458

that the above declaration gets stored in a

   459

simple_type_declaration_c, and not in a string_type_declaration_c

   460

as would be expected!

   461

   462

   463

/***********************/

   464

/* B 1.5.1 - Functions */

   465

/***********************/

   466

   467

   468

Issue 1

   469

=======

   470

   471

 Due to reduce/reduce conflicts between identifiers

   472

being reduced to either a variable or an enumerator value,

   473

we were forced to keep a symbol table of the names

   474

of all declared variables. Variables are no longer

   475

created from simple identifier_token, but from

   476

variable_name_token.

   477

   478

 BUT, in functions the function name may be used as

   479

a variable! In order to be able to parse this correctly,

   480

the token parser (flex) must return a variable_name_token

   481

when it comes across the function name, while parsing

   482

the function itself.

   483

We do this by inserting the function name into the variable

   484

symbol table, and having flex return a variable_name_token

   485

whenever it comes across it.

   486

When we finish parsing the function the variable name

   487

symbol table is cleared of all entries, and the function

   488

name is inserted into the library element symbol table. This

   489

means that from then onwards flex will return a

   490

derived_function_name_token whenever it comes across the

   491

function name.

   492

   493

In order to insert the function name into the variable_name

   494

symbol table BEFORE the function body gets parsed, we

   495

need the parser to reduce a construct that contains the

   496

the function name. That is why we created the extra

   497

construct 'function_name_declaration', i.e. to force

   498

the parser to reduce it, before parsing the function body,

   499

and therefore get an oportunity to insert the function name

   500

into the variable name symbol table!

   501

   502

   503

/********************************/

   504

/* B 3.2.4 Iteration Statements */

   505

/********************************/

   506

   507

Issue 1

   508

=======

   509

   510

For the 'FOR' iteration loop

   511

   512

  FOR control_variable ASSIGN expression TO expression BY expression DO statement_list END_FOR

   513

   514

The spec declares the control variable in the syntax as

   515

   516

  control_variable: identifier;

   517

   518

but then defines the semantics of control_variable (Section 3.3.2.4)