7 %  vi: spell spelllang=en

     7 %  vi: spell spelllang=en tw=78

   136       Documentation}

    85       Preliminary Documentation}

   141     \vspace{\fill}

    90     \vspace{\fill} {\Large Dipl.-Ing. (FH) Florian Pose,

   142     {\Large Florian Pose, \url{fp@igh-essen.com}\\[1ex]

    91     \url{fp@igh-essen.com}\\[1ex] Ingenieurgemeinschaft \IgH}

   143       Ingenieurgemeinschaft \IgH}

   156 \lstlistoflistings

   104 %\lstlistoflistings

   253 \item Common ``realtime interface'' for applications, that want to use

   201 \item Common ``Application Interface'' for applications, that want to use

   386 section~\ref{sec:ecdev}) and the ``Realtime Interface'' (see

   334 section~\ref{sec:ecdev}) and the ``Application Interface'' (see

   406 \textit{examples} directory, see chapter~\ref{chapter:examples}}, but have to

   354 \textit{examples/} directory.}, but have to be generated or written by the

   407 be generated or written by the user. An application module can ``request'' a

   355 user. An application module can ``request'' a master through the application

   408 master through the realtime interface (see section~\ref{sec:ecrt}). If this

   356 interface (see section~\ref{sec:ecrt}). If this succeeds, the module has the

   409 succeeds, the module has the control over the master: It can provide a bus

   357 control over the master: It can provide a bus configuration and exchange

   410 configuration and exchange process data.

   358 process data.

   446 \section{General behavior} % FIXME

   394 \section{General Behavior} % FIXME

   454 \label{sec:mastermodule}

   404 \label{sec:mastermod}

   480 \lstinline+ecrt_master_request()+ function of the realtime interface (see

   430 \lstinline+ecrt_master_request()+ function of the application interface (see

   644   \caption{Master configuration structures}

   610   \caption{Master Configuration}

   671 to know the appropriate kind of lock. Imagine, the application uses RTAI

   637 to know the appropriate kind of lock. For example if the application uses RTAI

   672 functionality, then ordinary kernel semaphores would not be sufficient. For

   638 functionality, ordinary kernel semaphores would not be sufficient. For that, an

   673 that, an important design decision was made: The application that reserved a

   639 important design decision was made: The application that reserved a master must

   674 master must have the total control, therefore it has to take responsibility for

   640 have the total control, therefore it has to take responsibility for providing

   675 providing the appropriate locking mechanisms. If another instance wants to

   641 the appropriate locking mechanisms. If another instance wants to access the

   676 access the master, it has to request the master lock by callbacks, that have to

   642 master, it has to request the master lock by callbacks, that have to be set by

   677 be set by the application. Moreover the application can deny access to the

   643 the application. Moreover the application can deny access to the master if it

   678 master if it considers it to be awkward at the moment.

   644 considers it to be awkward at the moment.

   687 Figure~\ref{fig:locks} exemplary shows, how two processes share one master: The

   653 Figure~\ref{fig:locks} exemplary shows, how two processes share one master:

   688 application's cyclic task uses the master for process data exchange, while the

   654 The application's cyclic task uses the master for process data exchange, while

   689 master-internal EoE process uses it to communicate with EoE-capable slaves.

   655 the master-internal EoE process uses it to communicate with EoE-capable

   690 Both have to acquire the master lock before access: The application task can

   656 slaves.  Both have to acquire the master lock before access: The application

   691 access the lock natively, while the EoE process has to use the callbacks.

   657 task can access the lock natively, while the EoE process has to use the

   692 Section~\ref{sec:concurrency} gives an example, of how to implement this.

   658 callbacks.  See the application interface documentation

   693 

   659 (section~\ref{sec:ecrt} of how to use the locking callbacks.

   694 %------------------------------------------------------------------------------

   660 

   695 

   661 %------------------------------------------------------------------------------

   696 \chapter{Ethernet devices}

   662 

   663 \chapter{Ethernet Devices}

   699 The EtherCAT protocol is based on the Ethernet standard. That's why the master

   666 The EtherCAT protocol is based on the Ethernet standard, so a master relies on

   700 relies on standard Ethernet hardware to communicate with the bus.

   667 standard Ethernet hardware to communicate with the bus.

   704 the master can use to connect to an EtherCAT bus.

   671 a master can use to connect to an EtherCAT bus.

   705 

   706 Section~\ref{sec:networkdrivers} offers an overview of general Linux

   707 network driver modules, while section~\ref{sec:requirements} will show

   708 the requirements to an EtherCAT-enabled network driver. Finally,

   709 sections~\ref{sec:seldev} to~\ref{sec:patching} show how to fulfill

   710 these requirements and implement such a driver module.

   758 \paragraph{The net\_device structure}

   719 \paragraph{The \lstinline+net_device+ Structure}

   761 The driver registers a \textit{net\_device} structure for each device

   722 The driver registers a \lstinline+net_device+ structure for each device to

   762 to communicate with the network stack and to create a ``network

   723 communicate with the network stack and to create a ``network interface''. In

   763 interface''. In case of an Ethernet driver, this interface appears as

   724 case of an Ethernet driver, this interface appears as \textit{ethX}, where X is

   764 \textit{ethX}, where X is a number assigned by the kernel on

   725 a number assigned by the kernel on registration. The \lstinline+net_device+

   765 registration. The \textit{net\_device} structure receives events

   726 structure receives events (either from user space or from the network stack)

   766 (either from user space or from the network stack) via several

   727 via several callbacks, which have to be set before registration. Not every

   767 callbacks, which have to be set before registration. Not every

   728 callback is mandatory, but for reasonable operation the ones below are needed

   768 callback is mandatory, but for reasonable operation the ones below are

   729 in any case:

   769 needed in any case:

   730 

   731 \newsavebox\boxopen

   732 \sbox\boxopen{\lstinline+open()+}

   733 \newsavebox\boxstop

   734 \sbox\boxstop{\lstinline+stop()+}

   735 \newsavebox\boxxmit

   736 \sbox\boxxmit{\lstinline+hard_start_xmit()+}

   737 \newsavebox\boxstats

   738 \sbox\boxstats{\lstinline+get_stats()+}

   773 \item[open()] This function is called when network communication has to be

   742 \item[\usebox\boxopen] This function is called when network communication has

   774 started, for example after a command \textit{ifconfig ethX up} from user

   743 to be started, for example after a command \lstinline+ip link set ethX up+ from

   775 space. Frame reception has to be enabled by the driver.

   744 user space. Frame reception has to be enabled by the driver.

   776 

   745 

   777 \item[stop()] The purpose of this callback is to ``close'' the device, i.~e.

   746 \item[\usebox\boxstop] The purpose of this callback is to ``close'' the device,

   778 make the hardware stop receiving frames.

   747 i.~e.  make the hardware stop receiving frames.

   779 

   748 

   780 \item[hard\_start\_xmit()] This function is cal\-led for each frame that has

   749 \item[\usebox\boxxmit] This function is called for each frame that has to be

   781 to be transmitted. The network stack passes the frame as a pointer to an

   750 transmitted. The network stack passes the frame as a pointer to an

   782 \textit{sk\_buff} structure (``socket buffer''\index{Socket buffer}, see

   751 \lstinline+sk_buff+ structure (``socket buffer''\index{Socket buffer}, see

   785 \item[get\_stats()] This call has to return a pointer to the device's

   754 \item[\usebox\boxstats] This call has to return a pointer to the device's

   786 \textit{net\_device\_stats} structure, which permanently has to be filled with

   755 \lstinline+net_device_stats+ structure, which permanently has to be filled with

   795 \paragraph{The netif Interface}

   764 \paragraph{The \lstinline+netif+ Interface}

   799 via the \lstinline+netif_*()+ calls. For example, on successful device

   768 via the \lstinline+netif_*()+ calls. For example, on successful device opening,

   800 opening, the network stack has to be notified, that it can now pass frames to

   769 the network stack has to be notified, that it can now pass frames to the

   801 the interface.  This is done by calling \lstinline+netif_start_queue()+. After

   770 interface. This is done by calling \lstinline+netif_start_queue()+. After this

   802 this call, the \lstinline+hard_start_xmit()+ callback can be called by the

   771 call, the \lstinline+hard_start_xmit()+ callback can be called by the network

   803 network stack.  Furthermore a network driver usually manages a frame

   772 stack. Furthermore a network driver usually manages a frame transmission queue.

   804 transmission queue. If this gets filled up, the network stack has to be told

   773 If this gets filled up, the network stack has to be told to stop passing

   805 to stop passing further frames for a while. This happens with a call to

   774 further frames for a while. This happens with a call to

   806 \lstinline+netif_stop_queue()+.  If some frames have been sent, and there is

   775 \lstinline+netif_stop_queue()+. If some frames have been sent, and there is

   815 packed into a so-called ``socket buffer'' for that (see below).

   784 included in a so-called ``socket buffer'' for that (see below).

   820 Socket buffers are the basic data type for the whole network stack.  They

   789 Socket buffers are the basic data type for the whole network stack. They serve

   821 serve as containers for network data and are able to quickly add data headers

   790 as containers for network data and are able to quickly add data headers and

   822 and footers, or strip them off again. Therefore a socket buffer consists of an

   791 footers, or strip them off again. Therefore a socket buffer consists of an

   824 (\textit{head}), beginning of data (\textit{data}), end of data

   793 (\lstinline+head+), beginning of data (\lstinline+data+), end of data

   825 (\textit{tail}) and end of buffer (\textit{end}). In addition, a socket buffer

   794 (\lstinline+tail+) and end of buffer (\lstinline+end+). In addition, a socket

   826 holds network header information and (in case of received data) a pointer to

   795 buffer holds network header information and (in case of received data) a

   827 the \textit{net\_device}, it was received on. There exist functions that

   796 pointer to the \lstinline+net_device+, it was received on. There exist

   828 create a socket buffer (\lstinline+dev_alloc_skb()+), add data either from

   797 functions that create a socket buffer (\lstinline+dev_alloc_skb()+), add data

   829 front (\lstinline+skb_push()+) or back (\lstinline+skb_put()+), remove data

   798 either from front (\lstinline+skb_push()+) or back (\lstinline+skb_put()+),

   830 from front (\lstinline+skb_pull()+) or back (\lstinline+skb_trim()+), or

   799 remove data from front (\lstinline+skb_pull()+) or back

   831 delete the buffer (\lstinline+kfree_skb()+).  A socket buffer is passed from

   800 (\lstinline+skb_trim()+), or delete the buffer (\lstinline+kfree_skb()+).  A

   832 layer to layer, and is freed by the layer that uses it the last time. In case

   801 socket buffer is passed from layer to layer, and is freed by the layer that

   833 of sending, freeing has to be done by the network driver.

   802 uses it the last time. In case of sending, freeing has to be done by the

   803 network driver.

   838 \label{sec:requirements}

   808 \label{sec:ethercatdrivers}

   846 exclusive access to the Ethernet hardware. This implies that the network

   816 exclusive access to the Ethernet hardware. This implies that the network device

   847 device must not be connected to the kernel's network stack as usual, because

   817 must not be connected to the kernel's network stack as usual, because the

   848 the kernel would try to use it as an ordinary Ethernet device.

   818 kernel would try to use it as an ordinary Ethernet device.

   853 EtherCAT frames travel through the logical EtherCAT ring and are then sent

   823 EtherCAT frames travel through the logical EtherCAT ring and are then sent back

   854 back to the master. Communication is highly deterministic: A frame is sent and

   824 to the master. Communication is highly deterministic: A frame is sent and will

   855 will be received again after a constant time.  Therefore, there is no need to

   825 be received again after a constant time, so there is no need to notify the

   856 notify the driver about frame reception: The master can instead query the

   826 driver about frame reception: The master can instead query the hardware for

   857 hardware for received frames.

   827 received frames, if it expects them to be already received.

   864   \includegraphics[width=.8\textwidth]{images/interrupt}

   834   \includegraphics[width=.9\textwidth]{images/interrupt}

   884 The interrupt-less operation is desirable, because there is simply no need for

   854 The interrupt-less operation is desirable, because hardware interrupts are not

   885 an interrupt. Moreover hardware interrupts are not conducive in improving the

   855 conducive in improving the driver's realtime behaviour: Their indeterministic

   886 driver's realtime behaviour: Their indeterministic incidences contribute to

   856 incidences contribute to increasing the jitter. Besides, if a realtime

   887 increasing the jitter. Besides, if a realtime extension (like RTAI) is used,

   857 extension (like RTAI) is used, some additional effort would have to be made to

   888 some additional effort would have to be made to prioritize interrupts.

   858 prioritize interrupts.

   930 After loading the master module, at least one EtherCAT-capable network

   900 After loading the master module, at least one EtherCAT-capable network driver

   931 driver module has to be loaded, that connects one of its devices to

   901 module has to be loaded, that offers its devices to the master (see

   932 the master. To specify an EtherCAT device and the master to connect

   902 section~\ref{sec:ecdev}. The master module knows the devices to choose from the

   933 to, all EtherCAT-capable network driver modules should provide two

   903 module parameters (see section~\ref{sec:mastermod}). If the init script is used

   934 module parameters:

   904 to start the master, the drivers and devices to use can be specified in the

   935 

   905 sysconfig file (see section~\ref{sec:sysconfig}).

   936 \begin{description}

   906 

   937 \item[ec\_device\_index] PCI device index of the device that is

   907 %------------------------------------------------------------------------------

   938   connected to the EtherCAT bus. If this parameter is left away, all

   908 

   939   devices found are treated as ordinary Ethernet devices. Default:

   909 \section{EtherCAT Device Interface}

   940   $-1$

   941 \item[ec\_master\_index] Index of the master to connect to. Default:

   942   $0$

   943 \end{description}

   944 

   945 The following command loads the EtherCAT-capable RTL8139 device

   946 driver, telling it to handle the second device as an EtherCAT device

   947 and connecting it to the first master:

   948 

   949 \begin{lstlisting}[gobble=2]

   950   # `\textbf{modprobe ec\_8139too ec\_device\_index=1}`

   951 \end{lstlisting}

   952 

   953 Usually, this command does not have to be entered manually, but is

   954 called by the EtherCAT init script. See section~\ref{sec:init} for

   955 more information.

   956 

   957 %------------------------------------------------------------------------------

   958 

   959 \section{The Device Interface}

   968 The master module provides a ``device interface'' for network device

   918 The master module provides a ``device interface'' for network device drivers.

   969 drivers. To use this interface, a network device driver module must

   919 To use this interface, a network device driver module must include the header

   970 include the header

   920 \textit{devices/ecdev.h}\nomenclature{ecdev}{EtherCAT Device}, coming with the

   971 \textit{devices/ecdev.h}\nomenclature{ecdev}{EtherCAT Device}, coming

   921 EtherCAT master code. This header offers a function interface for EtherCAT

   972 with the EtherCAT master code. This header offers a function interface

   922 devices. All functions of the device interface are named with the prefix

   973 for EtherCAT devices which is explained below. All functions of the

   923 \lstinline+ecdev+.

   974 device interface are named with the prefix \textit{ecdev}.

   924 

   975 

   925 The documentation of the device interface can be found in the header file or in

   976 \paragraph{Device Registration}

   926 the appropriate module of the interface documentation (see

   977 

   927 section~\ref{sec:gendoc} for generation instructions).

   978 A network device driver can connect a physical device to an EtherCAT

   928 

   979 master with the \textit{ecdev\_register()} function.

   929 \ldots % FIXME general description of the device interface

   980 

   981 \begin{lstlisting}[gobble=2,language=C]

   982   ec_device_t *ecdev_register(unsigned int master_index,

   983                               struct net_device *net_dev,

   984                               ec_isr_t isr,

   985                               struct module *module);

   986 \end{lstlisting}

   987 

   988 The first parameter \textit{master\_index} must be the index of the

   989 EtherCAT master to connect to (see section~\ref{sec:mastermod}),

   990 followed by \textit{net\_dev}, the pointer to the corresponding

   991 net\_device structure, which represents the network device to connect.

   992 The third parameter \textit{isr} must be a pointer to the interrupt

   993 service routine (ISR\index{ISR}) handling the device. The master will

   994 later execute the ISR in order to receive frames and to update the

   995 device status. The last parameter \textit{module} must be the pointer

   996 to the device driver module, which is usually accessible via the macro

   997 \textit{THIS\_MODULE} (see next paragraph). On success, the function

   998 returns a pointer to an \textit{ec\_device\_t} object, which has to be

   999 specified when calling further functions of the device interface.

  1000 Therefore the device module has to store this pointer for future use.

  1001 In error case, the \textit{ecdev\_register()} returns \textit{NULL},

  1002 which means that the device could not be registered.  The reason for

  1003 this is printed to \textit{Syslog}\index{Syslog}. In this case, the

  1004 device module is supposed to abort the module initialisation and let

  1005 the \textit{insmod} command fail.

  1006 

  1007 \paragraph{Implicit Dependencies}

  1008 

  1009 The reason for the module pointer has to be specified at device registration is

  1010 a non-trivial one: The master has to know about the module, because there will

  1011 be an implicit dependency between the device module and a later connected

  1012 application module: When an application module connects to the master, the use

  1013 count of the master module will be increased, so that the master module can not

  1014 be unloaded for the time of the connection. This is reasonable, and so

  1015 automatically done by the kernel. The kernel knows about this dependency,

  1016 because the application module uses kernel symbols provided by the master

  1017 module.  Moreover it is mandatory, that the device module can be unloaded

  1018 neither, because it is implicitly used by the application module, too.

  1019 Unloading it would lead to a fatal situation, because the master would have no

  1020 device to send and receive frames for the application. This dependency can not

  1021 be detected automatically, because the application module does not use any

  1022 symbols of the device module. Therefore the master explicitly increments the

  1023 use counter of the connected device module upon connection of an application

  1024 and decrements it, if it disconnects again. In this manner, it is impossible to

  1025 unload a device module while the master is in use. This is done with the kernel

  1026 function pair \textit{try\_module\_get()}

  1027 \index{try\_module\_get@\textit{try\_module\_get()}} and \textit{module\_put()}

  1028 \index{module\_put@\textit{module\_put()}}. The first one increases the use

  1029 count of a module and only fails, if the module is currently being unloaded.

  1030 The last one decreases the use count again and never fails. Both functions take

  1031 a pointer to the module as their argument, which the device module therefore

  1032 has to specify upon device registration.

  1033 

  1034 \paragraph{Device Unregistering}

  1035 

  1036 The deregistration of a device is usually done in the device module's cleanup

  1037 function, by calling the \textit{ecdev\_unregister()} function and specifying

  1038 the master index and a pointer to the device object again.

  1039 

  1040 \begin{lstlisting}[gobble=2,language=C]

  1041   void ecdev_unregister(unsigned int master_index,

  1042                         ec_device_t *device);

  1043 \end{lstlisting}

  1044 

  1045 This function can fail too (if the master index is invalid, or the

  1046 given device was not registered), but due to the fact, that this

  1047 failure can not be dealt with appropriately, because the device module

  1048 is unloading anyway, the failure code would not be of any interest. So

  1049 the function has a void return value.

  1050 

  1051 \paragraph{Starting the Master}

  1052 

  1053 When a device has been initialized completely and is ready to send and

  1054 receive frames, the master has to be notified about this by calling

  1055 the \textit{ecdev\_start()} function.

  1056 

  1057 \begin{lstlisting}[gobble=2,language=C]

  1058   int ecdev_start(unsigned int master_index);

  1059 \end{lstlisting}

  1060 

  1061 The master will then enter ``Idle Mode'' and start scanning the bus

  1062 (and possibly handling EoE slaves). Moreover it will make the bus

  1063 accessible via Sysfs interface and react to user interactions. The

  1064 function takes one parameter \textit{master\_index}, which has to be

  1065 the same as at the call to \textit{ecdev\_register()}. The return

  1066 value will be non-zero if the starting process failed. In this case

  1067 the device module is supposed to abort the init sequence and make the

  1068 init function return an error code.

  1069 

  1070 \paragraph{Stopping the Master}

  1071 

  1072 Before a device can be unregistered, the master has to be stopped by

  1073 calling the \textit{ecdev\_stop()} function. It will stop processing

  1074 messages of EoE slaves and leave ``Idle Mode''. The only parameter is

  1075 \textit{master\_index}. This function can not fail.

  1076 

  1077 \begin{lstlisting}[gobble=2,language=C]

  1078   void ecdev_stop(unsigned int master_index);

  1079 \end{lstlisting}

  1080 

  1081 A subsequent call to \textit{ecdev\_unregister()} will now unregister

  1082 the device savely.

  1083 

  1084 \paragraph{Receiving Frames}

  1085 

  1086 The interrupt service routine handling device events usually has a

  1087 section where new frames are fetched from the hardware and forwarded

  1088 to the kernel network stack via \textit{netif\_receive\_skb()}. For an

  1089 EtherCAT-capable device, this has to be replaced by calling the

  1090 \textit{ecdev\_receive()} function to forward the received data to the

  1091 connected EtherCAT master instead.

  1092 

  1093 \begin{lstlisting}[gobble=2,language=C]

  1094   void ecdev_receive(ec_device_t *device,

  1095                      const void *data,

  1096                      size_t size);

  1097 \end{lstlisting}

  1098 

  1099 This function takes 3 arguments, a pointer to the device object

  1100 (\textit{device}), a pointer to the received data, and the size of the

  1101 received data. The data range has to include the Ethernet headers

  1102 starting with the destination address and reach up to the last octet

  1103 of EtherCAT data, excluding the FCS. Most network devices handle the

  1104 FCS in hardware, so it is not seen by the driver code and therefore

  1105 doesn't have to be cut off manually.

  1106 

  1107 \paragraph{Handling the Link Status}

  1108 

  1109 Information about the link status (i.~e. if there is a carrier signal detected

  1110 on the physical port) is also important to the master. This information is

  1111 usually gathered by the ISR and should be forwarded to the master by calling

  1112 the \textit{ecdev\_link\_state()} function. The master then can react on this

  1113 and warn the application of a lost link.

  1114 

  1115 \begin{lstlisting}[gobble=2,language=C]

  1116   void ecdev_link_state(ec_device_t *device,

  1117                         uint8_t new_state);

  1118 \end{lstlisting}

  1119 

  1120 The parameter \textit{device} has to be a pointer to the device object

  1121 returned by \textit{ecdev\_\-register()}. With the second parameter

  1122 \textit{new\_state}, the new link state is passed: 1, if the link went

  1123 up, and 0, if it went down.

  1131 This section will demonstrate, how to make a standard Ethernet driver

   937 This section will describe, how to make a standard Ethernet driver

  1132 EtherCAT-capable. The below code examples are taken out of the

   938 EtherCAT-capable. Unfortunately, there is no standard procedure to enable an

  1133 modified RealTek RTL8139 driver coming with the EtherCAT master

   939 Ethernet driver for use with the EtherCAT master, but there are a few common

  1134 (\textit{devices/8139too.c}). The driver was originally developed by

   940 techniques.

  1135 Donald Becker, and is currently maintained by Jeff Garzik.

  1136 

  1137 Unfortunately, there is no standard procedure to enable an Ethernet

  1138 driver for use with the EtherCAT master, but there are a few common

  1139 techniques, that are described in this section.

  1142 \item A first simple rule is, that \textit{netif\_*()}-calls must be

   943 

  1143   strictly avoided for all EtherCAT devices. As mentioned before,

   944 \item A first simple rule is, that \lstinline+netif_*()+ calls must be avoided

  1144   EtherCAT devices have no connection to the network stack, and

   945 for all EtherCAT devices. As mentioned before, EtherCAT devices have no

  1145   therefore must not call its interface functions.

   946 connection to the network stack, and therefore must not call its interface

  1146 \item Another important thing is, that EtherCAT devices should be

   947 functions.

  1147   operated without interrupts. So any calls of registering interrupt

   948 

  1148   handlers and enabling interrupts at hardware level must be avoided,

   949 \item Another important thing is, that EtherCAT devices should be operated

  1149   too.

   950 without interrupts. So any calls of registering interrupt handlers and enabling

  1150 \item The master does not use a new socket buffer for each send

   951 interrupts at hardware level must be avoided, too.

  1151   operation: Instead there is a fix one allocated on master

   952 

  1152   initialization. This socket buffer is filled with an EtherCAT frame

   953 \item The master does not use a new socket buffer for each send operation:

  1153   with every send operation and passed to the

   954 Instead there is a fix one allocated on master initialization. This socket

  1154   \textit{hard\_start\_xmit()} callback. For that it is necessary,

   955 buffer is filled with an EtherCAT frame with every send operation and passed to

  1155   that the socket buffer is not be freed by the network driver as

   956 the \lstinline+hard_start_xmit()+ callback. For that it is necessary, that the

  1156   usual.

   957 socket buffer is not be freed by the network driver as usual.

   958 

  1159 As mentioned before, the driver will handle both EtherCAT and ordinary

   961 An Ethernet driver usually handles several Ethernet devices, each described by

  1160 Ethernet devices. This implies, that for each device-dependent

   962 a \lstinline+net_device+ structure with a \lstinline+priv_data+ field to

  1161 operation, it has to be checked if an EtherCAT device is involved, or

   963 attach driver-dependent data to the structure. To distinguish between normal

  1162 just an Ethernet device. For means of simplicity, this example driver

   964 Ethernet devices and the ones used by EtherCAT masters, the private data

  1163 will only handle one EtherCAT device. This makes the case

   965 structure used by the driver could be extended by a pointer, that points to an

  1164 differentiations easier.

   966 \lstinline+ec_device_t+ object returned by \lstinline+ecdev_offer()+ (see

  1165 

   967 section~\ref{sec:ecdev}) if the device is used by a master and otherwise is

  1166 \paragraph{Global Variables}

   968 zero.

  1167 

   969 

  1168 First of all, there have to be additional global variables declared,

   970 The RealTek RTL-8139 Fast Ethernet driver is a ``simple'' Ethernet driver and

  1169 as shown in the listing:

   971 can be taken as an example to patch new drivers. The interesting sections can

  1170 

   972 be found by searching the string ``ecdev" in the file

  1171 \begin{lstlisting}[gobble=2,language=C,numbers=left]

   973 \textit{devices/8139too-2.6.24-ethercat.c}.

  1172   static int ec_device_index = -1;

  1173   static int ec_device_master_index = 0;

  1174   static ec_device_t *rtl_ec_dev;

  1175   struct net_device *rtl_ec_net_dev = NULL;

  1176 \end{lstlisting}

  1177 

  1178 \begin{description}

  1179 \item[\linenum{1} -- \linenum{2}] To

  1180   comply to the requirements for parameters of EtherCAT device modules

  1181   described in section~\ref{sec:seldev}, there have to be additional

  1182   parameter variables: \textit{ec\_\-device\_\-index} holds the index

  1183   of the EtherCAT device and defaults to $-1$ (no EtherCAT device),

  1184   while \textit{ec\_device\_master\_index} stores index of the master,

  1185   the single device will be connected to. Default: $0$

  1186 \item[\linenum{3}] \textit{rtl\_ec\_dev} will be

  1187   the pointer to the later registered RealTek EtherCAT device, which

  1188   can be used as a parameter for device methods.

  1189 \item[\linenum{4}] \textit{rtl\_ec\_net\_dev} is

  1190   a pointer to the \textit{net\_device} structure of the dedicated

  1191   device and is set while scanning the PCI bus and finding the device

  1192   with the specified index. This is done inside the

  1193   \textit{pci\_module\_init()} function executed as the first thing on

  1194   module loading.

  1195 \end{description}

  1196 

  1197 \paragraph{Module Initialization}

  1198 

  1199 Below is the (shortened) coding of the device driver's module init

  1200 function:

  1201 

  1202 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1203   static int __init rtl8139_init_module(void)

  1204   {

  1205           if (pci_module_init(&rtl8139_pci_driver) < 0) {

  1206                   printk(KERN_ERR "Failed to init PCI mod.\n");

  1207                   goto out_return;

  1208           }

  1209 

  1210           if (rtl_ec_net_dev) {

  1211                   printk(KERN_INFO "Registering"

  1212                          " EtherCAT device...\n");

  1213                   if (!(rtl_ec_dev =

  1214                           ecdev_register(ec_device_master_index,

  1215                                          rtl_ec_net_dev,

  1216                                          rtl8139_interrupt,

  1217                                          THIS_MODULE))) {

  1218                           printk(KERN_ERR "Failed to reg."

  1219                                           " EtherCAT device!\n");

  1220                           goto out_unreg_pci;

  1221                   }

  1222 

  1223                   printk(KERN_INFO "Starting EtherCAT"

  1224                          " device...\n");

  1225                   if (ecdev_start(ec_device_master_index)) {

  1226                           printk(KERN_ERR "Failed to start"

  1227                                  " EtherCAT device!\n");

  1228                           goto out_unreg_ec;

  1229                   }

  1230           } else {

  1231                   printk(KERN_WARNING "No EtherCAT device"

  1232                          " registered!\n");

  1233           }

  1234 

  1235           return 0;

  1236 

  1237       out_unreg_ec:

  1238           ecdev_unregister(ec_device_master_index, rtl_ec_dev);

  1239       out_unreg_pci:

  1240           pci_unregister_driver(&rtl8139_pci_driver);

  1241       out_return:

  1242           return -1;

  1243   }

  1244 \end{lstlisting}

  1245 

  1246 \begin{description}

  1247 \item[\linenum{3}] This call initializes all

  1248   RTL8139-compatible devices found on the pci bus. If a device with

  1249   index \textit{ec\_device\_index} is found, a pointer to its

  1250   \textit{net\_device} structure is stored in

  1251   \textit{rtl\_ec\_net\_dev} for later use (see next listings).

  1252 \item[\linenum{8}] If the specified device was

  1253   found, \textit{rtl\_ec\_net\_dev} is non-zero.

  1254 \item[\linenum{11}] The device is connected to

  1255   the specified master with a call to \textit{ecdev\_register()}. If

  1256   this fails, module loading is aborted.

  1257 \item[\linenum{23}] The device registration was

  1258   successful and the master is started. This can fail, which aborts

  1259   module loading.

  1260 \item[\linenum{29}] If no EtherCAT device was

  1261   found, a warning is output.

  1262 \end{description}

  1263 

  1264 \paragraph{Device Searching}

  1265 

  1266 During the PCI initialization phase, a variable \textit{board\_idx} is

  1267 increased for each RTL8139-compatible device found. The code below is

  1268 executed for each device:

  1269 

  1270 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1271   if (board_idx == ec_device_index) {

  1272           rtl_ec_net_dev = dev;

  1273           strcpy(dev->name, "ec0");

  1274   }

  1275 \end{lstlisting}

  1276 

  1277 \begin{description}

  1278 \item[\linenum{1}] The device with the specified

  1279   index will be the EtherCAT device.

  1280 \end{description}

  1281 

  1282 \paragraph{Avoiding Device Registration}

  1283 

  1284 Later in the PCI initialization phase, the net\_devices get

  1285 registered. This has to be avoided for EtherCAT devices and so this is

  1286 a typical example for an EtherCAT case differentiation:

  1287 

  1288 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1289   if (dev != rtl_ec_net_dev) {

  1290           i = register_netdev(dev);

  1291           if (i) goto err_out;

  1292   }

  1293 \end{lstlisting}

  1294 

  1295 \begin{description}

  1296 \item[\linenum{1}] If the current net\_device is

  1297   not the EtherCAT device, it is registered at the network stack.

  1298 \end{description}

  1299 

  1300 \paragraph{Avoiding Interrupt Registration}

  1301 

  1302 In the next two listings, there is an interrupt requested and the

  1303 device's interrupts are enabled. This also has to be encapsulated by

  1304 if-clauses, because interrupt operation is not wanted for EtherCAT

  1305 devices.

  1306 

  1307 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1308   if (dev != rtl_ec_net_dev) {

  1309           retval = request_irq(dev->irq, rtl8139_interrupt,

  1310                                SA_SHIRQ, dev->name, dev);

  1311           if (retval) return retval;

  1312   }

  1313 \end{lstlisting}

  1314 

  1315 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1316   if (dev != rtl_ec_net_dev) {

  1317           /* Enable all known interrupts by setting

  1318              the interrupt mask. */

  1319           RTL_W16(IntrMask, rtl8139_intr_mask);

  1320   }

  1321 \end{lstlisting}

  1322 

  1323 \paragraph{Frame Sending}

  1324 

  1325 The listing below shows an excerpt of the function representing the

  1326 \textit{hard\_start\_xmit()} callback of the net\_device.

  1327 

  1328 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1329   /* Note: the chip doesn't have auto-pad! */

  1330   if (likely(len < TX_BUF_SIZE)) {

  1331           if (len < ETH_ZLEN)

  1332                   memset(tp->tx_buf[entry], 0, ETH_ZLEN);

  1333           skb_copy_and_csum_dev(skb, tp->tx_buf[entry]);

  1334           if (dev != rtl_ec_net_dev) {

  1335                   dev_kfree_skb(skb);

  1336           }

  1337   } else {

  1338           if (dev != rtl_ec_net_dev) {

  1339                   dev_kfree_skb(skb);

  1340           }

  1341           tp->stats.tx_dropped++;

  1342           return 0;

  1343   }

  1344 \end{lstlisting}

  1345 

  1346 \begin{description}

  1347 \item[\linenum{6} + \linenum{10}] The

  1348   master uses a fixed socket buffer for transmission, which is reused

  1349   and may not be freed.

  1350 \end{description}

  1351 

  1352 \paragraph{Frame Receiving}

  1353 

  1354 During ordinary frame reception, a socket buffer is created and filled

  1355 with the received data. This is not necessary for an EtherCAT device:

  1356 

  1357 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1358   if (dev != rtl_ec_net_dev) {

  1359           /* Malloc up new buffer, compatible with net-2e. */

  1360           /* Omit the four octet CRC from the length. */

  1361 

  1362           skb = dev_alloc_skb (pkt_size + 2);

  1363           if (likely(skb)) {

  1364                   skb->dev = dev;

  1365                   skb_reserve(skb, 2); /* 16 byte align

  1366                                           the IP fields. */

  1367                   eth_copy_and_sum(skb, &rx_ring[ring_off + 4],

  1368                                    pkt_size, 0);

  1369                   skb_put(skb, pkt_size);

  1370                   skb->protocol = eth_type_trans(skb, dev);

  1371 

  1372                   dev->last_rx = jiffies;

  1373                   tp->stats.rx_bytes += pkt_size;

  1374                   tp->stats.rx_packets++;

  1375 

  1376                   netif_receive_skb (skb);

  1377           } else {

  1378                   if (net_ratelimit())

  1379                           printk(KERN_WARNING

  1380                                  "%s: Memory squeeze, dropping"

  1381                                  " packet.\n", dev->name);

  1382                   tp->stats.rx_dropped++;

  1383           }

  1384   } else {

  1385           ecdev_receive(rtl_ec_dev,

  1386                         &rx_ring[ring_offset + 4], pkt_size);

  1387           dev->last_rx = jiffies;

  1388           tp->stats.rx_bytes += pkt_size;

  1389           tp->stats.rx_packets++;

  1390   }

  1391 \end{lstlisting}

  1392 

  1393 \begin{description}

  1394 \item[\linenum{28}] If the device is an EtherCAT

  1395   device, no socket buffer is allocated. Instead a pointer to the data

  1396   (which is still in the device's receive ring) is passed to the

  1397   EtherCAT master. Unnecessary copy operations are avoided.

  1398 \item[\linenum{30} -- \linenum{32}] The

  1399   device's statistics are updated as usual.

  1400 \end{description}

  1401 

  1402 \paragraph{Link State}

  1403 

  1404 The link state (i.~e. if there is a carrier signal detected on the

  1405 receive port) is determined during execution of the ISR. The listing

  1406 below shows the different processing for Ethernet and EtherCAT

  1407 devices:

  1408 

  1409 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1410   if (dev != rtl_ec_net_dev) {

  1411           if (tp->phys[0] >= 0) {

  1412                   mii_check_media(&tp->mii, netif_msg_link(tp),

  1413                                   init_media);

  1414           }

  1415   } else {

  1416           void __iomem *ioaddr = tp->mmio_addr;

  1417           uint16_t link = RTL_R16(BasicModeStatus)

  1418                           & BMSR_LSTATUS;

  1419           ecdev_link_state(rtl_ec_dev, link ? 1 : 0);

  1420   }

  1421 \end{lstlisting}

  1422 

  1423 \begin{description}

  1424 \item[\linenum{3}] The ``media check'' is done

  1425   via the media independent interface (MII\nomenclature{MII}{Media

  1426     Independent Interface}), a standard interface for Fast Ethernet

  1427   devices.

  1428 \item[\linenum{7} -- \linenum{10}] For

  1429   EtherCAT devices, the link state is fetched manually from the

  1430   appropriate device register, and passed to the EtherCAT master by

  1431   calling \textit{ecdev\_\-link\_\-state()}.

  1432 \end{description}

  1433 

  1434 \paragraph{Module Cleanup}

  1435 

  1436 Below is the module's cleanup function:

  1437 

  1438 \begin{lstlisting}[gobble=2,language=C,numbers=left]

  1439   static void __exit rtl8139_cleanup_module (void)

  1440   {

  1441           printk(KERN_INFO "Cleaning up RTL8139-EtherCAT"

  1442                  " module...\n");

  1443 

  1444           if (rtl_ec_net_dev) {

  1445                   printk(KERN_INFO "Stopping device...\n");

  1446                   ecdev_stop(ec_device_master_index);

  1447                   printk(KERN_INFO "Unregistering device...\n");

  1448                   ecdev_unregister(ec_device_master_index,

  1449                                    rtl_ec_dev);

  1450                   rtl_ec_dev = NULL;

  1451           }

  1452 

  1453           pci_unregister_driver(&rtl8139_pci_driver);

  1454 

  1455           printk(KERN_INFO "RTL8139-EtherCAT module"

  1456                  " cleaned up.\n");

  1457   }

  1458 \end{lstlisting}

  1459 

  1460 \begin{description}

  1461 

  1462 \item[\linenum{6}] Stopping and deregistration is only done, if a device was

  1463 registered before.

  1464 

  1465 \item[\linenum{8}] The master is first stopped, so it does not access the

  1466 device any more.

  1467 

  1468 \item[\linenum{10}] After this, the device is unregistered. The master is now

  1469 ``orphaned''.

  1470 

  1471 \end{description}

  1725 are gathered in a single object of the \textit{ec\_fsm\_t} class. This is

  1227 are gathered in a single object of the \lstinline+ec_fsm_master_t+ class. This

  1726 advantageous, because there is always one instance of every state machine

  1228 is advantageous, because there is always one instance of every state machine

  1731 If a closer look is taken to the above listing, it can be seen that

  1233 If a closer look is taken to the above listing, it can be seen that the

  1732 the actions executed (the ``outputs'' of the state machine) only

  1234 actions executed (the ``outputs'' of the state machine) only depend on the

  1733 depend on the current state. This accords to the ``Moore'' model

  1235 current state. This accords to the ``Moore'' model introduced in

  1734 introduced in section~\ref{sec:fsmtheory}. As mentioned, the ``Mealy''

  1236 section~\ref{sec:fsmtheory}. As mentioned, the ``Mealy'' model offers a higher

  1735 model offers a higher flexibility, which can be seen in the listing

  1237 flexibility, which can be seen in the listing below:

  1736 below:

  1752 \item[\linenum{3} + \linenum{7}] The

  1253 

  1753   state function executes the actions depending on the state

  1254 \item[\linenum{3} + \linenum{7}] The state function executes the actions

  1754   transition, that is about to be done.

  1255 depending on the state transition, that is about to be done.

  1256 

  1775 This model is oftenly used in the master. It combines the best aspects

  1277 This model is often used in the master. It combines the best aspects of both

  1776 of both approaches.

  1278 approaches.

  1780 To avoid having too much states, certain functions of the EtherCAT master state

  1282 To avoid having too much states, certain functions of the EtherCAT master

  1781 machine have been sourced out into sub state machines.  This helps to

  1283 state machine have been sourced out into sub state machines.  This helps to

  1813 \item[\linenum{3}] \textit{change\_state} is the

  1315 

  1814   state pointer of the state change state machine. The state function,

  1316 \item[\linenum{3}] \lstinline+change_state+ is the state pointer of the state

  1815   the pointer points on, is executed\ldots

  1317 change state machine. The state function, the pointer points on, is

  1816 \item[\linenum{6}] \ldots either until the state

  1318 executed\ldots

  1817   machine terminates with the error state \ldots

  1319 

  1818 \item[\linenum{11}] \ldots or until the state

  1320 \item[\linenum{6}] \ldots either until the state machine terminates with the

  1819   machine terminates in the end state. Until then, the ``higher''

  1321 error state \ldots

  1820   state machine remains in the current state and executes the sub

  1322 

  1821   state machine again in the next cycle.

  1323 \item[\linenum{11}] \ldots or until the state machine terminates in the end

  1324 state. Until then, the ``higher'' state machine remains in the current state

  1325 and executes the sub state machine again in the next cycle.

  1326 

  1826 The below sections describe every state machine used in the EtherCAT

  1331 The below sections describe every state machine used in the EtherCAT master.

  1827 master. The textual descriptions of the state machines contain

  1332 The textual descriptions of the state machines contain references to the

  1828 references to the transitions in the corresponding state transition

  1333 transitions in the corresponding state transition diagrams, that are marked

  1829 diagrams, that are marked with an arrow followed by the name of the

  1334 with an arrow followed by the name of the successive state. Transitions caused

  1830 successive state.  Transitions caused by trivial error cases (i.~e. no

  1335 by trivial error cases (i.~e. no response from slave) are not described

  1831 response from slave) are not described explicitly. These transitions

  1336 explicitly. These transitions are drawn as dashed arrows in the diagrams.

  1832 are drawn as dashed arrows in the diagrams.

  1337 

  1833 

  1338 %------------------------------------------------------------------------------

  1834 %------------------------------------------------------------------------------

  1339 

  1835 

  1340 \section{The Master State Machine}

  1836 \section{The Operation State Machine}

  1341 \label{sec:fsm-master}

  1837 \label{sec:fsm-op}

  1342 \index{FSM!Master}

  1838 \index{FSM!Operation}

  1343 

  1839 

  1344 The master state machine is executed in the context of the master thread.

  1840 The Operation state machine is executed by calling the

  1345 Figure~\ref{fig:fsm-master} shows its transition diagram. Its purposes are:

  1841 \textit{ecrt\_master\_run()} method in cyclic realtime code. Its

  1842 purpose is to monitor the bus and to reconfigure slaves after a bus

  1843 failure or power failure. Figure~\ref{fig:fsm-op} shows its transition

  1844 diagram.

  1848   \includegraphics[width=.8\textwidth]{images/fsm-op}

  1349   \includegraphics[width=\textwidth]{graphs/fsm_master}

  1849   \caption{Transition diagram of the operation state machine}

  1350   \caption{Transition diagram of the master state machine}

  1850   \label{fig:fsm-op}

  1351   \label{fig:fsm-master}

  1854 \item[START] This is the beginning state of the operation state

  1355 

  1855   machine. There is a datagram issued, that queries the ``AL Control

  1356 \item[Bus monitoring] The bus topology is monitored. If it changes, the bus is

  1856   Response'' attribute \cite[section~5.3.2]{alspec} of all slaves via

  1357 (re-)scanned.

  1857   broadcast. In this way, all slave states and the number of slaves

  1358 

  1858   responding can be determined.  $\rightarrow$~BROADCAST

  1359 \item[Slave configuration] The application-layer states of the slaves are

  1859 

  1360 monitored. If a slave is not in the state it supposed to be, the slave is

  1860 \item[BROADCAST] The broadcast datagram is evaluated. A change in the number of

  1361 (re-)configured.

  1861 responding slaves is treated as a topology change. If the number of slaves is

  1362 

  1862 not as expected, the bus is marked as ``tainted''. In this state, no slave

  1363 \item[Request handling] Requests (either originating from the application or

  1863 reconfiguration is possible, because the assignment of known slaves and those

  1364 from external sources) are handled. A request is a job that the master shall

  1864 present on the bus is ambiguous. If the number of slaves is considered as

  1365 process asynchronously, for example an SII access, Sdo access, or similar.

  1865 right, the bus is marked for validation, because it turned from tainted to

  1866 normal state and it has to be checked, if all slaves are valid. Now, the state

  1867 of every single slave has to be determined. For that, a (unicast) datagram is

  1868 issued, that queries the first slave's ``AL Control Response'' attribute.

  1869 $\rightarrow$~READ STATES

  1870 

  1871 \item[READ STATES] If the current slave did not respond to its configured

  1872 station address, it is marked as offline, and the next slave is queried.

  1873 $\rightarrow$~READ STATES

  1874 

  1875   If the slave responded, it is marked as online and its current state

  1876   is stored. The next slave is queried. $\rightarrow$~READ STATES

  1877 

  1878   If all slaves have been queried, and the bus is marked for

  1879   validation, the validation is started by checking the first slaves

  1880   vendor ID. $\rightarrow$~VALIDATE VENDOR

  1881 

  1882   If no validation has to be done, it is checked, if all slaves are in

  1883   the state they are supposed to be. If not, the first of slave with

  1884   the wrong state is reconfigured and brought in the required state.

  1885   $\rightarrow$~CONFIGURE SLAVES

  1886 

  1887   If all slaves are in the correct state, the state machine is

  1888   restarted. $\rightarrow$~START

  1889 

  1890 \item[CONFIGURE SLAVES] The slave configuration state machine is

  1891   executed until termination. $\rightarrow$~CONFIGURE SLAVES

  1892 

  1893   If there are still slaves in the wrong state after another check,

  1894   the first of these slaves is configured and brought into the correct

  1895   state again. $\rightarrow$~CONFIGURE SLAVES

  1896 

  1897   If all slaves are in the correct state, the state machine is

  1898   restarted. $\rightarrow$~START

  1899 

  1900 \item[VALIDATE VENDOR] The SII state machine is executed until

  1901   termination. If the slave has the wrong vendor ID, the state machine

  1902   is restarted. $\rightarrow$~START

  1903 

  1904   If the slave has the correct vendor ID, its product ID is queried.

  1905   $\rightarrow$~VALIDATE PRODUCT

  1906 

  1907 \item[VALIDATE PRODUCT] The SII state machine is executed until

  1908   termination. If the slave has the wrong product ID, the state

  1909   machine is restarted. $\rightarrow$~START

  1910 

  1911   If the slave has the correct product ID, the next slave's vendor ID

  1912   is queried. $\rightarrow$~VALIDATE VENDOR

  1913 

  1914   If all slaves have the correct vendor IDs and product codes, the

  1915   configured station addresses can be safely rewritten. This is done

  1916   for the first slave marked as offline.

  1917   $\rightarrow$~REWRITE ADDRESSES

  1918 

  1919 \item[REWRITE ADDRESSES] If the station address was successfully written, it is

  1920 searched for the next slave marked as offline. If there is one, its address is

  1921 reconfigured, too.  $\rightarrow$~REWRITE ADDRESSES

  1922 

  1923   If there are no more slaves marked as offline, the state machine is

  1924   restarted. $\rightarrow$~START

  1925 \end{description}

  1926 

  1927 %------------------------------------------------------------------------------

  1928 

  1929 \section{The Idle State Machine}

  1930 \label{sec:fsm-idle}