19 \usepackage{svn}

    19 \usepackage[nofancy]{rcsinfo}

    65 \SVN $Date$

    65 \rcsInfo $RcsId$

    66 \SVN $Revision$

    96     {\Large Essen, \SVNDate\\[1ex]

    95     {\Large Essen, \rcsInfoLongDate\\[1ex]

    97       Revision \SVNRevision}

    96       Revision \rcsInfoRevision}

   175 \item Comes with EtherCAT-capable drivers for several common Ethernet devices.

   174 \item Comes with EtherCAT-capable native drivers for several common Ethernet

   175 chips, as well as a generic driver for all chips supported by the Linux

   176 kernel.

   179   \item The Ethernet hardware is operated without interrupts.

   180   \item The native drivers operate the hardware without interrupts.

   180 

   181 

   181   \item Drivers for additional Ethernet hardware can easily be implemented

   182   \item Native drivers for additional Ethernet hardware can easily be

   182   using the common device interface (see sec.~\ref{sec:ecdev}) provided by the

   183   implemented using the common device interface (see sec.~\ref{sec:ecdev})

   183   master module.

   184   provided by the master module.

   185 

   186   \item For any other hardware, the generic driver can be used. It uses the

   187   lower layers of the Linux network stack.

   195   \item RTAI\nomenclature{RTAI}{Realtime Application Interface},

   199   \item RTAI\nomenclature{RTAI}{Realtime Application Interface} \cite{rtai},

   197   Systems}, etc.

   201   Systems}, RT-Preempt \cite{rt-preempt}, etc.

   365   \includegraphics[width=.9\textwidth]{images/architecture}

   369   \includegraphics[width=\textwidth]{images/architecture}

   936 hardware. There are device driver modules that handle Ethernet hardware, which

   940 hardware.

   937 a master can use to connect to an EtherCAT bus.

   941 

   942 \paragraph{Native Ethernet Device Drivers} There are native device driver

   943 modules (see sec.~\ref{sec:native-drivers}) that handle Ethernet hardware,

   944 which a master can use to connect to an EtherCAT bus. They offer their

   945 Ethernet hardware to the master module via the device interface (see

   946 sec.~\ref{sec:ecdev}) and must be capable to prepare Ethernet devices either

   947 for EtherCAT (realtime) operation or for ``normal'' operation using the

   948 kernel's network stack. The advantage of this approach is that the master can

   949 operate nearly directly on the hardware, which allows a high performance. The

   950 disadvantage is, that there has to be an EtherCAT-capable version of the

   951 original Ethernet driver.

   952 

   953 \paragraph{Generic Ethernet Device Driver} From master version 1.5, there is a

   954 generic Ethernet device driver module (see sec.~\ref{sec:generic-driver}),

   955 that uses the lower layers of the network stack to connect to the hardware.

   956 The advantage is, that arbitrary Ethernet hardware can be used for EtherCAT

   957 operation, independently of the actual hardware driver (so all Linux Ethernet

   958 drivers are supported without modifications). The disadvantage is, that this

   959 approach does not support realtime extensions like RTAI, because the Linux

   960 network stack is addressed. Moreover the performance is a little worse than

   961 the native approach, because the Ethernet frame data have to traverse the

   962 network stack.

   950 \paragraph{Tasks of a Network Driver}

   975 \paragraph{Tasks of a Network Driver} Network device drivers usually handle

   951 

   976 the lower two layers of the OSI model, that is the physical layer and the

   952 Network device drivers usually handle the lower two layers of the OSI model,

   977 data-link layer. A network device itself natively handles the physical layer

   953 that is the physical layer and the data-link layer. A network device itself

   978 issues: It represents the hardware to connect to the medium and to send and

   954 natively handles the physical layer issues: It represents the hardware to

   979 receive data in the way, the physical layer protocol describes. The network

   955 connect to the medium and to send and receive data in the way, the physical

   980 device driver is responsible for getting data from the kernel's networking

   956 layer protocol describes. The network device driver is responsible for getting

   981 stack and forwarding it to the hardware, that does the physical transmission.

   957 data from the kernel's networking stack and forwarding it to the hardware,

   982 If data is received by the hardware respectively, the driver is notified

   958 that does the physical transmission.  If data is received by the hardware

   983 (usually by means of an interrupt) and has to read the data from the hardware

   959 respectively, the driver is notified (usually by means of an interrupt) and

   984 memory and forward it to the network stack. There are a few more tasks, a

   960 has to read the data from the hardware memory and forward it to the network

   985 network device driver has to handle, including queue control, statistics and

   961 stack. There are a few more tasks, a network device driver has to handle,

   986 device dependent features.

   962 including queue control, statistics and device dependent features.

   987 

   963 

   988 \paragraph{Driver Startup} Usually, a driver searches for compatible devices

   964 \paragraph{Driver Startup}

   989 on module loading.  For PCI drivers, this is done by scanning the PCI bus and

   965 

   990 checking for known device IDs. If a device is found, data structures are

   966 Usually, a driver searches for compatible devices on module loading.

   991 allocated and the device is taken into operation.

   967 For PCI drivers, this is done by scanning the PCI bus and checking for

   992 

   968 known device IDs. If a device is found, data structures are allocated

   993 \paragraph{Interrupt Operation}\index{Interrupt} A network device usually

   969 and the device is taken into operation.

   994 provides a hardware interrupt that is used to notify the driver of received

   970 

   995 frames and success of transmission, or errors, respectively. The driver has to

   971 \paragraph{Interrupt Operation}

   996 register an interrupt service routine

   972 \index{Interrupt}

   997 (ISR\index{ISR}\nomenclature{ISR}{Interrupt Service Routine}), that is

   973 

   998 executed each time, the hardware signals such an event. If the interrupt was

   974 A network device usually provides a hardware interrupt that is used to

   999 thrown by the own device (multiple devices can share one hardware interrupt),

   975 notify the driver of received frames and success of transmission, or

  1000 the reason for the interrupt has to be determined by reading the device's

   976 errors, respectively. The driver has to register an interrupt service

  1001 interrupt register. For example, if the flag for received frames is set, frame

   977 routine (ISR\index{ISR}\nomenclature{ISR}{Interrupt Service Routine}),

  1002 data has to be copied from hardware to kernel memory and passed to the network

   978 that is executed each time, the hardware signals such an event. If the

  1003 stack.

   979 interrupt was thrown by the own device (multiple devices can share one

  1004 

   980 hardware interrupt), the reason for the interrupt has to be determined

  1005 \paragraph{The \lstinline+net_device+ Structure}\index{net\_device} The driver

   981 by reading the device's interrupt register. For example, if the flag

  1006 registers a \lstinline+net_device+ structure for each device to communicate

   982 for received frames is set, frame data has to be copied from hardware

  1007 with the network stack and to create a ``network interface''. In case of an

   983 to kernel memory and passed to the network stack.

  1008 Ethernet driver, this interface appears as \textit{ethX}, where X is a number

   984 

  1009 assigned by the kernel on registration. The \lstinline+net_device+ structure

   985 \paragraph{The \lstinline+net_device+ Structure}

  1010 receives events (either from userspace or from the network stack) via several

   986 \index{net\_device}

  1011 callbacks, which have to be set before registration. Not every callback is

   987 

  1012 mandatory, but for reasonable operation the ones below are needed in any case:

   988 The driver registers a \lstinline+net_device+ structure for each device to

   989 communicate with the network stack and to create a ``network interface''. In

   990 case of an Ethernet driver, this interface appears as \textit{ethX}, where X

   991 is a number assigned by the kernel on registration. The \lstinline+net_device+

   992 structure receives events (either from userspace or from the network stack)

   993 via several callbacks, which have to be set before registration. Not every

   994 callback is mandatory, but for reasonable operation the ones below are needed

   995 in any case:

  1030 \paragraph{The \lstinline+netif+ Interface}

  1047 \paragraph{The \lstinline+netif+ Interface}\index{netif} All other

  1031 \index{netif}

  1048 communication in the direction interface $\to$ network stack is done via the

  1032 

  1049 \lstinline+netif_*()+ calls. For example, on successful device opening, the

  1033 All other communication in the direction interface $\to$ network stack is done

  1050 network stack has to be notified, that it can now pass frames to the

  1034 via the \lstinline+netif_*()+ calls. For example, on successful device opening,

  1035 the network stack has to be notified, that it can now pass frames to the

  1038 stack. Furthermore a network driver usually manages a frame transmission queue.

  1053 stack. Furthermore a network driver usually manages a frame transmission

  1039 If this gets filled up, the network stack has to be told to stop passing

  1054 queue.  If this gets filled up, the network stack has to be told to stop

  1040 further frames for a while. This happens with a call to

  1055 passing further frames for a while. This happens with a call to

  1052 \paragraph{Socket Buffers}

  1067 \paragraph{Socket Buffers}\index{Socket buffer} Socket buffers are the basic

  1053 \index{Socket buffer}

  1068 data type for the whole network stack. They serve as containers for network

  1054 

  1069 data and are able to quickly add data headers and footers, or strip them off

  1055 Socket buffers are the basic data type for the whole network stack. They serve

  1070 again. Therefore a socket buffer consists of an allocated buffer and several

  1056 as containers for network data and are able to quickly add data headers and

  1071 pointers that mark beginning of the buffer (\lstinline+head+), beginning of

  1057 footers, or strip them off again. Therefore a socket buffer consists of an

  1072 data (\lstinline+data+), end of data (\lstinline+tail+) and end of buffer

  1058 allocated buffer and several pointers that mark beginning of the buffer

  1073 (\lstinline+end+). In addition, a socket buffer holds network header

  1059 (\lstinline+head+), beginning of data (\lstinline+data+), end of data

  1074 information and (in case of received data) a pointer to the

  1060 (\lstinline+tail+) and end of buffer (\lstinline+end+). In addition, a socket

  1075 \lstinline+net_device+, it was received on. There exist functions that create

  1061 buffer holds network header information and (in case of received data) a

  1076 a socket buffer (\lstinline+dev_alloc_skb()+), add data either from front

  1062 pointer to the \lstinline+net_device+, it was received on. There exist

  1077 (\lstinline+skb_push()+) or back (\lstinline+skb_put()+), remove data from

  1063 functions that create a socket buffer (\lstinline+dev_alloc_skb()+), add data

  1078 front (\lstinline+skb_pull()+) or back (\lstinline+skb_trim()+), or delete the

  1064 either from front (\lstinline+skb_push()+) or back (\lstinline+skb_put()+),

  1079 buffer (\lstinline+kfree_skb()+).  A socket buffer is passed from layer to

  1065 remove data from front (\lstinline+skb_pull()+) or back

  1080 layer, and is freed by the layer that uses it the last time. In case of

  1066 (\lstinline+skb_trim()+), or delete the buffer (\lstinline+kfree_skb()+).  A

  1081 sending, freeing has to be done by the network driver.

  1067 socket buffer is passed from layer to layer, and is freed by the layer that

  1082 

  1068 uses it the last time. In case of sending, freeing has to be done by the

  1083 %------------------------------------------------------------------------------

  1069 network driver.

  1084 

  1070 

  1085 \section{Native EtherCAT Device Drivers}

  1071 %------------------------------------------------------------------------------

  1086 \label{sec:native-drivers}

  1072 

  1087 

  1073 \section{EtherCAT Device Drivers}

  1088 There are a few requirements, that applies to Ethernet hardware when used with

  1074 \label{sec:drivers}

  1089 a native Ethernet driver with EtherCAT functionality.

  1075 

  1090 

  1076 There are a few requirements for Ethernet network devices to function as

  1091 \paragraph{Dedicated Hardware} For performance and realtime purposes, the

  1077 EtherCAT devices, when connected to an EtherCAT bus.

  1092 EtherCAT master needs direct and exclusive access to the Ethernet hardware.

  1078 

  1093 This implies that the network device must not be connected to the kernel's

  1079 \paragraph{Dedicated Interfaces}

  1094 network stack as usual, because the kernel would try to use it as an ordinary

  1080 

  1095 Ethernet device.

  1081 For performance and realtime purposes, the EtherCAT master needs direct and

  1096 

  1082 exclusive access to the Ethernet hardware. This implies that the network device

  1097 \paragraph{Interrupt-less Operation}\index{Interrupt} EtherCAT frames travel

  1083 must not be connected to the kernel's network stack as usual, because the

  1098 through the logical EtherCAT ring and are then sent back to the master.

  1084 kernel would try to use it as an ordinary Ethernet device.

  1099 Communication is highly deterministic: A frame is sent and will be received

  1085 

  1100 again after a constant time, so there is no need to notify the driver about

  1086 \paragraph{Interrupt-less Operation}

  1101 frame reception: The master can instead query the hardware for received

  1087 \index{Interrupt}

  1102 frames, if it expects them to be already received.

  1088 

  1089 EtherCAT frames travel through the logical EtherCAT ring and are then sent back

  1090 to the master. Communication is highly deterministic: A frame is sent and will

  1091 be received again after a constant time, so there is no need to notify the

  1092 driver about frame reception: The master can instead query the hardware for

  1093 received frames, if it expects them to be already received.

  1126 \paragraph{Ethernet and EtherCAT Devices}

  1135 \paragraph{Ethernet and EtherCAT Devices} Another issue lies in the way Linux

  1127 

  1136 handles devices of the same type.  For example, a

  1128 Another issue lies in the way Linux handles devices of the same type.  For

  1137 PCI\nomenclature{PCI}{Peripheral Component Interconnect, Computer Bus} driver

  1129 example, a PCI\nomenclature{PCI}{Peripheral Component Interconnect, Computer

  1138 scans the PCI bus for devices it can handle. Then it registers itself as the

  1130 Bus} driver scans the PCI bus for devices it can handle. Then it registers

  1139 responsible driver for all of the devices found. The problem is, that an

  1131 itself as the responsible driver for all of the devices found. The problem is,

  1140 unmodified driver can not be told to ignore a device because it will be used

  1132 that an unmodified driver can not be told to ignore a device because it will

  1141 for EtherCAT later. There must be a way to handle multiple devices of the same

  1133 be used for EtherCAT later. There must be a way to handle multiple devices of

  1142 type, where one is reserved for EtherCAT, while the other is treated as an

  1134 the same type, where one is reserved for EtherCAT, while the other is treated

  1143 ordinary Ethernet device.

  1135 as an ordinary Ethernet device.

  1163 \section{Device Selection}

  1171 \section{Generic EtherCAT Device Driver}

  1164 \label{sec:deviceselection}

  1172 \label{sec:generic-driver}

  1165 

  1173 

  1166 After loading the master module, at least one EtherCAT-capable network driver

  1174 Since there are approaches to enable the complete Linux kernel for realtime

  1167 module has to be loaded, that offers its devices to the master (see

  1175 operation \cite{rt-preempt}, it is possible to operate without native

  1168 sec.~\ref{sec:ecdev}. The master module knows the devices to choose from the

  1176 implementations of EtherCAT-capable Ethernet device drivers and use the Linux

  1169 module parameters (see sec.~\ref{sec:mastermod}). If the init script is used

  1177 network stack instead. Fig.~\ref{fig:arch} shows the ``Generic Ethernet Driver

  1170 to start the master, the drivers and devices to use can be specified in the

  1178 Module'', that connects to local Ethernet devices via the network stack. The

  1171 sysconfig file (see sec.~\ref{sec:sysconfig}).

  1179 kernel module is named \lstinline+ec_generic+ and can be loaded after the

  1180 master module like a native EtherCAT-capable Ethernet driver.

  1181 

  1182 The generic device driver scans the network stack for interfaces, that have

  1183 been registered by Ethernet device drivers. It offers all possible devices to

  1184 the EtherCAT master. If the master accepts a device, the generic driver

  1185 creates a packet socket (see \lstinline+man 7 packet+) with

  1186 \lstinline+socket_type+ set to \lstinline+SOCK_RAW+, bound to that device. All

  1187 functions of the device interface (see sec.~\ref{sec:ecdev}) will then operate

  1188 on that socket.

  1189 

  1190 Below are the advantages of this solution:

  1191 

  1192 \begin{itemize}

  1193 \item Any Ethernet hardware, that is covered by a Linux Ethernet driver can be

  1194 used for EtherCAT.

  1195 \item No modifications have to be made to the actual Ethernet drivers.

  1196 \end{itemize}

  1197 

  1198 The generic approach has the following disadvantages:

  1199 

  1200 \begin{itemize}

  1201 \item The performance is a little worse than the native approach, because the

  1202 frame data have to traverse the lower layers of the network stack.

  1203 \item It is not possible to use in-kernel realtime extensions like RTAI with

  1204 the generic driver, because the network stack code uses dynamic memory

  1205 allocations and other things, that could cause the system to freeze in

  1206 realtime context.

  1207 \end{itemize}

  1208 

  1209 %------------------------------------------------------------------------------

  1210 

  1211 \section{Providing Ethernet Devices}

  1212 \label{sec:providing-devices}

  1213 

  1214 After loading the master module, additional module(s) have to be loaded to

  1215 offer devices to the master(s) (see sec.~\ref{sec:ecdev}). The master module

  1216 knows the devices to choose from the module parameters (see

  1217 sec.~\ref{sec:mastermod}). If the init script is used to start the master, the

  1218 drivers and devices to use can be specified in the sysconfig file (see

  1219 sec.~\ref{sec:sysconfig}).

  1220 

  1221 Modules offering Ethernet devices can be

  1222 

  1223 \begin{itemize}

  1224 \item native EtherCAT-capable network driver modules (see

  1225 sec.~\ref{sec:native-drivers}) or

  1226 \item the generic EtherCAT device driver module (see

  1227 sec.~\ref{sec:generic-driver}).

  1228 \end{itemize}

  1199 \section{Patching Network Drivers}

  1256 \section{Patching Native Network Drivers}

  1204 EtherCAT-capable. Unfortunately, there is no standard procedure to enable an

  1261 EtherCAT-capable, using the native approach (see

  1205 Ethernet driver for use with the EtherCAT master, but there are a few common

  1262 sec.~\ref{sec:native-drivers}). Unfortunately, there is no standard procedure

  1206 techniques.

  1263 to enable an Ethernet driver for use with the EtherCAT master, but there are a

  1264 few common techniques.

  2608 Wireshark~\cite{wireshark}, for example.

  2666 Wireshark~\cite{wireshark}, for example. It is also possible to listen to

  2609 

  2667 local network interfaces on the machine running the EtherCAT master directly.

  2610 For convenience, so-called ``debug interfaces'' are supported. Debug

  2668 If the generic Ethernet driver (see sec.~\ref{sec:generic-driver}) is used,

  2611 interfaces are virtual network interfaces allowing to capture EtherCAT traffic

  2669 the network monitor can directly listen on the network interface connected to

  2612 with a network monitor (like Wireshark or tcpdump) running on the master

  2670 the EtherCAT bus.

  2613 machine without using external hardware. To use this functionality, the master

  2671 

  2614 sources have to be configured with the \lstinline+--enable-debug-if+ switch

  2672 When using native Ethernet drivers (see sec.~\ref{sec:native-drivers}), there

  2615 (see sec.~\ref{sec:installation}).

  2673 are no local network interfaces to listen to, because the Ethernet devices

  2674 used for EtherCAT are not registered at the network stack. For that case,

  2675 so-called ``debug interfaces'' are supported, which are virtual network

  2676 interfaces allowing to capture EtherCAT traffic with a network monitor (like

  2677 Wireshark or tcpdump) running on the master machine without using external

  2678 hardware. To use this functionality, the master sources have to be configured

  2679 with the \lstinline+--enable-debug-if+ switch (see

  2680 sec.~\ref{sec:installation}).

  2647 interfaces have to be allocated dynamically. Some Linux realtime extensions do

  2712 interfaces have to be allocated dynamically. Some Linux realtime extensions

  2648 not allow this in realtime context!

  2713 (like RTAI) do not allow this in realtime context!

  3052 \url{http://www.rtai.org}, 2006.

  3129 \url{https://www.rtai.org}, 2010.

  3130 

  3131 \bibitem{rt-preempt} RT PREEMPT HOWTO.

  3132 \url{http://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO}, 2010.