fp@369: %------------------------------------------------------------------------------ fp@369: % fp@369: % IgH EtherCAT Master Documentation fp@369: % fp@369: % $Id$ fp@369: % fp@1095: % vi: spell spelllang=en fp@1095: % fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: % fp@1085: % Conventions fp@1085: % The IgH EtherCAT Master fp@1085: % Feature Summary fp@1085: % License fp@1085: % Architecture fp@1085: % Phases fp@1085: % Behavior (Scanning) TODO fp@1085: % Application Interface fp@1085: % Interface version fp@1085: % Master Requesting and Releasing fp@1085: % Master Locking fp@1085: % Slave configuration fp@1085: % Configuring Pdo assignment and mapping fp@1085: % Domains (memory) fp@1085: % Pdo entry registration fp@1085: % Sdo configuration fp@1085: % Sdo access fp@1085: % Cyclic operation fp@1085: % Ethernet Devices fp@1085: % Device Interface fp@1085: % Device Modules fp@1085: % Network Driver Basics fp@1085: % EtherCAT Network Drivers fp@1085: % Device Selection fp@1085: % The Device Interface fp@1085: % Patching Network Drivers fp@1085: % The Master's State Machines fp@1085: % Master fp@1085: % Slave scanning fp@1085: % SII fp@1085: % Pdo assign/mapping fp@1085: % Slave configuration fp@1085: % State change fp@1085: % Pdo assign/mapping fp@1085: % CoE upload/download/information fp@1085: % Mailbox Protocol Implementations fp@1085: % Ethernet-over-EtherCAT (EoE) fp@1085: % CANopen-over-EtherCAT (CoE) fp@1085: % User Space fp@1085: % The ethercat command fp@1085: % System Integration fp@1085: % The EtherCAT Init Script fp@1085: % The EtherCAT Sysconfig File fp@1085: % Monitoring and Debugging fp@1085: % Installation fp@1085: % Example applications fp@1085: % Bibliography fp@1085: % Glossary fp@1085: % fp@1085: fp@369: \documentclass[a4paper,12pt,BCOR6mm,bibtotoc,idxtotoc]{scrbook} fp@369: fp@369: \usepackage[latin1]{inputenc} fp@369: \usepackage[automark,headsepline]{scrpage2} fp@369: \usepackage{graphicx} fp@369: \usepackage{makeidx} fp@369: \usepackage[refpage]{nomencl} fp@369: \usepackage{listings} fp@369: \usepackage{svn} fp@369: \usepackage{textcomp} fp@369: \usepackage{url} fp@1085: \usepackage{SIunits} fp@371: \usepackage[pdfpagelabels,plainpages=false]{hyperref} fp@369: fp@369: \setlength{\parskip}{0.8ex plus 0.8ex minus 0.5ex} fp@369: \setlength{\parindent}{0mm} fp@369: fp@369: \setcounter{secnumdepth}{\subsubsectionlevel} fp@369: fp@369: \DeclareFontShape{OT1}{cmtt}{bx}{n} fp@369: { fp@369: <5><6><7><8><9><10><10.95><12><14.4><17.28><20.74><24.88>cmttb10 fp@369: }{} fp@369: fp@369: \lstset{basicstyle=\ttfamily\small,numberstyle=\tiny,aboveskip=4mm, fp@1085: belowskip=2mm,escapechar=`} fp@369: \renewcommand\lstlistlistingname{List of Listings} fp@369: fp@917: % Workaround for lstlistoflistings bug fp@917: \makeatletter% --> De-TeX-FAQ fp@917: \renewcommand*{\lstlistoflistings}{% fp@917: \begingroup fp@917: \if@twocolumn fp@917: \@restonecoltrue\onecolumn fp@917: \else fp@917: \@restonecolfalse fp@917: \fi fp@917: \lol@heading fp@917: \setlength{\parskip}{\z@}% fp@917: \setlength{\parindent}{\z@}% fp@917: \setlength{\parfillskip}{\z@ \@plus 1fil}% fp@917: \@starttoc{lol}% fp@917: \if@restonecol\twocolumn\fi fp@917: \endgroup fp@917: } fp@917: \makeatother% --> \makeatletter fp@917: fp@369: \renewcommand\nomname{Glossary} fp@369: fp@369: \newcommand{\IgH}{\raisebox{-0.7667ex} fp@369: {\includegraphics[height=2.2ex]{images/ighsign}}} fp@369: fp@369: \SVN $Date$ fp@369: \SVN $Revision$ fp@369: fp@1085: \newcommand{\masterversion}{1.4.0} fp@1085: \newcommand{\linenum}[1]{\normalfont\textcircled{\tiny #1}} fp@487: fp@369: \makeindex fp@917: \makenomenclature fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \begin{document} fp@369: fp@369: \pagenumbering{roman} fp@369: \pagestyle{empty} fp@369: fp@369: \begin{titlepage} fp@369: \begin{center} fp@369: \rule{\textwidth}{1.5mm} fp@369: fp@369: {\Huge\bf IgH \includegraphics[height=2.4ex]{images/ethercat} fp@487: Master \masterversion\\[1ex] fp@369: Documentation} fp@369: fp@369: \vspace{1ex} fp@369: \rule{\textwidth}{1.5mm} fp@369: fp@369: \vspace{\fill} fp@369: {\Large Florian Pose, \url{fp@igh-essen.com}\\[1ex] fp@369: Ingenieurgemeinschaft \IgH} fp@369: fp@369: \vspace{\fill} fp@369: {\Large Essen, \SVNDate\\[1ex] fp@369: Revision \SVNRevision} fp@369: \end{center} fp@369: \end{titlepage} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \tableofcontents fp@369: \listoftables fp@369: \listoffigures fp@369: \lstlistoflistings fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \newpage fp@369: \pagestyle{scrheadings} fp@369: fp@369: \section*{Conventions} fp@369: \addcontentsline{toc}{section}{Conventions} fp@369: \markleft{Conventions} fp@369: fp@369: The following typographic conventions are used: fp@369: fp@369: \begin{itemize} fp@1085: fp@1085: \item \textit{Italic face} is used for newly introduced terms and file names. fp@1085: fp@1085: \item \texttt{Typewriter face} is used for code examples and command line fp@1085: output. fp@1085: fp@1085: \item \texttt{\textbf{Bold typewriter face}} is used for user input in command fp@1085: lines. fp@1085: fp@369: \end{itemize} fp@369: fp@1085: Data values and addresses are usually specified as hexadecimal values. These fp@1085: are marked in the \textit{C} programming language style with the prefix fp@1085: \lstinline+0x+ (example: \lstinline+0x88A4+). Unless otherwise noted, address fp@1085: values are specified as byte addresses. fp@369: fp@369: Function names are always printed with parentheses, but without fp@1085: parameters. So, if a function \lstinline+ecrt_request_master()+ has fp@1085: empty parentheses, this shall not imply that it has no parameters. fp@1085: fp@1085: If shell commands have to be entered, this is marked by a dollar prompt: fp@1085: fp@1085: \begin{lstlisting} fp@1085: $ fp@369: \end{lstlisting} fp@369: fp@369: Further, if a shell command has to be entered as the superuser, the fp@1085: prompt is a mesh: fp@1085: fp@1085: \begin{lstlisting} fp@1085: # fp@369: \end{lstlisting} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \chapter{The IgH EtherCAT Master} fp@369: \label{chapter:master} fp@369: \pagenumbering{arabic} fp@369: fp@1085: This chapter covers some general information about the EtherCAT master. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \section{Feature Summary} fp@369: \label{sec:summary} fp@369: \index{Master!Features} fp@369: fp@1085: The list below gives a short summary of the master features. fp@369: fp@369: \begin{itemize} fp@1085: fp@1085: \item Designed as a kernel module for Linux 2.6. fp@1085: fp@1086: \item Implemented according to IEC 61158-12 \cite{dlspec} \cite{alspec}. fp@1086: fp@1085: \item Comes with EtherCAT-capable drivers for several common Ethernet devices. fp@1085: fp@369: \begin{itemize} fp@1085: fp@369: \item The Ethernet hardware is operated without interrupts. fp@1085: fp@1085: \item Drivers for additional Ethernet hardware can easily be implemented fp@1085: using the common device interface (see section~\ref{sec:ecdev}) provided by fp@1085: the master module. fp@1085: fp@369: \end{itemize} fp@1085: fp@1085: \item The master module supports multiple EtherCAT masters running in fp@1085: parallel. fp@1085: fp@1085: \item The master code supports any Linux realtime extension through its fp@1085: independent architecture. fp@1085: fp@369: \begin{itemize} fp@1085: fp@1085: \item RTAI\nomenclature{RTAI}{Realtime Application Interface}, fp@1085: ADEOS\nomenclature{ADEOS}{Adaptive Domain Environment for Operating fp@1085: Systems}, etc. fp@1085: fp@369: \item It runs well even without realtime extensions. fp@1085: fp@369: \end{itemize} fp@1085: fp@1085: \item Common ``realtime interface'' for applications, that want to use fp@1085: EtherCAT functionality (see section~\ref{sec:ecrt}). fp@1085: fp@1085: \item \textit{Domains} are introduced, to allow grouping of process fp@1085: data transfers with different slave groups and task periods. fp@1085: fp@369: \begin{itemize} fp@1085: fp@1085: \item Handling of multiple domains with different task periods. fp@1085: fp@1085: \item Automatic calculation of process data mapping, FMMU and sync manager fp@1085: configuration within each domain. fp@1085: fp@369: \end{itemize} fp@1085: fp@1085: \item Communication through several finite state machines. fp@1085: fp@369: \begin{itemize} fp@1085: fp@1085: \item Automatic bus scanning after topology changes. fp@1085: fp@1085: \item Bus monitoring during operation. fp@1085: fp@1085: \item Automatic reconfiguration of slaves (for example after power failure) fp@1085: during operation. fp@1085: fp@369: \end{itemize} fp@1085: fp@1085: \item CANopen-over-EtherCAT (CoE) fp@1085: fp@369: \begin{itemize} fp@1085: fp@1085: \item Sdo upload, download and information service. fp@1085: fp@1085: \item Slave configuration via Sdos. fp@1085: fp@1085: \item Sdo access from user-space and from the application. fp@1085: fp@369: \end{itemize} fp@1085: fp@1085: \item Ethernet-over-EtherCAT (EoE) fp@1085: fp@369: \begin{itemize} fp@1085: fp@1085: \item Transparent use of EoE slaves via virtual network interfaces. fp@1085: fp@1085: \item Natively supports either a switched or a routed EoE network fp@1085: architecture. fp@1085: fp@369: \end{itemize} fp@1085: fp@1085: \item User space command-line-tool ``ethercat``. fp@1085: fp@369: \begin{itemize} fp@1085: fp@1085: \item Showing the current bus with slaves, Pdos and Sdos. fp@1085: \item Showing the bus configuration. fp@1085: \item Showing domains and process data. fp@1085: \item Setting the master's debug level. fp@1085: \item Writing alias addresses. fp@1085: \item Sdo uploading/downloading. fp@1085: \item Reading/writing a slave's SII. fp@1085: \item Setting slave states. fp@1085: \item Generate slave description XML. fp@1085: fp@369: \end{itemize} fp@1085: fp@369: \item Seamless system integration though LSB\nomenclature{LSB}{Linux fp@369: Standard Base} compliance. fp@1085: fp@369: \begin{itemize} fp@1085: fp@1095: \item Master and network device configuration via sysconfig files. fp@1085: fp@1085: \item Init script for master control. fp@1085: fp@369: \end{itemize} fp@1085: fp@369: \item Virtual read-only network interface for monitoring and debugging fp@369: purposes. fp@1085: fp@369: \end{itemize} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \section{License} fp@369: \label{sec:license} fp@369: fp@369: The master code is released under the terms and conditions of the GNU fp@369: General Public License\index{GPL} \cite{gpl} (version 2). Other fp@369: developers, that want to use EtherCAT with Linux systems, are invited fp@369: to use the master code or even participate on development. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \chapter{Architecture} fp@1085: \label{sec:arch} fp@369: \index{Master!Architecture} fp@369: fp@369: The EtherCAT master is integrated into the Linux 2.6 kernel. This was fp@1085: an early design decision, which has been made for several reasons: fp@369: fp@369: \begin{itemize} fp@1085: fp@1085: \item Kernel code has significantly better realtime characteristics, i.~e. fp@1085: less latency than user space code. It was foreseeable, that a fieldbus master fp@1085: has a lot of cyclic work to do. Cyclic work is usually triggered by timer fp@1085: interrupts inside the kernel. The execution delay of a function that processes fp@1085: timer interrupts is less, when it resides in kernel space, because there is no fp@1085: need of time-consuming context switches to a user space process. fp@1085: fp@369: \item It was also foreseeable, that the master code has to directly fp@1085: communicate with the Ethernet hardware. This has to be done in the kernel fp@1085: anyway (through network device drivers), which is one more reason for the fp@1085: master code being in kernel space. fp@1085: fp@369: \end{itemize} fp@369: fp@1085: Figure~\ref{fig:arch} gives a general overview of the master architecture. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@1085: \includegraphics[width=.9\textwidth]{images/architecture} fp@369: \caption{Master architecture} fp@1085: \label{fig:arch} fp@369: \end{figure} fp@369: fp@369: \paragraph{Master Module} fp@369: \index{Master module} fp@369: fp@1086: Kernel module containing one or more EtherCAT master instances (see fp@1086: section~\ref{sec:mastermod}), the ``Device Interface'' (see fp@1086: section~\ref{sec:ecdev}) and the ``Realtime Interface'' (see fp@1086: section~\ref{sec:ecrt}). fp@369: fp@369: \paragraph{Device Modules} fp@369: \index{Device modules} fp@369: fp@1086: EtherCAT-capable Ethernet device driver modules\index{Device modules}, that fp@1086: offer their devices to the EtherCAT master via the device interface (see fp@1086: section~\ref{sec:ecdev}). These modified network drivers can handle network fp@1086: devices used for EtherCAT operation and ``normal'' Ethernet devices in fp@1086: parallel. A master can accept a certain device and then is able to send and fp@1086: receive EtherCAT frames. Ethernet devices declined by the master module are fp@1086: connected to the kernel's network stack as usual. fp@1085: fp@1085: \paragraph{Application Modules} fp@1086: \index{Application module} fp@1086: fp@1086: Kernel modules, that use the EtherCAT master (usually for cyclic exchange of fp@1086: process data with EtherCAT slaves). These modules are not part of the EtherCAT fp@1086: master code\footnote{Although there are some examples provided in the fp@1086: \textit{examples} directory, see chapter~\ref{chapter:examples}}, but have to fp@1086: be generated or written by the user. An application module can ``request'' a fp@1086: master through the realtime interface (see section~\ref{sec:ecrt}). If this fp@1086: succeeds, the module has the control over the master: It can provide a bus fp@1086: configuration and exchange process data. fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1085: \section{Phases} fp@1085: \index{Master phases} fp@1085: fp@1086: The EtherCAT master runs through several phases (see fig.~\ref{fig:phases}): fp@1085: fp@1085: \begin{figure}[htbp] fp@1085: \centering fp@1085: \includegraphics[width=.9\textwidth]{images/phases} fp@1085: \caption{Master phases and transitions} fp@1085: \label{fig:phases} fp@1085: \end{figure} fp@1085: \begin{description} fp@1085: fp@1085: \item[Orphaned phase]\index{Orphaned phase} This mode takes effect, when the fp@1086: master still waits for its Ethernet device to connect. No bus communication is fp@1086: possible until then. fp@1086: fp@1086: \item[Idle phase]\index{Idle phase} takes effect when the master has accepted fp@1086: an Ethernet device, but is not requested by any application yet. The master fp@1085: runs its state machine (see section~\ref{sec:fsm-master}), that automatically fp@1085: scans the bus for slaves and executes pending operations from the user space fp@1085: interface (for example Sdo access). The command-line tool can be used to access fp@1085: the bus, but there is no process data exchange because of the missing bus fp@1085: configuration. fp@1085: fp@1085: \item[Operation phase]\index{Operation phase} The master is requested by an fp@1085: application that can provide a bus configuration and exchange process data. fp@1085: fp@1085: \end{description} fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1085: \section{General behavior} % FIXME fp@1085: \index{Master behavior} fp@1085: fp@1085: \ldots fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1086: \section{Master Module} fp@1086: \label{sec:mastermodule} fp@1085: \index{Master module} fp@1085: fp@1086: The EtherCAT master kernel module \textit{ec\_master} can contain multiple fp@1086: master instances. Each master waits for a certain Ethernet device identified fp@1086: by its MAC address\index{MAC address}. These addresses have to be specified on fp@1086: module loading via the \textit{main\_devices} module parameter. The number of fp@1086: master instances to initialize is taken from the number of MAC addresses fp@1086: given. fp@1086: fp@1086: The below command loads the master module with a single master instance that fp@1086: waits for the Ethernet device with the MAC address fp@1086: \lstinline+00:0E:0C:DA:A2:20+. The master will be accessible via index $0$. fp@1086: fp@1086: \begin{lstlisting} fp@1086: # `\textbf{modprobe ec\_master main\_devices=00:0E:0C:DA:A2:20}` fp@1086: \end{lstlisting} fp@1086: fp@1086: MAC addresses for multiple masters have to be separated by commas: fp@1086: fp@1086: \begin{lstlisting} fp@1086: # `\textbf{modprobe ec\_master main\_devices=00:0E:0C:DA:A2:20,00:e0:81:71:d5:1c}` fp@1086: \end{lstlisting} fp@1086: fp@1086: The two masters can be addressed by their indices 0 and 1 respectively (see fp@1086: figure~\ref{fig:masters}). The master index is needed for the fp@1086: \lstinline+ecrt_master_request()+ function of the realtime interface (see fp@1086: section~\ref{sec:ecrt}) and the \lstinline+--master+ option of the fp@1086: \textit{ethercat} command-line tool (see section~\ref{sec:ethercat}), which fp@1086: defaults to $0$. fp@1085: fp@1085: \begin{figure}[htbp] fp@1085: \centering fp@1085: \includegraphics[width=.5\textwidth]{images/masters} fp@1085: \caption{Multiple masters in one module} fp@1085: \label{fig:masters} fp@1085: \end{figure} fp@1085: fp@1086: \paragraph{Init script} fp@1086: \index{Init script} fp@1086: fp@1086: Most probably you won't want to load the master module and the Ethernet driver fp@1086: modules manually, but start the master as a service. See fp@1086: section~\ref{sec:system} on how to do this. fp@1086: fp@1086: \paragraph{Syslog} fp@1086: fp@1086: The master module outputs information about it's state and events to the fp@1086: kernel ring buffer. These also end up in the system logs. The above module fp@1086: loading command should result in the messages below: fp@1086: fp@1086: \begin{lstlisting} fp@1086: # `\textbf{dmesg | tail -2}` fp@1086: EtherCAT: Master driver `\masterversion` fp@1086: EtherCAT: 2 masters waiting for devices. fp@1086: fp@1086: # `\textbf{tail -2 /var/log/messages}` fp@1086: Jul 4 10:22:45 ethercat kernel: EtherCAT: Master driver `\masterversion` fp@1086: Jul 4 10:22:45 ethercat kernel: EtherCAT: 2 masters waiting fp@1086: for devices. fp@1086: \end{lstlisting} fp@1086: fp@1086: All EtherCAT master output is prefixed with \lstinline+EtherCAT+ which makes fp@1086: searching the logs easier. fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1085: \section{Handling of Process Data} % FIXME fp@369: \label{sec:processdata} fp@369: fp@1085: \ldots fp@1085: fp@369: \paragraph{Process Data Image} fp@369: \index{Process data} fp@369: fp@1085: The slaves offer their inputs and outputs by presenting the master so-called fp@1085: ``Process Data Objects'' (Pdos\index{Pdo}). The available Pdos can be fp@1085: determined by reading out the slave's TXPDO and RXPDO E$^2$PROM categories. The fp@1085: application can register the Pdos for data exchange during cyclic operation. fp@1085: The sum of all registered Pdos defines the ``process data image'', which is fp@1085: exchanged via the ``Logical ReadWrite'' datagrams introduced fp@369: in~\cite[section~5.4.2.4]{dlspec}. fp@369: fp@369: \paragraph{Process Data Domains} fp@369: \index{Domain} fp@369: fp@1085: The process data image can be easily managed by creating so-called fp@814: ``domains'', which group Pdos and allocate the datagrams needed to fp@369: exchange them. Domains are mandatory for process data exchange, so fp@369: there has to be at least one. They were introduced for the following fp@369: reasons: fp@369: fp@369: \begin{itemize} fp@369: \item The maximum size of a ``Logical ReadWrite'' datagram is limited fp@369: due to the limited size of an Ethernet frame: The maximum data size fp@369: is the Ethernet data field size minus the EtherCAT frame header, fp@369: EtherCAT datagram header and EtherCAT datagram footer: $1500 - 2 - fp@369: 12 - 2 = 1484$ octets. If the size of the process data image exceeds fp@369: this limit, multiple frames have to be sent, and the image has to be fp@369: partitioned for the use of multiple datagrams. A domain manages this fp@369: automatically. fp@814: \item Not every Pdo has to be exchanged with the same frequency: The fp@814: values of Pdos can vary slowly over time (for example temperature fp@369: values), so exchanging them with a high frequency would just waste fp@369: bus bandwidth. For this reason, multiple domains can be created, to fp@814: group different Pdos and so allow separate exchange. fp@369: \end{itemize} fp@369: fp@369: There is no upper limit for the number of domains, but each domain fp@369: occupies one FMMU in each slave involved, so the maximum number of fp@369: domains is also limited by the slaves' capabilities. fp@369: fp@369: \paragraph{FMMU Configuration} fp@369: \index{FMMU!Configuration} fp@369: fp@1085: An application can register Pdos for process data exchange. Every fp@814: Pdo is part of a memory area in the slave's physical memory, that is fp@369: protected by a sync manager \cite[section~6.7]{dlspec} for fp@369: synchronized access. In order to make a sync manager react on a fp@369: datagram accessing its memory, it is necessary to access the last byte fp@369: covered by the sync manager. Otherwise the sync manager will not react fp@369: on the datagram and no data will be exchanged. That is why the whole fp@369: synchronized memory area has to be included into the process data fp@814: image: For example, if a certain Pdo of a slave is registered for fp@369: exchange with a certain domain, one FMMU will be configured to map the fp@814: complete sync-manager-protected memory, the Pdo resides in. If a fp@814: second Pdo of the same slave is registered for process data exchange fp@814: within the same domain, and this Pdo resides in the same fp@814: sync-manager-protected memory as the first Pdo, the FMMU configuration fp@369: is not touched, because the appropriate memory is already part of the fp@814: domain's process data image. If the second Pdo belongs to another fp@369: sync-manager-protected area, this complete area is also included into fp@369: the domains process data image. See figure~\ref{fig:fmmus} for an fp@369: overview, how FMMU's are configured to map physical memory to logical fp@369: process data images. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=\textwidth]{images/fmmus} fp@1085: \caption{FMMU configuration for several domains} fp@369: \label{fig:fmmus} fp@369: \end{figure} fp@369: fp@1085: \paragraph{Process Data Pointers} % FIXME fp@1085: fp@1085: The figure also demonstrates the way, the application can access the exchanged fp@1085: process data: At Pdo registration, the application has to provide the address fp@1085: of a process data pointer. Upon calculation of the domain image and allocation fp@1085: of process data memory, this pointer is redirected to the appropriate location fp@1085: inside the domain's process data memory and can later be easily dereferenced by fp@1085: the module code. fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1085: \chapter{Application Interface} fp@1085: \label{sec:ecrt} fp@1085: \index{Application interface} fp@1085: fp@1094: The application interface provides functions and data structures for fp@1094: applications to access and use an EtherCAT master. The complete documentation fp@1094: of the interface is included as Doxygen~\cite{doxygen} comments in the header fp@1094: file \textit{include/ecrt.h}. You can either directly view the file comments fp@1094: or generate an HTML documentation as described in section~\ref{sec:gendoc}. fp@1094: fp@1094: The following sections cover a general description of the application fp@1094: interface. fp@1094: fp@1094: Every application should use the master in two steps: fp@1094: fp@1094: \begin{description} fp@1094: fp@1094: \item[Configuration] The master is requested and the configuration is applied. fp@1094: Domains are created Slaves are configured and Pdo entries are registered (see fp@1094: section~\ref{sec:masterconfig}). fp@1094: fp@1094: \item[Operation] Cyclic code is run, process data is exchanged (see fp@1094: section~\ref{sec:cyclic}). fp@1094: fp@1094: \end{description} fp@1094: fp@1094: %------------------------------------------------------------------------------ fp@1094: fp@1094: \section{Master Configuration} fp@1094: \label{sec:masterconfig} fp@1094: fp@1094: \ldots fp@1094: fp@1094: \begin{figure}[htbp] fp@1094: \centering fp@1094: \includegraphics[width=.8\textwidth]{images/app-config} fp@1094: \caption{Master configuration structures} fp@1094: \label{fig:app-config} fp@1094: \end{figure} fp@1094: fp@1094: %------------------------------------------------------------------------------ fp@1094: fp@1094: \section{Cyclic Operation} fp@1094: \label{sec:cyclic} fp@1094: fp@1094: \ldots fp@1094: % FIXME PDOS endianess fp@1094: fp@1094: fp@1094: %------------------------------------------------------------------------------ fp@1094: fp@1094: \section{Concurrent Master Access} % FIXME fp@1085: \label{sec:concurr} fp@1085: \index{Concurrency} fp@1085: fp@1085: In some cases, one master is used by several instances, for example when an fp@1085: application does cyclic process data exchange, and there are EoE-capable slaves fp@1085: that require to exchange Ethernet data with the kernel (see fp@1085: section~\ref{sec:eoeimp}). For this reason, the master is a shared resource, fp@1085: and access to it has to be sequentialized. This is usually done by locking with fp@1085: semaphores, or other methods to protect critical sections. fp@1085: fp@1085: The master itself can not provide locking mechanisms, because it has no chance fp@1085: to know the appropriate kind of lock. Imagine, the application uses RTAI fp@1085: functionality, then ordinary kernel semaphores would not be sufficient. For fp@1085: that, an important design decision was made: The application that reserved a fp@1085: master must have the total control, therefore it has to take responsibility for fp@1085: providing the appropriate locking mechanisms. If another instance wants to fp@1085: access the master, it has to request the master lock by callbacks, that have to fp@1085: be set by the application. Moreover the application can deny access to the fp@1085: master if it considers it to be awkward at the moment. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@1085: \includegraphics[width=.6\textwidth]{images/master-locks} fp@1085: \caption{Concurrent master access} fp@1085: \label{fig:locks} fp@369: \end{figure} fp@369: fp@1085: Figure~\ref{fig:locks} exemplary shows, how two processes share one master: The fp@1085: application's cyclic task uses the master for process data exchange, while the fp@1085: master-internal EoE process uses it to communicate with EoE-capable slaves. fp@1085: Both have to acquire the master lock before access: The application task can fp@1085: access the lock natively, while the EoE process has to use the callbacks. fp@1085: Section~\ref{sec:concurrency} gives an example, of how to implement this. fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1085: \chapter{Ethernet devices} fp@1085: \label{sec:devices} fp@1085: fp@1085: The EtherCAT protocol is based on the Ethernet standard. That's why the master fp@1085: relies on standard Ethernet hardware to communicate with the bus. fp@1085: fp@1085: The term \textit{device} is used as a synonym for Ethernet network interface fp@1085: hardware. There are device driver modules that handle Ethernet hardware, which fp@1085: the master can use to connect to an EtherCAT bus. fp@369: fp@369: Section~\ref{sec:networkdrivers} offers an overview of general Linux fp@369: network driver modules, while section~\ref{sec:requirements} will show fp@369: the requirements to an EtherCAT-enabled network driver. Finally, fp@369: sections~\ref{sec:seldev} to~\ref{sec:patching} show how to fulfill fp@369: these requirements and implement such a driver module. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{Network Driver Basics} fp@369: \label{sec:networkdrivers} fp@369: \index{Network drivers} fp@369: fp@369: EtherCAT relies on Ethernet hardware and the master needs a physical fp@369: Ethernet device to communicate with the bus. Therefore it is necessary fp@369: to understand how Linux handles network devices and their drivers, fp@369: respectively. fp@369: fp@369: \paragraph{Tasks of a Network Driver} fp@369: fp@1095: Network device drivers usually handle the lower two layers of the OSI model, fp@1095: that is the physical layer and the data-link layer. A network device itself fp@1095: natively handles the physical layer issues: It represents the hardware to fp@1095: connect to the medium and to send and receive data in the way, the physical fp@1095: layer protocol describes. The network device driver is responsible for getting fp@1095: data from the kernel's networking stack and forwarding it to the hardware, fp@1095: that does the physical transmission. If data is received by the hardware fp@1095: respectively, the driver is notified (usually by means of an interrupt) and fp@1095: has to read the data from the hardware memory and forward it to the network fp@1095: stack. There are a few more tasks, a network device driver has to handle, fp@1095: including queue control, statistics and device dependent features. fp@369: fp@369: \paragraph{Driver Startup} fp@369: fp@369: Usually, a driver searches for compatible devices on module loading. fp@369: For PCI drivers, this is done by scanning the PCI bus and checking for fp@369: known device IDs. If a device is found, data structures are allocated fp@369: and the device is taken into operation. fp@369: fp@369: \paragraph{Interrupt Operation} fp@369: \index{Interrupt} fp@369: fp@369: A network device usually provides a hardware interrupt that is used to fp@1085: notify the driver of received frames and success of transmission, or fp@369: errors, respectively. The driver has to register an interrupt service fp@369: routine (ISR\index{ISR}\nomenclature{ISR}{Interrupt Service Routine}), fp@369: that is executed each time, the hardware signals such an event. If the fp@369: interrupt was thrown by the own device (multiple devices can share one fp@369: hardware interrupt), the reason for the interrupt has to be determined fp@369: by reading the device's interrupt register. For example, if the flag fp@369: for received frames is set, frame data has to be copied from hardware fp@369: to kernel memory and passed to the network stack. fp@369: fp@369: \paragraph{The net\_device structure} fp@369: \index{net\_device} fp@369: fp@369: The driver registers a \textit{net\_device} structure for each device fp@369: to communicate with the network stack and to create a ``network fp@369: interface''. In case of an Ethernet driver, this interface appears as fp@369: \textit{ethX}, where X is a number assigned by the kernel on fp@369: registration. The \textit{net\_device} structure receives events fp@1085: (either from user space or from the network stack) via several fp@369: callbacks, which have to be set before registration. Not every fp@369: callback is mandatory, but for reasonable operation the ones below are fp@369: needed in any case: fp@369: fp@369: \begin{description} fp@1095: fp@1095: \item[open()] This function is called when network communication has to be fp@1095: started, for example after a command \textit{ifconfig ethX up} from user fp@1095: space. Frame reception has to be enabled by the driver. fp@1095: fp@1095: \item[stop()] The purpose of this callback is to ``close'' the device, i.~e. fp@1095: make the hardware stop receiving frames. fp@1095: fp@1095: \item[hard\_start\_xmit()] This function is cal\-led for each frame that has fp@1095: to be transmitted. The network stack passes the frame as a pointer to an fp@1095: \textit{sk\_buff} structure (``socket buffer''\index{Socket buffer}, see fp@1095: below), which has to be freed after sending. fp@1095: fp@1095: \item[get\_stats()] This call has to return a pointer to the device's fp@1095: \textit{net\_device\_stats} structure, which permanently has to be filled with fp@1095: frame statistics. This means, that every time a frame is received, sent, or an fp@1095: error happened, the appropriate counter in this structure has to be increased. fp@1095: fp@1095: \end{description} fp@1095: fp@1095: The actual registration is done with the \lstinline+register_netdev()+ call, fp@1095: unregistering is done with \lstinline+unregister_netdev()+. fp@369: fp@369: \paragraph{The netif Interface} fp@369: \index{netif} fp@369: fp@1085: All other communication in the direction interface $\to$ network stack is done fp@1095: via the \lstinline+netif_*()+ calls. For example, on successful device fp@1095: opening, the network stack has to be notified, that it can now pass frames to fp@1095: the interface. This is done by calling \lstinline+netif_start_queue()+. After fp@1095: this call, the \lstinline+hard_start_xmit()+ callback can be called by the fp@1095: network stack. Furthermore a network driver usually manages a frame fp@1095: transmission queue. If this gets filled up, the network stack has to be told fp@1095: to stop passing further frames for a while. This happens with a call to fp@1095: \lstinline+netif_stop_queue()+. If some frames have been sent, and there is fp@1095: enough space again to queue new frames, this can be notified with fp@1095: \lstinline+netif_wake_queue()+. Another important call is fp@1095: \lstinline+netif_receive_skb()+\footnote{This function is part of the NAPI fp@1095: (``New API''), that replaces the kernel 2.4 technique for interfacing to the fp@1095: network stack (with \lstinline+netif_rx()+). NAPI is a technique to improve fp@1095: network performance on Linux. Read more in fp@1095: \url{http://www.cyberus.ca/~hadi/usenix-paper.tgz}.}: It passes a frame to the fp@1095: network stack, that was just received by the device. Frame data has to be fp@1085: packed into a so-called ``socket buffer'' for that (see below). fp@369: fp@369: \paragraph{Socket Buffers} fp@369: \index{Socket buffer} fp@369: fp@1095: Socket buffers are the basic data type for the whole network stack. They fp@1095: serve as containers for network data and are able to quickly add data headers fp@1095: and footers, or strip them off again. Therefore a socket buffer consists of an fp@1095: allocated buffer and several pointers that mark beginning of the buffer fp@1095: (\textit{head}), beginning of data (\textit{data}), end of data fp@1095: (\textit{tail}) and end of buffer (\textit{end}). In addition, a socket buffer fp@1095: holds network header information and (in case of received data) a pointer to fp@1095: the \textit{net\_device}, it was received on. There exist functions that fp@1095: create a socket buffer (\lstinline+dev_alloc_skb()+), add data either from fp@1095: front (\lstinline+skb_push()+) or back (\lstinline+skb_put()+), remove data fp@1095: from front (\lstinline+skb_pull()+) or back (\lstinline+skb_trim()+), or fp@1095: delete the buffer (\lstinline+kfree_skb()+). A socket buffer is passed from fp@1095: layer to layer, and is freed by the layer that uses it the last time. In case fp@1095: of sending, freeing has to be done by the network driver. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{EtherCAT Device Drivers} fp@369: \label{sec:requirements} fp@369: fp@1095: There are a few requirements for Ethernet network devices to function as fp@1095: EtherCAT devices, when connected to an EtherCAT bus. fp@369: fp@369: \paragraph{Dedicated Interfaces} fp@369: fp@1095: For performance and realtime purposes, the EtherCAT master needs direct and fp@1095: exclusive access to the Ethernet hardware. This implies that the network fp@1095: device must not be connected to the kernel's network stack as usual, because fp@1095: the kernel would try to use it as an ordinary Ethernet device. fp@369: fp@369: \paragraph{Interrupt-less Operation} fp@369: \index{Interrupt} fp@369: fp@1095: EtherCAT frames travel through the logical EtherCAT ring and are then sent fp@1095: back to the master. Communication is highly deterministic: A frame is sent and fp@1095: will be received again after a constant time. Therefore, there is no need to fp@1095: notify the driver about frame reception: The master can instead query the fp@1095: hardware for received frames. fp@1095: fp@1095: Figure~\ref{fig:interrupt} shows two workflows for cyclic frame transmission fp@1095: and reception with and without interrupts. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.8\textwidth]{images/interrupt} fp@369: \caption{Interrupt Operation versus Interrupt-less Operation} fp@369: \label{fig:interrupt} fp@369: \end{figure} fp@369: fp@1095: In the left workflow ``Interrupt Operation'', the data from the last cycle is fp@1095: first processed and a new frame is assembled with new datagrams, which is then fp@1095: sent. The cyclic work is done for now. Later, when the frame is received fp@1095: again by the hardware, an interrupt is triggered and the ISR is executed. The fp@1095: ISR will fetch the frame data from the hardware and initiate the frame fp@1095: dissection: The datagrams will be processed, so that the data is ready for fp@1095: processing in the next cycle. fp@1095: fp@1095: In the right workflow ``Interrupt-less Operation'', there is no hardware fp@1095: interrupt enabled. Instead, the hardware will be polled by the master by fp@1095: executing the ISR. If the frame has been received in the meantime, it will be fp@1095: dissected. The situation is now the same as at the beginning of the left fp@1095: workflow: The received data is processed and a new frame is assembled and fp@1095: sent. There is nothing to do for the rest of the cycle. fp@1095: fp@1095: The interrupt-less operation is desirable, because there is simply no need for fp@1095: an interrupt. Moreover hardware interrupts are not conducive in improving the fp@1095: driver's realtime behaviour: Their indeterministic incidences contribute to fp@1095: increasing the jitter. Besides, if a realtime extension (like RTAI) is used, fp@1095: some additional effort would have to be made to prioritize interrupts. fp@369: fp@369: \paragraph{Ethernet and EtherCAT Devices} fp@369: fp@1095: Another issue lies in the way Linux handles devices of the same type. For fp@1095: example, a PCI\nomenclature{PCI}{Peripheral Component Interconnect, Computer fp@1095: Bus} driver scans the PCI bus for devices it can handle. Then it registers fp@1095: itself as the responsible driver for all of the devices found. The problem is, fp@1095: that an unmodified driver can not be told to ignore a device because it will fp@1095: be used for EtherCAT later. There must be a way to handle multiple devices of fp@1095: the same type, where one is reserved for EtherCAT, while the other is treated fp@369: as an ordinary Ethernet device. fp@369: fp@1095: For all this reasons, the author decided that the only acceptable solution is fp@1095: to modify standard Ethernet drivers in a way that they keep their normal fp@1095: functionality, but gain the ability to treat one or more of the devices as fp@1095: EtherCAT-capable. fp@369: fp@369: Below are the advantages of this solution: fp@369: fp@369: \begin{itemize} fp@369: \item No need to tell the standard drivers to ignore certain devices. fp@369: \item One networking driver for EtherCAT and non-EtherCAT devices. fp@369: \item No need to implement a network driver from scratch and running fp@369: into issues, the former developers already solved. fp@369: \end{itemize} fp@369: fp@369: The chosen approach has the following disadvantages: fp@369: fp@369: \begin{itemize} fp@369: \item The modified driver gets more complicated, as it must handle fp@369: EtherCAT and non-EtherCAT devices. fp@369: \item Many additional case differentiations in the driver code. fp@1085: \item Changes and bug fixes on the standard drivers have to be ported fp@369: to the Ether\-CAT-capable versions from time to time. fp@369: \end{itemize} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{Device Selection} fp@1085: \label{sec:deviceselection} fp@369: fp@369: After loading the master module, at least one EtherCAT-capable network fp@369: driver module has to be loaded, that connects one of its devices to fp@369: the master. To specify an EtherCAT device and the master to connect fp@369: to, all EtherCAT-capable network driver modules should provide two fp@369: module parameters: fp@369: fp@369: \begin{description} fp@369: \item[ec\_device\_index] PCI device index of the device that is fp@369: connected to the EtherCAT bus. If this parameter is left away, all fp@369: devices found are treated as ordinary Ethernet devices. Default: fp@369: $-1$ fp@369: \item[ec\_master\_index] Index of the master to connect to. Default: fp@369: $0$ fp@369: \end{description} fp@369: fp@369: The following command loads the EtherCAT-capable RTL8139 device fp@369: driver, telling it to handle the second device as an EtherCAT device fp@369: and connecting it to the first master: fp@369: fp@1085: \begin{lstlisting}[gobble=2] fp@379: # `\textbf{modprobe ec\_8139too ec\_device\_index=1}` fp@369: \end{lstlisting} fp@369: fp@369: Usually, this command does not have to be entered manually, but is fp@369: called by the EtherCAT init script. See section~\ref{sec:init} for fp@369: more information. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{The Device Interface} fp@369: \label{sec:ecdev} fp@369: \index{Device interface} fp@369: fp@369: An anticipation to the section about the master module fp@369: (section~\ref{sec:mastermod}) has to be made in order to understand fp@369: the way, a network device driver module can connect a device to a fp@369: specific EtherCAT master. fp@369: fp@369: The master module provides a ``device interface'' for network device fp@369: drivers. To use this interface, a network device driver module must fp@369: include the header fp@369: \textit{devices/ecdev.h}\nomenclature{ecdev}{EtherCAT Device}, coming fp@369: with the EtherCAT master code. This header offers a function interface fp@369: for EtherCAT devices which is explained below. All functions of the fp@369: device interface are named with the prefix \textit{ecdev}. fp@369: fp@369: \paragraph{Device Registration} fp@369: fp@369: A network device driver can connect a physical device to an EtherCAT fp@369: master with the \textit{ecdev\_register()} function. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C] fp@369: ec_device_t *ecdev_register(unsigned int master_index, fp@369: struct net_device *net_dev, fp@369: ec_isr_t isr, fp@369: struct module *module); fp@369: \end{lstlisting} fp@369: fp@369: The first parameter \textit{master\_index} must be the index of the fp@369: EtherCAT master to connect to (see section~\ref{sec:mastermod}), fp@369: followed by \textit{net\_dev}, the pointer to the corresponding fp@369: net\_device structure, which represents the network device to connect. fp@369: The third parameter \textit{isr} must be a pointer to the interrupt fp@369: service routine (ISR\index{ISR}) handling the device. The master will fp@369: later execute the ISR in order to receive frames and to update the fp@369: device status. The last parameter \textit{module} must be the pointer fp@369: to the device driver module, which is usually accessible via the macro fp@369: \textit{THIS\_MODULE} (see next paragraph). On success, the function fp@369: returns a pointer to an \textit{ec\_device\_t} object, which has to be fp@369: specified when calling further functions of the device interface. fp@369: Therefore the device module has to store this pointer for future use. fp@369: In error case, the \textit{ecdev\_register()} returns \textit{NULL}, fp@369: which means that the device could not be registered. The reason for fp@1085: this is printed to \textit{Syslog}\index{Syslog}. In this case, the fp@369: device module is supposed to abort the module initialisation and let fp@369: the \textit{insmod} command fail. fp@369: fp@369: \paragraph{Implicit Dependencies} fp@369: fp@1085: The reason for the module pointer has to be specified at device registration is fp@1085: a non-trivial one: The master has to know about the module, because there will fp@1085: be an implicit dependency between the device module and a later connected fp@1085: application module: When an application module connects to the master, the use fp@1085: count of the master module will be increased, so that the master module can not fp@1085: be unloaded for the time of the connection. This is reasonable, and so fp@1085: automatically done by the kernel. The kernel knows about this dependency, fp@1085: because the application module uses kernel symbols provided by the master fp@1085: module. Moreover it is mandatory, that the device module can be unloaded fp@1085: neither, because it is implicitly used by the application module, too. fp@1085: Unloading it would lead to a fatal situation, because the master would have no fp@1085: device to send and receive frames for the application. This dependency can not fp@1085: be detected automatically, because the application module does not use any fp@1085: symbols of the device module. Therefore the master explicitly increments the fp@1085: use counter of the connected device module upon connection of an application fp@1085: and decrements it, if it disconnects again. In this manner, it is impossible to fp@1085: unload a device module while the master is in use. This is done with the kernel fp@1085: function pair \textit{try\_module\_get()} fp@1085: \index{try\_module\_get@\textit{try\_module\_get()}} and \textit{module\_put()} fp@1085: \index{module\_put@\textit{module\_put()}}. The first one increases the use fp@1085: count of a module and only fails, if the module is currently being unloaded. fp@1085: The last one decreases the use count again and never fails. Both functions take fp@1085: a pointer to the module as their argument, which the device module therefore fp@1085: has to specify upon device registration. fp@369: fp@369: \paragraph{Device Unregistering} fp@369: fp@1085: The deregistration of a device is usually done in the device module's cleanup fp@1085: function, by calling the \textit{ecdev\_unregister()} function and specifying fp@1085: the master index and a pointer to the device object again. fp@1085: fp@1085: \begin{lstlisting}[gobble=2,language=C] fp@369: void ecdev_unregister(unsigned int master_index, fp@369: ec_device_t *device); fp@369: \end{lstlisting} fp@369: fp@369: This function can fail too (if the master index is invalid, or the fp@369: given device was not registered), but due to the fact, that this fp@369: failure can not be dealt with appropriately, because the device module fp@369: is unloading anyway, the failure code would not be of any interest. So fp@369: the function has a void return value. fp@369: fp@369: \paragraph{Starting the Master} fp@369: fp@369: When a device has been initialized completely and is ready to send and fp@369: receive frames, the master has to be notified about this by calling fp@369: the \textit{ecdev\_start()} function. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C] fp@369: int ecdev_start(unsigned int master_index); fp@369: \end{lstlisting} fp@369: fp@369: The master will then enter ``Idle Mode'' and start scanning the bus fp@369: (and possibly handling EoE slaves). Moreover it will make the bus fp@369: accessible via Sysfs interface and react to user interactions. The fp@369: function takes one parameter \textit{master\_index}, which has to be fp@369: the same as at the call to \textit{ecdev\_register()}. The return fp@369: value will be non-zero if the starting process failed. In this case fp@369: the device module is supposed to abort the init sequence and make the fp@369: init function return an error code. fp@369: fp@369: \paragraph{Stopping the Master} fp@369: fp@369: Before a device can be unregistered, the master has to be stopped by fp@369: calling the \textit{ecdev\_stop()} function. It will stop processing fp@369: messages of EoE slaves and leave ``Idle Mode''. The only parameter is fp@369: \textit{master\_index}. This function can not fail. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C] fp@369: void ecdev_stop(unsigned int master_index); fp@369: \end{lstlisting} fp@369: fp@369: A subsequent call to \textit{ecdev\_unregister()} will now unregister fp@369: the device savely. fp@369: fp@369: \paragraph{Receiving Frames} fp@369: fp@369: The interrupt service routine handling device events usually has a fp@369: section where new frames are fetched from the hardware and forwarded fp@369: to the kernel network stack via \textit{netif\_receive\_skb()}. For an fp@369: EtherCAT-capable device, this has to be replaced by calling the fp@369: \textit{ecdev\_receive()} function to forward the received data to the fp@369: connected EtherCAT master instead. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C] fp@369: void ecdev_receive(ec_device_t *device, fp@369: const void *data, fp@369: size_t size); fp@369: \end{lstlisting} fp@369: fp@369: This function takes 3 arguments, a pointer to the device object fp@369: (\textit{device}), a pointer to the received data, and the size of the fp@369: received data. The data range has to include the Ethernet headers fp@369: starting with the destination address and reach up to the last octet fp@369: of EtherCAT data, excluding the FCS. Most network devices handle the fp@369: FCS in hardware, so it is not seen by the driver code and therefore fp@369: doesn't have to be cut off manually. fp@369: fp@369: \paragraph{Handling the Link Status} fp@369: fp@1085: Information about the link status (i.~e. if there is a carrier signal detected fp@1085: on the physical port) is also important to the master. This information is fp@1085: usually gathered by the ISR and should be forwarded to the master by calling fp@1085: the \textit{ecdev\_link\_state()} function. The master then can react on this fp@1085: and warn the application of a lost link. fp@1085: fp@1085: \begin{lstlisting}[gobble=2,language=C] fp@369: void ecdev_link_state(ec_device_t *device, fp@369: uint8_t new_state); fp@369: \end{lstlisting} fp@369: fp@369: The parameter \textit{device} has to be a pointer to the device object fp@369: returned by \textit{ecdev\_\-register()}. With the second parameter fp@369: \textit{new\_state}, the new link state is passed: 1, if the link went fp@369: up, and 0, if it went down. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{Patching Network Drivers} fp@369: \label{sec:patching} fp@369: \index{Network drivers} fp@369: fp@369: This section will demonstrate, how to make a standard Ethernet driver fp@369: EtherCAT-capable. The below code examples are taken out of the fp@369: modified RealTek RTL8139 driver coming with the EtherCAT master fp@369: (\textit{devices/8139too.c}). The driver was originally developed by fp@369: Donald Becker, and is currently maintained by Jeff Garzik. fp@369: fp@369: Unfortunately, there is no standard procedure to enable an Ethernet fp@369: driver for use with the EtherCAT master, but there are a few common fp@369: techniques, that are described in this section. fp@369: fp@369: \begin{enumerate} fp@369: \item A first simple rule is, that \textit{netif\_*()}-calls must be fp@369: strictly avoided for all EtherCAT devices. As mentioned before, fp@369: EtherCAT devices have no connection to the network stack, and fp@369: therefore must not call its interface functions. fp@369: \item Another important thing is, that EtherCAT devices should be fp@369: operated without interrupts. So any calls of registering interrupt fp@369: handlers and enabling interrupts at hardware level must be avoided, fp@369: too. fp@369: \item The master does not use a new socket buffer for each send fp@369: operation: Instead there is a fix one allocated on master fp@369: initialization. This socket buffer is filled with an EtherCAT frame fp@369: with every send operation and passed to the fp@369: \textit{hard\_start\_xmit()} callback. For that it is necessary, fp@369: that the socket buffer is not be freed by the network driver as fp@369: usual. fp@369: \end{enumerate} fp@369: fp@369: As mentioned before, the driver will handle both EtherCAT and ordinary fp@369: Ethernet devices. This implies, that for each device-dependent fp@369: operation, it has to be checked if an EtherCAT device is involved, or fp@369: just an Ethernet device. For means of simplicity, this example driver fp@369: will only handle one EtherCAT device. This makes the case fp@369: differentiations easier. fp@369: fp@369: \paragraph{Global Variables} fp@369: fp@369: First of all, there have to be additional global variables declared, fp@369: as shown in the listing: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: static int ec_device_index = -1; fp@369: static int ec_device_master_index = 0; fp@369: static ec_device_t *rtl_ec_dev; fp@369: struct net_device *rtl_ec_net_dev = NULL; fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{1} -- \linenum{2}] To fp@369: comply to the requirements for parameters of EtherCAT device modules fp@369: described in section~\ref{sec:seldev}, there have to be additional fp@369: parameter variables: \textit{ec\_\-device\_\-index} holds the index fp@369: of the EtherCAT device and defaults to $-1$ (no EtherCAT device), fp@369: while \textit{ec\_device\_master\_index} stores index of the master, fp@369: the single device will be connected to. Default: $0$ fp@1085: \item[\linenum{3}] \textit{rtl\_ec\_dev} will be fp@369: the pointer to the later registered RealTek EtherCAT device, which fp@369: can be used as a parameter for device methods. fp@1085: \item[\linenum{4}] \textit{rtl\_ec\_net\_dev} is fp@369: a pointer to the \textit{net\_device} structure of the dedicated fp@369: device and is set while scanning the PCI bus and finding the device fp@369: with the specified index. This is done inside the fp@369: \textit{pci\_module\_init()} function executed as the first thing on fp@369: module loading. fp@369: \end{description} fp@369: fp@369: \paragraph{Module Initialization} fp@369: fp@369: Below is the (shortened) coding of the device driver's module init fp@369: function: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: static int __init rtl8139_init_module(void) fp@369: { fp@369: if (pci_module_init(&rtl8139_pci_driver) < 0) { fp@369: printk(KERN_ERR "Failed to init PCI mod.\n"); fp@369: goto out_return; fp@369: } fp@369: fp@369: if (rtl_ec_net_dev) { fp@369: printk(KERN_INFO "Registering" fp@369: " EtherCAT device...\n"); fp@369: if (!(rtl_ec_dev = fp@369: ecdev_register(ec_device_master_index, fp@369: rtl_ec_net_dev, fp@369: rtl8139_interrupt, fp@369: THIS_MODULE))) { fp@369: printk(KERN_ERR "Failed to reg." fp@369: " EtherCAT device!\n"); fp@369: goto out_unreg_pci; fp@369: } fp@369: fp@369: printk(KERN_INFO "Starting EtherCAT" fp@369: " device...\n"); fp@369: if (ecdev_start(ec_device_master_index)) { fp@369: printk(KERN_ERR "Failed to start" fp@369: " EtherCAT device!\n"); fp@369: goto out_unreg_ec; fp@369: } fp@369: } else { fp@369: printk(KERN_WARNING "No EtherCAT device" fp@369: " registered!\n"); fp@369: } fp@369: fp@369: return 0; fp@369: fp@369: out_unreg_ec: fp@369: ecdev_unregister(ec_device_master_index, rtl_ec_dev); fp@369: out_unreg_pci: fp@369: pci_unregister_driver(&rtl8139_pci_driver); fp@369: out_return: fp@369: return -1; fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{3}] This call initializes all fp@369: RTL8139-compatible devices found on the pci bus. If a device with fp@369: index \textit{ec\_device\_index} is found, a pointer to its fp@369: \textit{net\_device} structure is stored in fp@369: \textit{rtl\_ec\_net\_dev} for later use (see next listings). fp@1085: \item[\linenum{8}] If the specified device was fp@369: found, \textit{rtl\_ec\_net\_dev} is non-zero. fp@1085: \item[\linenum{11}] The device is connected to fp@369: the specified master with a call to \textit{ecdev\_register()}. If fp@369: this fails, module loading is aborted. fp@1085: \item[\linenum{23}] The device registration was fp@369: successful and the master is started. This can fail, which aborts fp@369: module loading. fp@1085: \item[\linenum{29}] If no EtherCAT device was fp@369: found, a warning is output. fp@369: \end{description} fp@369: fp@369: \paragraph{Device Searching} fp@369: fp@369: During the PCI initialization phase, a variable \textit{board\_idx} is fp@369: increased for each RTL8139-compatible device found. The code below is fp@369: executed for each device: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: if (board_idx == ec_device_index) { fp@369: rtl_ec_net_dev = dev; fp@369: strcpy(dev->name, "ec0"); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{1}] The device with the specified fp@369: index will be the EtherCAT device. fp@369: \end{description} fp@369: fp@369: \paragraph{Avoiding Device Registration} fp@369: fp@369: Later in the PCI initialization phase, the net\_devices get fp@369: registered. This has to be avoided for EtherCAT devices and so this is fp@369: a typical example for an EtherCAT case differentiation: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: if (dev != rtl_ec_net_dev) { fp@369: i = register_netdev(dev); fp@369: if (i) goto err_out; fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{1}] If the current net\_device is fp@369: not the EtherCAT device, it is registered at the network stack. fp@369: \end{description} fp@369: fp@369: \paragraph{Avoiding Interrupt Registration} fp@369: fp@369: In the next two listings, there is an interrupt requested and the fp@369: device's interrupts are enabled. This also has to be encapsulated by fp@369: if-clauses, because interrupt operation is not wanted for EtherCAT fp@369: devices. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: if (dev != rtl_ec_net_dev) { fp@369: retval = request_irq(dev->irq, rtl8139_interrupt, fp@369: SA_SHIRQ, dev->name, dev); fp@369: if (retval) return retval; fp@369: } fp@369: \end{lstlisting} fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: if (dev != rtl_ec_net_dev) { fp@369: /* Enable all known interrupts by setting fp@369: the interrupt mask. */ fp@369: RTL_W16(IntrMask, rtl8139_intr_mask); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \paragraph{Frame Sending} fp@369: fp@1085: The listing below shows an excerpt of the function representing the fp@369: \textit{hard\_start\_xmit()} callback of the net\_device. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: /* Note: the chip doesn't have auto-pad! */ fp@369: if (likely(len < TX_BUF_SIZE)) { fp@369: if (len < ETH_ZLEN) fp@369: memset(tp->tx_buf[entry], 0, ETH_ZLEN); fp@369: skb_copy_and_csum_dev(skb, tp->tx_buf[entry]); fp@369: if (dev != rtl_ec_net_dev) { fp@369: dev_kfree_skb(skb); fp@369: } fp@369: } else { fp@369: if (dev != rtl_ec_net_dev) { fp@369: dev_kfree_skb(skb); fp@369: } fp@369: tp->stats.tx_dropped++; fp@369: return 0; fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{6} + \linenum{10}] The fp@369: master uses a fixed socket buffer for transmission, which is reused fp@369: and may not be freed. fp@369: \end{description} fp@369: fp@369: \paragraph{Frame Receiving} fp@369: fp@369: During ordinary frame reception, a socket buffer is created and filled fp@369: with the received data. This is not necessary for an EtherCAT device: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: if (dev != rtl_ec_net_dev) { fp@369: /* Malloc up new buffer, compatible with net-2e. */ fp@369: /* Omit the four octet CRC from the length. */ fp@369: fp@369: skb = dev_alloc_skb (pkt_size + 2); fp@369: if (likely(skb)) { fp@369: skb->dev = dev; fp@369: skb_reserve(skb, 2); /* 16 byte align fp@369: the IP fields. */ fp@369: eth_copy_and_sum(skb, &rx_ring[ring_off + 4], fp@369: pkt_size, 0); fp@369: skb_put(skb, pkt_size); fp@369: skb->protocol = eth_type_trans(skb, dev); fp@369: fp@369: dev->last_rx = jiffies; fp@369: tp->stats.rx_bytes += pkt_size; fp@369: tp->stats.rx_packets++; fp@369: fp@369: netif_receive_skb (skb); fp@369: } else { fp@369: if (net_ratelimit()) fp@369: printk(KERN_WARNING fp@369: "%s: Memory squeeze, dropping" fp@369: " packet.\n", dev->name); fp@369: tp->stats.rx_dropped++; fp@369: } fp@369: } else { fp@369: ecdev_receive(rtl_ec_dev, fp@369: &rx_ring[ring_offset + 4], pkt_size); fp@369: dev->last_rx = jiffies; fp@369: tp->stats.rx_bytes += pkt_size; fp@369: tp->stats.rx_packets++; fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{28}] If the device is an EtherCAT fp@369: device, no socket buffer is allocated. Instead a pointer to the data fp@369: (which is still in the device's receive ring) is passed to the fp@369: EtherCAT master. Unnecessary copy operations are avoided. fp@1085: \item[\linenum{30} -- \linenum{32}] The fp@369: device's statistics are updated as usual. fp@369: \end{description} fp@369: fp@369: \paragraph{Link State} fp@369: fp@369: The link state (i.~e. if there is a carrier signal detected on the fp@369: receive port) is determined during execution of the ISR. The listing fp@369: below shows the different processing for Ethernet and EtherCAT fp@369: devices: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: if (dev != rtl_ec_net_dev) { fp@369: if (tp->phys[0] >= 0) { fp@369: mii_check_media(&tp->mii, netif_msg_link(tp), fp@369: init_media); fp@369: } fp@369: } else { fp@369: void __iomem *ioaddr = tp->mmio_addr; fp@369: uint16_t link = RTL_R16(BasicModeStatus) fp@369: & BMSR_LSTATUS; fp@369: ecdev_link_state(rtl_ec_dev, link ? 1 : 0); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{3}] The ``media check'' is done fp@369: via the media independent interface (MII\nomenclature{MII}{Media fp@369: Independent Interface}), a standard interface for Fast Ethernet fp@369: devices. fp@1085: \item[\linenum{7} -- \linenum{10}] For fp@369: EtherCAT devices, the link state is fetched manually from the fp@369: appropriate device register, and passed to the EtherCAT master by fp@369: calling \textit{ecdev\_\-link\_\-state()}. fp@369: \end{description} fp@369: fp@369: \paragraph{Module Cleanup} fp@369: fp@369: Below is the module's cleanup function: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: static void __exit rtl8139_cleanup_module (void) fp@369: { fp@369: printk(KERN_INFO "Cleaning up RTL8139-EtherCAT" fp@369: " module...\n"); fp@369: fp@369: if (rtl_ec_net_dev) { fp@369: printk(KERN_INFO "Stopping device...\n"); fp@369: ecdev_stop(ec_device_master_index); fp@369: printk(KERN_INFO "Unregistering device...\n"); fp@369: ecdev_unregister(ec_device_master_index, fp@369: rtl_ec_dev); fp@369: rtl_ec_dev = NULL; fp@369: } fp@369: fp@369: pci_unregister_driver(&rtl8139_pci_driver); fp@369: fp@369: printk(KERN_INFO "RTL8139-EtherCAT module" fp@369: " cleaned up.\n"); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: fp@1085: \item[\linenum{6}] Stopping and deregistration is only done, if a device was fp@1085: registered before. fp@1085: fp@1085: \item[\linenum{8}] The master is first stopped, so it does not access the fp@1085: device any more. fp@1085: fp@1085: \item[\linenum{10}] After this, the device is unregistered. The master is now fp@1085: ``orphaned''. fp@1085: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \chapter{State Machines} fp@369: \label{sec:fsm} fp@369: \index{FSM} fp@369: fp@1085: Many parts of the EtherCAT master are implemented as \textit{finite state fp@1085: machines} (FSMs\nomenclature{FSM}{Finite State Machine}). Though this leads fp@1085: to a higher grade of complexity in some aspects, is opens many new fp@1085: possibilities. fp@369: fp@369: The below short code example exemplary shows how to read all slave fp@369: states and moreover illustrates the restrictions of ``sequential'' fp@369: coding: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: ec_datagram_brd(datagram, 0x0130, 2); // prepare datagram fp@369: if (ec_master_simple_io(master, datagram)) return -1; fp@369: slave_states = EC_READ_U8(datagram->data); // process datagram fp@369: \end{lstlisting} fp@369: fp@1085: The \textit{ec\_master\_simple\_io()} function provides a simple interface for fp@1085: synchronously sending a single datagram and receiving the result\footnote{For fp@1085: all communication issues have been meanwhile sourced out into state machines, fp@1085: the function is deprecated and stopped existing. Nevertheless it is adequate fp@1085: for showing it's own restrictions.}. Internally, it queues the specified fp@1085: datagram, invokes the \textit{ec\_master\_send\_datagrams()} function to send fp@1085: a frame with the queued datagram and then waits actively for its reception. fp@369: fp@369: This sequential approach is very simple, reflecting in only three fp@369: lines of code. The disadvantage is, that the master is blocked for the fp@369: time it waits for datagram reception. There is no difficulty when only fp@369: one instance is using the master, but if more instances want to fp@369: (synchronously\footnote{At this time, synchronous master access will fp@369: be adequate to show the advantages of an FSM. The asynchronous fp@369: approach will be discussed in section~\ref{sec:eoeimp}}) use the fp@369: master, it is inevitable to think about an alternative to the fp@369: sequential model. fp@369: fp@369: Master access has to be sequentialized for more than one instance fp@369: wanting to send and receive datagrams synchronously. With the present fp@369: approach, this would result in having one phase of active waiting for fp@369: each instance, which would be non-acceptable especially in realtime fp@369: circumstances, because of the huge time overhead. fp@369: fp@369: A possible solution is, that all instances would be executed fp@369: sequentially to queue their datagrams, then give the control to the fp@369: next instance instead of waiting for the datagram reception. Finally, fp@369: bus IO is done by a higher instance, which means that all queued fp@369: datagrams are sent and received. The next step is to execute all fp@369: instances again, which then process their received datagrams and issue fp@369: new ones. fp@369: fp@369: This approach results in all instances having to retain their state, fp@369: when giving the control back to the higher instance. It is quite fp@369: obvious to use a \textit{finite state machine} model in this case. fp@369: Section~\ref{sec:fsmtheory} will introduce some of the theory used, fp@369: while the listings below show the basic approach by coding the example fp@369: from above as a state machine: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: // state 1 fp@369: ec_datagram_brd(datagram, 0x0130, 2); // prepare datagram fp@369: ec_master_queue(master, datagram); // queue datagram fp@369: next_state = state_2; fp@369: // state processing finished fp@369: \end{lstlisting} fp@369: fp@369: After all instances executed their current state and queued their fp@369: datagrams, these are sent and received. Then the respective next fp@369: states are executed: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: // state 2 fp@369: if (datagram->state != EC_DGRAM_STATE_RECEIVED) { fp@369: next_state = state_error; fp@369: return; // state processing finished fp@369: } fp@369: slave_states = EC_READ_U8(datagram->data); // process datagram fp@369: // state processing finished. fp@369: \end{lstlisting} fp@369: fp@369: See section~\ref{sec:statemodel} for an introduction to the fp@369: state machine programming concept used in the master code. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{State Machine Theory} fp@369: \label{sec:fsmtheory} fp@369: \index{FSM!Theory} fp@369: fp@369: A finite state machine \cite{automata} is a model of behavior with fp@369: inputs and outputs, where the outputs not only depend on the inputs, fp@369: but the history of inputs. The mathematical definition of a finite fp@369: state machine (or finite automaton) is a six-tuple $(\Sigma, \Gamma, fp@369: S, s_0, \delta, \omega)$, with fp@369: fp@369: \begin{itemize} fp@369: \item the input alphabet $\Sigma$, with $\Sigma \neq fp@369: \emptyset$, containing all input symbols, fp@369: \item the output alphabet $\Gamma$, with $\Gamma \neq fp@369: \emptyset$, containing all output symbols, fp@369: \item the set of states $S$, with $S \neq \emptyset$, fp@369: \item the set of initial states $s_0$ with $s_0 \subseteq S, s_0 \neq fp@369: \emptyset$ fp@369: \item the transition function $\delta: S \times \Sigma \rightarrow S fp@369: \times \Gamma$ fp@369: \item the output function $\omega$. fp@369: \end{itemize} fp@369: fp@369: The state transition function $\delta$ is often specified by a fp@369: \textit{state transition table}, or by a \textit{state transition fp@369: diagram}. The transition table offers a matrix view of the state fp@369: machine behavior (see table~\ref{tab:statetrans}). The matrix rows fp@369: correspond to the states ($S = \{s_0, s_1, s_2\}$) and the columns fp@369: correspond to the input symbols ($\Gamma = \{a, b, \varepsilon\}$). fp@369: The table contents in a certain row $i$ and column $j$ then represent fp@369: the next state (and possibly the output) for the case, that a certain fp@369: input symbol $\sigma_j$ is read in the state $s_i$. fp@369: fp@369: \begin{table}[htbp] fp@369: \caption{A typical state transition table} fp@369: \label{tab:statetrans} fp@369: \vspace{2mm} fp@369: \centering fp@369: \begin{tabular}{l|ccc} fp@369: & $a$ & $b$ & $\varepsilon$\\ \hline fp@369: $s_0$ & $s_1$ & $s_1$ & $s_2$\\ fp@369: $s_1$ & $s_2$ & $s_1$ & $s_0$\\ fp@369: $s_2$ & $s_0$ & $s_0$ & $s_0$\\ \hline fp@369: \end{tabular} fp@369: \end{table} fp@369: fp@369: The state diagram for the same example looks like the one in fp@369: figure~\ref{fig:statetrans}. The states are represented as circles or fp@369: ellipses and the transitions are drawn as arrows between them. Close fp@369: to a transition arrow can be the condition that must be fulfilled to fp@369: allow the transition. The initial state is marked by a filled black fp@369: circle with an arrow pointing to the respective state. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.5\textwidth]{images/statetrans} fp@369: \caption{A typical state transition diagram} fp@369: \label{fig:statetrans} fp@369: \end{figure} fp@369: fp@369: \paragraph{Deterministic and non-deterministic state machines} fp@369: fp@369: A state machine can be deterministic, meaning that for one state and fp@369: input, there is one (and only one) following state. In this case, the fp@369: state machine has exactly one starting state. Non-deterministic state fp@369: machines can have more than one transitions for a single state-input fp@369: combination. There is a set of starting states in the latter case. fp@369: fp@369: \paragraph{Moore and Mealy machines} fp@369: fp@369: There is a distinction between so-called \textit{Moore machines}, and fp@369: \textit{Mealy machines}. Mathematically spoken, the distinction lies fp@369: in the output function $\omega$: If it only depends on the current fp@369: state ($\omega: S \rightarrow \Gamma$), the machine corresponds to the fp@369: ``Moore Model''. Otherwise, if $\omega$ is a function of a state and fp@369: the input alphabet ($\omega: S \times \Sigma \rightarrow \Gamma$) the fp@369: state machine corresponds to the ``Mealy model''. Mealy machines are fp@369: the more practical solution in most cases, because their design allows fp@369: machines with a minimum number of states. In practice, a mixture of fp@369: both models is often used. fp@369: fp@369: \paragraph{Misunderstandings about state machines} fp@369: fp@1085: There is a phenomenon called ``state explosion'', that is often taken as a fp@1085: counter-argument against general use of state machines in complex environments. fp@1085: It has to be mentioned, that this point is misleading~\cite{fsmmis}. State fp@1085: explosions happen usually as a result of a bad state machine design: Common fp@1085: mistakes are storing the present values of all inputs in a state, or not fp@1085: dividing a complex state machine into simpler sub state machines. The EtherCAT fp@1085: master uses several state machines, that are executed hierarchically and so fp@1085: serve as sub state machines. These are also described below. fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1085: \section{The Master's State Model} fp@369: \label{sec:statemodel} fp@369: fp@369: This section will introduce the techniques used in the master to fp@369: implement state machines. fp@369: fp@369: \paragraph{State Machine Programming} fp@369: fp@369: There are certain ways to implement a state machine in \textit{C} fp@369: code. An obvious way is to implement the different states and actions fp@369: by one big case differentiation: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: enum {STATE_1, STATE_2, STATE_3}; fp@369: int state = STATE_1; fp@369: fp@369: void state_machine_run(void *priv_data) { fp@369: switch (state) { fp@369: case STATE_1: fp@369: action_1(); fp@369: state = STATE_2; fp@369: break; fp@369: case STATE_2: fp@369: action_2() fp@369: if (some_condition) state = STATE_1; fp@369: else state = STATE_3; fp@369: break; fp@369: case STATE_3: fp@369: action_3(); fp@369: state = STATE_1; fp@369: break; fp@369: } fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: For small state machines, this is an option. The disadvantage is, that fp@369: with an increasing number of states the code soon gets complex and an fp@369: additional case differentiation is executed each run. Besides, lots of fp@369: indentation is wasted. fp@369: fp@369: The method used in the master is to implement every state in an own fp@369: function and to store the current state function with a function fp@369: pointer: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: void (*state)(void *) = state1; fp@369: fp@369: void state_machine_run(void *priv_data) { fp@369: state(priv_data); fp@369: } fp@369: fp@369: void state1(void *priv_data) { fp@369: action_1(); fp@369: state = state2; fp@369: } fp@369: fp@369: void state2(void *priv_data) { fp@369: action_2(); fp@369: if (some_condition) state = state1; fp@369: else state = state2; fp@369: } fp@369: fp@369: void state3(void *priv_data) { fp@369: action_3(); fp@369: state = state1; fp@369: } fp@369: \end{lstlisting} fp@369: fp@1085: In the master code, state pointers of all state machines\footnote{All except fp@1085: for the EoE state machine, because multiple EoE slaves have to be handled in fp@1085: parallel. For this reason each EoE handler object has its own state pointer.} fp@1085: are gathered in a single object of the \textit{ec\_fsm\_t} class. This is fp@1085: advantageous, because there is always one instance of every state machine fp@1085: available and can be started on demand. fp@369: fp@369: \paragraph{Mealy and Moore} fp@369: fp@369: If a closer look is taken to the above listing, it can be seen that fp@369: the actions executed (the ``outputs'' of the state machine) only fp@369: depend on the current state. This accords to the ``Moore'' model fp@369: introduced in section~\ref{sec:fsmtheory}. As mentioned, the ``Mealy'' fp@369: model offers a higher flexibility, which can be seen in the listing fp@369: below: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: void state7(void *priv_data) { fp@369: if (some_condition) { fp@369: action_7a(); fp@369: state = state1; fp@369: } fp@369: else { fp@369: action_7b(); fp@369: state = state8; fp@369: } fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{3} + \linenum{7}] The fp@369: state function executes the actions depending on the state fp@369: transition, that is about to be done. fp@369: \end{description} fp@369: fp@369: The most flexible alternative is to execute certain actions depending fp@369: on the state, followed by some actions dependent on the state fp@369: transition: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@369: void state9(void *priv_data) { fp@369: action_9(); fp@369: if (some_condition) { fp@369: action_9a(); fp@369: state = state7; fp@369: } fp@369: else { fp@369: action_9b(); fp@369: state = state10; fp@369: } fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: This model is oftenly used in the master. It combines the best aspects fp@369: of both approaches. fp@369: fp@369: \paragraph{Using Sub State Machines} fp@369: fp@1085: To avoid having too much states, certain functions of the EtherCAT master state fp@1085: machine have been sourced out into sub state machines. This helps to fp@1085: encapsulate the related workflows and moreover avoids the ``state explosion'' fp@1085: phenomenon described in section~\ref{sec:fsmtheory}. If the master would fp@1085: instead use one big state machine, the number of states would be a multiple of fp@1085: the actual number. This would increase the level of complexity to a fp@369: non-manageable grade. fp@369: fp@369: \paragraph{Executing Sub State Machines} fp@369: fp@369: If a state machine starts to execute a sub state machine, it usually fp@369: remains in one state until the sub state machine terminates. This is fp@369: usually done like in the listing below, which is taken out of the fp@369: slave configuration state machine code: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left] fp@813: void ec_fsm_slaveconf_safeop(ec_fsm_t *fsm) fp@369: { fp@369: fsm->change_state(fsm); // execute state change fp@369: // sub state machine fp@369: fp@369: if (fsm->change_state == ec_fsm_error) { fp@369: fsm->slave_state = ec_fsm_end; fp@369: return; fp@369: } fp@369: fp@369: if (fsm->change_state != ec_fsm_end) return; fp@369: fp@369: // continue state processing fp@369: ... fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{3}] \textit{change\_state} is the fp@369: state pointer of the state change state machine. The state function, fp@369: the pointer points on, is executed\ldots fp@1085: \item[\linenum{6}] \ldots either until the state fp@369: machine terminates with the error state \ldots fp@1085: \item[\linenum{11}] \ldots or until the state fp@369: machine terminates in the end state. Until then, the ``higher'' fp@369: state machine remains in the current state and executes the sub fp@369: state machine again in the next cycle. fp@369: \end{description} fp@369: fp@369: \paragraph{State Machine Descriptions} fp@369: fp@369: The below sections describe every state machine used in the EtherCAT fp@369: master. The textual descriptions of the state machines contain fp@369: references to the transitions in the corresponding state transition fp@369: diagrams, that are marked with an arrow followed by the name of the fp@369: successive state. Transitions caused by trivial error cases (i.~e. no fp@369: response from slave) are not described explicitly. These transitions fp@369: are drawn as dashed arrows in the diagrams. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{The Operation State Machine} fp@369: \label{sec:fsm-op} fp@369: \index{FSM!Operation} fp@369: fp@369: The Operation state machine is executed by calling the fp@369: \textit{ecrt\_master\_run()} method in cyclic realtime code. Its fp@369: purpose is to monitor the bus and to reconfigure slaves after a bus fp@369: failure or power failure. Figure~\ref{fig:fsm-op} shows its transition fp@369: diagram. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.8\textwidth]{images/fsm-op} fp@369: \caption{Transition diagram of the operation state machine} fp@369: \label{fig:fsm-op} fp@369: \end{figure} fp@369: fp@369: \begin{description} fp@369: \item[START] This is the beginning state of the operation state fp@369: machine. There is a datagram issued, that queries the ``AL Control fp@369: Response'' attribute \cite[section~5.3.2]{alspec} of all slaves via fp@369: broadcast. In this way, all slave states and the number of slaves fp@379: responding can be determined. $\rightarrow$~BROADCAST fp@369: fp@1085: \item[BROADCAST] The broadcast datagram is evaluated. A change in the number of fp@1085: responding slaves is treated as a topology change. If the number of slaves is fp@1085: not as expected, the bus is marked as ``tainted''. In this state, no slave fp@1085: reconfiguration is possible, because the assignment of known slaves and those fp@1085: present on the bus is ambiguous. If the number of slaves is considered as fp@1085: right, the bus is marked for validation, because it turned from tainted to fp@1085: normal state and it has to be checked, if all slaves are valid. Now, the state fp@1085: of every single slave has to be determined. For that, a (unicast) datagram is fp@1085: issued, that queries the first slave's ``AL Control Response'' attribute. fp@1085: $\rightarrow$~READ STATES fp@1085: fp@1085: \item[READ STATES] If the current slave did not respond to its configured fp@1085: station address, it is marked as offline, and the next slave is queried. fp@1085: $\rightarrow$~READ STATES fp@369: fp@369: If the slave responded, it is marked as online and its current state fp@379: is stored. The next slave is queried. $\rightarrow$~READ STATES fp@369: fp@369: If all slaves have been queried, and the bus is marked for fp@369: validation, the validation is started by checking the first slaves fp@379: vendor ID. $\rightarrow$~VALIDATE VENDOR fp@369: fp@369: If no validation has to be done, it is checked, if all slaves are in fp@369: the state they are supposed to be. If not, the first of slave with fp@369: the wrong state is reconfigured and brought in the required state. fp@379: $\rightarrow$~CONFIGURE SLAVES fp@369: fp@369: If all slaves are in the correct state, the state machine is fp@379: restarted. $\rightarrow$~START fp@369: fp@369: \item[CONFIGURE SLAVES] The slave configuration state machine is fp@379: executed until termination. $\rightarrow$~CONFIGURE SLAVES fp@369: fp@369: If there are still slaves in the wrong state after another check, fp@369: the first of these slaves is configured and brought into the correct fp@379: state again. $\rightarrow$~CONFIGURE SLAVES fp@369: fp@369: If all slaves are in the correct state, the state machine is fp@379: restarted. $\rightarrow$~START fp@369: fp@369: \item[VALIDATE VENDOR] The SII state machine is executed until fp@369: termination. If the slave has the wrong vendor ID, the state machine fp@379: is restarted. $\rightarrow$~START fp@369: fp@369: If the slave has the correct vendor ID, its product ID is queried. fp@379: $\rightarrow$~VALIDATE PRODUCT fp@369: fp@369: \item[VALIDATE PRODUCT] The SII state machine is executed until fp@369: termination. If the slave has the wrong product ID, the state fp@379: machine is restarted. $\rightarrow$~START fp@369: fp@369: If the slave has the correct product ID, the next slave's vendor ID fp@379: is queried. $\rightarrow$~VALIDATE VENDOR fp@369: fp@369: If all slaves have the correct vendor IDs and product codes, the fp@369: configured station addresses can be safely rewritten. This is done fp@369: for the first slave marked as offline. fp@379: $\rightarrow$~REWRITE ADDRESSES fp@369: fp@1095: \item[REWRITE ADDRESSES] If the station address was successfully written, it is fp@1095: searched for the next slave marked as offline. If there is one, its address is fp@1095: reconfigured, too. $\rightarrow$~REWRITE ADDRESSES fp@369: fp@369: If there are no more slaves marked as offline, the state machine is fp@379: restarted. $\rightarrow$~START fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{The Idle State Machine} fp@369: \label{sec:fsm-idle} fp@369: \index{FSM!Idle} fp@369: fp@1085: The Idle state machine is executed by a kernel thread, if no application is fp@1085: connected. Its purpose is to make slave information available to user space, fp@1085: operate EoE-capable slaves, read and write SII contents and test slave fp@1085: functionality. Figure~\ref{fig:fsm-idle} shows its transition diagram. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.8\textwidth]{images/fsm-idle} fp@369: \caption{Transition diagram of the idle state machine} fp@369: \label{fig:fsm-idle} fp@369: \end{figure} fp@369: fp@369: \begin{description} fp@369: \item[START] The beginning state of the idle state machine. Similar to fp@369: the operation state machine, a broadcast datagram is issued, to fp@369: query all slave states and the number of slaves. fp@379: $\rightarrow$~BROADCAST fp@369: fp@369: \item[BROADCAST] The number of responding slaves is evaluated. If it fp@369: has changed since the last time, this is treated as a topology fp@369: change and the internal list of slaves is cleared and rebuild fp@369: completely. The slave scan state machine is started for the first fp@379: slave. $\rightarrow$~SCAN FOR SLAVES fp@369: fp@369: If no topology change happened, every single slave state is fetched. fp@379: $\rightarrow$~READ STATES fp@369: fp@369: \item[SCAN FOR SLAVES] The slave scan state machine is executed until fp@379: termination. $\rightarrow$~SCAN FOR SLAVES fp@369: fp@369: If there is another slave to scan, the slave scan state machine is fp@379: started again. $\rightarrow$~SCAN FOR SLAVES fp@369: fp@369: If all slave information has been fetched, slave addresses are fp@369: calculated and EoE processing is started. Then, the state machine is fp@379: restarted. $\rightarrow$~START fp@369: fp@369: \item[READ STATES] If the slave did not respond to the query, it is fp@369: marked as offline. The next slave is queried. fp@379: $\rightarrow$~READ STATES fp@369: fp@369: If the slave responded, it is marked as online. And the next slave fp@379: is queried. $\rightarrow$~READ STATES fp@369: fp@369: If all slave states have been determined, it is checked, if any fp@369: slaves are not in the state they supposed to be. If this is true, fp@369: the slave configuration state machine is started for the first of fp@379: them. $\rightarrow$~CONFIGURE SLAVES fp@369: fp@369: If all slaves are in the correct state, it is checked, if any fp@369: E$^2$PROM write operations are pending. If this is true, the first fp@369: pending operation is executed by starting the SII state machine for fp@379: writing access. $\rightarrow$~WRITE EEPROM fp@369: fp@369: If all these conditions are false, there is nothing to do and the fp@379: state machine is restarted. $\rightarrow$~START fp@369: fp@369: \item[CONFIGURE SLAVES] The slave configuration state machine is fp@379: executed until termination. $\rightarrow$~CONFIGURE SLAVES fp@369: fp@369: After this, it is checked, if another slave needs a state change. If fp@369: this is true, the slave state change state machine is started for fp@379: this slave. $\rightarrow$~CONFIGURE SLAVES fp@369: fp@369: If all slaves are in the correct state, it is determined, if any fp@369: E$^2$PROM write operations are pending. If this is true, the first fp@369: pending operation is executed by starting the SII state machine for fp@379: writing access. $\rightarrow$~WRITE EEPROM fp@369: fp@369: If all prior conditions are false, the state machine is restarted. fp@379: $\rightarrow$~START fp@369: fp@369: \item[WRITE EEPROM] The SII state machine is executed until fp@379: termination. $\rightarrow$~WRITE EEPROM fp@369: fp@369: If the current word has been written successfully, and there are fp@369: still word to write, the SII state machine is started for the next fp@379: word. $\rightarrow$~WRITE EEPROM fp@369: fp@369: If all words have been written successfully, the new E$^2$PROM fp@369: contents are evaluated and the state machine is restarted. fp@379: $\rightarrow$~START fp@369: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{The Slave Scan State Machine} fp@369: \label{sec:fsm-scan} fp@369: \index{FSM!Slave Scan} fp@369: fp@369: The slave scan state machine, which can be seen in fp@369: figure~\ref{fig:fsm-slavescan}, leads through the process of fetching fp@369: all slave information. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.6\textwidth]{images/fsm-slavescan} fp@369: \caption{Transition diagram of the slave scan state machine} fp@369: \label{fig:fsm-slavescan} fp@369: \end{figure} fp@369: fp@369: \begin{description} fp@369: \item[START] In the beginning state of the slave scan state machine, fp@369: the station address is written to the slave, which is always the fp@369: ring position~+~$1$. In this way, the address 0x0000 (default fp@369: address) is not used, which makes it easy to detect unconfigured fp@379: slaves. $\rightarrow$~ADDRESS fp@369: fp@369: \item[ADDRESS] The writing of the station address is verified. After fp@369: that, the slave's ``AL Control Response'' attribute is queried. fp@379: $\rightarrow$~STATE fp@369: fp@369: \item[STATE] The AL state is evaluated. A warning is output, if the fp@369: slave has still the \textit{Change} bit set. After that, the slave's fp@369: ``DL Information'' attribute is queried. fp@379: $\rightarrow$~BASE fp@369: fp@369: \item[BASE] The queried base data are evaluated: Slave type, revision fp@369: and build number, and even more important, the number of supported fp@369: sync managers and FMMUs are stored. After that, the slave's data fp@369: link layer information is read from the ``DL Status'' attribute at fp@379: address 0x0110. $\rightarrow$~DATALINK fp@369: fp@369: \item[DATALINK] In this state, the DL information is evaluated: This fp@369: information about the communication ports contains, if the link is fp@369: up, if the loop has been closed and if there is a carrier detected fp@369: on the RX side of each port. fp@369: fp@369: Then, the state machine starts measuring the size of the slave's fp@369: E$^2$PROM contents. This is done by subsequently reading out each fp@369: category header, until the last category is reached (type 0xFFFF). fp@369: This procedure is started by querying the first category header at fp@369: word address 0x0040 via the SII state machine. fp@379: $\rightarrow$~EEPROM SIZE fp@369: fp@369: \item[EEPROM SIZE] The SII state machine is executed until fp@379: termination. $\rightarrow$~EEPROM SIZE fp@369: fp@369: If the category type does not mark the end of the categories, the fp@369: position of the next category header is determined via the length of fp@369: the current category, and the SII state machine is started again. fp@379: $\rightarrow$~EEPROM SIZE fp@369: fp@369: If the size of the E$^2$PROM contents has been determined, memory is fp@369: allocated, to read all the contents. The SII state machine is fp@379: started to read the first word. $\rightarrow$~EEPROM DATA fp@369: fp@369: \item[EEPROM DATA] The SII state machine is executed until fp@379: termination. $\rightarrow$~EEPROM DATA fp@369: fp@369: Two words have been read. If more than one word is needed, the two fp@369: words are written in the allocated memory. Otherwise only one word fp@369: (the last word) is copied. If more words are to read, the SII state fp@369: machine is started again to read the next two words. fp@379: $\rightarrow$~EEPROM DATA fp@369: fp@369: The complete E$^2$PROM contents have been read. The slave's identity fp@369: object and mailbox information are evaluated. Moreover the category fp@369: types STRINGS, GENERAL, SYNC and PDO are evaluated. The slave fp@379: scanning has been completed. $\rightarrow$~END fp@369: fp@369: \item[END] Slave scanning has been finished. fp@369: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{The Slave Configuration State Machine} fp@369: \label{sec:fsm-conf} fp@369: \index{FSM!Slave Configuration} fp@369: fp@369: The slave configuration state machine, which can be seen in fp@369: figure~\ref{fig:fsm-slaveconf}, leads through the process of fp@369: configuring a slave and bringing it to a certain state. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.6\textwidth]{images/fsm-slaveconf} fp@369: \caption{Transition diagram of the slave configuration state fp@369: machine} fp@369: \label{fig:fsm-slaveconf} fp@369: \end{figure} fp@369: fp@369: \begin{description} fp@369: \item[INIT] The state change state machine has been initialized to fp@379: bring the slave into the INIT state. Now, the slave state change fp@379: state machine is executed until termination. $\rightarrow$~INIT fp@369: fp@369: If the slave state change failed, the configuration has to be fp@379: aborted. $\rightarrow$~END fp@379: fp@379: The slave state change succeeded and the slave is now in INIT state. fp@379: If this is the target state, the configuration is finished. fp@379: $\rightarrow$~END fp@369: fp@369: If the slave does not support any sync managers, the sync manager fp@369: configuration can be skipped. The state change state machine is fp@379: started to bring the slave into PREOP state. fp@379: $\rightarrow$~PREOP fp@369: fp@369: Sync managers are configured conforming to the sync manager category fp@369: information provided in the slave's E$^2$PROM. The corresponding fp@379: datagram is issued. $\rightarrow$~SYNC fp@369: fp@369: \item[SYNC] If the sync manager configuration datagram is accepted, fp@369: the sync manager configuration was successful. The slave may now fp@379: enter the PREOP state, and the state change state machine is fp@379: started. $\rightarrow$~PREOP fp@369: fp@369: \item[PREOP] The state change state machine is executed until fp@379: termination. $\rightarrow$~PREOP fp@369: fp@369: If the state change failed, the configuration has to be aborted. fp@379: $\rightarrow$~END fp@379: fp@379: If the PREOP state was the target state, the configuration is fp@379: finished. $\rightarrow$~END fp@369: fp@369: If the slave supports no FMMUs, the FMMU configuration can be fp@814: skipped. If the slave has Sdos to configure, it is begun with fp@814: sending the first Sdo. $\rightarrow$~SDO\_CONF fp@814: fp@814: If no Sdo configurations are provided, the slave can now directly be fp@813: brought into the SAFEOP state and the state change state machine is fp@813: started again. $\rightarrow$~SAFEOP fp@369: fp@814: Otherwise, all supported FMMUs are configured according to the Pdos fp@369: requested via the master's realtime interface. The appropriate fp@379: datagram is issued. $\rightarrow$~FMMU fp@369: fp@369: \item[FMMU] The FMMU configuration datagram was accepted. If the slave fp@814: has Sdos to configure, it is begun with sending the first Sdo. fp@379: $\rightarrow$~SDO\_CONF fp@379: fp@813: Otherwise, the slave can now be brought into the SAFEOP state. The fp@379: state change state machine is started. fp@813: $\rightarrow$~SAFEOP fp@369: fp@369: \item[SDO\_CONF] The CoE state machine is executed until termination. fp@379: $\rightarrow$~SDO\_CONF fp@369: fp@814: If another Sdo has to be configured, a new Sdo download sequence is fp@379: begun. $\rightarrow$~SDO\_CONF fp@379: fp@813: Otherwise, the slave can now be brought into the SAFEOP state. The fp@379: state change state machine is started. fp@813: $\rightarrow$~SAFEOP fp@813: fp@813: \item[SAFEOP] The state change state machine is executed until fp@813: termination. $\rightarrow$~SAFEOP fp@369: fp@369: If the state change failed, the configuration has to be aborted. fp@379: $\rightarrow$~END fp@379: fp@813: If the SAFEOP state was the target state, the configuration is fp@379: finished. $\rightarrow$~END fp@379: fp@379: The slave can now directly be brought into the OP state and the fp@379: state change state machine is started a last time. fp@379: $\rightarrow$~OP fp@369: fp@369: \item[OP] The state change state machine is executed until fp@379: termination. $\rightarrow$~OP fp@369: fp@369: If the state change state machine terminates, the slave fp@369: configuration is finished, regardless of its success. fp@379: $\rightarrow$~END fp@369: fp@369: \item[END] The termination state. fp@369: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{The State Change State Machine} fp@369: \label{sec:fsm-change} fp@369: \index{FSM!State Change} fp@369: fp@369: The state change state machine, which can be seen in fp@369: figure~\ref{fig:fsm-change}, leads through the process of changing a fp@369: slave's state. This implements the states and transitions described in fp@369: \cite[section~6.4.1]{alspec}. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.9\textwidth]{images/fsm-change} fp@369: \caption{Transition diagram of the state change state machine} fp@369: \label{fig:fsm-change} fp@369: \end{figure} fp@369: fp@369: \begin{description} fp@369: \item[START] The beginning state, where a datagram with the state fp@369: change command is written to the slave's ``AL Control Request'' fp@379: attribute. Nothing can fail. $\rightarrow$~CHECK fp@369: fp@369: \item[CHECK] After the state change datagram has been sent, the ``AL fp@369: Control Response'' attribute is queried with a second datagram. fp@379: $\rightarrow$~STATUS fp@369: fp@369: \item[STATUS] The read memory contents are evaluated: While the fp@369: parameter \textit{State} still contains the old slave state, the fp@369: slave is busy with reacting on the state change command. In this fp@369: case, the attribute has to be queried again. fp@379: $\rightarrow$~STATUS fp@369: fp@369: In case of success, the \textit{State} parameter contains the new fp@369: state and the \textit{Change} bit is cleared. The slave is in the fp@379: requested state. $\rightarrow$~END fp@369: fp@369: If the slave can not process the state change, the \textit{Change} fp@369: bit is set: Now the master tries to get the reason for this by fp@369: querying the \textit{AL Status Code} parameter. fp@379: $\rightarrow$~CODE fp@369: fp@1085: \item[END] If the state machine ends in this state, the slave's state fp@369: change has been successful. fp@369: fp@369: \item[CODE] The status code query has been sent. Reading the fp@369: \textit{AL Status Code} might fail, because not all slaves support fp@369: this parameter. Anyway, the master has to acknowledge the state fp@369: change error by writing the current slave state to the ``AL Control fp@369: Request'' attribute with the \textit{Acknowledge} bit set. fp@379: $\rightarrow$~ACK fp@369: fp@369: \item[ACK] After that, the ``AL Control Response'' attribute is fp@369: queried for the state of the acknowledgement. fp@379: $\rightarrow$~CHECK ACK fp@369: fp@369: \item[CHECK ACK] If the acknowledgement has been accepted by the fp@369: slave, the old state is kept. Still, the state change was fp@379: unsuccessful. $\rightarrow$~ERROR fp@369: fp@369: If the acknowledgement is ignored by the slave, a timeout happens. fp@369: In any case, the overall state change was unsuccessful. fp@379: $\rightarrow$~ERROR fp@369: fp@369: If there is still now response from the slave, but the timer did not fp@369: run out yet, the slave's ``AL Control Response'' attribute is fp@379: queried again. $\rightarrow$~CHECK ACK fp@369: fp@369: \item[ERROR] If the state machine ends in this state, the slave's fp@369: state change was unsuccessful. fp@369: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{The SII State Machine} fp@369: \label{sec:fsm-sii} fp@369: \index{FSM!SII} fp@369: fp@369: The SII\index{SII} state machine (shown in figure~\ref{fig:fsm-sii}) fp@369: implements the process of reading or writing E$^2$PROM data via the fp@369: Slave Information Interface described in \cite[section~5.4]{alspec}. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.9\textwidth]{images/fsm-sii} fp@369: \caption{Transition diagram of the SII state machine} fp@369: \label{fig:fsm-sii} fp@369: \end{figure} fp@369: fp@369: \begin{description} fp@369: \item[READ\_START] The beginning state for reading access, where the fp@369: read request and the requested address are written to the SII fp@369: attribute. Nothing can fail up to now. fp@379: $\rightarrow$~READ\_CHECK fp@369: fp@369: \item[READ\_CHECK] When the SII read request has been sent fp@369: successfully, a timer is started. A check/fetch datagram is issued, fp@369: that reads out the SII attribute for state and data. fp@379: $\rightarrow$~READ\_FETCH fp@369: fp@369: \item[READ\_FETCH] Upon reception of the check/fetch datagram, the fp@369: \textit{Read Operation} and \textit{Busy} parameters are checked: fp@369: \begin{itemize} fp@369: \item If the slave is still busy with fetching E$^2$PROM data into fp@369: the interface, the timer is checked. If it timed out, the reading fp@379: is aborted ($\rightarrow$~ERROR), if not, the check/fetch datagram fp@379: is issued again. $\rightarrow$~READ\_FETCH fp@369: fp@369: \item If the slave is ready with reading data, these are copied from fp@369: the datagram and the read cycle is completed. fp@379: $\rightarrow$~END fp@369: \end{itemize} fp@369: \end{description} fp@369: fp@369: The write access states behave nearly the same: fp@369: fp@369: \begin{description} fp@369: \item[WRITE\_START] The beginning state for writing access, fp@369: respectively. A write request, the target address and the data word fp@369: are written to the SII attribute. Nothing can fail. fp@379: $\rightarrow$~WRITE\_CHECK fp@369: fp@369: \item[WRITE\_CHECK] When the SII write request has been sent fp@369: successfully, the timer is started. A check datagram is issued, that fp@369: reads out the SII attribute for the state of the write operation. fp@379: $\rightarrow$~WRITE\_CHECK2 fp@369: fp@369: \item[WRITE\_CHECK2] Upon reception of the check datagram, the fp@369: \textit{Write Operation} and \textit{Busy} parameters are checked: fp@369: \begin{itemize} fp@369: \item If the slave is still busy with writing E$^2$PROM data, the fp@369: timer is checked. If it timed out, the operation is aborted fp@379: ($\rightarrow$~ERROR), if not, the check datagram is issued again. fp@379: $\rightarrow$~WRITE\_CHECK2 fp@369: \item If the slave is ready with writing data, the write cycle is fp@379: completed. $\rightarrow$~END fp@369: \end{itemize} fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \chapter{Mailbox Protocol Implementations} fp@369: \index{Mailbox} fp@369: fp@369: The EtherCAT master implements the EoE and the CoE mailbox fp@369: protocols. See the below section for details. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{Ethernet-over-EtherCAT (EoE)} fp@369: \label{sec:eoeimp} fp@369: \index{EoE} fp@369: fp@369: The EtherCAT master implements the Ethernet-over-EtherCAT mailbox fp@369: protocol to enable the tunneling of Ethernet frames to special slaves, fp@369: that can either have physical Ethernet ports to forward the frames to, fp@369: or have an own IP stack to receive the frames. fp@369: fp@369: \paragraph{Virtual Network Interfaces} fp@369: fp@369: The master creates a virtual EoE network interface for every fp@369: EoE-capable slave. These interfaces are called \textit{eoeX}, where X fp@369: is a number provided by the kernel on interface registration. Frames fp@369: sent to these interfaces are forwarded to the associated slaves by the fp@369: master. Frames, that are received by the slaves, are fetched by the fp@369: master and forwarded to the virtual interfaces. fp@369: fp@369: This bears the following advantages: fp@369: fp@369: \begin{itemize} fp@369: \item Flexibility: The user can decide, how the EoE-capable slaves are fp@369: interconnected with the rest of the world. fp@369: \item Standard tools can be used to monitor the EoE activity and to fp@369: configure the EoE interfaces. fp@369: \item The Linux kernel's layer-2-bridging implementation (according to fp@369: the IEEE 802.1D MAC Bridging standard) can be used natively to fp@369: bridge Ethernet traffic between EoE-capable slaves. fp@369: \item The Linux kernel's network stack can be used to route packets fp@369: between EoE-capable slaves and to track security issues, just like fp@369: having physical network interfaces. fp@369: \end{itemize} fp@369: fp@369: \paragraph{EoE Handlers} fp@369: fp@1085: The virtual EoE interfaces and the related functionality is encapsulated in the fp@1085: \textit{ec\_eoe\_t} class (see section~\ref{sec:class-eoe}). So the master fp@1085: does not create the network interfaces directly: This is done inside the fp@1085: constructor of the \textit{ec\_eoe\_t} class. An object of this class is called fp@1085: ``EoE handler'' below. An EoE handler additionally contains a frame queue. Each fp@1085: time, the kernel passes a new socket buffer for sending via the interface's fp@369: \textit{hard\_start\_xmit()} callback, the socket buffer is queued for fp@1085: transmission by the EoE state machine (see below). If the queue gets filled up, fp@1085: the passing of new socket buffers is suspended with a call to fp@1085: \textit{netif\_stop\_queue()}. fp@369: fp@369: \paragraph{Static Handler Creation} fp@369: fp@369: The master creates a pool of EoE handlers at startup, that are coupled fp@369: to EoE-capable slaves on demand. The lifetime of the corresponding fp@369: network interfaces is equal to the lifetime of the master module. fp@369: This approach is opposed to creating the virtual network interfaces on fp@369: demand (i.~e. on running across a new EoE-capable slave). The latter fp@1085: approach was considered as difficult, because of several reasons: fp@369: fp@369: \begin{itemize} fp@369: \item The \textit{alloc\_netdev()} function can sleep and must be fp@369: called from a non-interrupt context. This reduces the flexibility of fp@369: choosing an appropriate method for cyclic EoE processing. fp@369: \item Unregistering network interfaces requires them to be ``down'', fp@369: which can not be guaranteed upon sudden disappearing of an fp@369: EoE-capable slave. fp@369: \item The connection to the EoE-capable slaves must be as continuous fp@369: as possible. Especially the transition from idle to operation mode fp@369: (and vice versa) causes the rebuilding of the internal data fp@369: structures. These transitions must be as transparent as possible for fp@369: the instances using the network interfaces. fp@369: \end{itemize} fp@369: fp@1095: \paragraph{Number of Handlers} % FIXME fp@369: fp@369: The master module has a parameter \textit{ec\_eoeif\_count} to specify fp@369: the number of EoE interfaces (and handlers) per master to create. This fp@369: parameter can either be specified when manually loading the master fp@369: module, or (when using the init script) by setting the fp@379: \$EOE\_INTERFACES variable in the sysconfig file (see fp@369: section~\ref{sec:sysconfig}). Upon loading of the master module, the fp@369: virtual interfaces become available: fp@369: fp@1085: \begin{lstlisting}[gobble=2] fp@379: # `\textbf{ifconfig -a}` fp@369: eoe0 Link encap:Ethernet HWaddr 00:11:22:33:44:06 fp@369: BROADCAST MULTICAST MTU:1500 Metric:1 fp@369: RX packets:0 errors:0 dropped:0 overruns:0 frame:0 fp@369: TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 fp@369: collisions:0 txqueuelen:1000 fp@369: RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) fp@369: fp@369: eoe1 Link encap:Ethernet HWaddr 00:11:22:33:44:07 fp@369: BROADCAST MULTICAST MTU:1500 Metric:1 fp@369: RX packets:0 errors:0 dropped:0 overruns:0 frame:0 fp@369: TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 fp@369: collisions:0 txqueuelen:1000 fp@369: RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) fp@369: ... fp@369: \end{lstlisting} fp@369: fp@369: \paragraph{Coupling of EoE Slaves} fp@369: fp@369: During execution of the slave scan state machine (see fp@369: section~\ref{sec:fsm-scan}), the master determines the supported fp@369: mailbox protocols. This is done by examining the ``Supported Mailbox fp@369: Protocols'' mask field at word address 0x001C of the SII\index{SII}. fp@369: If bit 1 is set, the slave supports the EoE protocol. After slave fp@369: scanning, the master runs through all slaves again and couples each fp@369: EoE-capable slave to a free EoE handler. It can happen, that there are fp@369: not enough EoE handlers to cover all EoE-capable slaves. In this case, fp@369: the number of EoE handlers must be increased accordingly. fp@369: fp@369: \paragraph{EoE State Machine} fp@369: \index{FSM!EoE} fp@369: fp@369: Every EoE handler owns an EoE state machine, that is used to send fp@369: frames to the coupled slave and receive frames from the it via the EoE fp@369: communication primitives. This state machine is showed in fp@369: figure~\ref{fig:fsm-eoe}. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.7\textwidth]{images/fsm-eoe} fp@369: \caption{Transition diagram of the EoE state machine} fp@369: \label{fig:fsm-eoe} fp@369: \end{figure} fp@369: fp@369: \begin{description} fp@369: \item[RX\_START] The beginning state of the EoE state machine. A fp@369: mailbox check datagram is sent, to query the slave's mailbox for new fp@379: frames. $\rightarrow$~RX\_CHECK fp@369: fp@369: \item[RX\_CHECK] The mailbox check datagram is received. If the fp@369: slave's mailbox did not contain data, a transmit cycle is started. fp@379: $\rightarrow$~TX\_START fp@369: fp@369: If there are new data in the mailbox, a datagram is sent to fetch fp@379: the new data. $\rightarrow$~RX\_FETCH fp@369: fp@369: \item[RX\_FETCH] The fetch datagram is received. If the mailbox data fp@369: do not contain a ``EoE Fragment request'' command, the data are fp@369: dropped and a transmit sequence is started. fp@379: $\rightarrow$~TX\_START fp@369: fp@369: If the received Ethernet frame fragment is the first fragment, a new fp@369: socket buffer is allocated. In either case, the data are copied into fp@369: the correct position of the socket buffer. fp@369: fp@369: If the fragment is the last fragment, the socket buffer is forwarded fp@369: to the network stack and a transmit sequence is started. fp@379: $\rightarrow$~TX\_START fp@369: fp@369: Otherwise, a new receive sequence is started to fetch the next fp@379: fragment. $\rightarrow$~RX\_\-START fp@369: fp@369: \item[TX\_START] The beginning state of a transmit sequence. It is fp@1085: checked, if the transmission queue contains a frame to send. If not, fp@379: a receive sequence is started. $\rightarrow$~RX\_START fp@369: fp@369: If there is a frame to send, it is dequeued. If the queue was fp@369: inactive before (because it was full), the queue is woken up with a fp@369: call to \textit{netif\_wake\_queue()}. The first fragment of the fp@379: frame is sent. $\rightarrow$~TX\_SENT fp@369: fp@369: \item[TX\_SENT] It is checked, if the first fragment was sent fp@369: successfully. If the current frame consists of further fragments, fp@379: the next one is sent. $\rightarrow$~TX\_SENT fp@369: fp@369: If the last fragment was sent, a new receive sequence is started. fp@379: $\rightarrow$~RX\_START fp@369: \end{description} fp@369: fp@369: \paragraph{EoE Processing} fp@369: fp@369: To execute the EoE state machine of every active EoE handler, there fp@369: must be a cyclic process. The easiest thing would be to execute the fp@369: EoE state machines synchronously to the operation state machine (see fp@369: section~\ref{sec:fsm-op}) with every realtime cycle. This approach has fp@369: the following disadvantages: fp@369: fp@369: \begin{itemize} fp@1085: fp@1085: \item Only one EoE fragment can be sent or received every few cycles. This fp@1085: causes the data rate to be very low, because the EoE state machines are not fp@1085: executed in the time between the application cycles. Moreover, the data rate fp@1085: would be dependent on the period of the application task. fp@1085: fp@1085: \item The receiving and forwarding of frames to the kernel requires the dynamic fp@1085: allocation of frames. Some realtime extensions do not support calling memory fp@1085: allocation functions in realtime context, so the EoE state machine may not be fp@1085: executed with each application cycle. fp@1085: fp@369: \end{itemize} fp@369: fp@369: To overcome these problems, an own cyclic process is needed to fp@369: asynchronously execute the EoE state machines. For that, the master fp@369: owns a kernel timer, that is executed each timer interrupt. This fp@369: guarantees a constant bandwidth, but poses the new problem of fp@369: concurrent access to the master. The locking mechanisms needed for fp@369: this are introduced in section~\ref{sec:concurr}. fp@369: Section~\ref{sec:concurrency} gives practical implementation examples. fp@369: fp@1085: \paragraph{Idle phase} fp@1085: fp@1085: EoE data must also be exchanged in idle phase, to guarantee the continuous fp@1085: availability of the connection to the EoE-capable slaves. Although there is no fp@1085: application connected in this case, the master is still accessed by the master fp@1085: state machine (see section~\ref{sec:fsm-master}). With the EoE timer running in fp@1085: addition, there is still concurrency, that has to be protected by a lock. fp@1085: Therefore the master owns an internal spinlock that is used protect master fp@1085: access during idle phase. fp@369: fp@369: \paragraph{Automatic Configuration} fp@369: fp@379: By default, slaves are left in INIT state during idle mode. If an EoE fp@379: interface is set to running state (i.~e. with the \textit{ifconfig up} fp@379: command), the requested slave state of the related slave is fp@379: automatically set to OP, whereupon the idle state machine will attempt fp@379: to configure the slave and put it into operation. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{CANopen-over-EtherCAT (CoE)} fp@369: \label{sec:coeimp} fp@369: \index{CoE} fp@369: fp@369: The CANopen-over-EtherCAT protocol \cite[section~5.6]{alspec} is used fp@369: to configure slaves on application level. Each CoE-capable slave fp@814: provides a list of Sdos for this reason. fp@814: fp@814: \paragraph{Sdo Configuration} fp@814: fp@1085: The Sdo configurations have to be provided by the application. This is done fp@1085: via the \textit{ecrt\_slave\_conf\_sdo*()} methods (see fp@1085: section~\ref{sec:ecrt-slave}), that are part of the realtime interface. The fp@1085: slave stores the Sdo configurations in a linked list, but does not apply them fp@1085: at once. fp@369: fp@814: \paragraph{Sdo Download State Machine} fp@814: fp@814: The best time to apply Sdo configurations is during the slave's PREOP fp@379: state, because mailbox communication is already possible and slave's fp@379: application will start with updating input data in the succeeding fp@814: SAFEOP state. Therefore the Sdo configuration has to be part of the fp@379: slave configuration state machine (see section~\ref{sec:fsm-conf}): It fp@814: is implemented via an Sdo download state machine, that is executed fp@813: just before entering the slave's SAFEOP state. In this way, it is fp@814: guaranteed that the Sdo configurations are applied each time, the fp@379: slave is reconfigured. fp@369: fp@814: The transition diagram of the Sdo Download state machine can be seen fp@369: in figure~\ref{fig:fsm-coedown}. fp@369: fp@369: \begin{figure}[htbp] fp@369: \centering fp@369: \includegraphics[width=.9\textwidth]{images/fsm-coedown} fp@369: \caption{Transition diagram of the CoE download state machine} fp@369: \label{fig:fsm-coedown} fp@369: \end{figure} fp@369: fp@369: \begin{description} fp@369: \item[START] The beginning state of the CoE download state fp@814: machine. The ``Sdo Download Normal Request'' mailbox command is fp@379: sent. $\rightarrow$~REQUEST fp@369: fp@369: \item[REQUEST] It is checked, if the CoE download request has been fp@369: received by the slave. After that, a mailbox check command is issued fp@379: and a timer is started. $\rightarrow$~CHECK fp@369: fp@369: \item[CHECK] If no mailbox data is available, the timer is checked. fp@369: \begin{itemize} fp@814: \item If it timed out, the Sdo download is aborted. fp@379: $\rightarrow$~ERROR fp@369: \item Otherwise, the mailbox is queried again. fp@379: $\rightarrow$~CHECK fp@369: \end{itemize} fp@369: fp@369: If the mailbox contains new data, the response is fetched. fp@379: $\rightarrow$~RESPONSE fp@369: fp@369: \item[RESPONSE] If the mailbox response could not be fetched, the data fp@814: is invalid, the wrong protocol was received, or a ``Abort Sdo fp@814: Transfer Request'' was received, the Sdo download is aborted. fp@379: $\rightarrow$~ERROR fp@369: fp@814: If a ``Sdo Download Normal Response'' acknowledgement was received, fp@814: the Sdo download was successful. $\rightarrow$~END fp@814: fp@814: \item[END] The Sdo download was successful. fp@814: fp@814: \item[ERROR] The Sdo download was aborted due to an error. fp@369: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \chapter{User Space} fp@369: \label{sec:user} fp@369: \index{User space} fp@369: fp@369: For the master runs as a kernel module, accessing it is natively fp@1085: limited to analyzing Syslog messages and controlling using modutils. fp@369: fp@369: It is necessary to implement further interfaces, that make it easier fp@369: to access the master from user space and allow a finer influence. It fp@369: should be possible to view and to change special parameters at runtime. fp@369: fp@369: Bus visualization is a second point: For development and debugging fp@369: purposes it would be nice, if one could show the connected slaves with fp@369: a single command. fp@369: fp@369: Another aspect is automatic startup and configuration. If the master fp@369: is to be integrated into a running system, it must be able to fp@369: automatically start with a persistent configuration. fp@369: fp@369: A last thing is monitoring EtherCAT communication. For debugging fp@369: purposes, there had to be a way to analyze EtherCAT datagrams. The fp@369: best way would be with a popular network analyzer, like Wireshark fp@369: \cite{wireshark} (the former Ethereal) or others. fp@369: fp@369: This section covers all those points and introduces the interfaces and fp@369: tools to make all that possible. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1087: \section{Command-line Tool} fp@1087: \label{sec:ethercat} fp@1087: fp@1087: % --master fp@1087: fp@1087: \subsection{Character devices} fp@1087: \label{sec:cdev} fp@1087: fp@1087: Each master instance will get a character device as a user-space interface. fp@1087: The devices are named \textit{/dev/EtherCATX}, where $X$ is the index of the fp@1087: master. fp@1087: fp@1087: % FIXME fp@1087: % udev fp@1087: % rights fp@1087: fp@1087: %------------------------------------------------------------------------------ fp@1087: fp@1087: \subsection{Listing the bus} fp@1087: fp@1087: Slave information can be gathered with the subcommand \lstinline+slaves+: fp@1087: fp@1087: \begin{lstlisting} fp@1087: $ `\textbf{ethercat slaves}` fp@1087: 0 0:0 PREOP + EK1100 Ethernet Kopplerklemme (2A E-Bus) fp@1087: 1 5555:0 PREOP + EL3162 2K. Ana. Eingang 0-10V fp@1087: 2 5555:1 PREOP + EL4102 2K. Ana. Ausgang 0-10V fp@1087: 3 5555:2 PREOP + EL2004 4K. Dig. Ausgang 24V, 0,5A fp@1087: \end{lstlisting} fp@1087: fp@1087: Every slave found is displayed as one text row. The columns have the following fp@1087: meanings: fp@1087: fp@1087: \begin{enumerate} fp@1087: fp@1087: \item Ring position in the bus. fp@1087: fp@1087: \item Alias and position (see section~\ref{sec:addr}). fp@1087: fp@1087: \item Application-layer state. fp@1087: fp@1087: \item Error flag: \lstinline!+! means that the slave is ok, \lstinline+E+ fp@1087: means that an error has occurred during scanning or configuration. fp@1087: fp@1087: \item The slave's name, as it appears in the ``general'' SII category. If no fp@1087: name is found, the slave's vendor ID and product code are listed. fp@1087: fp@1087: \end{enumerate} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \subsection{SII Access} fp@1085: \label{sec:siiaccess} fp@1085: \index{SII!Access} fp@369: fp@1087: It is possible to directly read or write the complete SII contents of the fp@1087: slaves. This was introduced for the reasons below: fp@369: fp@369: \begin{itemize} fp@1087: fp@1087: \item The format of the SII data is still in development and categories can be fp@1087: added in the future. With read and write access, the complete memory contents fp@1087: can be easily backed up and restored. fp@1087: fp@1087: \item Some SII data fields have to be altered (like the alias address). A quick fp@1087: writing must be possible for that. fp@1087: fp@1087: \item Through reading access, analyzing category data is possible from user fp@1087: space. fp@1087: fp@369: \end{itemize} fp@369: fp@1087: Reading out SII data is as easy as other commands. Though the data are in fp@1087: binary format, analysis is easier with a tool like \textit{hexdump}: fp@1087: fp@1087: \begin{lstlisting} fp@1087: $ `\textbf{ethercat sii\_read --slave 3 | hexdump}` fp@1087: 0000000 0103 0000 0000 0000 0000 0000 0000 008c fp@1087: 0000010 0002 0000 3052 07f0 0000 0000 0000 0000 fp@1087: 0000020 0000 0000 0000 0000 0000 0000 0000 0000 fp@1087: ... fp@1087: \end{lstlisting} fp@1087: fp@1087: Backing up SII contents can easily done with a redirection: fp@1087: fp@1087: \begin{lstlisting} fp@1087: $ `\textbf{ethercat sii\_read --slave 3 > sii-of-slave3.bin}` fp@1087: \end{lstlisting} fp@1087: fp@1087: To download SII contents to a slave, writing access to the master's character fp@1087: device is necessary (see section~\ref{sec:cdev}). fp@1087: fp@1087: \begin{lstlisting} fp@1087: # `\textbf{ethercat sii\_write --slave 3 sii-of-slave3.bin}` fp@1087: \end{lstlisting} fp@1087: fp@1087: The SII contents will be checked for validity and then sent to the slave. The fp@1087: write operation may take a few seconds. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{System Integration} fp@369: \label{sec:system} fp@369: fp@1086: To integrate the EtherCAT master as a service into a running system, it comes fp@1086: with an init script and a sysconfig file, that are described below. fp@1086: fp@1086: \subsection{Init Script} fp@369: \label{sec:init} fp@369: \index{Init script} fp@369: fp@1086: The EtherCAT master init script conforms to the requirements of the ``Linux fp@1086: Standard Base'' (LSB\index{LSB}, \cite{lsb}). The script is installed to fp@1086: \textit{etc/init.d/ethercat} below the installation prefix and has to be copied fp@1086: (or better: linked) to the appropriate location (see fp@1086: section~\ref{sec:install}), before the master can be inserted as a service. fp@1086: Please note, that the init script depends on the sysconfig file described fp@1086: below. fp@1086: fp@1086: To provide service dependencies (i.~e. which services have to be started before fp@1086: others) inside the init script code, LSB defines a special comment block. fp@1086: System tools can extract this information to insert the EtherCAT init script at fp@1086: the correct place in the startup sequence: fp@1086: fp@1086: \lstinputlisting[firstline=38,lastline=48] fp@1086: {../script/init.d/ethercat} fp@1086: fp@1086: \subsection{Sysconfig} fp@1086: \label{sec:sysconfig} fp@1086: \index{Sysconfig file} fp@1086: fp@1086: For persistent configuration, the init script uses a sysconfig file installed fp@1086: to \textit{etc/sysconfig/ethercat} (below the installation prefix), that is fp@1086: mandatory for the init script. The sysconfig file contains all configuration fp@1086: variables needed to operate one or more masters. The documentation is inside fp@1086: the file and included below: fp@1086: fp@1086: \lstinputlisting[numbers=left,firstline=9,basicstyle=\ttfamily\scriptsize] fp@1086: {../script/sysconfig/ethercat} fp@1086: fp@1086: \subsection{Service} fp@1086: \label{sec:service} fp@1086: \index{Service} fp@1086: fp@1086: After the init script and the sysconfig file are placed into the right fp@1086: location, the EtherCAT master can be inserted as a service. The different Linux fp@1086: distributions offer different ways to mark a service for starting and stopping fp@1086: in certain runlevels. For example, SUSE Linux provides the \textit{insserv} fp@1086: command: fp@1086: fp@1086: \begin{lstlisting} fp@1086: # `\textbf{insserv ethercat}` fp@369: \end{lstlisting} fp@369: fp@369: The init script can also be used for manually starting and stopping fp@369: the EtherCAT master. It has to be executed with one of the parameters fp@379: \texttt{start}, \texttt{stop}, \texttt{restart} or \texttt{status}. fp@369: fp@1085: \begin{lstlisting}[gobble=2] fp@379: # `\textbf{/etc/init.d/ethercat restart}` fp@369: Shutting down EtherCAT master done fp@369: Starting EtherCAT master done fp@369: \end{lstlisting} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{Monitoring and Debugging} fp@369: \label{sec:debug} fp@369: \index{Monitoring} fp@369: fp@1085: For debugging purposes, every EtherCAT master registers a read-only network fp@1085: interface \textit{ecX}, where X is a number, provided by the kernel on device fp@1085: registration. While it is ``up'', the master forwards every frame sent and fp@1085: received to this interface. fp@1085: fp@1085: This makes it possible to connect an network monitor (like Wireshark or fp@1085: tcpdump) to the debug interface and monitor the EtherCAT frames. fp@1085: fp@1085: % FIXME schedule() fp@1085: It has to be considered, that can be frame rate can be very high. The master fp@1085: state machine usually runs every kernel timer interrupt (usually up to fp@1085: \unit{1}{\kilo\hertz}) and with a connected application, the rate can be even fp@369: higher. fp@369: fp@369: \paragraph{Attention:} The socket buffers needed for the operation of fp@369: the debugging interface have to be allocated dynamically. Some Linux fp@369: realtime extensions do not allow this in realtime context! fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \chapter{Timing Aspects} fp@369: \label{sec:timing} fp@369: fp@1085: Although EtherCAT's timing is highly deterministic and therefore timing issues fp@1085: are rare, there are a few aspects that can (and should be) dealt with. fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \subsection{Realtime Interface Profiling} fp@369: \label{sec:timing-profile} fp@369: \index{Realtime!Profiling} fp@369: fp@1085: One of the most important timing aspects are the execution times of the fp@369: realtime interface functions, that are called in cyclic context. These fp@1085: functions make up an important part of the overall timing of the application. fp@1085: To measure the timing of the functions, the following code was used: fp@369: fp@369: \begin{lstlisting}[gobble=2,language=C] fp@369: c0 = get_cycles(); fp@369: ecrt_master_receive(master); fp@369: c1 = get_cycles(); fp@369: ecrt_domain_process(domain1); fp@369: c2 = get_cycles(); fp@369: ecrt_master_run(master); fp@369: c3 = get_cycles(); fp@369: ecrt_master_send(master); fp@369: c4 = get_cycles(); fp@369: \end{lstlisting} fp@369: fp@1085: Between each call of an interface function, the CPU timestamp counter is read. fp@1085: The counter differences are converted to \micro\second\ with help of the fp@1085: \lstinline+cpu_khz+ variable, that contains the number of increments per fp@1085: \milli\second. fp@1085: fp@1085: For the actual measuring, a system with a \unit{2.0}{\giga\hertz} CPU was used, fp@1085: that ran the above code in an RTAI thread with a period of fp@1085: \unit{100}{\micro\second}. The measuring was repeated $n = 100$ times and the fp@1085: results were averaged. These can be seen in table~\ref{tab:profile}. fp@369: fp@369: \begin{table}[htpb] fp@369: \centering fp@1085: \caption{Profiling of a Realtime Cycle on a \unit{2.0}{\giga\hertz} fp@1085: Processor} fp@369: \label{tab:profile} fp@369: \vspace{2mm} fp@369: \begin{tabular}{l|r|r} fp@1085: Element & Mean Duration [\second] & Standard Deviancy [\micro\second] \\ fp@369: \hline fp@369: \textit{ecrt\_master\_receive()} & 8.04 & 0.48\\ fp@369: \textit{ecrt\_domain\_process()} & 0.14 & 0.03\\ fp@369: \textit{ecrt\_master\_run()} & 0.29 & 0.12\\ fp@369: \textit{ecrt\_master\_send()} & 2.18 & 0.17\\ \hline fp@369: Complete Cycle & 10.65 & 0.69\\ \hline fp@369: \end{tabular} fp@369: \end{table} fp@369: fp@1085: It is obvious, that the functions accessing hardware make up the fp@369: lion's share. The \textit{ec\_master\_receive()} executes the ISR of fp@369: the Ethernet device, analyzes datagrams and copies their contents into fp@369: the memory of the datagram objects. The \textit{ec\_master\_send()} fp@369: assembles a frame out of different datagrams and copies it to the fp@369: hardware buffers. Interestingly, this makes up only a quarter of the fp@369: receiving time. fp@369: fp@1085: The functions that only operate on the masters internal data structures are fp@1085: very fast ($\Delta t < \unit{1}{\micro\second}$). Interestingly the runtime of fp@1085: \textit{ec\_domain\_process()} has a small standard deviancy relative to the fp@1085: mean value, while this ratio is about twice as big for fp@1085: \textit{ec\_master\_run()}: This probably results from the latter function fp@1085: having to execute code depending on the current state and the different state fp@1085: functions are more or less complex. fp@1085: fp@1085: For a realtime cycle makes up about \unit{10}{\micro\second}, the theoretical fp@1085: frequency can be up to \unit{100}{\kilo\hertz}. For two reasons, this frequency fp@369: keeps being theoretical: fp@369: fp@369: \begin{enumerate} fp@1085: fp@1085: \item The processor must still be able to run the operating system between the fp@1085: realtime cycles. fp@1085: fp@1085: \item The EtherCAT frame must be sent and received, before the next realtime fp@1085: cycle begins. The determination of the bus cycle time is difficult and covered fp@1085: in section~\ref{sec:timing-bus}. fp@1085: fp@369: \end{enumerate} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \subsection{Bus Cycle Measuring} fp@369: \label{sec:timing-bus} fp@369: \index{Bus cycle} fp@369: fp@369: For measuring the time, a frame is ``on the wire'', two timestamps fp@369: must be be taken: fp@369: fp@369: \begin{enumerate} fp@369: \item The time, the Ethernet hardware begins with physically sending fp@369: the frame. fp@369: \item The time, the frame is completely received by the Ethernet fp@369: hardware. fp@369: \end{enumerate} fp@369: fp@369: Both times are difficult to determine. The first reason is, that the fp@369: interrupts are disabled and the master is not notified, when a frame fp@369: is sent or received (polling would distort the results). The second fp@369: reason is, that even with interrupts enabled, the time from the event fp@369: to the notification is unknown. Therefore the only way to confidently fp@369: determine the bus cycle time is an electrical measuring. fp@369: fp@1085: Anyway, the bus cycle time is an important factor when designing realtime code, fp@1085: because it limits the maximum frequency for the cyclic task of the application. fp@1085: In practice, these timing parameters are highly dependent on the hardware and fp@1085: often a trial and error method must be used to determine the limits of the fp@1085: system. fp@1085: fp@1085: The central question is: What happens, if the cycle frequency is too high? The fp@1085: answer is, that the EtherCAT frames that have been sent at the end of the cycle fp@1085: are not yet received, when the next cycle starts. First this is noticed by fp@1085: \textit{ecrt\_domain\_process()}, because the working counter of the process fp@1085: data datagrams were not increased. The function will notify the user via fp@1085: Syslog\footnote{To limit Syslog output, a mechanism has been implemented, that fp@1085: outputs a summarized notification at maximum once a second.}. In this case, the fp@1085: process data keeps being the same as in the last cycle, because it is not fp@1085: erased by the domain. When the domain datagrams are queued again, the master fp@1085: notices, that they are already queued (and marked as sent). The master will fp@1085: mark them as unsent again and output a warning, that datagrams were fp@1085: ``skipped''. fp@1085: fp@1085: On the mentioned \unit{2.0}{\giga\hertz} system, the possible cycle frequency fp@1085: can be up to \unit{25}{\kilo\hertz} without skipped frames. This value can fp@1085: surely be increased by choosing faster hardware. Especially the RealTek network fp@1085: hardware could be replaced by a faster one. Besides, implementing a dedicated fp@1085: ISR for EtherCAT devices would also contribute to increasing the latency. These fp@1085: are two points on the author's to-do list. fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1085: \chapter{Installation} fp@1085: \label{sec:installation} fp@1085: \index{Master!Installation} fp@369: fp@1094: \section{Building the software} fp@1094: fp@1094: The current EtherCAT master code is available at~\cite{etherlab} or can be fp@1094: obtained from the EtherLab CD. The \textit{tar.bz2} file has to be unpacked fp@1094: with the commands below (or similar): fp@369: fp@1085: \begin{lstlisting}[gobble=2] fp@487: `\$` `\textbf{tar xjf ethercat-\masterversion.tar.bz2}` fp@487: `\$` `\textbf{cd ethercat-\masterversion/}` fp@374: \end{lstlisting} fp@374: fp@374: The tarball was created with GNU Autotools, so the build process fp@487: follows the below commands: fp@369: fp@1085: \begin{lstlisting}[gobble=2] fp@379: `\$` `\textbf{./configure}` fp@1085: `\$` `\textbf{make}` fp@487: `\$` `\textbf{make modules}` fp@374: \end{lstlisting} fp@374: fp@1085: Table~\ref{tab:config} lists important configuration switches and options. fp@1085: fp@1085: \begin{table} fp@1085: \caption{Configuration options} fp@1085: \label{tab:config} fp@1085: \vspace{2mm} fp@1085: \begin{tabular}{l|p{.3\textwidth}|l} fp@1085: fp@1085: \bf Option/Switch & \bf Description & \bf Default\\\hline fp@1085: fp@1085: \lstinline+--prefix+ & Installation prefix & \textit{/opt/etherlab}\\ fp@1085: fp@1085: \lstinline+--with-linux-dir+ & Linux kernel sources & Use running kernel\\ fp@1085: fp@1085: \lstinline+--with-rtai-dir+ & RTAI path (only for RTAI example) & \\ fp@1085: fp@1085: \hline fp@1085: fp@1085: \lstinline+--enable-eoe+ & Enable EoE support & yes\\ fp@1085: fp@1085: \lstinline+--enable-cycles+ & Use CPU timestamp counter. Enable this on Intel fp@1085: architecture to get finer timing calculation. & no\\ fp@1085: fp@1085: \lstinline+--enable-debug-if+ & Create a debug interface for each master & no\\ fp@1085: fp@1085: \lstinline+--enable-debug-ring+ & Create a debug ring to record frames & no\\ fp@1085: fp@1085: \hline fp@1085: fp@1085: \lstinline+--enable-8139too+ & Build the 8139too driver & yes\\ fp@1085: fp@1085: \lstinline+--with-8139too-kernel+ & 8139too kernel & $\dagger$\\ fp@1085: fp@1085: \lstinline+--enable-e100+ & Build the e100 driver & no\\ fp@1085: fp@1085: \lstinline+--with-e100-kernel+ & e100 kernel & $\dagger$\\ fp@1085: fp@1085: \lstinline+--enable-forcedeth+ & Enable forcedeth driver & no\\ fp@1085: fp@1085: \lstinline+--with-forcedeth-kernel+ & forcedeth kernel & $\dagger$\\ fp@1085: fp@1085: \lstinline+--enable-e1000+ & Enable e1000 driver & no\\ fp@1085: fp@1085: \lstinline+--with-e1000-kernel+ & e1000 kernel & $\dagger$\\ fp@1085: fp@1085: \lstinline+--enable-r8169+ & Enable r8169 driver & no\\ fp@1085: fp@1085: \lstinline+--with-r8169-kernel+ & r8169 kernel & $\dagger$\\ fp@1085: fp@1085: \end{tabular} fp@1085: \vspace{2mm} fp@1085: fp@1085: \begin{description} fp@1085: fp@1085: \item[$\dagger$] If this option is not specified, the kernel version to use is fp@1085: extracted from the Linux kernel sources. fp@1085: fp@1085: \end{description} fp@1085: fp@1085: \end{table} fp@487: fp@1094: \section{Building the documentation} fp@1094: \label{sec:gendoc} fp@1094: fp@1094: The source code is documented using Doxygen~\cite{doxygen}. To build the HTML fp@1094: documentation, you must have the Doxygen software installed. The below command fp@1095: will generate the documents in the subdirectory \textit{doxygen-output}: fp@1094: fp@1094: \begin{lstlisting} fp@1094: $ `\textbf{make doc}` fp@1094: \end{lstlisting} fp@1094: fp@1094: To view them, point your browser to \textit{doxygen-output/html/index.html}. fp@1094: fp@1094: \section{Installation} fp@1094: fp@1106: The below commands have to be entered as \textit{root}: The first one will fp@1106: install the EtherCAT header, init script, sysconfig file and the user space fp@1106: tools to the prefix path. The second one will install the kernel modules to the fp@1106: kernel's modules directory. The following \lstinline+depmod+ call is necessary fp@1106: to include the kernel modules into the \textit{modules.dep} file to make it fp@1106: available to the \lstinline+modprobe+ command, used in the init script. fp@369: fp@1094: \begin{lstlisting} fp@1106: # `\textbf{make install}` fp@1094: # `\textbf{make modules\_install}` fp@1106: # `\textbf{depmod}` fp@369: \end{lstlisting} fp@369: fp@1095: If the target kernel's modules directory is not under \textit{/lib/modules}, a fp@1095: different destination directory can be specified with the \lstinline+DESTDIR+ fp@1095: make variable. For example: fp@487: fp@1094: \begin{lstlisting} fp@1094: # `\textbf{make DESTDIR=/vol/nfs/root modules\_install}` fp@487: \end{lstlisting} fp@487: fp@487: This command will install the compiled kernel modules to fp@487: \textit{/vol/nfs/root/lib/modules}, prepended by the kernel release. fp@487: fp@1085: If the EtherCAT master shall be run as a service\footnote{Even if the EtherCAT fp@1085: master shall not be loaded on system startup, the use of the init script is fp@1086: recommended for manual (un-)loading.} (see section~\ref{sec:system}), the init fp@1086: script and the sysconfig file have to be copied (or linked) to the appropriate fp@1086: locations. The below example is suitable for SUSE Linux. It may vary for other fp@1086: distributions. fp@1085: fp@1107: % FIXME relative ln -s? fp@1094: \begin{lstlisting} fp@1094: # `\textbf{cd /opt/etherlab}` fp@1094: # `\textbf{cp etc/sysconfig/ethercat /etc/sysconfig/}` fp@1094: # `\textbf{ln -s etc/init.d/ethercat /etc/init.d/}` fp@1094: # `\textbf{insserv ethercat}` fp@374: \end{lstlisting} fp@374: fp@376: Now the sysconfig file \texttt{/etc/sysconfig/ethercat} (see fp@1085: section~\ref{sec:sysconfig}) has to be customized. The minimal customization fp@1085: is to set the \lstinline+MASTER0_DEVICE+ variable to the MAC address of the fp@1085: Ethernet device to use (or \lstinline+ff:ff:ff:ff:ff:ff+ to use the first fp@1085: device offered) and selecting the driver(s) to load via the fp@1085: \lstinline+DEVICE_MODULES+ variable. fp@369: fp@369: After the basic configuration is done, the master can be started with fp@369: the below command: fp@369: fp@1094: \begin{lstlisting} fp@1094: # `\textbf{/etc/init.d/ethercat start}` fp@369: \end{lstlisting} fp@369: fp@369: The operation of the master can be observed by looking at the fp@1085: Syslog\index{Syslog} messages, which should look like the ones below. If fp@1085: EtherCAT slaves are connected to the master's EtherCAT device, the activity fp@1085: indicators should begin to flash. fp@369: fp@369: \begin{lstlisting}[numbers=left] fp@1085: EtherCAT: Master driver `\masterversion` fp@1085: EtherCAT: 1 master waiting for devices. fp@1085: EtherCAT Intel(R) PRO/1000 Network Driver - version 6.0.60-k2 fp@1085: Copyright (c) 1999-2005 Intel Corporation. fp@1085: PCI: Found IRQ 12 for device 0000:01:01.0 fp@1085: PCI: Sharing IRQ 12 with 0000:00:1d.2 fp@1085: PCI: Sharing IRQ 12 with 0000:00:1f.1 fp@1085: EtherCAT: Accepting device 00:0E:0C:DA:A2:20 for master 0. fp@1085: EtherCAT: Starting master thread. fp@1085: ec_e1000: ec0: e1000_probe: Intel(R) PRO/1000 Network fp@1085: Connection fp@1085: ec_e1000: ec0: e1000_watchdog_task: NIC Link is Up 100 Mbps fp@1085: Full Duplex fp@1085: EtherCAT: Link state changed to UP. fp@1085: EtherCAT: 7 slave(s) responding. fp@1085: EtherCAT: Slave states: PREOP. fp@1085: EtherCAT: Scanning bus. fp@1085: EtherCAT: Bus scanning completed in 431 ms. fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: fp@1085: \item[\linenum{1} -- \linenum{2}] The master module is loading, and one master fp@1085: is initialized. fp@1085: fp@1085: \item[\linenum{3} -- \linenum{8}] The EtherCAT-capable e1000 driver is fp@1085: loading. The master accepts the device with the address fp@1085: \lstinline+00:0E:0C:DA:A2:20+. fp@1085: fp@1085: \item[\linenum{9} -- \linenum{16}] The master goes to idle phase, starts its fp@1085: state machine and begins scanning the bus. fp@1085: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \chapter{Application examples} fp@1085: \label{chapter:examples} fp@1085: fp@1085: This chapter will give practical examples of how to use the EtherCAT master via fp@1085: the realtime interface by writing an application module. fp@1085: fp@1085: %------------------------------------------------------------------------------ fp@1085: fp@1085: \section{Minimal Example} fp@369: \label{sec:mini} fp@369: \index{Examples!Minimal} fp@369: fp@1085: This section will explain the use of the EtherCAT master from a minimal kernel fp@1085: module. The complete module code is obtainable as a part of the EtherCAT master fp@1085: code release (see~\cite{etherlab}, file \textit{examples/mini/mini.c}). fp@1085: fp@1085: The minimal example uses a kernel timer (software interrupt) to generate a fp@1085: cyclic task. After the timer function is executed, it re-adds itself with a fp@1085: delay of one \textit{jiffy}\index{jiffies}, which results in a timer frequency fp@1085: of \textit{HZ}\nomenclature{HZ}{Kernel macro containing the timer interrupt fp@1085: frequency} fp@369: fp@369: The module-global variables, needed to operate the master can be seen fp@369: in listing~\ref{lst:minivar}. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={Minimal fp@369: variables},label=lst:minivar] fp@369: struct timer_list timer; fp@369: fp@369: ec_master_t *master = NULL; fp@369: ec_domain_t *domain1 = NULL; fp@369: fp@369: void *r_dig_in, *r_ana_out; fp@369: fp@369: ec_pdo_reg_t domain1_pdos[] = { fp@369: {"1", Beckhoff_EL1014_Inputs, &r_dig_in}, fp@369: {"2", Beckhoff_EL4132_Ouput1, &r_ana_out}, fp@369: {} fp@369: }; fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{1}] There is a timer object fp@369: declared, that is needed to tell the kernel to install a timer and fp@369: execute a certain function, if it runs out. This is done by a fp@369: variable of the \textit{timer\_list} structure. fp@1085: \item[\linenum{3} -- \linenum{4}] There fp@369: is a pointer declared, that will later point to a requested EtherCAT fp@369: master. Additionally there is a pointer to a domain object needed, fp@369: that will manage process data IO. fp@1085: \item[\linenum{6}] The pointers \textit{r\_*} fp@369: will later point to the \underline{r}aw process data values inside fp@369: the domain memory. The addresses they point to will be set during a fp@369: call to \textit{ec\_\-master\_\-activate()}, that will create the fp@369: domain memory and configure the mapped process data image. fp@1085: \item[\linenum{8} -- \linenum{12}] The fp@814: configuration of the mapping of certain Pdos in a domain can easily fp@369: be done with the help of an initialization array of the fp@369: \textit{ec\_pdo\_reg\_t} type, defined as part of the realtime fp@369: interface. Each record must contain the ASCII bus-address of the fp@369: slave (see section~\ref{sec:addr}), the slave's vendor ID and fp@814: product code, and the index and subindex of the Pdo to map (these fp@369: four fields can be specified in junction, by using one of the fp@369: defines out of the \textit{include/ecdb.h} header). The last field fp@369: has to be the address of the process data pointer, so it can later fp@369: be redirected appropriately. Attention: The initialization array fp@369: must end with an empty record (\textit{\{\}})! fp@369: \end{description} fp@369: fp@1085: The initialization of the minimal application is done by the ``Minimal init fp@1085: function'' in listing~\ref{lst:miniinit}. fp@1085: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={Minimal init fp@369: function},label={lst:miniinit}] fp@369: int __init init_mini_module(void) fp@369: { fp@369: if (!(master = ecrt_request_master(0))) { fp@369: goto out_return; fp@369: } fp@369: fp@369: if (!(domain1 = ecrt_master_create_domain(master))) { fp@369: goto out_release_master; fp@369: } fp@369: fp@369: if (ecrt_domain_register_pdo_list(domain1, fp@369: domain1_pdos)) { fp@369: goto out_release_master; fp@369: } fp@369: fp@369: if (ecrt_master_activate(master)) { fp@369: goto out_release_master; fp@369: } fp@369: fp@369: ecrt_master_prepare(master); fp@369: fp@369: init_timer(&timer); fp@369: timer.function = run; fp@369: timer.expires = jiffies + 10; fp@369: add_timer(&timer); fp@369: fp@369: return 0; fp@369: fp@369: out_release_master: fp@369: ecrt_release_master(master); fp@369: out_return: fp@369: return -1; fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{3}] It is tried to request the fp@369: first EtherCAT master (index 0). On success, the fp@369: \textit{ecrt\_\-request\_\-master()} function returns a pointer to fp@369: the reserved master, that can be used as an object to following fp@369: functions calls. On failure, the function returns \textit{NULL}. fp@1085: \item[\linenum{7}] In order to exchange process fp@369: data, a domain object has to be created. The fp@369: \textit{ecrt\_\-master\_\-create\_domain()} function also returns a fp@369: pointer to the created domain, or \textit{NULL} in error case. fp@1085: \item[\linenum{11}] The registration of domain fp@814: Pdos with an initialization array results in a single function call. fp@369: Alternatively the data fields could be registered with individual fp@369: calls of \textit{ecrt\_domain\_register\_pdo()}. fp@1085: \item[\linenum{16}] After the configuration of fp@369: process data mapping, the master can be activated for cyclic fp@369: operation. This will configure all slaves and bring them into fp@379: OP state. fp@1085: \item[\linenum{20}] This call is needed to avoid fp@369: a case differentiation in cyclic operation: The first operation in fp@369: cyclic mode is a receive call. Due to the fact, that there is fp@369: nothing to receive during the first cycle, there had to be an fp@369: \textit{if}-statement to avoid a warning. A call to fp@369: \textit{ec\_master\_prepare()} sends a first datagram containing a fp@369: process data exchange datagram, so that the first receive call will fp@369: not fail. fp@1085: \item[\linenum{22} -- \linenum{25}] The fp@369: master is now ready for cyclic operation. The kernel timer that fp@369: cyclically executes the \textit{run()} function is initialized and fp@369: started. fp@369: \end{description} fp@369: fp@369: The coding of a cleanup function fo the minimal module can be seen in fp@369: listing~\ref{lst:miniclean}. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={Minimal cleanup fp@369: function},label={lst:miniclean}] fp@369: void __exit cleanup_mini_module(void) fp@369: { fp@369: del_timer_sync(&timer); fp@369: ecrt_master_deactivate(master); fp@369: ecrt_release_master(master); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{3}] To cleanup the module, it it fp@369: necessary to stop the cyclic processing. This is done by a call to fp@369: \textit{del\_timer\_sync()} which safely removes a queued timer fp@369: object. It is assured, that no cyclic work will be done after this fp@369: call returns. fp@1085: \item[\linenum{4}] This call deactivates the fp@379: master, which results in all slaves being brought to their INIT fp@379: state again. fp@1085: \item[\linenum{5}] This call releases the master, fp@369: removes any existing configuration and silently starts the idle fp@369: mode. The value of the master pointer is invalid after this call and fp@369: the module can be safely unloaded. fp@369: \end{description} fp@369: fp@369: The final part of the minimal module is that for the cyclic work. Its fp@369: coding can be seen in listing~\ref{lst:minirun}. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={Minimal cyclic fp@369: function},label={lst:minirun}] fp@369: void run(unsigned long data) fp@369: { fp@369: static uint8_t dig_in_0; fp@369: fp@369: ecrt_master_receive(master); fp@369: ecrt_domain_process(domain1); fp@369: fp@369: dig_in_0 = EC_READ_BIT(r_dig_in, 0); fp@369: EC_WRITE_S16(r_ana_out, dig_in_0 * 0x3FFF); fp@369: fp@369: ecrt_master_run(master); fp@369: ecrt_master_send(master); fp@369: fp@369: timer.expires += 1; // frequency = HZ fp@369: add_timer(&timer); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: fp@1085: \item[\linenum{5}] The cyclic processing starts with receiving datagrams, that fp@1085: were sent in the last cycle. The frames containing these datagrams have to be fp@1085: received by the network interface card prior to this call. fp@1085: fp@1085: \item[\linenum{6}] The process data of domain 1 has been automatically copied fp@1085: into domain memory while datagram reception. This call checks the working fp@1085: counter for changes and re-queues the domain's datagram for sending. fp@1085: fp@1085: \item[\linenum{8}] This is an example for reading out a bit-oriented process fp@1085: data value (i.~e. bit 0) via the \textit{EC\_READ\_BIT()} macro. See fp@1085: section~\ref{sec:macros} for more information about those macros. fp@1085: fp@1085: \item[\linenum{9}] This line shows how to write a signed, 16-bit process data fp@1085: value. In this case, the slave is able to output voltages of fp@1085: \unit{-10--+10}{\volt} with a resolution of \unit{16}{bit}. This write command fp@1085: outputs either \unit{0}{\volt} or \unit{+5}{\volt}, depending of the value of fp@1085: \textit{dig\_in\_0}. fp@1085: fp@1085: \item[\linenum{11}] This call runs the master's operation state machine (see fp@1085: section~\ref{sec:fsm-op}). A single state is processed, and datagrams are fp@1085: queued. Mainly bus observation is done: The bus state is determined and in case fp@1085: of slaves that lost their configuration, reconfiguration is tried. fp@1085: fp@1085: \item[\linenum{12}] This method sends all queued datagrams, in this case the fp@1085: domain's datagram and one of the master state machine. In best case, all fp@1085: datagrams fit into one frame. fp@1085: fp@1085: \item[\linenum{14} -- \linenum{15}] Kernel timers are implemented as fp@1085: ``one-shot'' timers, so they have to be re-added after each execution. The time fp@1085: of the next execution is specified in \textit{jiffies} and will happen at the fp@1085: time of the next system timer interrupt. This results in the \textit{run()} fp@1085: function being executed with a frequency of \textit{HZ}. fp@1085: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@1085: \section{RTAI Example} fp@369: \label{sec:rtai} fp@369: \index{Examples!RTAI} fp@369: fp@369: The whole code can be seen in the EtherCAT master code release fp@369: (see~\cite{etherlab}, file \textit{examples/rtai/rtai\_sample.c}). fp@369: fp@369: Listing~\ref{lst:rtaivar} shows the defines and global variables fp@369: needed for a minimal RTAI module with EtherCAT processing. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={RTAI task fp@369: declaration},label={lst:rtaivar}] fp@369: #define FREQUENCY 10000 fp@369: #define TIMERTICKS (1000000000 / FREQUENCY) fp@369: fp@369: RT_TASK task; fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{1} -- \linenum{2}] RTAI fp@369: takes the cycle period as nanoseconds, so the easiest way is to fp@369: define a frequency and convert it to a cycle time in nanoseconds. fp@1085: \item[\linenum{4}] The \textit{task} variable fp@369: later contains information about the running RTAI task. fp@369: \end{description} fp@369: fp@369: Listing~\ref{lst:rtaiinit} shows the module init function for the RTAI fp@369: module. Most lines are the same as in listing~\ref{lst:miniinit}, fp@369: differences come up when starting the cyclic code. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={RTAI module init fp@369: function},label={lst:rtaiinit}] fp@369: int __init init_mod(void) fp@369: { fp@369: RTIME requested_ticks, tick_period, now; fp@369: fp@369: if (!(master = ecrt_request_master(0))) { fp@369: goto out_return; fp@369: } fp@369: fp@369: if (!(domain1 = ecrt_master_create_domain(master))) { fp@369: goto out_release_master; fp@369: } fp@369: fp@369: if (ecrt_domain_register_pdo_list(domain1, fp@369: domain1_pdos)) { fp@369: goto out_release_master; fp@369: } fp@369: fp@369: if (ecrt_master_activate(master)) { fp@369: goto out_release_master; fp@369: } fp@369: fp@369: ecrt_master_prepare(master); fp@369: fp@369: requested_ticks = nano2count(TIMERTICKS); fp@369: tick_period = start_rt_timer(requested_ticks); fp@369: fp@369: if (rt_task_init(&task, run, 0, 2000, 0, 1, NULL)) { fp@369: goto out_stop_timer; fp@369: } fp@369: fp@369: now = rt_get_time(); fp@369: if (rt_task_make_periodic(&task, now + tick_period, fp@369: tick_period)) { fp@369: goto out_stop_task; fp@369: } fp@369: fp@369: return 0; fp@369: fp@369: out_stop_task: fp@369: rt_task_delete(&task); fp@369: out_stop_timer: fp@369: stop_rt_timer(); fp@369: out_deactivate: fp@369: ecrt_master_deactivate(master); fp@369: out_release_master: fp@369: ecrt_release_master(master); fp@369: out_return: fp@369: return -1; fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{24} -- \linenum{25}] The fp@369: nanoseconds are converted to RTAI timer ticks and an RTAI timer is fp@369: started. \textit{tick\_period} will be the ``real'' number of ticks fp@369: used for the timer period (which can be different to the requested fp@369: one). fp@1085: \item[\linenum{27}] The RTAI task is initialized fp@369: by specifying the cyclic function, the parameter to hand over, the fp@369: stack size, priority, a flag that tells, if the function will use fp@369: floating point operations and a signal handler. fp@1085: \item[\linenum{32}] The task is made periodic by fp@369: specifying a start time and a period. fp@369: \end{description} fp@369: fp@369: The cleanup function of the RTAI module in listing~\ref{lst:rtaiclean} fp@369: is nearly as simple as that of the minimal module. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={RTAI module fp@369: cleanup function},label={lst:rtaiclean}] fp@369: void __exit cleanup_mod(void) fp@369: { fp@369: rt_task_delete(&task); fp@369: stop_rt_timer(); fp@369: ecrt_master_deactivate(master); fp@369: ecrt_release_master(master); fp@369: rt_sem_delete(&master_sem); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{2}] The RTAI task will be stopped fp@369: and deleted. fp@1085: \item[\linenum{3}] After that, the RTAI timer can fp@369: be stopped. fp@369: \end{description} fp@369: fp@369: The rest is the same as for the minimal module. fp@369: fp@369: Worth to mention is, that the cyclic function of the RTAI module fp@369: (listing~\ref{lst:rtairun}) has a slightly different architecture. The fp@369: function is not executed until returning for every cycle, but has an fp@369: infinite loop in it, that is placed in a waiting state for the rest of fp@369: each cycle. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={RTAI module cyclic fp@369: function},label={lst:rtairun}] fp@369: void run(long data) fp@369: { fp@369: while (1) { fp@369: ecrt_master_receive(master); fp@369: ecrt_domain_process(domain1); fp@369: fp@369: k_pos = EC_READ_U32(r_ssi_input); fp@369: fp@369: ecrt_master_run(master); fp@369: ecrt_master_send(master); fp@369: fp@369: rt_task_wait_period(); fp@369: } fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{3}] The \textit{while (1)} loop fp@369: executes for the lifetime of the RTAI task. fp@1085: \item[\linenum{12}] The fp@369: \textit{rt\_task\_wait\_period()} function sets the process into a fp@369: sleeping state until the beginning of the next cycle. It also fp@369: checks, if the cyclic function has to be terminated. fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \section{Concurrency Example} fp@369: \label{sec:concurrency} fp@369: \index{Examples!Concurrency} fp@369: fp@1085: As mentioned before, there can be concurrent access to the EtherCAT master. The fp@1085: application and a EoE\index{EoE} process can compete for master access, for fp@1085: example. In this case, the module has to provide the locking mechanism, because fp@1085: it depends on the module's architecture which lock has to be used. The module fp@1085: makes this locking mechanism available to the master through the master's fp@1085: locking callbacks. fp@369: fp@369: In case of RTAI, the lock can be an RTAI semaphore, as shown in fp@1085: listing~\ref{lst:convar}. A normal Linux semaphore would not be appropriate, fp@1085: because it could not block the RTAI task due to RTAI running in a higher domain fp@1085: than the Linux kernel (see~\cite{rtai}). fp@1085: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={RTAI semaphore for fp@369: concurrent access},label={lst:convar}] fp@369: SEM master_sem; fp@369: \end{lstlisting} fp@369: fp@369: The module has to implement the two callbacks for requesting and fp@369: releasing the master lock. An exemplary coding can be seen in fp@369: listing~\ref{lst:conlock}. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={RTAI locking fp@369: callbacks for concurrent access},label={lst:conlock}] fp@369: int request_lock(void *data) fp@369: { fp@369: rt_sem_wait(&master_sem); fp@369: return 0; fp@369: } fp@369: fp@369: void release_lock(void *data) fp@369: { fp@369: rt_sem_signal(&master_sem); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{1}] The \textit{request\_lock()} fp@369: function has a data parameter. The master always passes the value, fp@369: that was specified when registering the callback function. This can fp@369: be used for handing the master pointer. Notice, that it has an fp@369: integer return value (see line 4). fp@1085: \item[\linenum{3}] The call to fp@369: \textit{rt\_sem\_wait()} either returns at once, when the semaphore fp@369: was free, or blocks until the semaphore is freed again. In any case, fp@369: the semaphore finally is reserved for the process calling the fp@369: request function. fp@1085: \item[\linenum{4}] When the lock was requested fp@369: successfully, the function should return 0. The module can prohibit fp@369: requesting the lock by returning non-zero (see paragraph ``Tuning fp@369: the jitter'' below). fp@1085: \item[\linenum{7}] The \textit{release\_lock()} fp@369: function gets the same argument passed, but has a void return value, fp@369: because is always succeeds. fp@1085: \item[\linenum{9}] The \textit{rt\_sem\_signal()} fp@369: function frees the semaphore, that was prior reserved with fp@369: \textit{rt\_sem\_wait()}. fp@369: \end{description} fp@369: fp@369: In the module's init function, the semaphore must be initialized, and fp@369: the callbacks must be passed to the EtherCAT master: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={Module init fp@369: function for concurrent access},label={lst:coninit}] fp@369: int __init init_mod(void) fp@369: { fp@369: RTIME tick_period, requested_ticks, now; fp@369: fp@369: rt_sem_init(&master_sem, 1); fp@369: fp@369: if (!(master = ecrt_request_master(0))) { fp@369: goto out_return; fp@369: } fp@369: fp@369: ecrt_master_callbacks(master, request_lock, fp@369: release_lock, NULL); fp@369: // ... fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{5}] The call to fp@369: \textit{rt\_sem\_init()} initializes the semaphore and sets its fp@369: value to 1, meaning that only one process can reserve the semaphore fp@369: without blocking. fp@1085: \item[\linenum{11}] The callbacks are passed to fp@369: the master with a call to \textit{ecrt\_master\_callbacks()}. The fp@369: last parameter is the argument, that the master should pass with fp@369: each call to a callback function. Here it is not used and set to fp@369: \textit{NULL}. fp@369: \end{description} fp@369: fp@369: For the cyclic function being only one competitor for master access, fp@369: it has to request the lock like any other process. There is no need to fp@369: use the callbacks (which are meant for processes of lower priority), fp@369: so it can access the semaphore directly: fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={RTAI cyclic fp@369: function for concurrent access},label={lst:conrun}] fp@369: void run(long data) fp@369: { fp@369: while (1) { fp@369: rt_sem_wait(&master_sem); fp@369: fp@369: ecrt_master_receive(master); fp@369: ecrt_domain_process(domain1); fp@369: fp@369: k_pos = EC_READ_U32(r_ssi_input); fp@369: fp@369: ecrt_master_run(master); fp@369: ecrt_master_send(master); fp@369: fp@369: rt_sem_signal(&master_sem); fp@369: rt_task_wait_period(); fp@369: } fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: fp@1085: \item[\linenum{4}] Every access to the master has to be preceded by a call to fp@1085: \textit{rt\_sem\_wait()}, because another instance might currently access the fp@1085: master. fp@1085: fp@1085: \item[\linenum{14}] When cyclic processing finished, the semaphore has to be fp@1085: freed again, so that other processes have the possibility to access the master. fp@1085: fp@369: \end{description} fp@369: fp@369: A little change has to be made to the cleanup function in case of fp@369: concurrent master access. fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={RTAI module fp@369: cleanup function for concurrent access},label={lst:conclean}] fp@369: void __exit cleanup_mod(void) fp@369: { fp@369: rt_task_delete(&task); fp@369: stop_rt_timer(); fp@369: ecrt_master_deactivate(master); fp@369: ecrt_release_master(master); fp@369: rt_sem_delete(&master_sem); fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{7}] Upon module cleanup, the fp@369: semaphore has to be deleted, so that memory can be freed. fp@369: \end{description} fp@369: fp@369: \paragraph{Tuning the Jitter} fp@369: \index{Jitter} fp@369: fp@1085: Concurrent access leads to higher jitter for the application task, because fp@1085: there are situations, in which the task has to wait for a process of lower fp@1085: priority to finish accessing the master. In most cases this is acceptable, fp@1085: because a master access cycle (receive/process/send) only takes fp@1085: \unit{10-20}{\micro\second} on recent systems, what would be the maximum fp@1085: additional jitter. However some applications demand a minimum jitter. For this fp@1085: reason the master access can be prohibited by the application: If the time, fp@1085: another process wants to access the master, is to close to the beginning of the fp@1085: next application cycle, the module can disallow, that the lock is taken. In fp@1085: this case, the request callback has to return $1$, meaning that the lock has fp@1085: not been taken. The foreign process must abort its master access and try again fp@1085: next time. fp@1085: fp@1085: This measure helps to significantly reducing the jitter produced by concurrent fp@1085: master access. Below are excerpts of an example coding: fp@1085: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={Variables for fp@369: jitter reduction},label={lst:redvar}] fp@369: #define FREQUENCY 10000 // RTAI task frequency in Hz fp@369: // ... fp@369: cycles_t t_last_cycle = 0; fp@369: const cycles_t t_critical = cpu_khz * 1000 / FREQUENCY fp@369: - cpu_khz * 30 / 1000; fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: fp@1085: \item[\linenum{3}] The variable \textit{t\_last\_cycle} holds the timer ticks fp@1085: at the beginning of the last realtime cycle. fp@1085: fp@1085: \item[\linenum{4}] \textit{t\_critical} contains the number of ticks, that may fp@1085: have passed since the beginning of the last cycle, until there is no more fp@1085: foreign access possible. It is calculated by subtracting the ticks for fp@1085: \unit{30}{\micro\second} from the ticks for a complete cycle. fp@1085: fp@369: \end{description} fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={Cyclic function fp@369: with reduced jitter},label={lst:redrun}] fp@369: void run(long data) fp@369: { fp@369: while (1) { fp@369: t_last_cycle = get_cycles(); fp@369: rt_sem_wait(&master_sem); fp@369: // ... fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: \item[\linenum{4}] The ticks of the beginning of fp@369: the current realtime cycle are taken before reserving the semaphore. fp@369: \end{description} fp@369: fp@1085: \begin{lstlisting}[gobble=2,language=C,numbers=left,caption={Request callback fp@369: for reduced jitter},label={lst:redreq}] fp@369: int request_lock(void *data) fp@369: { fp@369: // too close to the next RT cycle: deny access. fp@369: if (get_cycles() - t_last_cycle > t_critical) fp@369: return -1; fp@369: fp@369: // allow access fp@369: rt_sem_wait(&master_sem); fp@369: return 0; fp@369: } fp@369: \end{lstlisting} fp@369: fp@369: \begin{description} fp@1085: fp@1085: \item[\linenum{4}] If the time of request is too close to the next realtime fp@1085: cycle (here: \unit{<30}{\micro\second} before the estimated beginning), the fp@1085: locking is denied. The requesting process must abort its cycle. fp@1085: fp@369: \end{description} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \begin{thebibliography}{99} fp@1094: fp@1094: \bibitem{etherlab} Ingenieurgemeinschaft IgH: EtherLab -- Open Source Toolkit fp@1094: for rapid realtime code generation under Linux with Simulink/RTW and EtherCAT fp@1094: technology. \url{http://etherlab.org/en}, 2008. fp@1094: fp@369: \bibitem{dlspec} IEC 61158-4-12: Data-link Protocol Specification. fp@1095: International Electrotechnical Commission (IEC), 2005. fp@1094: fp@1094: \bibitem{alspec} IEC 61158-6-12: Application Layer Protocol Specification. fp@1095: International Electrotechnical Commission (IEC), 2005. fp@1094: fp@1094: \bibitem{gpl} GNU General Public License, Version 2. fp@1094: \url{http://www.gnu.org/licenses/gpl.txt}. August~9, 2006. fp@1094: fp@1094: \bibitem{lsb} Linux Standard Base. fp@1094: \url{http://www.linuxfoundation.org/en/LSB}. August~9, 2006. fp@1094: fp@1094: \bibitem{wireshark} Wireshark. \url{http://www.wireshark.org}. 2008. fp@1094: fp@1094: \bibitem{automata} {\it Hopcroft, J.~E. / Ullman, J.~D.}: Introduction to fp@1094: Automata Theory, Languages and Computation. Adison-Wesley, Reading, fp@1094: Mass.~1979. fp@1094: fp@369: \bibitem{fsmmis} {\it Wagner, F. / Wolstenholme, P.}: State machine fp@1094: misunderstandings. In: IEE journal ``Computing and Control Engineering'', fp@1094: 2004. fp@1094: fp@1094: \bibitem{rtai} RTAI. The RealTime Application Interface for Linux from DIAPM. fp@1094: \url{http://www.rtai.org}, 2006. fp@1094: fp@1094: \bibitem{doxygen} Doxygen. Source code documentation generator tool. fp@1094: \url{http://www.stack.nl/~dimitri/doxygen}, 2008. fp@1094: fp@369: \end{thebibliography} fp@369: fp@917: \printnomenclature fp@369: \addcontentsline{toc}{chapter}{\nomname} fp@369: \markleft{\nomname} fp@369: fp@369: \printindex fp@369: \markleft{Index} fp@369: fp@369: %------------------------------------------------------------------------------ fp@369: fp@369: \end{document} fp@369: fp@369: %------------------------------------------------------------------------------