765 callbacks, which have to be set before registration. Not every |
766 callbacks, which have to be set before registration. Not every |
766 callback is mandatory, but for reasonable operation the ones below are |
767 callback is mandatory, but for reasonable operation the ones below are |
767 needed in any case: |
768 needed in any case: |
768 |
769 |
769 \begin{description} |
770 \begin{description} |
770 \item[int (*open)(struct net\_device *)] This function is called when |
771 |
771 network communication has to be started, for example after a command |
772 \item[open()] This function is called when network communication has to be |
772 \textit{ifconfig ethX up} from user space. Frame reception has to be |
773 started, for example after a command \textit{ifconfig ethX up} from user |
773 enabled by the driver. |
774 space. Frame reception has to be enabled by the driver. |
774 \item[int (*stop)(struct net\_device *)] The purpose of this callback |
775 |
775 is to ``close'' the device, i.~e. make the hardware stop receiving |
776 \item[stop()] The purpose of this callback is to ``close'' the device, i.~e. |
776 frames. |
777 make the hardware stop receiving frames. |
777 \item[int (*hard\_start\_xmit)(struct sk\_buff *, struct net\_device |
778 |
778 *)] This function is cal\-led for each frame that has to be |
779 \item[hard\_start\_xmit()] This function is cal\-led for each frame that has |
779 transmitted. The network stack passes the frame as a pointer to an |
780 to be transmitted. The network stack passes the frame as a pointer to an |
780 \textit{sk\_buff} structure (``socket buffer''\index{Socket buffer}, |
781 \textit{sk\_buff} structure (``socket buffer''\index{Socket buffer}, see |
781 see below), which has to be freed after sending. |
782 below), which has to be freed after sending. |
782 \item[struct net\_device\_stats *(*get\_stats)(struct net\_device *)] |
783 |
783 This call has to return a pointer to the device's |
784 \item[get\_stats()] This call has to return a pointer to the device's |
784 \textit{net\_device\_stats} structure, which permanently has to be |
785 \textit{net\_device\_stats} structure, which permanently has to be filled with |
785 filled with frame statistics. This means, that every time a frame is |
786 frame statistics. This means, that every time a frame is received, sent, or an |
786 received, sent, or an error happened, the appropriate counter in |
787 error happened, the appropriate counter in this structure has to be increased. |
787 this structure has to be increased. |
788 |
788 \end{description} |
789 \end{description} |
789 |
790 |
790 The actual registration is done with the \textit{register\_netdev()} |
791 The actual registration is done with the \lstinline+register_netdev()+ call, |
791 call, unregistering is done with \textit{unregister\_netdev()}. |
792 unregistering is done with \lstinline+unregister_netdev()+. |
792 |
793 |
793 \paragraph{The netif Interface} |
794 \paragraph{The netif Interface} |
794 \index{netif} |
795 \index{netif} |
795 |
796 |
796 All other communication in the direction interface $\to$ network stack is done |
797 All other communication in the direction interface $\to$ network stack is done |
797 via the \textit{netif\_*} calls. For example, on successful device opening, the |
798 via the \lstinline+netif_*()+ calls. For example, on successful device |
798 network stack has to be notified, that it can now pass frames to the interface. |
799 opening, the network stack has to be notified, that it can now pass frames to |
799 This is done by calling \textit{netif\_start\_queue()}. After this call, the |
800 the interface. This is done by calling \lstinline+netif_start_queue()+. After |
800 \textit{hard\_start\_xmit()} callback can be called by the network stack. |
801 this call, the \lstinline+hard_start_xmit()+ callback can be called by the |
801 Furthermore a network driver usually manages a frame transmission queue. If |
802 network stack. Furthermore a network driver usually manages a frame |
802 this gets filled up, the network stack has to be told to stop passing further |
803 transmission queue. If this gets filled up, the network stack has to be told |
803 frames for a while. This happens with a call to \textit{netif\_stop\_queue()}. |
804 to stop passing further frames for a while. This happens with a call to |
804 If some frames have been sent, and there is enough space again to queue new |
805 \lstinline+netif_stop_queue()+. If some frames have been sent, and there is |
805 frames, this can be notified with \textit{netif\_wake\_queue()}. Another |
806 enough space again to queue new frames, this can be notified with |
806 important call is \textit{netif\_receive\_skb()}\footnote{This function is part |
807 \lstinline+netif_wake_queue()+. Another important call is |
807 of the NAPI (``New API''), that replaces the ``old'' kernel 2.4 technique for |
808 \lstinline+netif_receive_skb()+\footnote{This function is part of the NAPI |
808 interfacing to the network stack (with \textit{netif\_rx()}). NAPI is a |
809 (``New API''), that replaces the kernel 2.4 technique for interfacing to the |
809 technique to improve network performance on Linux. Read more in |
810 network stack (with \lstinline+netif_rx()+). NAPI is a technique to improve |
810 \url{http://www.cyberus.ca/~hadi/usenix-paper.tgz}}: It passes a frame to the |
811 network performance on Linux. Read more in |
811 network stack, that was just received by the device. Frame data has to be |
812 \url{http://www.cyberus.ca/~hadi/usenix-paper.tgz}.}: It passes a frame to the |
|
813 network stack, that was just received by the device. Frame data has to be |
812 packed into a so-called ``socket buffer'' for that (see below). |
814 packed into a so-called ``socket buffer'' for that (see below). |
813 |
815 |
814 \paragraph{Socket Buffers} |
816 \paragraph{Socket Buffers} |
815 \index{Socket buffer} |
817 \index{Socket buffer} |
816 |
818 |
817 Socket buffers are the basic data type for the whole network stack. |
819 Socket buffers are the basic data type for the whole network stack. They |
818 They serve as containers for network data and are able to quickly add |
820 serve as containers for network data and are able to quickly add data headers |
819 data headers and footers, or strip them off again. Therefore a socket |
821 and footers, or strip them off again. Therefore a socket buffer consists of an |
820 buffer consists of an allocated buffer and several pointers that mark |
822 allocated buffer and several pointers that mark beginning of the buffer |
821 beginning of the buffer (\textit{head}), beginning of data |
823 (\textit{head}), beginning of data (\textit{data}), end of data |
822 (\textit{data}), end of data (\textit{tail}) and end of buffer |
824 (\textit{tail}) and end of buffer (\textit{end}). In addition, a socket buffer |
823 (\textit{end}). In addition, a socket buffer holds network header |
825 holds network header information and (in case of received data) a pointer to |
824 information and (in case of received data) a pointer to the |
826 the \textit{net\_device}, it was received on. There exist functions that |
825 \textit{net\_device}, it was received on. There exist functions that |
827 create a socket buffer (\lstinline+dev_alloc_skb()+), add data either from |
826 create a socket buffer (\textit{dev\_alloc\_skb()}), add data either |
828 front (\lstinline+skb_push()+) or back (\lstinline+skb_put()+), remove data |
827 from front (\textit{skb\_push()}) or back (\textit{skb\_put()}), |
829 from front (\lstinline+skb_pull()+) or back (\lstinline+skb_trim()+), or |
828 remove data from front (\textit{skb\_pull()}) or back |
830 delete the buffer (\lstinline+kfree_skb()+). A socket buffer is passed from |
829 (\textit{skb\_trim()}), or delete the buffer (\textit{kfree\_skb()}). |
831 layer to layer, and is freed by the layer that uses it the last time. In case |
830 A socket buffer is passed from layer to layer, and is freed by the |
832 of sending, freeing has to be done by the network driver. |
831 layer that uses it the last time. In case of sending, freeing has to |
|
832 be done by the network driver. |
|
833 |
833 |
834 %------------------------------------------------------------------------------ |
834 %------------------------------------------------------------------------------ |
835 |
835 |
836 \section{EtherCAT Device Drivers} |
836 \section{EtherCAT Device Drivers} |
837 \label{sec:requirements} |
837 \label{sec:requirements} |
838 |
838 |
839 There are a few requirements for Ethernet network devices to function |
839 There are a few requirements for Ethernet network devices to function as |
840 as EtherCAT devices, when connected to an EtherCAT bus. |
840 EtherCAT devices, when connected to an EtherCAT bus. |
841 |
841 |
842 \paragraph{Dedicated Interfaces} |
842 \paragraph{Dedicated Interfaces} |
843 |
843 |
844 For performance and realtime purposes, the EtherCAT master needs |
844 For performance and realtime purposes, the EtherCAT master needs direct and |
845 direct and exclusive access to the Ethernet hardware. This implies |
845 exclusive access to the Ethernet hardware. This implies that the network |
846 that the network device must not be connected to the kernel's network |
846 device must not be connected to the kernel's network stack as usual, because |
847 stack as usual, because the kernel would try to use it as an ordinary |
847 the kernel would try to use it as an ordinary Ethernet device. |
848 Ethernet device. |
|
849 |
848 |
850 \paragraph{Interrupt-less Operation} |
849 \paragraph{Interrupt-less Operation} |
851 \index{Interrupt} |
850 \index{Interrupt} |
852 |
851 |
853 EtherCAT frames travel through the logical EtherCAT ring and are then |
852 EtherCAT frames travel through the logical EtherCAT ring and are then sent |
854 sent back to the master. Communication is highly deterministic: A |
853 back to the master. Communication is highly deterministic: A frame is sent and |
855 frame is sent and will be received again after a constant time. |
854 will be received again after a constant time. Therefore, there is no need to |
856 Therefore, there is no need to notify the driver about frame |
855 notify the driver about frame reception: The master can instead query the |
857 reception: The master can instead query the hardware for received |
856 hardware for received frames. |
858 frames. |
857 |
859 |
858 Figure~\ref{fig:interrupt} shows two workflows for cyclic frame transmission |
860 Figure~\ref{fig:interrupt} shows two workflows for cyclic frame |
859 and reception with and without interrupts. |
861 transmission and reception with and without interrupts. |
|
862 |
860 |
863 \begin{figure}[htbp] |
861 \begin{figure}[htbp] |
864 \centering |
862 \centering |
865 \includegraphics[width=.8\textwidth]{images/interrupt} |
863 \includegraphics[width=.8\textwidth]{images/interrupt} |
866 \caption{Interrupt Operation versus Interrupt-less Operation} |
864 \caption{Interrupt Operation versus Interrupt-less Operation} |
867 \label{fig:interrupt} |
865 \label{fig:interrupt} |
868 \end{figure} |
866 \end{figure} |
869 |
867 |
870 In the left workflow ``Interrupt Operation'', the data from the last |
868 In the left workflow ``Interrupt Operation'', the data from the last cycle is |
871 cycle is first processed and a new frame is assembled with new |
869 first processed and a new frame is assembled with new datagrams, which is then |
872 datagrams, which is then sent. The cyclic work is done for now. |
870 sent. The cyclic work is done for now. Later, when the frame is received |
873 Later, when the frame is received again by the hardware, an interrupt |
871 again by the hardware, an interrupt is triggered and the ISR is executed. The |
874 is triggered and the ISR is executed. The ISR will fetch the frame |
872 ISR will fetch the frame data from the hardware and initiate the frame |
875 data from the hardware and initiate the frame dissection: The |
873 dissection: The datagrams will be processed, so that the data is ready for |
876 datagrams will be processed, so that the data is ready for processing |
874 processing in the next cycle. |
877 in the next cycle. |
875 |
878 |
876 In the right workflow ``Interrupt-less Operation'', there is no hardware |
879 In the right workflow ``Interrupt-less Operation'', there is no |
877 interrupt enabled. Instead, the hardware will be polled by the master by |
880 hardware interrupt enabled. Instead, the hardware will be polled by |
878 executing the ISR. If the frame has been received in the meantime, it will be |
881 the master by executing the ISR. If the frame has been received in the |
879 dissected. The situation is now the same as at the beginning of the left |
882 meantime, it will be dissected. The situation is now the same as at |
880 workflow: The received data is processed and a new frame is assembled and |
883 the beginning of the left workflow: The received data is processed and |
881 sent. There is nothing to do for the rest of the cycle. |
884 a new frame is assembled and sent. There is nothing to do for the rest |
882 |
885 of the cycle. |
883 The interrupt-less operation is desirable, because there is simply no need for |
886 |
884 an interrupt. Moreover hardware interrupts are not conducive in improving the |
887 The interrupt-less operation is desirable, because there is simply no |
885 driver's realtime behaviour: Their indeterministic incidences contribute to |
888 need for an interrupt. Moreover hardware interrupts are not conducive |
886 increasing the jitter. Besides, if a realtime extension (like RTAI) is used, |
889 in improving the driver's realtime behaviour: Their indeterministic |
887 some additional effort would have to be made to prioritize interrupts. |
890 incidences contribute to increasing the jitter. Besides, if a realtime |
|
891 extension (like RTAI) is used, some additional effort would have to be |
|
892 made to prioritize interrupts. |
|
893 |
888 |
894 \paragraph{Ethernet and EtherCAT Devices} |
889 \paragraph{Ethernet and EtherCAT Devices} |
895 |
890 |
896 Another issue lies in the way Linux handles devices of the same type. |
891 Another issue lies in the way Linux handles devices of the same type. For |
897 For example, a PCI\nomenclature{PCI}{Peripheral Component |
892 example, a PCI\nomenclature{PCI}{Peripheral Component Interconnect, Computer |
898 Interconnect, Computer Bus} driver scans the PCI bus for devices it |
893 Bus} driver scans the PCI bus for devices it can handle. Then it registers |
899 can handle. Then it registers itself as the responsible driver for all |
894 itself as the responsible driver for all of the devices found. The problem is, |
900 of the devices found. The problem is, that an unmodified driver can |
895 that an unmodified driver can not be told to ignore a device because it will |
901 not be told to ignore a device because it will be used for EtherCAT |
896 be used for EtherCAT later. There must be a way to handle multiple devices of |
902 later. There must be a way to handle multiple devices of the same |
897 the same type, where one is reserved for EtherCAT, while the other is treated |
903 type, where one is reserved for EtherCAT, while the other is treated |
|
904 as an ordinary Ethernet device. |
898 as an ordinary Ethernet device. |
905 |
899 |
906 For all this reasons, the author has decided that the only acceptable |
900 For all this reasons, the author decided that the only acceptable solution is |
907 solution is to modify standard Ethernet drivers in a way that they |
901 to modify standard Ethernet drivers in a way that they keep their normal |
908 keep their normal functionality, but gain the ability to treat one or |
902 functionality, but gain the ability to treat one or more of the devices as |
909 more of the devices as EtherCAT-capable. |
903 EtherCAT-capable. |
910 |
904 |
911 Below are the advantages of this solution: |
905 Below are the advantages of this solution: |
912 |
906 |
913 \begin{itemize} |
907 \begin{itemize} |
914 \item No need to tell the standard drivers to ignore certain devices. |
908 \item No need to tell the standard drivers to ignore certain devices. |