Configuring µC/TCP-IP (Part 1)

TCP/IP communication involves a somewhat complex collection of protocols, and configuring a TCP/IP stack may be challenging if you are not familiar with these protocols. This blog post is the first in a two-part series in which we will go over some basic principles on how to configure Micrium's µC/TCP-IP stack along with some reasoning behind the settings. The information in the two posts is geared primarily towards those using TCP sockets and not UDP. However, much of the content in the first post is applicable to all projects based on µC/TCP-IP, regardless of the underlying transport protocol.

To make the most of this series of posts, you should keep the needs and objectives of your project in mind. The following will assist in navigating the two posts.

  • Do you need to optimize your system to save on memory or streamline performance? The descriptions of net_dev_cfg.c and net_cfg.h in the first post give an informative look at some key µC/TCP-IP configuration parameters.
  • Is your application locking up after receiving several packets? The first post's content on net_dev_cfg.c explains how the stack utilizes buffers to process packets.
  • Are you expecting high levels of traffic in your network? The Stack Window Size section in the second post can help with getting the best performance out of your system.
  • Are you trying to set up a server that can service multiple clients? The Sockets Configuration section in Part 2 will help get you started.
  • Are you looking to monitor and analyze the network traffic handled by your system? The TCP/IP Analysis Tools section in the second post will help.

Configuring µC/TCP-IP typically involves making changes to at least two files: net_dev_cfg.c and net_cfg.h. We examine key definitions in these files below.

net_dev_cfg.c (Device-Level Configuration)

The most important fields here are the number of large and small buffers and the number of descriptors. If these are not given adequate values, they will cause issues within your application. These configurations are the source of most problems in µC/TCP-IP-based projects, as they have a direct impact on the memory usage and performance of the system.

const NET_DEV_ETHER_MCU  NetDev_Cfg_MCU_Ether0 = {
        NET_IF_MEM_TYPE_MAIN,      /* Receive Buffer Memory Pool Type             */
        1536u,                     /* Large Rx Buffer Size (in Octets)            */
        12u,                       /* Number of Large Rx Buffers                  */
        32u,                       /* Alignment of Receive Buffers (in Octets)    */
        0u,                        /* Desired offset from base receive…           */   
                                   /* …index (if needed)                          */
        NET_IF_MEM_TYPE_MAIN,      /* Transmit Buffer Memory Pool Type            */
        1536u,                     /* Large Tx Buffer Size (in Octets)            */
        10u,                       /* Number of Large Tx Buffers                  */
        60u,                       /* Small Tx Buffer Size (in Octets)            */
        10u,                       /* Number of Small Tx Buffers                  */ 
        32u,                       /* Alignment of Transmit Buffers (in Octets)   */ 
        0u,                        /* Offset from base transmit index (if needed) */ 
        0x00000000u,               /* Base Address of Dedicated Memory…           */
                                   /* (if available)                              */
        0u,                        /* Size of dedicated memory in octets…         */
                                   /* …(if needed)                                */
        5u,                        /* Number of device receive descriptors        */
        10u,                       /* Number of device transmit descriptors       */

A partial example of the network interface configuration struct normally declared in net_dev_cfg.c is shown above. Details on several commonly modified fields in this structure are provided below.

Large Rx Buffer Size: When configuring the size of your large receive buffers in µC/TCP-IP, you should take into account the driver you are using in your application and the required alignment of the buffers in your device. In the above example, the large receive buffers are set to a size of 1536. This follows from the maximum Ethernet frame size of 1518. Since the buffers for the example device must be aligned to 32 bytes and 1518 is not an even factor of 32, this value is bumped up to the nearest evenly aligned 32-bit value, 1536. Under most circumstances, the size of the large receive buffers should not be less than 1518 so that each buffer can handle the largest allowable Ethernet frame.

Number of Large Rx Buffers: You must have at least one defined buffer. Ideally, you would set the number of large receive buffers in order to avoid scenarios in which the stack consumes every buffer available. This prevents the application from creating a deadlock or prematurely closing sockets due to dropped/missed TCP packets. The optimal setting for a system depends on a number of factors and requires some trial by the developer to find that "golden" value that works best. More on this later.

Large Tx Buffer Size: To maximize throughput, this should be configured as the maximum packet size (plus any additional space needed to meet the aforementioned alignment requirements). To reduce RAM usage, you can reduce the buffers' size but the performance will also be affected (usually negatively). Ultimately, the performance impact of any change in the size of the large transmit buffers is determined by the profile of the network traffic generated by your application. If your code mainly sends data in small chunks, a decrease in the buffer size shouldn't substantially reduce performance. On the other hand, if your application is written to perform transmit operations involving relatively large amounts of data, a decrease in the buffer size could have a substantial impact on the throughput that you observe.

Number of Large Tx Buffers: As with the large receive buffers, the optimal number of large transmit buffers depends on several factors. Thus, in your system, you may need to experiment a bit in order to find the best value for this configuration setting. In all cases, the total number of Tx buffers (large and small combined) must be at least one.

Small Tx Buffer Size: This buffer size value should be configured equal to NET_BUF_DATA_SIZE_MIN. This includes any additional bytes that may be required to make this value properly aligned with your devices pre-set alignment. If configured larger than this, a deadlock situation could occur. Generally, small Tx buffers are utilized for transmission of packets without a data payload like TCP, ACK and RST segments. The small buffers are not recommended for use with data payload transmission by the application. In order to avoid this we recommend setting them to a size where only "payload-less" packets can utilize them. This may vary depending on your interface's size requirements but can be found in the hardware reference manual if not already known.

Number of Small Tx Buffers: Ideally, there should be one small buffer for every two to four sockets being used. If performance is very important more small buffers can improve this, although the exact gains will differ according to the transmit behavior of the application.

Number of Rx Descriptors: This value should be between 50% - 70% of the interface's number of receive buffers.

Number of Tx Descriptors: This should be equal to the total of all large and small transmit buffers configured for the interface.

Size of Typical TCP Buffer

Size of Typical TCP Buffer

Descriptor Chain with some descriptors currently processing buffers and one empty descriptor.

Descriptor chain with some descriptors currently processing buffers and one empty descriptor

Descriptors manage buffers. They indicate which buffers are available to receive incoming packets. When a buffer pointed to by a descriptor is loaded with a packet it is then passed to the software for processing, which is when the own bit is SW owned.

Descriptors manage buffers. They indicate which buffers are available to receive incoming packets. When a buffer pointed to by a descriptor is loaded with a packet it is then passed to the software for processing, which is when the own bit is SW owned.

Own Bit used to indicated if the descriptor is currently owned by the hardware or the software

Own Bit used to indicated if the descriptor is currently owned by the hardware or the software

Here Buffer 1 and 2 received incoming packets. They are then passed onto the software to be processed. Buffer 3 is still HW owned and awaiting a packet. While Buffer 1 and 2 are processed by the SW the descriptors can point to other empty buffers to keep the queue going. This is one example why it's important to have  more buffers than descriptors in your system.

Here Buffer 1 and 2 received incoming packets. They are then passed onto the software to be processed. Buffer 3 is still HW owned and awaiting a packet. While Buffer 1 and 2 are processed by the SW the descriptors can point to other empty buffers to keep the queue going. This is one example why it's important to have more buffers than descriptors in your system.

This shows how buffers 4 and 5 are being queued up by the descriptors for incoming packets. Buffer 1 and 2 continue to process while the descriptors continue queuing up the next available buffers. Once Buffer 1 and 2 finish they will be available once again for new incoming packets.

This shows how buffers 4 and 5 are being queued up by the descriptors for incoming packets. Buffer 1 and 2 continue to process while the descriptors continue queuing up the next available buffers. Once Buffer 1 and 2 finish they will be available once again for new incoming packets.

The above images illustrate how the receive buffers and descriptors that are configured via net_dev_cfg.c interact within the stack. If there are not enough free buffers to associate with descriptors while packets are processed, performance can suffer. This is the reason for the earlier recommendation to keep the number of receive descriptors at 50% to 70% of the number of buffers.

More documentation on the configurable parameters in net_dev_cfg.c can be found on our website: Network Interface Configuration

Your settings in net_dev_cfg.c will influence how you should configure the defines in net_cfg.h. We consider important settings from the latter file below.

net_cfg.h: (Higher-Level Configuration)

NET_CFG_IF_RX_Q_SIZE: You should define this constant to reflect the total number of receive buffers in your application across all devices. If you have one interface this should be the same as the number of receive buffers defined in net_dev_cfg.c for that interface. If you have more than one, it should be the same as the total number of receive buffers on all devices.

NET_CFG_IF_TX_DEALLOC_Q_SIZE: The value of this constant should equal the total number of large and small transmit buffers defined across ALL interfaces in the application. If you have one interface in your application then this should be the sum of that interface's large and small transmit buffers. If you have two or more this should be the total of the buffers on each instantiated interface.

NET_IF_CFG_MAX_NBR_IF: This should equal the number of interface devices you plan to implement in your project. You do NOT always need to define this many interface structures in net_dev_cfg.c since, in some cases, you can use the same interface definition for multiple instances.

NET_SOCK_CFG_SOCK_NBR_TCP: This configuration determines how many TCP sockets can be created in the system. Every call to NetSock_Open() requires a socket from the corresponding TCP or UDP define and can be used to determine how many total sockets are expected to be created at any given time. Closed sockets are again free to be used with another NetSock_Open() call.

NET_SOCK_CFG_SOCK_NBR_UDP: This configures how many UDP sockets can be created in the system. The same principle applies for defining this as for the TCP configuration except that the UDP version should account for NetSock_Open() calls to only UDP sockets.

NET_IPv4_CFG_IF_MAX_NBR_ADDR: This parameter allows multiple IP addresses to be configured to one interface. It is useful for systems which require multiple connections but do not have the resources to assign each to a different hardware device. This is the IPv4 configuration.

NET_IPv6_CFG_IF_MAX_NBR_ADDR: This is the same as the IPv4 configuration above but for IPv6.

NET_SOCK_CFG_CONN_ACCEPT_Q_SIZE_MAX: This definition establishes the maximum size of the connection queue for each socket. This is used when remote devices try to open up multiple connections and several of them need to be queued while the initial request is processed. When configuring a listen socket you must specify the connection queue size. The size indicates how many connections can be placed in the socket accept queue. This can be used to limit how many buffers a socket can consume to prevent sockets from using up all available buffers.

NET_SOCK_CFG_RX_Q_SIZE_OCTET: This represents the number of bytes that can be queued up by a socket when receiving. This value should not exceed the amount of space available in the system's buffers. This value will need to be fine-tuned for each individual application for the best performance possible. It can be modified for each socket at run time but cannot exceed this default value. Note that this define acts as a load balancer between network applications. At a maximum value, high bandwidth applications would consume all available buffers preventing other applications from receiving anything. Setting this configuration is the equivalent of using SO_RECVBUF in sockets.

NET_SOCK_CFG_TX_Q_SIZE_OCTET: This number represents how many bytes can be queued up by a socket for transmission. Similar to the RX configuration, it is a load balancer for UDP connections. µC/TCP-IP’s transmission function will report back how many bytes were transmitted at the end of a transmit call. If the data to be sent exceeds the buffer size, the stack will inform the application that xx Bytes out of yy Bytes were sent and another call to transmit must be done to finish the transmission. This configuration is the equivalent of using SO_SNDBUF in BSD sockets.

Optional: These definitions are considered "advanced," meaning that they are normally commented. If you'd like to modify them, then you should simply remove the commenting.

NET_TCP_DFLT_RX_WIN_SIZE_OCTET: This defines the window size used in reception. More configuration details are provided in the second post of this series.

NET_TCP_DFLT_TX_WIN_SIZE_OCTET: This defines the window size used for transmission. More configuration details are provided in the second post of this series.

Conclusion

In this post, we covered some of the basics of configuration in µC/TCP-IP. We began at the driver level, reviewing configuration parameters contained in net_dev_cfg.c and examining how the buffers and descriptors whose allocation is controlled by this file interact. We then moved up to the levels of the stack configured via net_cfg.h and walked through a few of this file's key definitions. In the next post, we'll tackle slightly more advanced configuration topics, such as TCP window size, and we'll introduce a tool that can help µC/TCP-IP users monitor performance and confirm that the stack is configured as expected.

Tags: ,

Questions or Comments?

Have a question or a suggestion for a future article?
Don't hesitate to contact us and let us know!
All comments and ideas are welcome.