JNOS Commands Manual - Appendix D

Appendix D

APPENDIX D

Of PACLEN, MTU, MSS, and More

Note: This appendix is taken from the JNOS40 Config Manaual
written by Johan Reinalda and William Thompson c. 1994.


Setting Bufsize, Paclen, Maxframe, MTU, MSS and Window

Many NOS users are confused by these parameters and do not
know how to set them properly. This appendix will first review
these parameters and then discuss how to choose values for them.
Special emphasis is given to avoiding interoperability problems
that may appear when communicating with non-Nos implementations
of AX.25.

1. AX25 Parameters

1.1. Paclen

Paclen limits the size of the data field in an AX.25 I-frame. This
value does not include the AX.25 protocol header (source,
destination, digipeater addresses, control and pid bytes).
The AX.25 V2 protocol specifies a maximum of 256 bytes for the
paclen. Be aware that some AX.25 implementations can not handle
paclen greater then this, however NOS derived systems can handle
this situation. This may cause interoperability problems.
Even Nos may have trouble with certain KISS TNCs because of
fixed-size buffers. The original KISS TNC code for the TNC-2
by K3MC can handle frames limited in size only by the RAM in
the TNC, but some other KISS TNCs cannot. Since unconnected-mode
(datagram) AX.25 uses UI frames, the paclen value has no effect in
unconnected mode. IE.  if you run IP in Datagram mode, you can
still have an MTU > Paclen ! (As long as your receive buffers
can handle the mtu size; see INTERFACE BUFFERS...)

In JNOS40, and JNOS v1.05 (and greater), paclen can be set on
a per interface basis with the 'ifconfig  paclen ###'
command. Thus you can have a paclen of 256 on one interface and
a smaller or larger on other interfaces. When you first attach an
interface, the paclen defaults to the value of the 'ax25 paclen'
setting (this defaults to 256.) If you want to carry netrom
data over ax.25 links, the system needs to make sure the paclen
is large enough to handle the netrom MTU sized data. Net/rom
mtu is a maximum of 236; with a 20 byte header for the routing,
this gives a data size of 256 to be send with ax.25 packets.

In most versions of NOS.EXE, you can get problems when you use
netrom and set the paclen < 256. When the ax.25 layer gets to send
netrom frames, it cannot handle the whole data in one packet. It
will then split it up in several smaller frames, using the AX.25
Version 2.1 fragmentation scheme. However, TheNet, Net/Rom, G8BPQ,
MSYS and other nodes CAN NOT handle this type of fragmentation,
resulting in impossible connections ! However, if you are running
JNOS40, or JNOS v1.05 (or greater) for the PC, you need not worry
about this. These 2 version of NOS have been modified to never
provide more then paclen sized netrom data to the ax.25 layer! 
Thus the above problem will never happen...

1.2. Maxframe

This parameter controls the number of I-frames that may be
send on an AX.25 connection before it must stop and wait for
an acknowledgement. Since the AX.25/LAPB sequence number field
is 3 bits wide, this number cannot be larger than 7. It can be
shown that the optimal value is 1.

Since unconnected-mode (datagram) AX.25 uses UI frames that
do not have sequence numbers, this parameter does not apply to
unconnected mode.

2. IP and TCP Parameters

2.1. MTU

The MTU (Maximum Transmission Unit) is an interface parameter
that limits the size of the largest IP datagram that it may
handle. The MTU is the sum of the size of the data inside the
IP header (TCP of UDP most likely), and the size of the IP
header itself. IP datagrams routed to an interface that are
larger than its MTU are each split into two or more IP fragments.
Each fragment has its own IP header and is handled by the network
as if it were a distinct IP datagram, but when it arrives at the
destination it is held by the IP layer until all of the other
fragments belonging to the original datagram have arrived. Then
they are reassembled back into the complete, original IP datagram.
The minimum acceptable interface MTU is 28 bytes: 20 bytes for the
IP (fragment) header, plus 8 bytes of data. There is no default
MTU in Nos; it must be explicitly specified for each interface
as part of the 'attach' command. This is not the case for the
netrom interface. Since ip data is carried inside a 'regular'
netrom frame, the ip data should never be larger then the maximum
data size the netrom protocol can handle. The default mtu here
is 236, which is the protocol maximum.

If you plan on running IP in Datagram mode (the default), you can
have an MTU that is larger then the paclen for that interface.
However, you should still be aware of the constraints of the
receiver buffer size! (See INTERFACE BUFFERS)

If you plan on using IP in Virtual Connect, or VC, mode, you
should be aware of the following. If you set the IP MTU larger
then paclen for the interface, you will hand a packet to the
ax.25 layer that is larger then what it can send in one packet.
Thus you will get AX.25 V2.1 fragmentation. If you are exchanging
ip data with NOS based stations only, you have no problems (other
then the fragmentation).

If you are interfacing with stations other then NOS bases systems,
you again will have troubles, like with netrom. They will most
likely not be able to handle the packets correctly. Thus be aware
that in this case you should set the mtu to equal the paclen,
to avoid these problems.

2.2. MSS

MSS (Maximum Segment Size) is a TCP-level parameter that limits
the amount of data that the remote TCP will send in a single TCP
packet. MSS values are exchanged in the SYN (connection request)
packets that open a TCP connection. In the Nos implementation
of TCP, the MSS actually used by TCP is further reduced in order
to avoid fragmentation at the local IP interface. That is,
the local TCP asks IP for the MTU of the interface that will be
used to reach the destination. It then subtracts 40 from the
MTU value to allow for the overhead of the TCP (20 bytes) and IP
(20 bytes) headers. If the result is less than the MSS received
from the remote TCP, it is used instead.

2.3. Window

This is a TCP-level parameter that controls how much data the
local TCP will allow the remote TCP to send before it must stop
and wait for an acknowledgment. The actual window value used by
TCP when deciding how much more data to send is referred to
as the effective window. This is the smaller of two values: the
window advertised by the remote TCP minus the unacknowledged data
in flight, and the congestion window, an automatically computed
time-varying estimate of how much data the network can handle.

2.4. Discussion

2.4.1. IP Fragmentation vs. AX.25 Segmentation

IP-level fragmentation often makes it possible to interconnect
two dissimilar networks, but it is best avoided whenever possible.
One reason is that when a single IP fragment is lost, all other
fragments belonging to the same datagram are effectively
also lost and the entire datagram must be retransmitted by
the source. Even without loss, fragments require the allocation
of temporary buffer memory at the destination, and it is never
easy to decide how long to wait for missing fragments before
giving up and discarding those that have already arrived.
A reassembly timer controls this process. In Nos it is
(re)initialized with the 'ip rtimer' parameter (default 30
seconds) whenever progress is made in reassembling a datagram
(i.e., a new fragment is received). It is not necessary that
all of the fragments belonging to a datagram arrive within a
single time-out interval, only that the interval between fragments
be less than the time-out.

Most commercial sub networks that carry IP have MTUs of 576 or
more, so interconnecting them with sub networks having smaller
values can result in considerable fragmentation. For this reason,
IP implementors working with links or subnets having unusually
small packet size limits are encouraged to use transparent 
fragmentation, that is, to devise schemes to break up large
IP datagrams into a sequence of link or subnet frames that are
immediately reassembled on the other end of the link or subnet
into the original, whole IP datagram without the use of IP-level
fragmentation.

Such a scheme is provided in AX.25 Version 2.1. It can break a
large IP or NET/ROM datagram into a series of paclen-sized AX.25
segments (not to be confused with TCP segments), one per AX.25
I-frame, for transmission. Subsequently, it can reassemble 
them into a single datagram at the other end of the link before 
handing it up to the IP or NET/ROM module.

Unfortunately, the segmentation procedure is a new feature in
AX.25 and is not yet widely implemented; in fact, Nos and derived
software, is so far the only known implementation. For regular 
NOS systems this can create some interoperability problems between 
Nos and non-Nos nodes. However, JNOS40 and JNOS 1.05 (or later) 
have provisions built in to avoid these problems. This problem is 
discussed further in the section on setting the MTU.

2.4.2. Setting MTU

TCP/IP header overhead considerations similar to those of the AX.25
layer when setting paclen apply when choosing an MTU. However,
certain sub network types supported by Nos have well-established
MTUs, and these should always be used unless you know what
you're doing: 1500 bytes for Ethernet, and 508 bytes for ARCNET.
The MTU for PPP is automatically negotiated, and defaults to 1500.
Other subnet types, including SLIP and AX.25, are not as well
standardized.

SLIP has no official MTU, but the most common implementation
(for BSD UNIX) uses an MTU of 1006 bytes. Although Nos has no
hard wired limit on the size of a received SLIP frame, this is
not true for all other systems. Interoperability problems may
therefore result if larger MTUs are used in Nos.

Choosing an MTU for an AX.25 interface is more complex. When the
interface operates in datagram (UI-frame) mode, the paclen
parameter does not apply. The MTU effectively becomes the paclen
of the link. However, when using AX.25 CONNECTIONS to send IP
packets, data will be automatically segmented into I-frames no
larger than paclen bytes. Unfortunately, as also mentioned
earlier, Nos is so far the only known implementation of the new
AX.25 segmentation procedure. Thus, if you are exchanging IP over
connections (i.e. VC mode) with none-NOS based systems, it might be
needed to avoid AX.25 segmentation. Thus you should make sure that
packets larger than paclen are never handed to AX.25. In order
to assure this, you should not set the MTU greater then the paclen.

On the other hand, if you do not send IP over connections, but in
datagram mode, you can use a larger MTU. Here, you are limited
by the receive buffer size (and the tolerance of other network
users for your large packets and proportionally greater bandwidth
share !).

For connections carrying IP between NOS systems (i.e. NOS-NOS VC
mode), you can let AX.25 segmentation do it's thing. If you choose
an MTU on the order of 1000-1500 bytes, you can largely avoid
IP-level fragmentation and reduce TCP/IP-level header overhead
on file transfers to a very low level. And you are still free to
pick whatever paclen value is appropriate for the link.

2.4.5. Setting MSS

The setting of this TCP-level parameter is somewhat less critical
than the IP and AX.25 level parameters already discussed,
mainly because it is automatically lowered according to the MTU
of the local interface when a connection is created. Although 
this is, strictly speaking, a protocol layering violation (TCP is 
not supposed to have any knowledge of the workings of lower layers) 
this technique does work well in practice. However, it can be fooled;
for example, if a routing change occurs after the connection has
been opened and the new local interface has a smaller MTU than
the previous one, IP fragmentation may occur in the local system.

The only drawback to setting a large MSS is that it might cause
avoidable fragmentation at some other point within the network
path if it includes a "bottleneck" subnet with an MTU smaller
than that of the local interface. (Unfortunately, there is
presently no way to know when this is the case.

There is ongoing work within the Internet Engineering Task Force
on a "MTU Discovery" procedure to determine the largest datagram
that may be sent over a given path without fragmentation, but it
is not yet complete.) Also, since the MSS you specify is sent
to the remote system, and not all other TCPs do the MSS-lowering
procedure yet, this might cause the remote system to generate
IP fragments unnecessarily.

On the other hand, a too-small MSS can result in a considerable
performance loss, especially when operating over fast LANs and
networks that can handle larger packets. So the best value for
MSS is probably 40 less than the largest MTU on your system,
with the 40-byte margin allowing for the TCP and IP headers. For
example, if you have a SLIP interface with a 1006 byte MTU and
an Ethernet interface with a 1500 byte MTU, set MSS to 1460
bytes. This allows you to receive maximum-sized Ethernet packets,
assuming the path to your system does not have any bottleneck
subnets with smaller MTUs.

2.4.6. Setting Window

A sliding window protocol like TCP cannot transfer more than
one window's worth of data per round trip time interval. So
this TCP-level parameter controls the ability of the remote TCP
to keep a long "pipe" full. That is, when operating over a path
with many hops, offering a large TCP window will help keep all
those hops busy when you're receiving data. On the other hand,
offering too large a window can congest the network if it cannot
buffer all that data. Fortunately, new algorithms for dynamic
controlling the effective TCP flow control window have been
developed over the past few years and are now widely deployed.
Nos includes them, and you can watch them in action with the
'tcp status ' or 'socket <#>' commands. Look at the cwind
(congestion window) value.

In most cases it is safe to set the TCP window to a small integer
multiple of the MSS, (e.g. 4times), or larger if necessary
to fully utilize a high bandwidth*delay product path. One thing
to keep in mind, however, is that advertising a certain TCP 
window value declares that the system has that much buffer 
space available for incoming data. Nos does not actually 
pre-allocate this space; it keeps it in a common pool and may well
overbook" it, exploiting the fact that many TCP connections
are idle for long periods and gambling that most applications
will read incoming data from an active connection as soon as it
arrives, thereby quickly freeing the buffer memory. However,
it is possible to run Nos out of memory if excessive TCP window
sizes are advertised and either the applications go to sleep
indefinitely (e.g. suspended Telnet sessions) or a lot of
out-of-sequence data arrives. It is wise to keep an eye on the
amount of available memory and to decrease the TCP window size (or
limit the number of simultaneous connections) if it gets too low.

Depending on the channel access method and link level protocol,
the use of a window setting that exceeds the MSS may cause an
increase in channel collisions. In particular, collisions between
data packets and returning acknowledgments during a bulk file
transfer may become common. Although this is, strictly peaking,
not TCP's fault, it is possible to work around the problem at
the TCP level by decreasing the window so that the protocol
operates in stop-and-wait mode. This is done by making the
window value equal to the MSS.

2.5. Summary

In most cases, the default values provided by JNOS40 for each
of these parameters will work correctly and give reasonable
performance. Only in special circumstances such as operation over
a very poor link or experimentation with high speed modems should
it be necessary to change them.



Interface Buffers

There are two different types of buffers associated with attaching
interfaces. The first type is the ring buffer or fifo (first in,
first out) that is used when attaching the serial port. It is
used for receiving only. This buffer is allocated just once,
and is used throughout the life of the interface. The asynchronous
(or serial port) receiver interrupt code puts characters in this
buffer in a circular fashion. When the end of the buffer is 
reached, the next character is stored at the beginning and 
continues through the buffer again.

The receiver process for the serial interface reads the characters
from this fifo-buffer into memory buffers, or mbufs, that are
used internally to handle and pass around data. Setting the size
of the fifo buffer is an empirical process. A good place to
start is to set it to twice the size of the packet length used
over the serial port. Once you have the interface running, you
can monitor the usage of the fifo buffer with the 'asy' command.
This will show you the 'buf hi' value which is the same as 'sw hi'
for PC based NOS. 'buf hi' is the highest value of the number of
characters that were waiting in the fifo buffer to be read by the
receiver process. If 'buf hi' is close to the fifo buffer size, or
if the 'asy' command shows buffer overflows ('buf over' for JNOS40
code), you should increase the buffer size. If however the number
is significantly smaller, you could decrease the buffer size.

The second type of interface buffer is also a receiver buffer.
However, this type of buffer is allocated, then released as
needed. This buffer almost always is allocated during interrupt
service routines, i.e., when the interrupts are off! In order
to keep the service routine short, the buffer is allocated from
a special 'interrupt buffer queue' because a regular memory
allocation would take far too long.

The two internal modem drivers use this second type of buffer.
Since one interrupt buffer pool services all the drivers that need
buffers during interrupt, the size and number of these buffers are
quite critical to a system's 'well-being'. A buffer acquired from
the interrupt buffer pool needs to be large enough to handle the
largest packet received from any of the internal modem interfaces.

A good rule to estimate the size of the interrupt buffers needed
is as follows:

Set 'memory ibufsize' to the (largest + 'extra') of:

1 - the largest ax.25 paclen parameter of the internal modem
interfaces, OR

2 - the largest ip mtu parameter of the internal modem interfaces.

    PLUS 'extra' which accounts for the longest possible
    ax.25 header (source, destination, control and digipeaters). 
    Use 80 bytes for' extra' as a general rule.

The MTU is important because on ax.25 interfaces where the MTU
is larger than the paclen, there is a possibility of receiving
larger ip frames when IP traffic is carried in Datagram mode.
This possibility does not exist if IP traffic is carried in
Virtual Connect (VC) mode, since the data will be split in several
paclen-sized ax.25 packets. (This is AX.25 V2.1 fragmentation at
work for you!) When using VC mode you can ignore the MTU values
when finding the ibuf size.

The above discussion assumes that paclen and mtu values are
coordinated between all users in your area packet network. If this
is not the case, someone else might send packets larger then what
your system can handle. Such packets will cause receive buffer
overflow and will be dumped! If you have an ax.25 interface with
paclen of 256, mtu of 256, and another with paclen of 256 and mtu
of 512, you should set the 'memory ibufsize' to at least 512 +
80 ! JNOS40 ibufs default to 600 bytes, and should suffice for
paclen's or mtu's up to 512. NOS.EXE has a default ibufsize
of 2k (ie 2048 bytes), wich suffices for the standard Ethernet
MTU of 1500 bytes. Determining the number of buffers needed
is another empirical process. Start with, say, ten buffers (ie
'memory nibufs 10'). If you get a lot of memory ibuffails in the
'mem stat' display, you should increase the number of buffers.

NOTE: if you are not using one or more of the internal modems or
any drivers that require interrupt buffers, there is no need to
keep them around. In that case, simply set 'mem nibuf 0'.


End of Appendix D