Introduction

RtBrick Full Stack (RBFS) supports various types of interfaces, including physical and logical interfaces. On hardware platforms, RBFS physical interfaces represent the ports of a switch. This guide describes how to configure and verify RBFS interfaces. Features like routing protocols or access services will typically run on top of the interfaces.

Interface Types

Physical Interfaces

In RBFS, physical interfaces (IFP) typically represent the physical ports of a hardware switch. For example, ifp-0/0/1 represents switch port 1. On the physical interface level, you can configure various parameters associated with Layer 1 of the ISO/OSI reference model.

Logical Interfaces

For each physical interface, you can create one or multiple interface units also referred to as logical interfaces (IFL) in RBFS. A logical interface is associated with the Layer 2 operation. In addition, you can configure Layer 3 parameters like IP addresses on interface units, and assign interface units to routing instances.

Loopback Interfaces

A loopback interface is typically used to represent and identify a device itself. Loopback interfaces are preferred because they do not depend on the status of a physical port, and will always be up. Please note, although loopback interfaces are virtual interfaces, there are also represented as physical interfaces and interface units in RBFS, reflecting Layer 1 and Layer 2/3 operation.

Host Interfaces

Linux virtual ethernet (veth) interfaces connect an LXC container with the Linux host OS. In RBFS, a veth interface to the Linux bridge lxcbr0 is created by default. In virtual topologies, you can create additional veth interfaces and Linux bridges. RBFS host interfaces represent veth interfaces in RBFS.
For example, if the container interface eth1 connects to the host interface vethXYZ123, ifp-0/0/1 can be bound to eth1 to represent it in RBFS. Host interfaces can be used like any other physical interface.

Memory Interfaces

Memory interfaces (memif) are virtual interfaces used for creating virtual topologies. They connect multiple containers running RBFS to each other. When configuring memif interfaces:

  • Endpoints match on the memif ID, i.e. the memif ID needs to be the same on both ends.

  • memif IDs need to be unique on the host.

  • The memif interface name is locally significant only.

  • One endpoint needs to be configured as a master, while the other one is configured as a slave.

Interface Numbering

RBFS interface numbers match the port numbers on the switch faceplate. An interface is named in the ifp-<chassis-ID>/<front-panel-block-number>/<port> format. For example, ifp-0/0/1.

  • Chassis ID—always 0 for the currently supported platforms

  • Front Panel Block—represents group of ports on the faceplate

  • Port—matches the port number on the switch faceplate

Virtual interfaces follow the same structure, for example, lo-0/0/1 or memif-0/0/1.

Logical interfaces are numbered: ifl-<Node ID>/<Chip ID>/<Port ID>/<Unit ID>, for example ifl-0/0/1/1.

Community Support for Interfaces

You can tag an interface address with a community or extended community. RBFS will create a direct route for each interface address. If a community or extended community is configured for an interface address, RBFS will add it to the direct route. Communities can be used in policies. For example, when redistributing direct routes, you can match these communities and define desired policy rules.

Unnumbered Interfaces

An unnumbered interface is a point-to-point interface that is not explicitly configured with a dedicated IP address and subnet. Instead, it borrows (or links to) from a loopback interface, and uses it as the source IP address for packets originating from the interface. The IP unnumbered interface can "borrow" the IP address from another interface that is already configured on the switch, thereby conserving network and address space.

When configuring IP unnumbered on an interface:

  • The IP address of the numbered interface cannot be borrowed.

  • IP addresses can be borrowed from a loopback interface by logical interfaces but they cannot be borrowed and vice versa.

Path MTU Discovery

The Path MTU Discovery technique determines the maximum transmission unit between two hosts so that IP fragmentation can be avoided. By default, path MTU discovery is enabled in RBFS.

When RBFS receives MTU-violated packets, it will respond with the following ICMP error message:

ICMPv4 Message Types

The type field identifies the type of the message sent by the host. The type field contains more specific information about the error condition.

The table below lists the ICMPv4 message types.

Type Description

3

Destination Unreachable.
This alerts a source host of delivery problems encountered while trying to reach the destination.

Destination Unreachable uses the following code values to further describe the function of the ICMP message being sent.

Code Description

4

Fragmentation Needed and Don’t-Fragment (DF) was Set.
This message occurs when a router receives a packet that requires fragmentation, but the router has the DF DF flag turned on.

ICMPv6 Message Types

The type field identifies the type of the message sent by the host. The type field contains more specific information about the error condition.

The table below lists the ICMPv6 message type.

Type Description

2

Packet Too Big.
A Packet Too Big MUST be sent by a router in response to a packet that it cannot forward because the packet is larger than the MTU of the outgoing link.

Code Description

0

No code

You can change this behaviour by enabling fragmentation. For more information about enabling hostpath fragmentation, see the section "Enabling Hostpath Fragmentation" below.

All outgoing packets are validated against the configured MTU on the egress path.

  • If MTU is violated and MTU-profile action is drop, then packets are dropped in hardware.

  • If MTU is violated and MTU-profile action is redirect-to-cpu, a 20MB policer is used to protect the CPU-port from overwhelming MTU-violated traffic, and packets are sent to the CPU port.

  • When fragmentation is enabled, one of the following operations takes place.

    • If the DF bit is not set in the received packet (only for IPv4), the packets are fragmented and sent to the outgoing port.

    • If the DF bit is set in a packet, it drops the packet and sends an ICMP error message back to the source.

  • When fragmentation is disabled, packets are dropped and ICMP error messages are sent to the source.

For information about configuring the MTU profile, see [mtu-profile-config].

IP Fragmentation

If the maximum transmission unit (MTU) of an outgoing interface is less than the original packet which needs to be routed, the packet needs to be fragmented.

RBFS supports IP fragmentation on the QMX and QAX platforms but not on the Q2C platform. However, currently, there is no support for IP fragmentation in the QMX, QAX, or Q2C hardware. Due to this limitation, on the QMX and QAX platforms, the packets are sent to the CPU, and the fragmentation is handled by the CPU therefore the rate for these packets is significantly reduced.

If the packet that needs to be fragmented and the Do-Not-Fragment (DF) bit is specified, then the device is going to send an ICMP Error code "fragmentation needed and DF set" to the source.

By default, IPv6 fragmentation is handled at source. When the transit device receives an MTU-violated packet, it sends a "Packet Too Big" ICMPv6 message that it cannot forward the packet because it is larger than the MTU of the outgoing link.

Guidelines and Limitations of IP Fragmentation

The following guidelines and limitations apply to IP Fragmentation:

  • If a packet is larger than the negotiated subscriber MTU size, it will be fragmented (on the QMX platform); whereas on the Q2C platform, such a packet will be dropped. You can control the fragmentation on the Q2C platform by configuring the set forwarding-options fragmentation ipv4 state CPU command. For more information about configuring fragmentation, see "2.2.2. Enabling Hostpath Fragmentation."

  • The fragmented packets do not go over the regular QoS path in the egress pipeline.

  • There will be no ICMP error message sent in response to MTU-violated multicast packets.

MTU Profile

The Maximum Transmission Unit (MTU) is the size of the packet that is allowed in the network. In the new generation silicon like Broadcom Qumran2C (Q2C), resources are conserved by creating profiles of the resources, and multiple entities like IFP, IFL and L3 interfaces utilize these profiles. To better manage MTU resources and platform capabilities, RBFS supports configuring MTU profiles and attaching these profiles to the attachment points.

Attachment Points

The MTU profiles are attached to the interface entities like physical (IFP), logical (IFL) and L3 interfaces. RBFS supports the below attachment points for MTU profiles:

  • Port-level

  • L3 interface level (IPv4 and IPv6)

  • PPPoE subscriber level (L2 IFL)

MTU Size

A user-configured MTU size can range from 64 to 9216 in RBFS.

For MTU profiles of type "pppoe", users should provide L3 MTU size (IPv4/IPv6 headers).
MTU Type

An MTU type specifies the attachment point of the MTU profile. The MTU types supported are as follows:

  • physical: When checking MTU, the entire packet size is considered.

  • ipv4: MTU check is based on IPv4 headers.

  • ipv6: MTU check is based on IPv6 headers.

  • ip: MTU profile of type IP.

  • pppoe: The MTU profile is applied to the PPPoE subscriber interface and the user is required to provide the L3 MTU size. Based on its best match algorithm, the Subscriber Management service associates these profiles with PPPoE subscribers.

  • MTU profiles for L3 logical interfaces must be explicitly configured, and if not configured, no default MTU size is set for IPv4 and IPv6.

  • On the Q2C platform, L3 interfaces can only be configured with IPv4 MTU profile or IPv6 MTU profile, but not both. However, with the type "ip" MTU profile, you can configure MTU for both IPv4 and IPv6 traffic with a common MTU size.

  • On the QAX platform, only physical MTU profiles are supported.

MTU Action

The MTU action defines the action to be taken when the MTU check fails. Currently, RBFS supports “drop” as an action.

MTU Profile Limitations

The following limitations apply to the MTU profile:

  • There is a limit to how many MTUs can be used by each hardware.

    • On the Q2C platform, the limit is as follows:

      • Maximum number of MTU profiles: 8

      • Maximum number of L3 MTU profiles: 3 (MTU type: IP/IPv4/IPv6)

      • Maximum number of PPPoE MTU profiles: 6 (including the default PPPoE profile)

      • Maximum number of physical MTU profiles: 7

    • On the QAX platform, the limit is as follows:

      • Maximum number of physical MTU profiles: 1

Supported Platforms

Not all features are necessarily supported on each hardware platform. Refer to the Platform Guide for the features and the sub-features that are or are not supported by each platform.

Guidelines & Limitations

QAX-based Platforms

An additional restriction applies to ports on QAX-based platforms: because of hardware design, physical ports are grouped into quads (groups of 4, also known as port groups). Each quad must have the same physical parameters: speed, link-training, duplex.

The following tables are provided for easy identification of ports that need to have the same physical settings:

Edgecore 7316-26XB Port Groups:

Port Speed Duplex Port Group

ifp-0/0/0

100G

Full

0

ifp-0/0/1

100G

Full

1

ifp-0/1/0

10G

Full

2

ifp-0/1/1

10G

Full

ifp-0/1/2

10G

Full

ifp-0/1/3

10G

Full

ifp-0/1/4

10G

Full

3

ifp-0/1/5

10G

Full

ifp-0/1/6

10G

Full

ifp-0/1/7

10G

Full

ifp-0/1/8

10G

Full

4

ifp-0/1/9

10G

Full

ifp-0/1/10

10G

Full

ifp-0/1/11

10G

Full

ifp-0/1/12

10G

Full

5

ifp-0/1/13

10G

Full

ifp-0/1/14

10G

Full

ifp-0/1/15

10G

Full

ifp-0/1/16

25G

Full

6

ifp-0/1/17

25G

Full

ifp-0/1/18

25G

Full

ifp-0/1/19

25G

Full

ifp-0/1/20

25G

Full

7

ifp-0/1/21

25G

Full

ifp-0/1/22

25G

Full

ifp-0/1/23

25G

Full

UfiSpace S9500-22XST Port Groups:

Port Speed Duplex Port Group

ifp-0/0/0

10G

Full

8

ifp-0/0/1

10G

Full

ifp-0/0/2

10G

Full

ifp-0/0/3

10G

Full

ifp-0/0/4

10G

Full

4

ifp-0/0/5

10G

Full

ifp-0/0/6

10G

Full

ifp-0/0/7

10G

Full

ifp-0/0/8

10G

Full

11

ifp-0/0/9

10G

Full

ifp-0/0/10

10G

Full

ifp-0/0/11

10G

Full

ifp-0/0/12

25G

Full

3

ifp-0/0/13

25G

Full

ifp-0/0/14

25G

Full

ifp-0/0/15

25G

Full

ifp-0/0/16

25G

Full

2

ifp-0/0/17

25G

Full

ifp-0/0/18

25G

Full

ifp-0/0/19

25G

Full

ifp-0/0/20

100G

Full

0

ifp-0/0/21

100G

Full

1

A PHY Quad can be associated with network interface (NIF) ports of identical type only. For example, a quad cannot be a mix of XLGE and XE ports. An exception is GE and XE ports which can coexist in the same quad. This means all the ports in a port group should have the same physical interface configuration (that is, speed/duplex/link-training). Ports in a port group are only allowed to support 1G and 10G speeds; any other combination is not allowed. If a port within a port group is misconfigured, then it would require changing the speeds/interface type of all ports within the port group to a different type and then back into the original type.