Download PDF
Home

1. Installing the ONL and RBFS

The ONL Operating System Installation and upgrade guide provides the steps to install and configure the ONL OS along with Rtbrick Full Stack (RBFS) on Wedge-100BF based on Tofino chipset and AS-5916 based on Qumran chipset.

1.1. Preparing to Install the RtBrick Full Stack (RBFS)

All components of a RBFS run in a Linux container. You must install Linux containers based on Ubuntu 18.04 and other software like python 2.7 library before you install the RtBrick Full Stack (RBFS).

1.2. Installing on the Wedge 100-BF Box with Tofino Chipset

The wedge 100BF switch is a Tofino chipset based bare metal switch. It contains a console port and a management ethernet port for manageability. It contains two main boards that manages the whole system. The two boards are the BMC CPU module and the main CPU module.

1.2.1. BMC CPU module

This is the baseboard management controller CPU board. This CPU board runs the OpenBMC software that performs the following management functions:

  • Reset orchestration

  • PSU management

  • Voltage management

  • Temperature monitoring and fan control

1.2.2. Main CPU module

This is the main CPU board that can run the barefoot SDE natively or integrated with a Network Operating System (NOS). This board also can contain Open Network Installer Environment (ONIE) for automatic provisioning of systems. The supported NOS is either Open Network Linux (ONL) or Ubuntu. RBFS runs on a custom Open Network Linux (ONL) OS.

1.2.3. Management Access

On initial startup of the Wedge 100BF, connect to the BMC CPU through the CONSOLE port on the front panel.

Note Console settings are 9600/8-N-1(9600 baud, 8 data bits, no parity, and 1 stop bit) Once logged in to the BMC CPU, you can connect to the main CPU by means of the sol.sh script.
onl connecting console port
Figure 1. Connecting to the Console Port

Figure 1 shows the initial console connection. The credentials for logging in are: root/0penBmc, as shown in Figure 2.

onl logging running sol
Figure 2. Logging in and Running the sol.sh script.

While attached to the console port, you can send a break in order to exit to the main board prompt using the following key combination: CTRL + L + Backspace for a Microsoft keyboard and CTRL + L + DEL for a MAC system. This break usage is shown in Figure 3.

onl exit main board prompt
Figure 3. Exiting to the Main Board Prompt

After you’ve made the network settings to connect both CPUs to your management network, you can SSH into each CPU through the MGMT Ethernet port on the front panel.

Important Because there are two boards, both of them accessible through the same management port, make sure that they are assigned different IP addresses to avoid IP address “collisions” and access issues. If you are using DHCP, you can use the hardware address to control the address allocation. This method is explained in the provisioning steps below.
Note The management network cannot use the 192.168.0.0/24 subnet. This is because there is an internal network between the BMC CPU and the main CPU that uses IP addresses 192.168.0.1 and 192.168.0.2. Because the BMC CPU uses a non-persistent file system, its subnet cannot be changed without a recompilation.

Normally, by default, the main CPU bootloader (GRUB) is loaded with the Open Network Installer Environment (ONIE)

1.3. ONIE

The Open Network Installer Environment (ONIE) is an open source install environment that runs on a minimal Linux/Busybox environment. This environment allows end-users to install a supported network OS to provision and provide the box with the desired networking functions. Currently, a variety of network OSes are supported. To load the RtBrick Full Stack (RBFS), we provide a custom version of Open Network Linux (ONL) OS which contains the RBFS preloaded. When the device is booted up initially, the GRUB bootloader will contain various options as shown in Figure 4 below.

onl grub options
Figure 4. The GRUB Options

In most cases, the default option of the GRUB bootloader is to launch the Install OS mode for the ONIE environment. The device bootup can be watched once you are connected to the console port of the box. Once in the ONIE environment, the device starts up a variety of services.

The main services of concern are:

DHCP Service: The DHCP service performs two major tasks. First, if a DHCP server is already running on the network that the management port is connected to, DHCP sends out the DHCP client requests on the management port to retrieve and configure the management interface connectivity details. Second, as part of the request, DHCP provides a set of information that can be used by the DHCP server to provide a path to a network OS. If this information is provided by the DHCP server, then the system proceeds to download the network OS image and install the OS automatically. The Zero Touch Provisioning (ZTP) service uses this DHCP function.

SSH Service: The SSH service provides secure connectivity to the box through the management interface, as long as the management interface is configured with a reachable IP address on the network. The user name is root and there is no password.

When the box comes up in ONIE mode, the prompt shown in Figure 5 is displayed when you login through the console port or using SSH.

onl onie mode prompt
Figure 5. The ONIE Mode Prompt
Tip If the box does not automatically boot into ONIE mode, you can go to the BIOS and change the boot order.

1.4. Installing ONIE

If your Barefoot Networks Wedge 100B system did not come pre-installed with the desired ONIE boot loader, follow the steps in this section to install or re-install ONIE on the Wedge 100B main CPU.

1.5. Create the ONIE Bootable Image

Download the ONIE image from the Barefoot Networks support portal, or get it from your Barefoot Networks account representative.

Convert the ISO into a bootable image and load it onto a USB device. Use the commands for Linux or MAC-Based hosts to create the bootable USB.

On Linux-based hosts:
  1. Find the partition on which the USB drive is mounted. For example, /dev/sdb1.

  2. Flash the image onto the USB device:

    sudo dd if=./onie-recovery-x86_64-accton_wedge100bf_65x-r0.iso of=/dev/sdb1 +
On MAC-based hosts:
  1. Find the partition on which the USB drive is mounted. For example, /dev/sdb1.

  2. Convert the ISO into a bootable image:

    hdiutil convert -format UDRW -o onie-recovery-x86_64-accton_wedge100bf_65x- r0.img onie-recovery-x86_64-accton_wedge100bf_65x-r0.iso
  3. Flash the image onto the USB device:

    sudo dd if=./onie-recovery-x86_64-accton_wedge100bf_65x-r0.img.dmg of=/dev/disk2 bs=1m

1.6. Boot the Main CPU from the ONIE Image

  1. Connect a console session to the BMC CPU on the Barefoot Wedge 100B. For instructions, see the section on Installing on the Wedge 100-BF Box with Tofino Chipset.

  2. Insert your bootable USB drive into the USB port on the Wedge 100B front panel.

  3. Reboot the Wedge 100B system. At the BMC prompt, type the following. After the restart, you must run the sol.sh script, which starts a terminal session on the Wedge 100B main CPU:

    root@bmc:~# wedge_power.sh reset;
    sol.sh Power reset microserver ... Done
  4. Press the <ESC> or <DEL> key to enter the Aptio Setup Utility. In the Boot menu, set Boot mode select to LEGACY and set Boot Option #1 to USB Device.

  5. Press F4 to save and exit. (If you’re logged in from a MacOS machine using default settings, you may need to press fn + F4).

  6. When you reach the ONIE boot screen under GRUB, select ONIE EMBED MODE, which installs ONIE.

  7. When the ONIE installation is done, remove the USB device before the switch reboots.

  8. When the ONIE prompt appears (user is root, no password is required), type the following:

    ONIE:/ # onie-discovery-stop
    Stopping: discover... done.
  9. Verify that the ONIE installation was successful by checking the ONIE log file:

    ONIE:/ # tail /var/log/onie.log

1.7. Installing on the AS5916-54XK Switch Based on the Qumran Chipset

The AS5916-54XK switch is a 48*10G/1GbE SFP+ with 6*100G/40G QSFP28 fixed ports. All 10G ports and 100G uplink ports are placed on the front panel as default, but the console/management ports might be on the rear panel due the front panel space constraints.

By default, the switch comes preloaded with Open Network Installer Environment (ONIE). It contains a console port, a management ethernet port for manageability.

2. RBFS Customized ONL Image

Open Network Linux (ONL) is a Debian-based open source OS that is supported by various bare-metal switch vendors as a network OS. It has an active community that adds support for the various current and upcoming bare-metal switches.

ONL on its own cannot create network capabilities for its underlying hardware. Because of this lack, ONL is often customized for networking. RtBrick follows the same model: we preload our full stack LXC image into the ONL image. As soon as this custom ONL image is installed, the bare-metal switch is ready for installation of the RBFS.

The ONL image has the username/password combination of root/onl.

2.1. Installation of the ONL Image

The ONL image can be installed on the box using one of three methods:

  • Manual installation

  • Auto-provisioning using DHCP

  • Auto-provisioning using ZTP

Each of these methods is detailed in the following sections.

2.2. Manual Installation

To install the ONL image manually, you have to make sure that the box is turned on and running the ONIE environment (this is the default behavior). You also must make sure that the management interface has a reachable IP address. You can either use a DHCP server, or assign an IP address manually.

2.2.1. DHCP Server Provides the Management IP Address

You can use a DHCP server on the network connected to the management ethernet port to assign an IP address from a configured range. If you do not want a random IP address to be assigned, you can configure the DHCP server to provide an IP address based on the management MAC address.

Here is a sample DHCP configuration:

    ddns-update-style none
default-lease-time 7200;
    max-lease-time 7200;
authoritative;
    log-facility local7;

# This config assigns a fixed IP address to the device with the mac address provided
    host leaf1 {
        hardware ethernet
    cc:37:ab:e0:a3:ed; fixed-address
    172.16.240.6;
    }
# This config provides a pool of IP addresses to assign to devices that requests them via DHCP
    subnet 172.16.240.0 netmask 255.255.255.0 {
        range 172.16.240.10 172.16.240.20;
        option routers 172.16.240.1;
        option domain-name-servers 8.8.8.8;
    }

2.2.2. Manual Configuration of the Management IP Address

If you have connected to the box through the consoles port, you can manually configure the IP address on the management interface.

2.2.3. Manual Installation Continued

Once you have a valid management IP address for the box, do the following:

  1. Copy the RBFS ONL image to the box using SCP (username and password are root/(no password).

  2. Login to box using the console port of SSH.

  3. Use this command to start the installation of the ONL image with RBFS:

    onie-nos-install <path to the image>

The installation procedure takes about 5 to 10 minutes. The installation process can be observed on the console port.

2.3. Auto-Provisioning Using DHCP

The second way to install the ONL image on the box is by using DHCP to auto-provision the image.

In addition to having DHCP automatically or manually supply a management IP address, you must also have an HTTP/FTP/TFTP server on the management network.

You load the ONL image onto the HTTP/FTP/TFTP server so that the box can access the image installer and download it. An example URL for this could be:

tftp://172.16.240.2/rtbrick/rtbrick-onl-leaf-installer

You must configure the DHCP server with the bootfile-name option specific for this box using the MAC address. A sample configuration is provided below. Note that this configuration in an extension to the DHCP configuration (see the option bootfile-name) used to provide the management IP address.

    ddns-update-style
none default-lease-
time 7200;
    max-lease-time 7200;
authoritative;
log-facility local7;

# This config assigns a fixed IP address to the device with the mac address provided
    host leaf1 {
        hardware ethernet
    cc:37:ab:e0:a3:ed; fixed-
    address 172.16.240.6;
    # This option provides the path of the image to the ONIE, which inturn downloads the image and installs it automatically
    option bootfile-name
"tftp://172.16.240.2/rtbrick/ONL_TOFINO_ZTP_LEAF_INSTALLER";
    }
    # This config provides a pool of IP addresses to assign to devices that requests them via DHCP
    subnet 172.16.240.0 netmask 255.255.255.0 {
        range 172.16.240.10 172.16.240.20;
        option routers 172.16.240.1;
        option domain-name-servers 8.8.8.8;
}

When the DHCP server responds to the request from the box with the bootfile-name option, it triggers ONIE to automatically download and install the ONL image. The installation procedure takes about 5 to 10 minutes. The installation process can be observed on the console port.

2.4. Auto-Provisioning Using ZTP

Auto-provisioning using Zero Touch Provisioning (ZTP) is described in a separate document.

3. Verification

The installation process modifies the GRUB bootloader to load the installed ONL OS by default.

After successful installation, the box automatically reboots and loads the ONL OS. If there is any failure in the installation, the box enters the ONIE environment after the reboot.

Subsequent reboots of the box also bring up the ONL OS by default because the GRUB bootloader has been modified to load ONL by default after a successful installation. After a successful installation, you can login to the ONL OS using the username/password combination root/onl.

The RBFS stack runs inside an LXC container preloaded into the ONL image. Once the ONL OS comes up, the LXC container is auto-started (this also happens on reboots). The container takes a while to load and bring up the BCM device drivers.

Once all the initialization is complete, which takes about 4-5 minutes, you can see the container up and running using the lxc-ls -f command.

The name of the preloaded RBFS container is rtbrick. The username and password for the logging into the container is ubuntu/ubuntu.

After you have logged into the container, you can check the list of physical interfaces using command rtb fwdd show interfaces physical.

The command output shows whether the drivers have been loaded correctly or not (their Admin Status is up). If they have loaded correctly, this means that the installation and bring-up of the RBFS ONL image is successful.

Figure 5 shows a successful installation of the ONL with RBFS.

onl successful installation rbfs
Figure 6. Successful Installation of RBFS

4. Uninstallation

The ONL OS can be uninstalled manually using the ONIE shell. To do this, login to the box via the console port and reboot the box.

The box comes back up with the GRUB bootloader page, which provides option to go to the ONL OS as well as the ONIE environment.

Select ONIE to uninstall the network OS (ONL). This option is shown in Figure 6.

onl uninstallation
Figure 7. ONIE for Uninstallation of RBFS

Then select the Uninstall OS option from the next menu. This triggers the uninstallation of the ONL OS environment and resets the state of the box to its default environment (usually ONIE). This is shown in Figure 7.

onl uninstall os option
Figure 8. The Uninstall OS Option

Once the uninstallation is completed, the box will reboot and come up in ONIE mode.


©Copyright 2020 RtBrick, Inc. All rights reserved. The information contained herein is subject to change without notice. The trademarks, logos and service marks ("Marks") displayed in this documentation are the property of RtBrick in the United States and other countries. Use of the Marks are subject to RtBrick’s Term of Use Policy, available at https://www.rtbrick.com/privacy. Use of marks belonging to other parties is for informational purposes only.