Taking advantage of virtualization to support multiple OSs and their related workloads on one hardware platform is becoming more and more the norm. As the number of virtualized workloads grows, it's becoming ever more important to understand the capabilities of virtual networking and how to configure virtual networking to help maximize your IT investment. I'll take a look at the components of virtual networking, how to configure virtual network support for multiple workloads, and steps that you can take to make your virtual network environment more robust.

Background

Let's start by looking at the components of virtual networking. For our purposes, we'll break down virtual networking into two broad areas: virtual Ethernet adapters and network configuration (including routing). Often virtual Ethernet adapters have been misunderstood to be a component of the IBM i (i5/OS) operating system—they are not. Virtual Ethernet adapters are a feature of the hypervisor, or firmware, of the overall hardware system. On POWER5- and POWER6-based systems, the adapters are configured through the Hardware Management Console (HMC) or the Integrated Virtualization Manager (IVM). Prior to POWER5 systems, the adapters were configured through the partition management function of system service tools.

Regardless of which partitioning tool is being used, the concept is the same. Imagine that the system has a rack of network switches (virtual, of course) and that through the configuration of the virtual Ethernet adapter you can virtually cable up a virtual Ethernet adapter to one of the switches—this is done through the virtual LAN ID portion of the adapter configuration. Pre-POWER5 systems have up to 16 of these virtual switches, whereas POWER5 and POWER6 systems have up to 4,094 virtual Ethernet switches. Now it's just standard TCP/IP configuration: Partitions with virtual Ethernet adapters configured with the same virtual LAN (VLAN) ID can communicate through a standard TCP/IP stack, as Figure 1 shows.

Communication between virtual Ethernet adapters on the same VLAN ID is accomplished through the transfer of packets across the memory bus of the system. This provides for fast, secure communications between partitions and can be useful when large amounts of data need to be transferred between workloads (OSs) on the same physical system. But what if you want to be able to access a workload from a point on the network that's external to the physical system? That is where bridging or routing the virtual LAN to a physical LAN becomes important.

Let's look next at different ways that the virtual LAN that is constructed through the definition of the virtual Ethernet adapters and can be bridged, or routed, to a physical network. These methods include proxy ARP, Network Address Translation (NAT), direct routing, and Integrated Virtual Ethernet (IVE).

Proxy ARP: The Basics

Proxy ARP, or transparent subnetting, is the creation of a smaller network whose range of addresses is totally contained within a larger network. To understand how proxy ARP works, one must first have a basic understanding of TCP/IP networking. Anyone who has been around computers for any length of time has likely encountered the need to review, or possibly set, the parameters of the network card on their system. There are three network parameters in particular that concern us:

• Network address: the unique address within the network assigned to the network adapter. The network adapter will respond to traffic when the network packet has a destination address that matches the network address assigned to the adapter.
• Subnet mask: indicates the size of the local network that the network card is attached to. Put another way, it is the size of the network that can be reached without having to route the traffic through an intervening router. Shortly I'll describe how the subnet mask and network address are used together to determine the range of address for the local network.
• Router (or gateway): indicates the address of the router that traffic destined for a network address outside of the local network should be sent to. The router is responsible for forwarding traffic to the appropriate target network by sending it on to other routers until the router for the destined network is finally reached.

Every network is represented by a range of addresses with a "network" address as the first address and a "broadcast" address as the last address. The range of addresses, sometimes referred to as the broadcast range, is determined by the adapter address and the broadcast address. As an example, if the TCP/IP address assigned to an adapter is 10.1.1.11 and the broadcast address is 255.255.255.0 (referred to as class C), then the range of usable (assignable) addresses is 10.1.1.1 through 10.1.1.254, where 10.1.1.0 is the network address and 10.1.1.255 is the broadcast range. When communicating with another network adapter, the destination address is checked to see whether it falls within the broadcast range. If it does, then the traffic is simply broadcast across the network—which means that all adapters on the network can see the traffic but only the adapter with the destination address will actually respond to the traffic. Traffic with a destination address not within the broadcast range of the source address will be routed to the address indicated as the router or gateway and from there will continue to be routed across different routers/gateways until the router on the destination network has received the traffic, at which point it will be broadcast across that network.

With this understanding as the backdrop, we can take a look at proxy ARP and how it can be used to construct a smaller network for the virtual Ethernet connections that can be routed across a larger physical network. In the case of virtual networking for logical partitioning, a subnet of addresses is created for the virtual Ethernet adapters configured for the LPARs. One of those virtual Ethernet adapters needs to be allocated to the IBM i partition that also has a connection on the real/physical network. Consider the diagram in Figure 2, which updates the diagram in Figure 1 to include the proxy ARP configuration information.

Let's look first at the two ranges of addresses, as Figure 3 shows. Recall from our earlier discussion that the range of addresses in a network is determined by the adapters network address and subnet mask.

What you should notice from the table in Figure 3 is that the range of addresses for the virtual network is wholly contained within the range of addresses for the network that the physical adapter is connected to. There are two additional rules that the subnet of addresses must adhere to:

• The size of the subnet, as indicated by the subnet mask, must be a power of two.
• The subnet must fall on what is referred to as a subnet boundary; that is, it must be possible to fit a whole number of subnets within a larger class C network.

A subnet calculator is usually a good tool to help ensure that the range of addresses selected meets these rules. A free subnet calculator that I like to use is the IP Subnet Calculator from WildPackets, which you can download.

Once proxy ARP has been correctly set up, traffic will flow between the virtual and physical networks through a relationship that will be created between the virtual and physical Ethernet adapters on the IBM i partition. Let's examine how the configuration is established.

On the IBM i partition, a virtual Ethernet adapter is configured with the following parameters:

• IP address in the subnet range
• subnet mask that indicates the size of the subnet
• associated local interface of the IP address of a physical adapter to perform the routing

In addition to these configuration settings, on the IBM i partition the TCP/IP attribute Datagram Forwarding must be set to *YES to allow traffic to flow between the two networks.

On the guest partition(s) for which traffic is to be routed, the virtual Ethernet adapter is configured with the following parameters:

• IP address in the subnet range
• subnet mask that indicates the size of the subnet
• gateway (or router) defined as the IP address of the IBM i partition on the virtual LAN

The last item is crucial as it is this item that establishes that traffic outside of the subnet is to be routed to the IBM i partition, which will then use the associated local interface to route the traffic to the physical network.

Consider the following performance implication with regard to bridging virtual networks across a physical network. Often, the virtual network adapters will be configured to support jumbo frame sizes (9000 bytes in the case of IBM i). This is great for virtual LAN communication (communication between multiple adapters on the virtual adapter), as it lets us pass data along at the gigabit speed that the adapters support.

But consider what will happen when that same virtual adapter is associated with a physical network adapter that's configured with the standard frame size of 1500 bytes—fragmentation! Essentially we are taking a large pipe of data from the virtual network and trying to funnel it through a smaller pipe—the network packet will be fragmented into multiple parts to fit into the smaller pipe. I've seen numerous customer environments that experience performance issues with the virtual LAN traffic that are fixed once the frame sizes of the virtual and physical network adapters are configured to match. General rule of thumb: When associating a virtual network adapter to a physical adapter, ensure that the frame size of the virtual adapter matches the frame size of the physical adapter.

Proxy ARP: Making It Fault Tolerant

We can take the proxy ARP configuration one step further and make it tolerant to outages on the physical network adapter. Consider the configuration that we just discussed. If the physical adapter at address 10.1.1.10 becomes unavailable (maybe someone tripped over the Ethernet cable), then external network traffic for all connections on the virtual LAN will be disrupted. Let's look at how we can minimize network outages for the virtual LAN.

Instead of creating the TCP/IP interface with a single associated local interface, we can create the IBM i virtual LAN interface as a virtual IP address (VIPA) through iSeries Navigator and configure it to proxy over more than one interface. Let's start by viewing the details of the properties of such an interface, as Figure 4 shows.

In this configuration, the address 10.1.1.33 is configured to be proxied across the 10.1.1.11 or 10.1.1.10 addresses. When the 10.1.1.33 address is defined, it is defined with the subnet mask indicating the size of the network just as we did in our earlier example. The difference is in the details. In this configuration, when the 10.1.1.33 interface is active, the 10.1.1.11 address will proxy (or respond) for that address. If for some reason the 10.1.1.11 interface becomes unavailable, the 10.1.1.10 address will immediately take over responding for the 10.1.1.12 address. And since the 10.1.1.33 interface's configuration has a subnet mask that indicates multiple addresses in the network, the physical interface will respond for all those addresses, and then the traffic will be broadcast on the virtual LAN. Let's examine how this will look in regard to virtual LANs, as Figure 5 shows.

There are two items to note here. First, this is not load balancing. The whole purpose of this configuration is to provide fault tolerance for the virtual network. Second, the order of addresses in the preferred interface list matters. The VIPA (or address on the virtual LAN, in our case) will be proxied across the first available address in the preferred interface list. Only if and when that interface becomes unavailable will a subsequent address in the preferred interface list take over. And, when a higher (or more preferred) address becomes available, traffic will immediately start to be proxied across that interface, as the list indicates it is a more preferred interface then the lower interface.

Network Address Translation

Another method to look at is Network Address Translation (NAT). With NAT, a set of private addresses is configured for the virtual Ethernet interfaces, and a set of public addresses for those interfaces is configured against the physical interface on the IBM i partition, as Figure 6 shows. Translation rules are configured, via iSeries Navigator, to map the public address to the private addresses and vice versa.

Notice that in this configuration the addresses assigned to the virtual LAN interfaces are truly private addresses. Also note that all the public addresses for the interfaces are assigned to the physical interface of the IBM i partition. From a performance perspective, NAT will tend not to perform as well as proxy ARP. This is because each network packet that comes across the physical interface will have to be inspected and modified to change the public address to the private address as the traffic is received from the physical network, and to change the private address to the public address as the traffic is transitioned from the virtual network back to the physical network.

The only way to configure NAT on IBM i is to do so via iSeries Navigator. All options for NAT can be found by selecting Network, IP Policies, Packet Rules. The packet rules will be used to define two addresses for each partition: a private address on the virtual LAN and a public address on the physical network. Additionally, the packet rules will be used to define translation rules between the private and public addresses.

Direct Routing

Another method of routing traffic between physical and virtual networks is through direct routing. With direct routing, routing statements are placed in the actual network routers. Static routes are defined manually, and the routes are advertised via RIP or other protocols. The routes point to the real adapter address (i.e., the IBM i interface on the physical network), and DNS entries point to the virtual Ethernet addresses. The route table is checked, and the network packet is forwarded to the destination. As with the other methods, all network traffic is routed through the routing (in our case IBM i) partition. The methods for configuration of direct routing depend on the actual router and are beyond the scope of this article.

Integrated Virtual Ethernet

So far, the methods I've outlined for network routing involve leveraging the network stack of the IBM i partition and can be implemented on any IBM i platform that supports virtual networking and logical partitioning. The last method I'll discuss is leveraging the Integrated Virtual Ethernet (IVE) capability of the POWER6 systems. The IVE is a physical network adapter that has built-in support for multiple logical ports per physical ports. The IVE can essentially be abstracted to represent multiple logical addresses per physical address—up to 16 logical ports per physical port.

There are three variants of the IVE: a dual-port 1GB, dual-port 10GB, and a quad-port 1GB Ethernet adapter that can abstract up to 32 logical ports on the dual-port cards and up to 64 logical ports on the quad-port cards. You configure the IVE through either the HMC or IVM. Once a logical port has been mapped to a logical port, it's configured in the same way as a physical adapter allocated to the partition would be configured. There's no need for additional configuration such as that explained earlier with proxy ARP and NAT. Also note that the IVE card cannot be changed; that is, the IVE is not a customer-installable feature, and the IVE ordered/shipped with the system can't be changed.

Take the Next Step

Virtualization features such as virtual networking enable the incorporation of additional workloads beyond those that one would expect to typically be able to support through physical hardware resources. I hope this article has helped shed some light on how you can leverage virtual networking in your environments.

Erwin Earley is a managing consultant at IBM who has worked with the Rochester, Minnesota, development lab since 1996. Erwin currently heads up the Open Community Center of Competency in the IBM i Technology Center. He has worked in the IT industry since 1980 and has experience with several UNIX variants as well as Linux and IBM i.