Power blades, the IBM BladeCenter JS12 and JS22, have increased the number of hardware platforms for IBM i applications. Using proven PowerVM technologies such as logical partitioning (LPAR) and dynamic resource movement, the JS12 and JS22 provide a novel approach to workload consolidation, efficiency, and power cost reduction for IT environments.
Here, I provide tips and techniques on making the BladeCenter solution work, drawing on experience from the front line of planning for, implementing, troubleshooting and helping design the next stage of IBM i on a Power blade. Together, we'll trace the life cycle of an implementation to help you navigate any challenging terrain.
The Planning Stage
This phase begins with the simple question, “Is IBM i on a Power blade right for this environment?” Alternatives to this solution are to deploy IBM i on a POWER6 processor-based IBM Power rack server, and to implement Windows applications on x86-architecture blade servers with an Internet SCSI (iSCSI) connection to IBM i on a rack server. Considerations include
- the number, type and size of IBM i servers (AS/400, iSeries, System i)
- the number and size of Windows servers
- operating system levels and plans to upgrade
- type of existing storage (integrated vs. SAN)
- existing skills (IBM i, Power hardware, Windows, SAN, AIX/VIOS)
- potential power and cooling savings with the BladeCenter
Almost all IBM i and BladeCenter implementations involve Windows integration on blade servers as well. Frequently, migrating IBM i to a Power blade is the final step of an ongoing BladeCenter consolidation project. The closest equivalent to running IBM i on a traditional Power server is the BladeCenter S because of its integrated SAS storage. This type of implementation is best suited to a small IT environment with several servers. For larger installations, including Storage Area Network (SAN) and a BladeCenter H is most appropriate. Keep in mind that in both cases, the necessary skill set includes non-IBM i storage management (BladeCenter S or SAN) as well as working with BladeCenter hardware and Virtual I/O Server (VIOS)-based virtualization.
Operating system and application levels. Running IBM i on a Power blade requires the Licensed Internal Code (LIC) and operating system to be at 6.1. IBM i 5.4, even with modification level 5 of the LIC, will not work. You must verify that all applications used are supported and have been tested to work on IBM i 6.1. Note that there is no special certification for IBM i on a Power blade; the only requirement is that applications run on IBM i 6.1 in general. You use the same IBM i LIC and OS code on Power blades as on IBM Power servers. If the existing OS level is not 6.1 already, you should investigate readiness for IBM i 6.1 program conversion.
The mandate to upgrade to IBM i 6.1 can present a problem for many clients. In that case, a good approach is to consider implementing the solution in two stages: the rest of the configuration first (typically Windows on x86-architecture blades), followed by IBM i on a Power blade. This method allows for IBM i 6.1 testing while you install and test the rest of the environment.
Blade server and BladeCenter hardware. The critical issue here is to address the hardware needs of the entire solution. As previously mentioned, the final configuration almost always involves other blade servers, most often x86-architecture blades running Windows. Each blade server requires the correct type of expansion adapter(s), dependent on the type of storage and network connectivity planned. Those adapters correspond to I/O modules in the BladeCenter. As of this writing, IBM i on a Power blade supports two expansion adapters: a CFFh combination Fibre Channel and Ethernet card, and a CFFv SAS card.
In the BladeCenter H, the CFFh expansion adapter uses the high-speed I/O fabric (I/O module bays 7–10) to provide additional 1 Gb Ethernet ports and access to Fibre Channel SAN. Plan for obtaining or migrating I/O modules and expansion adapters for the rest of the blades accordingly. The CFFh combination adapter is not supported in the BladeCenter S. The CFFv adapter uses I/O module bays 3 and 4 to provide access to the integrated SAS disks in the BladeCenter, SAS-attached external storage, and SAS-attached tape in the BladeCenter S. The CFFv adapter also provides access to SAS-attached external storage and SAS-attached tape in the BladeCenter H.
Because x86-architecture blades have a much broader selection of expansion adapters available, a BladeCenter configuration sufficient for the needs of all blade servers is certainly achievable by planning with the final configuration in mind. Note that the BladeCenter JS12 and JS22 servers support additional expansion adapters, but the CFFh and CFFv adapters are the only ones tested with and supported for IBM i with VIOS on a Power blade. Although several BladeCenter interoperability documents exist, use IBM i on a Power Blade Supported Environments for configurations involving IBM i.
Storage. The supported storage types for IBM i on a Power blade are SAS with BladeCenter S (integrated disks in the BladeCenter itself and DS3200), and SAS and Fibre Channel with BladeCenter H (DS3200 and several Fibre Channel SAN subsystems). It is not possible to use a Fibre Channel SAN subsystem for IBM i on BladeCenter S, because the CFFh combination Fibre Channel and Ethernet adapter is not supported in that BladeCenter. Although you can use a Fibre Channel SAN subsystem for x86-architecture blades, doing so involves replacing the SAS switches in the BladeCenter with Fibre Channel ones. Such a replacement renders the SAS disks in the BladeCenter itself inaccessible and diminishes the inherent value of BladeCenter S.
The choice of storage depends on the number of physical disk arms required, the existing or planned storage management skills of the site administrators and, to a lesser degree, the disk capacity needed. Capacity is typically much less of a factor than the number of disk arms, as is traditionally the case with production IBM i workloads. The storage options available for both BladeCenter S and BladeCenter H tend to provide sufficient capacity for the intended environment (small vs. medium).
The storage subsystems available to IBM i on a Power blade support both SAS and Serial ATA (SATA) physical disks (DS3200) or Fibre Channel and SATA disks (DS3400, DS4700, DS4800, DS5000). SATA drives are much slower (7,200 RPM vs. 15,000 RPM for supported SAS and Fibre Channel disks), and they tend to be slightly less reliable. For these reasons, IBM strongly recommends that you create logical units (LUNs) for production IBM i workloads using only SAS or Fibre Channel physical disks. It is also recommended that you isolate the LUNs for each IBM i LPAR on a separate RAID array in the storage subsystem; this is an IBM i–specific guideline that other workloads do not share.
These and other performance recommendations should determine the final hardware configuration of the storage subsystem(s) to use with IBM i and BladeCenter. For a detailed performance discussion of IBM i on a Power blade, consult the IBM Power Systems Performance Capability Reference manual.
The Implementation Stage
The main installation document for this solution is IBM i on a Power Blade Read-me First, which continues to be updated as new features become available. It attempts to describe in detail the IBM i–specific steps of the complete implementation, and it includes links to many other relevant documents. This section does not list all configuration steps, but instead provides tips to use with the Read-me First guide.
Hardware installation. A great deal of excellent documentation exists for installing a BladeCenter and all of its components, such as the Advanced Management Module (AMM) and I/O modules. Installing x86-architecture blade servers is also well documented. Provided that you carefully plan for the blade server expansion adapters and BladeCenter I/O modules, there are no differences in those installations because IBM i on a Power blade will also be implemented. Be sure that you always update all BladeCenter component firmware to the latest available level before installing any blade servers. Note that I/O modules from different manufacturers tend to have different update instructions, but those instructions are provided together with the update code.
Before installing the Power blade(s), install the correct expansion adapter(s), shipped separately. If you order a BladeCenter S with one or more Power blades with IBM i preinstalled, the CFFv SAS adapter(s) will already be installed. Both IBM BladeCenter JS12 and JS22 can be hot-plugged while a BladeCenter is running.
When you insert a Power blade into the BladeCenter chassis, the blade service processor (SP) starts, similar to the stage of connecting a rack IBM Power server to a power outlet. Power blades use a similar SP to that on rack IBM Power servers, which carries the same type of system firmware (including the Power Hypervisor). The SP communicates with the BladeCenter AMM, so that several minutes after Power blade installation, the BladeCenter's machine type and model, serial number, and SP firmware level are available in the AMM interface. Be sure to verify that any expansion adapters are also reporting correctly in the AMM.
Storage configuration. The key to configuring storage for IBM i on BladeCenter S or BladeCenter H is to use the IBM i–specific guidelines in the IBM Power Systems Performance Capability Reference manual to determine which configuration to create; then use documentation specific to the storage subsystem for instructions on how to create that configuration. For storage subsystems supported by the sizing tool Disk Magic, create IBM i–specific models to determine the correct number of physical disks and LUNs. You should configure standard open storage AIX/VIOS LUNs for IBM i, attached to the physical expansion adapter owned by VIOS on the Power blade. Once available in VIOS, entire LUNs or physical SAS disks should be virtualized to IBM i, without using storage pools (or volume groups) and logical volumes in VIOS. This is a unique recommendation for IBM i based on performance tests; for AIX and Linux clients of VIOS, you'll typically use the logical volume.
It is best to create the planned storage configuration before powering on the JS12 or JS22. Once the blade is powered on, VIOS will recognize all physical disks or LUNs attached to any expansion adapters on the blade. Many SAN administrators prefer to have the host server running when configuring the LUN attachment on the storage subsystem to enable the SAN fabric to detect the Fibre Channel port(s) on the server.
By recording the worldwide port names (WWPNs) of the Fibre Channel expansion adapter on the Power blade, you can configure the storage subsystem without powering on the blade (you'll find the WWPNs in the AMM). However, there is no harm in starting the blade so that VIOS boots and the Fibre Channel ports on the expansion adapter become active in the SAN fabric. When the LUN assignment on the storage subsystem is complete, VIOS will not autodetect the new disk resources if it is already running. Use the VIOS command 'cfgdev' with no parameters to make it scan for hardware changes and detect the new LUNs. You must use the same command every time a new disk for IBM i is physically attached to VIOS, or when the DVD drive in the BladeCenter is reassigned to a Power blade.
Software installation. All Power blades can be ordered with VIOS preinstalled on one of the embedded drives on the blade or, in the case of BladeCenter S, on one of the disks in the chassis. Furthermore, if a JS12 or JS22 is ordered together with a BladeCenter S and the correct feature codes are entered, IBM i can also be preinstalled on SAS disks in the BladeCenter S. In this case, powering on the blade will start both the VIOS and IBM i LPARs, similar to an IBM Power server preinstalled with IBM i. Regardless of the preinstall options you choose, both VIOS and IBM i install media will ship with the Power blade.
To verify whether VIOS has been installed on the blade, open a console to it and power it on from the AMM. The VIOS login prompt will appear after several minutes, or Open Firmware will report that no operating system has been detected. To check whether IBM i has been installed (assuming this is a BladeCenter S), configure networking in VIOS and use the Integrated Virtualization Manager (IVM) to check whether an IBM i LPAR exists. If it does, IBM manufacturing has also installed IBM i on disks in the BladeCenter S.
Although VIOS and IBM i may be preinstalled on a Power blade, further configuration is necessary before they can be used. The AMM provides two different methods-- Serial over LAN and Remote Control--to open a console to the blade and thus to VIOS.
You must use the VIOS command line to configure networking before IVM can be employed for LPAR and virtual resource management. It is not possible to open a console to an IBM i LPAR on a blade through the AMM; instead, use Operations Console (LAN), or LAN console. It is critical to understand that LAN console for IBM i on a blade works the same way as on a rack server. Networking for IBM i on the blade is different: It uses Virtual Ethernet adapters and a Virtual Ethernet Bridge in VIOS to gain access to the physical LAN through an I/O module in the BladeCenter. However, once you've configured networking for IBM i in IVM, the process of establishing a LAN console connection and the configuration on the PC client used for the LAN console are the same as for any other Power server.
There are two vital requirements: You must configure a unique IP address on the PC specifically for the LAN console (VIOS’ or any other existing IP address cannot be used), and the PC must be on the same subnet for the initial LAN console connection to be established. Once the LAN console is connected and the IP information is stored in the LIC, you can use the same IP address from any PC to establish a new LAN console connection (provided routing is available to the IBM i LPAR on the blade). After a console session to IBM i is available, you should configure a separate production network interface using a second Virtual Ethernet adapter.
If IBM i is not preinstalled on the blade, such as with a BladeCenter H, follow these general steps:
- Configure storage for IBM i and assign it to VIOS on the correct blade.
- Use IVM to create the IBM i LPAR, assigning storage, network, and optical resources to it.
- Configure the LAN console.
- Start the IBM i LPAR in IVM and perform the installation.
The simplest method of installing IBM i is to use the DVD drive in the BladeCenter (the DVD drive must be assigned to the correct blade). Remember to use 'cfgdev' in VIOS if the DVD drive was previously assigned to a different blade. The DVD drive is then allocated to the IBM i LPAR in the IVM, and a D-mode IPL is performed from the LIC install image. Alternatively, you can import all IBM i install images into the media library in VIOS, and then perform the installation from disk in VIOS.
Post-Installation Stage
Once you've completed the implementation, you'll need to update code or install fixes as part of regular maintenance. The IBM i on a Power blade solution includes the following components with their own firmware or software that may require updates and fixes:
- AMM
- Blade expansion adapters
- IBM i
- I/O modules
- SP on the Power blade
- VIOS
Although updating the AMM happens independently of the blade servers, the rest of these components cannot be updated concurrently while the IBM i LPAR is running without interrupting I/O operations. In the case of SP firmware, it is different from the SP on rack servers, which allows applying most firmware updates concurrently.
You update SP and expansion adapter firmware on the Power blade from the VIOS command line, not from the AMM or IBM i. VIOS updates are also performed on the command line and require a reboot of VIOS and, therefore, IBM i downtime. However, the actual update can be performed concurrently, while the reboot is delayed until a planned downtime opportunity is available. Applying IBM i PTFs is no different from the process on a rack server. The Advanced Systems Management Interface (ASMI) is available only to IBM Service. For further information on performing code updates, consult IBM i on a Power Blade Read-me First.
Ready to Go
With this information, you are well on your way to implementing the IBM i and BladeCenter solution in your environment. You'll find the benefits of blade servers well worth the installation effort.
Vess Natchev (vess@us.ibm.com) is an advisory software engineer with Lab-based Services in IBM’s Systems and Technology Group. He is a worldwide technical lead in the areas of Power System and storage virtualization and Power blades. His responsibilities include client solution design and implementation services, initial proofs of leading-edge technology, and contributing to new product design based on client requirements.