Using z10 HiperDispatch for vertical CPU management

The new HiperDispatch feature for the IBM z10 mainframe manages CPU vertically instead of horizontally. This new feature could boost performance on large transactional workloads.

IBM's first dispatcher went into their operating system when they enabled multiprogramming in the 1970s. In the 90s, IBM introduced the Processor Resource/Systems Manager (PR/SM) Hypervisor, which enabled users to run one or more instances of the operating system as logical partitions (LPAR's) on the same hardware. Up until now the two dispatchers have had little notion that the other existed. With the arrival of z10 processors, however, IBM brought them closer together.

First, a discussion of z10 architecture: The brains of a z10 are organized into "books," up to four of them in a Central Electronic Complex (CEC). Each book contains from 1 to 16 Processor Units (PU's), along with several levels of cache and memory. Two of the cache levels, L1 and L1.5, are local to each processor. The L2 cache and main memory are available to every PU in all the books. The books talk to each other in a star-configured bus.

As you've probably already surmised, there's a penalty to pay for the unlucky unit of work that gets dispatched on a different PU in the same book; it loses the L1 and L1.5 cache. The situation only gets worse when the operating system puts the work on a PU in a different book. Not only does it need to rebuild its cache, but its favorite memory pages must be fetched from the other book, costing precious cycles. IBM designed HiperDispatch to avoid that overhead.

HiperDispatch and CPU management
CPU management before the z10 was "horizontal." That is, z/OS had logical processors (LP's) on which it tried to distribute work evenly. PR/SM, for its part, had to apportion physical processing units (PU's) according to weights set by the technical support staff. z10 and HiperDispatch add the option of "vertical" CPU management.

In vertical CPU management LP's belong to one of three classes:

  1. Vertical High – The LP receives 100% of its PU share. The system also tries to keep a strong affinity between the LP and PU to help cache hit ratios.
  2. Vertical Medium – These LP's get somewhere between 0% and 100% of a PU share.
  3. Vertical Low – The LP gets 0% of a PU. In fact, this type of LP may become "parked" if there's nothing to do or the physical resources aren't available on the CEC. They might become active if the LPAR needs a processor and there's available room in the CEC.

z/OS arranges the LP's into "affinity groups" and tries to keep a unit of work in the same group in the hopes of building an affinity with a physical PU. The strength of the affinity reduces the overhead of fetching memory between books. PR/SM "quasi-dedicates" (IBM's phrase) designated vertical high LP's to physical processors. Left over processor share is given to vertical medium LP's. PR/SM also supplies z/OS with processor topology so the operating system understands PU and book boundaries.

Another consequence of HiperDispatch is z/OS will try to keep the vertical high processors as busy as possible as opposed to the old way of spreading the work evenly. Therefore, it wouldn't be unusual to see several LP's with very high utilization while a couple others are mostly idle or parked.

What type of work benefits from HiperDispatch? The classic IBM reply pertains; "Your mileage will vary." There are, however, broad outlines:

  • One-book LPAR's and batch will run about the same.
  • CPU bound workloads will see little benefit as they're generally tied to a processor anyway.
  • Transactional type workloads that experience many interrupts and dispatches will see some improvement.
  • Big, multi-book LPAR's with transactional workloads will probably get the biggest boost.

IBM also recommends a Workload Management (WLM) policy review before implementing HiperDispatch. Some work that formerly belonged in the same service class (e.g, CICS and WebSphere) may need to be broken out. An IBM white paper titled, "Planning Considerations for HiperDispatch Mode" recommends promoting highly interactive WebSphere above relatively CPU-bound CICS and DB2. The same document has a nice explanation of various HiperDispatch definitions along with other chestnuts of wisdom.

ABOUT THE AUTHOR:For 24 years, Robert Crawford has worked off and on as a CICS systems programmer. He is experienced in debugging and tuning applications and has written in COBOL, Assembler and C++ using VSAM, DLI and DB2.

Dig Deeper on IT infrastructure management and planning

Cloud Computing
and ESG