kentoh - Fotolia
A souped-up parallel file system by Pavilion Data Systems brings dense, distributed multiprotocol flash storage capabilities to supercomputing workloads.
The release of Pavilion HyperOS 3.0 marks a key enhancement for the flash startup, which is trying to ride a wave of interest in computational storage. Pavilion launched several years ago amid a wave of rack-scale NVMe startups, most of which have been gobbled up in acquisition or ceased operations.
Pavilion HyperOS 3.0 parallelizes compute, NVMe flash and storage networking in tandem. The vendor's Pavilion Hyperparallel Flash Array uses native TCP to cluster multiple controllers in a global namespace for block, file and object data. The namespace can be clustered across multiple chassis and racks.
Storage scales to 2.2 PB of raw capacity with up to 72 NVMe solid-state drives (SSDs). The arrays include 40 network ports for 100 gigabit Ethernet or InfiniBand. Enhanced quality of service allows users to set four tiers for each volume, based on capacity limits or application IOs.
Pavilion's flash array is designed around a virtualized NVMe-oF network switch. The cacheless layout allows the controllers to write to any drive using remote direct memory access. The effect is shared hosting with performance equivalent to direct-attached storage. Pavilion's projected use cases include AI, big data, IoT and high-performance computing.
"We architected the storage controllers, the network, and everything in between [to deliver] hyperparallelism and predictable performance," said Costa Hasapopoulos, Pavilion Data chief field technology officer.
Pavilion flash with scalable namespace
Pavilion Data storage software exploits highly optimized flash hardware with predictable performance, said Tom Coughlin, president of storage analyst firm Coughlin Associates.
"There isn't any special hardware sauce. They bought really fast components and put them together so that the software doesn't get in the way of performance," Coughlin said.
With the upgrade, Pavilion said it intends to step up its pursuit of enterprise customers that rely heavily on GPU-based applications. HyperOS was refreshed to support 10 active-active line cards per rack, with two controllers per line card. Each controller runs a native HyperOS instance for coherency.
Pavilion claims the Hyperparallel Flash Array delivers up to 20 million IOPS in 4U. The vendor rates block storage reads at 120 GB/s and writes at 90 GB/s. Within the global namespace, Pavilion rates reads of 60 GB/s and writes of 40 GB/s for both files and objects.
Customers can start with a minimum of two Pavilion line cards, or four controllers, and scale capacity or compute processing separately. Pavilion said its tested performance allows linear scaling up to eight flash racks, 160 controllers and 16.6 TB of compressed storage capacity. A controller can independently serve a specific storage protocol or be used in a multicontroller setup within the same box and across multiple chassis.
The controller namespace could be devoted to NFS or S3, or shared across chassis. New Pavilion flash versioning guards against dropped runs in firmware. While writing data to 4K blocks, Pavilion Data tags the version number and places it in metadata to verify drive parity, checksum and automate rebuilds.
Coughlin said the controllers also enable highly parallel processing to deliver storage performance and reduce any resource contention that may arise.
Solving the compute-storage riddle
Modern applications drive the need for more parallelization of the hardware stack. Nvidia leads the way with its beta-tested GPUDirect Storage that allows data to be written from its DGX supercomputers directly to storage.
Other all-flash startups offer systems that push storage and processing in close proximity. The Fungible converged storage system is based on its custom data processing unit and NVMe-oF stack. Vast Data Universal Storage software supports any workload, while Lightbit Labs' software-defined block storage pools NVMe capacity from Ethernet-connected servers.
Eric Burgener, a research vice president of storage at analyst firm IDC, said Pavilion's technique is based on the simultaneous acceleration of all hardware, coupled with enterprise data services.
"The real differentiator for Pavilion is performance density. What I've heard from their customers are things like 'I moved from a 24U system down to a 4U system and I'm getting better performance.' Simpler is better," Burgener said.
Pricing for the Pavilion Hyperparallel Flash Array was not publicly disclosed. CEO Gurpreet Singh said Pavilion's "blockbuster" last quarter included customer wins at Sony Pictures, Canon and PayPal, and a reseller deal with Penguin Computing. Pavilion Data has received more than $58 million from venture investors since launching in 2014, including a $25 million funding haul last year.