eagle-i University of PennsylvaniaUniversity of Pennsylvania
See it in Search

High-performance Computing (Penn)

Director: Kaylor, James L., MSE, CISM


The PennHPC facility opened in April of 2013 to meet the increasing growth in genomics processing and storage, as well as growth in other scientific areas requiring computational capacity such as imaging and biostatistics/bioinformatics. The cluster is managed by two fulltime system administrators, and is located at the Philadelphia Technology Park, a Tier-3, SSAE 16/SAS 70 Type II Audit compliant colocation/datacenter facility.





  • High performance compute cluster ( Computer cluster )

    "The 144 IBM iDataPlex cluster nodes each house two eight-core Intel E5-2665 2.4Ghz Xeon Processors, 192 or 256 Gigabytes of RAM, and a 500 Gigabyte local hard drive. These nodes are sub-divided into virtual processing cores, with the capability to provision up to 5,100 virtual cores at 6GB of RAM per virtual core. Cluster and storage interconnection is provided by a 40Gbps Ethernet fabric with 10Gbps to each node. The cluster nodes are attached to 1.8 Petabytes of IBM Storwise V7000 disk storage, housed in two separate performance tiers (no backup). The disk is presented to the compute nodes via an ten-node IBM Scale out Network Attached Storage (SONAS) system leveraging the IBM General Parallel File System (GPFS). Computational job scheduling/queuing and cluster management is orchestrated by the IBM Platform Computing (LSF) suite of products."

  • Scale out Network Attached Storage ( Network attached storage )

    "The cluster nodes are attached to 1.8 Petabytes of IBM Storwise V7000 disk storage, housed in two separate performance tiers (no backup). The disk is presented to the compute nodes via a ten-node IBM Scale out Network Attached Storage (SONAS) system leveraging the IBM General Parallel File System (GPFS)."

  • SpectraLogic T950 tape library ( Network attached storage )

    SpectraLogic T950 tape library houses 1910 LTO-5 tapes, 290 LTO-6 tapes, and 12 LTO-6 drives with a total raw capacity of 3.6 Petabytes of storage.


  • Data archiving service ( Data storage service )

    "The PMACS HPC system houses 1.8PB of mirrored tape storage for both archive and backup uses. Archive storage provides clients with a primary data storage target that will look and function as any other hard drive, though often at a slower rate of data transfer. The archive is 100% client accessible, and clients need only to mount the drive on their computer and begin to copy/move data to/from the drive. PMACS HPC staff will provide the necessary information to mount the drive, and assist clients with initial setup. Data can be moved from an existing drive share on the PMACS HPC disk system, a scientific instrument, a desktop, or any other device that can map an NFS or CIFS share.

    Archive storage is NOT meant for any data that will be regularly accessed, nor any data upon which computation must be performed. The archive is “active” in that end-user files are available in real-time, however the purpose of the archive is long-term storage of infrequently access files.

    Archive performance is variable, and will depend upon how heavily the system is being used at any given time. No data ingress or egress service levels/performance are guaranteed, however average data retrieval times will be available once the system has been operational for a period of time over which performance metrics can be calculated. "

  • Data transfer service ( Support service )

    "There is no charge assessed for data transfer to/from the PMACS HPC system."

  • Disk storage service ( Access service )

    "The PMACS HPC houses 1.8 Petabtyes (1,800 Terabytes) of disk storage. Some of this storage is used for system utility functions; however most of the storage (about 80%) is available for the storage and processing of HPC jobs.

    Data is accessible via both NFS (Linux/Unix/Mac) and CIFS (Windows) drive shares. Shares can be mapped to client desktops, laptops, as well as biomedical equipment, such as high-throughput sequencers capable of mapping NFS and/or CIFS shares.

    Once data exists on the PMACS HPC disk storage it is then accessible by the compute cores and queue management software, and can become part of computational jobs.

    Storage rates apply to any data stored whether short or long-term. Rates are calculated regularly throughout the month.

    Disk storage costs do not include backup services. Data protection included in disk storage costs is based upon a technology called Redundant Array of Independent Disks (RAID) which protects data should a drive or drives in the system fail. Backup services are available, however backup services incur the “Archive/Backup Storage” rate detailed below, and are on an as-requested basis. "

  • High performance computing ( Access service )

    "The computational hour can also be referred to as a “core-hour” or “virtual-core-hour.” The system is configured in such a manner as to allow more computational-hours than the number of physical processors in the system. This virtualization provides a greater computational capacity than if each physical processing core was utilized individually, therefore minimizing queue wait times for jobs to complete. Each virtual core is assigned up to 8GB of RAM, however alternate RAM configurations are available as-needed."

    "Cluster Operating System is Red Hat Enterprise Linux v. 6.4, and this version is installed on all compute nodes. Alternate OS installations are available on an as-needed basis, by request only. Cluster software is either loaded on the individual node hard drives by HPC staff, providing the best performance, or is run from a client’s share on the SONAS file system, generally not requiring any HPC staff involvement. Clients needing software installation should place a service request in the PMACS HPC queue, or e-mail to pmacshpc@med.upenn.edu."

    "The PMACS HPC uses the IBM Platform LSF job scheduling system."

  • HPC consultation/customization service ( Support service )

Web Links:

Last updated: 2020-01-23T11:09:16.481-05:00

Copyright © 2016 by the President and Fellows of Harvard College
The eagle-i Consortium is supported by NIH Grant #5U24RR029825-02 / Copyright 2016