Knights Landing

From SciNetWiki
Jump to: navigation, search
Intel Xeon Phi (Knights Landing )
KNL-DAP-Adams-Pass.jpg
Installed August 2016
Operating System Linux Centos 7.2
Number of Nodes 4
Interconnect QDR Infiniband
Ram/Node 96GB DDR4 + 16GB MCDRAM
Cores/Node 64
Login/Devel Node knl01
Vendor Compilers icc,ifort
Queue Submission none

This is develop/test system of four x86_64 self-hosted 2nd Generation Intel Xeon Phi (Knights Landing, KNL) nodes, aka an Intel "Ninja" platform. Each node has one 64-core Intel(R) Xeon Phi(TM) CPU 7210 @ 1.30GHz with 4 threads per core. These systems are not add-on accelerators, but instead act as full-fledged processors running a regular linux operating system. They are configured with 96GB of DDR4 system RAM along with 16GB of very fast MCDRAM, see here for details. The nodes are interconnected to the rest of the clusters with QDR Infiniband and shares the regular SciNet GPFS filesystems.

Contents

Login

First login via ssh with your SciNet account at login.scinet.utoronto.ca, and from there you can proceed to knl01,knl02,knl03,knl04.

KNL Operational Modes

The four nodes all have identical hardware, however there are multiple options that control how the MCDRAM High Bandwidth Memory (HBM) is accessed. Mode changes are not dynamic and require the node to be rebooted to take affect.

Clustering

Currently all KNL nodes have the Cluster Mode configured to "Quadrant". See this article for more details about the clustering options that contorl how memory is accessed on the KNL.

Memory

Two nodes, knl01,kn02 have the MCDRAM configured as "Cache" mode and the other two knl03,kn04 are configured with the "Flat" memory mode. See this article for more details about the MCDRAM memory modes.

Initially when you first compile/port your code, use the Cache mode nodes, however if you wish to try and optimize memory performance by directly using the HBM memory with the memkind library or the numactl options, use the Flat nodes.

user@knl03$ numactl --membind 1 ./mycode

Queue

Currently there is no queue, be nice.

Software

Software is available using the standard modules framework used on other SciNet systems, however is separate from the GPC modules as the KNL has a newer Centos7 based operating system.

Compilers

The Xeon Phi uses the standard intel compilers.

module load intel/16.0.3

MPI

IntelMPI is currently the default MPI

module load intelmpi/5.1.3.219

References

Personal tools
Namespaces
Variants
Actions
Systems
Knowledge Base
Wiki
Toolbox