Using Chapel on Intel "Knights Corner"¶
Warning
KNC support is deprecated as of Chapel 1.13 and support will be entirely removed for the 1.14 release.
The following information is assembled to help Chapel users get up and running on Intel Xeon Phi, Knights Corner (KNC).
The initial implementation runs in self-hosted mode only. That is, the entire Chapel program runs on the KNC co-processor. For multilocale, GASNet must use the MPI conduit (segment everything) as all communication must bounce through the host processor.
Current Limitations¶
There are currently a number of limitations for KNC builds:
Only the Intel compiler supports compilation for KNC:
CHPL_TARGET_COMPILER=intel
orCHPL_TARGET_COMPILER=cray-prgenv-intel
Only non-native comm substrates can be used with KNC:
CHPL_COMM_SUBSTRATE=mpi
orCHPL_COMM_SUBSTRATE=udp
Additionally the following configurations are not supported:
Configuration Reason CHPL_REGEXP=re2 sfence not supported CHPL_GMP=system system version not built for KNC
Getting started¶
For vanilla Intel compiler:¶
Set
CHPL_TARGET_COMPILER=intel
andCHPL_TARGET_ARCH=knc
.Build Chapel as normal. Running
printchplenv
should look something like this:CHPL_HOST_PLATFORM: linux64 CHPL_HOST_COMPILER: gnu CHPL_TARGET_PLATFORM: linux64 CHPL_TARGET_COMPILER: intel CHPL_TARGET_ARCH: knc CHPL_LOCALE_MODEL: flat CHPL_COMM: none CHPL_TASKS: qthreads CHPL_LAUNCHER: none CHPL_TIMERS: generic CHPL_MEM: jemalloc CHPL_MAKE: gmake CHPL_ATOMICS: intrinsics CHPL_GMP: none CHPL_HWLOC: hwloc CHPL_REGEXP: none CHPL_WIDE_POINTERS: struct CHPL_LLVM: none CHPL_AUX_FILESYS: none
Build your Chapel program as normal:
chpl hello.chpl
For Cray XC:¶
For Cray machines, only the aprun
launcher is supported. In addition,
CHPL_TASKS=muxed
and CHPL_COMM=ugni
are not supported.
Load the proper modules. The order in which you do these module commands matters.
First, swap in the Intel PE module. For example, if you have the Cray PE module loaded:
module swap PrgEnv-cray PrgEnv-intel
Unload
libsci
,atp
(unsupported) and anycraype
processor type modules:module unload cray-libsci atp craype-ivybridge
Load the KNC
craype
module:module load craype-intel-knc
Unload any
craype-hugepages
module you have loaded, for example:module unload craype-hugepages2M
Build Chapel as normal. Running
printchplenv
should look something like this:CHPL_HOST_PLATFORM: cray-xc CHPL_HOST_COMPILER: gnu CHPL_TARGET_PLATFORM: cray-xc CHPL_TARGET_COMPILER: cray-prgenv-intel CHPL_TARGET_ARCH: knc CHPL_LOCALE_MODEL: flat CHPL_COMM: gasnet CHPL_COMM_SUBSTRATE: mpi CHPL_GASNET_SEGMENT: everything CHPL_TASKS: qthreads CHPL_LAUNCHER: aprun CHPL_TIMERS: generic CHPL_MEM: jemalloc CHPL_MAKE: gmake CHPL_ATOMICS: intrinsics CHPL_NETWORK_ATOMICS: none CHPL_GMP: gmp CHPL_HWLOC: hwloc CHPL_REGEXP: none CHPL_WIDE_POINTERS: struct CHPL_LLVM: none CHPL_AUX_FILESYS: none
Build your Chapel program as normal:
chpl hello.chpl
Request some nodes with KNCs. You can get the node numbers using cnselect:
cnselect -e name.eq.Xeon_Phi
or:
cnselect -e name.eq.MIC
Next, use qsub to request the nodes. For example, to request nodes 12-16 for interactive using PBSPro:
qsub -I -lmppwidth=5,mppnppn=1,mppnodes=\"12-16\"
Change to your working directory:
cd $PBS_O_WORKDIR
Run your program as you would normally.
./a.out -nl 5