Sanitizers

Sanitizers are a compiler feature that support instrumenting programs to do dynamic analysis to catch many classes of bugs at runtime.

AddressSanitizer (ASan) is a fast memory error detector. It consists of a compiler instrumentation module and a run-time library. ASan is similar to valgrind in that it can help identify memory errors. ASan is much faster than valgrind, but does require recompilation. Note that only GCC and Clang support sanitizers.

Building the Compiler

The following environment variables are required to build the Chapel compiler with sanitizers enabled:

export CHPL_TARGET_MEM=cstdlib
export CHPL_SANITIZE_EXE=address

Running the Compiled Program

After building a compiler configured like the above, you can build and run a Chapel program with AddressSanitizer enabled by setting the following environment variables:

chpl hello.chpl
export ASAN_OPTIONS="use_sigaltstack=0,detect_leaks=0"
./hello

Limitations

AddressSanitizer should be able to detect almost all memory errors for single locale configurations using CHPL_COMM=none. However, ASan cannot detect all classes of memory errors in multilocale configurations and the extent of the support depends on the CHPL_COMM setting. As a specific example some configurations will not be able to detect invalid remote reads/writes since ASan only knows about local memory. In the following program the invalid write to the array may not be detected for configurations that use RDMA to perform writes.

var A: [1..10] int;
on Locales[numLocales-1] {
  A[11] = 11; // Out of bounds write
}

This particular example would be caught by Chapel bounds checking, which are enabled by default but disabled with --fast. To get bounds checking and optimizations you can use --fast --bounds-checks or --fast --checks.

CHPL_COMM=gasnet with CHPL_COMM_SUBSTRATE=udp should be able to detect most memory errors since remote reads/writes are performed with active messages instead of RDMA.

CHPL_COMM=ugni without hugepages can detect some, but not all invalid remote reads/writes, but comes with a large performance cost. Hugepages can be unloaded with module unload $(module -t list 2>&1 | grep craype-hugepages)

Configuration Limitations

The above options are needed because not all third-party libraries support sanitizers. In particular:

Sanitizers hook into the system allocator, so using jemalloc is not supported
By default the gcc address sanitizer will enable leak checking, but Chapel intentionally leaks some memory in the runtime, so we disable that tracking for now. See Debugging Chapel Programs for more info about debugging memory leaks in Chapel.
An upstream bug can result in false-positives for some gcc versions. use_sigaltstack=0 works around this.

Other Sanitizers

Currently, only AddressSanitizer is regularly tested, but the options passed to CHPL_SANITIZE_EXE are passed directly to the backend compiler’s -fsanitize= option, so other sanitizers can be enabled. For example, to use C undefined behavior sanitizer and address sanitizer set CHPL_SANITIZE_EXE=undefined,memory.