DynamicLoading

Usage

use DynamicLoading;

or

import DynamicLoading;

Support for dynamic loading in Chapel.

Note

As of the 2.9 release, prototypical support has been added for loading and executing code from Chapel programs at runtime. See the section Loading Chapel Programs at Runtime below.

This module provides the ability to load a binary at runtime. Procedures contained in a dynamically loaded binary can be retrieved and called on any locale without compile-time knowledge of their names or locations.

A hypothetical C binary could contain a procedure named foo:

// Compile with: 'cc -shared -fPIC -o TestBinary TestBinary.c'
//
// Your C compiler must support compiling a shared library as well
// as generating "position independent code". If your compiler
// offers some other way of preserving symbol tables (e.g., the
// '-rdynamic' flag) for use with dynamic loading, you can compile
// a regular executable with that flag as well.
//

#include <stdio.h>

void foo(void);
void foo(void) {
  printf("Hello world from %s!\n", __FUNCTION__);
}

int main(void) {
  foo();
  return 0;
}

This binary can be can be loaded in Chapel at runtime as follows:

use DynamicLoading;

// A binary may or may not exist at this path.
const path = './TestBinary.so';

// If loading fails an error will be issued and the 'try!' will halt.
const bin = try! binary.load(path);

And a procedure named foo with type proc(): void can be retrieved:

// Declare the type of the procedure.
type P = proc(): void;

// Retrieve a procedure named 'foo' from 'bin' with the type 'P'.
const p1 = try! bin.retrieve('foo', P);

// Call the procedure.
p1();

When a procedure is retrieved from a loaded binary, the returned procedure value is callable on any locale despite binary.retrieve() only being called on a single locale. The returned procedure is considered to be extern and this is reflected in its type.

Note

Currently, only procedures can be retrieved from loaded binaries. Support for retrieving references to data stored in a binary could be added in the future.

Loading Chapel Programs at Runtime

Chapel supports the ability to dynamically load other Chapel programs while having all loaded Chapel programs utilize a single shared copy of the runtime.

Warning

This feature is highly unstable (even more than the rest of this module). As of the 2.9 release, it is only supported when CHPL_COMM=gasnet and CHPL_LIB_PIC=pic. Safety checks have not been added yet, so dynamically loaded code may break in unexpected ways if these constraints are not met.

Additionally, while it should be possible to execute parallel and distributed code from a loaded Chapel library, it is possible that certain (combinations of) features may have bugs and fail to work correctly.

To utilize this feature, both the Chapel program that is doing the loading (the “executable”) and the Chapel library that is loaded (the “library”) must be compiled with the flag --no-builtin-runtime.

This flag causes a Chapel program to be linked against a shared library variant of the Chapel runtime rather than to statically link in a copy as has traditionally been done.

In order to use --no-builtin-runtime, you must make sure your Chapel configuration was built with CHPL_LIB_PIC=pic so that the runtime, executable, and library can be compiled with position-independent code.

Additionally, the library must be compiled with --library --dynamic.

Once compiled, the library can be loaded by the executable using the same process that is described in the previous sections of this module.

Do note that only export procedures may be called from the library. This restriction may be removed in the future.

Note

You may get strange link-time errors when trying to run the executable. If this happens, it is usually because of a bug that occurs when the dynamic variant of the Chapel runtime library was built.

If you’ve built Chapel from source, a workaround is to touch a source file in the Chapel runtime and then rebuild it.

record binary

A wrapper around a dynamically loaded binary.

proc init=(rhs: binary)

The initialized binary refers to the same binary stored in rhs.

proc deinit()

operator =(ref lhs: binary, rhs: binary)

After assignment, lhs will refer to the same binary stored in rhs.

proc type load(path: string) throws

Create a record representing a dynamically loaded binary using a string that stores the path to a binary file. The file may be any executable binary, though a “shared library” or “dynamic library” is by far the most common and well supported.

The implementation will attempt to load a binary on all locales. If loading should fail on any locale then the entire process will be aborted and an error will be thrown.

Dynamic loading can fail for a variety of reasons. It can fail if the binary could not be loaded on any locale. It can fail if no file was found at path or if a file was found but it did not have a suitable representation (e.g., the file format was not natively executable, or it was not marked as executable).

Arguments:

path : string – The path to the binary file to dynamically load

Throws:

DynLoadError – if dynamic loading fails

proc retrieve(name: string, type t) throws  where isProcedureType(t)

Fetch a procedure from a dynamically loaded binary. Throws an error if no procedure could be found.

Warning

The procedure type t provided when calling binary.retrieve() is used verbatim and is not checked against the type of the underlying symbol in any way. If the type provided does not match the actual type of the underlying procedure, then the resulting behavior when the retrieved procedure is called is undefined. Crashes or other bugs could occur.

The retrieved procedure is considered to be extern and will use an extern calling convention, the same as is used for other extern procedures declared in Chapel code. This impacts how it can be used. For example, it cannot currently be assigned to a procedure value that is not also considered extern.

Arguments:

• name : string – The name of the procedure to retrieve

• t – The type of the retrieved procedure

Throws:

DynLoadError – if procedure lookup fails