Interoperability

Chapel’s interoperability features support cooperation between Chapel and other languages. They provide the ability to create software systems that incorporate both Chapel and non-Chapel components. Thus, they support the reuse of existing software components while leveraging the unique features of the Chapel language.

Interoperability can be broken down in terms of the exchange of types, variables and procedures, and whether these are imported or exported. An overview of procedure importing and exporting is provided in Interoperability Overview. Details on sharing types, variables and procedures are supplied in Shared Language Elements.

Note

Future:

At present, the backend language for Chapel is C, which makes it relatively easy to call C libraries from Chapel and vice versa. To support a variety of platforms without requiring recompilation, it may be desirable to move to an intermediate-language model.

In that case, each supported platform must minimally support that virtual machine. However, in addition to increased portability, a virtual machine model may expose elements of the underlying machine’s programming model (hardware task queues, automated garbage collection, etc.) that are not easily rendered in C. In addition, the virtual machine model can support run-time task migration.

The remainder of this chapter documents Chapel support of interoperability through the existing C-language backend.

Interoperability Overview

The following two subsections provide an overview of calling externally-defined (C) routines in Chapel, and setting up Chapel routines so they can be called from external (C) code.

Calling External Functions

To use an external function in a Chapel program, it is necessary to inform the Chapel compiler of that routine’s signature through an external function declaration. This permits Chapel to bind calls to that function signature during function resolution. The user must also supply a definition for the referenced function by naming a C source file, an object file or an object library on the chpl command line.

An external procedure declaration has the following syntax:

external-procedure-declaration-statement:
  'extern' external-name[OPT] 'proc' identifier argument-list return-intent[OPT] return-type[OPT] ;

Chapel code will call the external function using the parameter types supplied in the extern declaration. Therefore, in general, the type of each argument in the supplied argument-list must be the Chapel equivalent of the corresponding external type.

The return value of the function can be used by Chapel only if its type is declared using the optional return-type specifier. If it is omitted, Chapel assumes that no value is returned, or equivalently that the function returns void.

It is possible to use the external-name syntax to create an extern function that presents a different name to Chapel code than the name of the function actually used when linking. The external-name expression must evaluate to a param string. For example, the code below declares a function callable in Chapel as c_atoi but that will actually link with the C atoi function.

extern "atoi" proc c_atoi(arg:c_ptrConst(c_char)):c_int;

At present, external iterators are not supported.

Note

Future:

The overloading of function names is also not supported directly in the compiler. However, one can use the external-name syntax to supply a name to be used by the linker. In this way, function overloading can be implemented “by hand”. This syntax also supports calling external C++ routines: The external-name to use is the mangled function name generated by the external compilation environment [1].

Note

Future:

Dynamic dispatch (polymorphism) is also unsupported in this version. But this is not ruled out in future versions. Since Chapel already supports type-based procedure declaration and resolution, it is a small step to translate a type-relative extern method declaration into a virtual method table entry. The mangled name of the correct external function must be supplied for each polymorphic type available. However, most likely the generation of .chpl header files from C and C++ libraries can be fully automated.

There are three ways to supply to the Chapel compiler the definition of an external function: as a C source file (.c or .h), as an object file and as an object library. It is platform-dependent whether static libraries (archives), dynamic libraries or both are supported. See the chpl man page for more information on how these file types are handled.

Calling Chapel Functions

To call a Chapel procedure from external code, it is necessary to expose the corresponding function symbol to the linker. This is done by adding the export linkage specifier to the function definition. The export specifier ensures that the corresponding procedure will be resolved, even if it is not called within the Chapel program or library being compiled.

An exported procedure declaration has the following syntax:

exported-procedure-declaration-statement:
  'export' external-name[OPT] 'proc' identifier argument-list return-intent[OPT] return-type[OPT]
    function-body

external-name:
  expression

The rest of the procedure declaration is the same as for a non-exported function. An exported procedure can be called from within Chapel as well. Currently, iterators cannot be exported.

As with the extern-name for extern proc, if this syntax element is provided, then it must be a param string and will be used to determine the name of the function to use when linking. For example, the code below declares a function callable in C as chapel_addone but it is callable from Chapel code as addone:

export "chapel_addone" proc addone(arg:c_int):c_int {
  return arg+1;
}

Note

Future.

Currently, exported functions cannot have generic, param or type arguments. This is because such functions actually represent a family of functions, specific versions of which are instantiated as need during function resolution.

Instantiating all possible versions of a template function is not practical in general. However, if explicit instantiation were supported in Chapel, an explicit instantiation with the export linkage specifier would clearly indicate that the matching template function was to be instantiated with the given param values and argument types.

Shared Language Elements

This section provides details on how to share Chapel types, variables and procedures with external code. It is written assuming that the intermediate language is C.

Shared Types

This subsection discusses how specific types are shared between Chapel and external code.

Referring to Standard C Types

When referring to standard C types within Chapel code, they must either be expressed:

• using the type aliases provided by the CTypes module,

• using an external type declaration (see the next section), or

• in terms of their Chapel equivalents (e.g., C double corresponds to Chapel real(64))

  Note

  Note that a challenge to this last approach is that certain C types—particularly integers—can vary in size across implementations. As a result, using the type aliases in CTypes for such cases is safer from a portability perspective.

Referring to External C Types

An externally-defined type can be referenced using a external type declaration with the following syntax.

external-type-alias-declaration-statement:
  'extern' external-name[OPT] 'type' type-alias-declaration-list ;

In each type-alias-declaration, if the type-expression part is supplied, then Chapel uses the supplied type specifier internally. Otherwise, it treats the named type as an opaque type. The definition for an external type must be supplied by a C header file named on the chpl command line.

If an optional external-name is supplied, it provides the name of the symbol(s) in C, where the given Chapel identifier(s) are used to refer to them within Chapel code.

Fixed-size C array types can be described within Chapel using the c_array type defined by the standard CTypes module. For example, the C typedef

typedef double vec[3];

can be described in Chapel using

extern type vec = c_array(c_double, 3);

Referring to External C Structs and Unions

External C struct and union types can be referred to within Chapel by prefixing a Chapel record definition with the extern keyword.

external-record-declaration-statement:
  'extern' external-name[OPT] simple-record-declaration-statement

For example, consider an external C structure defined in foo.h called fltdbl.

typedef struct _fltdbl {
  float x;
  double y;
} fltdbl;

This type could be referred to within a Chapel program using

extern record fltdbl {
  var x: real(32);
  var y: real(64);
}

and defined by supplying foo.h on the chpl command line.

The same applies for a C union. An example would be such:

typedef union _someUnion {
  float x;
  double y;
} someUnion;

and this type could be referred to within a Chapel program using

extern union someUnion {
  var x: real(32);
  var y: real(64);
}

Within the Chapel declaration, some or all of the fields from the C structure or union may be omitted. The order of these fields need not match the order they were specified within the C code. Any fields that are not specified (or that cannot be specified because there is no equivalent Chapel type) cannot be referenced within the Chapel code. Some effort is made to preserve the values of the undefined fields when copying these structs but Chapel cannot guarantee the contents or memory story of fields of which it has no knowledge.

If the optional external-name is supplied, then it is used verbatim as the exported struct symbol.

A C header file containing the struct’s (or union’s) definition in C must be specified on the chpl compiler command line. Note that only typdef’d C structures or unions are supported by default. That is, in the C header file, the struct or union must be supplied with a type name through a typedef declaration. If this is not true, you can use the external-name part to apply the struct (or union) specifier. As an example of this, given a C declaration of:

struct Vec3 {
  double x, y, z;
};

in Chapel you would refer to this struct via

extern "struct Vec3" record Vec3 {
  var x, y, z: real(64);
}

Note that the above examples apply for C unions as well, so an example for non-typedef’d C union would be like this:

union noTypedefUnion {
   float x;
   double y;
   int64_t z;
};

referring to this union would be allowed in Chapel, via:

extern "union noTypedefUnion" union noTypedefUnion {
    var x: real(32);
    var y: real(64);
    var z: int(64);
}

Opaque Types

It is possible refer to external pointer-based C types that cannot be described in Chapel by using the “opaque” keyword. As the name implies, these types are opaque as far as Chapel is concerned and cannot be used for operations other than argument passing and assignment.

For example, Chapel could be used to call an external C function that returns a pointer to a structure (that can’t or won’t be described as a pointer to an external record) as follows:

extern proc returnStructPtr(): opaque;

var structPtr: opaque = returnStructPtr();

However, because the type of structPtr is opaque, it can be used only in assignments and the arguments of functions expecting the same underlying type.

var copyOfStructPtr = structPtr;

extern proc operateOnStructPtr(ptr: opaque);
operateOnStructPtr(structPtr);

Like a void* in C, Chapel’s opaque carries no information regarding the underlying type. It therefore subverts type safety, and should be used with caution.

Shared Data

This subsection discusses how to access external variables and constants.

A C variable or constant can be referred to within Chapel by prefixing its declaration with the extern keyword. For example:

extern var bar: foo;

would tell the Chapel compiler about an external C variable named bar of type foo. Similarly,

extern const baz: int(32);

would refer to an external 32-bit integer constant named baz in the C code. In practice, external consts can be used to provide Chapel definitions for #defines and enum symbols in addition to traditional C constants.

Implementation Notes.

Note that since params must be known to Chapel at compile-time and the Chapel compiler does not necessarily parse C code, external params are not supported.

Shared Procedures

This subsection provides additional detail and examples for calling external procedures from Chapel and for exporting Chapel functions for external use.

Calling External C Functions

To call an external C function, a prototype of the routine must appear in the Chapel code. This is accomplished by providing the Chapel signature of the function preceded by the extern keyword. For example, for a C function foo() that takes no arguments and returns nothing, the prototype would be:

extern proc foo();

To refer to the return value of a C function, its type must be supplied through a return-type clause in the prototype. [2]

If the above function returns a C double, it would be declared as:

extern proc foo(): real;

Similarly, for external functions that expect arguments, the types of those arguments types may be declared in Chapel using explicit argument type specifiers.

The types of function arguments may be omitted from the external procedure declaration, in which case they are inferred based on the Chapel callsite. For example, the Chapel code

extern proc foo(x: int, y): real;
var a, b: int;
foo(a, b);

would imply that the external function foo takes two 64-bit integer values and returns a 64-bit real. External function declarations with omitted type arguments can also be used call external C macros.

External function arguments can be declared using the default-expression syntax. In this case, the default argument will be supplied by the Chapel compiler if the corresponding actual argument is omitted at the callsite. For example:

extern proc foo(x: int, y = 1.2): real;
foo(0);

Would cause external function foo() to be invoked with the arguments 0 and 1.2.

C varargs functions can be declared using Chapel’s variable-argument-expression syntax (...). For example, the C printf function can be declared in Chapel as

extern proc printf(fmt: c_ptrConst(c_char), vals...?numvals): int;

External C functions or macros that accept type arguments can also be prototyped in Chapel by declaring the argument as a type. For example:

extern foo(type t);

Calling such a routine with a Chapel type will cause the type identifier (e.g., ’int’) to be passed to the routine. [3]

Calling Chapel Procedures Externally

To call a Chapel procedure from external code, the procedure name must be exported using the export keyword. An exported procedure taking no arguments and returning a 64-bit integer can be declared as:

export proc foo(): int { ... }

If the optional external-name is supplied, that is the name used in linking with external code. For example, if we declare

export "myModule_foo" proc foo(): int { ... }

then the name foo is used to refer to the procedure within chapel code, whereas a call to the same function from C code would appear as myModule_foo();. If the external name is omitted, then its internal name is also used externally.

When a procedure is exported, all of the types and functions on which it depends are also exported. Iterators cannot be explicitly exported.

Argument Passing

The manner in which arguments are passed to an external function can be controlled using argument intents. The following table shows the correspondence between Chapel intents and C argument type declarations. These correspondences pertain to both imported and exported function signatures.

Chapel	C
T	const T
in T	T
ref T	T*
const ref T	const T*
param
type	char*

Currently, param arguments are not allowed in an extern function declaration, and type args are passed as a string containing the name of the actual type being passed. Note that the level of indirection is changed when passing arguments to a C function using the ref or const ref intent. The C code implementing that function must dereference the argument to extract its value.

Variable Initialization

When default initializing a variable of extern type, the compiler will arrange to fill its memory with zero bytes. However, an extern record can define a proc init() in order to define how it should be default initialized. See also Record Initialization and Variable Lifetimes.

Note

Future versions may allow an extern record to include proc init= and proc deinit but these cannot yet be counted on. One challenge in this area is that C code working with the same type will not call the copy or deinit function.

[1]

In UNIX-like programming environments, nm and grep can be used to find the mangled name of a given function within an object file or object library.

[2]

The return type cannot be inferred, since an extern procedure declaration has no body.

[3]

In practice, this will typically only be useful if the external function is a macro or built-in that can handle type identifiers.