C Interoperability¶

Pointer and String Types¶

Chapel supports four C pointer types: c_void_ptr, c_ptr(T), c_string, and c_fn_ptr. These types are the same as C types:

c_void_ptr is void*
c_ptr(T) is T*
c_string is const char*
c_fn_ptr represents a C function pointer (with unspecified arg and return types)

Note that in some cases, a ref argument intent may be used in place of c_void_ptr or c_ptr(T).

All four of these pointer types may only point to local memory. The intent is that they will be used to interoperate with C libraries that run within a single locale. In addition, these pointer types must be treated carefully as it is possible to create the same kinds of problems as in C - in particular, it is necessary know that one of these types points to valid memory before it is used. As in C, these pointers could point to a stack variable that is no longer valid, or to memory that has been freed. The Chapel language makes no effort to extend a variable's lifetime if it is converted in some manner to a C pointer.

// both of these correspond to void fn(int* x)
extern proc byRef(ref x:c_int);
extern proc byPtr(x:c_ptr(c_int));

var x:c_int = 7;
byRef(x); // ref argument intent allows the variable to be passed directly
byPtr(c_ptrTo(x)); // c_ptr argument must be constructed explicitly

extern proc foo(f: c_fn_ptr);

foo(c_ptrTo(bar));

proc bar(x: c_int): c_double {
  ...
}

Declaring External C Types¶

You can refer to other external C types using 'extern' plus the normal type alias keyword. By leaving off any sort of type definition, you are telling the Chapel compiler that it can't assume it knows anything about how the type is represented or how to operate on it (though the compiler may make copies of values of the type using assignment and/or memcpy()).

Effectively, such a declaration defines a new primitive type in Chapel. For example, the following declaration says that there is an external type named 'foo':

extern type foo;

This permits you to declare variables of type 'foo' and to declare external functions that accept or return arguments of type 'foo'. Because Chapel knows nothing about the type 'foo' such variables cannot be manipulated within Chapel apart from passing them between routines. The type symbol 'foo' must be declared in a C header file provided on Chapel's compiler command-line.

If an external C type can be described in Chapel, that definition can be given in Chapel, which permits the compiler to deal with the type directly, as it would any other Chapel type alias. For example, if the external type 'foo' was a 64-bit integer, you could describe it in Chapel using:

extern type foo = int(64);

Static, fixed-size C array types can be described within Chapel using its homogeneous tuple type. For example, the following C typedef:

typedef double vec[3];

could be described in Chapel using:

extern type vec = 3*real(64);

To refer to more complex C types like external structs or pointers to structs, see the section on Declaring External C Structs below.

Declaring External C Variables and Constants¶

A C variable or constant can be referred to within Chapel by prefixing its declaration with the extern keyword. For example:

extern var bar: foo;

would tell the Chapel compiler about an external C variable named 'bar' of type 'foo'. Similarly:

extern const baz: int(32);

would refer to an external 32-bit integer constant named 'baz' in the C code. In practice, external consts can be used to provide Chapel definitions for #defines and enum symbols in addition to traditional C constants.

Note that it is not currently possible to explicitly declare an extern param. In the future, it might be possible to use an extern block to import #define constants that are known at compile time as param constants within Chapel.

Declaring External C Functions¶

To make a call to an external C function, you will need to prototype the routine in your Chapel code using the extern keyword. For example, for a C function foo() that takes no arguments and returns nothing, the prototype would appear as follows:

extern proc foo();

C functions that return values which you wish to refer to within your Chapel program must have those return types declared. Note that the Chapel compiler will not infer the return type as it does for Chapel functions. To make the function above return a C "double", it would be declared:

extern proc foo(): real;

Similarly, external functions that expect arguments must declare those arguments in Chapel.

Types of function arguments may be omitted, in which case the types will be inferred based on the Chapel callsite. For example, the following Chapel code:

extern proc foo(x: int, y): real;

var a, b: int;

foo(a, b);

Would imply that external function foo() is able to take two 32-bit integer values and that it returns a 64-bit real ('double' in C). External function declarations with omitted type arguments can be used to support calls to external C macros or varargs functions that accept multiple argument signatures.

Default arguments can be declared for external function arguments, in which case the Chapel compiler will supply the default argument value if it is omitted at the callsite. For example:

extern proc foo(x: int, y = 1.2): real;

foo(0);

Would cause external function foo() to be called with the arguments 0 and 1.2.

C varargs functions can be declared using Chapel's varargs ("...") syntax. For example, the following declaration prototypes C's printf function:

extern proc printf(fmt: c_string, vals...?numvals): int;

Note that it can also be prototyped more trivially/less accurately as follows:

extern proc printf(args...): int;

which relies on the callsite to pass in reasonable arguments (otherwise, the C compilation step will likely fail).

External C functions or macros that accept type arguments can also be prototyped in Chapel by declaring the argument as a type. For example:

extern foo(type t);

Calling such a routine with a Chapel type will cause the type identifier (e.g., 'int') to be passed to the routine. In practice, this will typically only be useful if the external function is a macro or built-in (like sizeof()) that can handle type identifiers.

Extern functions with array arguments are handled as a special case within the compiler. As an example:

extern proc foo(x: [] int, n: int);

This procedure definition will match up to a C function with the prototype of

void foo(int64_t* x, int64_t n);

The Chapel compiler will then rewrite any calls to foo like this:

foo(x, 10); // -> foo(c_ptrTo(x), 10);

Note that this same technique won't work for distributed rectangular arrays, nor for associative, sparse, or opaque arrays because their data isn't necessarily stored using a representation that translates trivially to a C array.

It is possible to provide the Chapel compiler with a different name for the function than the name available to other Chapel code. For example, if there is a C function called foo_in_c returning an int, we might use the following declaration to provide that function to other Chapel code with the name foo_in_chapel

extern "foo_in_c" proc foo_in_chapel(): c_int;
writeln(foo_in_chapel()); // will generate a call to foo_in_c

Frequently Asked Questions About Declaring External Routines¶

How do I pass a Chapel variable to an external routine that expects a pointer?

Today, the preferred way to do this is to declare the argument as having 'ref' intent. This should cause the Chapel compiler to pass a pointer to the argument. For example, given a C function:

void foo(double* x);

This could be called in Chapel using:

extern proc foo(ref x: real);

var myVal: real = 1.2;

foo(myVal);

Declaring External C Structs¶

External C struct types can be referred to within Chapel by prefixing a Chapel record definition with the extern keyword. For example, given an external C structure defined in foo.h called fltdbl:

typedef struct _fltdbl {
  float x;
  double y;
} fltdbl;

This type could be referred to within a Chapel program using:

extern record fltdbl {
}

Within the Chapel declaration, some or all of the fields from the C structure may be specified. Any fields that are not specified (or that cannot be specified because there is no equivalent Chapel type) cannot be referenced within the Chapel code. The order of these fields need not match the order they were specified within the C code. As an example, the following declaration would permit access to both fields x and y within variables of type fltdbl:

extern record fltdbl {
  var x: real(32);
  var y: real(64);
}

as would the following declaration:

extern record fltdbl {
  var y: real(64);
  var x: real(32);
}

Alternatively, the external declaration could only mention one of the fields. For example, the following declaration would permit field y to be accessed by a Chapel program, but not field x (though both would still be stored in any variable of type 'fltdbl').

extern record fltdbl {
  var y: real(64);
}

Alternatively, the external declaration can avoid mentioning any fields, which would permit a variable of that struct type to be passed between Chapel and C routines, but without permitting its fields to be accessed within the Chapel program:

extern record fltdbl {
}

A C header file containing the struct's definition in C must be specified on the chpl compiler command line or in a require statement as described in Expressing Dependencies.

To work with a C structure that is not typedef'd, just name the C type name after the extern keyword. In the example below, the struct stat type does not have a corresponding typedef in C. Therefore, we inform the Chapel compiler that the C name for the type is struct stat:

extern "struct stat" record chpl_stat_type {
  var st_size: off_t;
}

proc getFileSize(path:c_string) : int {
  extern proc stat(x: c_string, ref buf:chpl_stat_type): c_int;
  var buf: chpl_stat_type;

  if (chpl_stat_function(path, buf) == 0) {
    return buf.st_size;
  }
  return -1;
}

writeln(getFileSize("stat-example.chpl"));

Note that external record types only support assignment from records of matching type. In particular, Chapel's normal mechanisms that perform record assignment by field name are not used for external records. This restriction could be lifted in the future if considered useful to users.

Referring to External C Pointer-to-Structs ("extern classes")¶

You can also refer to an external C pointer-to-struct types by considering it to be an 'extern class' in Chapel. The declaration style is similar to that described above, it simply has different implications on the underlying C types.

As an example, given the declaration:

extern class D {
  var x: real;
}

The requirements on the corresponding C code are:

1. There must be a struct type that is typedef'd to have the name _D.
2. A pointer-to-_D type must be typedef'd to have the name D.
3. The _D struct type must contain a field named 'x' of type double. Like external records/structs, it may also contain other fields that will simply be ignored by the Chapel compiler.

Thus, the following C typedef would fulfill the external Chapel class declaration shown above:

typedef struct __D {
  double x;
  int y;
} _D, *D;

where the Chapel compiler would not know about the 'y' field and therefore could not refer to it or manipulate it.

Opaque Types¶

NOTE: This support may eventually be deprecated as other 'extern' features become increasingly flexible and robust.

You can refer to other external pointer-based C types that cannot be described in Chapel using the "opaque" keyword. As the name implies, these types are opaque as far as Chapel is concerned and cannot be used for operations other than argument passing and assignment (to/from other similarly opaque types).

For example, Chapel could be used to call an external C function that returns a pointer to a structure (that we can't or won't describe as an external class using the previous mechanism) as follows:

extern proc returnStructPtr(): opaque;

var structPtr: opaque = returnStructPtr();

However, because structPtr's type is opaque, it cannot be used for much apart from assigning it to other opaque variables of matching underlying type, or passing it back to an external C routine that expects a pointer-to-struct of that type:

extern proc operateOnStructPtr(ptr: opaque);

var copyOfStructPtr = structPtr;

operateOnStructPtr(structPtr);

#defines¶

The extern block functionality allows simple #defines to be used from Chapel code. Simple defines take no arguments and define an integer value or use another #define. For example:

extern {
 #define NEG_ONE (-1)
 #define MY_NUMBER (NEG_ONE)
}
writeln(MY_NUMBER);

However, it is easy to create #defines that are not supported because the Chapel compiler is unable to determine their expression type. In particular, any #define taking arguments is not currently supported. For example, the ADD definition below has a resulting expression type that cannot be determined by the Chapel compiler:

extern {
  #define ADD(x,y) ((x)+(y)) // NOT SUPPORTED
}
var x = ADD(1,2); // ERROR - ADD not defined in Chapel.

If the library you are using depends on these types, the current solution is to create inline functions to call the define with the appropriate argument types; for example:

extern {
  #define ADD(x,y) ((x)+(y))
  static inline int ADD_int_int(int x, int y) { return ADD(x,y); }
}
var sum = ADD_int_int(1,2);
writeln(sum);

Pointer Types¶

See the section Pointer and String Types above for background on how the Chapel programs can work with C pointer types. Any pointer type used in an extern block will be made visible to the Chapel program as c_ptr(T) or c_string (for const char* types).

For example:

extern {
  static void setItToOne(int* x) { *x = 1; }
  // will translate automatically into
  //  extern proc setItToOne(x:c_ptr(c_int));

  // The Chapel compiler can't know if X is used as an array,
  // if the argument will come from a Chapel variable, and in more general
  // cases, the best way to handle multiple levels of pointers. For example:
  static void setSpace(int** x) {
    static int space[10];
    *x = space;
  }
  // translates automatically into
  //  extern proc returnSpace( x:c_ptr(c_ptr(c_int)) );

  static void setString(const char** x) { *x = "My String"; }
}
var x:c_int;
setItToOne(c_ptrTo(x));

var space:c_ptr(c_int);
setSpace(c_ptrTo(space));

var str:c_string;
setString(c_ptrTo(str));
writeln(toString(str));

As you can see in this example, using the extern block might result in more calls to c_ptrTo() when using the generated extern declarations, because the compiler cannot automatically distinguish between several common cases (passing an array vs. passing in an argument by reference).

Example¶

Here's a more complete example:

module MyCModule {
  extern {
    static int foo(int b) { return b + 1; }
  }
}

writeln("Hello");
writeln(MyCModule.foo(7));

This prints out

Hello
8

which demonstrates full integration between Chapel and the C function it calls.

Parallel Safety¶

If external routines are to be called from Chapel within parallel execution contexts, they should be parallel-safe. As with internal Chapel routines, it is also the responsibility of the Chapel programmer to call external routines in a manner that preserves the integrity of objects accessible to those routines. Simply put, objects shared across Chapel tasks must be kept parallel-safe within Chapel.

Multiple Locales¶

Since the extern C code does not generally have any support for multiple locales, it is important to take care when using this code from multiple locales. Here are some things to be aware of:

• extern global variables will refer to a local version of that variable on each locale. These variables might become different if they are changed differently on different locales.
• c_ptr is always generated as a narrow pointer type (in other words, it does not encode the locale storing the pointed-to value - just an address). That means that passing a c_ptr from one locale to another and then using it on the second locale will probably result in a core dump.

Initialization¶

Chapel variables of extern type are not generally initialized automatically. Be sure to manually initialize Chapel variables of extern type.

In the future, we would like to support automatic zero initialization of such variables and a way to provide their default initializer.

Working with c_ptr¶

A c_ptr type can be constructed from a Chapel variable using the c_ptrTo() function; for example:

var i:c_int;
var i_ptr = c_ptrTo(i); // now i_ptr has type c_ptr(c_int) == int* in C

Since a C pointer might refer to a single variable or an array, the c_ptr type supports 0-based array indexing and dereferencing. In addition, it is possible to allocate and free space for one or more elements and return the result as a c_ptr. See the following example:

var cArray = c_calloc(c_int, 10);
for i in 0..#10 {
 cArray[i] = i:c_int;
}
// c_ptr.deref() always refers to the first element.
cArray.deref() = 17;
for i in 0..#10 {
 writeln(cArray[i]);
}
c_free(cArray);

Variables of type c_ptr can be compared against or set to nil.

Working with strings¶

If you need to call a C function and provide a Chapel string, you may need to convert the Chapel string to a C string first. Chapel string literals will automatically convert to C strings. A Chapel string variable can be converted using the c_str method.

myprint.h:

void myprint(const char* str);

myprint.c:

void myprint(const char* str) {
  printf("%s\n", str);
}

myprint.chpl:

extern proc myprint(str:c_string);

// string literal is automatically converted to a c_string
myprint("hello");

// a string variable must be converted with .c_str()
var s = "goodbye";
myprint(s.c_str());