Curl

Usage

use Curl;

or

import Curl;

Low-level support for many network protocols with ‘libcurl’.

This module provides support for libcurl, enabling Chapel programs to work with many network protocols. This module is a low-level C wrapper. For a simpler interface not specific to Curl, please see URL.

The curl homepage describes libcurl thus:

libcurl is a free and easy-to-use client-side URL transfer library, supporting
DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP.  libcurl supports
SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload,
proxies, cookies, user+password authentication (Basic, Digest, NTLM,
Negotiate, Kerberos), file transfer resume, http proxy tunneling and more!

Dependencies

The Curl module in Chapel is dependent on libcurl being installed and available to Chapel programs.

Using Curl Support in Chapel

Simple uses of Curl work through the generic URL module. This module allows a URL to be opened as a IO.fileReader or :record:`IO.fileWriter.

use URL;
var urlreader = openUrlReader("http://example.com");
var str:bytes;
// Output each line read from the URL to stdout
while(urlreader.readLine(str)) {
  write(str);
}

The Curl module includes require statements to include the Curl header and library, so the above example can be compiled simply with:

chpl curl-example.chpl

More complex uses of Curl can make use of the extern types provided in this module. In that event, please see the libcurl documentation for how to use these functions. Note that it is possible to use setopt to adjust the settings for a the result of URL.openUrlReader or URL.openUrlWriter before starting the connection. The Curl module also exposes the basic libcurl API. This can be used to make an HTTP POST or PATCH request as shown below.

Many times when we are connecting to a URL (FTP, IMAP, SMTP, HTTP) we have to give extra information to the Curl handle. This is done via the setopt() interface. Documentation on the various options, as well as the functions that are referenced below can be found here

Due to the large number of constants in the Curl interface, and the fact that these constants vary for different versions of Curl, this module does not provide extern declarations for all constants. However these are trivial to add to your own programs. For example, the below example declares CURLOPT_VERBOSE as a CURLoption like this:

extern const CURLOPT_VERBOSE:CURLoption;

Here is a full program enabling verbose output from Curl while downloading:

// This example changes the Curl options before connecting
use URL;
use Curl;
var reader = openUrlReader("https://example.com");
var str:bytes;
// Set verbose output from curl
extern const CURLOPT_VERBOSE:CURLoption;
Curl.setopt(reader, CURLOPT_VERBOSE, true);

// now read into the bytes
reader.readAll(str);
writeln(str);
reader.close();

Here is an example program using lower-level libcurl functions to issue a POST request:

// This example uses the curl_easy_ interface from libcurl
// to POST some json data.
use CTypes;
import Curl;

extern const CURLOPT_CUSTOMREQUEST: Curl.CURLoption;
extern const CURLOPT_HTTPHEADER: Curl.CURLoption;
extern const CURLOPT_POSTFIELDS: Curl.CURLoption;
extern const CURLOPT_URL: Curl.CURLoption;
extern const CURLOPT_WRITEFUNCTION: Curl.CURLoption;

// Called with the contents of the server's response; does nothing with it.
// Else libcurl writes it to stdout.
proc null_write_callback(ptr: c_ptr(c_char), size: c_size_t, nmemb: c_size_t, userdata: c_ptr(void)) {
  return size * nmemb;
}

var curl = Curl.curl_easy_init();

var args = new Curl.slist();
args.append("Accept: application/json");
args.append("Content-Type: application/json");
Curl.curl_easy_setopt(curl, CURLOPT_HTTPHEADER, args);

var jsonPayload = '{"foo": "bar"}';
Curl.curl_easy_setopt(curl, CURLOPT_POSTFIELDS, jsonPayload);

Curl.curl_easy_setopt(curl, CURLOPT_URL, 'http://localhost:3000/posts');
Curl.curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, 'POST');
Curl.curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, c_ptrTo(null_write_callback):c_ptr(void));

var ret = Curl.curl_easy_perform(curl);

args.free();
Curl.curl_easy_cleanup(curl);

Curl Support Types and Functions

proc getCurlHandle(ch): c_ptr(CURL) throws  where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)

Returns the CURL handle connected to a channel opened with URL.openUrlReader or URL.openUrlWriter.

proc setopt(ch, opt: c_int, arg): bool throws  where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)

This function is the equivalent to the curl_easy_setopt function in libcurl. It sets information on the curl file handle attached to a channel that can change libcurl’s behavior.

Arguments

• ch – a curl channel created with openUrlReader or openUrlWriter

• opt – the curl option to set.

• arg : int, string, bool, or slist – the value to set the curl option specified by opt.

proc setopt(ch, args ...?k) throws  where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)

Set curl options on a curl file attached to a channel.

For example, you might do:

extern const CURLOPT_USERNAME:CURLoption;
extern const CURLOPT_PASSWORD:CURLoption;

setopt(curlfile, (CURLOPT_USERNAME, username),
                 (CURLOPT_PASSWORD, password));

Arguments

• ch – a IO.fileReader or IO.fileWriter

• args – any number of tuples of the form (curl_option, value). This function will call setopt on each pair in turn.

record slist

A linked list of strings used in many curl setopt calls. This type corresponds to the libcurl type curl_slist.

Note

Memory in the list is not automatically managed. It is necessary to call slist.free to free the slist when it is no longer used.

proc ref slist.append(str: string) throws

Append the string argument to an slist. This function is the same as calling curl_slist_append

This function halts if an error is encountered. Future versions will support returning an error code instead of halting.

Arguments

str – a string argument to append

proc slist.free()

Free an slist. Chapel programs must call this function after using an slist. Programs must ensure that there are no ongoing connections using this slist when it is freed.

const CURLE_OK: c_int

Successful result for CURL easy API calls

const CURLM_OK: c_int

Successful result for CURL multi API calls

type CURL

A CURL easy handle. Most CURL functions accept a c_ptr(CURL).

type CURLM

A CURL multi handle.

record curl_slist

A CURL string list

type CURLoption = c_int

CURLoption identifies options for curl_easy_setopt.

type CURLcode = c_int

The return type of CURL easy API functions

type CURLMcode = c_int

The return type of CURL multi API functions

type CURLINFO = c_int

CURLINFO identifies info to get with curl_easy_getinfo

type curl_off_t = int(64)

curl_off_t is a file offset used by the CURL library

proc curl_easy_init(): c_ptr(CURL)

See https://curl.haxx.se/libcurl/c/curl_easy_init.html

proc curl_easy_getinfo(handle: c_ptr(CURL), info: CURLINFO, arg): CURLcode

See https://curl.haxx.se/libcurl/c/curl_easy_getinfo.html

proc curl_easy_setopt(handle: c_ptr(CURL), option: CURLoption, arg): CURLcode

See https://curl.haxx.se/libcurl/c/curl_easy_setopt.html

Handles Chapel arg types appropriate to each option:

For options accepting a C long, accepts integral and boolean types.

For options accepting a C pointer, accepts c_ptr and c_ptr(void).

For options accepting a C string, accepts string and bytes and c_ptr(char).

For options accepting a libcurl slist, accepts Curl.slist and c_ptr(slist).

For options accepting a callback function, accepts c_ptr(void).

For options accepting an offset, accepts integral types.

Returns the libcurl error code CURLE_OK on success, other codes on error.

proc curl_easy_setopt_long(curl: c_ptr(CURL), option: CURLoption, arg: c_long): CURLcode

Helper function for curl_easy_setopt when passing a numeric argument

proc curl_easy_setopt_ptr(curl: c_ptr(CURL), option: CURLoption, arg: c_ptr(void)): CURLcode

Helper function for curl_easy_setopt when passing a pointer argument

proc curl_easy_setopt_offset(curl: c_ptr(CURL), option: CURLoption, offset: int(64)): CURLcode

Helper function for curl_easy_setopt when passing an offset argument

proc curl_easy_getinfo_ptr(curl: c_ptr(CURL), info: CURLINFO, arg: c_ptr(void)): CURLcode

Helper function for curl_easy_getinfo when passing a pointer argument. Generally this is a pointer to the value to be set.

proc curl_easy_perform(curl: c_ptr(CURL)): CURLcode

See https://curl.haxx.se/libcurl/c/curl_easy_perform.html

proc curl_easy_cleanup(curl: c_ptr(CURL)): void

See https://curl.haxx.se/libcurl/c/curl_easy_cleanup.html

proc curl_easy_pause(curl: c_ptr(CURL), bitmask: c_int): CURLcode

See https://curl.haxx.se/libcurl/c/curl_easy_pause.html

proc curl_multi_init(): c_ptr(CURLM)

See https://curl.haxx.se/libcurl/c/curl_multi_init.html

proc curl_multi_add_handle(curlm: c_ptr(CURLM), curl: c_ptr(CURL)): CURLMcode

See https://curl.haxx.se/libcurl/c/curl_multi_add_handle.html

proc curl_multi_timeout(curlm: c_ptr(CURLM), ref timeout: c_long): CURLMcode

See https://curl.haxx.se/libcurl/c/curl_multi_timeout.html

proc curl_multi_fdset(curlm: c_ptr(CURLM), read_fd_set: c_ptr(fd_set), write_fd_set: c_ptr(fd_set), exc_fd_set: c_ptr(fd_set), ref max_fd: c_int): CURLMcode

See https://curl.haxx.se/libcurl/c/curl_multi_fdset.html

proc curl_multi_perform(curlm: c_ptr(CURLM), ref running_handles): CURLMcode

See https://curl.haxx.se/libcurl/c/curl_multi_perform.html

proc curl_multi_remove_handle(curlm: c_ptr(CURLM), curl: c_ptr(CURL)): CURLMcode

See https://curl.haxx.se/libcurl/c/curl_multi_remove_handle.html

proc curl_multi_cleanup(curlm: c_ptr(CURLM)): CURLcode

See https://curl.haxx.se/libcurl/c/curl_multi_cleanup.html

proc curl_slist_append(csl: c_ptr(curl_slist), char: c_ptrConst(c_char)): c_ptr(curl_slist)

See https://curl.haxx.se/libcurl/c/curl_slist_append.html

proc curl_slist_free_all(csl: c_ptr(curl_slist))

See https://curl.haxx.se/libcurl/c/curl_slist_free_all.html