Curl¶

Usage

use Curl;

or

import Curl;

Low-level support for many network protocols with ‘libcurl’.

This module provides support for libcurl, enabling Chapel programs to work with many network protocols. This module is a low-level C wrapper. For a simpler interface not specific to Curl, please see URL.

The curl homepage describes libcurl thus:

libcurl is a free and easy-to-use client-side URL transfer library, supporting
DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP.  libcurl supports
SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload,
proxies, cookies, user+password authentication (Basic, Digest, NTLM,
Negotiate, Kerberos), file transfer resume, http proxy tunneling and more!

Dependencies¶

The Curl module in Chapel is dependent on libcurl being installed and available to Chapel programs.

Using Curl Support in Chapel¶

Simple uses of Curl work through the generic URL module. This module allows a URL to be opened as a IO.fileReader or :record:`IO.fileWriter.

use URL;
var urlreader = openUrlReader("http://example.com");
var str:bytes;
// Output each line read from the URL to stdout
while(urlreader.readLine(str)) {
  write(str);
}

The Curl module includes require statements to include the Curl header and library, so the above example can be compiled simply with:

chpl curl-example.chpl

More complex uses of Curl can make use of the extern types provided in this module. In that event, please see the libcurl documentation for how to use these functions. Note that it is possible to use setopt to adjust the settings for a the result of URL.openUrlReader or URL.openUrlWriter before starting the connection. The Curl module also exposes the basic libcurl API. This can be used to make an HTTP POST or PATCH request as shown below.

Many times when we are connecting to a URL (FTP, IMAP, SMTP, HTTP) we have to give extra information to the Curl handle. This is done via the setopt() interface. Documentation on the various options, as well as the functions that are referenced below can be found here

Due to the large number of constants in the Curl interface, and the fact that these constants vary for different versions of Curl, this module does not provide extern declarations for all constants. However these are trivial to add to your own programs. For example, the below example declares CURLOPT_VERBOSE as a CURLoption like this:

extern const CURLOPT_VERBOSE:CURLoption;

Here is a full program enabling verbose output from Curl while downloading:

// This example changes the Curl options before connecting
use URL;
use Curl;
var reader = openUrlReader("https://example.com");
var str:bytes;
// Set verbose output from curl
extern const CURLOPT_VERBOSE:CURLoption;
Curl.setopt(reader, CURLOPT_VERBOSE, true);

// now read into the bytes
reader.readAll(str);
writeln(str);
reader.close();

Here is an example program using lower-level libcurl functions to issue a POST request:

// This example uses the curl_easy_ interface from libcurl
// to POST some json data.
use CTypes;
import Curl;

extern const CURLOPT_CUSTOMREQUEST: Curl.CURLoption;
extern const CURLOPT_HTTPHEADER: Curl.CURLoption;
extern const CURLOPT_POSTFIELDS: Curl.CURLoption;
extern const CURLOPT_URL: Curl.CURLoption;
extern const CURLOPT_WRITEFUNCTION: Curl.CURLoption;

// Called with the contents of the server's response; does nothing with it.
// Else libcurl writes it to stdout.
proc null_write_callback(ptr: c_ptr(c_char), size: c_size_t, nmemb: c_size_t, userdata: c_ptr(void)) {
  return size * nmemb;
}

var curl = Curl.curl_easy_init();

var args = new Curl.slist();
args.append("Accept: application/json");
args.append("Content-Type: application/json");
Curl.curl_easy_setopt(curl, CURLOPT_HTTPHEADER, args);

var jsonPayload = '{"foo": "bar"}';
Curl.curl_easy_setopt(curl, CURLOPT_POSTFIELDS, jsonPayload);

Curl.curl_easy_setopt(curl, CURLOPT_URL, 'http://localhost:3000/posts');
Curl.curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, 'POST');
Curl.curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, c_ptrTo(null_write_callback):c_ptr(void));

var ret = Curl.curl_easy_perform(curl);

args.free();
Curl.curl_easy_cleanup(curl);

Curl Support Types and Functions¶

proc getCurlHandle(ch) : c_ptr(CURL) throws where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)¶: Returns the CURL handle connected to a channel opened with URL.openUrlReader or URL.openUrlWriter.

proc setopt(ch, opt: c_int, arg) : bool throws where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)¶

This function is the equivalent to the curl_easy_setopt function in libcurl. It sets information on the curl file handle attached to a channel that can change libcurl’s behavior.

Arguments:

ch – a curl channel created with openUrlReader or openUrlWriter
opt – the curl option to set.
arg : int, string, bool, or slist – the value to set the curl option specified by opt.

proc setopt(ch, args ...?k) throws where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)

Set curl options on a curl file attached to a channel.

For example, you might do:

extern const CURLOPT_USERNAME:CURLoption;
extern const CURLOPT_PASSWORD:CURLoption;

setopt(curlfile, (CURLOPT_USERNAME, username),
                 (CURLOPT_PASSWORD, password));

Arguments:

ch – a IO.fileReader or IO.fileWriter
args – any number of tuples of the form (curl_option, value). This function will call setopt on each pair in turn.

record slist¶: A linked list of strings used in many curl setopt calls. This type corresponds to the libcurl type curl_slist.

Note

Memory in the list is not automatically managed. It is necessary to call slist.free to free the slist when it is no longer used.

proc ref slist.append(str: string) throws¶

Append the string argument to an slist. This function is the same as calling curl_slist_append

This function halts if an error is encountered. Future versions will support returning an error code instead of halting.

Arguments:: str – a string argument to append

proc slist.free()¶: Free an slist. Chapel programs must call this function after using an slist. Programs must ensure that there are no ongoing connections using this slist when it is freed.

const CURLE_OK : c_int¶: Successful result for CURL easy API calls

const CURLM_OK : c_int¶: Successful result for CURL multi API calls

type CURL¶: A CURL easy handle. Most CURL functions accept a c_ptr(CURL).

type CURLM¶: A CURL multi handle.

record curl_slist¶: A CURL string list

type CURLoption = c_int¶: CURLoption identifies options for curl_easy_setopt.

type CURLcode = c_int¶: The return type of CURL easy API functions

type CURLMcode = c_int¶: The return type of CURL multi API functions

type CURLINFO = c_int¶: CURLINFO identifies info to get with curl_easy_getinfo

type curl_off_t = int(64)¶: curl_off_t is a file offset used by the CURL library

proc curl_easy_init() : c_ptr(CURL)¶: See https://curl.haxx.se/libcurl/c/curl_easy_init.html

proc curl_easy_getinfo(handle: c_ptr(CURL), info: CURLINFO, arg) : CURLcode¶: See https://curl.haxx.se/libcurl/c/curl_easy_getinfo.html

proc curl_easy_setopt(handle: c_ptr(CURL), option: CURLoption, arg) : CURLcode¶

See https://curl.haxx.se/libcurl/c/curl_easy_setopt.html

Handles Chapel arg types appropriate to each option:

For options accepting a C long, accepts integral and boolean types.

For options accepting a C pointer, accepts c_ptr and c_ptr(void).

For options accepting a C string, accepts string and bytes and c_ptr(char).

For options accepting a libcurl slist, accepts Curl.slist and c_ptr(slist).

For options accepting a callback function, accepts c_ptr(void).

For options accepting an offset, accepts integral types.

Returns the libcurl error code CURLE_OK on success, other codes on error.

proc curl_easy_setopt_long(curl: c_ptr(CURL), option: CURLoption, arg: c_long) : CURLcode¶: Helper function for curl_easy_setopt when passing a numeric argument

proc curl_easy_setopt_ptr(curl: c_ptr(CURL), option: CURLoption, arg: c_ptr(void)) : CURLcode¶: Helper function for curl_easy_setopt when passing a pointer argument

proc curl_easy_setopt_offset(curl: c_ptr(CURL), option: CURLoption, offset: int(64)) : CURLcode¶: Helper function for curl_easy_setopt when passing an offset argument

proc curl_easy_getinfo_ptr(curl: c_ptr(CURL), info: CURLINFO, arg: c_ptr(void)) : CURLcode¶: Helper function for curl_easy_getinfo when passing a pointer argument. Generally this is a pointer to the value to be set.

proc curl_easy_perform(curl: c_ptr(CURL)) : CURLcode¶: See https://curl.haxx.se/libcurl/c/curl_easy_perform.html

proc curl_easy_cleanup(curl: c_ptr(CURL)) : void¶: See https://curl.haxx.se/libcurl/c/curl_easy_cleanup.html

proc curl_easy_pause(curl: c_ptr(CURL), bitmask: c_int) : CURLcode¶: See https://curl.haxx.se/libcurl/c/curl_easy_pause.html

proc curl_multi_init() : c_ptr(CURLM)¶: See https://curl.haxx.se/libcurl/c/curl_multi_init.html

proc curl_multi_add_handle(curlm: c_ptr(CURLM), curl: c_ptr(CURL)) : CURLMcode¶: See https://curl.haxx.se/libcurl/c/curl_multi_add_handle.html

proc curl_multi_timeout(curlm: c_ptr(CURLM), ref timeout: c_long) : CURLMcode¶: See https://curl.haxx.se/libcurl/c/curl_multi_timeout.html

proc curl_multi_fdset(curlm: c_ptr(CURLM), read_fd_set: c_ptr(fd_set), write_fd_set: c_ptr(fd_set), exc_fd_set: c_ptr(fd_set), ref max_fd: c_int) : CURLMcode¶: See https://curl.haxx.se/libcurl/c/curl_multi_fdset.html

proc curl_multi_perform(curlm: c_ptr(CURLM), ref running_handles) : CURLMcode¶: See https://curl.haxx.se/libcurl/c/curl_multi_perform.html

proc curl_multi_remove_handle(curlm: c_ptr(CURLM), curl: c_ptr(CURL)) : CURLMcode¶: See https://curl.haxx.se/libcurl/c/curl_multi_remove_handle.html

proc curl_multi_cleanup(curlm: c_ptr(CURLM)) : CURLcode¶: See https://curl.haxx.se/libcurl/c/curl_multi_cleanup.html

proc curl_slist_append(csl: c_ptr(curl_slist), char: c_ptrConst(c_char)) : c_ptr(curl_slist)¶: See https://curl.haxx.se/libcurl/c/curl_slist_append.html

proc curl_slist_free_all(csl: c_ptr(curl_slist))¶: See https://curl.haxx.se/libcurl/c/curl_slist_free_all.html