.. default-domain:: chpl
.. module:: Curl
:synopsis: Low-level support for many network protocols with 'libcurl'.
Curl
====
**Usage**
.. code-block:: chapel
use Curl;
or
.. code-block:: chapel
import Curl;
Low-level support for many network protocols with 'libcurl'.
This module provides support for libcurl, enabling Chapel programs
to work with many network protocols. This module is a low-level C
wrapper. For a simpler interface not specific to Curl, please see
:mod:`URL`.
The `curl homepage `_ describes libcurl thus::
libcurl is a free and easy-to-use client-side URL transfer library, supporting
DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. libcurl supports
SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload,
proxies, cookies, user+password authentication (Basic, Digest, NTLM,
Negotiate, Kerberos), file transfer resume, http proxy tunneling and more!
Dependencies
------------
The Curl module in Chapel is dependent on libcurl being installed and available
to Chapel programs.
Using Curl Support in Chapel
----------------------------
Simple uses of Curl work through the generic :mod:`URL` module. This module
allows a URL to be opened as a :record:`IO.fileReader` or
`:record:`IO.fileWriter`.
.. code-block:: chapel
use URL;
var urlreader = openUrlReader("http://example.com");
var str:bytes;
// Output each line read from the URL to stdout
while(urlreader.readLine(str)) {
write(str);
}
The Curl module includes ``require`` statements to include the Curl header and
library, so the above example can be compiled simply with:
.. code-block:: sh
chpl curl-example.chpl
More complex uses of Curl can make use of the extern types provided in this
module. In that event, please see the libcurl documentation for how to use
these functions. Note that it is possible to use :proc:`setopt` to adjust the
settings for a the result of :proc:`URL.openUrlReader` or
:proc:`URL.openUrlWriter` before starting the connection. The Curl module
also exposes the basic libcurl API. This can be used to make an HTTP
POST or PATCH request as shown below.
Many times when we are connecting to a URL (FTP, IMAP, SMTP, HTTP) we have to
give extra information to the Curl handle. This is done via the setopt()
interface. Documentation on the various options, as well as the functions
that are referenced below can be found `here
`_
Due to the large number of constants in the Curl interface, and the fact that
these constants vary for different versions of Curl, this module does not
provide extern declarations for all constants. However these are trivial to add
to your own programs. For example, the below example declares
``CURLOPT_VERBOSE`` as a ``CURLoption`` like this:
.. code-block:: chapel
extern const CURLOPT_VERBOSE:CURLoption;
Here is a full program enabling verbose output from Curl while downloading:
.. code-block:: chapel
// This example changes the Curl options before connecting
use URL;
use Curl;
var reader = openUrlReader("https://example.com");
var str:bytes;
// Set verbose output from curl
extern const CURLOPT_VERBOSE:CURLoption;
Curl.setopt(reader, CURLOPT_VERBOSE, true);
// now read into the bytes
reader.readAll(str);
writeln(str);
reader.close();
Here is an example program using lower-level libcurl functions to
issue a POST request:
.. code-block:: chapel
// This example uses the curl_easy_ interface from libcurl
// to POST some json data.
use CTypes;
import Curl;
extern const CURLOPT_CUSTOMREQUEST: Curl.CURLoption;
extern const CURLOPT_HTTPHEADER: Curl.CURLoption;
extern const CURLOPT_POSTFIELDS: Curl.CURLoption;
extern const CURLOPT_URL: Curl.CURLoption;
extern const CURLOPT_WRITEFUNCTION: Curl.CURLoption;
// Called with the contents of the server's response; does nothing with it.
// Else libcurl writes it to stdout.
proc null_write_callback(ptr: c_ptr(c_char), size: c_size_t, nmemb: c_size_t, userdata: c_ptr(void)) {
return size * nmemb;
}
var curl = Curl.curl_easy_init();
var args = new Curl.slist();
args.append("Accept: application/json");
args.append("Content-Type: application/json");
Curl.curl_easy_setopt(curl, CURLOPT_HTTPHEADER, args);
var jsonPayload = '{"foo": "bar"}';
Curl.curl_easy_setopt(curl, CURLOPT_POSTFIELDS, jsonPayload);
Curl.curl_easy_setopt(curl, CURLOPT_URL, 'http://localhost:3000/posts');
Curl.curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, 'POST');
Curl.curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, c_ptrTo(null_write_callback):c_ptr(void));
var ret = Curl.curl_easy_perform(curl);
args.free();
Curl.curl_easy_cleanup(curl);
Curl Support Types and Functions
--------------------------------
.. function:: proc getCurlHandle(ch): c_ptr(CURL) throws where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)
Returns the ``CURL`` handle connected to a channel opened with
:proc:`URL.openUrlReader` or :proc:`URL.openUrlWriter`.
.. function:: proc setopt(ch, opt: c_int, arg): bool throws where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)
This function is the equivalent to the
`curl_easy_setopt `_
function in libcurl. It sets information on the curl file handle
attached to a channel that can change libcurl's behavior.
:arg ch: a curl channel created with openUrlReader or openUrlWriter
:arg opt: the curl option to set.
:arg arg: the value to set the curl option specified by opt.
:type arg: `int`, `string`, `bool`, or `slist`
.. function:: proc setopt(ch, args ...?k) throws where isSubtype(ch.type, fileReader) || isSubtype(ch.type, fileWriter)
Set curl options on a curl file attached to a channel.
For example, you might do:
.. code-block:: chapel
extern const CURLOPT_USERNAME:CURLoption;
extern const CURLOPT_PASSWORD:CURLoption;
setopt(curlfile, (CURLOPT_USERNAME, username),
(CURLOPT_PASSWORD, password));
:arg ch: a :record:`IO.fileReader` or :record:`IO.fileWriter`
:arg args: any number of tuples of the form (curl_option, value).
This function will call ``setopt`` on each pair in turn.
.. record:: slist
A linked list of strings used in many curl setopt calls. This type
corresponds to the libcurl type curl_slist.
.. note::
Memory in the list is not automatically managed. It is necessary
to call :proc:`slist.free` to free the slist when it is no longer used.
.. method:: proc ref slist.append(str: string) throws
Append the string argument to an slist. This function is the same
as calling
`curl_slist_append `_
This function halts if an error is encountered. Future versions will
support returning an error code instead of halting.
:arg str: a string argument to append
.. method:: proc slist.free()
Free an slist. Chapel programs must call this function after using an slist.
Programs must ensure that there are no ongoing connections using
this slist when it is freed.
.. data:: const CURLE_OK: c_int
Successful result for CURL easy API calls
.. data:: const CURLM_OK: c_int
Successful result for CURL multi API calls
.. type:: type CURL
A CURL easy handle. Most CURL functions accept a ``c_ptr(CURL)``.
.. type:: type CURLM
A CURL multi handle.
.. record:: curl_slist
A CURL string list
.. type:: type CURLoption = c_int
CURLoption identifies options for ``curl_easy_setopt``.
.. type:: type CURLcode = c_int
The return type of CURL easy API functions
.. type:: type CURLMcode = c_int
The return type of CURL multi API functions
.. type:: type CURLINFO = c_int
CURLINFO identifies info to get with ``curl_easy_getinfo``
.. type:: type curl_off_t = int(64)
curl_off_t is a file offset used by the CURL library
.. function:: proc curl_easy_init(): c_ptr(CURL)
See https://curl.haxx.se/libcurl/c/curl_easy_init.html
.. function:: proc curl_easy_getinfo(handle: c_ptr(CURL), info: CURLINFO, arg): CURLcode
See https://curl.haxx.se/libcurl/c/curl_easy_getinfo.html
.. function:: proc curl_easy_setopt(handle: c_ptr(CURL), option: CURLoption, arg): CURLcode
See https://curl.haxx.se/libcurl/c/curl_easy_setopt.html
Handles Chapel arg types appropriate to each option:
For options accepting a C long, accepts integral and boolean types.
For options accepting a C pointer, accepts c_ptr and c_ptr(void).
For options accepting a C string, accepts string and bytes and c_ptr(char).
For options accepting a libcurl slist, accepts Curl.slist and c_ptr(slist).
For options accepting a callback function, accepts c_ptr(void).
For options accepting an offset, accepts integral types.
Returns the libcurl error code CURLE_OK on success, other codes on error.
.. function:: proc curl_easy_setopt_long(curl: c_ptr(CURL), option: CURLoption, arg: c_long): CURLcode
Helper function for ``curl_easy_setopt`` when passing a numeric argument
.. function:: proc curl_easy_setopt_ptr(curl: c_ptr(CURL), option: CURLoption, arg: c_ptr(void)): CURLcode
Helper function for ``curl_easy_setopt`` when passing a pointer argument
.. function:: proc curl_easy_setopt_offset(curl: c_ptr(CURL), option: CURLoption, offset: int(64)): CURLcode
Helper function for ``curl_easy_setopt`` when passing an offset argument
.. function:: proc curl_easy_getinfo_ptr(curl: c_ptr(CURL), info: CURLINFO, arg: c_ptr(void)): CURLcode
Helper function for ``curl_easy_getinfo`` when passing a pointer argument.
Generally this is a pointer to the value to be set.
.. function:: proc curl_easy_perform(curl: c_ptr(CURL)): CURLcode
See https://curl.haxx.se/libcurl/c/curl_easy_perform.html
.. function:: proc curl_easy_cleanup(curl: c_ptr(CURL)): void
See https://curl.haxx.se/libcurl/c/curl_easy_cleanup.html
.. function:: proc curl_easy_pause(curl: c_ptr(CURL), bitmask: c_int): CURLcode
See https://curl.haxx.se/libcurl/c/curl_easy_pause.html
.. function:: proc curl_multi_init(): c_ptr(CURLM)
See https://curl.haxx.se/libcurl/c/curl_multi_init.html
.. function:: proc curl_multi_add_handle(curlm: c_ptr(CURLM), curl: c_ptr(CURL)): CURLMcode
See https://curl.haxx.se/libcurl/c/curl_multi_add_handle.html
.. function:: proc curl_multi_timeout(curlm: c_ptr(CURLM), ref timeout: c_long): CURLMcode
See https://curl.haxx.se/libcurl/c/curl_multi_timeout.html
.. function:: proc curl_multi_fdset(curlm: c_ptr(CURLM), read_fd_set: c_ptr(fd_set), write_fd_set: c_ptr(fd_set), exc_fd_set: c_ptr(fd_set), ref max_fd: c_int): CURLMcode
See https://curl.haxx.se/libcurl/c/curl_multi_fdset.html
.. function:: proc curl_multi_perform(curlm: c_ptr(CURLM), ref running_handles): CURLMcode
See https://curl.haxx.se/libcurl/c/curl_multi_perform.html
.. function:: proc curl_multi_remove_handle(curlm: c_ptr(CURLM), curl: c_ptr(CURL)): CURLMcode
See https://curl.haxx.se/libcurl/c/curl_multi_remove_handle.html
.. function:: proc curl_multi_cleanup(curlm: c_ptr(CURLM)): CURLcode
See https://curl.haxx.se/libcurl/c/curl_multi_cleanup.html
.. function:: proc curl_slist_append(csl: c_ptr(curl_slist), char: c_ptrConst(c_char)): c_ptr(curl_slist)
See https://curl.haxx.se/libcurl/c/curl_slist_append.html
.. function:: proc curl_slist_free_all(csl: c_ptr(curl_slist))
See https://curl.haxx.se/libcurl/c/curl_slist_free_all.html