Subprocess

Usage

use Subprocess;

or

import Subprocess;

Support launching and interacting with other programs.

Using functions in this module, one can create a subprocess and possibly capture its output. It is also possible to provide input to a subprocess.

To start a subprocess, use spawn or spawnshell. To wait for the subprocess process to finish, use the subprocess.wait or subprocess.communicate functions.

This example program produces a listing of files in the current directory with names that begin with test. by using the ls command. The output will be mixed in with the Chapel program’s output.

use Subprocess;

var sub = spawn(["ls", "test.*"]);
sub.wait();

This version also runs the ls command but uses a pipe to read the output from the ls command.

use Subprocess;

var sub = spawn(["ls", "test.*"], stdout=pipeStyle.pipe);

var line:string;
while sub.stdout.readLine(line) {
  write("ls returned: ", line);
}

sub.wait();

Here is an example program that provides input to a subprocess in addition to capturing its output. This version uses the cat command, which just prints back its input.

use Subprocess;

var sub = spawn(["cat"], stdin=pipeStyle.bufferAll, stdout=pipeStyle.pipe);

sub.stdin.writeln("Hello");
sub.stdin.writeln("World");

sub.communicate();

var line:string;
while sub.stdout.readLine(line) {
  write("Got line: ", line);
}

// prints out:
// Got line: Hello
// Got line: World

Here is a final example in which the Chapel program uses 2 tasks to work with a subprocess. One task is producing data and the other task is consuming it.

use Subprocess;

var input = ["a", "b", "c"];

var sub = spawn(["cat"], stdin=pipeStyle.pipe, stdout=pipeStyle.pipe);
cobegin {
  {
    // one task writes data to the subprocess
    for x in input {
      sub.stdin.writeln(x);
    }
    // this close is important; otherwise the other task blocks forever
    sub.stdin.close();
  }

  {
    var line:string;
    while sub.stdout.readln(line) {
      writeln("Got line ", line);
    }
  }
}
sub.wait();

// prints out:
// Got line: a
// Got line: b
// Got line: c

Note

Creating a subprocess that uses pipeStyle.pipe to provide input or capture output does not work when using the ugni communications layer with hugepages enabled and when using more than one locale. In this circumstance, the program will halt with an error message. These scenarios do work when using GASNet instead of the ugni layer.

The Deprecated ‘kind’ Field

Prior to the 1.32 release, configuring a subprocess record to use binary IO required using the kind field (of type iokind) to enable binary IO and set the desired endianness. The 1.32 release introduced serializers, and deprecated use of the iokind type in favor of using Serializers and Deserializers to configure a given fileWriter or fileReader for a desired format.

Users may now create aliases of stdin and stdout with different serialization formatting by using the withSerializer and withDeserializer methods. For example, consider the following program that writes the numbers 1 through 10 in binary to the hexdump utility:

use IO, Subprocess;

var sub = spawn(["hexdump", "-C"], stdin=pipeStyle.pipe, stdout=pipeStyle.pipe);

// Use 'withSerializer' to create a binary-serializing alias of 'sub.stdin'
var bin = sub.stdin.withSerializer(binarySerializer);

for i in 1..10 do bin.write(i:uint(8));

sub.communicate();

var line : string;
while sub.stdout.readLine(line) do
  write(line);

This program prints:

00000000  01 02 03 04 05 06 07 08  09 0a                    |..........|
0000000a

Please refer to binarySerializer and binaryDeserializer for more information on their supported format.

record subprocess

This record represents a subprocess.

Note that the subprocess will not be waited for if this record goes out of scope. Channels opened to communicate with the subprocess will be closed if the record goes out of scope, however.

Generally, it is important to call subprocess.wait to wait for the process to complete. If the parent process is using pipes to communicate with the subprocess, the parent process may call subprocess.close in order to close the pipes and free any buffers. Such calls are generally not needed since the channels will be closed when the subprocess record is automatically destroyed.

param kind: iokind = iokind.dynamic

Warning

the ‘kind’ field is deprecated, please use Serializers or Deserializers with stdin, stdout, and stderr channels instead. Channels can be configured with the ‘fileWriter.withSerializer’ or ‘fileReader.withDeserializer’ methods.

The kind of a subprocess is used to create the types for stdin, stdout, or stderr.

param locking: bool

As with kind, this value is used to create the types for any channels that are necessary.

var pid: int(64)

The Process ID number of the spawned process

var running: bool

false if this library knows that the subprocess is not running

var exitCode: int

The exit status from the subprocess, or possibly a value >= 256 if there was en error when creating the subprocess

proc stdin throws

Access the stdin pipe for the subprocess. The parent process can write to the subprocess through this pipe if the subprocess was created with stdin=pipeStyle.pipe.

Throws

SystemError – If the subprocess does not have a stdin pipe open.

proc stdout throws

Access the stdout pipe for the subprocess. The parent process can read from the subprocess through this pipe if the subprocess was created with stdout=pipeStyle.pipe.

Throws

SystemError – If the subprocess does not have a stdout pipe open.

proc stderr throws

Access the stderr pipe for the subprocess. The parent process can read from the subprocess through this pipe if the subprocess was created with stderr=pipeStyle.pipe.

Throws

SystemError – If the subprocess does not have a stderr pipe open.

enum pipeStyle { forward, close, pipe, stdout, bufferAll }

Styles of piping to use in a subprocess.

enum constant forward

forward indicates that the child process should inherit the stdin/stdout/stderr of this process.

enum constant close

close indicates that the child process should close its stdin/stdout/stderr.

enum constant pipe

pipe indicates that the spawn operation should set up a pipe between the parent process and the child process so that the parent process can provide input to the child process or capture its output.

enum constant stdout

stdout indicates that the stderr stream of the child process should be forwarded to its stdout stream.

enum constant bufferAll

bufferAll is the same as pipe, but when used for stdin causes all data to be buffered and sent on the communicate() call. This avoids certain deadlock scenarios where stdout or stderr are pipe. In particular, without bufferAll, the sub-process might block on writing output which will not be consumed until the communicate() call.

proc spawn(args: [] string, env: [] string = Subprocess.empty_env, executable = "", stdin: ?t = pipeStyle.forward, stdout: ?u = pipeStyle.forward, stderr: ?v = pipeStyle.forward, param locking = true) throws

Create a subprocess.

Arguments

  • args – An array of strings storing the command to run and its arguments. The command to run is always the first argument. The command could be found in the current PATH or it could be a full path to a file to execute. If the executable argument is set, the first argument will be the name of the spawned program provided to that program and typically shown in process listings.

  • env – An array of strings storing the environment to use when spawning the child process instead of forwarding the current environment. By default, this argument is an empty array. In that case, the current environment will be forwarded to the child process.

  • executable – By default, the executable argument is “”. In that case, the program to launch is the first element of the args array. If the executable argument is provided, it will be used instead of the first element of the args array as the program to launch. In either case, the program could be found by searching the PATH.

  • stdin – indicates how the standard input of the child process should be handled. It could be pipeStyle.forward, pipeStyle.close, pipeStyle.pipe, or a file descriptor number to use. Defaults to pipeStyle.forward.

  • stdout – indicates how the standard output of the child process should be handled. It could be pipeStyle.forward, pipeStyle.close, pipeStyle.pipe, or a file descriptor number to use. Defaults to pipeStyle.forward.

  • stderr – indicates how the standard error of the child process should be handled. It could be pipeStyle.forward, pipeStyle.close, pipeStyle.pipe, pipeStyle.stdout, or a file descriptor number to use. Defaults to pipeStyle.forward.

  • locking – Should channels created use locking? This argument is used to set subprocess.locking in the resulting subprocess. Defaults to true.

Returns

a subprocess with kind and locking set according to the arguments.

Throws

IllegalArgumentError – Thrown when args is an empty array.

proc spawn(args: [] string, env: [] string = Subprocess.empty_env, executable = "", stdin: ?t = pipeStyle.forward, stdout: ?u = pipeStyle.forward, stderr: ?v = pipeStyle.forward, param kind = iokind.dynamic, param locking = true) throws

Warning

‘spawn’ with a ‘kind’ is deprecated

proc spawnshell(command: string, env: [] string = Subprocess.empty_env, stdin: ?t = pipeStyle.forward, stdout: ?u = pipeStyle.forward, stderr: ?v = pipeStyle.forward, executable = "/bin/sh", shellarg = "-c", param locking = true) throws

Create a subprocess by invoking a shell.

Note

Since the command string is passed to a shell, it is very unsecure to pass user input to this command without proper quoting.

Arguments

  • command – A string representing the command to run. This string will be interpreted by the shell.

  • env – An array of strings storing the environment to use when spawning the child process instead of forwarding the current environment. By default, this argument is an empty array. In that case, the current environment will be forwarded to the child process.

  • stdin – indicates how the standard input of the child process should be handled. It could be pipeStyle.forward, pipeStyle.close, pipeStyle.pipe, or a file descriptor number to use. Defaults to pipeStyle.forward.

  • stdout – indicates how the standard output of the child process should be handled. It could be pipeStyle.forward, pipeStyle.close, pipeStyle.pipe, or a file descriptor number to use. Defaults to pipeStyle.forward.

  • stderr – indicates how the standard error of the child process should be handled. It could be pipeStyle.forward, pipeStyle.close, pipeStyle.pipe, pipeStyle.stdout, or a file descriptor number to use. Defaults to pipeStyle.forward.

  • executable – By default, the executable argument is “/bin/sh”. That directs the subprocess to run the /bin/sh shell in order to interpret the command string.

  • shellarg – An argument to pass to the shell before the command string. By default this is “-c”.

  • locking – Should channels created use locking? This argument is used to set subprocess.locking in the resulting subprocess. Defaults to true.

Returns

a subprocess with kind and locking set according to the arguments.

Throws

IllegalArgumentError – Thrown when command is an empty string.

proc spawnshell(command: string, env: [] string = Subprocess.empty_env, stdin: ?t = pipeStyle.forward, stdout: ?u = pipeStyle.forward, stderr: ?v = pipeStyle.forward, executable = "/bin/sh", shellarg = "-c", param kind = iokind.dynamic, param locking = true) throws

Warning

‘spawnshell’ with a ‘kind’ argument is deprecated

proc ref subprocess.poll() throws

Check to see if a child process has terminated. If the child process has terminated, after this call, running will be false.

Throws

  • InterruptedError – when the poll was interrupted by a signal.

  • SystemError – if something else has gone wrong when polling the subprocess.

proc ref subprocess.wait(buffer = true) throws

Wait for a child process to complete. After this function returns, running is false and exitCode stores the exit code returned by the subprocess.

If buffer is true, then this call will handle cases in which stdin, stdout, or stderr for the child process is pipeStyle pipe by writing any input to the child process and buffering up the output of the child process as necessary while waiting for it to terminate. It will do so in the same manner as subprocess.communicate.

Note

Do not use buffer false when using pipe for stdin, stdout, or stderr. If buffer is false, this function does not try to send any buffered input to the child process and so could result in a hang if the child process is waiting for input to finish. Similarly, this function does not consume the output from the child process and so the child process could hang while waiting to write data to its output while the parent process is waiting for it to complete (and not consuming its output).

Arguments

buffer – if true, buffer input and output pipes (see above).

Throws

  • BlockingIoError – when there weren’t sufficient resources to perform one of the required actions

  • InterruptedError – when the call was interrupted in some way.

  • BrokenPipeError – when a pipe for the subprocess closed early.

  • SystemError – when invalid values were passed to the subprocess’s stdin, or something else went wrong when shutting down the subprocess.

proc ref subprocess.communicate() throws

Wait for a child process to complete. After this function returns, running is false and exitCode stores the exit code returned by the subprocess.

This function handles cases in which stdin, stdout, or stderr for the child process is pipeStyle.pipe by writing any input to the child process and buffering up the output of the child process as necessary while waiting for it to terminate.

Throws

  • BlockingIoError – when there weren’t sufficient resources to perform one of the required actions

  • InterruptedError – when the call was interrupted in some way.

  • BrokenPipeError – when a pipe for the subprocess closed early.

  • SystemError – when something went wrong when shutting down the subprocess

proc ref subprocess.close() throws

Close any open channels and pipes for interacting with a subprocess. This function does not wait for the subprocess to complete. Note that it is generally not necessary to call this function since these channels will be closed when the subprocess record goes out of scope.

proc subprocess.sendPosixSignal(signal: int) throws

Send a signal to a child process.

Declarations for POSIX.1.2008 signals are provided in the OS.POSIX module. These include SIGABRT, SIGALRM, SIGBUS, SIGCHLD, SIGCONT, SIGFPE, SIGHUP, SIGILL, SIGINT, SIGKILL, SIGPIPE, SIGQUIT, SIGSEGV, SIGSTOP, SIGTERM, SIGTRAP, SIGTSTP, SIGTTIN, SIGTTOU, SIGURG, SIGUSR1, SIGUSR2, SIGXCPU, SIGXFSZ.

See your system’s documentation for their meaning:

man signal

Other values for signal are system-specific and can be declared in this way, for example:

extern const SIGPOLL: c_int;
Arguments

signal – the signal to send

proc subprocess.abort() throws

Request an abnormal termination of the child process. The associated signal, SIGABRT, may be caught and handled by the child process. See subprocess.sendPosixSignal.

proc subprocess.alarm() throws

Send the child process an alarm signal. The associated signal, SIGALRM, may be caught and handled by the child process. See subprocess.sendPosixSignal.

proc subprocess.kill() throws

Unconditionally kill the child process. The associated signal, SIGKILL, cannot be caught by the child process. See subprocess.sendPosixSignal.

proc subprocess.terminate() throws

Request termination of the child process. The associated signal, SIGTERM, may be caught and handled by the child process. See subprocess.sendPosixSignal.