IO

Usage

use IO;

or

import IO;

Submodules

• FormattedIO

Support for a variety of kinds of input and output.

Input/output (I/O) facilities in Chapel include the types file, fileReader and fileWriter; the constants stdin, stdout and stderr; the functions open, file.close, file.reader, file.writer, fileReader.read, fileWriter.write, and many others.

Automatically Available Symbols

Note

These symbols can also be accessed using IO. as their qualified access prefix.

All Chapel programs include write, writeln and writef by default. This allows for a simple implementation of a Hello World program:

writeln("Hello, World!");
// outputs
// Hello, World!

I/O Overview

A file in Chapel identifies a file in the underlying operating system. Reads to a file are done via one or more fileReaders associated with the file and writes to a file are done via one or more fileWriters. Each fileReader or fileWriter uses a buffer to provide sequential read or write access to its file.

For example, the following program opens a file and writes an integer to it:

use IO;

try {
  // open the file "test-file.txt" for writing, creating it if
  // it does not exist yet.
  var myFile = open("test-file.txt", ioMode.cw);

  // create a fileWriter starting at the beginning of the file
  // (this fileWriter will not be used in parallel, so does not need to use
  // locking)
  var myFileWriter = myFile.writer(locking=false);

  var x: int = 17;

  // This function will write the human-readable text version of x;
  // binary I/O is also possible.
  myFileWriter.write(x);

  // Now test-file.txt contains:
  // 17

} catch e: Error {
  // Generally speaking, the I/O functions throw errors.  Handling these
  // errors is application-dependent and is left out of this example for
  // brevity.  Please see the documentation for individual functions for more
  // details about errors that they can throw.
}

Then, the following program can be used to read the integer:

use IO;

try {
  // open the file "test-file.txt" for reading only
  var myFile = open("test-file.txt", ioMode.r);

  // create a fileReader starting at the beginning of the file
  // (this fileReader will not be used in parallel, so does not need to use
  // locking)
  var myFileReader = myFile.reader(locking=false);

  var x: int;

  // Now read a textual integer. Note that the
  // fileReader.read function returns a bool to indicate
  // if it read something or if the end of the file
  // was reached before something could be read.
  var readSomething = myFileReader.read(x);

  writeln("Read integer ", x);
  // prints out:
  // Read integer 17

} catch e: Error {
  // Generally speaking, the I/O functions throw errors.  Handling these
  // errors is application-dependent and is left out of this example for
  // brevity.  Please see the documentation for individual functions for more
  // details about errors that they can throw.
}

The read functions allow one to read values into variables as the following example demonstrates. It shows three ways to read values into a pair of variables x and y.

use IO;

var x: int;
var y: real;
/* reading into variable expressions, returning
   true if the values were read, false on EOF */
var ok:bool = read(x, y);

/* reading via a single type argument */
x = read(int);
y = read(real);

/* reading via multiple type arguments */
(x, y) = read(int, real);

Design Rationale

Since fileReaders and fileWriters operate independently, concurrent I/O to the same open file is possible without contending for locks. Furthermore, since the fileReader or fileWriter (and not the file) stores the current file offset, it is straightforward to create programs that access the same open file in parallel. Note that such parallel access is not possible in C when multiple threads are using the same FILE* to write to different regions of a file because of the race condition between fseek and fwrite. Because of these issues, Chapel programmers wishing to perform I/O will need to know how to open files as well as create fileReaders and fileWriters.

The ‘serialize’ and ‘deserialize’ Methods

A Chapel program can implement serialize and deserialize methods on a user-defined data type to define how that type is deserialized from a fileReader or serialized to a fileWriter. The method signatures for non-class types are:

proc T.serialize(writer: fileWriter(locking=false, ?),
                 ref serializer: ?st) throws {/*...*/}

proc ref T.deserialize(reader: fileReader(locking=false, ?),
                       ref deserializer: ?dt) throws {/*...*/}

The signatures for classes are slightly different:

override proc T.serialize(writer: fileWriter(locking=false, ?),
                          ref serializer: ?st) throws {/*...*/}

override proc T.deserialize(reader: fileReader(locking=false, ?),
                            ref deserializer: ?dt) throws {/*...*/}

The serializer and deserializer arguments must satisfy the Serializer API and the Deserializer API, respectively.

Basic Usage

Implementations of serialize and deserialize methods are not necessarily required to utilize their serializer and deserializer arguments, and can instead trivially read and write from their fileReader and fileWriter arguments. For example:

// A record 'R' that serializes as an integer
record R : writeSerializable {
  var x : int;

  proc serialize(writer: fileWriter(locking=false, ?),
                 ref serializer: ?st) {
    writer.write(x);
  }
}

var val = new R(5);
writeln(val); // prints '5'

Using Serializers and Deserializers

Serializers and Deserializers support a variety of methods to support serializing various kinds of types. These methods can be used to serialize or deserialize a type in a format-agnostic way. For example, consider a simple ‘point’ type:

record point : writeSerializable {
  var x : int;
  var y : int;
}

The default implementation of point’s serialize method will naturally serialize point as a record. In the default serialization format, this would look something like (x = 2, y = 4). In the JSON serialization format, the output would instead be {"x":4, "y":2}. While this may be perfectly acceptable, what if the author of point wished to always serialize a point as a tuple?

Serializers and Deserializers have “start” methods that begin serialization or deserialization of a type, and then return a helper object that implements methods to continue the process. To begin serializing point as a tuple, a user may invoke the startTuple method on the serializer, passing in the fileWriter to use when writing serialized output and the number of elements in the tuple. The returned value from startTuple is a helper object that implements writeElement and endTuple methods:

proc point.serialize(writer: fileWriter(locking=false, ?),
                     ref serializer: ?st) {
  // Start serializing and get the helper object
  // '2' represents the number of tuple elements to be serialized
  var ser = serializer.startTuple(writer, 2);

  ser.writeElement(x); // serialize 'x' as a tuple element
  ser.writeElement(y); // serialize 'y' as a tuple element

  // End serialization of the tuple
  ser.endTuple();
}

Now, when using different Serializers like the defaultSerializer or the jsonSerializer, the point type can be serialized without introducing special cases for each format:

use IO, JSON;

var p = new point(4, 2);

// Prints '(4, 2)' in the default serialization format
stdout.writeln(p);

// Prints '[4, 2]' in the JSON serialization format
var jsonWriter = stdout.withSerializer(jsonSerializer);
jsonWriter.writeln(p);

A similar API exists for deserialization that would allow for deserializing a point as a tuple. Please refer to the IO Serializers technote for more detail on the various kinds of types that can be serialized and deserialized. As of Chapel 1.32 the supported type-kinds are Classes, Records, Tuples, Arrays, Lists, and Maps.

Compiler-Generated Default Methods

Default serialize methods are created for all types for which a user-defined serialize method is not provided.

Classes will be serialized as a ‘Class’ type-kind using the Serializer API, and will invoke their parent serialize method before serializing their own fields.

Records will be serialized as a ‘Record’ type-kind using the Serializer API, and will serialize each field in the record.

Default deserialize methods are created for all types for which a user-defined deserialize method is not provided. The default deserialize methods will mirror the relevant API calls in the default serialize methods.

For more information on the default serialization format, please refer to the defaultSerializer and defaultDeserializer types.

If the compiler sees a user-defined implementation of the serialize method, the deserialize method, or the deserializing initializer, then the compiler may choose to not automatically generate any of the other unimplemented methods. This is out of concern that the user has intentionally deviated from the default implementation of serialization and deserialization.

Types with compiler-generated versions of these methods do not need to explicitly indicate that they satisfy any of the relevant serialization interfaces (such as writeSerializable).

Note

Note that it is not currently possible to read and write circular data structures with these mechanisms.

Files

There are several functions that open a file and return a file including open, openTempFile, openMemFile, the file initializer that takes an int argument, and the file initializer that takes a c_FILE argument.

Once a file is open, it is necessary to create associated fileReader(s) and/or fileWriter(s) - see file.reader and file.writer - to read from and/or write to the file.

Use the file.fsync function to explicitly synchronize the file to ensure that file data is committed to the file’s underlying device for persistence.

Files, fileReaders, and fileWriters will be kept alive while there are variables referring to them and closed when all variables referring to them have gone out of scope. However, each can be closed explicitly with close methods. Note that file.close will not work if the file has open fileReaders and/or fileWriters.

Note

Escaped strings can be used for paths on systems where UTF-8 file names are not enforced.

Functions for fileReader and fileWriter Creation

file.writer creates a fileWriter for writing to a file, and file.reader creates a fileReader for reading from a file.

The helper functions openReader and openWriter can also be used to open a file and create a fileReader/fileWriter to it in a single step.

Synchronization of fileReader and fileWriter Data and Avoiding Data Races

FileReaders and fileWriters (and files) contain locks in order to keep their operation safe for multiple tasks. When creating a fileReader or fileWriter, it is possible to disable the lock (for performance reasons) by passing locking=false to e.g. file.writer(), or by using openReader/openWriter. Some fileReader and fileWriter methods should only be called on locked fileReaders or fileWriters. With these methods, it is possible to perform I/O “transactions” (see fileWriter.mark, e.g.). To use these methods, e.g., first lock the fileWriter with fileWriter.lock, call the methods you need, then unlock the fileWriter with fileWriter.unlock. Note that in the future, we may move to alternative ways of calling these functions that guarantee that they are not called on a fileReader or fileWriter without the appropriate locking.

Besides data races that can occur if locking is not used in fileWriters when it should be, it is also possible for there to be data races on file data that is buffered simultaneously in multiple fileReader/fileWriter combinations. The main way to avoid such data races is the fileWriter.flush synchronization operation. fileWriter.flush will make all writes to the fileWriter, if any, available to concurrent viewers of its associated file, such as other fileWriters, fileReaders or other applications accessing this file concurrently. See the note below for more details on the situation in which this kind of data race can occur.

Note

Since fileWriters can buffer data until fileWriter.flush is called, it is possible to write programs that have undefined behavior because of race conditions on fileWriter buffers. In particular, the problem comes up for programs that make:

• concurrent operations on multiple fileWriters and/or fileReaders that operate on overlapping regions of a file

• where at least one fileWriter is used along with other fileWriters or fileReaders

• and where data could be stored in more than one of the overlapping fileWriter’s buffers at the same time (i.e., write and read ordering are not enforced through fileWriter.flush and other means such as sync variables).

Note that it is possible in some cases to create a file that does not allow multiple fileWriters and/or fileReaders at different offsets. FileWriters created on such files will not change the file’s offset based on a region= offset argument. Instead, each read or write operation will use the file descriptor’s current offset. Therefore, only one fileWriter or fileReader should be created for files created in the following situations:

• with the file initializer that takes a c_FILE argument

• with the file initializer that takes an int argument, where the int represents a non-seekable system file descriptor

Performing I/O with FileReaders and FileWriters

FileReaders have a variety of read methods and fileWriters have a variety of write methods. The most common variety of these are generic methods that can read or write values of any type. For non-primitive types, the relevant deserialize or serialize method is used to control the I/O formatting; see The ‘serialize’ and ‘deserialize’ Methods. These functions generally take any number of arguments and throw if there was an error:

• fileWriter.write

• fileWriter.writeln

• FormattedIO.fileWriter.writef (see also Formatted I/O)

• fileReader.read

• fileReader.readln

• FormattedIO.fileReader.readf (see also Formatted I/O)

The fileWriter type also has the following methods for executing write operations with more specific types. These methods can provide finer control over the fileWriter’s behavior as well as some performance advantages over the generic write methods:

• fileWriter.writeBits

• fileWriter.writeCodepoint

• fileWriter.writeByte

• fileWriter.writeString

• fileWriter.writeBytes

• fileWriter.writeBinary

The fileReader type has similar methods for executing read operations with more specific types, where the goal of these methods is also to provide finer control over the fileReader’s behavior and the potential for performance advantages:

• fileReader.readBits

• fileReader.readCodepoint

• fileReader.readByte

• fileReader.readString

• fileReader.readBytes

• fileReader.readBinary

Additionally, the fileReader has the following methods which read arbitrary amounts of data from the file until some stop condition is met. These methods generally have multiple overloads for reading into values of different types:

• fileReader.readLine

• fileReader.lines

• fileReader.readThrough

• fileReader.readTo

• fileReader.readAll

Sometimes it’s important to flush the buffer in a fileWriter - to do that, use the fileWriter.flush() method. Flushing the buffer will make all writes available to other applications or other views of the file (e.g., it will call the OS call pwrite()). It is also possible to close a fileWriter, which will implicitly flush it and release any buffer memory used by the fileWriter. Note that if you need to ensure that data from a fileWriter is on disk, you’ll have to call fileWriter.flush or fileWriter.close and then file.fsync on the related file.

Functions for Closing FileReaders and FileWriters

A fileReader or fileWriter must be closed in order to free the resources allocated for it, to ensure that data written to it is visible to other fileReaders, or to allow the associated file to be closed.

See fileReader.close and fileWriter.close.

It is an error to perform any I/O operations on a fileReader or fileWriter that has been closed. It is an error to close a file when it has fileReaders and/or fileWriters that have not been closed.

Files, fileReaders and fileWriters are reference counted. Each file, fileReader and fileWriter is closed automatically when no references to it remain. For example, if a local variable is the only reference to a fileReader, the fileReader will be closed when that variable goes out of scope. Programs may also close a file, fileReader or fileWriter explicitly.

The stdin fileReader, and stdout and stderr fileWriters

Chapel provides the predefined fileReader stdin, and the predefined fileWriters stdout, and stderr to access the corresponding operating system streams standard input, standard output, and standard error.

stdin supports reading; stdout and stderr support writing.

All three are safe to use concurrently.

Unicode Support

Most I/O operations default to working with textual data in the UTF-8 encoding. This choice of UTF-8 matches the encoding used by the string type (see Strings).

To work with non-UTF-8 data, it’s necessary to use binary I/O routines (e.g. fileReader.readByte, fileReader.readBytes, fileReader.readBinary fileReader.readBits) or do I/O with a serializer or deserializer that uses a binary format, such as binaryDeserializer.

Generally speaking, if invalid UTF-8 is encountered when reading textual data, a SystemError will be throw with EILSEQ and the channel position will be left just after the first byte of UTF-8 that was determined to be invalid. Some routines have other error handling behavior as described in their documentation (for example, see fileReader.readThrough).

Error Handling

Most I/O routines throw an Error, which can be handled appropriately with try and catch (see the documentation for more detail).

Additionally, some subclasses of Error are commonly used within the I/O implementation. These are:

• OS.EofError - the end of file was reached

• OS.UnexpectedEofError - a read or write only returned part of the requested data

• OS.BadFormatError - data read did not adhere to the requested format

System Errors:

For other error cases, a general SystemError is typically thrown. These errors are often produced by less predictable circumstances that are more challenging to recover from. For example, a fileReader could run out of memory when attempting to allocate more buffer space.

As such, it is typically recommended that more specific errors are caught and recovered from separately from a SystemError. See the following example:

use IO;

const r = openReader("test.txt");

try {
  var i = r.read(int);
  // ...
} catch e: EofError {
  writeln("r is at EOF");
  // we're done reading

} catch e: UnexpectedEofError {
  writeln("unable to read an 'int'");
  // try to read something else? ...

} catch e: SystemError {
  writeln("system error in IO implementation: ", e);
  // try to recover from the error? ...

} catch e: Error {
  writeln("something else went wrong...");
}

I/O Transactions

An I/O transaction is a common pattern afforded by the IO interface that provides the ability to temporarily hold a particular region of a file in a fileReader or fileWriter’s buffer. This allows I/O operations within that region of the file to easily be undone in the event of some unexpected data or other errors.

To support I/O transactions, each fileReader and fileWriter is fitted with a mark stack which contains a series of file offsets. The region of the file between the minimum and maximum offset on the mark stack will always be retained in the buffer.

The steps of a typical I/O transaction are as follows:

• mark the current file offset with fileReader.mark or fileWriter.mark. This pushes the current offset onto the mark stack

• do a speculative I/O operation:

  • reading example: read 200 bytes followed by a b.

  • writing example: write 200 bytes without exceeding the fileWriter’s region.

• if the operation fails, revert the operation by calling fileReader.revert or fileWriter.revert. Subsequent operations will continue from the originally marked offset as if nothing happened.

• if the operation is successful, call fileReader.commit or fileWriter.commit to pop the value from the mark stack and continue performing I/O operations from the current offset.

Note that when the mark stack is emptied, a fileWriter is allowed to flush any portion of its buffer to its file and a fileReader is allowed to discard any portion of its buffer.

See the following example of a simple I/O transaction:

use IO;

var fr = openReader("file.txt");

// mark the current channel position
fr.mark();

// read an array of bytes
var a: [0..<200] uint(8);
fr.read(a);

// try to match a pattern
if fr.matchLiteral("b") {
  fr.commit(); // "b" was found, continue reading from the current offset
} else {
  fr.revert(); // "b" was't found, revert back to the marked position

  // try to read something else from the file, throw an error, etc.
}

Specifying the region of a fileReader or fileWriter

The fileReader and fileWriter types can be configured to own a specific region of their associated file.

When a fileReader or fileWriter is initialized using one of the following routines, the optional region argument can be set to designate some region of the file (a zero-based range of integers in bytes) that can be read from or written to:

• file.reader

• file.writer

• openReader

I/O operations that fall outside of the region are illegal. The region argument defaults to 0.., meaning that the owned region starts at the 0th byte, and extends indefinitely.

Note that fileReader.seek and fileWriter.seek can be used to adjust a fileReader or fileWriter’s region after initialization.

Creating a fileReader or fileWriter that points to a sub-region of a file can be useful for concurrently reading from or writing to multiple portions of a file from separate tasks. See the following example, which uses multiple tasks to concurrently read bytes from a binary file into an array of bytes:

use IO;

// the number of tasks to use
config const nWorkers = 8;

// open a (large) binary file
var f = open("file.dat", ioMode.r);

// compute how many bytes each worker will read
const nBytes = f.size,
      nPerLoc = nBytes/ nWorkers;

// create an array to hold the file contents
var a: [0..<nBytes] uint(8);

// concurrently read each worker's region into 'a'
coforall w in 0..<nWorkers {
  const myRegion = (w*nPerLoc)..<((w+1) * nPerLoc),
        fr = f.reader(region=myRegion, locking=false);

  fr.readBinary(a[myRegion]);
}

Locking Behavior of FileReaders and FileWriters

The fileReader and fileWriter types can be configured to lock access to their file when executing I/O operations to avoid race conditions with other fileReader or fileWriter instances that may be accessing the same file.

The locking field is a param and is thus part of the fileReader and fileWriter type. As such, it is possible to use type constraints to designate whether a reader or writer is locking. For example this could be useful in a procedure that relies on a reader argument being locking:

use IO;

proc readSomething(reader: fileReader(locking=true, ?)) {
  // use 'reader' concurrently with another fileReader/fileWriter   ...
}

The locking field can be set by passing the desired value to one of the following routines that create a fileReader or fileWriter:

• file.reader

• file.writer

• openReader

• openWriter

With a locking fileReader or fileWriter, one can obtain a lock manually by calling fileReader.lock or fileWriter.lock, and then release a lock by calling fileReader.unlock or fileWriter.unlock.

Note

The following methods will not automatically acquire/release a lock for locking=true:

• fileReader.mark

• fileWriter.mark

• fileReader.commit

• fileWriter.commit

• fileReader.revert

• fileWriter.revert

• fileReader.offset

• fileWriter.offset

Ensuring Successful I/O

It is possible - in some situations - for I/O to fail without returning an error. In cases where a programmer wants to be sure that there was no error writing the data to disk, it is important to call file.fsync to make sure that data has arrived on disk without an error. Many errors can be reported with a typical operation, but some errors can only be reported by the system during file.close or even file.fsync.

When a file (or fileWriter) is closed, data written to that file will be written to disk eventually by the operating system. If an application needs to be sure that the data is immediately written to persistent storage, it should use file.fsync prior to closing the file.

Correspondence with C I/O

It is not possible to seek, read, or write to a file directly; fileReaders and/or fileWriters must be created and used.

fileWriter.flush in Chapel has the same conceptual meaning as fflush() in C. However, fflush() is not necessarily called in fileWriter.flush(), unlike fsync(), which is actually called by file.fsync() in Chapel.

Automatically Included IO Functions

proc write(const args ...?n, sep: string = "")

Equivalent to try! stdout.write. See IO.fileWriter.write

proc writeln(const args ...?n, sep: string = "")

Equivalent to try! stdout.writeln. See IO.fileWriter.writeln

proc writef(fmt: ?t, const args ...?k)  where isStringType(t) || isBytesType(t)

Equivalent to try! stdout.writef. See FormattedIO.fileWriter.writef.

IO Functions and Types

enum ioMode { r = 1, cw = 2, rw = 3, cwr = 4, a = 5 }

The ioMode type is an enum. When used as arguments when opening files, its constants have the same meaning as the listed strings passed to fopen() in C. However, open() in Chapel does not necessarily invoke fopen() in C.

enum constant r = 1

Open an existing file for reading. (fopen() string is “r”)

enum constant cw = 2

Create a new file for writing. If the file already exists, its contents are truncated. (fopen() string is “w”)

enum constant rw = 3

Open an existing file for reading and writing. (fopen() string is “r+”)

enum constant cwr = 4

Same as ioMode.cw, but reading from the file is also allowed. (fopen() string is “w+”)

enum constant a = 5

Warning

ioMode.a is unstable and subject to change. It currently only supports one fileWriter at a time.

Open a file for appending, creating it if it does not exist. (fopen() string is “a”)

enum endianness { native = 0, big = 1, little = 2 }

The endianness type is an enum. When used as an argument to the fileReader or fileWriter methods, its constants have the following meanings:

enum constant native = 0

native means binary I/O is performed in the byte order that is native to the target platform.

enum constant big = 1

big means binary I/O is performed in big-endian byte order.

enum constant little = 2

little means binary I/O is performed in little-endian byte order.

type ioendian = endianness

Warning

ioendian is deprecated; please use endianness instead

record ioHintSet

A value of the ioHintSet type defines a set of hints to provide information about the operations that a file, fileReader or fileWriter will perform. These hints may be used by the implementation to select optimized versions of the I/O operations.

Most hints have POSIX equivalents associated with posix_fadvise() and posix_madvise().

This example depicts how an ioHintSet might be used.

use IO;

// define a set of hints using a union operation
var hints = ioHintSet.sequential | ioHintSet.prefetch;

// open a file using the hints
var f: file;
try! {
  f = open("file.txt", ioMode.r, hints=hints);
}

proc type empty

Defines an empty set, which provides no hints. Corresponds to ‘POSIX_*_NORMAL’.

proc type sequential

Suggests that the file will be accessed sequentially. Corresponds to ‘POSIX_*_SEQUENTIAL’

proc type random

Suggests that the file will be accessed randomly. Corresponds to ‘POSIX_*_RANDOM’.

proc type prefetch

Suggests that the runtime/OS should immediately begin prefetching the file contents. Corresponds to ‘POSIX_*_WILLNEED’.

proc type mmap(useMmap = true)

Suggests whether or not ‘mmap’ should be used to access the file contents.

• when useMmap is true, suggests that ‘mmap’ should be used

• when useMmap is false, suggests that ‘mmap’ should not be used and ‘pread’/’pwrite’ should be used instead

operator ioHintSet.|(lhs: ioHintSet, rhs: ioHintSet)

Compute the union of two hint sets

operator ioHintSet.&(lhs: ioHintSet, rhs: ioHintSet)

Compute the intersection of two hint sets

operator ioHintSet.==(lhs: ioHintSet, rhs: ioHintSet)

Compare two hint sets for equality

operator ioHintSet.!=(lhs: ioHintSet, rhs: ioHintSet)

Compare two hint sets for inequality

record file

The file type is implementation-defined. A value of the file type refers to the state that is used by the implementation to identify and interact with the OS file.

When a file formal argument has default intent, the actual is passed by const ref to the formal upon a function call, and the formal cannot be assigned within the function.

The default value of the file type does not represent any OS file. It is illegal to perform any I/O operations on the default value.

proc file.init(fp: c_ptr(c_FILE), hints = ioHintSet.empty, own = false) throws

Create a Chapel file that wraps around an open C file. A pointer to a C FILE object can be obtained via Chapel’s C Interoperability functionality.

Note

This is an alternative way to create a file. The main way to do so is via the open function.

Once the Chapel file is created, you will need to use file.reader to create a fileReader or file.writer to create a fileWriter to perform I/O operations on the C file.

Note

The resulting file value should only be used with one fileReader or fileWriter at a time. The I/O system will ignore the offsets when reading or writing to a file opened using this initializer.

Arguments:

• fp – a pointer to a C FILE. See c_FILE.

• hints – optional argument to specify any hints to the I/O system about this file. See ioHintSet.

• own – set to indicate if the c_FILE provided should be cleaned up when the file is closed. Defaults to false

Throws:

SystemError – If the C file could not be retrieved.

proc file.init(fileDescriptor: int, hints = ioHintSet.empty, own = false) throws

Create a Chapel file that works with a system file descriptor. Note that once the file is open, you will need to use a file.reader to create a fileReader or file.writer to create a fileWriter to actually perform I/O operations

Note

This is an alternative way to create a file. The main way to do so is via the open function.

Note

This function can be used to create Chapel files that refer to system file descriptors that do not support the seek functionality. For example, file descriptors that represent pipes or open socket connections have this property. In that case, the resulting file value should only be used with one fileReader or fileWriter at a time. The I/O system will ignore the fileReader offsets when reading (or the fileWriter offsets when writing) to files backed by non-seekable file descriptors.

Arguments:

• fileDescriptor – a system file descriptor.

• hints – optional argument to specify any hints to the I/O system about this file. See ioHintSet.

• own – set to indicate if the fileDescriptor provided should be cleaned up when the file is closed. Defaults to false

Throws:

SystemError – If the file descriptor could not be retrieved.

proc file.isOpen() : bool

Indicates if the file is currently open. Will return false for both closed and invalid files

proc file.close() throws

Close a file.

In order to free the resources allocated for a file, it must be closed using this method.

Closing a file does not guarantee immediate persistence of the performed updates, if any. In cases where immediate persistence is important, file.fsync should be used for that purpose prior to closing the file. In particular, even though closing the file might complete without errors, the data written might not persist in the event of a severe error like running out of storage space or power loss. See also Ensuring Successful I/O.

Files are automatically closed when the file variable goes out of scope and all fileReaders and fileWriters using that file are closed. Programs may also explicitly close a file using this method.

It is an error to perform any I/O operations on a file that has been closed. It is an error to close a file when it has fileReaders and/or fileWriters that have not been closed.

Throws:

SystemError – If the file could not be closed.

proc file.fsync() throws

Sync a file to disk.

Commits file data to the device associated with this file. Data written to the file by a fileWriter will only be guaranteed committed if the fileWriter has been closed or flushed.

This function will typically call the fsync system call.

Throws:

SystemError – If the file could not be synced.

proc file.path : string throws

Get the absolute path to an open file.

Note that not all files have a path (e.g. files opened with openMemFile), and that this procedure may not work on all operating systems.

The function Path.realPath is an alternative way to get the path to a file.

Returns:

the absolute path to the file

Return type:

string

Throws:

SystemError – If the path could not be retrieved.

proc file.size : int throws

Get the current size of an open file. Note that the size can always change if other fileWriters, tasks or programs are writing to the file.

Returns:

the current file size

Throws:

SystemError – If the size could not be retrieved.

proc open(path: string, mode: ioMode, hints = ioHintSet.empty) : file throws

Open a file on a filesystem. Note that once the file is open, you will need to use a file.reader to create a fileReader or file.writer to create a fileWriter to actually perform I/O operations

Arguments:

• path – which file to open (for example, “some/file.txt”).

• mode – specify whether to open the file for reading or writing and whether or not to create the file if it doesn’t exist. See ioMode.

• hints – optional argument to specify any hints to the I/O system about this file. See ioHintSet.

Returns:

an open file to the requested resource.

Throws:

• FileNotFoundError – If part of the provided path did not exist

• PermissionError – If part of the provided path had inappropriate permissions

• NotADirectoryError – If part of the provided path was expected to be a directory but was not

• SystemError – If the file could not be opened.

proc openTempFile(hints = ioHintSet.empty) : file throws

Open a temporary file. Note that once the file is open, you will need to use a file.reader to create a fileReader or file.writer to create a fileWriter to actually perform I/O operations.

The temporary file will be created in an OS-dependent temporary directory, for example “/tmp” is the typical location. The temporary file will be deleted upon closing.

Temporary files are opened with ioMode ioMode.cwr; that is, a new file is created that supports both writing and reading. When possible, it may be opened using OS support for temporary files in order to make sure that a new file is created only for use by the current application.

Arguments:

hints – optional argument to specify any hints to the I/O system about this file. See ioHintSet.

Returns:

an open temporary file.

Throws:

SystemError – If the temporary file could not be opened.

proc openMemFile() : file throws

Open a file that is backed by a buffer in memory that will not persist when the file is closed. Note that once the file is open, you will need to use a file.reader to create a fileReader or file.writer to create a fileWriter to actually perform I/O operations.

The resulting file supports both reading and writing.

Returns:

an open memory file.

Throws:

SystemError – If the memory buffered file could not be opened.

record fileReader

A fileReader supports sequential reading from an underlying file object. It can buffer data. Read operations on it might return old data.

The fileReader type is implementation-defined. A value of the fileReader type refers to the state that is used to implement the reading operations.

When a fileReader formal argument has default intent, the actual is passed by const ref to the formal upon a function call, and the formal cannot be assigned within the function.

The default value of the fileReader type is not associated with any file, and so cannot be used to perform I/O.

The fileReader type is generic.

param locking : bool

locking is a boolean indicating whether it is safe to use this fileReader concurrently (when true).

type deserializerType = defaultSerializeType(false)

deserializerType indicates the type of the deserializer that this fileReader will use to deserialize data.

proc fileReader.getFile()

Warning

The ‘fileReader.getFile()’ method may change based on feedback

Get the file type underlying a fileReader.

proc fileReader.deserializer ref : deserializerType

Return a mutable reference to this fileReader’s deserializer.

record fileWriter

A fileWriter supports sequential writing to an underlying file object. A fileWriter can buffer data. Write operations might not have an immediate effect. Use fileWriter.flush to control this buffering.

The fileWriter type is implementation-defined. A value of the fileWriter type refers to the state that is used to implement the writing operations.

When a fileWriter formal argument has default intent, the actual is passed by const ref to the formal upon a function call, and the formal cannot be assigned within the function.

The default value of the fileWriter type is not associated with any file, and so cannot be used to perform I/O.

The fileWriter type is generic.

param locking : bool

locking is a boolean indicating whether it is safe to use this fileWriter concurrently (when true).

type serializerType = defaultSerializeType(true)

serializerType indicates the type of the serializer that this fileWriter will use to serialize data.

proc fileWriter.getFile()

Warning

The ‘fileWriter.getFile()’ method may change based on feedback

Get the file type underlying a fileWriter.

proc fileWriter.serializer ref : serializerType

Return a mutable reference to this fileWriter’s serializer.

record defaultSerializer

The default Serializer used by fileWriter.

See the serializers technote for a general overview of Serializers and their usage.

Otherwise, please refer to the individual methods in this type for a description of the default IO format.

proc ref serializeValue(writer: fileWriter, const val: ?t) : void throws

Serialize val with writer.

Numeric values are serialized as though they were written with the format as %i for integers and %r for real numbers. Complex numbers are serialized as %z. Please refer to the section on Formatted IO for more information.

Booleans are serialized as the literal strings true or false.

string values are serialized using the same format as %s — that is, literally and without quotes. bytes values are also serialized literally without extra formatting.

Enums are serialized using the name of the corresponding value. For example with an enum like enum colors {red, green blue}, the value red would simply be serialized as red.

The nil value and nilable class variables storing nil will be serialized as the text nil.

Classes and records will have their serialize method invoked, passing in writer and this Serializer as arguments. Please see the serializers technote for more information.

Classes and records are expected to implement the writeSerializable or serializable interface.

Arguments:

• writer – The fileWriter used to write serialized output.

• val – The value to be serialized.

proc startClass(writer: fileWriter, name: string, size: int) throws

Start serializing a class by writing the character {.

Arguments:

• writer – The fileWriter to be used when serializing.

• name – The name of the class type.

• size – The number of fields in the class.

Returns:

A new AggregateSerializer

proc startRecord(writer: fileWriter, name: string, size: int) throws

Start serializing a record by writing the character (.

Arguments:

• writer – The fileWriter to be used when serializing.

• name – The name of the record type.

• size – The number of fields in the record.

Returns:

A new AggregateSerializer

record AggregateSerializer

Returned by startClass or startRecord to provide the API for serializing classes or records.

A class with integer fields ‘x’ and ‘y’ with values ‘0’ and ‘5’ would be serialized as:

{x = 0, y = 5}

A record with matching fields would be serialized in the same way, but would use ( and ) instead of { and }.

proc ref writeField(name: string, const field: ?) throws

Serialize field named name.

Serializes fields in the form ‘<name> = <field>’. Adds a comma before the name if this is not the first field.

proc ref startClass(writer: fileWriter, name: string, size: int) throws

Start serializing a nested class inside the current class. In this format inheritance is not represented and parent fields are printed before child fields. For example, the following classes with values x=5 and y=2.0:

class Parent {
  var x : int;
}

class Child: Parent {
  var y : real;
}

would be serialized as:

{x = 5, y = 2.0}

Arguments:

• writer – The fileWriter to be used when serializing. Must match the writer used to create current AggregateSerializer.

• name – The name of the class type.

• size – The number of fields in the class.

Returns:

A new AggregateSerializer

proc endClass() throws

Ends serialization of the current class by writing the character }

Note

It is an error to call methods on an AggregateSerializer after invoking ‘endClass’.

proc endRecord() throws

Ends serialization of the current record by writing the character )

Note

It is an error to call methods on an AggregateSerializer after invoking ‘endRecord’.

proc startTuple(writer: fileWriter, size: int) throws

Start serializing a tuple by writing the character (.

Arguments:

• writer – The fileWriter to be used when serializing.

• size – The number of elements in the tuple.

Returns:

A new TupleSerializer

record TupleSerializer

Returned by startTuple to provide the API for serializing tuples.

A tuple will be serialized as a comma-separated list between two parentheses. For example, the tuple literal (1, 2, 3) would be serialized as:

(1, 2, 3)

A 1-tuple will be serialized with a trailing comma. For example, the literal (4,) would be serialized as (4,).

proc ref writeElement(const element: ?) throws

Serialize element.

Writes a leading comma before serializing the element if this is not the first element in the tuple.

proc endTuple() throws

Ends serialization of the current tuple by writing the character ).

Adds a comma between the last value and ) if there was only one element.

proc startList(writer: fileWriter, size: int) throws

Start serializing a list by writing the character [.

Arguments:

• writer – The fileWriter to be used when serializing.

• size – The number of elements in the list.

Returns:

A new ListSerializer

record ListSerializer

Returned by startList to provide the API for serializing lists.

A list will be serialized as a comma-separated series of serialized elements between two square brackets. For example, serializing a list with elements 1, 2, and 3 will produce the text:

[1, 2, 3]

Empty lists will be serialized as [].

proc ref writeElement(const element: ?) throws

Serialize element.

Writes a leading comma before serializing the element if this is not the first element in the list.

proc endList() throws

Ends serialization of the current list by writing the character ].

proc startArray(writer: fileWriter, size: int) throws

Start serializing an array.

Arguments:

• writer – The fileWriter to be used when serializing.

• size – The number of elements in the array.

Returns:

A new ArraySerializer

record ArraySerializer

Returned by startArray to provide the API for serializing arrays.

In the default format, an array will be serialized as a whitespace-separated series of serialized elements.

A 1D array is serialized simply using spaces:

1 2 3 4

A 2D array is serialized using spaces between elements in a row, and prints newlines for new rows:

1 2 3
4 5 6
7 8 9

Arrays with three or more dimensions will be serialized as a series of 2D “panes”, with multiple newlines separating new dimensions:

1 2 3
4 5 6
7 8 9

10 11 12
13 14 15
16 17 18

19 20 21
22 23 24
25 26 27

Empty arrays result in no output to the fileWriter.

proc ref startDim(size: int) throws

Inform the ArraySerializer to start serializing a new dimension of size size.

proc ref endDim() throws

End the current dimension.

proc ref writeElement(const element: ?) throws

Serialize element.

Adds a space if this is not the first element in the row.

proc endArray() throws

Ends serialization of the current array.

proc startMap(writer: fileWriter, size: int) throws

Start serializing a map by writing the character {.

Arguments:

• writer – The fileWriter to be used when serializing.

• size – The number of entries in the map.

Returns:

A new MapSerializer

record MapSerializer

Returned by startMap to provide the API for serializing maps.

Maps are serialized as a comma-separated series of pairs between curly braces. Pairs are serialized with a : separating the key and value. For example, the keys 1, 2, and 3 with values corresponding to their squares would be serialized as:

{1: 1, 2: 4, 3: 9}

Empty maps will be serialized as {}.

proc ref writeKey(const key: ?) throws

Serialize key.

Adds a leading comma if this is not the first pair in the map.

proc writeValue(const val: ?) throws

Serialize val, preceded by the character :.

proc endMap() throws

Ends serialization of the current map by writing the character }

record defaultDeserializer

The default Deserializer used by fileReader.

See the serializers technote for a general overview of Deserializers and their usage.

Otherwise, please refer to defaultSerializer for a description of the default IO format. Individual methods on this type may clarify behavior specific to deserialization.

Note

Prior to the 1.32 release and the advent of the ‘serializers’ feature, the default implementation for reading classes and records permitted reading fields out of order. This functionality is not supported by the defaultDeserializer.

For an unspecified amount of time this module will retain the ability to disable automatic use of the defaultDeserializer by recompiling programs with the config-param useIOSerializers set to false.

Eventually, however, users must update their programs to account for reading fields out of order.

proc ref deserializeType(reader: fileReader, type readType) : readType throws

Deserialize type readType with reader.

Classes and records will be deserialized using an appropriate initializer, passing in reader and this Deserializer as arguments. If an initializer is unavailable, this method may invoke the class or record’s deserialize method. Please see the serializers technote for more information.

Classes and records are expected to implement either the initDeserializable or readDeserializable interfaces (or both). Alternatively, types implementing the entire serializable interface are also accepted.

Arguments:

• reader – The fileReader from which types are deserialized.

• readType – The type to be deserialized.

Returns:

A value of type readType.

proc ref deserializeValue(reader: fileReader, ref val: ?readType) : void throws

Deserialize from reader directly into val.

Like deserializeType, but reads into an initialized value rather than creating a new value. For classes and records, this method will first attempt to invoke a deserialize method. If the deserialize method is unavailable, this method may fall back on invoking a suitable initializer and assigning the resulting value into val. Please see the serializers technote for more.

Classes and records are expected to implement either the initDeserializable or readDeserializable interfaces (or both). Alternatively, types implementing the entire serializable interface are also accepted.

Arguments:

• reader – The fileReader from which values are deserialized.

• val – The value into which this Deserializer will deserialize.

proc startClass(reader: fileReader, name: string) throws

Start deserializing a class by reading the character {.

Arguments:

• reader – The fileReader to use when deserializing.

• name – The name of the class type

Returns:

A new AggregateDeserializer

proc startRecord(reader: fileReader, name: string) throws

Start deserializing a record by reading the character (.

Arguments:

• reader – The fileReader to use when deserializing.

• name – The name of the record type

Returns:

A new AggregateDeserializer

record AggregateDeserializer

Returned by startClass or startRecord to provide the API for deserializing classes or records.

See AggregateSerializer for details of the default format for classes and records.

proc readField(name: string, type fieldType) : fieldType throws

Deserialize a field named name of type fieldType.

Returns:

A deserialized value of type fieldType.

proc readField(name: string, ref field) throws

Deserialize a field named name in-place.

proc startClass(reader: fileReader, name: string) throws

Start deserializing a nested class inside the current class.

See defaultSerializer.AggregateSerializer.startClass for details on inheritance on the default format.

Returns:

A new AggregateDeserializer

proc endClass() throws

End deserialization of the current class by reading the character }.

proc endRecord() throws

End deserialization of the current record by reading the character ).

proc startTuple(reader: fileReader) throws

Start deserializing a tuple by reading the character (.

Arguments:

reader – The fileReader to use when deserializing.

Returns:

A new TupleDeserializer

record TupleDeserializer

Returned by startTuple to provide the API for deserializing tuples.

See TupleSerializer for details of the default format for tuples.

proc readElement(type eltType) : eltType throws

Deserialize an element of the tuple.

Returns:

A deserialized value of type eltType.

proc readElement(ref element) throws

Deserialize element in-place as an element of the tuple.

proc endTuple() throws

End deserialization of the current tuple by reading the character ).

proc ref startList(reader: fileReader) throws

Start deserializing a list by reading the character [.

Arguments:

reader – The fileReader to use when deserializing.

Returns:

A new ListDeserializer

record ListDeserializer

Returned by startList to provide the API for deserializing lists.

See ListSerializer for details of the default format for lists.

proc ref readElement(type eltType) : eltType throws

Deserialize an element of the list.

Returns:

A deserialized value of type eltType.

proc ref readElement(ref element) throws

Deserialize element in-place as an element of the list.

proc endList() throws

End deserialization of the current list by reading the character ].

proc hasMore() : bool throws

Returns:

Returns true if there are more elements to read.

proc startArray(reader: fileReader) throws

Start deserializing an array.

Arguments:

reader – The fileReader to use when deserializing.

Returns:

A new ArrayDeserializer

record ArrayDeserializer

Returned by startArray to provide the API for deserializing arrays.

See ArraySerializer for details of the default format for arrays.

proc ref startDim() throws

Inform the ArrayDeserializer to start deserializing a new dimension.

proc ref endDim() throws

End deserialization of the current dimension.

proc ref readElement(type eltType) : eltType throws

Deserialize an element of the array.

Returns:

A deserialized value of type eltType.

proc ref readElement(ref element) throws

Deserialize element in-place as an element of the array.

proc endArray() throws

End deserialization of the current array.

proc startMap(reader: fileReader) throws

Start deserializing a map by reading the character {.

Arguments:

reader – The fileReader to use when deserializing.

Returns:

A new MapDeserializer

record MapDeserializer

Returned by startMap to provide the API for deserializing maps.

See MapSerializer for details of the default format for map.

proc ref readKey(type keyType) : keyType throws

Deserialize and return a key of type keyType.

proc ref readKey(ref key) throws

Deserialize key in-place as a key of the map.

proc readValue(type valType) : valType throws

Deserialize and return a value of type valType.

proc readValue(ref value) throws

Deserialize value in-place as a value of the map.

proc endMap() throws

End deserialization of the current map by reading the character }.

proc hasMore() : bool throws

Returns:

Returns true if there are more elements to read.

Warning

Behavior of ‘hasMore’ is undefined when called between readKey and readValue.

config param warnBinaryStructured : bool = true

Warning

This config param is unstable and may be removed without advance notice

This config param allows users to disable a warning for reading and writing classes and strings with binarySerializer and binaryDeserializer following a format change in the 1.33 release.

record binarySerializer

A binary Serializer that implements a simple binary format.

This Serializer supports an endian field which may be configured at execution time.

See the serializers technote for a general overview of Serializers and their usage.

Warning

In the 1.32 release this format included bytes representing the length of a string. Also, classes were serialized beginning with a single byte to indicate whether the class value was nil. This behavior was changed in the subsequent release to provide users with a more flexible serializer that did not insert bytes that the user did not request. A compile-time warning will be issued to indicate that this behavior has changed. Users can recompile with -swarnBinaryStructured=false to silence the warning.

To mimic the old behavior, please use the unstable ObjectSerialization module.

const endian : endianness = endianness.native

‘endian’ represents the endianness of the binary output produced by this Serializer.

proc ref serializeValue(writer: fileWriter(serializerType = binarySerializer, locking = false, ?), const val: ?t) throws

Serialize val with writer.

Numeric values like integers, real numbers, and complex numbers are serialized directly to the associated fileWriter as binary data in the specified endianness.

Booleans are serialized as single byte unsigned values of either 0 or 1.

string values are serialized as a raw sequence of bytes that does not include a null terminator, nor any bytes representing length. This means that string values cannot be deserialized without manual intervention by users to decide how their strings should be stored such that they can be deserialized.

The nil value is serialized as a single unsigned byte of value 0.

Classes and records will have their serialize method invoked, passing in writer and this Serializer as arguments. Please see the serializers technote for more on the serialize method.

Classes and records are expected to implement the writeSerializable interface. The serializable interface is also acceptable.

Note

Serializing and deserializing enums is not stable in this format.

Arguments:

• writer – The fileWriter used to write serialized output.

• val – The value to be serialized.

proc startClass(writer: fileWriter(?), name: string, size: int) throws

Start serializing a class and return a new AggregateSerializer.

Arguments:

• writer – The fileWriter to be used when serializing.

• name – The name of the class type.

• size – The number of fields in the class.

Returns:

A new AggregateSerializer

proc startRecord(writer: fileWriter(?), name: string, size: int) throws

Start serializing a record and return a new AggregateSerializer.

Arguments:

• writer – The fileWriter to be used when serializing.

• name – The name of the record type.

• size – The number of fields in the class.

Returns:

A new AggregateSerializer

record AggregateSerializer

Returned by startClass or startRecord to provide the API for serializing classes or records.

In this simple binary format, classes and records do not begin or end with any bytes indicating size, and instead serialize their field values in binarySerializer’s format.

For example, a record with two uint(8) fields with values 1 and 2 would be serialized as 0x01 followed by 0x02 (in raw binary).

proc writeField(name: string, const field: ?T) throws

Serialize field in binarySerializer’s format.

proc startClass(writer, name: string, size: int) throws

Start serializing a nested class inside the current class. In this binary format, this has no impact on the serialized output.

proc endClass() throws

End deserialization of this class.

proc endRecord() throws

End deserialization of this record.

proc startTuple(writer: fileWriter(?), size: int) throws

Start serializing a tuple and return a new TupleSerializer.

Arguments:

• writer – The fileWriter to be used when serializing.

• size – The number of elements in the tuple.

Returns:

A new TupleSerializer

record TupleSerializer

Returned by startTuple to provide the API for serializing tuples.

In this simple binary format, tuples do not begin or end with any bytes indicating size, and instead serialize their elements sequentially in binarySerializer’s format.

proc writeElement(const element: ?T) throws

Serialize element in binarySerializer’s format.

proc endTuple() throws

Ends serialization of the current tuple.

proc startList(writer: fileWriter(?), size: int) throws

Start serializing a list by serializing size.

Arguments:

• writer – The fileWriter to be used when serializing.

• size – The number of elements in the list.

Returns:

A new ListSerializer

record ListSerializer

Returned by startList to provide the API for serializing lists.

In this simple binary format, lists begin with the serialization of an int representing the size of the list. This data is then followed by the binary serialization of the specified number of elements.

proc writeElement(const element: ?) throws

Serialize element in binarySerializer’s format.

proc endList() throws

Ends serialization of the current list.

proc startArray(writer: fileWriter(?), size: int) throws

Start serializing an array and return a new ArraySerializer.

Arguments:

• writer – The fileWriter to be used when serializing.

• size – The number of elements in the array.

Returns:

A new ArraySerializer

record ArraySerializer

Returned by startArray to provide the API for serializing arrays.

In this simple binary format, arrays are serialized element by element in the order indicated by the caller of writeElement. Dimensions and the start or end of the array are not represented.

proc startDim(size: int) throws

Start serializing a new dimension of the array.

proc endDim() throws

Ends serialization of this dimension.

proc writeElement(const element: ?) throws

Serialize element in binarySerializer’s format.

proc writeBulkElements(data: c_ptr(?eltType), numElements: int) throws  where isNumericType(eltType)

Serialize numElements number of elements in data, provided that the element type of data is a numeric type.

This performance-motivated implementation of the optional writeBulkElements will write the elements of data in the order in which they are represented in memory.

Note

This method is only optimized for the case where the binarySerializer has been configured for native endianness.

Warning

This method should only be called when the data argument is located on the same locale as the underlying file of this serializer. Otherwise the c_ptr will be invalid.

proc endArray() throws

Ends serialization of the current array.

proc startMap(writer: fileWriter(?), size: int) throws

Start serializing a map by serializing size.

Arguments:

• writer – The fileWriter to be used when serializing.

• size – The number of entries in the map.

Returns:

A new MapSerializer

record MapSerializer

Returned by startMap to provide the API for serializing maps.

In this simple binary format, maps begin with the serialization of an int representing the size of the map. This data is then followed by the binary serialization of the specified number of key-value pairs. The binary serialization of a key-value pair has no structure, and simply consists of the serialization of the key followed by the serialization of the value.

proc writeKey(const key: ?) throws

Serialize key in binarySerializer’s format.

proc writeValue(const val: ?) throws

Serialize val in binarySerializer’s format.

proc endMap() throws

Ends serialization of the current map.

record binaryDeserializer

A binary Deserializer that implements a simple binary format.

This Deserializer supports an endian field which may be configured at execution time.

See the serializers technote for a general overview of Deserializers and their usage.

Otherwise, please refer to binarySerializer for a description of the binary format. Individual methods on this type may clarify relevant behavior specific to deserialization

Note

Deserializing string or bytes types will result in an IllegalArgumentError because these types cannot currently be deserialized with the raw nature of the format.

Warning

In the 1.32 release this format included bytes representing the length of a string. Also, classes were serialized beginning with a single byte to indicate whether the class value was nil. This behavior was changed in the subsequent release to provide users with a more flexible deserializer that did not read bytes that the user did not request. A compile-time warning will be issued to indicate that this behavior has changed. Users can recompile with -swarnBinaryStructured=false to silence the warning.

To mimic the old behavior, please use the unstable ObjectSerialization module.

const endian : IO.endianness = IO.endianness.native

‘endian’ represents the endianness that this Deserializer should use when deserializing input.

proc ref deserializeType(reader: fileReader(?), type readType) : readType throws

Deserialize type readType with reader.

Classes and records will be deserialized using an appropriate initializer, passing in reader and this Deserializer as arguments. If an initializer is unavailable, this method may invoke the class or record’s deserialize method. Please see the serializers technote for more.

Classes and records are expected to implement either the initDeserializable or readDeserializable interfaces (or both). The serializable interface is also acceptable.

Arguments:

• reader – The fileReader from which types are deserialized.

• readType – The type to be deserialized.

Returns:

A value of type readType.

proc ref deserializeValue(reader: fileReader(?), ref val: ?readType) : void throws

Deserialize from reader directly into val.

Like deserializeType, but reads into an initialized value rather than creating a new value. For classes and records, this method will first attempt to invoke a deserialize method. If the deserialize method is unavailable, this method may fall back on invoking a suitable initializer and assigning the resulting value into val. Please see the serializers technote for more.

Classes and records are expected to implement either the readDeserializable or initDeserializable interfaces (or both). The serializable interface is also acceptable.

Arguments:

• reader – The fileReader from which values are deserialized.

• val – The value into which this Deserializer will deserialize.

proc startClass(reader: fileReader(?), name: string) throws

Start deserializing a class by returning an AggregateDeserializer.

Arguments:

• reader – The fileReader to use when deserializing.

• name – The name of the class type.

Returns:

A new AggregateDeserializer

proc startRecord(reader: fileReader(?), name: string) throws

Start deserializing a record by returning an AggregateDeserializer.

Arguments:

• reader – The fileReader to use when deserializing.

• name – The name of the record type.

Returns:

A new AggregateDeserializer

record AggregateDeserializer

Returned by startClass or startRecord to provide the API for deserializing classes or records.

See binarySerializer.AggregateSerializer for details of the binary format for classes and records.

proc readField(name: string, type fieldType) : fieldType throws

Deserialize and return a value of type fieldType.

proc readField(name: string, ref field) throws

Deserialize field in-place.

proc startClass(reader, name: string) throws

Start deserializing a nested class inside the current class.

See binarySerializer.AggregateSerializer.startClass for details on inheritance on the binary format.

Returns:

A new AggregateDeserializer

proc endClass() throws

End deserialization of the current class.

proc endRecord() throws

End deserialization of the current record.

proc startTuple(reader: fileReader(?)) throws

Start deserializing a tuple by returning a TupleDeserializer.

Arguments:

reader – The fileReader to use when deserializing.

Returns:

A new TupleDeserializer

record TupleDeserializer

Returned by startTuple to provide the API for deserializing tuples.

See binarySerializer.TupleSerializer for details of the binary format for tuples.

proc readElement(type eltType) : eltType throws

Deserialize an element of the tuple.

Returns:

A deserialized value of type eltType.

proc readElement(ref element) throws

Deserialize element in-place as an element of the tuple.

proc endTuple() throws

End deserialization of the current tuple.

proc startList(reader: fileReader(?)) throws

Start deserializing a list by returning a ListDeserializer.

Arguments:

reader – The fileReader to use when deserializing.

Returns:

A new ListDeserializer

record ListDeserializer

Returned by startList to provide the API for deserializing lists.

See binarySerializer.ListSerializer for details of the binary format for lists.

proc ref readElement(type eltType) : eltType throws

Deserialize an element of the list.

Returns:

A deserialized value of type eltType.

proc ref readElement(ref element) throws

Deserialize element in-place as an element of the list.

proc endList() throws

End deserialization of the current list.

Throws:

A BadFormatError if there are remaining elements.

proc hasMore() : bool throws

Returns:

Returns true if there are more elements to read.

proc startArray(reader: fileReader(?)) throws

Start deserializing an array by returning an ArrayDeserializer.

Arguments:

reader – The fileReader to use when deserializing.

Returns:

A new ArrayDeserializer

record ArrayDeserializer

Returned by startArray to provide the API for deserializing arrays.

See binarySerializer.ArraySerializer for details of the binary format for arrays.

proc startDim() throws

Inform the ArrayDeserializer to start deserializing a new dimension.

proc endDim() throws

End deserialization of the current dimension.

proc readElement(type eltType) : eltType throws

Deserialize an element of the list.

Returns:

A deserialized value of type eltType.

proc readElement(ref element) throws

Deserialize element in-place as an element of the array.

proc readBulkElements(data: c_ptr(?eltType), numElements: int) throws  where isNumericType(eltType)

Deserialize numElements number of elements into data, provided that the element type of data is a numeric type.

This performance-motivated implementation of the optional readBulkElements will read the elements of data in the order in which they are represented in memory.

Note

This method is only optimized for the case where the binaryDeserializer has been configured for native endianness.

Warning

This method should only be called when the data argument is located on the same locale as the underlying file of this deserializer. Otherwise the c_ptr will be invalid.

proc endArray() throws

End deserialization of the current array.

proc startMap(reader: fileReader(?)) throws

Start deserializing a map by returning a MapDeserializer.

Arguments:

reader – The fileReader to use when deserializing.

Returns:

A new MapDeserializer

record MapDeserializer

Returned by startMap to provide the API for deserializing maps.

See binarySerializer.MapSerializer for details of the binary format for map.

proc ref readKey(type keyType) : keyType throws

Deserialize and return a key of type keyType.

proc ref readKey(ref key) throws

Deserialize key in-place as a key of the map.

proc readValue(type valType) : valType throws

Deserialize and return a value of type valType.

proc readValue(ref value) throws

Deserialize value in-place as a value of the map.

proc endMap() throws

End deserialization of the current map.

Throws:

A BadFormatError if there are entries remaining.

proc hasMore() : bool throws

Returns:

Returns true if there are more elements to read.

Warning

Behavior of ‘hasMore’ is undefined when called between readKey and readValue.

proc fileReader.withDeserializer(type deserializerType) : fileReader(this.locking, deserializerType)

Create and return an alias of this fileReader configured to use deserializerType for deserialization. The provided deserializerType must be able to be default-initialized.

Warning

It is an error for the returned alias to outlive the original fileReader.

proc fileReader.withDeserializer(in deserializer: ?dt) : fileReader(this.locking, dt)

Create and return an alias of this fileReader configured to use deserializer for deserialization.

Warning

It is an error for the returned alias to outlive the original fileReader.

proc fileWriter.withSerializer(type serializerType) : fileWriter(this.locking, serializerType)

Create and return an alias of this fileWriter configured to use serializerType for serialization. The provided serializerType must be able to be default-initialized.

Warning

It is an error for the returned alias to outlive the original fileWriter.

proc fileWriter.withSerializer(in serializer: ?st) : fileWriter(this.locking, st)

Create and return an alias of this fileWriter configured to use serializer for serialization.

Warning

It is an error for the returned alias to outlive the original fileWriter.

proc fileReader.lock() throws

Acquire a fileReader’s lock. See Locking Behavior of FileReaders and FileWriters for more details.

Throws:

SystemError – If the lock could not be acquired.

proc fileWriter.lock() throws

Acquire a fileWriter’s lock. See Locking Behavior of FileReaders and FileWriters for more details.

Throws:

SystemError – If the lock could not be acquired.

proc fileReader.unlock()

Release a fileReader’s lock. See Locking Behavior of FileReaders and FileWriters for more details.

proc fileWriter.unlock()

Release a fileWriter’s lock. See Locking Behavior of FileReaders and FileWriters for more details.

proc fileReader.offset() : int(64)

Return the current offset of a fileReader.

If the fileReader can be used by multiple tasks, take care when doing operations that rely on the fileReader’s current offset. To prevent race conditions, lock the fileReader with fileReader.lock before calling fileReader.offset, then unlock it afterwards with fileReader.unlock.

Returns:

the current offset of the fileReader

proc fileWriter.offset() : int(64)

Return the current offset of a fileWriter.

If the fileWriter can be used by multiple tasks, take care when doing operations that rely on the fileWriter’s current offset. To prevent race conditions, lock the fileWriter with fileWriter.lock before calling fileWriter.offset, then unlock it afterwards with fileWriter.unlock.

Returns:

the current offset of the fileWriter

proc fileReader.advance(amount: int(64)) throws

Move a fileReader offset forward.

This routine will consume the next amount bytes from the file, storing them in the fileReader’s buffer. This can be useful for advancing to some known offset in the file before reading.

Note that calling fileReader.mark before advancing will cause at least amount bytes to be retained in memory until commit or revert are called. As such, it is typical to advance by a small number of bytes during an I/O transaction.

To make large adjustments to the offset, consider creating a new fileReader or using seek instead.

Throws:

• EofError – If EOF is reached before the requested number of bytes can be consumed. The offset will be left at EOF.

• SystemError – For other failures, for which fileReader offset is not moved.

proc fileWriter.advance(amount: int(64)) throws

Move a fileWriter offset forward.

This routine will populate the fileWriter’s buffer as the offset is moved forward by amount bytes. The buffer can be populated with any of the following data depending on the fileWriter’s configuration and whether it was marked before advancing:

• zeros

• bytes directly from the file

• bytes from a previously buffered portion of the file

The contents of the buffer will subsequently be written to the file by the buffering mechanism.

Note that calling fileWriter.mark before advancing will cause at least amount bytes to be retained in memory until commit or revert are called. As such, it is typical to advance by a small number of bytes during an I/O transaction.

To make large adjustments to the offset, consider creating a new fileWriter or using seek instead.

Throws:

• EofError – If EOF is reached before the offset can be advanced by the requested number of bytes. The offset will be left at EOF.

• SystemError – For other failures, for which fileWriter offset is not moved.

proc fileReader.advanceThrough(separator: ?t) throws  where t == string || t == bytes

Read until a separator is found, leaving the fileReader offset just after it.

If the separator cannot be found, the fileReader offset is left at EOF and an UnexpectedEofError is thrown.

Note

The implementation is faster for single-byte string or bytes separators.

Arguments:

separator – The separator to match with. Must be a string or bytes.

Throws:

• IllegalArgumentError – If the separator is empty

• EofError – If the fileReader offset was already at EOF.

• UnexpectedEofError – If the requested separator could not be found.

• SystemError – If data could not be read from the file. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.advanceThroughNewline() throws

Read until a newline is found, leaving the fileReader offset just after it.

If a newline cannot be found, the fileReader offset is left at EOF and an UnexpectedEofError is thrown.

Throws:

• EofError – If the fileReader offset was already at EOF.

• UnexpectedEofError – A newline couldn’t be found before the end of the file.

• SystemError – If data could not be read from the file. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.advanceTo(separator: ?t) throws  where t == string || t == bytes

Read until a separator is found, leaving the fileReader offset just before it.

If the separator cannot be found, the fileReader offset is left at EOF and an UnexpectedEofError is thrown.

Note

The implementation is faster for single-byte string or bytes separators.

Arguments:

separator – The separator to match with. Must be a string or bytes.

Throws:

• IllegalArgumentError – If the separator is empty

• EofError – If the fileReader offset is already at EOF.

• UnexpectedEofError – If the requested separator could not be found.

• SystemError – If data could not be read from the fileReader. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.mark() throws

Mark a fileReader - that is, save the current offset of the fileReader on its mark stack.

The mark stack stores several file offsets. The fileReader will keep the region of the file between its minimum and maximum mark stack values buffered in memory so that IO operations can be undone. As a result, it is possible to perform I/O transactions on a fileReader. The basic steps for an I/O transaction are:

• mark the current offset with fileReader.mark

• do something speculative (e.g. try to read 200 bytes of anything followed by a ‘B’)

• if the speculative operation was successful, commit the changes by calling fileReader.commit

• if the speculative operation was not successful, go back to the mark by calling fileReader.revert. Subsequent I/O operations will work as though nothing happened.

If a fileReader has locking==true, mark should only be called once it has been locked with fileReader.lock. The fileReader should not be unlocked with fileReader.unlock until after the mark has been committed with commit or reverted with revert.

See I/O Transactions for more.

Note

Note that it is possible to request an entire file be buffered in memory using this feature, for example by marking at offset=0 and then advancing to the end of the file. It is important to be aware of these memory space requirements.

Returns:

The offset that was marked

Throws:

SystemError – if marking the fileReader failed

proc fileWriter.mark() throws

Mark a fileWriter - that is, save the current offset of the fileWriter on its mark stack.

The mark stack stores several file offsets. The fileWriter will keep the region of the file between its minimum and maximum mark stack values buffered in memory so that IO operations can be undone. As a result, it is possible to perform I/O transactions on a fileWriter. The basic steps for an I/O transaction are:

• mark the current offset with fileWriter.mark

• do something speculative (e.g. try to write 200 bytes)

• if the speculative operation was successful, commit the changes by calling fileWriter.commit

• if the speculative operation was not successful, go back to the mark by calling fileWriter.revert. Subsequent I/O operations will work as though nothing happened.

If a fileWriter has locking==true, mark should only be called once it has been locked with fileWriter.lock. The fileWriter should not be unlocked with fileWriter.unlock until after the mark has been committed with commit or reverted with revert.

See I/O Transactions for more.

Note

Note that it is possible to request an entire file be buffered in memory using this feature, for example by marking at offset=0 and then advancing to the end of the file. It is important to be aware of these memory space requirements.

Returns:

The offset that was marked

Throws:

SystemError – if marking the fileWriter failed

proc fileReader.revert()

Abort an I/O transaction by popping from the fileReader’s mark stack and adjusting its position to that offset. See I/O Transactions for more.

This routine should only be called on a fileReader that has already been marked. If called on a fileReader with locking=true, the fileReader should have already been locked manually with lock before mark was called.

proc fileWriter.revert()

Abort an I/O transaction by popping from the fileWriter’s mark stack and adjusting its position to that offset. See I/O Transactions for more.

This routine should only be called on a fileWriter that has already been marked. If called on a fileWriter with locking=true, the fileWriter should have already been locked manually with lock before mark was called.

proc fileReader.commit()

Commit an I/O transaction by popping from the fileReader’s mark stack and leaving its position in the file unchanged. See I/O Transactions for more.

This routine should only be called on a fileReader that has already been marked. If called on a fileReader with locking=true, the fileReader should have already been locked manually with lock before mark was called.

proc fileWriter.commit()

Commit an I/O transaction by popping from the fileWriter’s mark stack and leaving its position in the file unchanged. See I/O Transactions for more.

This routine should only be called on a fileWriter that has already been marked. If called on a fileWriter with locking=true, the fileWriter should have already been locked manually with lock before mark was called.

proc fileReader.seek(region: range(?)) throws

Adjust a fileReader’s region. The fileReader’s buffer will be discarded.

This routine has the following constraints:

• the underlying file must be seekable (sockets and pipes are not seekable)

• the fileReader must be non-locking (to avoid race conditions if two tasks seek and read simultaneously)

• the fileReader must not be marked (see: fileReader.mark)

If the fileReader offset needs to be updated during an I/O transaction or if discarding the buffer will incur a performance penalty, consider using fileReader.advance instead.

Arguments:

region – the new region, measured in bytes and counting from 0. An upper bound can be omitted (e.g., r.seek(range=42..)). See region for more.

Warning

The region argument will ignore any specified stride other than 1.

Throws:

• SystemError – if seeking failed. Possible reasons include that the file is not seekable, or that the fileReader is marked.

• IllegalArgumentError – if region argument did not have a lower bound

proc fileWriter.seek(region: range(?)) throws

Adjust a fileWriter’s region. The fileWriter’s buffer will be discarded.

This routine has the following constraints:

• the underlying file must be seekable (sockets and pipes are not seekable)

• the fileWriter must be non-locking (to avoid race conditions if two tasks seek and read simultaneously)

• the fileWriter must not be marked (see: fileWriter.mark)

If the fileWriter offset needs to be updated during an I/O transaction or if discarding the buffer will incur a performance penalty, consider using fileWriter.advance instead.

Arguments:

region – the new region, measured in bytes and counting from 0. An upper bound can be omitted (e.g., w.seek(range=42..)). See region for more.

Warning

The region argument will ignore any specified stride other than 1.

Throws:

• SystemError – if seeking failed. Possible reasons include that the file is not seekable, or that the fileReader is marked.

• IllegalArgumentError – if region argument did not have a lower bound

proc fileReader.readWriteThisFromLocale()

Warning

‘readWriteThisFromLocale’ is unstable and may be removed or modified in a future release

Return the locale on which an ongoing I/O was started with a fileReader. This method will return nilLocale unless it is called on a fileReader that is the formal argument to a readThis method.

proc fileWriter.readWriteThisFromLocale()

Warning

‘readWriteThisFromLocale’ is unstable and may be removed or modified in a future release

Return the locale on which an ongoing I/O was started with a fileWriter. This method will return nilLocale unless it is called on a fileWriter that is the formal argument to a writeThis method.

proc openReader(path: string, param locking = false, region: range(?) = 0.., hints = ioHintSet.empty, in deserializer: ?dt = defaultSerializeVal(false)) : fileReader(locking, dt) throws

Open a file at a particular path and return a fileReader for it. This function is equivalent to calling open and then file.reader on the resulting file.

Arguments:

• path – which file to open (for example, “some/file.txt”).

• locking – compile-time argument to determine whether or not the fileReader should use locking; sets the corresponding parameter of the fileReader type. Defaults to false

• region – zero-based byte offset indicating where in the file the fileReader should start and stop reading. Defaults to 0.., meaning from the start of the file to no specified end point.

• hints – optional argument to specify any hints to the I/O system about this file. See ioHintSet.

• deserializer – deserializer to use when reading.

Returns:

an open fileReader to the requested resource.

Note

locking=true should only be used when a fileReader will be used by multiple tasks concurrently.

Warning

The region argument will ignore any specified stride other than 1.

Throws:

• FileNotFoundError – If part of the provided path did not exist

• PermissionError – If part of the provided path had inappropriate permissions

• NotADirectoryError – If part of the provided path was expected to be a directory but was not

• SystemError – If a fileReader could not be returned.

• IllegalArgumentError – If trying to read explicitly prior to byte 0.

proc openStringReader(const s: string, in deserializer: ?dt = defaultSerializeVal(false)) : fileReader(false, dt) throws

Warning

‘openStringReader’ is an experimental feature; its name and behavior are subject to change

Create a fileReader around a string

Note that the string is copied into a local memory file, so it can be modified after the fileReader is created without affecting the contents of the fileReader.

Arguments:

• s – the string to read from

• deserializer – deserializer to use when reading.

Returns:

a fileReader reading from the string

proc openBytesReader(const b: bytes, in deserializer: ?dt = defaultSerializeVal(false)) : fileReader(false, dt) throws

Warning

‘openBytesReader’ is an experimental feature; its name and behavior are subject to change

Create a fileReader around a bytes

Note that the bytes is copied into a local memory file, so it can be modified after the fileReader is created without affecting the contents of the fileReader.

Arguments:

• b – the bytes to read from

• deserializer – deserializer to use when reading.

Returns:

a fileReader reading from the string

proc openWriter(path: string, param locking = false, hints = ioHintSet.empty, in serializer: ?st = defaultSerializeVal(true)) : fileWriter(locking, st) throws

Open a file at a particular path and return a fileWriter for it. This function is equivalent to calling open with ioMode.cwr and then file.writer on the resulting file.

Arguments:

• path – which file to open (for example, “some/file.txt”).

• locking – compile-time argument to determine whether or not the fileWriter should use locking; sets the corresponding parameter of the fileWriter type. Defaults to false

• hints – optional argument to specify any hints to the I/O system about this file. See ioHintSet.

• serializer – serializer to use when writing.

Returns:

an open fileWriter to the requested resource.

Note

locking=true should only be used when a fileWriter will be used by multiple tasks concurrently.

Throws:

• FileNotFoundError – If part of the provided path did not exist

• PermissionError – If part of the provided path had inappropriate permissions

• NotADirectoryError – If part of the provided path was expected to be a directory but was not

• SystemError – If a fileWriter could not be returned.

• IllegalArgumentError – If trying to write explicitly prior to byte 0.

proc file.reader(param locking = false, region: range(?) = 0.., hints = ioHintSet.empty, in deserializer: ?dt = defaultSerializeVal(false)) : fileReader(locking, dt) throws

Create a fileReader that supports reading from a file. See I/O Overview.

The region= argument defines the portion of the file that the fileReader will read from. This is a byte offset; the beginning of the file is at the offset 0. The default for this argument enables the fileReader to access the entire file.

A fileReader will never read beyond its maximum end offset. In addition, reading from a fileReader beyond the end of the underlying file will not extend that file. Reading beyond the end of the file or beyond the end offset of the fileReader will produce the error OS.EofError (or just return false in many cases such as fileReader.read) to indicate that the end was reached.

Arguments:

• locking – compile-time argument to determine whether or not the fileReader should use locking; sets the corresponding parameter of the fileReader type. Defaults to false.

• region – zero-based byte offset indicating where in the file the fileReader should start and stop reading. Defaults to 0.. - meaning from the start of the file to no end point.

• hints – provide hints about the I/O that this fileReader will perform. See ioHintSet. The default value of ioHintSet.empty will cause the fileReader to use the hints provided when the file was opened.

• deserializer – deserializer to use when reading.

Warning

The region argument will ignore any specified stride other than 1.

Throws:

• SystemError – If a fileReader could not be returned.

• IllegalArgumentError – If trying to read explicitly prior to byte 0.

proc file.writer(param locking = false, region: range(?) = 0.., hints = ioHintSet.empty, in serializer: ?st = defaultSerializeVal(true)) : fileWriter(locking, st) throws

Create a fileWriter that supports writing to a file. See I/O Overview.

The region= argument defines the portion of the file that the fileWriter will write to. This is a byte offset; the beginning of the file is at the offset 0. The default for this argument enables the fileWriter to access the entire file.

When a fileWriter writes to a file, it will replace file data that was previously stored at the relevant offset. If the offset is beyond the end of the file, the file will be extended.

A fileWriter will never write beyond its maximum end offset. It will extend the file only as necessary to store data written to the fileWriter. In other words, specifying the high bound of the region argument here does not impact the file size directly; it impacts only the section of the file that this fileWriter can write to. After all fileWriters to a file are closed, that file will have a size equal to the last offset written to by any fileWriter.

Arguments:

• locking – compile-time argument to determine whether or not the fileWriter should use locking; sets the corresponding parameter of the fileWriter type. Defaults to false.

• region – zero-based byte offset indicating where in the file the fileWriter should start and stop writing. Defaults to 0.. - meaning from the start of the file to no specified end point.

• hints – provide hints about the I/O that this fileWriter will perform. See ioHintSet. The default value of ioHintSet.empty will cause the fileWriter to use the hints provided when the file was opened.

• serializer – serializer to use when writing.

Warning

The region argument will ignore any specified stride other than 1.

Throws:

• SystemError – If a fileWriter could not be returned.

• IllegalArgumentError – If trying to write explicitly prior to byte 0.

proc fileReader.readLiteral(literal: string, ignoreWhitespace = true) : void throws

Advances the offset of a fileReader within the file by reading the exact text of the given string literal from the fileReader.

If the string is not matched exactly, then the fileReader’s offset is unchanged. In such cases a OS.BadFormatError will be thrown, unless the end of the fileReader is encountered in which case an OS.EofError will be thrown.

By default this method will ignore leading whitespace in the file when attempting to read a literal (leading whitespace in the literal itself is still matched against whitespace in the file).

Arguments:

• literal – the string to be matched.

• ignoreWhitespace – determines whether leading whitespace is ignored.

Throws:

• BadFormatError – If literal could not be matched.

• EofError – If end of fileReader is encountered.

proc fileReader.readLiteral(literal: bytes, ignoreWhitespace = true) : void throws

Advances the offset of a fileReader by reading the exact bytes of the given literal from the fileReader.

If the bytes are not matched exactly, then the fileReader’s offset is unchanged. In such cases a OS.BadFormatError will be thrown, unless the end of the fileReader is encountered in which case an OS.EofError will be thrown.

By default this method will ignore leading whitespace in the file when attempting to read a literal (leading whitespace in the literal itself is still matched against whitespace in the file).

Arguments:

• literal – the bytes to be matched.

• ignoreWhitespace – determines whether leading whitespace is ignored.

Throws:

• BadFormatError – If literal could not be matched.

• EofError – If end of the fileReader is encountered.

proc fileReader.readNewline() : void throws

Advances the offset of the fileReader by reading a newline.

If a newline is not matched exactly, then the fileReader’s offset is unchanged. In such cases a OS.BadFormatError will be thrown, unless the end of the fileReader is encountered in which case an OS.EofError will be thrown. By default this method will ignore leading whitespace when attempting to read a newline.

Throws:

• BadFormatError – If a newline could not be matched.

• EofError – If end of the fileReader is encountered.

proc fileReader.matchLiteral(literal: string, ignoreWhitespace = true) : bool throws

Advances the offset of a fileReader by reading the exact text of the given string literal from the fileReader.

If the string is not matched exactly, then the fileReader’s offset is unchanged and this method will return false. In other words, this fileReader will return false in the cases where fileReader.readLiteral would throw a OS.BadFormatError or an OS.EofError.

By default this method will ignore leading whitespace in the file when attempting to read a literal (leading whitespace in the literal itself is still matched against whitespace in the file).

Arguments:

• literal – the string to be matched.

• ignoreWhitespace – determines whether leading whitespace is ignored.

Returns:

true if the read succeeded, and false on end of file or if the literal could not be matched.

proc fileReader.matchLiteral(literal: bytes, ignoreWhitespace = true) : bool throws

Advances the offset of a fileReader by reading the exact bytes of the given literal from the fileReader.

If the bytes are not matched exactly, then the fileReader’s offset is unchanged and this method will return false. In other words, this fileReader will return false in the cases where fileReader.readLiteral would throw a OS.BadFormatError or an OS.EofError.

By default this method will ignore leading whitespace in the file when attempting to read a literal (leading whitespace in the literal itself is still matched against whitespace in the file).

Arguments:

• literal – the bytes to be matched.

• ignoreWhitespace – determines whether leading whitespace is ignored.

Returns:

true if the read succeeded, and false on end of file or if the literal could not be matched.

proc fileReader.matchNewline() : bool throws

Advances the offset of the fileReader by reading a newline.

If a newline is not matched exactly, then the fileReader’s offset is unchanged and this method will return false. In other words, this fileReader will return false in the cases where fileReader.readNewline would throw a OS.BadFormatError or an OS.EofError.

By default this method will ignore leading whitespace when attempting to read a newline.

Returns:

true if the read succeeded, and false on end of file or if the newline could not be matched.

proc fileWriter.writeLiteral(literal: string) : void throws

Writes a string to the fileWriter, ignoring any formatting configured for this fileWriter.

proc fileWriter.writeLiteral(literal: bytes) : void throws

Writes bytes to the fileWriter, ignoring any formatting configured for this fileWriter.

proc fileWriter.writeNewline() : void throws

Writes a newline to the fileWriter, ignoring any formatting configured for this fileWriter.

iter fileReader.lines(stripNewline = false, type t = string) : t  where t == string || t == bytes

Iterate over all of the lines ending in \n in a fileReader - the fileReader lock will be held while iterating over the lines.

This iterator will halt on internal system errors. In the future, iterators are intended to be able to throw in such cases.

Example:

var r = openReader("ints.txt"),
    sum = 0;

forall line in r.lines() with (+ reduce sum) {
  sum += line:int;
}

Warning

This iterator executes on the current locale. This may impact performance if the current locale is not the same locale on which the fileReader was created.

Warning

Parallel iteration is not supported for sockets, pipes, or other non-file-based sources

Warning

Zippered parallel iteration is not yet supported for this iterator.

Arguments:

• stripNewline – Whether to strip the trailing \n from the line — defaults to false

• t – The type of the lines to read (either string or bytes) — defaults to string

Yields:

lines from the fileReader, by default with a trailing \n

iter fileReader.lines(stripNewline = false, type t = string, targetLocales: [] locale) : t  where t == string || t == bytes

Iterate over all of the lines ending in \n in a fileReader using multiple locales - the fileReader lock will be held while iterating over the lines.

This iterator will halt on internal system errors. In the future, iterators are intended to be able to throw in such cases.

Example:

var r = openReader("ints.txt"),
    sum = 0;

forall line in r.lines(targetLocales=Locales) with (+ reduce sum) {
  sum += line:int;
}

Warning

Parallel iteration is not supported for sockets, pipes, or other non-file-based sources

Warning

This procedure does not support reading from a memory-file (e.g., files opened with openMemFile)

Warning

Zippered parallel iteration is not yet supported for this iterator.

Arguments:

• stripNewline – Whether to strip the trailing \n from the line — defaults to false

• targetLocales – The locales on which to read the lines (parallel iteration only)

• t – The type of the lines to read (either string or bytes) — defaults to string

Yields:

lines from the fileReader, by default with a trailing \n

proc fileReader.read(ref args ...?k) : bool throws

Read one or more values from a fileReader. The fileReader’s lock will be held while reading the values — this protects against interleaved reads.

Arguments:

args – a series of variables to read into. Basic types are handled internally, but for other types this function will call value.deserialize() with a fileReader argument as described in The ‘serialize’ and ‘deserialize’ Methods.

Returns:

true if the read succeeded, and false on end of file.

Throws:

• UnexpectedEofError – If an EOF occurred while reading an item.

• SystemError – If data could not be read from the fileReader for another reason.

proc fileReader.readLine(ref a: [] ?t, maxSize = a.size, stripNewline = false) : int throws  where a.rank == 1 && a.isRectangular() && a.strides == strideKind.one && (t == uint(8) || t == int(8))

Read a line into an array of bytes.

Reads bytes from the fileReader until a \n is reached. Values are read in binary format (i.e., this method is not aware of UTF-8 encoding).

The array’s size is not changed to accommodate bytes. If a newline is not found before the array is filled, or maxSize bytes are read, a BadFormatError is thrown and the fileReader offset is returned to the position it had when this routine was called.

Arguments:

• a – A 1D DefaultRectangular non-strided array storing int(8) or uint(8). Values are overwritten.

• maxSize – The maximum number of bytes to store into the a array. Defaults to the size of the array.

• stripNewline – Whether to strip the trailing \n from the line. If true, the newline isn’t counted in the number of bytes read.

Returns:

The number of array elements set by this call, or 0 otherwise (i.e., the fileReader was already at EOF).

Throws:

• IllegalArgumentError – If maxSize > a.size

• BadFormatError – If the line is longer than maxSize. The fileReader’s offset is not moved in that case.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readLine(ref s: string, maxSize = -1, stripNewline = false) : bool throws

Read a line into a string. Reads until a \n is reached.

Arguments:

• s – the string to read into. Contents are overwritten.

• maxSize – The maximum number of codepoints to store into s. The default of -1 means to read an unlimited number of codepoints.

• stripNewline – Whether to strip the trailing \n from the line.

Returns:

true if a line was read without error, false upon EOF

Throws:

• BadFormatError – If the line is longer than maxSize. The fileReader offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readLine(ref b: bytes, maxSize = -1, stripNewline = false) : bool throws

Read a line into a bytes. Reads until a \n is reached.

Arguments:

• b – the bytes to receive the line. Contents are overwritten.

• maxSize – The maximum number of bytes to store into b. The default of -1 means to read an unlimited number of bytes.

• stripNewline – Whether to strip the trailing \n from the line.

Returns:

true if a line was read without error, false upon EOF

Throws:

• BadFormatError – If the line is longer than maxSize. The file offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readLine(type t = string, maxSize = -1, stripNewline = false) : t throws  where t == string || t == bytes

Read a line. Reads until a \n is reached.

Arguments:

• t – the type of data to read, which must be string or bytes. Defaults to string if not specified.

• maxSize – The maximum number of codepoints to read. The default of -1 means to read an unlimited number of codepoints.

• stripNewline – Whether to strip the trailing \n from the line.

Returns:

A string or bytes with the contents of the fileReader up to (and possibly including) the newline.

Throws:

• EofError – If nothing could be read because the fileReader was already at EOF.

• BadFormatError – If the line is longer than maxSize. The file offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readThrough(separator: ?t, maxSize = -1, stripSeparator = false) : t throws  where t == string || t == bytes

Read until the given separator is found, returning the contents of the fileReader through that point.

If the separator is found, the fileReader offset is left immediately after it. If the separator could not be found in the next maxSize bytes, a BadFormatError is thrown and the fileReader’s offset is not changed. Otherwise, if EOF is reached before finding the separator, the remainder of the fileReader’s contents are returned and the offset is left at EOF.

To match with multiple separators, or a more complex separator, use the overload of readThrough that accepts a regex separator.

Arguments:

• separator – The separator to match with. Must be a string or bytes.

• maxSize – The maximum number of bytes (for t==bytes) or codepoints (for t==string) to read. For the default value of -1, this method can read until EOF.

• stripSeparator – Whether to strip the separator from the returned string or bytes. If true, the returned value will not include the separator.

Returns:

A string or bytes with the contents of the fileReader up to (and possibly including) the separator.

Throws:

• IllegalArgumentError – If the separator is empty

• EofError – If nothing could be read because the fileReader was already at EOF.

• BadFormatError – If the separator was not found in the next maxSize bytes. The fileReader offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readThrough(separator: string, ref s: string, maxSize = -1, stripSeparator = false) : bool throws

Read until the given separator is found, returning the contents of the fileReader through that point.

See the above overload of this method for more details.

Arguments:

• separator – The separator to match with.

• s – The string to read into. Contents will be overwritten.

• maxSize – The maximum number of codepoints to read. For the default value of -1, this method can read until EOF.

• stripSeparator – Whether to strip the separator from the returned string. If true, the separator will not be included in s.

Returns:

true if something was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

• IllegalArgumentError – If the separator is empty

• BadFormatError – If the separator was not found in the next maxSize bytes. The fileReader offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readThrough(separator: bytes, ref b: bytes, maxSize = -1, stripSeparator = false) : bool throws

Read until the given separator is found, returning the contents of the fileReader through that point.

See the above overload of this method for more details.

Arguments:

• separator – The separator to match with.

• b – The bytes to read into. Contents will be overwritten.

• maxSize – The maximum number of bytes to read. For the default value of -1, this method can read until EOF.

• stripSeparator – Whether to strip the separator from the returned bytes. If true, the separator will not be included in b.

Returns:

true if something was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

• IllegalArgumentError – If the separator is empty

• BadFormatError – If the separator was not found in the next maxSize bytes. The fileReader offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readTo(separator: ?t, maxSize = -1) : t throws  where t == string || t == bytes

Read until the given separator is found, returning the contents of the fileReader up to that point.

If the separator is found, the fileReader offset is left immediately before it. If the separator could not be found in the next maxSize bytes, a BadFormatError is thrown and the fileReader’s offset is not changed. Otherwise, if EOF is reached before finding the separator, the remainder of the fileReader’s contents are returned and the offset is left at EOF.

To match with multiple separators, or a more complex separator, use the overload of readTo that accepts a regex separator.

Arguments:

• separator – The separator to match with. Must be a string or bytes.

• maxSize – The maximum number of bytes (for t==bytes) or codepoints (for t==string) to read. For the default value of -1, this method can read until EOF.

Returns:

A string or bytes with the contents of the fileReader up to the separator.

Throws:

• IllegalArgumentError – If the separator is empty

• EofError – If nothing could be read because the fileReader was already at EOF.

• BadFormatError – If the separator was not found in the next maxSize bytes. The fileReader offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readTo(separator: string, ref s: string, maxSize = -1) : bool throws

Read until the given separator is found, returning the contents of the fileReader up to that point.

See the above overload of this method for more details.

Arguments:

• separator – The separator to match with.

• s – The string to read into. Contents will be overwritten.

• maxSize – The maximum number of codepoints to read. For the default value of -1, this method will read until EOF.

Returns:

true if something was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

• IllegalArgumentError – If the separator is empty

• BadFormatError – If the separator was not found in the next maxSize bytes. The fileReader offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readTo(separator: bytes, ref b: bytes, maxSize = -1) : bool throws

Read until the given separator is found, returning the contents of the fileReader up to that point.

See the above overload of this method for more details.

Arguments:

• separator – The separator to match with.

• b – The bytes to read into. Contents will be overwritten.

• maxSize – The maximum number of bytes to read. For the default value of -1, this method will read until EOF.

Returns:

true if something was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

• IllegalArgumentError – If the separator is empty

• BadFormatError – If the separator was not found in the next maxSize bytes. The fileReader offset is not moved.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset is not moved by this routine.

proc fileReader.readAll(type t = bytes) : t throws  where t == string || t == bytes

Read the remaining contents of the fileReader into an instance of the specified type

Arguments:

t – the type to read into; must be string or bytes. Defaults to bytes if not specified.

Returns:

the contents of the fileReader as a t

Throws:

• EofError – If nothing could be read because the fileReader was already at EOF.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readAll(ref s: string) : int throws

Read the remaining contents of the fileReader into a string.

Note that any existing contents of the string are overwritten.

Arguments:

s – the string to read into

Returns:

the number of codepoints that were stored in s, or 0 if the fileReader is at EOF.

Return type:

int

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readAll(ref b: bytes) : int throws

Read the remaining contents of the fileReader into a bytes.

Note that any existing contents of the bytes are overwritten.

Arguments:

b – the bytes to read into

Returns:

the number of bytes that were stored in b, or 0 if the fileReader is at EOF.

Return type:

int

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readAll(ref a: [?d] ?t) : int throws  where a.rank == 1 && a.isRectangular() && a.strides == strideKind.one && (t == uint(8) || t == int(8))

Read the remaining contents of the fileReader into an array of bytes.

Note that this routine currently requires a 1D rectangular non-strided array.

If the remaining contents of the fileReader exceed the size of a, the first a.size bytes will be read into a, and then an InsufficientCapacityError will be thrown. In such a case, the fileReader offset is advanced a.size bytes from its original position.

Arguments:

a – the array of bytes to read into

Returns:

the number of bytes that were stored in a

Return type:

int

Throws:

• InsufficientCapacityError – If the fileReader’s contents do not fit into a.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readString(maxSize: int) : string throws

Read a given number of codepoints from a fileReader, returning a new string.

The string’s length may be less than maxSize if EOF is reached while reading. If nothing is read, the empty string ("") will be returned.

Arguments:

maxSize – the maximum number of codepoints to read from the fileReader

Returns:

a new string containing up to the next maxSize codepoints from the fileReader

Throws:

• EofError – If the fileReader offset was already at EOF.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readString(ref s: string, maxSize: int) : bool throws

Read a given number of codepoints from a fileReader into a string.

The updated string’s length may be less than maxSize if EOF is reached while reading. If nothing is read, it will be set to the empty string ("").

Arguments:

• s – the string to read into — contents will be overwritten

• maxSize – the maximum number of codepoints to read from the fileReader

Returns:

true if something was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readBytes(maxSize: int) : bytes throws

Read a given number of bytes from a fileReader, returning a new bytes.

The bytes’s length may be less than maxSize if EOF is reached while reading. If nothing is read, the empty bytes (b"") will be returned.

Arguments:

maxSize – the maximum number of bytes to read from the fileReader

Returns:

a new bytes containing up to the next maxSize bytes from the fileReader

Throws:

• EofError – If the fileReader offset was already at EOF.

• SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readBytes(ref b: bytes, maxSize: int) : bool throws

Read a given number of bytes from a fileReader into a bytes.

The updated bytes’s length may be less than maxSize if EOF is reached while reading. If nothing is read, it will be set to the empty bytes (b"").

Arguments:

• b – the bytes to read into — contents will be overwritten

• maxSize – the maximum number of bytes to read from the fileReader

Returns:

true if something was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readBits(ref x: integral, numBits: int) : bool throws

Read bits with binary I/O

Arguments:

• x – where to store the read bits. This value will have its numBits least-significant bits set.

• numBits – how many bits to read

Returns:

true if the bits were read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

• UnexpectedEofError – If EOF was encountered before numBits could be read.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.readBits(type resultType, numBits: int) : resultType throws

Read bits with binary I/O

Arguments:

• resultType – type of the value returned

• numBits – how many bits to read

Returns:

bits read. This value will have its numBits least-significant bits set

Throws:

• EofError – If the fileReader offset was already at EOF.

• UnexpectedEofError – If EOF was encountered before numBits could be read.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileWriter.writeBits(x: integral, numBits: int) : void throws

Write bits with binary I/O

Arguments:

• x – a value containing numBits bits to write the least-significant bits

• numBits – how many bits to write

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified range.

• IllegalArgumentError – If writing more bits than fit into x.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileWriter.writeCodepoint(codepoint: int) throws

Write a single Unicode codepoint to a fileWriter

Arguments:

codepoint – Unicode codepoint to write

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified range.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileReader.readCodepoint() : int throws

Read a single Unicode codepoint from a fileReader

Returns:

Unicode codepoint read

Throws:

• EofError – If the fileReader offset was already at EOF.

• UnexpectedEofError – If EOF was encountered while reading a codepoint.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.readCodepoint(ref codepoint: int) : bool throws

Read a single Unicode codepoint from a fileReader

Arguments:

codepoint – where to store the read codepoint

Returns:

true if the codepoint was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

• UnexpectedEofError – If EOF was encountered while reading a codepoint.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileWriter.writeByte(byte: uint(8)) throws

Write a single byte to a fileWriter

Arguments:

byte – the byte to write

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified range.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileReader.readByte() : uint(8) throws

Read a single byte from a fileReader

Returns:

the byte read

Throws:

• EofError – If the fileReader offset was already at EOF.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.readByte(ref byte: uint(8)) : bool throws

Read a single byte from a fileReader

Arguments:

byte – where to store the read byte

Returns:

true if the byte was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

SystemError – If data could not be read from the fileReader due to a system error.

config param IOSkipBufferingForLargeOps = true

Warning

IOSkipBufferingForLargeOps is unstable and could change or be removed in the future

Controll whether large read/write operations can bypass the IO runtime’s buffering mechanism.

This optimization is on by default as it can improve performance for large operations where buffering doesn’t significantly reduce the number of system I/O calls and thus adds unnecessary overhead.

To disable the optimization, compile with -sIOSkipBufferingForLargeOps=false.

Note that this flag controls an implementation-specific feature and thus is not part of the Chapel language specification.

proc fileWriter.writeString(s: string, size = s.size) throws

Write size codepoints from a string to a fileWriter

Arguments:

• s – the string to write

• size – the number of codepoints to write from the string

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified range.

• SystemError – If data could not be written to the fileWriter due to a system error.

• IllegalArgumentError – If size is larger than s.size

proc fileWriter.writeBytes(b: bytes, size = b.size) throws

Write size bytes from a bytes to a fileWriter

Arguments:

• b – the bytes to write

• size – the number of bytes to write from the bytes

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified range.

• SystemError – If data could not be written to the fileWriter due to a system error.

• IllegalArgumentError – If size is larger than b.size

proc fileWriter.writeBinary(ptr: c_ptr(?t), numBytes: int) throws

Write numBytes of data from a c_ptr to a fileWriter

Note that native endianness is always used.

If numBytes is not evenly divisible by the size of t, the remaining bytes will be ignored. For example, if the c_ptr’s internal type is 4 bytes in length, and numBytes=17, only 16 bytes will be written.

Warning

This method provides no protection against attempting to access invalid memory

Arguments:

• ptr – a c_ptr to some valid memory

• numBytes – the number of bytes to write

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified region.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileWriter.writeBinary(ptr: c_ptr(void), numBytes: int) throws

Write numBytes of data from a CTypes.c_ptr(void) to a fileWriter

The data are written to the file one byte at a time.

Warning

This method provides no protection against attempting to access invalid memory

Arguments:

• ptr – a c_ptr(void) to some valid memory

• numBytes – the number of bytes to write

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified region.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileWriter.writeBinary(arg: numeric, param endian: endianness = endianness.native) throws

Write a binary number to the fileWriter

Arguments:

• arg – number to be written

• endian – endianness compile-time argument that specifies the byte order in which to write the number. Defaults to endianness.native.

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified region.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileWriter.writeBinary(arg: numeric, endian: endianness) throws

Write a binary number to the fileWriter

Arguments:

• arg – number to be written

• endian – endianness specifies the byte order in which to write the number.

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified region.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileWriter.writeBinary(s: string, size: int = s.size) throws

Write a string to a fileWriter in binary format

Arguments:

• s – the string to write

• size – the number of codepoints to write from the string

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified region.

• SystemError – If data could not be written to the fileWriter due to a system error.

• IllegalArgumentError – If size is larger than s.size.

proc fileWriter.writeBinary(b: bytes, size: int = b.size) throws

Write a bytes to a fileWriter in binary format

Arguments:

• b – the bytes to write

• size – the number of bytes to write from the bytes

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified region.

• SystemError – If data could not be written to the fileWriter due to a system error.

• IllegalArgumentError – If size is larger than b.size.

proc fileWriter.writeBinary(const ref data: [?d] ?t, param endian: endianness = endianness.native) throws  where isSuitableForBinaryReadWrite(data) && data.strides == strideKind.one && (isIntegralType(t) || isRealType(t) || isImagType(t) || isComplexType(t))

Write an array of binary numbers to a fileWriter

Note that this routine currently requires a local rectangular non-strided array.

Arguments:

• data – an array of numbers to write to the fileWriter

• endian – endianness compile-time argument that specifies the byte order in which to read the numbers. Defaults to endianness.native.

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified region.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileWriter.writeBinary(const ref data: [] ?t, endian: endianness) throws  where isSuitableForBinaryReadWrite(data) && data.strides == strideKind.one && (isIntegralType(t) || isRealType(t) || isImagType(t) || isComplexType(t))

Write an array of binary numbers to a fileWriter

Note that this routine currently requires a local rectangular non-strided array.

Arguments:

• data – an array of numbers to write to the fileWriter

• endian – endianness specifies the byte order in which to write the number.

Throws:

• EofError – If the fileWriter offset was already at EOF.

• UnexpectedEofError – If the write operation exceeds the fileWriter’s specified region.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileReader.readBinary(ref arg: numeric, param endian: endianness = endianness.native) : bool throws

Read a binary number from the fileReader

Arguments:

• arg – number to be read

• endian – endianness compile-time argument that specifies the byte order in which to read the number. Defaults to endianness.native.

Returns:

true if the number was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

• UnexpectedEofError – If EOF was encountered while reading the number.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.readBinary(ref arg: numeric, endian: endianness) : bool throws

Read a binary number from the fileReader

Arguments:

• arg – number to be read

• endian – endianness specifies the byte order in which to read the number.

Returns:

true if the number was read, and false otherwise (i.e., the fileReader was already at EOF).

Throws:

• UnexpectedEofError – If EOF was encountered while reading the number.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.readBinary(ref s: string, maxSize: int) : bool throws

Read a specified number of codepoints into a string

The resulting string s may be smaller than maxSize if EOF is reached before reading the specified number of codepoints. Additionally, if nothing is read from the fileReader, s will be set to "" (the empty string) and the method will return false.

Note

This method always uses UTF-8 encoding regardless of the fileReader’s configuration

Arguments:

• s – the string to read into — this value is overwritten

• maxSize – the number of codepoints to read from the fileReader

Returns:

true if some codepoints were read, or false on EOF

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readBinary(ref b: bytes, maxSize: int) : bool throws

Read a specified number of bytes into a bytes

The bytes b may be smaller than maxSize if EOF is reached before reading the specified number of bytes. Additionally, if nothing is read from the fileReader, b will be set to b"" (the empty bytes) and the method will return false.

Arguments:

• b – the bytes to read into — this value is overwritten

• maxSize – the number of bytes to read from the fileReader

Returns:

true if some bytes were read, or false on EOF

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readBinary(ref data: [?d] ?t, param endian = endianness.native) : int throws  where isSuitableForBinaryReadWrite(data) && data.strides == strideKind.one && (isIntegralType(t) || isRealType(t) || isImagType(t) || isComplexType(t))

Read an array of binary numbers from a fileReader

Binary values of the type data.eltType are consumed from the fileReader until data is full or EOF is reached.

Note that this routine currently requires a local rectangular non-strided array.

Arguments:

• data – an array to read into – existing values are overwritten.

• endian – endianness compile-time argument that specifies the byte order in which to read the numbers in. Defaults to endianness.native.

Returns:

the number of values that were read into the array. This can be less than data.size if EOF was reached, or an error occurred, before filling the array.

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readBinary(ref data: [] ?t, endian: endianness) : int throws  where isSuitableForBinaryReadWrite(data) && data.strides == strideKind.one && (isIntegralType(t) || isRealType(t) || isImagType(t) || isComplexType(t))

Read an array of binary numbers from a fileReader

Binary values of the type data.eltType are consumed from the fileReader until data is full or EOF is reached.

Note that this routine currently requires a local rectangular non-strided array.

Arguments:

• data – an array to read into – existing values are overwritten.

• endian – endianness specifies the byte order in which to read the number.

Returns:

the number of values that were read into the array. This can be less than data.size if EOF was reached, or an error occurred, before filling the array.

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readBinary(ptr: c_ptr(?t), maxBytes: int) : int throws

Read up to maxBytes bytes from a fileReader into a c_ptr

Note that native endianness is always used.

If maxBytes is not evenly divisible by the size of t, then the remaining bytes are ignored.

Arguments:

• ptr – a c_ptr to some memory — existing values will be overwritten

• maxBytes – the maximum number of bytes to read from the fileReader

Returns:

the number of bytes that were read. this can be less than maxBytes if EOF was reached before reading the specified number of bytes, or if maxBytes is not evenly divisible by the size of t

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readBinary(ptr: c_ptr(void), maxBytes: int) : int throws

Read up to maxBytes bytes from a fileReader into a CTypes.c_ptr(void)

Note that data are read from the file one byte at a time.

Arguments:

• ptr – a c_ptr(void) to some memory — existing values will be overwritten

• maxBytes – the maximum number of bytes to read from the fileReader

Returns:

the number of bytes that were read. this can be less than maxBytes if EOF was reached before reading the specified number of bytes

Throws:

SystemError – If data could not be read from the fileReader due to a system error. In that event, the fileReader’s offset will be left near the position where the error occurred.

proc fileReader.readln(ref args ...?k) : bool throws

Warning

‘readln’ is unstable and may be removed or modified in a future release

Read values from a fileReader and then consume any bytes until newline is reached. The input will be consumed atomically - the fileReader lock will be held while reading all of the passed values.

Arguments:

args – a list of arguments to read. This routine can be called with zero or more such arguments. Basic types are handled internally, but for other types this function will call value.deserialize() with a fileReader argument as described in The ‘serialize’ and ‘deserialize’ Methods.

Returns:

true if the read succeeded, and false upon end of file.

Throws:

• UnexpectedEofError – If EOF was encountered before data could be read.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.read(type t) throws

Read a value of passed type.

For example, the following line of code reads a value of type int from stdin and uses it to initialize a variable x:

var x = stdin.read(int);

Arguments:

t – the type to read

Returns:

the value read

Throws:

• EofError – If the fileReader is already at EOF.

• UnexpectedEofError – If EOF was encountered before data could be fully read.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.readln(type t) throws

Warning

‘readln’ is unstable and may be removed or modified in a future release

Read a value of passed type followed by a newline.

Arguments:

t – the type to read

Returns:

the value read

Throws:

• EofError – If the fileReader is at already EOF.

• UnexpectedEofError – If EOF was encountered before data could be fully read.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.readln(type t ...?numTypes) throws  where numTypes > 1

Warning

‘readln’ is unstable and may be removed or modified in a future release

Read values of passed types followed by a newline and return a tuple containing the read values.

Arguments:

t – more than one type to read

Returns:

a tuple of the read values

Throws:

• EofError – If the fileReader is already at EOF.

• UnexpectedEofError – If EOF was encountered before data could be fully read.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileReader.read(type t ...?numTypes) throws  where numTypes > 1

Read values of passed types and return a tuple containing the read values. The fileReader’s lock will be held while reading — this protects against interleaved reads.

Arguments:

t – more than one type to read

Returns:

a tuple of the read values

Throws:

• EofError – If the fileReader is already at EOF.

• UnexpectedEofError – If EOF was encountered while more data was expected.

• SystemError – If data could not be read from the fileReader due to a system error.

proc fileWriter.write(const args ...?k, sep: string = "") throws

Write values to a fileWriter. The output will be produced atomically - the fileWriter lock will be held while writing all of the passed values.

Arguments:

• args – a list of arguments to write. Basic types are handled internally, but for other types this function will call value.serialize() with the fileWriter as an argument.

• sep – a string separator that is printed in between each argument. Defaults to the empty string. Note that specifying sep is currently an unstable feature pending further design.

Throws:

• EofError – If EOF is reached before all the arguments could be written.

• UnexpectedEofError – If EOF is encountered while writing one of the arguments.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileWriter.writeln(const args ...?k, sep: string = "") throws

Write values to a fileWriter followed by a newline. The output will be produced atomically - the fileWriter lock will be held while writing all of the passed values.

Arguments:

• args – a variable number of arguments to write. This method can be called with zero or more arguments. Basic types are handled internally, but for other types this function will call value.serialize() with the fileWriter as an argument.

• sep – a string separator that is printed in between each argument. Defaults to the empty string. Note that specifying sep is currently an unstable feature pending further design.

Throws:

• EofError – If EOF is reached before all the arguments could be written.

• UnexpectedEofError – If EOF is encountered while writing one of the arguments.

• SystemError – If data could not be written to the fileWriter due to a system error.

proc fileWriter.flush() throws

Makes all writes to the fileWriter, if any, available to concurrent viewers of its associated file, such as other fileWriters/fileReader or other applications accessing this file concurrently.

Unlike file.fsync, this does not commit the written data to the file’s device.

Throws:

SystemError – If the flush fails.

proc fileReader.assertEOF(errStr: string = "- Not at EOF")

Warning

‘assertEOF’ is unstable and may be removed or modified in a future release

Assert that a fileReader has reached end-of-file and that there was no error doing the read.

proc fileReader.close() throws

Close a fileReader

Throws:

SystemError – If the fileReader is not successfully closed.

proc fileWriter.close() throws

Close a fileWriter. Implicitly performs the fileWriter.flush operation (see Synchronization of fileReader and fileWriter Data and Avoiding Data Races).

Throws:

SystemError – If the fileWriter is not successfully closed.

proc fileReader.isClosed() : bool

Return true if a fileReader is currently closed.

proc fileWriter.isClosed() : bool

Return true if a fileWriter is currently closed.

const stdin : fileReader(true)

A locking fileReader instance that reads from standard input.

const stdout : fileWriter(true)

A locking fileWriter instance that writes to standard output.

const stderr : fileWriter(true)

A locking fileWriter instance that writes to standard error.

proc read(ref args ...?n) : bool throws

Equivalent to stdin.read. See fileReader.read

proc read(type t ...?numTypes) throws

Equivalent to stdin.read. See fileReader.read for types

proc readLine(ref a: [] ?t, maxSize = a.size, stripNewline = false) : int throws  where a.rank == 1 && a.isRectangular() && a.strides == strideKind.one && (t == uint(8) || t == int(8))

Equivalent to stdin.readLine. See fileReader.readLine

proc readLine(ref s: string, maxSize = -1, stripNewline = false) : bool throws

Equivalent to stdin.readLine. See fileReader.readLine

proc readLine(ref b: bytes, maxSize = -1, stripNewline = false) : bool throws

Equivalent to stdin.readLine. See fileReader.readLine

proc readLine(type t = string, maxSize = -1, stripNewline = false) : t throws  where t == string || t == bytes

Equivalent to stdin.readLine. See fileReader.readLine

proc readln(ref args ...?n) : bool throws

Warning

‘readln’ is unstable and may be removed or modified in a future release

Equivalent to stdin.readln. See fileReader.readln

proc readln(type t ...?numTypes) throws

Warning

‘readln’ is unstable and may be removed or modified in a future release

Equivalent to stdin.readln. See fileReader.readln for types