Protocol Buffers Support - Generated Chapel Code

This page describes exactly what Chapel code the protocol buffer compiler generates for protocol definitions using proto3 syntax. Please see Protocol Buffers Support in Chapel for information about how to use the protocol buffer support. Additionally, this document assumes familiarity with the proto3 language guide.

Note

The aim of this documentation is to provide a better understanding of the generated code. The user is not expected to modify the generated file.

Compiler Invocation

The protocol buffer compiler produces Chapel output when invoked with the --chpl_out command-line flag. The parameter to the --chpl_out option is the directory where you want the compiler to write your Chapel output. The compiler creates a single source file for each .proto file input, having a .chpl extension.

Only proto3 messages are supported by the Chapel code generator. Ensure that each .proto file begins with a declaration of:

syntax = "proto3";

Output Module Name

The name of the output file/module will be same as the package name. If the package name is not specified, the module takes the name of the proto file with all non-alphanumeric characters replaced by an underscore.

For example a file called address.proto without the package specifier will result in an output file called address.chpl with the generated code wrapped in a module of the same name. (Full implementation is not shown here.)

address.proto

syntax = "proto3";

message messageName {
...
}

address.chpl

module address {
  record messageName {
  ...
  }
}

The same proto file with a package declaration package myPackage; will result in a file called myPackage.chpl.

address.proto

syntax = "proto3";

package myPackage;

message messageName {
...
}

myPackage.chpl

module myPackage {
  record messageName {
  ...
  }
}

Messages

Given a simple message declaration:

message Foo {
  int32 num = 1;
}

The protocol buffer compiler generates a record called Foo, which has message field initializers and serialization/parsing methods for wire-type encoding. (Full implementation is not shown here.)

record Foo {

  /*
    Returns the package name of the proto file. If not declared the method returns
    an empty string.
  */
  proc packageName param { return "packageName"; }

  /*
    Returns the name of the proto message the record is derived from.
  */
  proc messageName param { return "Foo"; }

  /*
    Record fields will be generated corresponding to each proto message field.
  */
  var num: int(32);

  /*
    Used to store encoded byte stream of unknown fields encountered while parsing.

    As per proto3 documentation, unknown fields should be preserved and appended
    to the generated message byte stream.
  */
  var unknownFieldStream: bytes = "";

  /*
    User exposed method for serializing data to protobuf wire format.

    This is a wrapper method to the actual method.
  */
  proc serialize(ch) throws { ... }

  /*
    Contains the actual implementation for serializing data.

    Calls the `Append` functions of the user support library. It appends the
    `unknownFieldStream` at the end of the message.

    This should end up as a private method when supported, so a user should not
    call it directly.
  */
  proc _serialize(binCh) throws { ... }

  /*
    User exposed method for parsing data from protobuf wire format.

    This is a wrapper method to the actual method.
  */
  proc deserialize(ch) throws { ... }

  /*
    Contains the actual implementation for parsing data.

    Calls the `Consume` functions of the user support library. Appends unknown
    fields encountered to the `unknownFieldStream` variable.

    This should end up as a private method when supported, so a user should not
    call it directly.
  */
  proc _deserialize(binCh) throws { ... }

}

Any Message Type

For Any messages, you can call pack to pack a specified message into the current Any message, or unpack to unpack the current Any message to a specified message. Corresponding to an any message type field the plugin will generate a record field of Any type.

// Field "a"
var a: Any;

Fields

The protocol buffer compiler generates a Chapel record field for each field defined within a message. Methods equivalent to get and set in other languages are implicitly generated by the Chapel compiler, so do not need to be generated by the protocol buffer compiler.

Scalar Value Types

A scalar message field can have one of the following types, the table shows the type specified in the .proto file, and the corresponding generated Chapel type:

.proto Type	Chapel Type
double	real(64)
float	real(32)
int32	int(32)
int64	int(64)
uint32	uint(32)
uint64	uint(64)
sint32	int(32)
sint64	int(64)
fixed32	uint(32)
fixed64	uint(64)
sfixed32	int(32)
sfixed64	int(64)
bool	bool
string	string
bytes	bytes

Singular Fields

Every singular message field generates a record field variable of an appropriate Chapel type. Fetching a value from a field which hasn’t been explicitly set will return the default chapel value for that type. For example, a boolean field a will generate a variable of bool type, with default value of false:

// Field "a"
var a: bool;

Repeated Fields

Every repeated message field generates a list type. Fetching a value from a field which hasn’t been explicitly set will return an empty list. For example, a repeated string field a will generate a list of type string:

// Field "a"
var a: list(string);

Map Fields

Given this message definition:

message Foo {
  map<int32, bool> mapfield = 1;
}

The plugin will generate a Chapel map(int(32), bool) type field:

// Field "mapfield"
var mapfield: map(int(32), bool);

Enumerations

Given an enumeration definition like:

enum Color {
  RED = 0;
  GREEN = 5;
  BLUE = 1234;
}

The protocol buffer compiler will generate a Chapel enum type called Color with the same set of values.

The Color proto enum above would therefore become the following Chapel code:

enum Color {
  RED = 0,
  GREEN = 5,
  BLUE = 1234,
}

Oneof

Given a message with a oneof:

message Foo {
  oneof test_oneof {
     string name = 1;
     int32 serial_number = 2;
  }
}

The Chapel record corresponding to Foo will have name_ and serial_number_ fields along with explicit get/set type methods:

// Field "name"
var name_: string;
proc name {
  ...
}
proc ref name ref {
  ...
}

// Field "serial_number"
var serial_number_: int(32);
proc serial_number {
  ...
}
proc ref serial_number ref {
  ...
}

The explicit methods are declared to allow the user to set at most one of the fields in a oneof at a time. For example:

messageObj.name = "chapel";
messageObj.serial_number = 23;

Setting the value of serial_number after name will set name to its default value(”” in case of string).

Nested Types

A message can be declared inside another message. For example:

message Foo {
  message Bar {
    ...
  }
}

In this case, or if a message contains a nested enum declaration, the compiler will generate module level record/enum per nested type. This generated record or enum will have a name prefixed by the parent message name:

record Foo {
 ...
}

// Nested Types
record Foo_Bar {
  ...
}

Note

Nested records or declaration of enums in records are currently not supported in Chapel. Once we have support for these, we can declare nested types in the parent record and thus avoid the name prefix.