.. _readme-protoGenCodeGuide:

================================================
Protocol Buffers Support - Generated Chapel Code
================================================

This page describes exactly what Chapel code the protocol buffer compiler
generates for protocol definitions using ``proto3`` syntax. Please see
:ref:`readme-protobuf` for information about how to use the protocol
buffer support. Additionally, this document assumes familiarity with the
`proto3 language guide`_.

.. note::

  The aim of this documentation is to provide a better understanding of the
  generated code. The user is not expected to modify the generated file.

Compiler Invocation
-------------------
The protocol buffer compiler produces Chapel output when invoked with the ``--chpl_out``
command-line flag. The parameter to the ``--chpl_out`` option is the directory where
you want the compiler to write your Chapel output. The compiler creates a single
source file for each ``.proto`` file input, having a ``.chpl`` extension.

Only ``proto3`` messages are supported by the Chapel code generator. Ensure
that each ``.proto`` file begins with a declaration of:

.. code-block:: proto

  syntax = "proto3";

Output Module Name
------------------
The name of the output file/module will be same as the ``package`` name. If the
``package`` name is not specified, the module takes the name of the proto
file with all non-alphanumeric characters replaced by an ``underscore``.

For example a file called ``address.proto`` without the package specifier will
result in an output file called ``address.chpl`` with the generated code wrapped
in a module of the same name. (Full implementation is not shown here.)

``address.proto``

.. code-block:: proto

  syntax = "proto3";

  message messageName {
  ...
  }

``address.chpl``

.. code-block:: chpl

  module address {
    record messageName {
    ...
    }
  }

The same proto file with a package declaration ``package myPackage;`` will
result in a file called ``myPackage.chpl``.

``address.proto``

.. code-block:: proto

  syntax = "proto3";

  package myPackage;

  message messageName {
  ...
  }

``myPackage.chpl``

.. code-block:: chpl

  module myPackage {
    record messageName {
    ...
    }
  }

Messages
--------
Given a simple message declaration:

.. code-block:: proto
  
  message Foo {
    int32 num = 1;
  }
  
The protocol buffer compiler generates a record called ``Foo``, which has message
field initializers and serialization/parsing methods for wire-type encoding.
(Full implementation is not shown here.)

 .. code-block:: chpl
  
  record Foo {

    /*
      Returns the package name of the proto file. If not declared the method returns
      an empty string.
    */
    proc packageName param { return "packageName"; }

    /*
      Returns the name of the proto message the record is derived from.
    */
    proc messageName param { return "Foo"; }

    /*
      Record fields will be generated corresponding to each proto message field.
    */
    var num: int(32);
    
    /*
      Used to store encoded byte stream of unknown fields encountered while parsing.

      As per proto3 documentation, unknown fields should be preserved and appended
      to the generated message byte stream.
    */
    var unknownFieldStream: bytes = "";
    
    /*
      User exposed method for serializing data to protobuf wire format.

      This is a wrapper method to the actual method.
    */
    proc serialize(ch) throws { ... }
    
    /*
      Contains the actual implementation for serializing data.

      Calls the `Append` functions of the user support library. It appends the
      `unknownFieldStream` at the end of the message.

      This should end up as a private method when supported, so a user should not
      call it directly.
    */
    proc _serialize(binCh) throws { ... }
    
    /*
      User exposed method for parsing data from protobuf wire format.

      This is a wrapper method to the actual method.
    */
    proc deserialize(ch) throws { ... }
    
    /*
      Contains the actual implementation for parsing data.

      Calls the `Consume` functions of the user support library. Appends unknown
      fields encountered to the `unknownFieldStream` variable.

      This should end up as a private method when supported, so a user should not
      call it directly.
    */
    proc _deserialize(binCh) throws { ... }
  
  }

Any Message Type
^^^^^^^^^^^^^^^^
For Any messages, you can call ``pack`` to pack a specified message into the
current Any message, or ``unpack`` to unpack the current Any message to a specified
message. Corresponding to an any message type field the plugin will generate a record
field of ``Any`` type.

.. code-block:: chpl
  
  // Field "a"
  var a: Any;

Fields
------
The protocol buffer compiler generates a Chapel record field for each field defined
within a message. Methods equivalent to `get` and `set` in other languages are
implicitly generated by the Chapel compiler, so do not need to be generated by
the protocol buffer compiler.

Scalar Value Types
^^^^^^^^^^^^^^^^^^
A scalar message field can have one of the following types, the table shows the
type specified in the ``.proto`` file, and the corresponding generated Chapel type:

..
  This table is intended to match the order given by the proto3 language documentation.

.. list-table::
   :widths: 50 50
   :header-rows: 1

   * - .proto Type
     - Chapel Type
   * - double
     - real(64)
   * - float
     - real(32)
   * - int32
     - int(32)
   * - int64
     - int(64)
   * - uint32
     - uint(32)
   * - uint64
     - uint(64)
   * - sint32
     - int(32)
   * - sint64
     - int(64)
   * - fixed32
     - uint(32)
   * - fixed64
     - uint(64)
   * - sfixed32
     - int(32)
   * - sfixed64
     - int(64)
   * - bool
     - bool
   * - string
     - string
   * - bytes
     - bytes

Singular Fields
^^^^^^^^^^^^^^^
Every `singular`_ message field generates a record field variable of an appropriate Chapel type.
Fetching a value from a field which hasn't been explicitly set will return the 
default chapel value for that type. For example, a boolean field ``a`` will generate a
variable of ``bool`` type, with default value of ``false``:

.. code-block:: chpl
  
  // Field "a"
  var a: bool;
  
Repeated Fields
^^^^^^^^^^^^^^^
Every `repeated`_ message field generates a list type. Fetching a value from a field which
hasn't been explicitly set will return an empty list. For example, a repeated
string field ``a`` will generate a list of type ``string``:

.. code-block:: chpl
  
  // Field "a"
  var a: list(string);

Map Fields
^^^^^^^^^^
Given this message definition:

.. code-block:: proto

  message Foo {
    map<int32, bool> mapfield = 1;
  }

The plugin will generate a Chapel ``map(int(32), bool)`` type field:

.. code-block:: chpl

  // Field "mapfield"
  var mapfield: map(int(32), bool);

Enumerations
------------
Given an enumeration definition like:

.. code-block:: proto
  
  enum Color {
    RED = 0;
    GREEN = 5;
    BLUE = 1234;
  }
  
The protocol buffer compiler will generate a Chapel enum type called ``Color`` with the
same set of values.

The ``Color`` proto enum above would therefore become the following Chapel code:

.. code-block:: chpl
  
  enum Color {
    RED = 0,
    GREEN = 5,
    BLUE = 1234,
  }

Oneof
-----
Given a message with a oneof:

.. code-block:: proto

  message Foo {
    oneof test_oneof {
       string name = 1;
       int32 serial_number = 2;
    }
  }

The Chapel record corresponding to ``Foo`` will have ``name_`` and ``serial_number_``
fields along with explicit ``get/set`` type methods:

.. code-block:: chpl

  // Field "name"
  var name_: string;
  proc name {
    ...
  }
  proc ref name ref {
    ...
  }

  // Field "serial_number"
  var serial_number_: int(32);
  proc serial_number {
    ...
  }
  proc ref serial_number ref {
    ...
  }

The explicit methods are declared to allow the user to set at most one of the
fields in a oneof at a time. For example:

.. code-block:: chpl

  messageObj.name = "chapel";
  messageObj.serial_number = 23;

Setting the value of ``serial_number`` after ``name`` will set ``name`` to its
default value("" in case of string).

Nested Types
------------
A message can be declared inside another message. For example:

.. code-block:: proto

  message Foo {
    message Bar {
      ...
    }
  }

In this case, or if a message contains a nested enum declaration, the compiler will generate
module level record/enum per nested type. This generated record or enum will have a name prefixed
by the parent message name:

.. code-block:: chpl
  
  record Foo {
   ...
  }

  // Nested Types
  record Foo_Bar {
    ...
  }

.. note::

  Nested records or declaration of enums in records are currently not supported in
  Chapel. Once we have support for these, we can declare nested types in the parent
  record and thus avoid the name prefix.

.. _proto3 language guide: https://developers.google.com/protocol-buffers/docs/proto3
.. _singular: https://developers.google.com/protocol-buffers/docs/proto3#specifying_field_rules
.. _repeated: `singular`_