.. default-domain:: chpl

.. module:: Zarr
   :synopsis: Support for distributed reading and writing of Zarr stores. Support is

Zarr
====
**Usage**

.. code-block:: chapel

   use Zarr;


or

.. code-block:: chapel

   import Zarr;


Support for distributed reading and writing of Zarr stores. Support is
limited to v2 Zarr arrays stored on local filesystems. NFS is not supported.
The module uses c-blosc to compress and decompress chunks. Zarr
specification: https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html

.. data:: config param zarrProfiling = false

   Turns on/off profiling of Zarr IO 

.. iterfunction:: iter zarrProfilingResults() throws

   
   Returns a map of profiling results for Zarr IO operations. The keys are
   the names of the operations and the values are the total time spent in
   each operation across all threads. Requires that zarrProfiling be set to
   true.
   

.. record:: zarrMetadataV2

   .. attribute:: var zarr_format: int

   .. attribute:: var chunks: list(int)

   .. attribute:: var dtype: string

   .. attribute:: var shape: list(int)

.. record:: zarrMetadataV3

   Unused until support is added for v3.0 stores 

   .. attribute:: var zarr_format: int

   .. attribute:: var node_type: string

   .. attribute:: var shape: list(int)

   .. attribute:: var data_type: string

   .. attribute:: var dimension_names: list(string)

.. function:: proc getLocalChunks(D: domain(?), localD: domain(?), chunkShape: ?dimCount*int): domain(dimCount)

   Returns the domain of chunks that the calling locale is responsible for 

.. function:: proc readChunk(param dimCount: int, chunkPath: string, chunkDomain: domain(dimCount), ref arraySlice: [] ?t) throws

   
   Reads a chunk from storage and fills `arraySlice` with its corresponding
   values.
   
   :arg dimCount: Dimensionality of the array being read.
   
   :arg chunkPath: Relative or absolute path to the chunk being read.
   
   :arg chunkDomain: Array subdomain the chunk contains.
   
   :arg arraySlice: Reference to the portion of the array the calling locale stores.
   
   :throws Error: If the decompression fails
   

.. function:: proc writeChunk(param dimCount, chunkPath: string, chunkDomain: domain(dimCount), ref arraySlice: [] ?t, bloscLevel: int(32) = 9) throws

   
   Updates a chunk in storage with a locale's contribution to that chunk.
   The calling function is expected to manage synchronization among locales.
   If the locale contributes the entire chunk, it will immediately compress
   and write the chunk's data. If the contribution is partial, it decompresses
   the chunk, updates the necessary values, then compresses and writes the
   chunk to storage.
   
   :arg dimCount: Dimensionality of the array being written.
   
   :arg chunkPath: Relative or absolute path to the chunk being written.
   
   :arg chunkDomain: Array subdomain that the chunk contains.
   
   :arg arraySlice: The portion of the array that the calling locale
     contributes to this chunk.
   
   :arg bloscLevel: Compression level to use. 0 indicates no compression,
     9 (default) indicates maximum compression. Values outside of this range
     will be clipped to a value between 0 and 9.
   
   :throws Error: If the compression fails
   

.. function:: proc readZarrArray(directoryPath: string, type dtype, param dimCount: int, bloscThreads: int(32) = 1) throws

   
   Reads a v2.0 zarr store from storage, returning a block distributed array.
   Each locale reads and decompresses the chunks with elements in its
   subdomain. This method assumes a shared filesystem where all nodes can
   access the store directory.
   
   :arg directoryPath: Relative or absolute path to the root of the zarr
     store. The store is expected to contain a '.zarray' metadata file
   
   :arg dtype: Chapel type of the store's data
   
   :arg dimCount: Dimensionality of the zarr array
   
   :arg bloscThreads: The number of threads to use during decompression
     (default=1)
   

.. function:: proc writeZarrArray(directoryPath: string, ref A: [?domainType] ?dtype, chunkShape: ?dimCount*int, bloscThreads: int(32) = 1, bloscLevel: int(32) = 9) throws

   
   Writes an array to storage as a v2.0 zarr store. The array metadata and
   chunks will be stored within the `directoryPath` directory, which is created
   if it does not yet exist. The chunks will have the dimensions given in the
   `chunkShape` argument. This function writes chunks in parallel, and supports
   distributed execution. It assumes a shared filesystem where all nodes can
   access the store directory.
   
   :arg directoryPath: Relative or absolute path to the root of the zarr store.
     The directory and all necessary parent directories will be created if it
     does not exist.
   
   :arg A: The array to write to storage.
   
   :arg chunkShape: The dimension extents to use when breaking A into chunks.
   
   :arg bloscThreads: The number of threads to use during compression (default=1)
   
   :arg bloscLevel: Compression level to use. 0 indicates no compression,
     9 (default) indicates maximum compression.