UtilReplicatedVar¶

Usage

use UtilReplicatedVar;

Support for user-level replicated variables.

A "replicated" variable is a variable for which there is a copy on each locale. Referencing a replicated variable (in a stylized way, see below) accesses its copy on the current locale.

Features:

The variable's copies are not synchronized automatically among the locales.

Limitations:

It is "user-level", i.e. the user is required to handle the variable in specific ways to achieve the desired result.
Tree-shape communication (like for reductions) is not provided.
Using a replicated variable of an array type is not straightforward. Workaround: declare that array itself as replicated, then access it normally, e.g.:

var replArray: [MyDomain dmapped ReplicatedDist()] real;

How to use replicated variables¶

use UtilReplicatedVar;

// declare a replicated variable of the type 'MyType'
var myRepVar: [rcDomain] MyType;

// access its copy on the current locale (read or write) (either option)
... myRepVar(1) ...  // must use 1
... rcLocal(myRepVar) ...

// "replicate": assign 'valToRep' to copies on all locales
rcReplicate(myRepVar, valToRep);

// "collect": assign from each copy of 'myRepVar' to
// corresponding element of an array 'collected'
var collected: [LocaleSpace] MyType;
rcCollect(myRepVar, collected);

// access directly a remote copy on the locale 'remoteLoc' (read or write)
... rcRemote(myRepVar, remoteLoc) ...

Replicating over a subset of locales¶

To replicate a variable over a subset of locales, say myLocales, modify the above variable declarations as follows:

var myRepVar: [rcDomainBase dmapped ReplicatedDist(myLocales,
                 "over which to replicate 'myRepVar'")] MyType;
var collected: [myLocales.domain] MyType;

myLocales must be "consistent", as defined for ReplicatedDist. That is, for each ix in myLocales.domain, myLocales[ix] must equal Locales[ix].

Tip: if the domain of the desired array of locales cannot be described as a rectangular domain (which could be strided, multi-dimensional, and/or sparse), make that array's domain associative over int.

Declarations¶

const rcDomainBase = {rcDomainIx..rcDomainIx}¶: Use this domain when replicating over a subset of locales, as shown above.

const rcDomain = rcDomainBase dmapped new dmap(rcDomainMap)¶: Use this domain to declare a user-level replicated variable, as shown above .

proc rcReplicate(replicatedVar: [?D] ?MYTYPE, valToReplicate: MYTYPE): void¶: Assign a value valToReplicate to copies of the replicated variable replicatedVar on all locales.

proc rcCollect(replicatedVar: [?D] ?MYTYPE, collected: [?CD] MYTYPE): void¶: Copy the value of the replicated variable replicatedVar on each locale into the element of the array collected that corresponds to that locale.

proc rcLocal(replicatedVar: [?D] ?MYTYPE) ref: MYTYPE¶

Access the copy of replicatedVar on the current locale.

This is equivalent to replicatedVar[1].

proc rcRemote(replicatedVar: [?D] ?MYTYPE, remoteLoc: locale) ref: MYTYPE¶: Access the copy of replicatedVar on the locale remoteLoc.