/[MITgcm]/manual/s_phys_pkgs/text/exch2.tex
ViewVC logotype

Diff of /manual/s_phys_pkgs/text/exch2.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.9 by afe, Fri Mar 12 20:58:19 2004 UTC revision 1.12 by afe, Tue Mar 16 21:52:15 2004 UTC
# Line 10  Line 10 
10  %%    o automatically inserted at \section{Reference}  %%    o automatically inserted at \section{Reference}
11    
12    
13  \section{Extended Cubed Sphere Exchange}  \section{exch2: Extended Cubed Sphere \mbox{Topology}}
14  \label{sec:exch2}  \label{sec:exch2}
15    
16    
17  \subsection{Introduction}  \subsection{Introduction}
18    
19  The \texttt{exch2} package is an extension to the original cubed  The \texttt{exch2} package extends the original cubed
20  sphere topological configuration that allows more flexible domain  sphere topology configuration to allow more flexible domain
21  decomposition and parallelization.  Cube faces (also called  decomposition and parallelization.  Cube faces (also called
22  subdomains) may be divided into any number of tiles that divide evenly  subdomains) may be divided into any number of tiles that divide evenly
23  into the grid point dimensions of the subdomain.  Furthermore, the  into the grid point dimensions of the subdomain.  Furthermore, the
24  individual tiles may be run on separate processors in different  individual tiles may be run on separate processors in different
25  combinations, and whether exchanges between particular tiles occur  combinations, and whether exchanges between particular tiles occur
26  between different processors is determined at runtime.  This  between different processors is determined at runtime.  This
27  flexibility provides for manual load balancing across a relatively  flexibility provides for manual compile-time load balancing across a
28  arbitrary number of processors.  relatively arbitrary number of processors. \\
29    
30  The exchange parameters are declared in  The exchange parameters are declared in
31  \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}  \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}
32  and assigned in  and assigned in
33  \filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}. The  \filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}. The
34  validity of the cube topology depends on the \texttt{SIZE.h} file as  validity of the cube topology depends on the \file{SIZE.h} file as
35  detailed below.  Both files are generated by Matlab scripts in ??  detailed below.  The default files provided in the release configure a
36  check these in already! and should not be edited.  The default files  cubed sphere topology of six tiles, one per subdomain, each with
37  provided in the release configure a cubed sphere arrangement of six  32$\times$32 grid points, all running on a single processor.  Both
38  tiles, one per subdomain, each with 32$\times$32 grid points, all  files are generated by Matlab scripts in
39  running on a single processor.  Pregenerated examples of these files  \file{utils/exch2/matlab-topology-generator}; see Section
40  with alternate topologies are provided in ??.  \ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2}
41    for details on creating alternate topologies.  Pregenerated examples
42    of these files with alternate topologies are provided under
43    \file{utils/exch2/code-mods} along with the appropriate \file{SIZE.h}
44    file for single-processor execution.
45    
46  \subsection{Invoking exch2}  \subsection{Invoking exch2}
47    
48  To use exch2 with the cubed sphere, the following conditions must be met:  To use exch2 with the cubed sphere, the following conditions must be
49    met: \\
50    
51  - the exch2 package is included when \texttt{genmake2} is run.  The  $\bullet$ The exch2 package is included when \file{genmake2} is run.
52    easiest way to do this is to add the line \texttt{exch2} to the    The easiest way to do this is to add the line \code{exch2} to the
53    \texttt{profile.conf} file -- see Section \ref{sect:buildingCode}    \file{profile.conf} file -- see Section
54    for general details. \\    \ref{sect:buildingCode} \sectiontitle{Building the code} for general
55      details. \\
56  - an example of \texttt{W2\_EXCH2\_TOPOLOGY.h} and  
57    \texttt{w2\_e2setup.F} must reside in a directory containing code  $\bullet$ An example of \file{W2\_EXCH2\_TOPOLOGY.h} and
58    linked when \texttt{genmake2} runs.  The safest place to put these    \file{w2\_e2setup.F} must reside in a directory containing code
59    is the directory indicated in the \texttt{-mods=DIR} command line    linked when \file{genmake2} runs.  The safest place to put these
60    modifier (typically \texttt{../code}), or the build directory.  The    is the directory indicated in the \code{-mods=DIR} command line
61    default versions of these files reside in \texttt{pkg/exch2}, but    modifier (typically \file{../code}), or the build directory.  The
62    they should be left untouched to avoid breaking configurations other    default versions of these files reside in \file{pkg/exch2} and are
63    than the one you intend to modify.\\    linked automatically if no other versions exist elsewhere in the
64      link path, but they should be left untouched to avoid breaking
65  - files containing grid parameters, named    configurations other than the one you intend to modify.\\
66    \texttt{tile}xxx\texttt{.mitgrid} where xxx is \texttt{001} through  
67    \texttt{006}, must be in the working directory when the MITgcm  $\bullet$ Files containing grid parameters, named
68    executable is run.  These files are provided in the example    \file{tile00$n$.mitgrid} where $n$=[1,6] (one per subdomain), must
69    experiments for cubed sphere configurations with 32$\times$32 cube    be in the working directory when the MITgcm executable is run.
70    sides and are non-trivial to generate -- please contact MITgcm    These files are provided in the example experiments for cubed sphere
71    support if you want to generate files for other configurations.    configurations with 32$\times$32 cube sides and are non-trivial to
72    This is lame. ?? \\    generate -- please contact MITgcm support if you want to generate
73      files for other configurations. \\
74    
75    $\bullet$ As always when compiling MITgcm, the file \file{SIZE.h} must
76      be placed where \file{genmake2} will find it.  In particular for the
77      exch2, the domain decomposition specified in \file{SIZE.h} must
78      correspond with the particular configuration's topology specified in
79      \file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}.  Domain
80      decomposition issues particular to exch2 are addressed in Section
81      \ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2}
82      and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and MPI}; a more
83      general background on the subject relevant to MITgcm is presented in
84      Section \ref{sect:specifying_a_decomposition}
85      \sectiontitle{Specifying a decomposition}.\\
86    
87  As of the time of writing the following examples use exch2 and may be  As of the time of writing the following examples use exch2 and may be
88  used for guidance:  used for guidance:
# Line 80  verification/hs94.cs-32x32x5 Line 98  verification/hs94.cs-32x32x5
98    
99    
100    
101  \subsection{Generating Topology Files}  \subsection{Generating Topology Files for exch2}
102    \label{sec:topogen}
103    
104    Alternate cubed sphere topologies may be created using the Matlab
105    scripts in \file{utils/exch2/matlab-topology-generator}. Running the
106    m-file
107    \filelink{driver.m}{utils-exch2-matlab-topology-generator_driver.m}
108    from the Matlab prompt (there are no parameters to pass) generates
109    exch2 topology files \file{W2\_EXCH2\_TOPOLOGY.h} and
110    \file{w2\_e2setup.F} in the working directory and displays a figure of
111    the topology via Matlab.  The other m-files in the directory are
112    subroutines of \file{driver.m} and should not be run ``bare'' except
113    for development purposes. \\
114    
115    The parameters that determine the dimensions and topology of the
116    generated configuration are \code{nr}, \code{nb}, \code{ng},
117    \code{tnx} and \code{tny}, and all are assigned early in the script. \\
118    
119    The first three determine the size of the subdomains and
120    hence the size of the overall domain.  Each one determines the number
121    of grid points, and therefore the resolution, along the subdomain
122    sides in a ``great circle'' around each axis of the cube.  At the time
123    of this writing MITgcm requires these three parameters to be equal,
124    but they provide for future releases  to accomodate different
125    resolutions around the axes to allow (for example) greater resolution
126    around the equator.\\
127    
128    The parameters \code{tnx} and \code{tny} determine the dimensions of
129    the tiles into which the subdomains are decomposed, and must evenly
130    divide the integer assigned to \code{nr}, \code{nb} and \code{ng}.
131    The result is a rectangular tiling of the subdomain.  Figure
132    \ref{fig:24tile} shows one possible topology for a twenty-four tile
133    cube, and figure \ref{fig:12tile} shows one for twelve tiles. \\
134    
135    \begin{figure}
136    \begin{center}
137     \resizebox{4in}{!}{
138      \includegraphics{part6/s24t_16x16.ps}
139     }
140    \end{center}
141    
142    \caption{Plot of cubed sphere topology with a 32$\times$192 domain
143    divided into six 32$\times$32 subdomains, each of which is divided into four tiles
144    (\code{tnx=16, tny=16}) for a total of twenty-four tiles.
145    } \label{fig:24tile}
146    \end{figure}
147    
148    \begin{figure}
149    \begin{center}
150     \resizebox{4in}{!}{
151      \includegraphics{part6/s12t_16x32.ps}
152     }
153    \end{center}
154    \caption{Plot of cubed sphere topology with a 32$\times$192 domain
155    divided into six 32$\times$32 subdomains of two tiles each
156     (\code{tnx=16, tny=32}).
157    } \label{fig:12tile}
158    \end{figure}
159    
160    Tiles can be selected from the topology to be omitted from being
161    allocated memory and processors.  This tuning is useful in ocean
162    modeling for omitting tiles that fall entirely on land.  The tiles
163    omitted are specified in the file
164    \filelink{blanklist.txt}{utils-exch2-matlab-topology-generator_blanklist.txt}
165    by their tile number in the topology, separated by a newline. \\
166    
167    
168    
169    
170    \subsection{exch2, SIZE.h, and multiprocessing}
171    \label{sec:exch2mpi}
172    
173    Once the topology configuration files are created, the Fortran
174    parameters in \file{SIZE.h} must be configured to match.  Section
175    \ref{sect:specifying_a_decomposition} \sectiontitle{Specifying a
176    decomposition} provides a general description of domain decomposition
177    within MITgcm and its relation to \file{SIZE.h}. The current section
178    specifies certain constraints the exch2 package imposes as well as
179    describes how to enable parallel execution with MPI. \\
180    
181    As in the general case, the parameters \varlink{sNx}{sNx} and
182    \varlink{sNy}{sNy} define the size of the individual tiles, and so
183    must be assigned the same respective values as \code{tnx} and
184    \code{tny} in \file{driver.m}.\\
185    
186    The halo width parameters \varlink{OLx}{OLx} and \varlink{OLy}{OLy}
187    have no special bearing on exch2 and may be assigned as in the general
188    case. The same holds for \varlink{Nr}{Nr}, the number of vertical
189    levels in the model.\\
190    
191    The parameters \varlink{nSx}{nSx}, \varlink{nSy}{nSy},
192    \varlink{nPx}{nPx}, and \varlink{nPy}{nPy} relate to the number of
193    tiles and how they are distributed on processors.  When using exch2,
194    the tiles are stored in single dimension, and so
195    \code{\varlink{nSy}{nSy}=1} in all cases.  Since the tiles as
196    configured by exch2 cannot be split up accross processors without
197    regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well. \\
198    
199    The number of tiles MITgcm allocates and how they are distributed
200    between processors depends on \varlink{nPx}{nPx} and
201    \varlink{nSx}{nSx}.  \varlink{nSx}{nSx} is the number of tiles per
202    processor and \varlink{nPx}{nPx} the number of processors.  The total
203    number of tiles in the topology minus those listed in
204    \file{blanklist.txt} must equal \code{nSx*nPx}. \\
205    
206    The following is an example of \file{SIZE.h} for the twelve-tile
207    configuration illustrated in figure \ref{fig:12tile} running on
208    one processor: \\
209    
210    \begin{verbatim}
211          PARAMETER (
212         &           sNx =  16,
213         &           sNy =  32,
214         &           OLx =   2,
215         &           OLy =   2,
216         &           nSx =  12,
217         &           nSy =   1,
218         &           nPx =   1,
219         &           nPy =   1,
220         &           Nx  = sNx*nSx*nPx,
221         &           Ny  = sNy*nSy*nPy,
222         &           Nr  =   5)
223    \end{verbatim}
224    
225    The following is an example for the twentyfour-tile topology in figure
226    \ref{fig:24tile} running on six processors:
227    
228    \begin{verbatim}
229          PARAMETER (
230         &           sNx =  16,
231         &           sNy =  16,
232         &           OLx =   2,
233         &           OLy =   2,
234         &           nSx =   4,
235         &           nSy =   1,
236         &           nPx =   6,
237         &           nPy =   1,
238         &           Nx  = sNx*nSx*nPx,
239         &           Ny  = sNy*nSy*nPy,
240         &           Nr  =   5)
241    \end{verbatim}
242    
243    
244    
245    
246    
247  \subsection{Key Variables}  \subsection{Key Variables}
248    
249  The descriptions of the variables are divided up into scalars,  The descriptions of the variables are divided up into scalars,
250  one-dimensional arrays indexed to the tile number, and two and three  one-dimensional arrays indexed to the tile number, and two and three
251  dimensional arrays indexed to tile number and neighboring tile.  This  dimensional arrays indexed to tile number and neighboring tile.  This
252  division actually reflects the functionality of these variables: the  division reflects the functionality of these variables: The
253  scalars are common to every part of the topology, the tile-indexed  scalars are common to every part of the topology, the tile-indexed
254  arrays to individual tiles, and the arrays indexed to tile and  arrays to individual tiles, and the arrays indexed by tile and
255  neighbor to relationships between tiles and their neighbors.  neighbor to relationships between tiles and their neighbors. \\
256    
257  \subsubsection{Scalars}  \subsubsection{Scalars}
258    
259  The number of tiles in a particular topology is set with the parameter  The number of tiles in a particular topology is set with the parameter
260  \texttt{NTILES}, and the maximum number of neighbors of any tiles by  \code{NTILES}, and the maximum number of neighbors of any tiles by
261  \texttt{MAX\_NEIGHBOURS}.  These parameters are used for defining the  \code{MAX\_NEIGHBOURS}.  These parameters are used for defining the
262  size of the various one and two dimensional arrays that store tile  size of the various one and two dimensional arrays that store tile
263  parameters indexed to the tile number.\\  parameters indexed to the tile number and are assigned in the files
264    generated by \file{driver.m}.\\
265    
266  The scalar parameters \varlink{exch2\_domain\_nxt}{exch2_domain_nxt}  The scalar parameters \varlink{exch2\_domain\_nxt}{exch2_domain_nxt}
267  and \varlink{exch2\_domain\_nyt}{exch2_domain_nyt} express the number  and \varlink{exch2\_domain\_nyt}{exch2_domain_nyt} express the number
268  of tiles in the x and y global indices.  For example, the default  of tiles in the $x$ and $y$ global indices.  For example, the default
269  setup of six tiles has \texttt{exch2\_domain\_nxt=6} and  setup of six tiles has \code{exch2\_domain\_nxt=6} and
270  \texttt{exch2\_domain\_nyt=1}.  A topology of twenty-four square (in  \code{exch2\_domain\_nyt=1}.  A topology of twenty-four square tiles,
271  gridpoints) tiles, four (2x2) per subdomain, will have  four per subdomain (as in figure \ref{fig:24tile}), will have
272  \texttt{exch2\_domain\_nxt=12} and \texttt{exch2\_domain\_nyt=2}.  \code{exch2\_domain\_nxt=12} and \code{exch2\_domain\_nyt=2}.  Note
273  Note that these parameters express the tile layout to allow global  that these parameters express the tile layout to allow global data
274  data files that are tile-layout-neutral and have no bearing on the  files that are tile-layout-neutral and have no bearing on the internal
275  internal storage of the arrays.  The tiles are internally stored in a  storage of the arrays.  The tiles are internally stored in a range
276  range from \texttt{1,bi} (in the x axis) and y-axis variable  from [1,\varlink{bi}{bi}] the $x$ axis and $y$ axis variable
277  \texttt{bj} is generally ignored within the package.  \varlink{bj}{bj} is generally ignored within the package. \\
278    
279  \subsubsection{Arrays Indexed to Tile Number}  \subsubsection{Arrays Indexed to Tile Number}
280    
281  The following arrays are of size \texttt{NTILES}, are indexed to the  The following arrays are of size \code{NTILES}, are indexed to the
282  tile number, and the indices are omitted in their descriptions.  tile number, and the indices are omitted in their descriptions. \\
283    
284  The arrays \varlink{exch2\_tnx}{exch2_tnx} and  The arrays \varlink{exch2\_tnx}{exch2_tnx} and
285  \varlink{exch2\_tny}{exch2_tny} express the x and y dimensions of each  \varlink{exch2\_tny}{exch2_tny} express the $x$ and $y$ dimensions of
286  tile.  At present for each tile \texttt{exch2\_tnx=sNx} and  each tile.  At present for each tile \texttt{exch2\_tnx=sNx} and
287  \texttt{exch2\_tny=sNy}, as assigned in \texttt{SIZE.h}.  Future  \texttt{exch2\_tny=sNy}, as assigned in \file{SIZE.h} and described in
288  releases of MITgcm are to allow varying tile sizes.  section \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and
289    multiprocessing}.  Future releases of MITgcm are to allow varying tile
290    sizes. \\
291    
292  The location of the tiles' Cartesian origin within a subdomain are  The location of the tiles' Cartesian origin within a subdomain are
293  determined by the arrays \varlink{exch2\_tbasex}{exch2_tbasex} and  determined by the arrays \varlink{exch2\_tbasex}{exch2_tbasex} and
294  \varlink{exch2\_tbasey}{exch2_tbasey}.  These variables are used to  \varlink{exch2\_tbasey}{exch2_tbasey}.  These variables are used to
295  relate the location of the edges of the tiles to each other.  As an  relate the location of the edges of different tiles to each other.  As
296  example, in the default six-tile topology (the degenerate case) each  an example, in the default six-tile topology ??  each index in these
297  index in these arrays are set to 0.  The twenty-four, 32x32 cube face  arrays are set to \code{0}.  The twentyfour-tile case discussed above
298  case discussed above will have values of 0 or 16, depending on the  will have values of \code{0} or \code{16}, depending on the quadrant
299  quadrant the tile falls within the subdomain.  The array  the tile falls within the subdomain.  The array
300  \varlink{exch2\_myFace}{exch2_myFace} contains the number of the  \varlink{exch2\_myFace}{exch2_myFace} contains the number of the
301  cubeface/subdomain of each tile, numbered 1-6 in the case of the  subdomain of each tile, numbered \code{(1:6)} in the case of the
302  standard cube topology.  standard cube topology and indicated by \textbf{\textsf{f}}$n$ in
303    figures \ref{fig:12tile}) and \ref{fig:24tile}). \\
304    
305  The arrays \varlink{exch2\_txglobalo}{exch2_txglobalo} and  The elements of the arrays \varlink{exch2\_txglobalo}{exch2_txglobalo}
306  \varlink{exch2\_txglobalo}{exch2_txglobalo} are similar to  and \varlink{exch2\_txglobalo}{exch2_txglobalo} are similar to
307  \varlink{exch2\_tbasex}{exch2_tbasex} and  \varlink{exch2\_tbasex}{exch2_tbasex} and
308  \varlink{exch2\_tbasey}{exch2_tbasey}, but locate the tiles within the  \varlink{exch2\_tbasey}{exch2_tbasey}, but locate the tiles within the
309  global address space, similar to that used by global files.  global address space, similar to that used by global files. \\
310    
311  The arrays \varlink{exch2\_isWedge}{exch2_isWedge},  The arrays \varlink{exch2\_isWedge}{exch2_isWedge},
312  \varlink{exch2\_isEedge}{exch2_isEedge},  \varlink{exch2\_isEedge}{exch2_isEedge},
313  \varlink{exch2\_isSedge}{exch2_isSedge}, and  \varlink{exch2\_isSedge}{exch2_isSedge}, and
314  \varlink{exch2\_isNedge}{exch2_isNedge} are set to 1 if the indexed  \varlink{exch2\_isNedge}{exch2_isNedge} are set to \code{1} if the
315  tile lies on the edge of a subdomain, 0 if not.  The values are used  indexed tile lies on the edge of a subdomain, \code{0} if not.  The
316  within the topology generator to determine the orientation of  values are used within the topology generator to determine the
317  neighboring tiles and to indicate whether a tile lies on the corner of  orientation of neighboring tiles, and to indicate whether a tile lies
318  a subdomain.  The latter case indicates special exchange and numerical  on the corner of a subdomain.  The latter case requires special
319  handling for the singularities at the eight corners of the cube.  exchange and numerical handling for the singularities at the eight
320  \varlink{exch2\_nNeighbours}{exch2_nNeighbours} contains a count of  corners of the cube.  \varlink{exch2\_nNeighbours}{exch2_nNeighbours}
321  how many neighboring tiles each tile has, and is used for setting  contains a count of how many neighboring tiles each tile has, and is
322  bounds for looping over neighboring tiles.  used for setting bounds for looping over neighboring tiles.
323  \varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each  \varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each
324  tile, and is used in interprocess communication.  tile, and is used in interprocess communication.  \\
325    
326  \subsubsection{Arrays Indexed to Tile Number and Neighbor}  \subsubsection{Arrays Indexed to Tile Number and Neighbor}
327    
328  The following arrays are all of size \texttt{MAX\_NEIGHBOURS} $\times$  The following arrays are all of size
329  \texttt{NTILES} and describe the orientations between the the tiles.  \code{MAX\_NEIGHBOURS}$\times$\code{NTILES} and describe the
330    orientations between the the tiles. \\
331  The array \texttt{exch2\_neighbourId(a,T)} holds the tile number for  
332  each of the $n$ neighboring tiles.  The neighbor tiles are indexed  The array \code{exch2\_neighbourId(a,T)} holds the tile number
333  \texttt{(1,MAX\_NEIGHBOURS} in the order right to left on the north  \code{Tn} for each of the tile number \code{T}'s neighboring tiles
334  then south edges, and then top to bottom on the east and west edges.  \code{a}.  The neighbor tiles are indexed \code{(1:MAX\_NEIGHBOURS)}
335  Maybe throw in a fig here, eh?  in the order right to left on the north then south edges, and then top
336    to bottom on the east and west edges.  Maybe throw in a fig here, eh?
337  The \texttt{exch2\_opposingSend\_record(a,T)} array holds the index c  \\
338  in \texttt{exch2\_neighbourId(b,$T_{n}$)} that holds the tile number T.  
339  In other words,  The \code{exch2\_opposingSend\_record(a,T)} array holds the index
340    \code{b} in \texttt{exch2\_neighbourId(b,Tn)} that holds the tile
341    number \code{T}.  In other words,
342  \begin{verbatim}  \begin{verbatim}
343     exch2_neighbourId( exch2_opposingSend_record(a,T),     exch2_neighbourId( exch2_opposingSend_record(a,T),
344                        exch2_neighbourId(a,T) ) = T                        exch2_neighbourId(a,T) ) = T
345  \end{verbatim}  \end{verbatim}
346  and this provides a back-reference from the neighbor tiles.  This provides a back-reference from the neighbor tiles. \\
347    
348  The arrays \varlink{exch2\_pi}{exch2_pi},  The arrays \varlink{exch2\_pi}{exch2_pi},
349  \varlink{exch2\_pj}{exch2_pj}, \varlink{exch2\_oi}{exch2_oi},  \varlink{exch2\_pj}{exch2_pj}, \varlink{exch2\_oi}{exch2_oi},
350  \varlink{exch2\_oj}{exch2_oj}, \varlink{exch2\_oi\_f}{exch2_oi_f}, and  \varlink{exch2\_oj}{exch2_oj}, \varlink{exch2\_oi\_f}{exch2_oi_f}, and
351  \varlink{exch2\_oj\_f}{exch2_oj_f} specify the transformations in  \varlink{exch2\_oj\_f}{exch2_oj_f} specify the transformations in
352  exchanges between the neighboring tiles.  The dimensions of  exchanges between the neighboring tiles.  The dimensions of
353  \texttt{exch2\_pi(t,N,T)} and \texttt{exch2\_pj(t,N,T)} are the  \code{exch2\_pi(t,N,T)} and \code{exch2\_pj(t,N,T)} are the neighbor
354  neighbor ID \textit{N} and the tile number \textit{T} as explained  ID \code{N} and the tile number \code{T} as explained above, plus a
355  above, plus the transformation vector {\em t }, of length two.  The  vector of length 2 containing transformation factors \code{t}.  The
356  first element of the transformation vector indicates the factor by  first element of the transformation vector indicates the factor
357  which variables representing the same vector component of a tile will  \code{t} by which variables representing the same vector component of
358  be multiplied, and the second element indicates the transform to the  a tile \code{T} will be multiplied in exchanges with neighbor
359    \code{N}, and the second element indicates the transform to the
360  variable in the other direction.  As an example,  variable in the other direction.  As an example,
361  \texttt{exch2\_pi(1,N,T)} holds the transform of the i-component of a  \code{exch2\_pi(1,N,T)} holds the transform of the $i$ component of a
362  vector variable in tile \texttt{T} to the i-component of tile  vector variable in tile \code{T} to the $i$ component of tile
363  \texttt{T}'s neighbor \texttt{N}, and \texttt{exch2\_pi(2,N,T)} hold  \code{T}'s neighbor \code{N}, and \code{exch2\_pi(2,N,T)} hold the
364  the component of neighbor \texttt{N}'s j-component.  component of neighbor \code{N}'s $j$ component. \\
365    
366  Under the current cube topology, one of the two elements of  Under the current cube topology, one of the two elements of
367  \texttt{exch2\_pi} or \texttt{exch2\_pj} for a given tile \texttt{T}  \code{exch2\_pi} or \code{exch2\_pj} for a given tile \code{T} and
368  and neighbor \texttt{N} will be 0, reflecting the fact that the vector  neighbor \code{N} will be \code{0}, reflecting the fact that the two
369  components are orthogonal.  The other element will be 1 or -1,  vector components are orthogonal.  The other element will be 1 or -1,
370  depending on whether the components are indexed in the same or  depending on whether the components are indexed in the same or
371  opposite directions.  For example, the transform dimension of the  opposite directions.  For example, the transform vector of the arrays
372  arrays for all tile neighbors on the same subdomain will be [1,0],  for all tile neighbors on the same subdomain will be \code{(1,0)},
373  since all tiles on the same subdomain are oriented identically.  since all tiles on the same subdomain are oriented identically.  A
374  Vectors that correspond to the orthogonal dimension with the same  vector direction that corresponds to the orthogonal dimension with the
375  index direction will have [0,1], whereas those in the opposite index  same index direction in a particular tile-neighbor orientation will
376  direction will have [0,-1].  have \code{(0,1)}, whereas those in the opposite index direction will
377    have \code{(0,-1)}.  This needs some diagrams.
378    
379    
380  {\footnotesize  {\footnotesize
Legend:
Removed from v.1.9  
changed lines
  Added in v.1.12
  ViewVC Help
Powered by ViewVC 1.1.22