/[MITgcm]/manual/s_phys_pkgs/text/exch2.tex
ViewVC logotype

Diff of /manual/s_phys_pkgs/text/exch2.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.2 by afe, Wed Jan 28 18:08:22 2004 UTC revision 1.8 by edhill, Tue Feb 17 21:58:56 2004 UTC
# Line 10  Line 10 
10  %%    o automatically inserted at \section{Reference}  %%    o automatically inserted at \section{Reference}
11    
12    
13  \section{exch2: Extended Cubed Sphere Exchange}  \section{Extended Cubed Sphere Exchange}
14    \label{sec:exch2}
 \subsection{Introduction}  
   
 The exch2 package is an extension to the original cubed sphere exchanges  
 to allow more flexible domain decomposition and parallelization.  Cube faces  
 (subdomain) may be divided into whatever number of tiles that divide evenly  
 into the grid point dimensions of the subdomain.  Furthermore, the individual  
 tiles may be run on different processors in any combination, (tone this down  
 a bit), and whether exchanges between particular tiles occur between different  
 processors is decided at runtime.  
   
15    
16    
17    \subsection{Introduction}
18    
19    The \texttt{exch2} package is an extension to the original cubed
20    sphere exchanges to allow more flexible domain decomposition and
21    parallelization.  Cube faces (subdomains) may be divided into whatever
22    number of tiles that divide evenly into the grid point dimensions of
23    the subdomain.  Furthermore, the individual tiles may be run on
24    separate processors in different combinations, and whether exchanges
25    between particular tiles occur between different processors is
26    determined at runtime.
27    
28    The exchange parameters are declared in
29    \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}
30    and assigned in
31    \filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}, both in
32    the \texttt{pkg/exch2} directory.  The validity of the cube topology
33    depends on the \texttt{SIZE.h} file as detailed below.  Both files are
34    generated by Matlab scripts and should not be edited.  The default
35    files provided in the release set up a cube sphere arrangement of six
36    tiles, one per subdomain, each with 32x32 grid points, running on a
37    single processor.
38    
39    \subsection{Key Variables}
40    
41    The descriptions of the variables are divided up into scalars,
42    one-dimensional arrays indexed to the tile number, and two and three
43    dimensional arrays indexed to tile number and neighboring tile.  This
44    division actually reflects the functionality of these variables: the
45    scalars are common to every part of the topology, the tile-indexed
46    arrays to individual tiles, and the arrays indexed to tile and
47    neighbor to relationships between tiles and their neighbors.
48    
49    \subsubsection{Scalars}
50    
51    The number of tiles in a particular topology is set with the parameter
52    \texttt{NTILES}, and the maximum number of neighbors of any tiles by
53    \texttt{MAX\_NEIGHBOURS}.  These parameters are used for defining the
54    size of the various one and two dimensional arrays that store tile
55    parameters indexed to the tile number.
56    
57    The scalar parameters \varlink{exch2\_domain\_nxt}{exch2_domain_nxt}
58    and \varlink{exch2\_domain\_nyt}{exch2_domain_nyt} express the number
59    of tiles in the x and y global indices.  For example, the default
60    setup of six tiles has \texttt{exch2\_domain\_nxt=6} and
61    \texttt{exch2\_domain\_nyt=1}.  A topology of twenty-four square (in
62    gridpoints) tiles, four (2x2) per subdomain, will have
63    \texttt{exch2\_domain\_nxt=12} and \texttt{exch2\_domain\_nyt=2}.
64    Note that these parameters express the tile layout to allow global
65    data files that are tile-layout-neutral and have no bearing on the
66    internal storage of the arrays.  The tiles are internally stored in a
67    range from \texttt{1,bi} (in the x axis) and y-axis variable
68    \texttt{bj} is generally ignored within the package.
69    
70    \subsubsection{Arrays Indexed to Tile Number}
71    
72    The following arrays are of size \texttt{NTILES}, are indexed to the
73    tile number, and the indices are omitted in their descriptions.
74    
75    The arrays \varlink{exch2\_tnx}{exch2_tnx} and
76    \varlink{exch2\_tny}{exch2_tny} express the x and y dimensions of each
77    tile.  At present for each tile \texttt{exch2\_tnx=sNx} and
78    \texttt{exch2\_tny=sNy}, as assigned in \texttt{SIZE.h}.  Future
79    releases of MITgcm are to allow varying tile sizes.
80    
81    The location of the tiles' Cartesian origin within a subdomain are
82    determined by the arrays \varlink{exch2\_tbasex}{exch2_tbasex} and
83    \varlink{exch2\_tbasey}{exch2_tbasey}.  These variables are used to
84    relate the location of the edges of the tiles to each other.  As an
85    example, in the default six-tile topology (the degenerate case) each
86    index in these arrays are set to 0.  The twenty-four, 32x32 cube face
87    case discussed above will have values of 0 or 16, depending on the
88    quadrant the tile falls within the subdomain.  The array
89    \varlink{exch2\_myFace}{exch2_myFace} contains the number of the
90    cubeface/subdomain of each tile, numbered 1-6 in the case of the
91    standard cube topology.
92    
93    The arrays \varlink{exch2\_txglobalo}{exch2_txglobalo} and
94    \varlink{exch2\_txglobalo}{exch2_txglobalo} are similar to
95    \varlink{exch2\_tbasex}{exch2_tbasex} and
96    \varlink{exch2\_tbasey}{exch2_tbasey}, but locate the tiles within the
97    global address space, similar to that used by global files.
98    
99    The arrays \varlink{exch2\_isWedge}{exch2_isWedge},
100    \varlink{exch2\_isEedge}{exch2_isEedge},
101    \varlink{exch2\_isSedge}{exch2_isSedge}, and
102    \varlink{exch2\_isNedge}{exch2_isNedge} are set to 1 if the indexed
103    tile lies on the edge of a subdomain, 0 if not.  The values are used
104    within the topology generator to determine the orientation of
105    neighboring tiles and to indicate whether a tile lies on the corner of
106    a subdomain.  The latter case indicates special exchange and numerical
107    handling for the singularities at the eight corners of the cube.
108    \varlink{exch2\_nNeighbours}{exch2_nNeighbours} contains a count of
109    how many neighboring tiles each tile has, and is used for setting
110    bounds for looping over neighboring tiles.
111    \varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each
112    tile, and is used in interprocess communication.
113    
114    \subsubsection{Arrays Indexed to Tile Number and Neighbor}
115    
116    The following arrays are all of size \texttt{MAX\_NEIGHBOURS} $\times$
117    \texttt{NTILES} and describe the orientations between the the tiles.
118    
119    The array \texttt{exch2\_neighbourId(a,T)} holds the tile number for
120    each of the $n$ neighboring tiles.  The neighbor tiles are indexed
121    \texttt{(1,MAX\_NEIGHBOURS} in the order right to left on the north
122    then south edges, and then top to bottom on the east and west edges.
123    Maybe throw in a fig here, eh?
124    
125    The \texttt{exch2\_opposingSend\_record(a,T)} array holds the index c
126    in \texttt{exch2\_neighbourId(b,$T_{n}$)} that holds the tile number T.
127    In other words,
128    \begin{verbatim}
129       exch2_neighbourId( exch2_opposingSend_record(a,T),
130                          exch2_neighbourId(a,T) ) = T
131    \end{verbatim}
132    and this provides a back-reference from the neighbor tiles.
133    
134    The arrays \varlink{exch2\_pi}{exch2_pi},
135    \varlink{exch2\_pj}{exch2_pj}, \varlink{exch2\_oi}{exch2_oi},
136    \varlink{exch2\_oj}{exch2_oj}, \varlink{exch2\_oi\_f}{exch2_oi_f}, and
137    \varlink{exch2\_oj\_f}{exch2_oj_f} specify the transformations in
138    exchanges between the neighboring tiles.  The dimensions of
139    \texttt{exch2\_pi(t,N,T)} and \texttt{exch2\_pj(t,N,T)} are the
140    neighbor ID \textit{N} and the tile number \textit{T} as explained
141    above, plus the transformation vector {\em t }, of length two.  The
142    first element of the transformation vector indicates the factor by
143    which variables representing the same vector component of a tile will
144    be multiplied, and the second element indicates the transform to the
145    variable in the other direction.  As an example,
146    \texttt{exch2\_pi(1,N,T)} holds the transform of the i-component of a
147    vector variable in tile \texttt{T} to the i-component of tile
148    \texttt{T}'s neighbor \texttt{N}, and \texttt{exch2\_pi(2,N,T)} hold
149    the component of neighbor \texttt{N}'s j-component.
150    
151    Under the current cube topology, one of the two elements of
152    \texttt{exch2\_pi} or \texttt{exch2\_pj} for a given tile \texttt{T}
153    and neighbor \texttt{N} will be 0, reflecting the fact that the vector
154    components are orthogonal.  The other element will be 1 or -1,
155    depending on whether the components are indexed in the same or
156    opposite directions.  For example, the transform dimension of the
157    arrays for all tile neighbors on the same subdomain will be [1,0],
158    since all tiles on the same subdomain are oriented identically.
159    Vectors that correspond to the orthogonal dimension with the same
160    index direction will have [0,1], whereas those in the opposite index
161    direction will have [0,-1].
162    
163    
164    {\footnotesize
165    \begin{verbatim}
166    C      exch2_pi          :: X index row of target to source permutation
167    C                        :: matrix for each neighbour entry.            
168    C      exch2_pj          :: Y index row of target to source permutation
169    C                        :: matrix for each neighbour entry.            
170    C      exch2_oi          :: X index element of target to source
171    C                        :: offset vector for cell-centered quantities  
172    C                        :: of each neighbor entry.                    
173    C      exch2_oj          :: Y index element of target to source
174    C                        :: offset vector for cell-centered quantities  
175    C                        :: of each neighbor entry.                    
176    C      exch2_oi_f        :: X index element of target to source
177    C                        :: offset vector for face quantities          
178    C                        :: of each neighbor entry.                    
179    C      exch2_oj_f        :: Y index element of target to source
180    C                        :: offset vector for face quantities          
181    C                        :: of each neighbor entry.                    
182    \end{verbatim}
183    }
184    
185    
186    
Legend:
Removed from v.1.2  
changed lines
  Added in v.1.8
  ViewVC Help
Powered by ViewVC 1.1.22