/[MITgcm]/manual/s_phys_pkgs/text/exch2.tex
ViewVC logotype

Diff of /manual/s_phys_pkgs/text/exch2.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.3 by afe, Thu Jan 29 15:39:49 2004 UTC revision 1.7 by afe, Wed Feb 11 20:48:14 2004 UTC
# Line 18  Line 18 
18    
19  The exch2 package is an extension to the original cubed sphere exchanges  The exch2 package is an extension to the original cubed sphere exchanges
20  to allow more flexible domain decomposition and parallelization.  Cube faces  to allow more flexible domain decomposition and parallelization.  Cube faces
21  (subdomain) may be divided into whatever number of tiles that divide evenly  (subdomains) may be divided into whatever number of tiles that divide evenly
22  into the grid point dimensions of the subdomain.  Furthermore, the individual  into the grid point dimensions of the subdomain.  Furthermore, the individual
23  tiles may be run on different processors in any combination, (tone this down  tiles may be run on separate processors in different combinations,
24  a bit), and whether exchanges between particular tiles occur between different  and whether exchanges between particular tiles occur between different
25  processors is decided at runtime.  processors is determined at runtime.
26    
27    The exchange parameters are declared in {\em W2\_EXCH2\_TOPOLOGY.h} and
28    assigned in {\em w2\_e2setup.F}, both in the
29    {\em pkg/exch2} directory.  The validity of the cube topology depends
30    on the {\em SIZE.h} file as detailed below.  Both files are generated by
31    Matlab scripts and
32    should not be edited.  The default files provided in the release set up
33    a cube sphere arrangement of six tiles, one per subdomain, each with 32x32 grid
34    points, running on a single processor.  
35    
36    \subsection{Key Variables}
37    
38    The descriptions of the variables are divided up into scalars,
39    one-dimensional arrays indexed to the tile number, and two and three
40    dimensional
41    arrays indexed to tile number and neighboring tile.  This division
42    actually reflects  the functionality of these variables: the scalars
43    are common to every part of the topology, the tile-indexed arrays to
44    individual tiles, and the arrays indexed to tile and neighbor to
45    relationships between tiles and their neighbors.
46    
47    \subsubsection{Scalars}
48    
49    The number of tiles in a particular topology is set with the parameter
50    {\em NTILES}, and the maximum number of neighbors of any tiles by
51    {\em MAX\_NEIGHBOURS}.  These parameters are used for defining the size of
52    the various one and two dimensional arrays that store tile parameters
53    indexed to the tile number.
54    
55    The scalar parameters {\em exch2\_domain\_nxt} and
56    {\em exch2\_domain\_nyt} express the number of tiles in the x and y global
57    indices.  For example, the default setup of six tiles has
58    {\em exch2\_domain\_nxt=6} and {\em exch2\_domain\_nyt=1}.  A topology of
59    twenty-four square (in gridpoints) tiles, four (2x2) per subdomain, will
60    have {\em exch2\_domain\_nxt=12} and {\em exch2\_domain\_nyt=2}.  Note
61    that these parameters express the tile layout to allow global data files that
62    are tile-layout-neutral and have no bearing on the internal storage of the
63    arrays.  The tiles are internally stored in a range from {\em 1,bi} (in the
64    x axis) and y-axis variable {\em bj} is generally ignored within the package.
65    
66    \subsubsection{Arrays Indexed to Tile Number}
67    
68    The following arrays are of size {\em NTILES}, are indexed to the tile number,
69    and the indices are omitted in their descriptions.
70    
71    The arrays {\em exch2\_tnx} and {\em exch2\_tny}
72    express the x and y dimensions of each tile.  At present for each tile
73    {\em exch2\_tnx = sNx}
74    and {\em exch2\_tny = sNy}, as assigned in {\em SIZE.h}.  Future releases of
75    MITgcm are to allow varying tile sizes.
76    
77    The location of the tiles' Cartesian origin within a subdomain are determined
78    by the arrays {\em exch2\_tbasex} and {\em exch2\_tbasey}.  These variables
79    are used to relate the location of the edges of the tiles to each other.  As
80    an example, in the default six-tile topology (the degenerate case)
81    each index in these arrays are
82    set to 0.  The twenty-four, 32x32 cube face case discussed above will have
83    values of 0 or 16, depending on the quadrant the tile falls within the
84    subdomain.  {\em exch2\_myFace} contains the number of the
85    cubeface/subdomain of each tile, numbered 1-6 in the case of the standard
86    cube topology.  
87    
88    The arrays {\em exch2\_txglobalo} and {\em exch2\_txglobalo} are similar to
89    {\em exch2\_tbasex} and {\em exch2\_tbasey}, but locate the tiles within
90    the global address space, similar to that used by global files.  
91    
92    The arrays {\em exch2\_isWedge}, {\em exch2\_isEedge}, {\em exch2\_isSedge},
93    and {\em exch2\_isNedge} are set to 1 if the indexed tile lies on the edge
94    of a subdomain, 0 if not.  The values are used within the topology generator
95    to determine the orientation of neighboring tiles and to indicate whether
96    a tile lies on the corner of a subdomain.  The latter case indicates
97    special exchange and numerical handling for the singularities at the eight
98    corners of the cube.  {\em exch2\_isNedge} contains a count of how many
99    neighboring tiles each tile has, and is used for setting bounds for looping
100    over neighboring tiles.  {\em exch2\_tProc} holds the process rank of each tile,
101    and is used in interprocess communication.
102    
103    \subsubsection{Arrays Indexed to Tile Number and Neighbor}
104    
105    The following arrays are all of size {\em MAX\_NEIGHBOURS}x{\em NTILES} and
106    describe the orientations between the the tiles.
107    
108    The array {\em exch2\_neighbourId(a,T)} holds the tile number $T_{n}$ for each tile
109    {\em T}'s neighbor tile {\em a}.  The neighbor tiles are indexed {\em 1,MAX\_NEIGHBOURS }
110    in the order right to left on the north then south edges, and then top to bottom on the east
111    and west edges. maybe throw in a fig here, eh?  
112    
113    {\em exch2\_opposingSend\_record(a,T)} holds
114    the index c in {\em exch2\_neighbourId(b,$T_{n}$)} that holds the tile number T.
115    In other words,
116    
117    \begin{verbatim}  
118    exch2_neighbourId( exch2_opposingSend_record(a,T),
119                               exch2_neighbourId(a,T) ) = T
120    \end{verbatim}
121    
122    % {\em exch2\_neighbourId(exch2\_opposingSend\_record(a,T),exch2\_neighbourId(a,T))=T}.
123    % alternate version
124    
125    This is to provide a backreference from the neighbor tiles.
126    
127    The arrays {\em exch2\_pi }, {\em exch2\_pj }, {\em exch2\_oi },
128    {\em exch2\_oj }, {\em exch2\_oi\_f }, and {\em exch2\_oj\_f }  specify
129    the transformations in exchanges between the neighboring tiles.  The dimensions  
130    of {\em exch2\_pi(t,N,T) } and {\em exch2\_pj(t,N,T) } are the neighbor ID
131    { \em N } and the tile number {\em T } as explained above, plus the transformation
132    vector {\em t }, of length two.  The first element of the transformation vector indicates
133    the factor by which variables representing the same vector component  of a tile
134    will be multiplied, and the second element indicates the transform to the
135    variable in the other direction.  As an example, {\em exch2\_pi(1,N,T) } holds the
136    transform of the i-component of a vector variable in tile {\em T } to the i-component of
137    tile  {\em T }'s neighbor  {\em N }, and {\em exch2\_pi(2,N,T) } hold the component
138    of neighbor  {\em N }'s j-component.
139    
140    Under the current cube topology, one of the two elements of {\em exch2\_pi } or {\em exch2\_pj }
141    for a given tile   {\em T } and  neighbor  {\em N } will be 0, reflecting the fact that
142    the vector components are orthogonal.  The other element will be 1 or -1, depending on whether
143    the components are indexed in the same or opposite directions.  For example, the transform dimension
144    of the arrays for all tile neighbors on the same subdomain will be {\em [1 , 0] }, since all tiles on
145    the same subdomain are oriented identically.  Vectors that correspond to the orthogonal dimension with the
146    same index direction will have {\em [0 , 1] }, whereas those in the opposite index direction will have
147    {\em [0 , -1] }.
148    
149    
150    
151    
152    //
153    
154    \begin{verbatim}
155    
156    
157    
158    C      exch2_pi          :: X index row of target to source permutation
159    C                        :: matrix for each neighbour entry.            
160    C      exch2_pj          :: Y index row of target to source permutation
161    C                        :: matrix for each neighbour entry.            
162    C      exch2_oi          :: X index element of target to source
163    C                        :: offset vector for cell-centered quantities  
164    C                        :: of each neighbor entry.                    
165    C      exch2_oj          :: Y index element of target to source
166    C                        :: offset vector for cell-centered quantities  
167    C                        :: of each neighbor entry.                    
168    C      exch2_oi_f        :: X index element of target to source
169    C                        :: offset vector for face quantities          
170    C                        :: of each neighbor entry.                    
171    C      exch2_oj_f        :: Y index element of target to source
172    C                        :: offset vector for face quantities          
173    C                        :: of each neighbor entry.                    
174    \end{verbatim}
175    
176    
177    
Legend:
Removed from v.1.3  
changed lines
  Added in v.1.7
  ViewVC Help
Powered by ViewVC 1.1.22