--- manual/s_phys_pkgs/text/exch2.tex 2004/01/29 15:39:49 1.3 +++ manual/s_phys_pkgs/text/exch2.tex 2004/01/29 17:55:35 1.4 @@ -1,4 +1,4 @@ -% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.3 2004/01/29 15:39:49 afe Exp $ +% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.4 2004/01/29 17:55:35 afe Exp $ % $Name: $ %% * Introduction @@ -18,14 +18,107 @@ The exch2 package is an extension to the original cubed sphere exchanges to allow more flexible domain decomposition and parallelization. Cube faces -(subdomain) may be divided into whatever number of tiles that divide evenly +(subdomains) may be divided into whatever number of tiles that divide evenly into the grid point dimensions of the subdomain. Furthermore, the individual -tiles may be run on different processors in any combination, (tone this down -a bit), and whether exchanges between particular tiles occur between different -processors is decided at runtime. - - - +tiles may be run on separate processors in different combinations, +and whether exchanges between particular tiles occur between different +processors is determined at runtime. + +The exchange parameters are declared in {\em W2\_EXCH2\_TOPOLOGY.h} and +assigned in {\em w2\_e2setup.F}, both in the +{\em pkg/exch2} directory. The validity of the cube topology depends +on the {\em SIZE.h} file as detailed below. Both files are generated by +Matlab scripts and +should not be edited. The default files provided in the release set up +a cube sphere arrangement of six tiles, one per subdomain, each with 32x32 grid +points, running on a single processor. + +\subsection{Key Variables} + +The descriptions of the variables are divided up into scalars, +one-dimensional arrays indexed to the tile number, and two-dimensional +arrays indexed to tile number and neighboring tile. This division +actually reflects the functionality of these variables, not just the +whim of some FORTRAN enthusiast. + +\subsubsection{Scalars} + +The number of tiles in a particular topology is set with the parameter +{\em NTILES}, and the maximum number of neighbors of any tiles by +{\em MAX\_NEIGHBOURS}. These parameters are used for defining the size of +the various one and two dimensional arrays that store tile parameters +indexed to the tile number. + +The scalar parameters {\em exch2\_domain\_nxt} and +{\em exch2\_domain\_nyt} express the number of tiles in the x and y global +indices. For example, the default setup of six tiles has +{\em exch2\_domain\_nxt=6} and {\em exch2\_domain\_nyt=1}. A topology of +twenty-four square (in gridpoints) tiles, four (2x2) per subdomain, will +have {\em exch2\_domain\_nxt=12} and {\em exch2\_domain\_nyt=2}. Note +that these parameters express the tile layout to allow global data files that +are tile-layout-neutral and have no bearing on the internal storage of the +arrays. The tiles are internally stored in a range from {\em 1,bi} (in the +x axis) and y-axis variable {\em bj} is generally ignored within the package. + +\subsubsection{One-Dimensional Arrays} + +The following arrays are indexed to the tile number, and the indices are +omitted in their descriptions. + +The arrays {\em exch2\_tnx} and {\em exch2\_tny} +express the x and y dimensions of each tile. At present for each tile +{\em exch2\_tnx = sNx} +and {\em exch2\_tny = sNy}, as assigned in {\em SIZE.h}. Future releases of +MITgcm are to allow varying tile sizes. + +The location of the tiles' Cartesian origin within a subdomain are determined +by the arrays {\em exch2\_tbasex} and {\em exch2\_tbasey}. These + +\subsubsection{Two-Dimensional Arrays} + + +// + +\begin{verbatim} +C NTILES :: Number of tiles in this topology +C MAX_NEIGHBOURS :: Maximum number of neighbours any tile has. +C exch2_domain_nxt :: Total domain length in tiles. +C exch2_domain_nyt :: Maximum domain height in tiles. +C exch2_tnx :: Size in X for each tile. +C exch2_tny :: Size in Y for each tile. +C exch2_tbasex :: Tile offset in X within its sub-domain (cube face) +C exch2_tbasey :: Tile offset in Y within its sub-domain (cube face) +C exch2_tglobalxlo :: Tile base X index within global index space. +C exch2_tglobalylo :: Tile base Y index within global index space. +C exch2_isWedge :: 0 if West not at domain edge, 1 if it is. +C exch2_isNedge :: 0 if North not at domain edge, 1 if it is. +C exch2_isEedge :: 0 if East not at domain edge, 1 if it is. +C exch2_isSedge :: 0 if South not at domain edge, 1 if it is. +C exch2_myFace :: Cube face number used for I/O. +C exch2_nNeighbours :: Tile neighbour entries count. +C exch2_tProc :: Rank of process owning tile +C :: (filled at run time). +C exch2_neighbourId :: Tile number for each neighbour entry. +C exch2_opposingSend_record :: Record for entry in target tile send +C :: list that has this tile and face +C :: as its target. +C exch2_pi :: X index row of target to source permutation +C :: matrix for each neighbour entry. +C exch2_pj :: Y index row of target to source permutation +C :: matrix for each neighbour entry. +C exch2_oi :: X index element of target to source +C :: offset vector for cell-centered quantities +C :: of each neighbor entry. +C exch2_oj :: Y index element of target to source +C :: offset vector for cell-centered quantities +C :: of each neighbor entry. +C exch2_oi_f :: X index element of target to source +C :: offset vector for face quantities +C :: of each neighbor entry. +C exch2_oj_f :: Y index element of target to source +C :: offset vector for face quantities +C :: of each neighbor entry. +\end{verbatim}