--- manual/s_phys_pkgs/text/exch2.tex	2004/01/28 18:08:22	1.2
+++ manual/s_phys_pkgs/text/exch2.tex	2004/01/29 21:03:53	1.5
@@ -1,4 +1,4 @@
-% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.2 2004/01/28 18:08:22 afe Exp $
+% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.5 2004/01/29 21:03:53 afe Exp $
 % $Name:  $
 
 %%  * Introduction
@@ -11,19 +11,136 @@
 
 
 \section{exch2: Extended Cubed Sphere Exchange}
+\label{sec:exch2}
+
 
 \subsection{Introduction}
 
 The exch2 package is an extension to the original cubed sphere exchanges
 to allow more flexible domain decomposition and parallelization.  Cube faces
-(subdomain) may be divided into whatever number of tiles that divide evenly
+(subdomains) may be divided into whatever number of tiles that divide evenly
 into the grid point dimensions of the subdomain.  Furthermore, the individual
-tiles may be run on different processors in any combination, (tone this down 
-a bit), and whether exchanges between particular tiles occur between different
-processors is decided at runtime.
-
-
-
+tiles may be run on separate processors in different combinations,
+and whether exchanges between particular tiles occur between different
+processors is determined at runtime.
+
+The exchange parameters are declared in {\em W2\_EXCH2\_TOPOLOGY.h} and 
+assigned in {\em w2\_e2setup.F}, both in the 
+{\em pkg/exch2} directory.  The validity of the cube topology depends
+on the {\em SIZE.h} file as detailed below.  Both files are generated by 
+Matlab scripts and
+should not be edited.  The default files provided in the release set up
+a cube sphere arrangement of six tiles, one per subdomain, each with 32x32 grid
+points, running on a single processor.  
+
+\subsection{Key Variables}
+
+The descriptions of the variables are divided up into scalars,
+one-dimensional arrays indexed to the tile number, and two-dimensional
+arrays indexed to tile number and neighboring tile.  This division
+actually reflects  the functionality of these variables, not just the
+whim of some FORTRAN enthusiast.
+
+\subsubsection{Scalars}
+
+The number of tiles in a particular topology is set with the parameter
+{\em NTILES}, and the maximum number of neighbors of any tiles by 
+{\em MAX\_NEIGHBOURS}.  These parameters are used for defining the size of
+the various one and two dimensional arrays that store tile parameters
+indexed to the tile number.
+
+The scalar parameters {\em exch2\_domain\_nxt} and 
+{\em exch2\_domain\_nyt} express the number of tiles in the x and y global
+indices.  For example, the default setup of six tiles has 
+{\em exch2\_domain\_nxt=6} and {\em exch2\_domain\_nyt=1}.  A topology of
+twenty-four square (in gridpoints) tiles, four (2x2) per subdomain, will
+have {\em exch2\_domain\_nxt=12} and {\em exch2\_domain\_nyt=2}.  Note 
+that these parameters express the tile layout to allow global data files that
+are tile-layout-neutral and have no bearing on the internal storage of the
+arrays.  The tiles are internally stored in a range from {\em 1,bi} (in the
+x axis) and y-axis variable {\em bj} is generally ignored within the package.
+
+\subsubsection{One-Dimensional Arrays}
+
+The following arrays are of size {\em NTILES}, are indexed to the tile number, 
+and the indices are omitted in their descriptions.
+
+The arrays {\em exch2\_tnx} and {\em exch2\_tny} 
+express the x and y dimensions of each tile.  At present for each tile
+{\em exch2\_tnx = sNx} 
+and {\em exch2\_tny = sNy}, as assigned in {\em SIZE.h}.  Future releases of 
+MITgcm are to allow varying tile sizes.
+
+The location of the tiles' Cartesian origin within a subdomain are determined 
+by the arrays {\em exch2\_tbasex} and {\em exch2\_tbasey}.  These variables
+are used to relate the location of the edges of the tiles to each other.  As 
+an example, in the default six-tile topology (the degenerate case) 
+each index in these arrays are 
+set to 0.  The twenty-four, 32x32 cube face case discussed above will have
+values of 0 or 16, depending on the quadrant the tile falls within the 
+subdomain.  {\em exch2\_myFace} contains the number of the 
+cubeface/subdomain of each tile, numbered 1-6 in the case of the standard
+cube topology.  
+
+The arrays {\em exch2\_txglobalo} and {\em exch2\_txglobalo} are similar to
+{\em exch2\_tbasex} and {\em exch2\_tbasey}, but locate the tiles within
+the global address space, similar to that used by global files.  
+
+The arrays {\em exch2\_isWedge}, {\em exch2\_isEedge}, {\em exch2\_isSedge}, 
+and {\em exch2\_isNedge} are set to 1 if the indexed tile lies on the edge
+of a subdomain, 0 if not.  The values are used within the topology generator
+to determine the orientation of neighboring tiles and to indicate whether 
+a tile lies on the corner of a subdomain.  The latter case indicates 
+special exchange and numerical handling for the singularities at the eight 
+corners of the cube.  {\em exch2\_isNedge} contains a count of how many
+neighboring tiles each tile has, and is used for setting bounds for looping
+over neighboring tiles.  {\em exch2\_tProc} holds the process rank of each tile,
+and is used in interprocess communication.
+
+\subsubsection{Two-Dimensional Arrays}
+
+The following arrays are all of size {\em MAX\_NEIGHBOURS}x{\em NTILES} and
+describe the orientations between the the tiles.
+
+The array {\em exch2\_neighbourId(a,T)} holds the tile number $T_{n}$ for each tile 
+{\em T}'s neighbor tile {\em a}, and {\em exch2\_opposingSend\_record(a,T)} holds 
+the index c in {\em exch2\_neighbourId(b,$T_{n}$)} that holds the tile number T.
+In other words,
+
+\begin{verbatim}   
+exch2_neighbourId( exch2_opposingSend_record(a,T), exch2_neighbourId(a,T) ) = T
+\end{verbatim}
+
+{\em exch2\_neighbourId(exch2\_opposingSend\_record(a,T),exch2\_neighbourId(a,T))=T}.
+This is to provide a backreference from the neighbor tiles.
+
+
+//
+
+\begin{verbatim}
+
+
+C      exch2_neighbourId :: Tile number for each neighbour entry.        
+C      exch2_opposingSend_record :: Record for entry in target tile send 
+C                                :: list that has this tile and face     
+C                                :: as its target.                       
+C      exch2_pi          :: X index row of target to source permutation 
+C                        :: matrix for each neighbour entry.            
+C      exch2_pj          :: Y index row of target to source permutation 
+C                        :: matrix for each neighbour entry.            
+C      exch2_oi          :: X index element of target to source 
+C                        :: offset vector for cell-centered quantities  
+C                        :: of each neighbor entry.                     
+C      exch2_oj          :: Y index element of target to source 
+C                        :: offset vector for cell-centered quantities  
+C                        :: of each neighbor entry.                     
+C      exch2_oi_f        :: X index element of target to source 
+C                        :: offset vector for face quantities           
+C                        :: of each neighbor entry.                     
+C      exch2_oj_f        :: Y index element of target to source 
+C                        :: offset vector for face quantities           
+C                        :: of each neighbor entry.                     
+\end{verbatim}