--- manual/s_phys_pkgs/text/exch2.tex	2004/10/12 15:47:40	1.20
+++ manual/s_phys_pkgs/text/exch2.tex	2006/06/28 15:35:07	1.26
@@ -1,4 +1,4 @@
-% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.20 2004/10/12 15:47:40 edhill Exp $
+% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.26 2006/06/28 15:35:07 molod Exp $
 % $Name:  $
 
 %%  * Introduction
@@ -10,11 +10,11 @@
 %%    o automatically inserted at \section{Reference} 
 
 
-\section{exch2: Extended Cubed Sphere \mbox{Topology}}
+\subsection{exch2: Extended Cubed Sphere \mbox{Topology}}
 \label{sec:exch2}
 
 
-\subsection{Introduction}
+\subsubsection{Introduction}
 
 The \texttt{exch2} package extends the original cubed sphere topology
 configuration to allow more flexible domain decomposition and
@@ -23,7 +23,7 @@
 dimensions of the subdomain.  Furthermore, the tiles can run on
 separate processors individually or in groups, which provides for
 manual compile-time load balancing across a relatively arbitrary
-number of processors. \\
+number of processors.
 
 The exchange parameters are declared in
 \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}
@@ -41,7 +41,7 @@
 \file{utils/exch2/code-mods} along with the appropriate \file{SIZE.h}
 file for single-processor execution.
 
-\subsection{Invoking exch2}
+\subsubsection{Invoking exch2}
 
 To use exch2 with the cubed sphere, the following conditions must be
 met:
@@ -49,9 +49,10 @@
 \begin{itemize}
 \item The exch2 package is included when \file{genmake2} is run.  The
   easiest way to do this is to add the line \code{exch2} to the
-  \file{profile.conf} file -- see Section \ref{sect:buildingCode}
-  \sectiontitle{Building the code} for general details.
-  
+  \file{packages.conf} file -- see Section \ref{sect:buildingCode}
+  \sectiontitle{Building the code} for general
+  details.
+
 \item An example of \file{W2\_EXCH2\_TOPOLOGY.h} and
   \file{w2\_e2setup.F} must reside in a directory containing files
   symbolically linked by the \file{genmake2} script.  The safest place
@@ -62,21 +63,14 @@
   exist elsewhere in the build path, but they should be left untouched
   to avoid breaking configurations other than the one you intend to
   modify.
-  
+
 \item Files containing grid parameters, named \file{tile00$n$.mitgrid}
   where $n$=\code{(1:6)} (one per subdomain), must be in the working
   directory when the MITgcm executable is run.  These files are
   provided in the example experiments for cubed sphere configurations
-  with 32$\times$32 cube sides -- please contact
-  \begin{rawhtml}
-    <A href=''mailto:mitgcm-support@dev.mitgcm.org"> 
-  \end{rawhtml} 
-\begin{verbatim}
-MITgcm-support@mitgcm.org
-\end{verbatim}
-  \begin{rawhtml} </A> \end{rawhtml}
-  if you want to generate files for other configurations.
-  
+  with 32$\times$32 cube sides -- please contact MITgcm support if you
+  want to generate files for other configurations.
+
 \item As always when compiling MITgcm, the file \file{SIZE.h} must be
   placed where \file{genmake2} will find it.  In particular for exch2,
   the domain decomposition specified in \file{SIZE.h} must correspond
@@ -87,12 +81,10 @@
   and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and
     Multiprocessing}; a more general background on the subject
   relevant to MITgcm is presented in Section
-  \ref{sect:specifying_a_decomposition} \sectiontitle{Specifying a
-    decomposition}.
+  \ref{sect:specifying_a_decomposition}
+  \sectiontitle{Specifying a decomposition}.
 \end{itemize}
 
-
-
 At the time of this writing the following examples use exch2 and may
 be used for guidance:
 
@@ -107,7 +99,7 @@
 
 
 
-\subsection{Generating Topology Files for exch2}
+\subsubsection{Generating Topology Files for exch2}
 \label{sec:topogen}
 
 Alternate cubed sphere topologies may be created using the Matlab
@@ -193,27 +185,26 @@
 
 
 
-\subsection{exch2, SIZE.h, and Multiprocessing}
+\subsubsection{exch2, SIZE.h, and Multiprocessing}
 \label{sec:exch2mpi}
 
 Once the topology configuration files are created, the Fortran
 \code{PARAMETER}s in \file{SIZE.h} must be configured to match.
 Section \ref{sect:specifying_a_decomposition} \sectiontitle{Specifying
-a decomposition} provides a general description of domain
+  a decomposition} provides a general description of domain
 decomposition within MITgcm and its relation to \file{SIZE.h}. The
-current section specifies constraints that the exch2 package
-imposes and describes how to enable parallel execution with
-MPI. \\
+current section specifies constraints that the exch2 package imposes
+and describes how to enable parallel execution with MPI.
 
 As in the general case, the parameters \varlink{sNx}{sNx} and
 \varlink{sNy}{sNy} define the size of the individual tiles, and so
 must be assigned the same respective values as \code{tnx} and
-\code{tny} in \file{driver.m}.\\
+\code{tny} in \file{driver.m}.
 
 The halo width parameters \varlink{OLx}{OLx} and \varlink{OLy}{OLy}
 have no special bearing on exch2 and may be assigned as in the general
-case. The same holds for \varlink{Nr}{Nr}, the number of vertical 
-levels in the model.\\
+case. The same holds for \varlink{Nr}{Nr}, the number of vertical
+levels in the model.
 
 The parameters \varlink{nSx}{nSx}, \varlink{nSy}{nSy},
 \varlink{nPx}{nPx}, and \varlink{nPy}{nPy} relate to the number of
@@ -221,21 +212,28 @@
 the tiles are stored in the $x$ dimension, and so
 \code{\varlink{nSy}{nSy}=1} in all cases.  Since the tiles as
 configured by exch2 cannot be split up accross processors without
-regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well. \\
+regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well.
 
 The number of tiles MITgcm allocates and how they are distributed
 between processors depends on \varlink{nPx}{nPx} and
 \varlink{nSx}{nSx}.  \varlink{nSx}{nSx} is the number of tiles per
-processor and \varlink{nPx}{nPx} is the number of processors.  The total
-number of tiles in the topology minus those listed in
-\file{blanklist.txt} must equal \code{nSx*nPx}.  Note that in order to 
+processor and \varlink{nPx}{nPx} is the number of processors.  The
+total number of tiles in the topology minus those listed in
+\file{blanklist.txt} must equal \code{nSx*nPx}.  Note that in order to
 obtain maximum usage from a given number of processors in some cases,
-this restriction might entail sharing a processor with a tile that would 
-otherwise be excluded. \\
+this restriction might entail sharing a processor with a tile that
+would otherwise be excluded because it is topographically outside of
+the domain and therefore in \file{blanklist.txt}.  For example,
+suppose you have five processors and a domain decomposition of
+thirty-six tiles that allows you to exclude seven tiles.  To evenly
+distribute the remaining twenty-nine tiles among five processors, you
+would have to run one ``dummy'' tile to make an even six tiles per
+processor.  Such dummy tiles are \emph{not} listed in
+\file{blanklist.txt}.
 
 The following is an example of \file{SIZE.h} for the twelve-tile
-configuration illustrated in figure \ref{fig:12tile} running on 
-one processor: \\
+configuration illustrated in figure \ref{fig:12tile} running on one
+processor:
 
 \begin{verbatim}
       PARAMETER (
@@ -271,10 +269,7 @@
 \end{verbatim}
 
 
-
-
-
-\subsection{Key Variables}
+\subsubsection{Key Variables}
 
 The descriptions of the variables are divided up into scalars,
 one-dimensional arrays indexed to the tile number, and two and
@@ -284,7 +279,7 @@
 arrays to individual tiles, and the arrays indexed by tile and
 neighbor to relationships between tiles and their neighbors. \\
 
-\subsubsection{Scalars}
+Scalars:
 
 The number of tiles in a particular topology is set with the parameter
 \code{NTILES}, and the maximum number of neighbors of any tiles by
@@ -307,11 +302,11 @@
 $x$ axis, and the $y$ axis variable \varlink{bj}{bj} is assumed to 
 equal \code{1} throughout the package. \\
 
-\subsubsection{Arrays indexed to tile number}
+Arrays indexed to tile number:
 
 The following arrays are of length \code{NTILES} and are indexed to
 the tile number, which is indicated in the diagrams with the notation
-\code{tn}.  The indices are omitted in the descriptions. \\
+\textsf{t}$n$.  The indices are omitted in the descriptions. \\
 
 The arrays \varlink{exch2\_tnx}{exch2_tnx} and
 \varlink{exch2\_tny}{exch2_tny} express the $x$ and $y$ dimensions of
@@ -339,11 +334,11 @@
 
 The array \varlink{exch2\_myFace}{exch2_myFace} contains the number of
 the subdomain of each tile, in a range \code{(1:6)} in the case of the
-standard cube topology and indicated by \textbf{\textsf{fn}} in
-figures \ref{fig:12tile} and \ref{fig:24tile}. The
-\varlink{exch2\_nNeighbours}{exch2_nNeighbours} variable contains a
-count of the neighboring tiles each tile has, and sets the bounds for
-looping over neighboring tiles.  And
+standard cube topology and indicated by \textbf{\textsf{f}}$n$ in
+figures \ref{fig:12tile} and
+\ref{fig:24tile}. \varlink{exch2\_nNeighbours}{exch2_nNeighbours}
+contains a count of the neighboring tiles each tile has, and sets 
+the bounds for looping over neighboring tiles.
 \varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each
 tile, and is used in interprocess communication.  \\
 
@@ -360,7 +355,7 @@
 corners of the cube. \\
 
 
-\subsubsection{Arrays Indexed to Tile Number and Neighbor}
+Arrays Indexed to Tile Number and Neighbor:
 
 The following arrays have vectors of length \code{MAX\_NEIGHBOURS} and
 \code{NTILES} and describe the orientations between the the tiles. \\
@@ -504,7 +499,7 @@
 width of \code{T}'s northern edge, expanded by one into the halo. \\
 
 
-\subsection{Key Routines}
+\subsubsection{Key Routines}
 
 Most of the subroutines particular to exch2 handle the exchanges
 themselves and are of the same format as those described in
@@ -534,3 +529,10 @@
 \code{EXCH2\_SEND\_RX1} and \code{EXCH2\_SEND\_RX2}, which do most of
 the work using the variables discussed above. \\
 
+\subsubsection{Experiments and tutorials that use exch2}
+\label{sec:pkg:exch2:experiments}
+
+\begin{itemize}
+\item{Held Suarez tutorial, in tutorial\_held\_suarez\_cs verification directory, 
+described in section \ref{sect:eg-hs} }
+\end{itemize}
