--- manual/s_phys_pkgs/text/exch2.tex	2005/07/18 20:45:27	1.24
+++ manual/s_phys_pkgs/text/exch2.tex	2009/05/02 02:13:18	1.27
@@ -1,4 +1,4 @@
-% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.24 2005/07/18 20:45:27 molod Exp $
+% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.27 2009/05/02 02:13:18 jmc Exp $
 % $Name:  $
 
 %%  * Introduction
@@ -23,7 +23,7 @@
 dimensions of the subdomain.  Furthermore, the tiles can run on
 separate processors individually or in groups, which provides for
 manual compile-time load balancing across a relatively arbitrary
-number of processors. \\
+number of processors.
 
 The exchange parameters are declared in
 \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}
@@ -44,43 +44,46 @@
 \subsubsection{Invoking exch2}
 
 To use exch2 with the cubed sphere, the following conditions must be
-met: \\
+met:
 
-$\bullet$ The exch2 package is included when \file{genmake2} is run.
-  The easiest way to do this is to add the line \code{exch2} to the
-  \file{profile.conf} file -- see Section
-  \ref{sect:buildingCode} \sectiontitle{Building the code} for general
-  details. \\
+\begin{itemize}
+\item The exch2 package is included when \file{genmake2} is run.  The
+  easiest way to do this is to add the line \code{exch2} to the
+  \file{packages.conf} file -- see Section \ref{sect:buildingCode}
+  \sectiontitle{Building the code} for general
+  details.
 
-$\bullet$ An example of \file{W2\_EXCH2\_TOPOLOGY.h} and
+\item An example of \file{W2\_EXCH2\_TOPOLOGY.h} and
   \file{w2\_e2setup.F} must reside in a directory containing files
-  symbolically linked by the \file{genmake2} script.  The safest place to
-  put these is the directory indicated in the \code{-mods=DIR} command
-  line modifier (typically \file{../code}), or the build directory.
-  The default versions of these files reside in \file{pkg/exch2} and
-  are linked automatically if no other versions exist elsewhere in the
-  build path, but they should be left untouched to avoid breaking
-  configurations other than the one you intend to modify.\\
-
-$\bullet$ Files containing grid parameters, named
-  \file{tile00$n$.mitgrid} where $n$=\code{(1:6)} (one per subdomain),
-  must be in the working directory when the MITgcm executable is run.
-  These files are provided in the example experiments for cubed sphere
-  configurations with 32$\times$32 cube sides 
-  -- please contact MITgcm support if you want to generate
-  files for other configurations. \\
-
-$\bullet$ As always when compiling MITgcm, the file \file{SIZE.h} must
-  be placed where \file{genmake2} will find it.  In particular for
-  exch2, the domain decomposition specified in \file{SIZE.h} must
-  correspond with the particular configuration's topology specified in
+  symbolically linked by the \file{genmake2} script.  The safest place
+  to put these is the directory indicated in the \code{-mods=DIR}
+  command line modifier (typically \file{../code}), or the build
+  directory.  The default versions of these files reside in
+  \file{pkg/exch2} and are linked automatically if no other versions
+  exist elsewhere in the build path, but they should be left untouched
+  to avoid breaking configurations other than the one you intend to
+  modify.
+
+\item Files containing grid parameters, named \file{tile00$n$.mitgrid}
+  where $n$=\code{(1:6)} (one per subdomain), must be in the working
+  directory when the MITgcm executable is run.  These files are
+  provided in the example experiments for cubed sphere configurations
+  with 32$\times$32 cube sides -- please contact MITgcm support if you
+  want to generate files for other configurations.
+
+\item As always when compiling MITgcm, the file \file{SIZE.h} must be
+  placed where \file{genmake2} will find it.  In particular for exch2,
+  the domain decomposition specified in \file{SIZE.h} must correspond
+  with the particular configuration's topology specified in
   \file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}.  Domain
   decomposition issues particular to exch2 are addressed in Section
   \ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2}
-  and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and Multiprocessing}; a more
-  general background on the subject relevant to MITgcm is presented in
-  Section \ref{sect:specifying_a_decomposition}
-  \sectiontitle{Specifying a decomposition}.\\
+  and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and
+    Multiprocessing}; a more general background on the subject
+  relevant to MITgcm is presented in Section
+  \ref{sect:specifying_a_decomposition}
+  \sectiontitle{Specifying a decomposition}.
+\end{itemize}
 
 At the time of this writing the following examples use exch2 and may
 be used for guidance:
@@ -106,8 +109,8 @@
 from the Matlab prompt (there are no parameters to pass) generates
 exch2 topology files \file{W2\_EXCH2\_TOPOLOGY.h} and
 \file{w2\_e2setup.F} in the working directory and displays a figure of
-the topology via Matlab -- figures \ref{fig:6tile}, \ref{fig:12tile}, 
-and \ref{fig:24tile} are examples of the generated diagrams.  The other 
+the topology via Matlab -- figures \ref{fig:6tile}, \ref{fig:18tile}, 
+and \ref{fig:48tile} are examples of the generated diagrams.  The other 
 m-files in the directory are
 subroutines called from \file{driver.m} and should not be run ``bare'' except
 for development purposes. \\
@@ -128,39 +131,48 @@
 the tiles into which the subdomains are decomposed, and must evenly
 divide the integer assigned to \code{nr}, \code{nb} and \code{ng}.
 The result is a rectangular tiling of the subdomain.  Figure
-\ref{fig:24tile} shows one possible topology for a twenty-four-tile
-cube, and figure \ref{fig:12tile} shows one for twelve tiles. \\
+\ref{fig:48tile} shows one possible topology for a twenty-four-tile
+cube, and figure \ref{fig:6tile} shows one for six tiles. \\
 
 \begin{figure}
 \begin{center}
- \resizebox{4in}{!}{
-  \includegraphics{part6/s24t_16x16.ps}
+ \resizebox{6in}{!}{
+% \includegraphics{part6/s24t_16x16.ps}
+  \includegraphics{part6/adjust_cs.ps}
  }
 \end{center} 
 
 \caption{Plot of a cubed sphere topology with a 32$\times$192 domain
 divided into six 32$\times$32 subdomains, each of which is divided
-into four tiles of width \code{tnx=16} and height \code{tny=16} for a 
-total of twenty-four tiles.  The colored borders of the subdomains 
-represent the parameters \code{nr} (red), \code{nb} (blue), and 
-\code{ng} (green).  } \label{fig:24tile}
+into eight tiles of width \code{tnx=16} and height \code{tny=8} for a 
+total of forty-eight tiles. The colored borders of the subdomains 
+represent the parameters \code{nr} (red), \code{ng} (green), and
+\code{nb} (blue). 
+This tiling is used in the example
+verification/adjustment.cs-32x32x1/
+with the option (blanklist.txt) to remove the land-only 4 tiles 
+(11,12,13,14) which are filled in red on the plot.
+} \label{fig:48tile}
 \end{figure}
 
 \begin{figure}
 \begin{center}
- \resizebox{4in}{!}{
-  \includegraphics{part6/s12t_16x32.ps}
+ \resizebox{6in}{!}{
+% \includegraphics{part6/s12t_16x32.ps}
+  \includegraphics{part6/polarcap.ps}
  }
 \end{center} 
-\caption{Plot of a cubed sphere topology with a 32$\times$192 domain
-divided into six 32$\times$32 subdomains of two tiles each
- (\code{tnx=16, tny=32}).
-} \label{fig:12tile}
+\caption{Plot of a non-square cubed sphere topology with 
+6 subdomains of different size (nr=90,ng=360,nb=90),
+divided into one to four tiles each
+ (\code{tnx=90, tny=90}), resulting in a total of 18 tiles.
+} \label{fig:18tile}
 \end{figure}
 
 \begin{figure}
 \begin{center}
  \resizebox{4in}{!}{
+% \includegraphics{part6/s6t_32x32.ps}
   \includegraphics{part6/s6t_32x32.ps}
  }
 \end{center} 
@@ -188,21 +200,20 @@
 Once the topology configuration files are created, the Fortran
 \code{PARAMETER}s in \file{SIZE.h} must be configured to match.
 Section \ref{sect:specifying_a_decomposition} \sectiontitle{Specifying
-a decomposition} provides a general description of domain
+  a decomposition} provides a general description of domain
 decomposition within MITgcm and its relation to \file{SIZE.h}. The
-current section specifies constraints that the exch2 package
-imposes and describes how to enable parallel execution with
-MPI. \\
+current section specifies constraints that the exch2 package imposes
+and describes how to enable parallel execution with MPI.
 
 As in the general case, the parameters \varlink{sNx}{sNx} and
 \varlink{sNy}{sNy} define the size of the individual tiles, and so
 must be assigned the same respective values as \code{tnx} and
-\code{tny} in \file{driver.m}.\\
+\code{tny} in \file{driver.m}.
 
 The halo width parameters \varlink{OLx}{OLx} and \varlink{OLy}{OLy}
 have no special bearing on exch2 and may be assigned as in the general
-case. The same holds for \varlink{Nr}{Nr}, the number of vertical 
-levels in the model.\\
+case. The same holds for \varlink{Nr}{Nr}, the number of vertical
+levels in the model.
 
 The parameters \varlink{nSx}{nSx}, \varlink{nSy}{nSy},
 \varlink{nPx}{nPx}, and \varlink{nPy}{nPy} relate to the number of
@@ -210,7 +221,7 @@
 the tiles are stored in the $x$ dimension, and so
 \code{\varlink{nSy}{nSy}=1} in all cases.  Since the tiles as
 configured by exch2 cannot be split up accross processors without
-regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well. \\
+regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well.
 
 The number of tiles MITgcm allocates and how they are distributed
 between processors depends on \varlink{nPx}{nPx} and
@@ -227,20 +238,19 @@
 distribute the remaining twenty-nine tiles among five processors, you
 would have to run one ``dummy'' tile to make an even six tiles per
 processor.  Such dummy tiles are \emph{not} listed in
-\file{blanklist.txt}.\\
+\file{blanklist.txt}.
 
-
-The following is an example of \file{SIZE.h} for the twelve-tile
-configuration illustrated in figure \ref{fig:12tile} running on 
-one processor: \\
+The following is an example of \file{SIZE.h} for the six-tile
+configuration illustrated in figure \ref{fig:6tile} 
+running on one processor:
 
 \begin{verbatim}
       PARAMETER (
-     &           sNx =  16,
+     &           sNx =  32,
      &           sNy =  32,
      &           OLx =   2,
      &           OLy =   2,
-     &           nSx =  12,
+     &           nSx =   6,
      &           nSy =   1,
      &           nPx =   1,
      &           nPy =   1,
@@ -249,16 +259,16 @@
      &           Nr  =   5)
 \end{verbatim}
 
-The following is an example for the twenty-four-tile topology in
-figure \ref{fig:24tile} running on six processors:
+The following is an example for the forty-eight-tile topology in
+figure \ref{fig:48tile} running on six processors:
 
 \begin{verbatim}
       PARAMETER (
      &           sNx =  16,
-     &           sNy =  16,
+     &           sNy =   8,
      &           OLx =   2,
      &           OLy =   2,
-     &           nSx =   4,
+     &           nSx =   8,
      &           nSy =   1,
      &           nPx =   6,
      &           nPy =   1,
@@ -268,9 +278,6 @@
 \end{verbatim}
 
 
-
-
-
 \subsubsection{Key Variables}
 
 The descriptions of the variables are divided up into scalars,
@@ -295,9 +302,9 @@
 of tiles in the $x$ and $y$ global indices.  For example, the default
 setup of six tiles (Fig. \ref{fig:6tile}) has
 \code{exch2\_domain\_nxt=6} and \code{exch2\_domain\_nyt=1}.  A
-topology of twenty-four square tiles, four per subdomain (as in figure
-\ref{fig:24tile}), will have \code{exch2\_domain\_nxt=12} and
-\code{exch2\_domain\_nyt=2}.  Note that these parameters express the
+topology of forty-eight tiles, eight per subdomain (as in figure
+\ref{fig:48tile}), will have \code{exch2\_domain\_nxt=12} and
+\code{exch2\_domain\_nyt=4}.  Note that these parameters express the
 tile layout in order to allow global data files that are tile-layout-neutral.
 They have no bearing on the internal storage of the arrays.  The tiles
 are stored internally in a range from \code{\varlink{bi}{bi}=(1:NTILES)} in the
@@ -337,8 +344,8 @@
 The array \varlink{exch2\_myFace}{exch2_myFace} contains the number of
 the subdomain of each tile, in a range \code{(1:6)} in the case of the
 standard cube topology and indicated by \textbf{\textsf{f}}$n$ in
-figures \ref{fig:12tile} and
-\ref{fig:24tile}. \varlink{exch2\_nNeighbours}{exch2_nNeighbours}
+figures \ref{fig:6tile} and
+\ref{fig:48tile}. \varlink{exch2\_nNeighbours}{exch2_nNeighbours}
 contains a count of the neighboring tiles each tile has, and sets 
 the bounds for looping over neighboring tiles.
 \varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each
@@ -460,8 +467,8 @@
 \varlink{exch2\_jthi\_c}{exch2_jthi_c} hold the location and index
 bounds of the edge segment of the neighbor tile \code{N}'s subdomain
 that gets exchanged with the local tile \code{T}.  To take the example
-of tile \code{T=2} in the twelve-tile topology
-(Fig. \ref{fig:12tile}): \\
+of tile \code{T=2} in the forty-eight-tile topology
+(Fig. \ref{fig:48tile}): \\
 
 \begin{verbatim}
        exch2_itlo_c(4,2)=17
@@ -531,3 +538,10 @@
 \code{EXCH2\_SEND\_RX1} and \code{EXCH2\_SEND\_RX2}, which do most of
 the work using the variables discussed above. \\
 
+\subsubsection{Experiments and tutorials that use exch2}
+\label{sec:pkg:exch2:experiments}
+
+\begin{itemize}
+\item{Held Suarez tutorial, in tutorial\_held\_suarez\_cs verification directory, 
+described in section \ref{sect:eg-hs} }
+\end{itemize}
