--- manual/s_phys_pkgs/text/exch2.tex 2004/02/17 21:58:56 1.8 +++ manual/s_phys_pkgs/text/exch2.tex 2004/03/12 20:58:19 1.9 @@ -1,4 +1,4 @@ -% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.8 2004/02/17 21:58:56 edhill Exp $ +% $Header: /home/ubuntu/mnt/e9_copy/manual/s_phys_pkgs/text/exch2.tex,v 1.9 2004/03/12 20:58:19 afe Exp $ % $Name: $ %% * Introduction @@ -17,24 +17,70 @@ \subsection{Introduction} The \texttt{exch2} package is an extension to the original cubed -sphere exchanges to allow more flexible domain decomposition and -parallelization. Cube faces (subdomains) may be divided into whatever -number of tiles that divide evenly into the grid point dimensions of -the subdomain. Furthermore, the individual tiles may be run on -separate processors in different combinations, and whether exchanges -between particular tiles occur between different processors is -determined at runtime. +sphere topological configuration that allows more flexible domain +decomposition and parallelization. Cube faces (also called +subdomains) may be divided into any number of tiles that divide evenly +into the grid point dimensions of the subdomain. Furthermore, the +individual tiles may be run on separate processors in different +combinations, and whether exchanges between particular tiles occur +between different processors is determined at runtime. This +flexibility provides for manual load balancing across a relatively +arbitrary number of processors. The exchange parameters are declared in \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h} and assigned in -\filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}, both in -the \texttt{pkg/exch2} directory. The validity of the cube topology -depends on the \texttt{SIZE.h} file as detailed below. Both files are -generated by Matlab scripts and should not be edited. The default -files provided in the release set up a cube sphere arrangement of six -tiles, one per subdomain, each with 32x32 grid points, running on a -single processor. +\filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}. The +validity of the cube topology depends on the \texttt{SIZE.h} file as +detailed below. Both files are generated by Matlab scripts in ?? +check these in already! and should not be edited. The default files +provided in the release configure a cubed sphere arrangement of six +tiles, one per subdomain, each with 32$\times$32 grid points, all +running on a single processor. Pregenerated examples of these files +with alternate topologies are provided in ??. + +\subsection{Invoking exch2} + +To use exch2 with the cubed sphere, the following conditions must be met: + +- the exch2 package is included when \texttt{genmake2} is run. The + easiest way to do this is to add the line \texttt{exch2} to the + \texttt{profile.conf} file -- see Section \ref{sect:buildingCode} + for general details. \\ + +- an example of \texttt{W2\_EXCH2\_TOPOLOGY.h} and + \texttt{w2\_e2setup.F} must reside in a directory containing code + linked when \texttt{genmake2} runs. The safest place to put these + is the directory indicated in the \texttt{-mods=DIR} command line + modifier (typically \texttt{../code}), or the build directory. The + default versions of these files reside in \texttt{pkg/exch2}, but + they should be left untouched to avoid breaking configurations other + than the one you intend to modify.\\ + +- files containing grid parameters, named + \texttt{tile}xxx\texttt{.mitgrid} where xxx is \texttt{001} through + \texttt{006}, must be in the working directory when the MITgcm + executable is run. These files are provided in the example + experiments for cubed sphere configurations with 32$\times$32 cube + sides and are non-trivial to generate -- please contact MITgcm + support if you want to generate files for other configurations. + This is lame. ?? \\ + +As of the time of writing the following examples use exch2 and may be +used for guidance: + +\begin{verbatim} +verification/adjust_nlfs.cs-32x32x1 +verification/adjustment.cs-32x32x1 +verification/aim.5l_cs +verification/global_ocean.cs32x15 +verification/hs94.cs-32x32x5 +\end{verbatim} + + + + +\subsection{Generating Topology Files} \subsection{Key Variables} @@ -52,7 +98,7 @@ \texttt{NTILES}, and the maximum number of neighbors of any tiles by \texttt{MAX\_NEIGHBOURS}. These parameters are used for defining the size of the various one and two dimensional arrays that store tile -parameters indexed to the tile number. +parameters indexed to the tile number.\\ The scalar parameters \varlink{exch2\_domain\_nxt}{exch2_domain_nxt} and \varlink{exch2\_domain\_nyt}{exch2_domain_nyt} express the number