/[MITgcm]/manual/s_phys_pkgs/text/exch2.tex
ViewVC logotype

Diff of /manual/s_phys_pkgs/text/exch2.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.8 by edhill, Tue Feb 17 21:58:56 2004 UTC revision 1.11 by afe, Mon Mar 15 22:39:28 2004 UTC
# Line 10  Line 10 
10  %%    o automatically inserted at \section{Reference}  %%    o automatically inserted at \section{Reference}
11    
12    
13  \section{Extended Cubed Sphere Exchange}  \section{exch2: Extended Cubed Sphere \mbox{Topology}}
14  \label{sec:exch2}  \label{sec:exch2}
15    
16    
17  \subsection{Introduction}  \subsection{Introduction}
18    
19  The \texttt{exch2} package is an extension to the original cubed  The \texttt{exch2} package is an extension to the original cubed
20  sphere exchanges to allow more flexible domain decomposition and  sphere topological configuration that allows more flexible domain
21  parallelization.  Cube faces (subdomains) may be divided into whatever  decomposition and parallelization.  Cube faces (also called
22  number of tiles that divide evenly into the grid point dimensions of  subdomains) may be divided into any number of tiles that divide evenly
23  the subdomain.  Furthermore, the individual tiles may be run on  into the grid point dimensions of the subdomain.  Furthermore, the
24  separate processors in different combinations, and whether exchanges  individual tiles may be run on separate processors in different
25  between particular tiles occur between different processors is  combinations, and whether exchanges between particular tiles occur
26  determined at runtime.  between different processors is determined at runtime.  This
27    flexibility provides for manual compile-time load balancing across a
28    relatively arbitrary number of processors. \\
29    
30  The exchange parameters are declared in  The exchange parameters are declared in
31  \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}  \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}
32  and assigned in  and assigned in
33  \filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}, both in  \filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}. The
34  the \texttt{pkg/exch2} directory.  The validity of the cube topology  validity of the cube topology depends on the \file{SIZE.h} file as
35  depends on the \texttt{SIZE.h} file as detailed below.  Both files are  detailed below.  Both files are generated by Matlab scripts in
36  generated by Matlab scripts and should not be edited.  The default  \file{utils/exch2/matlab-topology-generator}; see Section
37  files provided in the release set up a cube sphere arrangement of six  \ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2}
38  tiles, one per subdomain, each with 32x32 grid points, running on a  for details on creating alternate topologies.  The default files
39  single processor.  provided in the release configure a cubed sphere topology of six
40    tiles, one per subdomain, each with 32$\times$32 grid points, all
41    running on a single processor.  Pregenerated examples of these files
42    with alternate topologies are provided under
43    \file{utils/exch2/code-mods} along with the appropriate \file{SIZE.h}
44    file for single-processor execution.
45    
46    \subsection{Invoking exch2}
47    
48    To use exch2 with the cubed sphere, the following conditions must be
49    met: \\
50    
51    $\bullet$ The exch2 package is included when \file{genmake2} is run.
52      The easiest way to do this is to add the line \code{exch2} to the
53      \file{profile.conf} file -- see Section
54      \ref{sect:buildingCode}\sectiontitle{Building the code} for general
55      details. \\
56    
57    $\bullet$ An example of \file{W2\_EXCH2\_TOPOLOGY.h} and
58      \file{w2\_e2setup.F} must reside in a directory containing code
59      linked when \file{genmake2} runs.  The safest place to put these
60      is the directory indicated in the \code{-mods=DIR} command line
61      modifier (typically \file{../code}), or the build directory.  The
62      default versions of these files reside in \file{pkg/exch2} and are
63      linked automatically if no other versions exist elsewhere in the
64      link path, but they should be left untouched to avoid breaking
65      configurations other than the one you intend to modify.\\
66    
67    $\bullet$ Files containing grid parameters, named
68      \file{tile???.mitgrid} where \file{???} is \file{001} through
69      \file{006} (one per subdomain), must be in the working directory
70      when the MITgcm executable is run.  These files are provided in the
71      example experiments for cubed sphere configurations with
72      32$\times$32 cube sides and are non-trivial to generate -- please
73      contact MITgcm support if you want to generate files for other
74      configurations. \\
75    
76    $\bullet$ As always when compiling MITgcm, the file \file{SIZE.h}
77      must be placed where \file{genmake2} will find it.  In particular
78      for the exch2, the domain decomposition specified in \file{SIZE.h}
79      must correspond with the particular configuration's topology
80      specified in \file{W2\_EXCH2\_TOPOLOGY.h} and
81      \file{w2\_e2setup.F}.  Domain decomposition issues particular to
82      exch2 are addressed in Section \ref{sec:topogen} \sectiontitle{Generating
83      Topology Files for exch2}; a more general background on the subject
84      relvant to MITgcm is presented in Section
85      \ref{sect:specifying_a_decomposition}\sectiontitle{Specifying a
86      decomposition}.\\
87    
88    As of the time of writing the following examples use exch2 and may be
89    used for guidance:
90    
91    \begin{verbatim}
92    verification/adjust_nlfs.cs-32x32x1
93    verification/adjustment.cs-32x32x1
94    verification/aim.5l_cs
95    verification/global_ocean.cs32x15
96    verification/hs94.cs-32x32x5
97    \end{verbatim}
98    
99    
100    
101    
102    \subsection{Generating Topology Files for exch2}
103    \label{sec:topogen}
104    
105    Alternate cubed sphere topologies may be created using the Matlab
106    scripts in \file{utils/exch2/matlab-topology-generator}. Running the
107    m-file \file{driver.m} from the Matlab prompt (there are no parameters
108    to pass) generates exch2 topology files \file{W2\_EXCH2\_TOPOLOGY.h}
109    and \file{w2\_e2setup.F} in the working directory and displays a
110    figure of the topology via Matlab.  The other m-files in the directory
111    are subroutines of \file{driver.m} and should not be run except for
112    development purposes. \\
113    
114    The parameters that determine the dimensions and topology of the
115    generated configuration are \code{nr}, \code{nb}, \code{ng},
116    \code{tnx} and \code{tny}, and all are assigned early in the script.
117    
118    The first three determine the size of the subdomains (cube faces) and
119    hence the size of the overall domain.  Each one determines the number
120    of grid points, and therefore the resolution, along the subdomain
121    sides in a ``great circle'' around each axis of the cube.  At the time
122    of this writing MITgcm requires these three parameters to be equal,
123    but they provide for future releases of MITgcm to accomodate different
124    resolutions around the axes to allow (for example) greater resolution
125    around the equator.\\
126    
127    The parameters \code{tnx} and \code{tny} determine the dimensions of
128    the tiles into which the subdomains are decomposed, and must evenly
129    divide the integer assigned to \code{nr}, \code{nb} and \code{ng}.
130    The result is a rectangular tiling of the subdomain.  Figure
131    \ref{fig:24tile} shows one possible topology for a twenty-four tile
132    cube, and figure \ref{fig:12tile} shows one for twelve tiles. \\
133    
134    \begin{figure}
135    \begin{center}
136     \resizebox{4in}{!}{
137      \includegraphics{part6/s24t_16x16.ps}
138     }
139    \end{center}
140    \caption{Plot of cubed sphere topology with a 32$\times$32 grid and
141    twenty-four tiles (\code{tnx=16, tny=16})
142    } \label{fig:24tile}
143    \end{figure}
144    
145    \begin{figure}
146    \begin{center}
147     \resizebox{4in}{!}{
148      \includegraphics{part6/s12t_16x32.ps}
149     }
150    \end{center}
151    \caption{Plot of cubed sphere topology with a 32$\times$32 grid and
152    twelve tiles (\code{tnx=16, tny=32})
153    } \label{fig:12tile}
154    \end{figure}
155    
156    Tiles can be selected from the topology to be omitted from being
157    allocated memory and processors.  This kind otuning is useful in
158    ocean modeling for omitting tiles that fall entirely on land.  The
159    tiles omitted are specified in the file \file{blanklist.txt} by
160    their tile number in the topology, separated by a newline. \\
161    
162    
163    
164    
165    
166    
167  \subsection{Key Variables}  \subsection{Key Variables}
168    
# Line 52  The number of tiles in a particular topo Line 180  The number of tiles in a particular topo
180  \texttt{NTILES}, and the maximum number of neighbors of any tiles by  \texttt{NTILES}, and the maximum number of neighbors of any tiles by
181  \texttt{MAX\_NEIGHBOURS}.  These parameters are used for defining the  \texttt{MAX\_NEIGHBOURS}.  These parameters are used for defining the
182  size of the various one and two dimensional arrays that store tile  size of the various one and two dimensional arrays that store tile
183  parameters indexed to the tile number.  parameters indexed to the tile number.\\
184    
185  The scalar parameters \varlink{exch2\_domain\_nxt}{exch2_domain_nxt}  The scalar parameters \varlink{exch2\_domain\_nxt}{exch2_domain_nxt}
186  and \varlink{exch2\_domain\_nyt}{exch2_domain_nyt} express the number  and \varlink{exch2\_domain\_nyt}{exch2_domain_nyt} express the number
Legend:
Removed from v.1.8  
changed lines
  Added in v.1.11
  ViewVC Help
Powered by ViewVC 1.1.22