/[MITgcm]/manual/s_phys_pkgs/text/exch2.tex
ViewVC logotype

Diff of /manual/s_phys_pkgs/text/exch2.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.8 by edhill, Tue Feb 17 21:58:56 2004 UTC revision 1.10 by afe, Mon Mar 15 20:11:56 2004 UTC
# Line 10  Line 10 
10  %%    o automatically inserted at \section{Reference}  %%    o automatically inserted at \section{Reference}
11    
12    
13  \section{Extended Cubed Sphere Exchange}  \section{exch2: Extended Cubed Sphere \mbox{Topology}}
14  \label{sec:exch2}  \label{sec:exch2}
15    
16    
17  \subsection{Introduction}  \subsection{Introduction}
18    
19  The \texttt{exch2} package is an extension to the original cubed  The \texttt{exch2} package is an extension to the original cubed
20  sphere exchanges to allow more flexible domain decomposition and  sphere topological configuration that allows more flexible domain
21  parallelization.  Cube faces (subdomains) may be divided into whatever  decomposition and parallelization.  Cube faces (also called
22  number of tiles that divide evenly into the grid point dimensions of  subdomains) may be divided into any number of tiles that divide evenly
23  the subdomain.  Furthermore, the individual tiles may be run on  into the grid point dimensions of the subdomain.  Furthermore, the
24  separate processors in different combinations, and whether exchanges  individual tiles may be run on separate processors in different
25  between particular tiles occur between different processors is  combinations, and whether exchanges between particular tiles occur
26  determined at runtime.  between different processors is determined at runtime.  This
27    flexibility provides for manual compile-time load balancing across a
28    relatively arbitrary number of processors. \\
29    
30  The exchange parameters are declared in  The exchange parameters are declared in
31  \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}  \filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h}
32  and assigned in  and assigned in
33  \filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}, both in  \filelink{pkg/exch2/w2\_e2setup.F}{pkg-exch2-w2_e2setup.F}. The
34  the \texttt{pkg/exch2} directory.  The validity of the cube topology  validity of the cube topology depends on the \texttt{SIZE.h} file as
35  depends on the \texttt{SIZE.h} file as detailed below.  Both files are  detailed below.  Both files are generated by Matlab scripts in
36  generated by Matlab scripts and should not be edited.  The default  \texttt{utils/exch2/matlab-topology-generator}; see Section
37  files provided in the release set up a cube sphere arrangement of six  \ref{sec:topogen} for details on creating alternate topologies.  The
38  tiles, one per subdomain, each with 32x32 grid points, running on a  default files provided in the release configure a cubed sphere
39  single processor.  topology of six tiles, one per subdomain, each with 32$\times$32 grid
40    points, all running on a single processor.  Pregenerated examples of
41    these files with alternate topologies are provided under
42    \texttt{utils/exch2/code-mods} along with the appropriate
43    \texttt{SIZE.h} file for single-processor execution.
44    
45    \subsection{Invoking exch2}
46    
47    To use exch2 with the cubed sphere, the following conditions must be
48    met: \\
49    
50    $\bullet$ The exch2 package is included when \texttt{genmake2} is run.  The
51      easiest way to do this is to add the line \texttt{exch2} to the
52      \texttt{profile.conf} file -- see Section \ref{sect:buildingCode}
53      for general details. \\
54    
55    $\bullet$ An example of \texttt{W2\_EXCH2\_TOPOLOGY.h} and
56      \texttt{w2\_e2setup.F} must reside in a directory containing code
57      linked when \texttt{genmake2} runs.  The safest place to put these
58      is the directory indicated in the \texttt{-mods=DIR} command line
59      modifier (typically \texttt{../code}), or the build directory.  The
60      default versions of these files reside in \texttt{pkg/exch2} and are
61      linked automatically if no other versions exist elsewhere in the
62      link path, but they should be left untouched to avoid breaking
63      configurations other than the one you intend to modify.\\
64    
65    $\bullet$ Files containing grid parameters, named
66      \texttt{tile}???\texttt{.mitgrid} where ??? is \texttt{001} through
67      \texttt{006} (one per subdomain), must be in the working directory
68      when the MITgcm executable is run.  These files are provided in the
69      example experiments for cubed sphere configurations with
70      32$\times$32 cube sides and are non-trivial to generate -- please
71      contact MITgcm support if you want to generate files for other
72      configurations. \\
73    
74    $\bullet$ As always when compiling MITgcm, the file \texttt{SIZE.h}
75      must be placed where \texttt{genmake2} will find it.  In particular
76      for the exch2, the domain decompositin specified in \texttt{SIZE.h}
77      must correspond with the particular configuration's topology
78      specified in \texttt{W2\_EXCH2\_TOPOLOGY.h} and
79      \texttt{w2\_e2setup.F}.  Domain decomposition issues particular to
80      exch2 are addressed in Section \ref{sec:topogen}: ``Generating
81      Topology Files for exch2''; a more general background on the subject
82      relvant to MITgcm is presented in Section
83      \ref{sect:specifying_a_decomposition}: ``Specifying a
84      decomposition''.\\
85    
86    As of the time of writing the following examples use exch2 and may be
87    used for guidance:
88    
89    \begin{verbatim}
90    verification/adjust_nlfs.cs-32x32x1
91    verification/adjustment.cs-32x32x1
92    verification/aim.5l_cs
93    verification/global_ocean.cs32x15
94    verification/hs94.cs-32x32x5
95    \end{verbatim}
96    
97    
98    
99    
100    \subsection{Generating Topology Files for exch2}
101    \label{sec:topogen}
102    
103    Alternate cubed sphere topologies may be created using the Matlab
104    scripts in \texttt{utils/exch2/matlab-topology-generator}. Running the
105    m-file \texttt{driver} from the Matlab prompt (without passing any
106    function parameters) generates exch2 topology files
107    \texttt{W2\_EXCH2\_TOPOLOGY.h} and \texttt{w2\_e2setup.F} in the
108    working directory and displays via Matlab a figure of the topology.
109    The other m-files in the directory are subroutines of \texttt{driver}
110    and should not be run except for development purposes. \\
111    
112    The parameters that determine the dimensions and topology of the
113    generated configuration are nr, nb, ng, tnx and tny, and all are
114    assigned early in the script.
115    
116    The first three determine the size of the subdomains (cube faces) and
117    hence the size of the overall domain.  Each one determines the number
118    of grid points, and therefore the resolution, along the subdomain
119    sides in a ``great circle'' around each axis of the cube.  At the time
120    of this writing MITgcm requires these three parameters to be equal,
121    but they provide for future releases of MITgcm to accomodate different
122    resolutions around the axes to allow (for example) greater resolution
123    around the equator.\\
124    
125    The parameters tnx and tny determine the dimensions of the tiles into
126    which the subdomains are decomposed, and must evenly divide the
127    integer assigned to nr, nb and ng.  The result is a rectangular tiling
128    of the subdomain.  Figure \ref{fig:24tile} shows one possible topology
129    for a twenty-four tile cube, and figure \ref{fig:12tile} shows one for
130    twelve tiles. \\
131    
132    \begin{figure}
133    \begin{center}
134     \resizebox{4in}{!}{
135      \includegraphics{part6/s24t_16x16.ps}
136     }
137    \end{center}
138    \caption{Plot of cubed sphere topology with a 32$\times$32 grid and
139    twenty-four tiles (tnx=16, tny=16)
140    } \label{fig:24tile}
141    \end{figure}
142    
143    \begin{figure}
144    \begin{center}
145     \resizebox{4in}{!}{
146      \includegraphics{part6/s12t_16x32.ps}
147     }
148    \end{center}
149    \caption{Plot of cubed sphere topology with a 32$\times$32 grid and
150    twelve tiles (tnx=16, tny=32)
151    } \label{fig:12tile}
152    \end{figure}
153    
154    Tiles can be selected from the topology to be omitted from being
155    allocated memory and processors.  This kind of tuning is useful in
156    ocean modeling for omitting tiles that fall entirely on land.  The
157    tiles omitted are specified in the file \texttt{blanklist.txt} by
158    their tile number in the topology, separated by a newline. \\
159    
160    
161    
162    
163    
164    
165  \subsection{Key Variables}  \subsection{Key Variables}
166    
# Line 52  The number of tiles in a particular topo Line 178  The number of tiles in a particular topo
178  \texttt{NTILES}, and the maximum number of neighbors of any tiles by  \texttt{NTILES}, and the maximum number of neighbors of any tiles by
179  \texttt{MAX\_NEIGHBOURS}.  These parameters are used for defining the  \texttt{MAX\_NEIGHBOURS}.  These parameters are used for defining the
180  size of the various one and two dimensional arrays that store tile  size of the various one and two dimensional arrays that store tile
181  parameters indexed to the tile number.  parameters indexed to the tile number.\\
182    
183  The scalar parameters \varlink{exch2\_domain\_nxt}{exch2_domain_nxt}  The scalar parameters \varlink{exch2\_domain\_nxt}{exch2_domain_nxt}
184  and \varlink{exch2\_domain\_nyt}{exch2_domain_nyt} express the number  and \varlink{exch2\_domain\_nyt}{exch2_domain_nyt} express the number
Legend:
Removed from v.1.8  
changed lines
  Added in v.1.10
  ViewVC Help
Powered by ViewVC 1.1.22