23 |
dimensions of the subdomain. Furthermore, the tiles can run on |
dimensions of the subdomain. Furthermore, the tiles can run on |
24 |
separate processors individually or in groups, which provides for |
separate processors individually or in groups, which provides for |
25 |
manual compile-time load balancing across a relatively arbitrary |
manual compile-time load balancing across a relatively arbitrary |
26 |
number of processors. \\ |
number of processors. |
27 |
|
|
28 |
The exchange parameters are declared in |
The exchange parameters are declared in |
29 |
\filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h} |
\filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h} |
44 |
\subsubsection{Invoking exch2} |
\subsubsection{Invoking exch2} |
45 |
|
|
46 |
To use exch2 with the cubed sphere, the following conditions must be |
To use exch2 with the cubed sphere, the following conditions must be |
47 |
met: \\ |
met: |
48 |
|
|
49 |
$\bullet$ The exch2 package is included when \file{genmake2} is run. |
\begin{itemize} |
50 |
The easiest way to do this is to add the line \code{exch2} to the |
\item The exch2 package is included when \file{genmake2} is run. The |
51 |
\file{profile.conf} file -- see Section |
easiest way to do this is to add the line \code{exch2} to the |
52 |
\ref{sect:buildingCode} \sectiontitle{Building the code} for general |
\file{packages.conf} file -- see Section \ref{sect:buildingCode} |
53 |
details. \\ |
\sectiontitle{Building the code} for general |
54 |
|
details. |
55 |
|
|
56 |
$\bullet$ An example of \file{W2\_EXCH2\_TOPOLOGY.h} and |
\item An example of \file{W2\_EXCH2\_TOPOLOGY.h} and |
57 |
\file{w2\_e2setup.F} must reside in a directory containing files |
\file{w2\_e2setup.F} must reside in a directory containing files |
58 |
symbolically linked by the \file{genmake2} script. The safest place to |
symbolically linked by the \file{genmake2} script. The safest place |
59 |
put these is the directory indicated in the \code{-mods=DIR} command |
to put these is the directory indicated in the \code{-mods=DIR} |
60 |
line modifier (typically \file{../code}), or the build directory. |
command line modifier (typically \file{../code}), or the build |
61 |
The default versions of these files reside in \file{pkg/exch2} and |
directory. The default versions of these files reside in |
62 |
are linked automatically if no other versions exist elsewhere in the |
\file{pkg/exch2} and are linked automatically if no other versions |
63 |
build path, but they should be left untouched to avoid breaking |
exist elsewhere in the build path, but they should be left untouched |
64 |
configurations other than the one you intend to modify.\\ |
to avoid breaking configurations other than the one you intend to |
65 |
|
modify. |
66 |
$\bullet$ Files containing grid parameters, named |
|
67 |
\file{tile00$n$.mitgrid} where $n$=\code{(1:6)} (one per subdomain), |
\item Files containing grid parameters, named \file{tile00$n$.mitgrid} |
68 |
must be in the working directory when the MITgcm executable is run. |
where $n$=\code{(1:6)} (one per subdomain), must be in the working |
69 |
These files are provided in the example experiments for cubed sphere |
directory when the MITgcm executable is run. These files are |
70 |
configurations with 32$\times$32 cube sides |
provided in the example experiments for cubed sphere configurations |
71 |
-- please contact MITgcm support if you want to generate |
with 32$\times$32 cube sides -- please contact MITgcm support if you |
72 |
files for other configurations. \\ |
want to generate files for other configurations. |
73 |
|
|
74 |
$\bullet$ As always when compiling MITgcm, the file \file{SIZE.h} must |
\item As always when compiling MITgcm, the file \file{SIZE.h} must be |
75 |
be placed where \file{genmake2} will find it. In particular for |
placed where \file{genmake2} will find it. In particular for exch2, |
76 |
exch2, the domain decomposition specified in \file{SIZE.h} must |
the domain decomposition specified in \file{SIZE.h} must correspond |
77 |
correspond with the particular configuration's topology specified in |
with the particular configuration's topology specified in |
78 |
\file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}. Domain |
\file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}. Domain |
79 |
decomposition issues particular to exch2 are addressed in Section |
decomposition issues particular to exch2 are addressed in Section |
80 |
\ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2} |
\ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2} |
81 |
and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and Multiprocessing}; a more |
and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and |
82 |
general background on the subject relevant to MITgcm is presented in |
Multiprocessing}; a more general background on the subject |
83 |
Section \ref{sect:specifying_a_decomposition} |
relevant to MITgcm is presented in Section |
84 |
\sectiontitle{Specifying a decomposition}.\\ |
\ref{sect:specifying_a_decomposition} |
85 |
|
\sectiontitle{Specifying a decomposition}. |
86 |
|
\end{itemize} |
87 |
|
|
88 |
At the time of this writing the following examples use exch2 and may |
At the time of this writing the following examples use exch2 and may |
89 |
be used for guidance: |
be used for guidance: |
191 |
Once the topology configuration files are created, the Fortran |
Once the topology configuration files are created, the Fortran |
192 |
\code{PARAMETER}s in \file{SIZE.h} must be configured to match. |
\code{PARAMETER}s in \file{SIZE.h} must be configured to match. |
193 |
Section \ref{sect:specifying_a_decomposition} \sectiontitle{Specifying |
Section \ref{sect:specifying_a_decomposition} \sectiontitle{Specifying |
194 |
a decomposition} provides a general description of domain |
a decomposition} provides a general description of domain |
195 |
decomposition within MITgcm and its relation to \file{SIZE.h}. The |
decomposition within MITgcm and its relation to \file{SIZE.h}. The |
196 |
current section specifies constraints that the exch2 package |
current section specifies constraints that the exch2 package imposes |
197 |
imposes and describes how to enable parallel execution with |
and describes how to enable parallel execution with MPI. |
|
MPI. \\ |
|
198 |
|
|
199 |
As in the general case, the parameters \varlink{sNx}{sNx} and |
As in the general case, the parameters \varlink{sNx}{sNx} and |
200 |
\varlink{sNy}{sNy} define the size of the individual tiles, and so |
\varlink{sNy}{sNy} define the size of the individual tiles, and so |
201 |
must be assigned the same respective values as \code{tnx} and |
must be assigned the same respective values as \code{tnx} and |
202 |
\code{tny} in \file{driver.m}.\\ |
\code{tny} in \file{driver.m}. |
203 |
|
|
204 |
The halo width parameters \varlink{OLx}{OLx} and \varlink{OLy}{OLy} |
The halo width parameters \varlink{OLx}{OLx} and \varlink{OLy}{OLy} |
205 |
have no special bearing on exch2 and may be assigned as in the general |
have no special bearing on exch2 and may be assigned as in the general |
206 |
case. The same holds for \varlink{Nr}{Nr}, the number of vertical |
case. The same holds for \varlink{Nr}{Nr}, the number of vertical |
207 |
levels in the model.\\ |
levels in the model. |
208 |
|
|
209 |
The parameters \varlink{nSx}{nSx}, \varlink{nSy}{nSy}, |
The parameters \varlink{nSx}{nSx}, \varlink{nSy}{nSy}, |
210 |
\varlink{nPx}{nPx}, and \varlink{nPy}{nPy} relate to the number of |
\varlink{nPx}{nPx}, and \varlink{nPy}{nPy} relate to the number of |
212 |
the tiles are stored in the $x$ dimension, and so |
the tiles are stored in the $x$ dimension, and so |
213 |
\code{\varlink{nSy}{nSy}=1} in all cases. Since the tiles as |
\code{\varlink{nSy}{nSy}=1} in all cases. Since the tiles as |
214 |
configured by exch2 cannot be split up accross processors without |
configured by exch2 cannot be split up accross processors without |
215 |
regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well. \\ |
regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well. |
216 |
|
|
217 |
The number of tiles MITgcm allocates and how they are distributed |
The number of tiles MITgcm allocates and how they are distributed |
218 |
between processors depends on \varlink{nPx}{nPx} and |
between processors depends on \varlink{nPx}{nPx} and |
229 |
distribute the remaining twenty-nine tiles among five processors, you |
distribute the remaining twenty-nine tiles among five processors, you |
230 |
would have to run one ``dummy'' tile to make an even six tiles per |
would have to run one ``dummy'' tile to make an even six tiles per |
231 |
processor. Such dummy tiles are \emph{not} listed in |
processor. Such dummy tiles are \emph{not} listed in |
232 |
\file{blanklist.txt}.\\ |
\file{blanklist.txt}. |
|
|
|
233 |
|
|
234 |
The following is an example of \file{SIZE.h} for the twelve-tile |
The following is an example of \file{SIZE.h} for the twelve-tile |
235 |
configuration illustrated in figure \ref{fig:12tile} running on |
configuration illustrated in figure \ref{fig:12tile} running on one |
236 |
one processor: \\ |
processor: |
237 |
|
|
238 |
\begin{verbatim} |
\begin{verbatim} |
239 |
PARAMETER ( |
PARAMETER ( |
269 |
\end{verbatim} |
\end{verbatim} |
270 |
|
|
271 |
|
|
|
|
|
|
|
|
|
|
|
272 |
\subsubsection{Key Variables} |
\subsubsection{Key Variables} |
273 |
|
|
274 |
The descriptions of the variables are divided up into scalars, |
The descriptions of the variables are divided up into scalars, |