| 23 |
dimensions of the subdomain. Furthermore, the tiles can run on |
dimensions of the subdomain. Furthermore, the tiles can run on |
| 24 |
separate processors individually or in groups, which provides for |
separate processors individually or in groups, which provides for |
| 25 |
manual compile-time load balancing across a relatively arbitrary |
manual compile-time load balancing across a relatively arbitrary |
| 26 |
number of processors. \\ |
number of processors. |
| 27 |
|
|
| 28 |
The exchange parameters are declared in |
The exchange parameters are declared in |
| 29 |
\filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h} |
\filelink{pkg/exch2/W2\_EXCH2\_TOPOLOGY.h}{pkg-exch2-W2_EXCH2_TOPOLOGY.h} |
| 44 |
\subsubsection{Invoking exch2} |
\subsubsection{Invoking exch2} |
| 45 |
|
|
| 46 |
To use exch2 with the cubed sphere, the following conditions must be |
To use exch2 with the cubed sphere, the following conditions must be |
| 47 |
met: \\ |
met: |
| 48 |
|
|
| 49 |
$\bullet$ The exch2 package is included when \file{genmake2} is run. |
\begin{itemize} |
| 50 |
The easiest way to do this is to add the line \code{exch2} to the |
\item The exch2 package is included when \file{genmake2} is run. The |
| 51 |
\file{profile.conf} file -- see Section |
easiest way to do this is to add the line \code{exch2} to the |
| 52 |
\ref{sect:buildingCode} \sectiontitle{Building the code} for general |
\file{packages.conf} file -- see Section \ref{sect:buildingCode} |
| 53 |
details. \\ |
\sectiontitle{Building the code} for general |
| 54 |
|
details. |
| 55 |
|
|
| 56 |
$\bullet$ An example of \file{W2\_EXCH2\_TOPOLOGY.h} and |
\item An example of \file{W2\_EXCH2\_TOPOLOGY.h} and |
| 57 |
\file{w2\_e2setup.F} must reside in a directory containing files |
\file{w2\_e2setup.F} must reside in a directory containing files |
| 58 |
symbolically linked by the \file{genmake2} script. The safest place to |
symbolically linked by the \file{genmake2} script. The safest place |
| 59 |
put these is the directory indicated in the \code{-mods=DIR} command |
to put these is the directory indicated in the \code{-mods=DIR} |
| 60 |
line modifier (typically \file{../code}), or the build directory. |
command line modifier (typically \file{../code}), or the build |
| 61 |
The default versions of these files reside in \file{pkg/exch2} and |
directory. The default versions of these files reside in |
| 62 |
are linked automatically if no other versions exist elsewhere in the |
\file{pkg/exch2} and are linked automatically if no other versions |
| 63 |
build path, but they should be left untouched to avoid breaking |
exist elsewhere in the build path, but they should be left untouched |
| 64 |
configurations other than the one you intend to modify.\\ |
to avoid breaking configurations other than the one you intend to |
| 65 |
|
modify. |
| 66 |
$\bullet$ Files containing grid parameters, named |
|
| 67 |
\file{tile00$n$.mitgrid} where $n$=\code{(1:6)} (one per subdomain), |
\item Files containing grid parameters, named \file{tile00$n$.mitgrid} |
| 68 |
must be in the working directory when the MITgcm executable is run. |
where $n$=\code{(1:6)} (one per subdomain), must be in the working |
| 69 |
These files are provided in the example experiments for cubed sphere |
directory when the MITgcm executable is run. These files are |
| 70 |
configurations with 32$\times$32 cube sides |
provided in the example experiments for cubed sphere configurations |
| 71 |
-- please contact MITgcm support if you want to generate |
with 32$\times$32 cube sides -- please contact MITgcm support if you |
| 72 |
files for other configurations. \\ |
want to generate files for other configurations. |
| 73 |
|
|
| 74 |
$\bullet$ As always when compiling MITgcm, the file \file{SIZE.h} must |
\item As always when compiling MITgcm, the file \file{SIZE.h} must be |
| 75 |
be placed where \file{genmake2} will find it. In particular for |
placed where \file{genmake2} will find it. In particular for exch2, |
| 76 |
exch2, the domain decomposition specified in \file{SIZE.h} must |
the domain decomposition specified in \file{SIZE.h} must correspond |
| 77 |
correspond with the particular configuration's topology specified in |
with the particular configuration's topology specified in |
| 78 |
\file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}. Domain |
\file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}. Domain |
| 79 |
decomposition issues particular to exch2 are addressed in Section |
decomposition issues particular to exch2 are addressed in Section |
| 80 |
\ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2} |
\ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2} |
| 81 |
and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and Multiprocessing}; a more |
and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and |
| 82 |
general background on the subject relevant to MITgcm is presented in |
Multiprocessing}; a more general background on the subject |
| 83 |
Section \ref{sect:specifying_a_decomposition} |
relevant to MITgcm is presented in Section |
| 84 |
\sectiontitle{Specifying a decomposition}.\\ |
\ref{sect:specifying_a_decomposition} |
| 85 |
|
\sectiontitle{Specifying a decomposition}. |
| 86 |
|
\end{itemize} |
| 87 |
|
|
| 88 |
At the time of this writing the following examples use exch2 and may |
At the time of this writing the following examples use exch2 and may |
| 89 |
be used for guidance: |
be used for guidance: |
| 191 |
Once the topology configuration files are created, the Fortran |
Once the topology configuration files are created, the Fortran |
| 192 |
\code{PARAMETER}s in \file{SIZE.h} must be configured to match. |
\code{PARAMETER}s in \file{SIZE.h} must be configured to match. |
| 193 |
Section \ref{sect:specifying_a_decomposition} \sectiontitle{Specifying |
Section \ref{sect:specifying_a_decomposition} \sectiontitle{Specifying |
| 194 |
a decomposition} provides a general description of domain |
a decomposition} provides a general description of domain |
| 195 |
decomposition within MITgcm and its relation to \file{SIZE.h}. The |
decomposition within MITgcm and its relation to \file{SIZE.h}. The |
| 196 |
current section specifies constraints that the exch2 package |
current section specifies constraints that the exch2 package imposes |
| 197 |
imposes and describes how to enable parallel execution with |
and describes how to enable parallel execution with MPI. |
|
MPI. \\ |
|
| 198 |
|
|
| 199 |
As in the general case, the parameters \varlink{sNx}{sNx} and |
As in the general case, the parameters \varlink{sNx}{sNx} and |
| 200 |
\varlink{sNy}{sNy} define the size of the individual tiles, and so |
\varlink{sNy}{sNy} define the size of the individual tiles, and so |
| 201 |
must be assigned the same respective values as \code{tnx} and |
must be assigned the same respective values as \code{tnx} and |
| 202 |
\code{tny} in \file{driver.m}.\\ |
\code{tny} in \file{driver.m}. |
| 203 |
|
|
| 204 |
The halo width parameters \varlink{OLx}{OLx} and \varlink{OLy}{OLy} |
The halo width parameters \varlink{OLx}{OLx} and \varlink{OLy}{OLy} |
| 205 |
have no special bearing on exch2 and may be assigned as in the general |
have no special bearing on exch2 and may be assigned as in the general |
| 206 |
case. The same holds for \varlink{Nr}{Nr}, the number of vertical |
case. The same holds for \varlink{Nr}{Nr}, the number of vertical |
| 207 |
levels in the model.\\ |
levels in the model. |
| 208 |
|
|
| 209 |
The parameters \varlink{nSx}{nSx}, \varlink{nSy}{nSy}, |
The parameters \varlink{nSx}{nSx}, \varlink{nSy}{nSy}, |
| 210 |
\varlink{nPx}{nPx}, and \varlink{nPy}{nPy} relate to the number of |
\varlink{nPx}{nPx}, and \varlink{nPy}{nPy} relate to the number of |
| 212 |
the tiles are stored in the $x$ dimension, and so |
the tiles are stored in the $x$ dimension, and so |
| 213 |
\code{\varlink{nSy}{nSy}=1} in all cases. Since the tiles as |
\code{\varlink{nSy}{nSy}=1} in all cases. Since the tiles as |
| 214 |
configured by exch2 cannot be split up accross processors without |
configured by exch2 cannot be split up accross processors without |
| 215 |
regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well. \\ |
regenerating the topology, \code{\varlink{nPy}{nPy}=1} as well. |
| 216 |
|
|
| 217 |
The number of tiles MITgcm allocates and how they are distributed |
The number of tiles MITgcm allocates and how they are distributed |
| 218 |
between processors depends on \varlink{nPx}{nPx} and |
between processors depends on \varlink{nPx}{nPx} and |
| 229 |
distribute the remaining twenty-nine tiles among five processors, you |
distribute the remaining twenty-nine tiles among five processors, you |
| 230 |
would have to run one ``dummy'' tile to make an even six tiles per |
would have to run one ``dummy'' tile to make an even six tiles per |
| 231 |
processor. Such dummy tiles are \emph{not} listed in |
processor. Such dummy tiles are \emph{not} listed in |
| 232 |
\file{blanklist.txt}.\\ |
\file{blanklist.txt}. |
|
|
|
| 233 |
|
|
| 234 |
The following is an example of \file{SIZE.h} for the twelve-tile |
The following is an example of \file{SIZE.h} for the twelve-tile |
| 235 |
configuration illustrated in figure \ref{fig:12tile} running on |
configuration illustrated in figure \ref{fig:12tile} running on one |
| 236 |
one processor: \\ |
processor: |
| 237 |
|
|
| 238 |
\begin{verbatim} |
\begin{verbatim} |
| 239 |
PARAMETER ( |
PARAMETER ( |
| 269 |
\end{verbatim} |
\end{verbatim} |
| 270 |
|
|
| 271 |
|
|
|
|
|
|
|
|
|
|
|
| 272 |
\subsubsection{Key Variables} |
\subsubsection{Key Variables} |
| 273 |
|
|
| 274 |
The descriptions of the variables are divided up into scalars, |
The descriptions of the variables are divided up into scalars, |