| 12 |
|
|
| 13 |
\section{exch2: Extended Cubed Sphere \mbox{Topology}} |
\section{exch2: Extended Cubed Sphere \mbox{Topology}} |
| 14 |
\label{sec:exch2} |
\label{sec:exch2} |
|
\label{sec:pkg:exch2} |
|
|
\begin{rawhtml} |
|
|
<!-- CMIREDIR:package_exch2: --> |
|
|
\end{rawhtml} |
|
| 15 |
|
|
| 16 |
|
|
| 17 |
\subsection{Introduction} |
\subsection{Introduction} |
| 44 |
\subsection{Invoking exch2} |
\subsection{Invoking exch2} |
| 45 |
|
|
| 46 |
To use exch2 with the cubed sphere, the following conditions must be |
To use exch2 with the cubed sphere, the following conditions must be |
| 47 |
met: |
met: \\ |
| 48 |
|
|
| 49 |
\begin{itemize} |
$\bullet$ The exch2 package is included when \file{genmake2} is run. |
| 50 |
\item The exch2 package is included when \file{genmake2} is run. The |
The easiest way to do this is to add the line \code{exch2} to the |
| 51 |
easiest way to do this is to add the line \code{exch2} to the |
\file{profile.conf} file -- see Section |
| 52 |
\file{profile.conf} file -- see Section \ref{sect:buildingCode} |
\ref{sect:buildingCode} \sectiontitle{Building the code} for general |
| 53 |
\sectiontitle{Building the code} for general details. |
details. \\ |
| 54 |
|
|
| 55 |
\item An example of \file{W2\_EXCH2\_TOPOLOGY.h} and |
$\bullet$ An example of \file{W2\_EXCH2\_TOPOLOGY.h} and |
| 56 |
\file{w2\_e2setup.F} must reside in a directory containing files |
\file{w2\_e2setup.F} must reside in a directory containing files |
| 57 |
symbolically linked by the \file{genmake2} script. The safest place |
symbolically linked by the \file{genmake2} script. The safest place to |
| 58 |
to put these is the directory indicated in the \code{-mods=DIR} |
put these is the directory indicated in the \code{-mods=DIR} command |
| 59 |
command line modifier (typically \file{../code}), or the build |
line modifier (typically \file{../code}), or the build directory. |
| 60 |
directory. The default versions of these files reside in |
The default versions of these files reside in \file{pkg/exch2} and |
| 61 |
\file{pkg/exch2} and are linked automatically if no other versions |
are linked automatically if no other versions exist elsewhere in the |
| 62 |
exist elsewhere in the build path, but they should be left untouched |
build path, but they should be left untouched to avoid breaking |
| 63 |
to avoid breaking configurations other than the one you intend to |
configurations other than the one you intend to modify.\\ |
| 64 |
modify. |
|
| 65 |
|
$\bullet$ Files containing grid parameters, named |
| 66 |
\item Files containing grid parameters, named \file{tile00$n$.mitgrid} |
\file{tile00$n$.mitgrid} where $n$=\code{(1:6)} (one per subdomain), |
| 67 |
where $n$=\code{(1:6)} (one per subdomain), must be in the working |
must be in the working directory when the MITgcm executable is run. |
| 68 |
directory when the MITgcm executable is run. These files are |
These files are provided in the example experiments for cubed sphere |
| 69 |
provided in the example experiments for cubed sphere configurations |
configurations with 32$\times$32 cube sides |
| 70 |
with 32$\times$32 cube sides -- please contact |
-- please contact MITgcm support if you want to generate |
| 71 |
\begin{rawhtml} |
files for other configurations. \\ |
| 72 |
<A href="mailto:mitgcm-support@dev.mitgcm.org"> |
|
| 73 |
\end{rawhtml} |
$\bullet$ As always when compiling MITgcm, the file \file{SIZE.h} must |
| 74 |
\begin{verbatim} |
be placed where \file{genmake2} will find it. In particular for |
| 75 |
MITgcm-support@mitgcm.org |
exch2, the domain decomposition specified in \file{SIZE.h} must |
| 76 |
\end{verbatim} |
correspond with the particular configuration's topology specified in |
|
\begin{rawhtml} </A> \end{rawhtml} |
|
|
if you want to generate files for other configurations. |
|
|
|
|
|
\item As always when compiling MITgcm, the file \file{SIZE.h} must be |
|
|
placed where \file{genmake2} will find it. In particular for exch2, |
|
|
the domain decomposition specified in \file{SIZE.h} must correspond |
|
|
with the particular configuration's topology specified in |
|
| 77 |
\file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}. Domain |
\file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}. Domain |
| 78 |
decomposition issues particular to exch2 are addressed in Section |
decomposition issues particular to exch2 are addressed in Section |
| 79 |
\ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2} |
\ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2} |
| 80 |
and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and |
and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and Multiprocessing}; a more |
| 81 |
Multiprocessing}; a more general background on the subject |
general background on the subject relevant to MITgcm is presented in |
| 82 |
relevant to MITgcm is presented in Section |
Section \ref{sect:specifying_a_decomposition} |
| 83 |
\ref{sect:specifying_a_decomposition} \sectiontitle{Specifying a |
\sectiontitle{Specifying a decomposition}.\\ |
|
decomposition}. |
|
|
\end{itemize} |
|
|
|
|
|
|
|
| 84 |
|
|
| 85 |
At the time of this writing the following examples use exch2 and may |
At the time of this writing the following examples use exch2 and may |
| 86 |
be used for guidance: |
be used for guidance: |
| 215 |
The number of tiles MITgcm allocates and how they are distributed |
The number of tiles MITgcm allocates and how they are distributed |
| 216 |
between processors depends on \varlink{nPx}{nPx} and |
between processors depends on \varlink{nPx}{nPx} and |
| 217 |
\varlink{nSx}{nSx}. \varlink{nSx}{nSx} is the number of tiles per |
\varlink{nSx}{nSx}. \varlink{nSx}{nSx} is the number of tiles per |
| 218 |
processor and \varlink{nPx}{nPx} is the number of processors. The total |
processor and \varlink{nPx}{nPx} is the number of processors. The |
| 219 |
number of tiles in the topology minus those listed in |
total number of tiles in the topology minus those listed in |
| 220 |
\file{blanklist.txt} must equal \code{nSx*nPx}. Note that in order to |
\file{blanklist.txt} must equal \code{nSx*nPx}. Note that in order to |
| 221 |
obtain maximum usage from a given number of processors in some cases, |
obtain maximum usage from a given number of processors in some cases, |
| 222 |
this restriction might entail sharing a processor with a tile that would |
this restriction might entail sharing a processor with a tile that |
| 223 |
otherwise be excluded. \\ |
would otherwise be excluded because it is topographically outside of |
| 224 |
|
the domain and therefore in \file{blanklist.txt}. For example, |
| 225 |
|
suppose you have five processors and a domain decomposition of |
| 226 |
|
thirty-six tiles that allows you to exclude seven tiles. To evenly |
| 227 |
|
distribute the remaining twenty-nine tiles among five processors, you |
| 228 |
|
would have to run one ``dummy'' tile to make an even six tiles per |
| 229 |
|
processor. Such dummy tiles are \emph{not} listed in |
| 230 |
|
\file{blanklist.txt}.\\ |
| 231 |
|
|
| 232 |
|
|
| 233 |
The following is an example of \file{SIZE.h} for the twelve-tile |
The following is an example of \file{SIZE.h} for the twelve-tile |
| 234 |
configuration illustrated in figure \ref{fig:12tile} running on |
configuration illustrated in figure \ref{fig:12tile} running on |
| 308 |
|
|
| 309 |
The following arrays are of length \code{NTILES} and are indexed to |
The following arrays are of length \code{NTILES} and are indexed to |
| 310 |
the tile number, which is indicated in the diagrams with the notation |
the tile number, which is indicated in the diagrams with the notation |
| 311 |
\code{tn}. The indices are omitted in the descriptions. \\ |
\textsf{t}$n$. The indices are omitted in the descriptions. \\ |
| 312 |
|
|
| 313 |
The arrays \varlink{exch2\_tnx}{exch2_tnx} and |
The arrays \varlink{exch2\_tnx}{exch2_tnx} and |
| 314 |
\varlink{exch2\_tny}{exch2_tny} express the $x$ and $y$ dimensions of |
\varlink{exch2\_tny}{exch2_tny} express the $x$ and $y$ dimensions of |
| 336 |
|
|
| 337 |
The array \varlink{exch2\_myFace}{exch2_myFace} contains the number of |
The array \varlink{exch2\_myFace}{exch2_myFace} contains the number of |
| 338 |
the subdomain of each tile, in a range \code{(1:6)} in the case of the |
the subdomain of each tile, in a range \code{(1:6)} in the case of the |
| 339 |
standard cube topology and indicated by \textbf{\textsf{fn}} in |
standard cube topology and indicated by \textbf{\textsf{f}}$n$ in |
| 340 |
figures \ref{fig:12tile} and \ref{fig:24tile}. The |
figures \ref{fig:12tile} and |
| 341 |
\varlink{exch2\_nNeighbours}{exch2_nNeighbours} variable contains a |
\ref{fig:24tile}. \varlink{exch2\_nNeighbours}{exch2_nNeighbours} |
| 342 |
count of the neighboring tiles each tile has, and sets the bounds for |
contains a count of the neighboring tiles each tile has, and sets |
| 343 |
looping over neighboring tiles. And |
the bounds for looping over neighboring tiles. |
| 344 |
\varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each |
\varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each |
| 345 |
tile, and is used in interprocess communication. \\ |
tile, and is used in interprocess communication. \\ |
| 346 |
|
|