12 |
|
|
13 |
\section{exch2: Extended Cubed Sphere \mbox{Topology}} |
\section{exch2: Extended Cubed Sphere \mbox{Topology}} |
14 |
\label{sec:exch2} |
\label{sec:exch2} |
|
\label{sec:pkg:exch2} |
|
|
\begin{rawhtml} |
|
|
<!-- CMIREDIR:package_exch2: --> |
|
|
\end{rawhtml} |
|
15 |
|
|
16 |
|
|
17 |
\subsection{Introduction} |
\subsection{Introduction} |
44 |
\subsection{Invoking exch2} |
\subsection{Invoking exch2} |
45 |
|
|
46 |
To use exch2 with the cubed sphere, the following conditions must be |
To use exch2 with the cubed sphere, the following conditions must be |
47 |
met: |
met: \\ |
48 |
|
|
49 |
\begin{itemize} |
$\bullet$ The exch2 package is included when \file{genmake2} is run. |
50 |
\item The exch2 package is included when \file{genmake2} is run. The |
The easiest way to do this is to add the line \code{exch2} to the |
51 |
easiest way to do this is to add the line \code{exch2} to the |
\file{profile.conf} file -- see Section |
52 |
\file{profile.conf} file -- see Section \ref{sect:buildingCode} |
\ref{sect:buildingCode} \sectiontitle{Building the code} for general |
53 |
\sectiontitle{Building the code} for general details. |
details. \\ |
54 |
|
|
55 |
\item An example of \file{W2\_EXCH2\_TOPOLOGY.h} and |
$\bullet$ An example of \file{W2\_EXCH2\_TOPOLOGY.h} and |
56 |
\file{w2\_e2setup.F} must reside in a directory containing files |
\file{w2\_e2setup.F} must reside in a directory containing files |
57 |
symbolically linked by the \file{genmake2} script. The safest place |
symbolically linked by the \file{genmake2} script. The safest place to |
58 |
to put these is the directory indicated in the \code{-mods=DIR} |
put these is the directory indicated in the \code{-mods=DIR} command |
59 |
command line modifier (typically \file{../code}), or the build |
line modifier (typically \file{../code}), or the build directory. |
60 |
directory. The default versions of these files reside in |
The default versions of these files reside in \file{pkg/exch2} and |
61 |
\file{pkg/exch2} and are linked automatically if no other versions |
are linked automatically if no other versions exist elsewhere in the |
62 |
exist elsewhere in the build path, but they should be left untouched |
build path, but they should be left untouched to avoid breaking |
63 |
to avoid breaking configurations other than the one you intend to |
configurations other than the one you intend to modify.\\ |
64 |
modify. |
|
65 |
|
$\bullet$ Files containing grid parameters, named |
66 |
\item Files containing grid parameters, named \file{tile00$n$.mitgrid} |
\file{tile00$n$.mitgrid} where $n$=\code{(1:6)} (one per subdomain), |
67 |
where $n$=\code{(1:6)} (one per subdomain), must be in the working |
must be in the working directory when the MITgcm executable is run. |
68 |
directory when the MITgcm executable is run. These files are |
These files are provided in the example experiments for cubed sphere |
69 |
provided in the example experiments for cubed sphere configurations |
configurations with 32$\times$32 cube sides |
70 |
with 32$\times$32 cube sides -- please contact |
-- please contact MITgcm support if you want to generate |
71 |
\begin{rawhtml} |
files for other configurations. \\ |
72 |
<A href="mailto:mitgcm-support@dev.mitgcm.org"> |
|
73 |
\end{rawhtml} |
$\bullet$ As always when compiling MITgcm, the file \file{SIZE.h} must |
74 |
\begin{verbatim} |
be placed where \file{genmake2} will find it. In particular for |
75 |
MITgcm-support@mitgcm.org |
exch2, the domain decomposition specified in \file{SIZE.h} must |
76 |
\end{verbatim} |
correspond with the particular configuration's topology specified in |
|
\begin{rawhtml} </A> \end{rawhtml} |
|
|
if you want to generate files for other configurations. |
|
|
|
|
|
\item As always when compiling MITgcm, the file \file{SIZE.h} must be |
|
|
placed where \file{genmake2} will find it. In particular for exch2, |
|
|
the domain decomposition specified in \file{SIZE.h} must correspond |
|
|
with the particular configuration's topology specified in |
|
77 |
\file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}. Domain |
\file{W2\_EXCH2\_TOPOLOGY.h} and \file{w2\_e2setup.F}. Domain |
78 |
decomposition issues particular to exch2 are addressed in Section |
decomposition issues particular to exch2 are addressed in Section |
79 |
\ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2} |
\ref{sec:topogen} \sectiontitle{Generating Topology Files for exch2} |
80 |
and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and |
and \ref{sec:exch2mpi} \sectiontitle{exch2, SIZE.h, and Multiprocessing}; a more |
81 |
Multiprocessing}; a more general background on the subject |
general background on the subject relevant to MITgcm is presented in |
82 |
relevant to MITgcm is presented in Section |
Section \ref{sect:specifying_a_decomposition} |
83 |
\ref{sect:specifying_a_decomposition} \sectiontitle{Specifying a |
\sectiontitle{Specifying a decomposition}.\\ |
|
decomposition}. |
|
|
\end{itemize} |
|
|
|
|
|
|
|
84 |
|
|
85 |
At the time of this writing the following examples use exch2 and may |
At the time of this writing the following examples use exch2 and may |
86 |
be used for guidance: |
be used for guidance: |
215 |
The number of tiles MITgcm allocates and how they are distributed |
The number of tiles MITgcm allocates and how they are distributed |
216 |
between processors depends on \varlink{nPx}{nPx} and |
between processors depends on \varlink{nPx}{nPx} and |
217 |
\varlink{nSx}{nSx}. \varlink{nSx}{nSx} is the number of tiles per |
\varlink{nSx}{nSx}. \varlink{nSx}{nSx} is the number of tiles per |
218 |
processor and \varlink{nPx}{nPx} is the number of processors. The total |
processor and \varlink{nPx}{nPx} is the number of processors. The |
219 |
number of tiles in the topology minus those listed in |
total number of tiles in the topology minus those listed in |
220 |
\file{blanklist.txt} must equal \code{nSx*nPx}. Note that in order to |
\file{blanklist.txt} must equal \code{nSx*nPx}. Note that in order to |
221 |
obtain maximum usage from a given number of processors in some cases, |
obtain maximum usage from a given number of processors in some cases, |
222 |
this restriction might entail sharing a processor with a tile that would |
this restriction might entail sharing a processor with a tile that |
223 |
otherwise be excluded. \\ |
would otherwise be excluded because it is topographically outside of |
224 |
|
the domain and therefore in \file{blanklist.txt}. For example, |
225 |
|
suppose you have five processors and a domain decomposition of |
226 |
|
thirty-six tiles that allows you to exclude seven tiles. To evenly |
227 |
|
distribute the remaining twenty-nine tiles among five processors, you |
228 |
|
would have to run one ``dummy'' tile to make an even six tiles per |
229 |
|
processor. Such dummy tiles are \emph{not} listed in |
230 |
|
\file{blanklist.txt}.\\ |
231 |
|
|
232 |
|
|
233 |
The following is an example of \file{SIZE.h} for the twelve-tile |
The following is an example of \file{SIZE.h} for the twelve-tile |
234 |
configuration illustrated in figure \ref{fig:12tile} running on |
configuration illustrated in figure \ref{fig:12tile} running on |
308 |
|
|
309 |
The following arrays are of length \code{NTILES} and are indexed to |
The following arrays are of length \code{NTILES} and are indexed to |
310 |
the tile number, which is indicated in the diagrams with the notation |
the tile number, which is indicated in the diagrams with the notation |
311 |
\code{tn}. The indices are omitted in the descriptions. \\ |
\textsf{t}$n$. The indices are omitted in the descriptions. \\ |
312 |
|
|
313 |
The arrays \varlink{exch2\_tnx}{exch2_tnx} and |
The arrays \varlink{exch2\_tnx}{exch2_tnx} and |
314 |
\varlink{exch2\_tny}{exch2_tny} express the $x$ and $y$ dimensions of |
\varlink{exch2\_tny}{exch2_tny} express the $x$ and $y$ dimensions of |
336 |
|
|
337 |
The array \varlink{exch2\_myFace}{exch2_myFace} contains the number of |
The array \varlink{exch2\_myFace}{exch2_myFace} contains the number of |
338 |
the subdomain of each tile, in a range \code{(1:6)} in the case of the |
the subdomain of each tile, in a range \code{(1:6)} in the case of the |
339 |
standard cube topology and indicated by \textbf{\textsf{fn}} in |
standard cube topology and indicated by \textbf{\textsf{f}}$n$ in |
340 |
figures \ref{fig:12tile} and \ref{fig:24tile}. The |
figures \ref{fig:12tile} and |
341 |
\varlink{exch2\_nNeighbours}{exch2_nNeighbours} variable contains a |
\ref{fig:24tile}. \varlink{exch2\_nNeighbours}{exch2_nNeighbours} |
342 |
count of the neighboring tiles each tile has, and sets the bounds for |
contains a count of the neighboring tiles each tile has, and sets |
343 |
looping over neighboring tiles. And |
the bounds for looping over neighboring tiles. |
344 |
\varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each |
\varlink{exch2\_tProc}{exch2_tProc} holds the process rank of each |
345 |
tile, and is used in interprocess communication. \\ |
tile, and is used in interprocess communication. \\ |
346 |
|
|