977 |
routines must be activated at compile time. Currently MPI libraries are |
routines must be activated at compile time. Currently MPI libraries are |
978 |
invoked by |
invoked by |
979 |
specifying the appropriate options file with the |
specifying the appropriate options file with the |
980 |
\begin{verbatim}-of\end{verbatim} flag when running the {\em genmake2} |
{\tt-of} flag when running the {\em genmake2} |
981 |
script, which generates the Makefile for compiling and linking MITgcm. |
script, which generates the Makefile for compiling and linking MITgcm. |
982 |
(Previously this was done by setting the {\em ALLOW\_USE\_MPI} and |
(Previously this was done by setting the {\em ALLOW\_USE\_MPI} and |
983 |
{\em ALWAYS\_USE\_MPI} flags in the {\em CPP\_EEOPTIONS.h} file.) More |
{\em ALWAYS\_USE\_MPI} flags in the {\em CPP\_EEOPTIONS.h} file.) More |
984 |
detailed information about the use of {\em genmake2} for specifying |
detailed information about the use of {\em genmake2} for specifying |
985 |
local compiler flags is located in section 3 ??\\ |
local compiler flags is located in section \ref{sect:genmake}.\\ |
986 |
|
|
987 |
|
|
988 |
\fbox{ |
\fbox{ |
989 |
\begin{minipage}{4.75in} |
\begin{minipage}{4.75in} |
990 |
|
Directory: {\em tools/build\_options}\\ |
991 |
File: {\em tools/genmake2} |
File: {\em tools/genmake2} |
992 |
\end{minipage} |
\end{minipage} |
993 |
} \\ |
} \\ |
1006 |
in the file {\em SIZE.h}. The parameter {\em mf} specifies that a text file |
in the file {\em SIZE.h}. The parameter {\em mf} specifies that a text file |
1007 |
called ``mf'' will be read to get a list of processor names on |
called ``mf'' will be read to get a list of processor names on |
1008 |
which the sixty-four processes will execute. The syntax of this file |
which the sixty-four processes will execute. The syntax of this file |
1009 |
is specified by the MPI distribution |
is specified by the MPI distribution. |
1010 |
\\ |
\\ |
1011 |
|
|
1012 |
\fbox{ |
\fbox{ |
1057 |
Allocation of processes to tiles in controlled by the routine |
Allocation of processes to tiles in controlled by the routine |
1058 |
{\em INI\_PROCS()}. For each process this routine sets |
{\em INI\_PROCS()}. For each process this routine sets |
1059 |
the variables {\em myXGlobalLo} and {\em myYGlobalLo}. |
the variables {\em myXGlobalLo} and {\em myYGlobalLo}. |
1060 |
These variables specify (in index space) the coordinate |
These variables specify in index space the coordinates |
1061 |
of the southern most and western most corner of the |
of the southernmost and westernmost corner of the |
1062 |
southern most and western most tile owned by this process. |
southernmost and westernmost tile owned by this process. |
1063 |
The variables {\em pidW}, {\em pidE}, {\em pidS} and {\em pidN} |
The variables {\em pidW}, {\em pidE}, {\em pidS} and {\em pidN} |
1064 |
are also set in this routine. These are used to identify |
are also set in this routine. These are used to identify |
1065 |
processes holding tiles to the west, east, south and north |
processes holding tiles to the west, east, south and north |
1068 |
communication routines. The above does not hold when the |
communication routines. The above does not hold when the |
1069 |
exch2 package is used -- exch2 sets its own parameters to |
exch2 package is used -- exch2 sets its own parameters to |
1070 |
specify the global indices of tiles and their relationships |
specify the global indices of tiles and their relationships |
1071 |
to each other. |
to each other. See the documentation on the exch2 package |
1072 |
|
(\ref{sec:exch2}) for |
1073 |
|
details. |
1074 |
\\ |
\\ |
1075 |
|
|
1076 |
\fbox{ |
\fbox{ |
1096 |
describes the information that is held and used. |
describes the information that is held and used. |
1097 |
|
|
1098 |
\begin{enumerate} |
\begin{enumerate} |
1099 |
\item {\bf Tile-tile connectivity information} For each tile the WRAPPER |
\item {\bf Tile-tile connectivity information} |
1100 |
sets a flag that sets the tile number to the north, south, east and |
For each tile the WRAPPER |
1101 |
|
sets a flag that sets the tile number to the north, |
1102 |
|
south, east and |
1103 |
west of that tile. This number is unique over all tiles in a |
west of that tile. This number is unique over all tiles in a |
1104 |
configuration. The number is held in the variables {\em tileNo} |
configuration. Except when using the cubed sphere and the exch2 package, |
1105 |
|
the number is held in the variables {\em tileNo} |
1106 |
( this holds the tiles own number), {\em tileNoN}, {\em tileNoS}, |
( this holds the tiles own number), {\em tileNoN}, {\em tileNoS}, |
1107 |
{\em tileNoE} and {\em tileNoW}. A parameter is also stored with each tile |
{\em tileNoE} and {\em tileNoW}. A parameter is also stored with each tile |
1108 |
that specifies the type of communication that is used between tiles. |
that specifies the type of communication that is used between tiles. |
1125 |
(see figure \ref{fig:communication_primitives}). The routine |
(see figure \ref{fig:communication_primitives}). The routine |
1126 |
{\em ini\_communication\_patterns()} is responsible for setting the |
{\em ini\_communication\_patterns()} is responsible for setting the |
1127 |
communication mode values for each tile. |
communication mode values for each tile. |
1128 |
\\ |
|
1129 |
|
When using the cubed sphere configuration with the exch2 package, the |
1130 |
|
relationships between tiles and their communication methods are set |
1131 |
|
by the package in other variables. See the exch2 package documentation |
1132 |
|
(\ref{sec:exch2} for details. |
1133 |
|
|
1134 |
|
|
1135 |
|
|
1136 |
\fbox{ |
\fbox{ |
1137 |
\begin{minipage}{4.75in} |
\begin{minipage}{4.75in} |
1267 |
the cube-sphere grid. In this class of grid a rotation may be required |
the cube-sphere grid. In this class of grid a rotation may be required |
1268 |
between tiles. Aligning the coordinate requiring rotation with the |
between tiles. Aligning the coordinate requiring rotation with the |
1269 |
tile decomposition, allows the coordinate transformation to |
tile decomposition, allows the coordinate transformation to |
1270 |
be embedded within a custom form of the \_EXCH primitive. |
be embedded within a custom form of the \_EXCH primitive. In these |
1271 |
|
cases \_EXCH is mapped to exch2 routines, as detailed in the exch2 |
1272 |
|
package documentation \ref{sec:exch2}. |
1273 |
|
|
1274 |
\item {\bf Reverse Mode} |
\item {\bf Reverse Mode} |
1275 |
The communication primitives \_EXCH and \_GSUM both employ |
The communication primitives \_EXCH and \_GSUM both employ |
1286 |
is set to the value {\em REVERSE\_SIMULATION}. This signifies |
is set to the value {\em REVERSE\_SIMULATION}. This signifies |
1287 |
ti the low-level routines that the adjoint forms of the |
ti the low-level routines that the adjoint forms of the |
1288 |
appropriate communication operation should be performed. |
appropriate communication operation should be performed. |
1289 |
|
|
1290 |
\item {\bf MAX\_NO\_THREADS} |
\item {\bf MAX\_NO\_THREADS} |
1291 |
The variable {\em MAX\_NO\_THREADS} is used to indicate the |
The variable {\em MAX\_NO\_THREADS} is used to indicate the |
1292 |
maximum number of OS threads that a code will use. This |
maximum number of OS threads that a code will use. This |
1391 |
This is done to allow a large number of variations on the exchange |
This is done to allow a large number of variations on the exchange |
1392 |
process to be maintained. One set of variations supports the |
process to be maintained. One set of variations supports the |
1393 |
cube sphere grid. Support for a cube sphere grid in MITgcm is based |
cube sphere grid. Support for a cube sphere grid in MITgcm is based |
1394 |
on having each face of the cube as a separate tile (or tiles). |
on having each face of the cube as a separate tile or tiles. |
1395 |
The exchange routines are then able to absorb much of the |
The exchange routines are then able to absorb much of the |
1396 |
detailed rotation and reorientation required when moving around the |
detailed rotation and reorientation required when moving around the |
1397 |
cube grid. The set of {\em \_EXCH} routines that contain the |
cube grid. The set of {\em \_EXCH} routines that contain the |