| 1 |
% $Header$ |
% $Header$ |
| 2 |
|
|
| 3 |
In this chapter we describe the software architecture and |
This chapter focuses on describing the {\bf WRAPPER} environment within which |
| 4 |
implementation strategy for the MITgcm code. The first part of this |
both the core numerics and the pluggable packages operate. The description |
| 5 |
chapter discusses the MITgcm architecture at an abstract level. In the second |
presented here is intended to be a detailed exposition and contains significant |
| 6 |
part of the chapter we described practical details of the MITgcm implementation |
background material, as well as advanced details on working with the WRAPPER. |
| 7 |
and of current tools and operating system features that are employed. |
The tutorial sections of this manual (see sections |
| 8 |
|
\ref{sect:tutorials} and \ref{sect:tutorialIII}) |
| 9 |
|
contain more succinct, step-by-step instructions on running basic numerical |
| 10 |
|
experiments, of varous types, both sequentially and in parallel. For many |
| 11 |
|
projects simply starting from an example code and adapting it to suit a |
| 12 |
|
particular situation |
| 13 |
|
will be all that is required. |
| 14 |
|
The first part of this chapter discusses the MITgcm architecture at an |
| 15 |
|
abstract level. In the second part of the chapter we described practical |
| 16 |
|
details of the MITgcm implementation and of current tools and operating system |
| 17 |
|
features that are employed. |
| 18 |
|
|
| 19 |
\section{Overall architectural goals} |
\section{Overall architectural goals} |
| 20 |
|
|
| 97 |
\resizebox{!}{4.5in}{\includegraphics{part4/fit_in_wrapper.eps}} |
\resizebox{!}{4.5in}{\includegraphics{part4/fit_in_wrapper.eps}} |
| 98 |
\end{center} |
\end{center} |
| 99 |
\caption{ |
\caption{ |
| 100 |
Numerical code is written too fit within a software support |
Numerical code is written to fit within a software support |
| 101 |
infrastructure called WRAPPER. The WRAPPER is portable and |
infrastructure called WRAPPER. The WRAPPER is portable and |
| 102 |
can be specialized for a wide range of specific target hardware and |
can be specialized for a wide range of specific target hardware and |
| 103 |
programming environments, without impacting numerical code that fits |
programming environments, without impacting numerical code that fits |
| 120 |
(UMA) and non-uniform memory access (NUMA) designs. Significant work has also |
(UMA) and non-uniform memory access (NUMA) designs. Significant work has also |
| 121 |
been undertaken on x86 cluster systems, Alpha processor based clustered SMP |
been undertaken on x86 cluster systems, Alpha processor based clustered SMP |
| 122 |
systems, and on cache-coherent NUMA (CC-NUMA) systems from Silicon Graphics. |
systems, and on cache-coherent NUMA (CC-NUMA) systems from Silicon Graphics. |
| 123 |
The MITgcm code, operating within the WRAPPER, is also used routinely used on |
The MITgcm code, operating within the WRAPPER, is also routinely used on |
| 124 |
large scale MPP systems (for example T3E systems and IBM SP systems). In all |
large scale MPP systems (for example T3E systems and IBM SP systems). In all |
| 125 |
cases numerical code, operating within the WRAPPER, performs and scales very |
cases numerical code, operating within the WRAPPER, performs and scales very |
| 126 |
competitively with equivalent numerical code that has been modified to contain |
competitively with equivalent numerical code that has been modified to contain |
| 781 |
forty grid points in y. The two sub-domains in each process will be computed |
forty grid points in y. The two sub-domains in each process will be computed |
| 782 |
sequentially if they are given to a single thread within a single process. |
sequentially if they are given to a single thread within a single process. |
| 783 |
Alternatively if the code is invoked with multiple threads per process |
Alternatively if the code is invoked with multiple threads per process |
| 784 |
the two domains in y may be computed on concurrently. |
the two domains in y may be computed concurrently. |
| 785 |
\item |
\item |
| 786 |
\begin{verbatim} |
\begin{verbatim} |
| 787 |
PARAMETER ( |
PARAMETER ( |
| 817 |
WRAPPER is shown in figure \ref{fig:wrapper_startup}. |
WRAPPER is shown in figure \ref{fig:wrapper_startup}. |
| 818 |
|
|
| 819 |
\begin{figure} |
\begin{figure} |
| 820 |
|
{\footnotesize |
| 821 |
\begin{verbatim} |
\begin{verbatim} |
| 822 |
|
|
| 823 |
MAIN |
MAIN |
| 846 |
|
|
| 847 |
|
|
| 848 |
\end{verbatim} |
\end{verbatim} |
| 849 |
|
} |
| 850 |
\caption{Main stages of the WRAPPER startup procedure. |
\caption{Main stages of the WRAPPER startup procedure. |
| 851 |
This process proceeds transfer of control to application code, which |
This process proceeds transfer of control to application code, which |
| 852 |
occurs through the procedure {\em THE\_MODEL\_MAIN()}. |
occurs through the procedure {\em THE\_MODEL\_MAIN()}. |
| 1426 |
|
|
| 1427 |
WRAPPER layer. |
WRAPPER layer. |
| 1428 |
|
|
| 1429 |
|
{\footnotesize |
| 1430 |
\begin{verbatim} |
\begin{verbatim} |
| 1431 |
|
|
| 1432 |
MAIN |
MAIN |
| 1454 |
|--THE_MODEL_MAIN :: Numerical code top-level driver routine |
|--THE_MODEL_MAIN :: Numerical code top-level driver routine |
| 1455 |
|
|
| 1456 |
\end{verbatim} |
\end{verbatim} |
| 1457 |
|
} |
| 1458 |
|
|
| 1459 |
Core equations plus packages. |
Core equations plus packages. |
| 1460 |
|
|
| 1461 |
|
{\footnotesize |
| 1462 |
\begin{verbatim} |
\begin{verbatim} |
| 1463 |
C |
C |
| 1464 |
C |
C |
| 1797 |
C :: events. |
C :: events. |
| 1798 |
C |
C |
| 1799 |
\end{verbatim} |
\end{verbatim} |
| 1800 |
|
} |
| 1801 |
|
|
| 1802 |
\subsection{Measuring and Characterizing Performance} |
\subsection{Measuring and Characterizing Performance} |
| 1803 |
|
|