1 |
% $Header$ |
% $Header$ |
2 |
|
|
3 |
This chapter focuses on describing the {\bf WRAPPER} environment within which |
This chapter focuses on describing the {\bf WRAPPER} environment |
4 |
both the core numerics and the pluggable packages operate. The description |
within which both the core numerics and the pluggable packages |
5 |
presented here is intended to be a detailed exposition and contains significant |
operate. The description presented here is intended to be a detailed |
6 |
background material, as well as advanced details on working with the WRAPPER. |
exposition and contains significant background material, as well as |
7 |
The tutorial sections of this manual (see sections |
advanced details on working with the WRAPPER. The tutorial sections |
8 |
\ref{sect:tutorials} and \ref{sect:tutorialIII}) |
of this manual (see sections \ref{sect:tutorials} and |
9 |
contain more succinct, step-by-step instructions on running basic numerical |
\ref{sect:tutorialIII}) contain more succinct, step-by-step |
10 |
experiments, of varous types, both sequentially and in parallel. For many |
instructions on running basic numerical experiments, of varous types, |
11 |
projects simply starting from an example code and adapting it to suit a |
both sequentially and in parallel. For many projects simply starting |
12 |
particular situation |
from an example code and adapting it to suit a particular situation |
13 |
will be all that is required. |
will be all that is required. The first part of this chapter |
14 |
The first part of this chapter discusses the MITgcm architecture at an |
discusses the MITgcm architecture at an abstract level. In the second |
15 |
abstract level. In the second part of the chapter we described practical |
part of the chapter we described practical details of the MITgcm |
16 |
details of the MITgcm implementation and of current tools and operating system |
implementation and of current tools and operating system features that |
17 |
features that are employed. |
are employed. |
18 |
|
|
19 |
\section{Overall architectural goals} |
\section{Overall architectural goals} |
20 |
\begin{rawhtml} |
\begin{rawhtml} |
25 |
three-fold |
three-fold |
26 |
|
|
27 |
\begin{itemize} |
\begin{itemize} |
28 |
\item We wish to be able to study a very broad range |
\item We wish to be able to study a very broad range of interesting |
29 |
of interesting and challenging rotating fluids problems. |
and challenging rotating fluids problems. |
30 |
\item We wish the model code to be readily targeted to |
\item We wish the model code to be readily targeted to a wide range of |
31 |
a wide range of platforms |
platforms |
32 |
\item On any given platform we would like to be |
\item On any given platform we would like to be able to achieve |
33 |
able to achieve performance comparable to an implementation |
performance comparable to an implementation developed and |
34 |
developed and specialized specifically for that platform. |
specialized specifically for that platform. |
35 |
\end{itemize} |
\end{itemize} |
36 |
|
|
37 |
These points are summarized in figure \ref{fig:mitgcm_architecture_goals} |
These points are summarized in figure |
38 |
which conveys the goals of the MITgcm design. The goals lead to |
\ref{fig:mitgcm_architecture_goals} which conveys the goals of the |
39 |
a software architecture which at the high-level can be viewed as consisting |
MITgcm design. The goals lead to a software architecture which at the |
40 |
of |
high-level can be viewed as consisting of |
41 |
|
|
42 |
\begin{enumerate} |
\begin{enumerate} |
43 |
\item A core set of numerical and support code. This is discussed in |
\item A core set of numerical and support code. This is discussed in |
71 |
\begin{center} |
\begin{center} |
72 |
\resizebox{!}{2.5in}{\includegraphics{part4/mitgcm_goals.eps}} |
\resizebox{!}{2.5in}{\includegraphics{part4/mitgcm_goals.eps}} |
73 |
\end{center} |
\end{center} |
74 |
\caption{ |
\caption{ The MITgcm architecture is designed to allow simulation of a |
75 |
The MITgcm architecture is designed to allow simulation of a wide |
wide range of physical problems on a wide range of hardware. The |
76 |
range of physical problems on a wide range of hardware. The computational |
computational resource requirements of the applications targeted |
77 |
resource requirements of the applications targeted range from around |
range from around $10^7$ bytes ($\approx 10$ megabytes) of memory to |
78 |
$10^7$ bytes ( $\approx 10$ megabytes ) of memory to $10^{11}$ bytes |
$10^{11}$ bytes ($\approx 100$ gigabytes). Arithmetic operation |
79 |
( $\approx 100$ gigabytes). Arithmetic operation counts for the applications of |
counts for the applications of interest range from $10^{9}$ floating |
80 |
interest range from $10^{9}$ floating point operations to more than $10^{17}$ |
point operations to more than $10^{17}$ floating point operations.} |
|
floating point operations.} |
|
81 |
\label{fig:mitgcm_architecture_goals} |
\label{fig:mitgcm_architecture_goals} |
82 |
\end{figure} |
\end{figure} |
83 |
|
|
86 |
<!-- CMIREDIR:wrapper: --> |
<!-- CMIREDIR:wrapper: --> |
87 |
\end{rawhtml} |
\end{rawhtml} |
88 |
|
|
89 |
A significant element of the software architecture utilized in |
A significant element of the software architecture utilized in MITgcm |
90 |
MITgcm is a software superstructure and substructure collectively |
is a software superstructure and substructure collectively called the |
91 |
called the WRAPPER (Wrappable Application Parallel Programming |
WRAPPER (Wrappable Application Parallel Programming Environment |
92 |
Environment Resource). All numerical and support code in MITgcm is written |
Resource). All numerical and support code in MITgcm is written to |
93 |
to ``fit'' within the WRAPPER infrastructure. Writing code to ``fit'' within |
``fit'' within the WRAPPER infrastructure. Writing code to ``fit'' |
94 |
the WRAPPER means that coding has to follow certain, relatively |
within the WRAPPER means that coding has to follow certain, relatively |
95 |
straightforward, rules and conventions (these are discussed further in |
straightforward, rules and conventions (these are discussed further in |
96 |
section \ref{sect:specifying_a_decomposition}). |
section \ref{sect:specifying_a_decomposition}). |
97 |
|
|
98 |
The approach taken by the WRAPPER is illustrated in figure |
The approach taken by the WRAPPER is illustrated in figure |
99 |
\ref{fig:fit_in_wrapper} which shows how the WRAPPER serves to insulate code |
\ref{fig:fit_in_wrapper} which shows how the WRAPPER serves to |
100 |
that fits within it from architectural differences between hardware platforms |
insulate code that fits within it from architectural differences |
101 |
and operating systems. This allows numerical code to be easily retargetted. |
between hardware platforms and operating systems. This allows |
102 |
|
numerical code to be easily retargetted. |
103 |
|
|
104 |
|
|
105 |
\begin{figure} |
\begin{figure} |
416 |
\subsection{Distributed memory communication} |
\subsection{Distributed memory communication} |
417 |
\label{sect:distributed_memory_communication} |
\label{sect:distributed_memory_communication} |
418 |
Many parallel systems are not constructed in a way where it is |
Many parallel systems are not constructed in a way where it is |
419 |
possible or practical for an application to use shared memory |
possible or practical for an application to use shared memory for |
420 |
for communication. For example cluster systems consist of individual computers |
communication. For example cluster systems consist of individual |
421 |
connected by a fast network. On such systems there is no notion of shared memory |
computers connected by a fast network. On such systems there is no |
422 |
at the system level. For this sort of system the WRAPPER provides support |
notion of shared memory at the system level. For this sort of system |
423 |
for communication based on a bespoke communication library |
the WRAPPER provides support for communication based on a bespoke |
424 |
(see figure \ref{fig:comm_msg}). The default communication library used is MPI |
communication library (see figure \ref{fig:comm_msg}). The default |
425 |
\ref{mpi}. However, it is relatively straightforward to implement bindings to |
communication library used is MPI \cite{MPI-std-20}. However, it is |
426 |
optimized platform specific communication libraries. For example the work |
relatively straightforward to implement bindings to optimized platform |
427 |
described in \ref{hoe-hill:99} substituted standard MPI communication for a |
specific communication libraries. For example the work described in |
428 |
highly optimized library. |
\ref{hoe-hill:99} substituted standard MPI communication for a highly |
429 |
|
optimized library. |
430 |
|
|
431 |
\subsection{Communication primitives} |
\subsection{Communication primitives} |
432 |
\label{sect:communication_primitives} |
\label{sect:communication_primitives} |