--- manual/s_getstarted/text/getting_started.tex 2004/03/24 20:53:12 1.22
+++ manual/s_getstarted/text/getting_started.tex 2004/10/13 05:06:25 1.26
@@ -1,4 +1,4 @@
-% $Header: /home/ubuntu/mnt/e9_copy/manual/s_getstarted/text/getting_started.tex,v 1.22 2004/03/24 20:53:12 edhill Exp $
+% $Header: /home/ubuntu/mnt/e9_copy/manual/s_getstarted/text/getting_started.tex,v 1.26 2004/10/13 05:06:25 cnh Exp $
% $Name: $
%\section{Getting started}
@@ -324,7 +324,7 @@
\end{itemize}
-\section{Example experiments}
+\section[MITgcm Example Experiments]{Example experiments}
\label{sect:modelExamples}
%% a set of twenty-four pre-configured numerical experiments
@@ -484,7 +484,7 @@
Once you have chosen the example you want to run, you are ready to
compile the code.
-\section{Building the code}
+\section[Building MITgcm]{Building the code}
\label{sect:buildingCode}
To compile the code, we use the {\em make} program. This uses a file
@@ -666,18 +666,17 @@
\end{verbatim}
-
-\subsection{Using \textit{genmake2}}
+\subsection{Using \texttt{genmake2}}
\label{sect:genmake}
To compile the code, first use the program \texttt{genmake2} (located
-in the \textit{tools} directory) to generate a Makefile.
+in the \texttt{tools} directory) to generate a Makefile.
\texttt{genmake2} is a shell script written to work with all
``sh''--compatible shells including bash v1, bash v2, and Bourne.
Internally, \texttt{genmake2} determines the locations of needed
files, the compiler, compiler options, libraries, and Unix tools. It
-relies upon a number of ``optfiles'' located in the {\em
- tools/build\_options} directory.
+relies upon a number of ``optfiles'' located in the
+\texttt{tools/build\_options} directory.
The purpose of the optfiles is to provide all the compilation options
for particular ``platforms'' (where ``platform'' roughly means the
@@ -828,6 +827,10 @@
``-standarddirs'' option)
\end{itemize}
+\item[\texttt{--mpi}] This option enables certain MPI features (using
+ CPP \texttt{\#define}s) within the code and is necessary for MPI
+ builds (see Section \ref{sect:mpi-build}).
+
\item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
soft-links and other bugs common with the \texttt{make} versions
provided by commercial Unix vendors, GNU \texttt{make} (sometimes
@@ -840,7 +843,7 @@
a Bourne, POSIX, or compatible) shell. The syntax in these
circumstances is:
\begin{center}
- \texttt{/bin/sh genmake2 -bash=/bin/sh [...options...]}
+ \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
\end{center}
where \texttt{/bin/sh} can be replaced with the full path and name
of the desired shell.
@@ -848,13 +851,111 @@
\end{description}
+\subsection{Building with MPI}
+\label{sect:mpi-build}
+
+Building MITgcm to use MPI libraries can be complicated due to the
+variety of different MPI implementations available, their dependencies
+or interactions with different compilers, and their often ad-hoc
+locations within file systems. For these reasons, its generally a
+good idea to start by finding and reading the documentation for your
+machine(s) and, if necessary, seeking help from your local systems
+administrator.
+
+The steps for building MITgcm with MPI support are:
+\begin{enumerate}
+
+\item Determine the locations of your MPI-enabled compiler and/or MPI
+ libraries and put them into an options file as described in Section
+ \ref{sect:genmake}. One can start with one of the examples in:
+ \begin{rawhtml}
+ \end{rawhtml}
+ \begin{center}
+ \texttt{MITgcm/tools/build\_options/}
+ \end{center}
+ \begin{rawhtml} \end{rawhtml}
+ such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
+ \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
+ hand. You may need help from your user guide or local systems
+ administrator to determine the exact location of the MPI libraries.
+ If libraries are not installed, MPI implementations and related
+ tools are available including:
+ \begin{itemize}
+ \item \begin{rawhtml}
+ \end{rawhtml}
+ MPICH
+ \begin{rawhtml} \end{rawhtml}
+
+ \item \begin{rawhtml}
+ \end{rawhtml}
+ LAM/MPI
+ \begin{rawhtml} \end{rawhtml}
+
+ \item \begin{rawhtml}
+ \end{rawhtml}
+ MPIexec
+ \begin{rawhtml} \end{rawhtml}
+ \end{itemize}
+
+\item Build the code with the \texttt{genmake2} \texttt{-mpi} option
+ (see Section \ref{sect:genmake}) using commands such as:
+{\footnotesize \begin{verbatim}
+ % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
+ % make depend
+ % make
+\end{verbatim} }
+
+\item Run the code with the appropriate MPI ``run'' or ``exec''
+ program provided with your particular implementation of MPI.
+ Typical MPI packages such as MPICH will use something like:
+\begin{verbatim}
+ % mpirun -np 4 -machinefile mf ./mitgcmuv
+\end{verbatim}
+ Sightly more complicated scripts may be needed for many machines
+ since execution of the code may be controlled by both the MPI
+ library and a job scheduling and queueing system such as PBS,
+ LoadLeveller, Condor, or any of a number of similar tools. A few
+ example scripts (those used for our \begin{rawhtml} \end{rawhtml}regular
+ verification runs\begin{rawhtml} \end{rawhtml}) are available
+ at:
+ \begin{rawhtml}
+ \end{rawhtml}
+ {\footnotesize \tt
+ http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm\_contrib/test\_scripts/ }
+ \begin{rawhtml} \end{rawhtml}
+
+\end{enumerate}
+
+An example of the above process on the MITgcm cluster (``cg01'') using
+the GNU g77 compiler and the mpich MPI library is:
-\section{Running the model}
+{\footnotesize \begin{verbatim}
+ % cd MITgcm/verification/exp5
+ % mkdir build
+ % cd build
+ % ../../../tools/genmake2 -mpi -mods=../code \
+ -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
+ % make depend
+ % make
+ % cd ../input
+ % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
+ -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
+\end{verbatim} }
+
+
+
+\section[Running MITgcm]{Running the model in prognostic mode}
\label{sect:runModel}
-If compilation finished succesfuully (section \ref{sect:buildModel})
-then an executable called {\em mitgcmuv} will now exist in the local
-directory.
+If compilation finished succesfuully (section \ref{sect:buildingCode})
+then an executable called \texttt{mitgcmuv} will now exist in the
+local directory.
To run the model as a single process (ie. not in parallel) simply
type:
@@ -958,7 +1059,7 @@
>> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
\end{verbatim}
-\section{Doing it yourself: customizing the code}
+\section[Customizing MITgcm]{Doing it yourself: customizing the code}
When you are ready to run the model in the configuration you want, the
easiest thing is to use and adapt the setup of the case studies