--- manual/s_getstarted/text/getting_started.tex 2004/02/16 02:42:10 1.20
+++ manual/s_getstarted/text/getting_started.tex 2004/06/04 15:32:38 1.25
@@ -1,4 +1,4 @@
-% $Header: /home/ubuntu/mnt/e9_copy/manual/s_getstarted/text/getting_started.tex,v 1.20 2004/02/16 02:42:10 edhill Exp $
+% $Header: /home/ubuntu/mnt/e9_copy/manual/s_getstarted/text/getting_started.tex,v 1.25 2004/06/04 15:32:38 edhill Exp $
% $Name: $
%\section{Getting started}
@@ -666,18 +666,17 @@
\end{verbatim}
-
-\subsection{Using \textit{genmake2}}
+\subsection{Using \texttt{genmake2}}
\label{sect:genmake}
To compile the code, first use the program \texttt{genmake2} (located
-in the \textit{tools} directory) to generate a Makefile.
+in the \texttt{tools} directory) to generate a Makefile.
\texttt{genmake2} is a shell script written to work with all
``sh''--compatible shells including bash v1, bash v2, and Bourne.
Internally, \texttt{genmake2} determines the locations of needed
files, the compiler, compiler options, libraries, and Unix tools. It
-relies upon a number of ``optfiles'' located in the {\em
- tools/build\_options} directory.
+relies upon a number of ``optfiles'' located in the
+\texttt{tools/build\_options} directory.
The purpose of the optfiles is to provide all the compilation options
for particular ``platforms'' (where ``platform'' roughly means the
@@ -772,6 +771,21 @@
the user's path. When these three items have been identified,
genmake2 will try to find an optfile that has a matching name.
+\item[\texttt{--pdefault='PKG1 PKG2 PKG3 ...'}] specifies the default
+ set of packages to be used. The normal order of precedence for
+ packages is as follows:
+ \begin{enumerate}
+ \item If available, the command line (\texttt{--pdefault}) settings
+ over-rule any others.
+
+ \item Next, \texttt{genmake2} will look for a file named
+ ``\texttt{packages.conf}'' in the local directory or in any of the
+ directories specified with the \texttt{--mods} option.
+
+ \item Finally, if neither of the above are available,
+ \texttt{genmake2} will use the \texttt{/pkg/pkg\_default} file.
+ \end{enumerate}
+
\item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
used for packages.
@@ -784,12 +798,6 @@
assumed that the two packages are compatible and will function
either with or without each other.
-\item[\texttt{--pdefault='PKG1 PKG2 PKG3 ...'}] specifies the default
- set of packages to be used.
-
- If not set, the default package list will be read from {\em
- pkg/pkg\_default}
-
\item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
automatic differentiation options file to be used. The file is
analogous to the ``optfile'' defined above but it specifies
@@ -819,22 +827,135 @@
``-standarddirs'' option)
\end{itemize}
+\item[\texttt{--mpi}] This option enables certain MPI features (using
+ CPP \texttt{\#define}s) within the code and is necessary for MPI
+ builds (see Section \ref{sect:mpi-build}).
+
\item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
soft-links and other bugs common with the \texttt{make} versions
provided by commercial Unix vendors, GNU \texttt{make} (sometimes
called \texttt{gmake}) should be preferred. This option provides a
means for specifying the make executable to be used.
+
+\item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
+ machines, the ``bash'' shell is unavailable. To run on these
+ systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
+ a Bourne, POSIX, or compatible) shell. The syntax in these
+ circumstances is:
+ \begin{center}
+ \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
+ \end{center}
+ where \texttt{/bin/sh} can be replaced with the full path and name
+ of the desired shell.
\end{description}
+\subsection{Building with MPI}
+\label{sect:mpi-build}
+
+Building MITgcm to use MPI libraries can be complicated due to the
+variety of different MPI implementations available, their dependencies
+or interactions with different compilers, and their often ad-hoc
+locations within file systems. For these reasons, its generally a
+good idea to start by finding and reading the documentation for your
+machine(s) and, if necessary, seeking help from your local systems
+administrator.
+
+The steps for building MITgcm with MPI support are:
+\begin{enumerate}
+
+\item Determine the locations of your MPI-enabled compiler and/or MPI
+ libraries and put them into an options file as described in Section
+ \ref{sect:genmake}. One can start with one of the examples in:
+ \begin{rawhtml}
+ \end{rawhtml}
+ \begin{center}
+ \texttt{MITgcm/tools/build\_options/}
+ \end{center}
+ \begin{rawhtml} \end{rawhtml}
+ such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
+ \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
+ hand. You may need help from your user guide or local systems
+ administrator to determine the exact location of the MPI libraries.
+ If libraries are not installed, MPI implementations and related
+ tools are available including:
+ \begin{itemize}
+ \item \begin{rawhtml}
+ \end{rawhtml}
+ MPICH
+ \begin{rawhtml} \end{rawhtml}
+
+ \item \begin{rawhtml}
+ \end{rawhtml}
+ LAM/MPI
+ \begin{rawhtml} \end{rawhtml}
+
+ \item \begin{rawhtml}
+ \end{rawhtml}
+ MPIexec
+ \begin{rawhtml} \end{rawhtml}
+ \end{itemize}
+
+\item Build the code with the \texttt{genmake2} \texttt{-mpi} option
+ (see Section \ref{sect:genmake}) using commands such as:
+{\footnotesize \begin{verbatim}
+ % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
+ % make depend
+ % make
+\end{verbatim} }
+
+\item Run the code with the appropriate MPI ``run'' or ``exec''
+ program provided with your particular implementation of MPI.
+ Typical MPI packages such as MPICH will use something like:
+\begin{verbatim}
+ % mpirun -np 4 -machinefile mf ./mitgcmuv
+\end{verbatim}
+ Sightly more complicated scripts may be needed for many machines
+ since execution of the code may be controlled by both the MPI
+ library and a job scheduling and queueing system such as PBS,
+ LoadLeveller, Condor, or any of a number of similar tools. A few
+ example scripts (those used for our \begin{rawhtml} \end{rawhtml}regular
+ verification runs\begin{rawhtml} \end{rawhtml}) are available
+ at:
+ \begin{rawhtml}
+ \end{rawhtml}
+ {\footnotesize \tt
+ http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm\_contrib/test\_scripts/ }
+ \begin{rawhtml} \end{rawhtml}
+
+\end{enumerate}
+
+An example of the above process on the MITgcm cluster (``cg01'') using
+the GNU g77 compiler and the mpich MPI library is:
+
+{\footnotesize \begin{verbatim}
+ % cd MITgcm/verification/exp5
+ % mkdir build
+ % cd build
+ % ../../../tools/genmake2 -mpi -mods=../code \
+ -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
+ % make depend
+ % make
+ % cd ../input
+ % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
+ -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
+\end{verbatim} }
+
+
\section{Running the model}
\label{sect:runModel}
-If compilation finished succesfuully (section \ref{sect:buildModel})
-then an executable called {\em mitgcmuv} will now exist in the local
-directory.
+If compilation finished succesfuully (section \ref{sect:buildingCode})
+then an executable called \texttt{mitgcmuv} will now exist in the
+local directory.
To run the model as a single process (ie. not in parallel) simply
type: