% $Header: /home/ubuntu/mnt/e9_copy/manual/s_getstarted/text/getting_started.tex,v 1.28 2004/10/14 14:54:24 cnh Exp $
% $Name: $
%\section{Getting started}
In this section, we describe how to use the model. In the first
section, we provide enough information to help you get started with
the model. We believe the best way to familiarize yourself with the
model is to run the case study examples provided with the base
version. Information on how to obtain, compile, and run the code is
found there as well as a brief description of the model structure
directory and the case study examples. The latter and the code
structure are described more fully in chapters
\ref{chap:discretization} and \ref{chap:sarch}, respectively. Here, in
this section, we provide information on how to customize the code when
you are ready to try implementing the configuration you have in mind.
\section{Where to find information}
\label{sect:whereToFindInfo}
A web site is maintained for release 2 (``Pelican'') of MITgcm:
\begin{rawhtml} \end{rawhtml}
\begin{verbatim}
http://mitgcm.org/pelican
\end{verbatim}
\begin{rawhtml} \end{rawhtml}
Here you will find an on-line version of this document, a
``browsable'' copy of the code and a searchable database of the model
and site, as well as links for downloading the model and
documentation, to data-sources, and other related sites.
There is also a web-archived support mailing list for the model that
you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
\begin{rawhtml} \end{rawhtml}
\begin{verbatim}
http://mitgcm.org/mailman/listinfo/mitgcm-support/
http://mitgcm.org/pipermail/mitgcm-support/
\end{verbatim}
\begin{rawhtml} \end{rawhtml}
Essentially all of the MITgcm web pages can be searched using a
popular web crawler such as Google or through our own search facility:
\begin{rawhtml} \end{rawhtml}
\begin{verbatim}
http://mitgcm.org/htdig/
\end{verbatim}
\begin{rawhtml} \end{rawhtml}
%%% http://www.google.com/search?q=hydrostatic+site%3Amitgcm.org
\section{Obtaining the code}
\label{sect:obtainingCode}
MITgcm can be downloaded from our system by following
the instructions below. As a courtesy we ask that you send e-mail to us at
\begin{rawhtml} \end{rawhtml}
MITgcm-support@mitgcm.org
\begin{rawhtml} \end{rawhtml}
to enable us to keep track of who's using the model and in what application.
You can download the model two ways:
\begin{enumerate}
\item Using CVS software. CVS is a freely available source code management
tool. To use CVS you need to have the software installed. Many systems
come with CVS pre-installed, otherwise good places to look for
the software for a particular platform are
\begin{rawhtml} \end{rawhtml}
cvshome.org
\begin{rawhtml} \end{rawhtml}
and
\begin{rawhtml} \end{rawhtml}
wincvs.org
\begin{rawhtml} \end{rawhtml}
.
\item Using a tar file. This method is simple and does not
require any special software. However, this method does not
provide easy support for maintenance updates.
\end{enumerate}
\subsection{Method 1 - Checkout from CVS}
\label{sect:cvs_checkout}
If CVS is available on your system, we strongly encourage you to use it. CVS
provides an efficient and elegant way of organizing your code and keeping
track of your changes. If CVS is not available on your machine, you can also
download a tar file.
Before you can use CVS, the following environment variable(s) should
be set within your shell. For a csh or tcsh shell, put the following
\begin{verbatim}
% setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
\end{verbatim}
in your .cshrc or .tcshrc file. For bash or sh shells, put:
\begin{verbatim}
% export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
\end{verbatim}
in your \texttt{.profile} or \texttt{.bashrc} file.
To get MITgcm through CVS, first register with the MITgcm CVS server
using command:
\begin{verbatim}
% cvs login ( CVS password: cvsanon )
\end{verbatim}
You only need to do a ``cvs login'' once.
To obtain the latest sources type:
\begin{verbatim}
% cvs co MITgcm
\end{verbatim}
or to get a specific release type:
\begin{verbatim}
% cvs co -P -r checkpoint52i_post MITgcm
\end{verbatim}
The MITgcm web site contains further directions concerning the source
code and CVS. It also contains a web interface to our CVS archive so
that one may easily view the state of files, revisions, and other
development milestones:
\begin{rawhtml} \end{rawhtml}
\begin{verbatim}
http://mitgcm.org/source_code.html
\end{verbatim}
\begin{rawhtml} \end{rawhtml}
As a convenience, the MITgcm CVS server contains aliases which are
named subsets of the codebase. These aliases can be especially
helpful when used over slow internet connections or on machines with
restricted storage space. Table \ref{tab:cvsModules} contains a list
of CVS aliases
\begin{table}[htb]
\centering
\begin{tabular}[htb]{|lp{3.25in}|}\hline
\textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
\texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
\texttt{MITgcm\_verif\_basic}
& Source code plus a small set of the verification examples
(\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
\texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
\texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
\texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
\texttt{MITgcm\_verif\_all} & Source code plus all of the
verification examples. \\\hline
\end{tabular}
\caption{MITgcm CVS Modules}
\label{tab:cvsModules}
\end{table}
The checkout process creates a directory called \textit{MITgcm}. If
the directory \textit{MITgcm} exists this command updates your code
based on the repository. Each directory in the source tree contains a
directory \textit{CVS}. This information is required by CVS to keep
track of your file versions with respect to the repository. Don't edit
the files in \textit{CVS}! You can also use CVS to download code
updates. More extensive information on using CVS for maintaining
MITgcm code can be found
\begin{rawhtml} \end{rawhtml}
here
\begin{rawhtml} \end{rawhtml}
.
It is important to note that the CVS aliases in Table
\ref{tab:cvsModules} cannot be used in conjunction with the CVS
\texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
they create can be changed to a different name following the check-out:
\begin{verbatim}
% cvs co MITgcm_verif_basic
% mv MITgcm MITgcm_verif_basic
\end{verbatim}
\subsection{Method 2 - Tar file download}
\label{sect:conventionalDownload}
If you do not have CVS on your system, you can download the model as a
tar file from the web site at:
\begin{rawhtml} \end{rawhtml}
\begin{verbatim}
http://mitgcm.org/download/
\end{verbatim}
\begin{rawhtml} \end{rawhtml}
The tar file still contains CVS information which we urge you not to
delete; even if you do not use CVS yourself the information can help
us if you should need to send us your copy of the code. If a recent
tar file does not exist, then please contact the developers through
the
\begin{rawhtml} \end{rawhtml}
MITgcm-support@mitgcm.org
\begin{rawhtml} \end{rawhtml}
mailing list.
\subsubsection{Upgrading from an earlier version}
If you already have an earlier version of the code you can ``upgrade''
your copy instead of downloading the entire repository again. First,
``cd'' (change directory) to the top of your working copy:
\begin{verbatim}
% cd MITgcm
\end{verbatim}
and then issue the cvs update command such as:
\begin{verbatim}
% cvs -q update -r checkpoint52i_post -d -P
\end{verbatim}
This will update the ``tag'' to ``checkpoint52i\_post'', add any new
directories (-d) and remove any empty directories (-P). The -q option
means be quiet which will reduce the number of messages you'll see in
the terminal. If you have modified the code prior to upgrading, CVS
will try to merge your changes with the upgrades. If there is a
conflict between your modifications and the upgrade, it will report
that file with a ``C'' in front, e.g.:
\begin{verbatim}
C model/src/ini_parms.F
\end{verbatim}
If the list of conflicts scrolled off the screen, you can re-issue the
cvs update command and it will report the conflicts. Conflicts are
indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
``$>>>>>>>$''. For example,
{\small
\begin{verbatim}
<<<<<<< ini_parms.F
& bottomDragLinear,myOwnBottomDragCoefficient,
=======
& bottomDragLinear,bottomDragQuadratic,
>>>>>>> 1.18
\end{verbatim}
}
means that you added ``myOwnBottomDragCoefficient'' to a namelist at
the same time and place that we added ``bottomDragQuadratic''. You
need to resolve this conflict and in this case the line should be
changed to:
{\small
\begin{verbatim}
& bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
\end{verbatim}
}
and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
Unless you are making modifications which exactly parallel
developments we make, these types of conflicts should be rare.
\paragraph*{Upgrading to the current pre-release version}
We don't make a ``release'' for every little patch and bug fix in
order to keep the frequency of upgrades to a minimum. However, if you
have run into a problem for which ``we have already fixed in the
latest code'' and we haven't made a ``tag'' or ``release'' since that
patch then you'll need to get the latest code:
\begin{verbatim}
% cvs -q update -A -d -P
\end{verbatim}
Unlike, the ``check-out'' and ``update'' procedures above, there is no
``tag'' or release name. The -A tells CVS to upgrade to the
very latest version. As a rule, we don't recommend this since you
might upgrade while we are in the processes of checking in the code so
that you may only have part of a patch. Using this method of updating
also means we can't tell what version of the code you are working
with. So please be sure you understand what you're doing.
\section{Model and directory structure}
The ``numerical'' model is contained within a execution environment
support wrapper. This wrapper is designed to provide a general
framework for grid-point models. MITgcmUV is a specific numerical
model that uses the framework. Under this structure the model is split
into execution environment support code and conventional numerical
model code. The execution environment support code is held under the
\textit{eesupp} directory. The grid point model code is held under the
\textit{model} directory. Code execution actually starts in the
\textit{eesupp} routines and not in the \textit{model} routines. For
this reason the top-level \textit{MAIN.F} is in the
\textit{eesupp/src} directory. In general, end-users should not need
to worry about this level. The top-level routine for the numerical
part of the code is in \textit{model/src/THE\_MODEL\_MAIN.F}. Here is
a brief description of the directory structure of the model under the
root tree (a detailed description is given in section 3: Code
structure).
\begin{itemize}
\item \textit{bin}: this directory is initially empty. It is the
default directory in which to compile the code.
\item \textit{diags}: contains the code relative to time-averaged
diagnostics. It is subdivided into two subdirectories \textit{inc}
and \textit{src} that contain include files (*.\textit{h} files) and
Fortran subroutines (*.\textit{F} files), respectively.
\item \textit{doc}: contains brief documentation notes.
\item \textit{eesupp}: contains the execution environment source code.
Also subdivided into two subdirectories \textit{inc} and
\textit{src}.
\item \textit{exe}: this directory is initially empty. It is the
default directory in which to execute the code.
\item \textit{model}: this directory contains the main source code.
Also subdivided into two subdirectories \textit{inc} and
\textit{src}.
\item \textit{pkg}: contains the source code for the packages. Each
package corresponds to a subdirectory. For example, \textit{gmredi}
contains the code related to the Gent-McWilliams/Redi scheme,
\textit{aim} the code relative to the atmospheric intermediate
physics. The packages are described in detail in section 3.
\item \textit{tools}: this directory contains various useful tools.
For example, \textit{genmake2} is a script written in csh (C-shell)
that should be used to generate your makefile. The directory
\textit{adjoint} contains the makefile specific to the Tangent
linear and Adjoint Compiler (TAMC) that generates the adjoint code.
The latter is described in details in part V.
\item \textit{utils}: this directory contains various utilities. The
subdirectory \textit{knudsen2} contains code and a makefile that
compute coefficients of the polynomial approximation to the knudsen
formula for an ocean nonlinear equation of state. The
\textit{matlab} subdirectory contains matlab scripts for reading
model output directly into matlab. \textit{scripts} contains C-shell
post-processing scripts for joining processor-based and tiled-based
model output.
\item \textit{verification}: this directory contains the model
examples. See section \ref{sect:modelExamples}.
\end{itemize}
\section[MITgcm Example Experiments]{Example experiments}
\label{sect:modelExamples}
%% a set of twenty-four pre-configured numerical experiments
The MITgcm distribution comes with more than a dozen pre-configured
numerical experiments. Some of these example experiments are tests of
individual parts of the model code, but many are fully fledged
numerical simulations. A few of the examples are used for tutorial
documentation in sections \ref{sect:eg-baro} - \ref{sect:eg-global}.
The other examples follow the same general structure as the tutorial
examples. However, they only include brief instructions in a text file
called {\it README}. The examples are located in subdirectories under
the directory \textit{verification}. Each example is briefly described
below.
\subsection{Full list of model examples}
\begin{enumerate}
\item \textit{exp0} - single layer, ocean double gyre (barotropic with
free-surface). This experiment is described in detail in section
\ref{sect:eg-baro}.
\item \textit{exp1} - Four layer, ocean double gyre. This experiment
is described in detail in section \ref{sect:eg-baroc}.
\item \textit{exp2} - 4x4 degree global ocean simulation with steady
climatological forcing. This experiment is described in detail in
section \ref{sect:eg-global}.
\item \textit{exp4} - Flow over a Gaussian bump in open-water or
channel with open boundaries.
\item \textit{exp5} - Inhomogenously forced ocean convection in a
doubly periodic box.
\item \textit{front\_relax} - Relaxation of an ocean thermal front (test for
Gent/McWilliams scheme). 2D (Y-Z).
\item \textit{internal wave} - Ocean internal wave forced by open
boundary conditions.
\item \textit{natl\_box} - Eastern subtropical North Atlantic with KPP
scheme; 1 month integration
\item \textit{hs94.1x64x5} - Zonal averaged atmosphere using Held and
Suarez '94 forcing.
\item \textit{hs94.128x64x5} - 3D atmosphere dynamics using Held and
Suarez '94 forcing.
\item \textit{hs94.cs-32x32x5} - 3D atmosphere dynamics using Held and
Suarez '94 forcing on the cubed sphere.
\item \textit{aim.5l\_zon-ave} - Intermediate Atmospheric physics.
Global Zonal Mean configuration, 1x64x5 resolution.
\item \textit{aim.5l\_XZ\_Equatorial\_Slice} - Intermediate
Atmospheric physics, equatorial Slice configuration. 2D (X-Z).
\item \textit{aim.5l\_Equatorial\_Channel} - Intermediate Atmospheric
physics. 3D Equatorial Channel configuration.
\item \textit{aim.5l\_LatLon} - Intermediate Atmospheric physics.
Global configuration, on latitude longitude grid with 128x64x5 grid
points ($2.8^\circ{\rm degree}$ resolution).
\item \textit{adjustment.128x64x1} Barotropic adjustment problem on
latitude longitude grid with 128x64 grid points ($2.8^\circ{\rm
degree}$ resolution).
\item \textit{adjustment.cs-32x32x1} Barotropic adjustment problem on
cube sphere grid with 32x32 points per face ( roughly $2.8^\circ{\rm
degree}$ resolution).
\item \textit{advect\_cs} Two-dimensional passive advection test on
cube sphere grid.
\item \textit{advect\_xy} Two-dimensional (horizontal plane) passive
advection test on Cartesian grid.
\item \textit{advect\_yz} Two-dimensional (vertical plane) passive
advection test on Cartesian grid.
\item \textit{carbon} Simple passive tracer experiment. Includes
derivative calculation. Described in detail in section
\ref{sect:eg-carbon-ad}.
\item \textit{flt\_example} Example of using float package.
\item \textit{global\_ocean.90x40x15} Global circulation with GM, flux
boundary conditions and poles.
\item \textit{global\_ocean\_pressure} Global circulation in pressure
coordinate (non-Boussinesq ocean model). Described in detail in
section \ref{sect:eg-globalpressure}.
\item \textit{solid-body.cs-32x32x1} Solid body rotation test for cube
sphere grid.
\end{enumerate}
\subsection{Directory structure of model examples}
Each example directory has the following subdirectories:
\begin{itemize}
\item \textit{code}: contains the code particular to the example. At a
minimum, this directory includes the following files:
\begin{itemize}
\item \textit{code/CPP\_EEOPTIONS.h}: declares CPP keys relative to
the ``execution environment'' part of the code. The default
version is located in \textit{eesupp/inc}.
\item \textit{code/CPP\_OPTIONS.h}: declares CPP keys relative to
the ``numerical model'' part of the code. The default version is
located in \textit{model/inc}.
\item \textit{code/SIZE.h}: declares size of underlying
computational grid. The default version is located in
\textit{model/inc}.
\end{itemize}
In addition, other include files and subroutines might be present in
\textit{code} depending on the particular experiment. See Section 2
for more details.
\item \textit{input}: contains the input data files required to run
the example. At a minimum, the \textit{input} directory contains the
following files:
\begin{itemize}
\item \textit{input/data}: this file, written as a namelist,
specifies the main parameters for the experiment.
\item \textit{input/data.pkg}: contains parameters relative to the
packages used in the experiment.
\item \textit{input/eedata}: this file contains ``execution
environment'' data. At present, this consists of a specification
of the number of threads to use in $X$ and $Y$ under multithreaded
execution.
\end{itemize}
In addition, you will also find in this directory the forcing and
topography files as well as the files describing the initial state
of the experiment. This varies from experiment to experiment. See
section 2 for more details.
\item \textit{results}: this directory contains the output file
\textit{output.txt} produced by the simulation example. This file is
useful for comparison with your own output when you run the
experiment.
\end{itemize}
Once you have chosen the example you want to run, you are ready to
compile the code.
\section[Building MITgcm]{Building the code}
\label{sect:buildingCode}
To compile the code, we use the {\em make} program. This uses a file
({\em Makefile}) that allows us to pre-process source files, specify
compiler and optimization options and also figures out any file
dependencies. We supply a script ({\em genmake2}), described in
section \ref{sect:genmake}, that automatically creates the {\em
Makefile} for you. You then need to build the dependencies and
compile the code.
As an example, let's assume that you want to build and run experiment
\textit{verification/exp2}. The are multiple ways and places to
actually do this but here let's build the code in
\textit{verification/exp2/input}:
\begin{verbatim}
% cd verification/exp2/input
\end{verbatim}
First, build the {\em Makefile}:
\begin{verbatim}
% ../../../tools/genmake2 -mods=../code
\end{verbatim}
The command line option tells {\em genmake} to override model source
code with any files in the directory {\em ./code/}.
On many systems, the {\em genmake2} program will be able to
automatically recognize the hardware, find compilers and other tools
within the user's path (``echo \$PATH''), and then choose an
appropriate set of options from the files contained in the {\em
tools/build\_options} directory. Under some circumstances, a user
may have to create a new ``optfile'' in order to specify the exact
combination of compiler, compiler flags, libraries, and other options
necessary to build a particular configuration of MITgcm. In such
cases, it is generally helpful to read the existing ``optfiles'' and
mimic their syntax.
Through the MITgcm-support list, the MITgcm developers are willing to
provide help writing or modifing ``optfiles''. And we encourage users
to post new ``optfiles'' (particularly ones for new machines or
architectures) to the
\begin{rawhtml} \end{rawhtml}
MITgcm-support@mitgcm.org
\begin{rawhtml} \end{rawhtml}
list.
To specify an optfile to {\em genmake2}, the syntax is:
\begin{verbatim}
% ../../../tools/genmake2 -mods=../code -of /path/to/optfile
\end{verbatim}
Once a {\em Makefile} has been generated, we create the dependencies:
\begin{verbatim}
% make depend
\end{verbatim}
This modifies the {\em Makefile} by attaching a [long] list of files
upon which other files depend. The purpose of this is to reduce
re-compilation if and when you start to modify the code. The {\tt make
depend} command also creates links from the model source to this
directory.
Next compile the code:
\begin{verbatim}
% make
\end{verbatim}
The {\tt make} command creates an executable called \textit{mitgcmuv}.
Additional make ``targets'' are defined within the makefile to aid in
the production of adjoint and other versions of MITgcm.
Now you are ready to run the model. General instructions for doing so are
given in section \ref{sect:runModel}. Here, we can run the model with:
\begin{verbatim}
./mitgcmuv > output.txt
\end{verbatim}
where we are re-directing the stream of text output to the file {\em
output.txt}.
\section[Running MITgcm]{Running the model in prognostic mode}
\label{sect:runModel}
If compilation finished succesfuully (section \ref{sect:buildingCode})
then an executable called \texttt{mitgcmuv} will now exist in the
local directory.
To run the model as a single process (ie. not in parallel) simply
type:
\begin{verbatim}
% ./mitgcmuv
\end{verbatim}
The ``./'' is a safe-guard to make sure you use the local executable
in case you have others that exist in your path (surely odd if you
do!). The above command will spew out many lines of text output to
your screen. This output contains details such as parameter values as
well as diagnostics such as mean Kinetic energy, largest CFL number,
etc. It is worth keeping this text output with the binary output so we
normally re-direct the {\em stdout} stream as follows:
\begin{verbatim}
% ./mitgcmuv > output.txt
\end{verbatim}
For the example experiments in {\em verification}, an example of the
output is kept in {\em results/output.txt} for comparison. You can compare
your {\em output.txt} with this one to check that the set-up works.
\subsection{Output files}
The model produces various output files. At a minimum, the instantaneous
``state'' of the model is written out, which is made of the following files:
\begin{itemize}
\item \textit{U.00000nIter} - zonal component of velocity field (m/s and $>
0 $ eastward).
\item \textit{V.00000nIter} - meridional component of velocity field (m/s
and $> 0$ northward).
\item \textit{W.00000nIter} - vertical component of velocity field (ocean:
m/s and $> 0$ upward, atmosphere: Pa/s and $> 0$ towards increasing pressure
i.e. downward).
\item \textit{T.00000nIter} - potential temperature (ocean: $^{0}$C,
atmosphere: $^{0}$K).
\item \textit{S.00000nIter} - ocean: salinity (psu), atmosphere: water vapor
(g/kg).
\item \textit{Eta.00000nIter} - ocean: surface elevation (m), atmosphere:
surface pressure anomaly (Pa).
\end{itemize}
The chain \textit{00000nIter} consists of ten figures that specify the
iteration number at which the output is written out. For example, \textit{%
U.0000000300} is the zonal velocity at iteration 300.
In addition, a ``pickup'' or ``checkpoint'' file called:
\begin{itemize}
\item \textit{pickup.00000nIter}
\end{itemize}
is written out. This file represents the state of the model in a condensed
form and is used for restarting the integration. If the C-D scheme is used,
there is an additional ``pickup'' file:
\begin{itemize}
\item \textit{pickup\_cd.00000nIter}
\end{itemize}
containing the D-grid velocity data and that has to be written out as well
in order to restart the integration. Rolling checkpoint files are the same
as the pickup files but are named differently. Their name contain the chain
\textit{ckptA} or \textit{ckptB} instead of \textit{00000nIter}. They can be
used to restart the model but are overwritten every other time they are
output to save disk space during long integrations.
\subsection{Looking at the output}
All the model data are written according to a ``meta/data'' file format.
Each variable is associated with two files with suffix names \textit{.data}
and \textit{.meta}. The \textit{.data} file contains the data written in
binary form (big\_endian by default). The \textit{.meta} file is a
``header'' file that contains information about the size and the structure
of the \textit{.data} file. This way of organizing the output is
particularly useful when running multi-processors calculations. The base
version of the model includes a few matlab utilities to read output files
written in this format. The matlab scripts are located in the directory
\textit{utils/matlab} under the root tree. The script \textit{rdmds.m} reads
the data. Look at the comments inside the script to see how to use it.
Some examples of reading and visualizing some output in {\em Matlab}:
\begin{verbatim}
% matlab
>> H=rdmds('Depth');
>> contourf(H');colorbar;
>> title('Depth of fluid as used by model');
>> eta=rdmds('Eta',10);
>> imagesc(eta');axis ij;colorbar;
>> title('Surface height at iter=10');
>> eta=rdmds('Eta',[0:10:100]);
>> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
\end{verbatim}