/[MITgcm]/manual/s_getstarted/text/getting_started.tex
ViewVC logotype

Annotation of /manual/s_getstarted/text/getting_started.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.24 - (hide annotations) (download) (as text)
Fri Apr 9 15:06:18 2004 UTC (21 years, 3 months ago) by edhill
Branch: MAIN
Changes since 1.23: +16 -6 lines
File MIME type: application/x-tex
 o add links

1 edhill 1.24 % $Header: /u/gcmpack/manual/part3/getting_started.tex,v 1.23 2004/04/08 02:24:23 edhill Exp $
2 adcroft 1.2 % $Name: $
3 adcroft 1.1
4 adcroft 1.4 %\section{Getting started}
5 adcroft 1.1
6 adcroft 1.4 In this section, we describe how to use the model. In the first
7     section, we provide enough information to help you get started with
8     the model. We believe the best way to familiarize yourself with the
9     model is to run the case study examples provided with the base
10     version. Information on how to obtain, compile, and run the code is
11     found there as well as a brief description of the model structure
12     directory and the case study examples. The latter and the code
13     structure are described more fully in chapters
14     \ref{chap:discretization} and \ref{chap:sarch}, respectively. Here, in
15     this section, we provide information on how to customize the code when
16     you are ready to try implementing the configuration you have in mind.
17    
18     \section{Where to find information}
19     \label{sect:whereToFindInfo}
20    
21 edhill 1.15 A web site is maintained for release 2 (``Pelican'') of MITgcm:
22     \begin{rawhtml} <A href=http://mitgcm.org/pelican/ target="idontexist"> \end{rawhtml}
23 adcroft 1.4 \begin{verbatim}
24 edhill 1.15 http://mitgcm.org/pelican
25 adcroft 1.4 \end{verbatim}
26 edhill 1.15 \begin{rawhtml} </A> \end{rawhtml}
27 adcroft 1.4 Here you will find an on-line version of this document, a
28     ``browsable'' copy of the code and a searchable database of the model
29     and site, as well as links for downloading the model and
30 edhill 1.15 documentation, to data-sources, and other related sites.
31 adcroft 1.4
32 edhill 1.15 There is also a web-archived support mailing list for the model that
33     you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
34     \begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
35     \begin{verbatim}
36     http://mitgcm.org/mailman/listinfo/mitgcm-support/
37     http://mitgcm.org/pipermail/mitgcm-support/
38     \end{verbatim}
39     \begin{rawhtml} </A> \end{rawhtml}
40     Essentially all of the MITgcm web pages can be searched using a
41     popular web crawler such as Google or through our own search facility:
42 edhill 1.16 \begin{rawhtml} <A href=http://mitgcm.org/mailman/htdig/ target="idontexist"> \end{rawhtml}
43 adcroft 1.1 \begin{verbatim}
44 edhill 1.15 http://mitgcm.org/htdig/
45 adcroft 1.1 \end{verbatim}
46 edhill 1.15 \begin{rawhtml} </A> \end{rawhtml}
47     %%% http://www.google.com/search?q=hydrostatic+site%3Amitgcm.org
48    
49    
50 adcroft 1.4
51     \section{Obtaining the code}
52     \label{sect:obtainingCode}
53 adcroft 1.1
54 cnh 1.7 MITgcm can be downloaded from our system by following
55     the instructions below. As a courtesy we ask that you send e-mail to us at
56 edhill 1.14 \begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
57     MITgcm-support@mitgcm.org
58 cnh 1.7 \begin{rawhtml} </A> \end{rawhtml}
59     to enable us to keep track of who's using the model and in what application.
60     You can download the model two ways:
61    
62     \begin{enumerate}
63 cnh 1.9 \item Using CVS software. CVS is a freely available source code management
64 cnh 1.7 tool. To use CVS you need to have the software installed. Many systems
65     come with CVS pre-installed, otherwise good places to look for
66     the software for a particular platform are
67     \begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
68     cvshome.org
69     \begin{rawhtml} </A> \end{rawhtml}
70     and
71     \begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
72     wincvs.org
73     \begin{rawhtml} </A> \end{rawhtml}
74     .
75    
76     \item Using a tar file. This method is simple and does not
77     require any special software. However, this method does not
78     provide easy support for maintenance updates.
79    
80     \end{enumerate}
81    
82 edhill 1.19 \subsubsection{Checkout from CVS}
83     \label{sect:cvs_checkout}
84    
85 adcroft 1.1 If CVS is available on your system, we strongly encourage you to use it. CVS
86     provides an efficient and elegant way of organizing your code and keeping
87     track of your changes. If CVS is not available on your machine, you can also
88     download a tar file.
89    
90 edhill 1.15 Before you can use CVS, the following environment variable(s) should
91     be set within your shell. For a csh or tcsh shell, put the following
92     \begin{verbatim}
93     % setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
94     \end{verbatim}
95     in your .cshrc or .tcshrc file. For bash or sh shells, put:
96 adcroft 1.1 \begin{verbatim}
97 edhill 1.15 % export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
98 adcroft 1.6 \end{verbatim}
99 edhill 1.20 in your \texttt{.profile} or \texttt{.bashrc} file.
100 adcroft 1.6
101 edhill 1.15
102     To get MITgcm through CVS, first register with the MITgcm CVS server
103     using command:
104 adcroft 1.6 \begin{verbatim}
105 adcroft 1.1 % cvs login ( CVS password: cvsanon )
106     \end{verbatim}
107 edhill 1.15 You only need to do a ``cvs login'' once.
108 adcroft 1.1
109 edhill 1.15 To obtain the latest sources type:
110     \begin{verbatim}
111     % cvs co MITgcm
112     \end{verbatim}
113     or to get a specific release type:
114 adcroft 1.1 \begin{verbatim}
115 edhill 1.16 % cvs co -P -r checkpoint52i_post MITgcm
116 adcroft 1.1 \end{verbatim}
117 edhill 1.15 The MITgcm web site contains further directions concerning the source
118     code and CVS. It also contains a web interface to our CVS archive so
119     that one may easily view the state of files, revisions, and other
120     development milestones:
121 edhill 1.17 \begin{rawhtml} <A href=''http://mitgcm.org/download'' target="idontexist"> \end{rawhtml}
122 edhill 1.15 \begin{verbatim}
123 edhill 1.17 http://mitgcm.org/source_code.html
124 edhill 1.15 \end{verbatim}
125     \begin{rawhtml} </A> \end{rawhtml}
126 adcroft 1.1
127 edhill 1.19 As a convenience, the MITgcm CVS server contains aliases which are
128     named subsets of the codebase. These aliases can be especially
129     helpful when used over slow internet connections or on machines with
130     restricted storage space. Table \ref{tab:cvsModules} contains a list
131     of CVS aliases
132     \begin{table}[htb]
133     \centering
134     \begin{tabular}[htb]{|lp{3.25in}|}\hline
135     \textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
136     \texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
137     \texttt{MITgcm\_verif\_basic}
138     & Source code plus a small set of the verification examples
139     (\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
140     \texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
141     \texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
142     \texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
143     \texttt{MITgcm\_verif\_all} & Source code plus all of the
144     verification examples. \\\hline
145     \end{tabular}
146     \caption{MITgcm CVS Modules}
147     \label{tab:cvsModules}
148     \end{table}
149 edhill 1.15
150     The checkout process creates a directory called \textit{MITgcm}. If
151     the directory \textit{MITgcm} exists this command updates your code
152     based on the repository. Each directory in the source tree contains a
153     directory \textit{CVS}. This information is required by CVS to keep
154     track of your file versions with respect to the repository. Don't edit
155     the files in \textit{CVS}! You can also use CVS to download code
156     updates. More extensive information on using CVS for maintaining
157     MITgcm code can be found
158 edhill 1.17 \begin{rawhtml} <A href=''http://mitgcm.org/usingcvstoget.html'' target="idontexist"> \end{rawhtml}
159 cnh 1.7 here
160     \begin{rawhtml} </A> \end{rawhtml}
161     .
162 edhill 1.19 It is important to note that the CVS aliases in Table
163     \ref{tab:cvsModules} cannot be used in conjunction with the CVS
164     \texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
165     they create can be changed to a different name following the check-out:
166     \begin{verbatim}
167     % cvs co MITgcm_verif_basic
168     % mv MITgcm MITgcm_verif_basic
169     \end{verbatim}
170 cnh 1.7
171 adcroft 1.1
172 edhill 1.19 \subsubsection{Conventional download method}
173 adcroft 1.4 \label{sect:conventionalDownload}
174 adcroft 1.1
175 adcroft 1.4 If you do not have CVS on your system, you can download the model as a
176 edhill 1.15 tar file from the web site at:
177 cnh 1.7 \begin{rawhtml} <A href=http://mitgcm.org/download target="idontexist"> \end{rawhtml}
178 adcroft 1.1 \begin{verbatim}
179     http://mitgcm.org/download/
180     \end{verbatim}
181 cnh 1.7 \begin{rawhtml} </A> \end{rawhtml}
182 adcroft 1.4 The tar file still contains CVS information which we urge you not to
183     delete; even if you do not use CVS yourself the information can help
184 edhill 1.15 us if you should need to send us your copy of the code. If a recent
185     tar file does not exist, then please contact the developers through
186 edhill 1.17 the
187     \begin{rawhtml} <A href=''mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
188     MITgcm-support@mitgcm.org
189     \begin{rawhtml} </A> \end{rawhtml}
190     mailing list.
191 adcroft 1.1
192 edhill 1.19 \subsubsection{Upgrading from an earlier version}
193 adcroft 1.12
194     If you already have an earlier version of the code you can ``upgrade''
195     your copy instead of downloading the entire repository again. First,
196     ``cd'' (change directory) to the top of your working copy:
197     \begin{verbatim}
198     % cd MITgcm
199     \end{verbatim}
200 edhill 1.15 and then issue the cvs update command such as:
201 adcroft 1.12 \begin{verbatim}
202 edhill 1.16 % cvs -q update -r checkpoint52i_post -d -P
203 adcroft 1.12 \end{verbatim}
204 edhill 1.16 This will update the ``tag'' to ``checkpoint52i\_post'', add any new
205 adcroft 1.12 directories (-d) and remove any empty directories (-P). The -q option
206     means be quiet which will reduce the number of messages you'll see in
207     the terminal. If you have modified the code prior to upgrading, CVS
208     will try to merge your changes with the upgrades. If there is a
209     conflict between your modifications and the upgrade, it will report
210     that file with a ``C'' in front, e.g.:
211     \begin{verbatim}
212     C model/src/ini_parms.F
213     \end{verbatim}
214     If the list of conflicts scrolled off the screen, you can re-issue the
215     cvs update command and it will report the conflicts. Conflicts are
216 edhill 1.15 indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
217     ``$>>>>>>>$''. For example,
218 edhill 1.17 {\small
219 adcroft 1.12 \begin{verbatim}
220     <<<<<<< ini_parms.F
221     & bottomDragLinear,myOwnBottomDragCoefficient,
222     =======
223     & bottomDragLinear,bottomDragQuadratic,
224     >>>>>>> 1.18
225     \end{verbatim}
226 edhill 1.17 }
227 adcroft 1.12 means that you added ``myOwnBottomDragCoefficient'' to a namelist at
228     the same time and place that we added ``bottomDragQuadratic''. You
229     need to resolve this conflict and in this case the line should be
230     changed to:
231 edhill 1.17 {\small
232 adcroft 1.12 \begin{verbatim}
233     & bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
234     \end{verbatim}
235 edhill 1.17 }
236 edhill 1.15 and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
237 adcroft 1.12 Unless you are making modifications which exactly parallel
238     developments we make, these types of conflicts should be rare.
239    
240     \paragraph*{Upgrading to the current pre-release version}
241    
242     We don't make a ``release'' for every little patch and bug fix in
243     order to keep the frequency of upgrades to a minimum. However, if you
244     have run into a problem for which ``we have already fixed in the
245     latest code'' and we haven't made a ``tag'' or ``release'' since that
246     patch then you'll need to get the latest code:
247     \begin{verbatim}
248     % cvs -q update -A -d -P
249     \end{verbatim}
250     Unlike, the ``check-out'' and ``update'' procedures above, there is no
251     ``tag'' or release name. The -A tells CVS to upgrade to the
252     very latest version. As a rule, we don't recommend this since you
253     might upgrade while we are in the processes of checking in the code so
254     that you may only have part of a patch. Using this method of updating
255     also means we can't tell what version of the code you are working
256     with. So please be sure you understand what you're doing.
257    
258 adcroft 1.4 \section{Model and directory structure}
259 adcroft 1.1
260 adcroft 1.12 The ``numerical'' model is contained within a execution environment
261     support wrapper. This wrapper is designed to provide a general
262     framework for grid-point models. MITgcmUV is a specific numerical
263     model that uses the framework. Under this structure the model is split
264     into execution environment support code and conventional numerical
265     model code. The execution environment support code is held under the
266     \textit{eesupp} directory. The grid point model code is held under the
267     \textit{model} directory. Code execution actually starts in the
268     \textit{eesupp} routines and not in the \textit{model} routines. For
269 edhill 1.17 this reason the top-level \textit{MAIN.F} is in the
270     \textit{eesupp/src} directory. In general, end-users should not need
271     to worry about this level. The top-level routine for the numerical
272     part of the code is in \textit{model/src/THE\_MODEL\_MAIN.F}. Here is
273     a brief description of the directory structure of the model under the
274     root tree (a detailed description is given in section 3: Code
275     structure).
276 adcroft 1.1
277     \begin{itemize}
278    
279 edhill 1.17 \item \textit{bin}: this directory is initially empty. It is the
280     default directory in which to compile the code.
281    
282 adcroft 1.1 \item \textit{diags}: contains the code relative to time-averaged
283 edhill 1.17 diagnostics. It is subdivided into two subdirectories \textit{inc}
284     and \textit{src} that contain include files (*.\textit{h} files) and
285     Fortran subroutines (*.\textit{F} files), respectively.
286 adcroft 1.1
287     \item \textit{doc}: contains brief documentation notes.
288 edhill 1.17
289     \item \textit{eesupp}: contains the execution environment source code.
290     Also subdivided into two subdirectories \textit{inc} and
291     \textit{src}.
292    
293     \item \textit{exe}: this directory is initially empty. It is the
294     default directory in which to execute the code.
295    
296     \item \textit{model}: this directory contains the main source code.
297     Also subdivided into two subdirectories \textit{inc} and
298     \textit{src}.
299    
300     \item \textit{pkg}: contains the source code for the packages. Each
301     package corresponds to a subdirectory. For example, \textit{gmredi}
302     contains the code related to the Gent-McWilliams/Redi scheme,
303     \textit{aim} the code relative to the atmospheric intermediate
304     physics. The packages are described in detail in section 3.
305    
306     \item \textit{tools}: this directory contains various useful tools.
307     For example, \textit{genmake2} is a script written in csh (C-shell)
308     that should be used to generate your makefile. The directory
309     \textit{adjoint} contains the makefile specific to the Tangent
310     linear and Adjoint Compiler (TAMC) that generates the adjoint code.
311     The latter is described in details in part V.
312    
313 adcroft 1.1 \item \textit{utils}: this directory contains various utilities. The
314 edhill 1.17 subdirectory \textit{knudsen2} contains code and a makefile that
315     compute coefficients of the polynomial approximation to the knudsen
316     formula for an ocean nonlinear equation of state. The
317     \textit{matlab} subdirectory contains matlab scripts for reading
318     model output directly into matlab. \textit{scripts} contains C-shell
319     post-processing scripts for joining processor-based and tiled-based
320     model output.
321    
322     \item \textit{verification}: this directory contains the model
323     examples. See section \ref{sect:modelExamples}.
324 adcroft 1.1
325     \end{itemize}
326    
327 adcroft 1.4 \section{Example experiments}
328     \label{sect:modelExamples}
329 adcroft 1.1
330 edhill 1.15 %% a set of twenty-four pre-configured numerical experiments
331    
332     The MITgcm distribution comes with more than a dozen pre-configured
333     numerical experiments. Some of these example experiments are tests of
334     individual parts of the model code, but many are fully fledged
335     numerical simulations. A few of the examples are used for tutorial
336     documentation in sections \ref{sect:eg-baro} - \ref{sect:eg-global}.
337     The other examples follow the same general structure as the tutorial
338     examples. However, they only include brief instructions in a text file
339     called {\it README}. The examples are located in subdirectories under
340     the directory \textit{verification}. Each example is briefly described
341     below.
342 adcroft 1.1
343 cnh 1.8 \subsection{Full list of model examples}
344 adcroft 1.1
345 cnh 1.8 \begin{enumerate}
346 edhill 1.17
347 adcroft 1.1 \item \textit{exp0} - single layer, ocean double gyre (barotropic with
348 edhill 1.15 free-surface). This experiment is described in detail in section
349     \ref{sect:eg-baro}.
350 adcroft 1.1
351 edhill 1.15 \item \textit{exp1} - Four layer, ocean double gyre. This experiment
352     is described in detail in section \ref{sect:eg-baroc}.
353    
354 adcroft 1.1 \item \textit{exp2} - 4x4 degree global ocean simulation with steady
355 edhill 1.15 climatological forcing. This experiment is described in detail in
356     section \ref{sect:eg-global}.
357    
358     \item \textit{exp4} - Flow over a Gaussian bump in open-water or
359     channel with open boundaries.
360    
361     \item \textit{exp5} - Inhomogenously forced ocean convection in a
362     doubly periodic box.
363 adcroft 1.1
364 cnh 1.8 \item \textit{front\_relax} - Relaxation of an ocean thermal front (test for
365 adcroft 1.1 Gent/McWilliams scheme). 2D (Y-Z).
366    
367 edhill 1.15 \item \textit{internal wave} - Ocean internal wave forced by open
368     boundary conditions.
369    
370 cnh 1.8 \item \textit{natl\_box} - Eastern subtropical North Atlantic with KPP
371 edhill 1.15 scheme; 1 month integration
372    
373     \item \textit{hs94.1x64x5} - Zonal averaged atmosphere using Held and
374     Suarez '94 forcing.
375    
376     \item \textit{hs94.128x64x5} - 3D atmosphere dynamics using Held and
377     Suarez '94 forcing.
378    
379 adcroft 1.1 \item \textit{hs94.cs-32x32x5} - 3D atmosphere dynamics using Held and
380 edhill 1.15 Suarez '94 forcing on the cubed sphere.
381    
382     \item \textit{aim.5l\_zon-ave} - Intermediate Atmospheric physics.
383     Global Zonal Mean configuration, 1x64x5 resolution.
384    
385     \item \textit{aim.5l\_XZ\_Equatorial\_Slice} - Intermediate
386     Atmospheric physics, equatorial Slice configuration. 2D (X-Z).
387    
388 adcroft 1.1 \item \textit{aim.5l\_Equatorial\_Channel} - Intermediate Atmospheric
389 edhill 1.15 physics. 3D Equatorial Channel configuration.
390    
391 cnh 1.8 \item \textit{aim.5l\_LatLon} - Intermediate Atmospheric physics.
392 edhill 1.15 Global configuration, on latitude longitude grid with 128x64x5 grid
393     points ($2.8^\circ{\rm degree}$ resolution).
394    
395     \item \textit{adjustment.128x64x1} Barotropic adjustment problem on
396     latitude longitude grid with 128x64 grid points ($2.8^\circ{\rm
397     degree}$ resolution).
398    
399     \item \textit{adjustment.cs-32x32x1} Barotropic adjustment problem on
400     cube sphere grid with 32x32 points per face ( roughly $2.8^\circ{\rm
401     degree}$ resolution).
402    
403 cnh 1.8 \item \textit{advect\_cs} Two-dimensional passive advection test on
404 edhill 1.15 cube sphere grid.
405    
406     \item \textit{advect\_xy} Two-dimensional (horizontal plane) passive
407     advection test on Cartesian grid.
408    
409     \item \textit{advect\_yz} Two-dimensional (vertical plane) passive
410     advection test on Cartesian grid.
411    
412     \item \textit{carbon} Simple passive tracer experiment. Includes
413     derivative calculation. Described in detail in section
414     \ref{sect:eg-carbon-ad}.
415 cnh 1.8
416     \item \textit{flt\_example} Example of using float package.
417 edhill 1.15
418     \item \textit{global\_ocean.90x40x15} Global circulation with GM, flux
419     boundary conditions and poles.
420 cnh 1.8
421 mlosch 1.13 \item \textit{global\_ocean\_pressure} Global circulation in pressure
422     coordinate (non-Boussinesq ocean model). Described in detail in
423     section \ref{sect:eg-globalpressure}.
424 edhill 1.15
425     \item \textit{solid-body.cs-32x32x1} Solid body rotation test for cube
426     sphere grid.
427 cnh 1.8
428     \end{enumerate}
429 adcroft 1.1
430 adcroft 1.4 \subsection{Directory structure of model examples}
431 adcroft 1.1
432     Each example directory has the following subdirectories:
433    
434     \begin{itemize}
435     \item \textit{code}: contains the code particular to the example. At a
436 edhill 1.16 minimum, this directory includes the following files:
437 adcroft 1.1
438 edhill 1.16 \begin{itemize}
439     \item \textit{code/CPP\_EEOPTIONS.h}: declares CPP keys relative to
440     the ``execution environment'' part of the code. The default
441     version is located in \textit{eesupp/inc}.
442    
443     \item \textit{code/CPP\_OPTIONS.h}: declares CPP keys relative to
444     the ``numerical model'' part of the code. The default version is
445     located in \textit{model/inc}.
446    
447     \item \textit{code/SIZE.h}: declares size of underlying
448     computational grid. The default version is located in
449     \textit{model/inc}.
450     \end{itemize}
451    
452     In addition, other include files and subroutines might be present in
453     \textit{code} depending on the particular experiment. See Section 2
454     for more details.
455 edhill 1.15
456     \item \textit{input}: contains the input data files required to run
457     the example. At a minimum, the \textit{input} directory contains the
458     following files:
459 adcroft 1.1
460 edhill 1.16 \begin{itemize}
461     \item \textit{input/data}: this file, written as a namelist,
462     specifies the main parameters for the experiment.
463    
464     \item \textit{input/data.pkg}: contains parameters relative to the
465     packages used in the experiment.
466    
467     \item \textit{input/eedata}: this file contains ``execution
468     environment'' data. At present, this consists of a specification
469     of the number of threads to use in $X$ and $Y$ under multithreaded
470     execution.
471     \end{itemize}
472 edhill 1.17
473     In addition, you will also find in this directory the forcing and
474     topography files as well as the files describing the initial state
475     of the experiment. This varies from experiment to experiment. See
476     section 2 for more details.
477 edhill 1.16
478     \item \textit{results}: this directory contains the output file
479     \textit{output.txt} produced by the simulation example. This file is
480     useful for comparison with your own output when you run the
481     experiment.
482 adcroft 1.1 \end{itemize}
483    
484 edhill 1.17 Once you have chosen the example you want to run, you are ready to
485     compile the code.
486 adcroft 1.1
487 adcroft 1.4 \section{Building the code}
488     \label{sect:buildingCode}
489    
490     To compile the code, we use the {\em make} program. This uses a file
491     ({\em Makefile}) that allows us to pre-process source files, specify
492     compiler and optimization options and also figures out any file
493 edhill 1.16 dependencies. We supply a script ({\em genmake2}), described in
494     section \ref{sect:genmake}, that automatically creates the {\em
495     Makefile} for you. You then need to build the dependencies and
496     compile the code.
497 adcroft 1.4
498     As an example, let's assume that you want to build and run experiment
499 edhill 1.16 \textit{verification/exp2}. The are multiple ways and places to
500     actually do this but here let's build the code in
501 adcroft 1.4 \textit{verification/exp2/input}:
502     \begin{verbatim}
503     % cd verification/exp2/input
504     \end{verbatim}
505     First, build the {\em Makefile}:
506     \begin{verbatim}
507 edhill 1.16 % ../../../tools/genmake2 -mods=../code
508 adcroft 1.4 \end{verbatim}
509     The command line option tells {\em genmake} to override model source
510     code with any files in the directory {\em ./code/}.
511    
512 edhill 1.16 On many systems, the {\em genmake2} program will be able to
513     automatically recognize the hardware, find compilers and other tools
514     within the user's path (``echo \$PATH''), and then choose an
515     appropriate set of options from the files contained in the {\em
516     tools/build\_options} directory. Under some circumstances, a user
517     may have to create a new ``optfile'' in order to specify the exact
518     combination of compiler, compiler flags, libraries, and other options
519     necessary to build a particular configuration of MITgcm. In such
520     cases, it is generally helpful to read the existing ``optfiles'' and
521     mimic their syntax.
522    
523     Through the MITgcm-support list, the MITgcm developers are willing to
524     provide help writing or modifing ``optfiles''. And we encourage users
525     to post new ``optfiles'' (particularly ones for new machines or
526 edhill 1.17 architectures) to the
527     \begin{rawhtml} <A href=''mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
528     MITgcm-support@mitgcm.org
529     \begin{rawhtml} </A> \end{rawhtml}
530     list.
531 edhill 1.16
532     To specify an optfile to {\em genmake2}, the syntax is:
533 adcroft 1.4 \begin{verbatim}
534 edhill 1.16 % ../../../tools/genmake2 -mods=../code -of /path/to/optfile
535 adcroft 1.4 \end{verbatim}
536    
537 edhill 1.16 Once a {\em Makefile} has been generated, we create the dependencies:
538 adcroft 1.4 \begin{verbatim}
539     % make depend
540     \end{verbatim}
541 edhill 1.16 This modifies the {\em Makefile} by attaching a [long] list of files
542     upon which other files depend. The purpose of this is to reduce
543     re-compilation if and when you start to modify the code. The {\tt make
544     depend} command also creates links from the model source to this
545     directory.
546 adcroft 1.1
547 edhill 1.16 Next compile the code:
548 adcroft 1.4 \begin{verbatim}
549     % make
550     \end{verbatim}
551     The {\tt make} command creates an executable called \textit{mitgcmuv}.
552 edhill 1.16 Additional make ``targets'' are defined within the makefile to aid in
553     the production of adjoint and other versions of MITgcm.
554 adcroft 1.4
555     Now you are ready to run the model. General instructions for doing so are
556     given in section \ref{sect:runModel}. Here, we can run the model with:
557     \begin{verbatim}
558     ./mitgcmuv > output.txt
559     \end{verbatim}
560     where we are re-directing the stream of text output to the file {\em
561     output.txt}.
562    
563    
564     \subsection{Building/compiling the code elsewhere}
565    
566     In the example above (section \ref{sect:buildingCode}) we built the
567     executable in the {\em input} directory of the experiment for
568     convenience. You can also configure and compile the code in other
569     locations, for example on a scratch disk with out having to copy the
570     entire source tree. The only requirement to do so is you have {\tt
571 edhill 1.16 genmake2} in your path or you know the absolute path to {\tt
572     genmake2}.
573 adcroft 1.4
574 edhill 1.16 The following sections outline some possible methods of organizing
575     your source and data.
576 adcroft 1.4
577     \subsubsection{Building from the {\em ../code directory}}
578    
579     This is just as simple as building in the {\em input/} directory:
580     \begin{verbatim}
581     % cd verification/exp2/code
582 edhill 1.16 % ../../../tools/genmake2
583 adcroft 1.4 % make depend
584     % make
585     \end{verbatim}
586     However, to run the model the executable ({\em mitgcmuv}) and input
587     files must be in the same place. If you only have one calculation to make:
588     \begin{verbatim}
589     % cd ../input
590     % cp ../code/mitgcmuv ./
591     % ./mitgcmuv > output.txt
592     \end{verbatim}
593 cnh 1.9 or if you will be making multiple runs with the same executable:
594 adcroft 1.4 \begin{verbatim}
595     % cd ../
596     % cp -r input run1
597     % cp code/mitgcmuv run1
598     % cd run1
599     % ./mitgcmuv > output.txt
600     \end{verbatim}
601    
602     \subsubsection{Building from a new directory}
603    
604     Since the {\em input} directory contains input files it is often more
605 cnh 1.9 useful to keep {\em input} pristine and build in a new directory
606 adcroft 1.4 within {\em verification/exp2/}:
607     \begin{verbatim}
608     % cd verification/exp2
609     % mkdir build
610     % cd build
611 edhill 1.16 % ../../../tools/genmake2 -mods=../code
612 adcroft 1.4 % make depend
613     % make
614     \end{verbatim}
615     This builds the code exactly as before but this time you need to copy
616     either the executable or the input files or both in order to run the
617     model. For example,
618     \begin{verbatim}
619     % cp ../input/* ./
620     % ./mitgcmuv > output.txt
621     \end{verbatim}
622     or if you tend to make multiple runs with the same executable then
623     running in a new directory each time might be more appropriate:
624     \begin{verbatim}
625     % cd ../
626     % mkdir run1
627     % cp build/mitgcmuv run1/
628     % cp input/* run1/
629     % cd run1
630     % ./mitgcmuv > output.txt
631     \end{verbatim}
632    
633 edhill 1.16 \subsubsection{Building on a scratch disk}
634 adcroft 1.4
635     Model object files and output data can use up large amounts of disk
636     space so it is often the case that you will be operating on a large
637     scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
638     following commands will build the model in {\em /scratch/exp2-run1}:
639     \begin{verbatim}
640     % cd /scratch/exp2-run1
641 edhill 1.16 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
642     -mods=~/MITgcm/verification/exp2/code
643 adcroft 1.4 % make depend
644     % make
645     \end{verbatim}
646     To run the model here, you'll need the input files:
647     \begin{verbatim}
648     % cp ~/MITgcm/verification/exp2/input/* ./
649     % ./mitgcmuv > output.txt
650     \end{verbatim}
651    
652     As before, you could build in one directory and make multiple runs of
653     the one experiment:
654     \begin{verbatim}
655     % cd /scratch/exp2
656     % mkdir build
657     % cd build
658 edhill 1.16 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
659     -mods=~/MITgcm/verification/exp2/code
660 adcroft 1.4 % make depend
661     % make
662     % cd ../
663     % cp -r ~/MITgcm/verification/exp2/input run2
664     % cd run2
665     % ./mitgcmuv > output.txt
666     \end{verbatim}
667    
668    
669 edhill 1.24 \subsection{Using \texttt{genmake2}}
670 adcroft 1.4 \label{sect:genmake}
671 adcroft 1.1
672 edhill 1.16 To compile the code, first use the program \texttt{genmake2} (located
673 edhill 1.24 in the \texttt{tools} directory) to generate a Makefile.
674 edhill 1.16 \texttt{genmake2} is a shell script written to work with all
675     ``sh''--compatible shells including bash v1, bash v2, and Bourne.
676     Internally, \texttt{genmake2} determines the locations of needed
677     files, the compiler, compiler options, libraries, and Unix tools. It
678 edhill 1.24 relies upon a number of ``optfiles'' located in the
679     \texttt{tools/build\_options} directory.
680 edhill 1.16
681     The purpose of the optfiles is to provide all the compilation options
682     for particular ``platforms'' (where ``platform'' roughly means the
683     combination of the hardware and the compiler) and code configurations.
684     Given the combinations of possible compilers and library dependencies
685     ({\it eg.} MPI and NetCDF) there may be numerous optfiles available
686     for a single machine. The naming scheme for the majority of the
687     optfiles shipped with the code is
688     \begin{center}
689     {\bf OS\_HARDWARE\_COMPILER }
690     \end{center}
691     where
692     \begin{description}
693     \item[OS] is the name of the operating system (generally the
694     lower-case output of the {\tt 'uname'} command)
695     \item[HARDWARE] is a string that describes the CPU type and
696     corresponds to output from the {\tt 'uname -m'} command:
697     \begin{description}
698     \item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
699     and athlon
700     \item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
701     \item[amd64] is AMD x86\_64 systems
702     \item[ppc] is for Mac PowerPC systems
703     \end{description}
704     \item[COMPILER] is the compiler name (generally, the name of the
705     FORTRAN executable)
706     \end{description}
707    
708     In many cases, the default optfiles are sufficient and will result in
709     usable Makefiles. However, for some machines or code configurations,
710     new ``optfiles'' must be written. To create a new optfile, it is
711     generally best to start with one of the defaults and modify it to suit
712     your needs. Like \texttt{genmake2}, the optfiles are all written
713     using a simple ``sh''--compatible syntax. While nearly all variables
714     used within \texttt{genmake2} may be specified in the optfiles, the
715     critical ones that should be defined are:
716    
717     \begin{description}
718     \item[FC] the FORTRAN compiler (executable) to use
719     \item[DEFINES] the command-line DEFINE options passed to the compiler
720     \item[CPP] the C pre-processor to use
721     \item[NOOPTFLAGS] options flags for special files that should not be
722     optimized
723     \end{description}
724    
725     For example, the optfile for a typical Red Hat Linux machine (``ia32''
726     architecture) using the GCC (g77) compiler is
727     \begin{verbatim}
728     FC=g77
729     DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
730     CPP='cpp -traditional -P'
731     NOOPTFLAGS='-O0'
732     # For IEEE, use the "-ffloat-store" option
733     if test "x$IEEE" = x ; then
734     FFLAGS='-Wimplicit -Wunused -Wuninitialized'
735     FOPTIM='-O3 -malign-double -funroll-loops'
736     else
737     FFLAGS='-Wimplicit -Wunused -ffloat-store'
738     FOPTIM='-O0 -malign-double'
739     fi
740     \end{verbatim}
741    
742     If you write an optfile for an unrepresented machine or compiler, you
743     are strongly encouraged to submit the optfile to the MITgcm project
744     for inclusion. Please send the file to the
745     \begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
746     \begin{center}
747     MITgcm-support@mitgcm.org
748     \end{center}
749     \begin{rawhtml} </A> \end{rawhtml}
750     mailing list.
751 adcroft 1.1
752 edhill 1.16 In addition to the optfiles, \texttt{genmake2} supports a number of
753     helpful command-line options. A complete list of these options can be
754     obtained from:
755     \begin{verbatim}
756     % genmake2 -h
757     \end{verbatim}
758    
759     The most important command-line options are:
760     \begin{description}
761    
762 edhill 1.17 \item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
763     should be used for a particular build.
764 edhill 1.16
765     If no "optfile" is specified (either through the command line or the
766     MITGCM\_OPTFILE environment variable), genmake2 will try to make a
767     reasonable guess from the list provided in {\em
768     tools/build\_options}. The method used for making this guess is
769     to first determine the combination of operating system and hardware
770     (eg. "linux\_ia32") and then find a working FORTRAN compiler within
771     the user's path. When these three items have been identified,
772     genmake2 will try to find an optfile that has a matching name.
773    
774 edhill 1.22 \item[\texttt{--pdefault='PKG1 PKG2 PKG3 ...'}] specifies the default
775     set of packages to be used. The normal order of precedence for
776     packages is as follows:
777     \begin{enumerate}
778     \item If available, the command line (\texttt{--pdefault}) settings
779     over-rule any others.
780    
781     \item Next, \texttt{genmake2} will look for a file named
782     ``\texttt{packages.conf}'' in the local directory or in any of the
783     directories specified with the \texttt{--mods} option.
784    
785     \item Finally, if neither of the above are available,
786     \texttt{genmake2} will use the \texttt{/pkg/pkg\_default} file.
787     \end{enumerate}
788    
789 edhill 1.17 \item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
790     used for packages.
791 edhill 1.16
792     If not specified, the default dependency file {\em pkg/pkg\_depend}
793     is used. The syntax for this file is parsed on a line-by-line basis
794     where each line containes either a comment ("\#") or a simple
795     "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
796     specifies a "must be used with" or a "must not be used with"
797     relationship, respectively. If no rule is specified, then it is
798     assumed that the two packages are compatible and will function
799     either with or without each other.
800    
801 edhill 1.17 \item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
802     automatic differentiation options file to be used. The file is
803     analogous to the ``optfile'' defined above but it specifies
804     information for the AD build process.
805 edhill 1.16
806     The default file is located in {\em
807     tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
808     and "TAMC" compilers. An alternate version is also available at
809     {\em tools/adjoint\_options/adjoint\_staf} that selects the newer
810     "STAF" compiler. As with any compilers, it is helpful to have their
811     directories listed in your {\tt \$PATH} environment variable.
812    
813 edhill 1.17 \item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
814     directories containing ``modifications''. These directories contain
815     files with names that may (or may not) exist in the main MITgcm
816     source tree but will be overridden by any identically-named sources
817     within the ``MODS'' directories.
818 edhill 1.16
819     The order of precedence for this "name-hiding" is as follows:
820     \begin{itemize}
821     \item ``MODS'' directories (in the order given)
822     \item Packages either explicitly specified or provided by default
823     (in the order given)
824     \item Packages included due to package dependencies (in the order
825     that that package dependencies are parsed)
826     \item The "standard dirs" (which may have been specified by the
827     ``-standarddirs'' option)
828     \end{itemize}
829    
830 edhill 1.23 \item[\texttt{--mpi}] This option enables certain MPI features (using
831     CPP \texttt{\#define}s) within the code and is necessary for MPI
832     builds (see Section \ref{sect:mpi-build}).
833    
834 edhill 1.17 \item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
835     soft-links and other bugs common with the \texttt{make} versions
836     provided by commercial Unix vendors, GNU \texttt{make} (sometimes
837     called \texttt{gmake}) should be preferred. This option provides a
838     means for specifying the make executable to be used.
839 edhill 1.21
840     \item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
841     machines, the ``bash'' shell is unavailable. To run on these
842     systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
843     a Bourne, POSIX, or compatible) shell. The syntax in these
844     circumstances is:
845     \begin{center}
846 edhill 1.23 \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
847 edhill 1.21 \end{center}
848     where \texttt{/bin/sh} can be replaced with the full path and name
849     of the desired shell.
850 adcroft 1.1
851 edhill 1.16 \end{description}
852 adcroft 1.1
853    
854 edhill 1.23 \subsection{Building with MPI}
855     \label{sect:mpi-build}
856    
857     Building MITgcm to use MPI libraries can be complicated due to the
858     variety of different MPI implementations available, their dependencies
859     or interactions with different compilers, and their often ad-hoc
860     locations within file systems. For these reasons, its generally a
861     good idea to start by finding and reading the documentation for your
862     machine(s) and, if necessary, seeking help from your local systems
863     administrator.
864    
865     The steps for building MITgcm with MPI support are:
866     \begin{enumerate}
867    
868     \item Determine the locations of your MPI-enabled compiler and/or MPI
869     libraries and put them into an options file as described in Section
870     \ref{sect:genmake}. One can start with one of the examples in:
871     \begin{rawhtml} <A
872     href="http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/tools/build_options/">
873     \end{rawhtml}
874     \begin{center}
875     \texttt{MITgcm/tools/build\_options/}
876     \end{center}
877     \begin{rawhtml} </A> \end{rawhtml}
878     such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
879     \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
880     hand. You may need help from your user guide or local systems
881     administrator to determine the exact location of the MPI libraries.
882     If libraries are not installed, MPI implementations and related
883     tools are available including:
884     \begin{itemize}
885     \item \begin{rawhtml} <A
886     href="http://www-unix.mcs.anl.gov/mpi/mpich/">
887     \end{rawhtml}
888     MPICH
889     \begin{rawhtml} </A> \end{rawhtml}
890    
891     \item \begin{rawhtml} <A
892     href="http://www.lam-mpi.org/">
893     \end{rawhtml}
894     LAM/MPI
895     \begin{rawhtml} </A> \end{rawhtml}
896    
897     \item \begin{rawhtml} <A
898     href="http://www.osc.edu/~pw/mpiexec/">
899     \end{rawhtml}
900     MPIexec
901     \begin{rawhtml} </A> \end{rawhtml}
902     \end{itemize}
903    
904     \item Build the code with the \texttt{genmake2} \texttt{-mpi} option
905     (see Section \ref{sect:genmake}) using commands such as:
906     {\footnotesize \begin{verbatim}
907     % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
908     % make depend
909     % make
910     \end{verbatim} }
911    
912     \item Run the code with the appropriate MPI ``run'' or ``exec''
913     program provided with your particular implementation of MPI.
914     Typical MPI packages such as MPICH will use something like:
915     \begin{verbatim}
916     % mpirun -np 4 -machinefile mf ./mitgcmuv
917     \end{verbatim}
918     Sightly more complicated scripts may be needed for many machines
919     since execution of the code may be controlled by both the MPI
920     library and a job scheduling and queueing system such as PBS,
921 edhill 1.24 LoadLeveller, Condor, or any of a number of similar tools. A few
922     example scripts (those used for our \begin{rawhtml} <A
923     href="http://mitgcm.org/testing.html"> \end{rawhtml}regular
924     verification runs\begin{rawhtml} </A> \end{rawhtml}) are available
925     at:
926     \begin{rawhtml} <A
927     href="http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">
928     \end{rawhtml}
929     {\footnotesize \tt
930     http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm\_contrib/test\_scripts/ }
931     \begin{rawhtml} </A> \end{rawhtml}
932 edhill 1.23
933     \end{enumerate}
934    
935    
936 adcroft 1.1
937 adcroft 1.4 \section{Running the model}
938     \label{sect:runModel}
939    
940 edhill 1.23 If compilation finished succesfuully (section \ref{sect:buildingCode})
941     then an executable called \texttt{mitgcmuv} will now exist in the
942     local directory.
943 adcroft 1.1
944 adcroft 1.4 To run the model as a single process (ie. not in parallel) simply
945     type:
946 adcroft 1.1 \begin{verbatim}
947 adcroft 1.4 % ./mitgcmuv
948 adcroft 1.1 \end{verbatim}
949 adcroft 1.4 The ``./'' is a safe-guard to make sure you use the local executable
950     in case you have others that exist in your path (surely odd if you
951     do!). The above command will spew out many lines of text output to
952     your screen. This output contains details such as parameter values as
953     well as diagnostics such as mean Kinetic energy, largest CFL number,
954     etc. It is worth keeping this text output with the binary output so we
955     normally re-direct the {\em stdout} stream as follows:
956 adcroft 1.1 \begin{verbatim}
957 adcroft 1.4 % ./mitgcmuv > output.txt
958 adcroft 1.1 \end{verbatim}
959    
960 edhill 1.17 For the example experiments in {\em verification}, an example of the
961 adcroft 1.4 output is kept in {\em results/output.txt} for comparison. You can compare
962     your {\em output.txt} with this one to check that the set-up works.
963 adcroft 1.1
964    
965    
966 adcroft 1.4 \subsection{Output files}
967 adcroft 1.1
968     The model produces various output files. At a minimum, the instantaneous
969     ``state'' of the model is written out, which is made of the following files:
970    
971     \begin{itemize}
972     \item \textit{U.00000nIter} - zonal component of velocity field (m/s and $>
973     0 $ eastward).
974    
975     \item \textit{V.00000nIter} - meridional component of velocity field (m/s
976     and $> 0$ northward).
977    
978     \item \textit{W.00000nIter} - vertical component of velocity field (ocean:
979     m/s and $> 0$ upward, atmosphere: Pa/s and $> 0$ towards increasing pressure
980     i.e. downward).
981    
982     \item \textit{T.00000nIter} - potential temperature (ocean: $^{0}$C,
983     atmosphere: $^{0}$K).
984    
985     \item \textit{S.00000nIter} - ocean: salinity (psu), atmosphere: water vapor
986     (g/kg).
987    
988     \item \textit{Eta.00000nIter} - ocean: surface elevation (m), atmosphere:
989     surface pressure anomaly (Pa).
990     \end{itemize}
991    
992     The chain \textit{00000nIter} consists of ten figures that specify the
993     iteration number at which the output is written out. For example, \textit{%
994     U.0000000300} is the zonal velocity at iteration 300.
995    
996     In addition, a ``pickup'' or ``checkpoint'' file called:
997    
998     \begin{itemize}
999     \item \textit{pickup.00000nIter}
1000     \end{itemize}
1001    
1002     is written out. This file represents the state of the model in a condensed
1003     form and is used for restarting the integration. If the C-D scheme is used,
1004     there is an additional ``pickup'' file:
1005    
1006     \begin{itemize}
1007     \item \textit{pickup\_cd.00000nIter}
1008     \end{itemize}
1009    
1010     containing the D-grid velocity data and that has to be written out as well
1011     in order to restart the integration. Rolling checkpoint files are the same
1012     as the pickup files but are named differently. Their name contain the chain
1013     \textit{ckptA} or \textit{ckptB} instead of \textit{00000nIter}. They can be
1014     used to restart the model but are overwritten every other time they are
1015     output to save disk space during long integrations.
1016    
1017 adcroft 1.4 \subsection{Looking at the output}
1018 adcroft 1.1
1019     All the model data are written according to a ``meta/data'' file format.
1020     Each variable is associated with two files with suffix names \textit{.data}
1021     and \textit{.meta}. The \textit{.data} file contains the data written in
1022     binary form (big\_endian by default). The \textit{.meta} file is a
1023     ``header'' file that contains information about the size and the structure
1024     of the \textit{.data} file. This way of organizing the output is
1025     particularly useful when running multi-processors calculations. The base
1026     version of the model includes a few matlab utilities to read output files
1027     written in this format. The matlab scripts are located in the directory
1028     \textit{utils/matlab} under the root tree. The script \textit{rdmds.m} reads
1029     the data. Look at the comments inside the script to see how to use it.
1030    
1031 adcroft 1.4 Some examples of reading and visualizing some output in {\em Matlab}:
1032     \begin{verbatim}
1033     % matlab
1034     >> H=rdmds('Depth');
1035     >> contourf(H');colorbar;
1036     >> title('Depth of fluid as used by model');
1037    
1038     >> eta=rdmds('Eta',10);
1039     >> imagesc(eta');axis ij;colorbar;
1040     >> title('Surface height at iter=10');
1041    
1042     >> eta=rdmds('Eta',[0:10:100]);
1043     >> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
1044     \end{verbatim}
1045 adcroft 1.1
1046     \section{Doing it yourself: customizing the code}
1047    
1048     When you are ready to run the model in the configuration you want, the
1049 edhill 1.17 easiest thing is to use and adapt the setup of the case studies
1050     experiment (described previously) that is the closest to your
1051     configuration. Then, the amount of setup will be minimized. In this
1052     section, we focus on the setup relative to the ``numerical model''
1053     part of the code (the setup relative to the ``execution environment''
1054     part is covered in the parallel implementation section) and on the
1055     variables and parameters that you are likely to change.
1056 adcroft 1.1
1057 adcroft 1.5 \subsection{Configuration and setup}
1058 adcroft 1.4
1059 edhill 1.17 The CPP keys relative to the ``numerical model'' part of the code are
1060     all defined and set in the file \textit{CPP\_OPTIONS.h }in the
1061     directory \textit{ model/inc }or in one of the \textit{code
1062     }directories of the case study experiments under
1063     \textit{verification.} The model parameters are defined and declared
1064     in the file \textit{model/inc/PARAMS.h }and their default values are
1065     set in the routine \textit{model/src/set\_defaults.F. }The default
1066     values can be modified in the namelist file \textit{data }which needs
1067     to be located in the directory where you will run the model. The
1068     parameters are initialized in the routine
1069     \textit{model/src/ini\_parms.F}. Look at this routine to see in what
1070     part of the namelist the parameters are located.
1071    
1072     In what follows the parameters are grouped into categories related to
1073     the computational domain, the equations solved in the model, and the
1074     simulation controls.
1075 adcroft 1.1
1076 adcroft 1.4 \subsection{Computational domain, geometry and time-discretization}
1077 adcroft 1.1
1078 edhill 1.17 \begin{description}
1079     \item[dimensions] \
1080    
1081 edhill 1.18 The number of points in the x, y, and r directions are represented
1082     by the variables \textbf{sNx}, \textbf{sNy} and \textbf{Nr}
1083     respectively which are declared and set in the file
1084     \textit{model/inc/SIZE.h}. (Again, this assumes a mono-processor
1085     calculation. For multiprocessor calculations see the section on
1086     parallel implementation.)
1087 edhill 1.17
1088     \item[grid] \
1089    
1090     Three different grids are available: cartesian, spherical polar, and
1091 edhill 1.18 curvilinear (which includes the cubed sphere). The grid is set
1092     through the logical variables \textbf{usingCartesianGrid},
1093     \textbf{usingSphericalPolarGrid}, and \textbf{usingCurvilinearGrid}.
1094     In the case of spherical and curvilinear grids, the southern
1095     boundary is defined through the variable \textbf{phiMin} which
1096     corresponds to the latitude of the southern most cell face (in
1097     degrees). The resolution along the x and y directions is controlled
1098     by the 1D arrays \textbf{delx} and \textbf{dely} (in meters in the
1099     case of a cartesian grid, in degrees otherwise). The vertical grid
1100     spacing is set through the 1D array \textbf{delz} for the ocean (in
1101     meters) or \textbf{delp} for the atmosphere (in Pa). The variable
1102     \textbf{Ro\_SeaLevel} represents the standard position of Sea-Level
1103     in ``R'' coordinate. This is typically set to 0m for the ocean
1104     (default value) and 10$^{5}$Pa for the atmosphere. For the
1105     atmosphere, also set the logical variable \textbf{groundAtK1} to
1106     \texttt{'.TRUE.'} which puts the first level (k=1) at the lower
1107 edhill 1.17 boundary (ground).
1108    
1109     For the cartesian grid case, the Coriolis parameter $f$ is set
1110 edhill 1.18 through the variables \textbf{f0} and \textbf{beta} which correspond
1111     to the reference Coriolis parameter (in s$^{-1}$) and
1112     $\frac{\partial f}{ \partial y}$(in m$^{-1}$s$^{-1}$) respectively.
1113     If \textbf{beta } is set to a nonzero value, \textbf{f0} is the
1114     value of $f$ at the southern edge of the domain.
1115 edhill 1.17
1116     \item[topography - full and partial cells] \
1117    
1118     The domain bathymetry is read from a file that contains a 2D (x,y)
1119     map of depths (in m) for the ocean or pressures (in Pa) for the
1120     atmosphere. The file name is represented by the variable
1121 edhill 1.18 \textbf{bathyFile}. The file is assumed to contain binary numbers
1122     giving the depth (pressure) of the model at each grid cell, ordered
1123     with the x coordinate varying fastest. The points are ordered from
1124     low coordinate to high coordinate for both axes. The model code
1125     applies without modification to enclosed, periodic, and double
1126     periodic domains. Periodicity is assumed by default and is
1127 edhill 1.17 suppressed by setting the depths to 0m for the cells at the limits
1128     of the computational domain (note: not sure this is the case for the
1129     atmosphere). The precision with which to read the binary data is
1130 edhill 1.18 controlled by the integer variable \textbf{readBinaryPrec} which can
1131 edhill 1.17 take the value \texttt{32} (single precision) or \texttt{64} (double
1132 edhill 1.18 precision). See the matlab program \textit{gendata.m} in the
1133     \textit{input} directories under \textit{verification} to see how
1134 edhill 1.17 the bathymetry files are generated for the case study experiments.
1135    
1136 edhill 1.18 To use the partial cell capability, the variable \textbf{hFacMin}
1137     needs to be set to a value between 0 and 1 (it is set to 1 by
1138     default) corresponding to the minimum fractional size of the cell.
1139     For example if the bottom cell is 500m thick and \textbf{hFacMin} is
1140     set to 0.1, the actual thickness of the cell (i.e. used in the code)
1141     can cover a range of discrete values 50m apart from 50m to 500m
1142     depending on the value of the bottom depth (in \textbf{bathyFile})
1143     at this point.
1144 edhill 1.17
1145     Note that the bottom depths (or pressures) need not coincide with
1146 edhill 1.18 the models levels as deduced from \textbf{delz} or \textbf{delp}.
1147     The model will interpolate the numbers in \textbf{bathyFile} so that
1148     they match the levels obtained from \textbf{delz} or \textbf{delp}
1149     and \textbf{hFacMin}.
1150 edhill 1.17
1151     (Note: the atmospheric case is a bit more complicated than what is
1152     written here I think. To come soon...)
1153    
1154     \item[time-discretization] \
1155    
1156     The time steps are set through the real variables \textbf{deltaTMom}
1157     and \textbf{deltaTtracer} (in s) which represent the time step for
1158     the momentum and tracer equations, respectively. For synchronous
1159     integrations, simply set the two variables to the same value (or you
1160     can prescribe one time step only through the variable
1161     \textbf{deltaT}). The Adams-Bashforth stabilizing parameter is set
1162     through the variable \textbf{abEps} (dimensionless). The stagger
1163     baroclinic time stepping can be activated by setting the logical
1164 edhill 1.18 variable \textbf{staggerTimeStep} to \texttt{'.TRUE.'}.
1165 adcroft 1.1
1166 edhill 1.17 \end{description}
1167 adcroft 1.1
1168    
1169 adcroft 1.4 \subsection{Equation of state}
1170 adcroft 1.1
1171 mlosch 1.13 First, because the model equations are written in terms of
1172     perturbations, a reference thermodynamic state needs to be specified.
1173     This is done through the 1D arrays \textbf{tRef} and \textbf{sRef}.
1174     \textbf{tRef} specifies the reference potential temperature profile
1175     (in $^{o}$C for the ocean and $^{o}$K for the atmosphere) starting
1176     from the level k=1. Similarly, \textbf{sRef} specifies the reference
1177     salinity profile (in ppt) for the ocean or the reference specific
1178     humidity profile (in g/kg) for the atmosphere.
1179    
1180     The form of the equation of state is controlled by the character
1181     variables \textbf{buoyancyRelation} and \textbf{eosType}.
1182 edhill 1.18 \textbf{buoyancyRelation} is set to \texttt{'OCEANIC'} by default and
1183     needs to be set to \texttt{'ATMOSPHERIC'} for atmosphere simulations.
1184     In this case, \textbf{eosType} must be set to \texttt{'IDEALGAS'}.
1185 mlosch 1.13 For the ocean, two forms of the equation of state are available:
1186 edhill 1.18 linear (set \textbf{eosType} to \texttt{'LINEAR'}) and a polynomial
1187     approximation to the full nonlinear equation ( set \textbf{eosType} to
1188     \texttt{'POLYNOMIAL'}). In the linear case, you need to specify the
1189     thermal and haline expansion coefficients represented by the variables
1190     \textbf{tAlpha} (in K$^{-1}$) and \textbf{sBeta} (in ppt$^{-1}$). For
1191     the nonlinear case, you need to generate a file of polynomial
1192     coefficients called \textit{POLY3.COEFFS}. To do this, use the program
1193 mlosch 1.13 \textit{utils/knudsen2/knudsen2.f} under the model tree (a Makefile is
1194     available in the same directory and you will need to edit the number
1195     and the values of the vertical levels in \textit{knudsen2.f} so that
1196     they match those of your configuration).
1197    
1198     There there are also higher polynomials for the equation of state:
1199     \begin{description}
1200 edhill 1.18 \item[\texttt{'UNESCO'}:] The UNESCO equation of state formula of
1201 mlosch 1.13 Fofonoff and Millard \cite{fofonoff83}. This equation of state
1202 edhill 1.18 assumes in-situ temperature, which is not a model variable; {\em its
1203     use is therefore discouraged, and it is only listed for
1204     completeness}.
1205     \item[\texttt{'JMD95Z'}:] A modified UNESCO formula by Jackett and
1206 mlosch 1.13 McDougall \cite{jackett95}, which uses the model variable potential
1207 edhill 1.18 temperature as input. The \texttt{'Z'} indicates that this equation
1208 mlosch 1.13 of state uses a horizontally and temporally constant pressure
1209     $p_{0}=-g\rho_{0}z$.
1210 edhill 1.18 \item[\texttt{'JMD95P'}:] A modified UNESCO formula by Jackett and
1211 mlosch 1.13 McDougall \cite{jackett95}, which uses the model variable potential
1212 edhill 1.18 temperature as input. The \texttt{'P'} indicates that this equation
1213 mlosch 1.13 of state uses the actual hydrostatic pressure of the last time
1214     step. Lagging the pressure in this way requires an additional pickup
1215     file for restarts.
1216 edhill 1.18 \item[\texttt{'MDJWF'}:] The new, more accurate and less expensive
1217 mlosch 1.13 equation of state by McDougall et~al. \cite{mcdougall03}. It also
1218     requires lagging the pressure and therefore an additional pickup
1219     file for restarts.
1220     \end{description}
1221     For none of these options an reference profile of temperature or
1222     salinity is required.
1223 adcroft 1.1
1224 adcroft 1.4 \subsection{Momentum equations}
1225 adcroft 1.1
1226 edhill 1.18 In this section, we only focus for now on the parameters that you are
1227     likely to change, i.e. the ones relative to forcing and dissipation
1228     for example. The details relevant to the vector-invariant form of the
1229     equations and the various advection schemes are not covered for the
1230     moment. We assume that you use the standard form of the momentum
1231     equations (i.e. the flux-form) with the default advection scheme.
1232     Also, there are a few logical variables that allow you to turn on/off
1233     various terms in the momentum equation. These variables are called
1234     \textbf{momViscosity, momAdvection, momForcing, useCoriolis,
1235     momPressureForcing, momStepping} and \textbf{metricTerms }and are
1236     assumed to be set to \texttt{'.TRUE.'} here. Look at the file
1237     \textit{model/inc/PARAMS.h }for a precise definition of these
1238     variables.
1239 adcroft 1.1
1240 edhill 1.17 \begin{description}
1241     \item[initialization] \
1242    
1243     The velocity components are initialized to 0 unless the simulation
1244     is starting from a pickup file (see section on simulation control
1245     parameters).
1246    
1247     \item[forcing] \
1248    
1249     This section only applies to the ocean. You need to generate
1250 edhill 1.18 wind-stress data into two files \textbf{zonalWindFile} and
1251     \textbf{meridWindFile} corresponding to the zonal and meridional
1252 edhill 1.17 components of the wind stress, respectively (if you want the stress
1253     to be along the direction of only one of the model horizontal axes,
1254     you only need to generate one file). The format of the files is
1255     similar to the bathymetry file. The zonal (meridional) stress data
1256     are assumed to be in Pa and located at U-points (V-points). As for
1257     the bathymetry, the precision with which to read the binary data is
1258 edhill 1.18 controlled by the variable \textbf{readBinaryPrec}. See the matlab
1259     program \textit{gendata.m} in the \textit{input} directories under
1260     \textit{verification} to see how simple analytical wind forcing data
1261     are generated for the case study experiments.
1262 edhill 1.17
1263     There is also the possibility of prescribing time-dependent periodic
1264     forcing. To do this, concatenate the successive time records into a
1265 edhill 1.18 single file (for each stress component) ordered in a (x,y,t) fashion
1266     and set the following variables: \textbf{periodicExternalForcing }to
1267     \texttt{'.TRUE.'}, \textbf{externForcingPeriod }to the period (in s)
1268     of which the forcing varies (typically 1 month), and
1269     \textbf{externForcingCycle} to the repeat time (in s) of the forcing
1270     (typically 1 year -- note: \textbf{ externForcingCycle} must be a
1271     multiple of \textbf{externForcingPeriod}). With these variables set
1272     up, the model will interpolate the forcing linearly at each
1273     iteration.
1274 edhill 1.17
1275     \item[dissipation] \
1276    
1277     The lateral eddy viscosity coefficient is specified through the
1278 edhill 1.18 variable \textbf{viscAh} (in m$^{2}$s$^{-1}$). The vertical eddy
1279     viscosity coefficient is specified through the variable
1280     \textbf{viscAz} (in m$^{2}$s$^{-1}$) for the ocean and
1281     \textbf{viscAp} (in Pa$^{2}$s$^{-1}$) for the atmosphere. The
1282     vertical diffusive fluxes can be computed implicitly by setting the
1283     logical variable \textbf{implicitViscosity }to \texttt{'.TRUE.'}.
1284     In addition, biharmonic mixing can be added as well through the
1285     variable \textbf{viscA4} (in m$^{4}$s$^{-1}$). On a spherical polar
1286     grid, you might also need to set the variable \textbf{cosPower}
1287     which is set to 0 by default and which represents the power of
1288     cosine of latitude to multiply viscosity. Slip or no-slip conditions
1289     at lateral and bottom boundaries are specified through the logical
1290     variables \textbf{no\_slip\_sides} and \textbf{no\_slip\_bottom}. If
1291     set to \texttt{'.FALSE.'}, free-slip boundary conditions are
1292     applied. If no-slip boundary conditions are applied at the bottom, a
1293     bottom drag can be applied as well. Two forms are available: linear
1294     (set the variable \textbf{bottomDragLinear} in s$ ^{-1}$) and
1295     quadratic (set the variable \textbf{bottomDragQuadratic} in
1296     m$^{-1}$).
1297 edhill 1.17
1298     The Fourier and Shapiro filters are described elsewhere.
1299    
1300     \item[C-D scheme] \
1301    
1302     If you run at a sufficiently coarse resolution, you will need the
1303     C-D scheme for the computation of the Coriolis terms. The
1304     variable\textbf{\ tauCD}, which represents the C-D scheme coupling
1305     timescale (in s) needs to be set.
1306    
1307     \item[calculation of pressure/geopotential] \
1308    
1309     First, to run a non-hydrostatic ocean simulation, set the logical
1310 edhill 1.18 variable \textbf{nonHydrostatic} to \texttt{'.TRUE.'}. The pressure
1311 edhill 1.17 field is then inverted through a 3D elliptic equation. (Note: this
1312     capability is not available for the atmosphere yet.) By default, a
1313     hydrostatic simulation is assumed and a 2D elliptic equation is used
1314     to invert the pressure field. The parameters controlling the
1315     behaviour of the elliptic solvers are the variables
1316 edhill 1.18 \textbf{cg2dMaxIters} and \textbf{cg2dTargetResidual } for
1317     the 2D case and \textbf{cg3dMaxIters} and
1318     \textbf{cg3dTargetResidual} for the 3D case. You probably won't need to
1319 edhill 1.17 alter the default values (are we sure of this?).
1320    
1321     For the calculation of the surface pressure (for the ocean) or
1322     surface geopotential (for the atmosphere) you need to set the
1323 edhill 1.18 logical variables \textbf{rigidLid} and \textbf{implicitFreeSurface}
1324     (set one to \texttt{'.TRUE.'} and the other to \texttt{'.FALSE.'}
1325     depending on how you want to deal with the ocean upper or atmosphere
1326     lower boundary).
1327 adcroft 1.1
1328 edhill 1.17 \end{description}
1329 adcroft 1.1
1330 adcroft 1.4 \subsection{Tracer equations}
1331 adcroft 1.1
1332 edhill 1.18 This section covers the tracer equations i.e. the potential
1333     temperature equation and the salinity (for the ocean) or specific
1334     humidity (for the atmosphere) equation. As for the momentum equations,
1335     we only describe for now the parameters that you are likely to change.
1336     The logical variables \textbf{tempDiffusion} \textbf{tempAdvection}
1337     \textbf{tempForcing}, and \textbf{tempStepping} allow you to turn
1338     on/off terms in the temperature equation (same thing for salinity or
1339     specific humidity with variables \textbf{saltDiffusion},
1340     \textbf{saltAdvection} etc.). These variables are all assumed here to
1341     be set to \texttt{'.TRUE.'}. Look at file \textit{model/inc/PARAMS.h}
1342     for a precise definition.
1343 adcroft 1.1
1344 edhill 1.17 \begin{description}
1345     \item[initialization] \
1346    
1347     The initial tracer data can be contained in the binary files
1348 edhill 1.18 \textbf{hydrogThetaFile} and \textbf{hydrogSaltFile}. These files
1349     should contain 3D data ordered in an (x,y,r) fashion with k=1 as the
1350     first vertical level. If no file names are provided, the tracers
1351     are then initialized with the values of \textbf{tRef} and
1352     \textbf{sRef} mentioned above (in the equation of state section). In
1353 edhill 1.17 this case, the initial tracer data are uniform in x and y for each
1354     depth level.
1355    
1356     \item[forcing] \
1357    
1358     This part is more relevant for the ocean, the procedure for the
1359     atmosphere not being completely stabilized at the moment.
1360    
1361     A combination of fluxes data and relaxation terms can be used for
1362 edhill 1.18 driving the tracer equations. For potential temperature, heat flux
1363 edhill 1.17 data (in W/m$ ^{2}$) can be stored in the 2D binary file
1364 edhill 1.18 \textbf{surfQfile}. Alternatively or in addition, the forcing can
1365     be specified through a relaxation term. The SST data to which the
1366     model surface temperatures are restored to are supposed to be stored
1367     in the 2D binary file \textbf{thetaClimFile}. The corresponding
1368     relaxation time scale coefficient is set through the variable
1369     \textbf{tauThetaClimRelax} (in s). The same procedure applies for
1370     salinity with the variable names \textbf{EmPmRfile},
1371     \textbf{saltClimFile}, and \textbf{tauSaltClimRelax} for freshwater
1372     flux (in m/s) and surface salinity (in ppt) data files and
1373     relaxation time scale coefficient (in s), respectively. Also for
1374     salinity, if the CPP key \textbf{USE\_NATURAL\_BCS} is turned on,
1375     natural boundary conditions are applied i.e. when computing the
1376     surface salinity tendency, the freshwater flux is multiplied by the
1377     model surface salinity instead of a constant salinity value.
1378 edhill 1.17
1379     As for the other input files, the precision with which to read the
1380     data is controlled by the variable \textbf{readBinaryPrec}.
1381     Time-dependent, periodic forcing can be applied as well following
1382     the same procedure used for the wind forcing data (see above).
1383    
1384     \item[dissipation] \
1385    
1386     Lateral eddy diffusivities for temperature and salinity/specific
1387 edhill 1.18 humidity are specified through the variables \textbf{diffKhT} and
1388     \textbf{diffKhS} (in m$^{2}$/s). Vertical eddy diffusivities are
1389     specified through the variables \textbf{diffKzT} and
1390     \textbf{diffKzS} (in m$^{2}$/s) for the ocean and \textbf{diffKpT
1391     }and \textbf{diffKpS} (in Pa$^{2}$/s) for the atmosphere. The
1392     vertical diffusive fluxes can be computed implicitly by setting the
1393     logical variable \textbf{implicitDiffusion} to \texttt{'.TRUE.'}.
1394     In addition, biharmonic diffusivities can be specified as well
1395     through the coefficients \textbf{diffK4T} and \textbf{diffK4S} (in
1396 edhill 1.17 m$^{4}$/s). Note that the cosine power scaling (specified through
1397 edhill 1.18 \textbf{cosPower}---see the momentum equations section) is applied to
1398     the tracer diffusivities (Laplacian and biharmonic) as well. The
1399 edhill 1.17 Gent and McWilliams parameterization for oceanic tracers is
1400     described in the package section. Finally, note that tracers can be
1401     also subject to Fourier and Shapiro filtering (see the corresponding
1402     section on these filters).
1403    
1404     \item[ocean convection] \
1405    
1406     Two options are available to parameterize ocean convection: one is
1407     to use the convective adjustment scheme. In this case, you need to
1408     set the variable \textbf{cadjFreq}, which represents the frequency
1409     (in s) with which the adjustment algorithm is called, to a non-zero
1410     value (if set to a negative value by the user, the model will set it
1411     to the tracer time step). The other option is to parameterize
1412     convection with implicit vertical diffusion. To do this, set the
1413 edhill 1.18 logical variable \textbf{implicitDiffusion} to \texttt{'.TRUE.'}
1414     and the real variable \textbf{ivdc\_kappa} to a value (in m$^{2}$/s)
1415 edhill 1.17 you wish the tracer vertical diffusivities to have when mixing
1416     tracers vertically due to static instabilities. Note that
1417 edhill 1.18 \textbf{cadjFreq} and \textbf{ivdc\_kappa}can not both have non-zero
1418     value.
1419 adcroft 1.1
1420 edhill 1.17 \end{description}
1421 adcroft 1.1
1422 adcroft 1.4 \subsection{Simulation controls}
1423 adcroft 1.1
1424 edhill 1.18 The model ''clock'' is defined by the variable \textbf{deltaTClock}
1425     (in s) which determines the IO frequencies and is used in tagging
1426     output. Typically, you will set it to the tracer time step for
1427     accelerated runs (otherwise it is simply set to the default time step
1428     \textbf{deltaT}). Frequency of checkpointing and dumping of the model
1429     state are referenced to this clock (see below).
1430 adcroft 1.1
1431 edhill 1.17 \begin{description}
1432     \item[run duration] \
1433    
1434     The beginning of a simulation is set by specifying a start time (in
1435 edhill 1.18 s) through the real variable \textbf{startTime} or by specifying an
1436 edhill 1.17 initial iteration number through the integer variable
1437     \textbf{nIter0}. If these variables are set to nonzero values, the
1438 edhill 1.18 model will look for a ''pickup'' file \textit{pickup.0000nIter0} to
1439     restart the integration. The end of a simulation is set through the
1440     real variable \textbf{endTime} (in s). Alternatively, you can
1441     specify instead the number of time steps to execute through the
1442     integer variable \textbf{nTimeSteps}.
1443 edhill 1.17
1444     \item[frequency of output] \
1445    
1446     Real variables defining frequencies (in s) with which output files
1447 edhill 1.18 are written on disk need to be set up. \textbf{dumpFreq} controls
1448 edhill 1.17 the frequency with which the instantaneous state of the model is
1449 edhill 1.18 saved. \textbf{chkPtFreq} and \textbf{pchkPtFreq} control the output
1450     frequency of rolling and permanent checkpoint files, respectively.
1451     See section 1.5.1 Output files for the definition of model state and
1452     checkpoint files. In addition, time-averaged fields can be written
1453     out by setting the variable \textbf{taveFreq} (in s). The precision
1454     with which to write the binary data is controlled by the integer
1455     variable w\textbf{riteBinaryPrec} (set it to \texttt{32} or
1456     \texttt{64}).
1457 adcroft 1.1
1458 edhill 1.17 \end{description}
1459 adcroft 1.1
1460 mlosch 1.13
1461     %%% Local Variables:
1462     %%% mode: latex
1463     %%% TeX-master: t
1464     %%% End:

  ViewVC Help
Powered by ViewVC 1.1.22