[MITgcm] Annotation of /manual/s_getstarted/text/getting

1	edhill	1.23	% $Header: /u/gcmpack/manual/part3/getting_started.tex,v 1.22 2004/03/24 20:53:12 edhill Exp $
2	adcroft	1.2	% $Name: $
3	adcroft	1.1
4	adcroft	1.4	%\section{Getting started}
5	adcroft	1.1
6	adcroft	1.4	In this section, we describe how to use the model. In the first
7			section, we provide enough information to help you get started with
8			the model. We believe the best way to familiarize yourself with the
9			model is to run the case study examples provided with the base
10			version. Information on how to obtain, compile, and run the code is
11			found there as well as a brief description of the model structure
12			directory and the case study examples. The latter and the code
13			structure are described more fully in chapters
14			\ref{chap:discretization} and \ref{chap:sarch}, respectively. Here, in
15			this section, we provide information on how to customize the code when
16			you are ready to try implementing the configuration you have in mind.
17
18			\section{Where to find information}
19			\label{sect:whereToFindInfo}
20
21	edhill	1.15	A web site is maintained for release 2 (``Pelican'') of MITgcm:
22			\begin{rawhtml} <A href=http://mitgcm.org/pelican/ target="idontexist"> \end{rawhtml}
23	adcroft	1.4	\begin{verbatim}
24	edhill	1.15	http://mitgcm.org/pelican
25	adcroft	1.4	\end{verbatim}
26	edhill	1.15	\begin{rawhtml} </A> \end{rawhtml}
27	adcroft	1.4	Here you will find an on-line version of this document, a
28			``browsable'' copy of the code and a searchable database of the model
29			and site, as well as links for downloading the model and
30	edhill	1.15	documentation, to data-sources, and other related sites.
31	adcroft	1.4
32	edhill	1.15	There is also a web-archived support mailing list for the model that
33			you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
34			\begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
35			\begin{verbatim}
36			http://mitgcm.org/mailman/listinfo/mitgcm-support/
37			http://mitgcm.org/pipermail/mitgcm-support/
38			\end{verbatim}
39			\begin{rawhtml} </A> \end{rawhtml}
40			Essentially all of the MITgcm web pages can be searched using a
41			popular web crawler such as Google or through our own search facility:
42	edhill	1.16	\begin{rawhtml} <A href=http://mitgcm.org/mailman/htdig/ target="idontexist"> \end{rawhtml}
43	adcroft	1.1	\begin{verbatim}
44	edhill	1.15	http://mitgcm.org/htdig/
45	adcroft	1.1	\end{verbatim}
46	edhill	1.15	\begin{rawhtml} </A> \end{rawhtml}
47			%%% http://www.google.com/search?q=hydrostatic+site%3Amitgcm.org
48
49
50	adcroft	1.4
51			\section{Obtaining the code}
52			\label{sect:obtainingCode}
53	adcroft	1.1
54	cnh	1.7	MITgcm can be downloaded from our system by following
55			the instructions below. As a courtesy we ask that you send e-mail to us at
56	edhill	1.14	\begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
57			MITgcm-support@mitgcm.org
58	cnh	1.7	\begin{rawhtml} </A> \end{rawhtml}
59			to enable us to keep track of who's using the model and in what application.
60			You can download the model two ways:
61
62			\begin{enumerate}
63	cnh	1.9	\item Using CVS software. CVS is a freely available source code management
64	cnh	1.7	tool. To use CVS you need to have the software installed. Many systems
65			come with CVS pre-installed, otherwise good places to look for
66			the software for a particular platform are
67			\begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
68			cvshome.org
69			\begin{rawhtml} </A> \end{rawhtml}
70			and
71			\begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
72			wincvs.org
73			\begin{rawhtml} </A> \end{rawhtml}
74			.
75
76			\item Using a tar file. This method is simple and does not
77			require any special software. However, this method does not
78			provide easy support for maintenance updates.
79
80			\end{enumerate}
81
82	edhill	1.19	\subsubsection{Checkout from CVS}
83			\label{sect:cvs_checkout}
84
85	adcroft	1.1	If CVS is available on your system, we strongly encourage you to use it. CVS
86			provides an efficient and elegant way of organizing your code and keeping
87			track of your changes. If CVS is not available on your machine, you can also
88			download a tar file.
89
90	edhill	1.15	Before you can use CVS, the following environment variable(s) should
91			be set within your shell. For a csh or tcsh shell, put the following
92			\begin{verbatim}
93			% setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
94			\end{verbatim}
95			in your .cshrc or .tcshrc file. For bash or sh shells, put:
96	adcroft	1.1	\begin{verbatim}
97	edhill	1.15	% export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
98	adcroft	1.6	\end{verbatim}
99	edhill	1.20	in your \texttt{.profile} or \texttt{.bashrc} file.
100	adcroft	1.6
101	edhill	1.15
102			To get MITgcm through CVS, first register with the MITgcm CVS server
103			using command:
104	adcroft	1.6	\begin{verbatim}
105	adcroft	1.1	% cvs login ( CVS password: cvsanon )
106			\end{verbatim}
107	edhill	1.15	You only need to do a ``cvs login'' once.
108	adcroft	1.1
109	edhill	1.15	To obtain the latest sources type:
110			\begin{verbatim}
111			% cvs co MITgcm
112			\end{verbatim}
113			or to get a specific release type:
114	adcroft	1.1	\begin{verbatim}
115	edhill	1.16	% cvs co -P -r checkpoint52i_post MITgcm
116	adcroft	1.1	\end{verbatim}
117	edhill	1.15	The MITgcm web site contains further directions concerning the source
118			code and CVS. It also contains a web interface to our CVS archive so
119			that one may easily view the state of files, revisions, and other
120			development milestones:
121	edhill	1.17	\begin{rawhtml} <A href=''http://mitgcm.org/download'' target="idontexist"> \end{rawhtml}
122	edhill	1.15	\begin{verbatim}
123	edhill	1.17	http://mitgcm.org/source_code.html
124	edhill	1.15	\end{verbatim}
125			\begin{rawhtml} </A> \end{rawhtml}
126	adcroft	1.1
127	edhill	1.19	As a convenience, the MITgcm CVS server contains aliases which are
128			named subsets of the codebase. These aliases can be especially
129			helpful when used over slow internet connections or on machines with
130			restricted storage space. Table \ref{tab:cvsModules} contains a list
131			of CVS aliases
132			\begin{table}[htb]
133			\centering
134			\begin{tabular}[htb]{\|lp{3.25in}\|}\hline
135			\textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
136			\texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
137			\texttt{MITgcm\_verif\_basic}
138			& Source code plus a small set of the verification examples
139			(\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
140			\texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
141			\texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
142			\texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
143			\texttt{MITgcm\_verif\_all} & Source code plus all of the
144			verification examples. \\\hline
145			\end{tabular}
146			\caption{MITgcm CVS Modules}
147			\label{tab:cvsModules}
148			\end{table}
149	edhill	1.15
150			The checkout process creates a directory called \textit{MITgcm}. If
151			the directory \textit{MITgcm} exists this command updates your code
152			based on the repository. Each directory in the source tree contains a
153			directory \textit{CVS}. This information is required by CVS to keep
154			track of your file versions with respect to the repository. Don't edit
155			the files in \textit{CVS}! You can also use CVS to download code
156			updates. More extensive information on using CVS for maintaining
157			MITgcm code can be found
158	edhill	1.17	\begin{rawhtml} <A href=''http://mitgcm.org/usingcvstoget.html'' target="idontexist"> \end{rawhtml}
159	cnh	1.7	here
160			\begin{rawhtml} </A> \end{rawhtml}
161			.
162	edhill	1.19	It is important to note that the CVS aliases in Table
163			\ref{tab:cvsModules} cannot be used in conjunction with the CVS
164			\texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
165			they create can be changed to a different name following the check-out:
166			\begin{verbatim}
167			% cvs co MITgcm_verif_basic
168			% mv MITgcm MITgcm_verif_basic
169			\end{verbatim}
170	cnh	1.7
171	adcroft	1.1
172	edhill	1.19	\subsubsection{Conventional download method}
173	adcroft	1.4	\label{sect:conventionalDownload}
174	adcroft	1.1
175	adcroft	1.4	If you do not have CVS on your system, you can download the model as a
176	edhill	1.15	tar file from the web site at:
177	cnh	1.7	\begin{rawhtml} <A href=http://mitgcm.org/download target="idontexist"> \end{rawhtml}
178	adcroft	1.1	\begin{verbatim}
179			http://mitgcm.org/download/
180			\end{verbatim}
181	cnh	1.7	\begin{rawhtml} </A> \end{rawhtml}
182	adcroft	1.4	The tar file still contains CVS information which we urge you not to
183			delete; even if you do not use CVS yourself the information can help
184	edhill	1.15	us if you should need to send us your copy of the code. If a recent
185			tar file does not exist, then please contact the developers through
186	edhill	1.17	the
187			\begin{rawhtml} <A href=''mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
188			MITgcm-support@mitgcm.org
189			\begin{rawhtml} </A> \end{rawhtml}
190			mailing list.
191	adcroft	1.1
192	edhill	1.19	\subsubsection{Upgrading from an earlier version}
193	adcroft	1.12
194			If you already have an earlier version of the code you can ``upgrade''
195			your copy instead of downloading the entire repository again. First,
196			``cd'' (change directory) to the top of your working copy:
197			\begin{verbatim}
198			% cd MITgcm
199			\end{verbatim}
200	edhill	1.15	and then issue the cvs update command such as:
201	adcroft	1.12	\begin{verbatim}
202	edhill	1.16	% cvs -q update -r checkpoint52i_post -d -P
203	adcroft	1.12	\end{verbatim}
204	edhill	1.16	This will update the ``tag'' to ``checkpoint52i\_post'', add any new
205	adcroft	1.12	directories (-d) and remove any empty directories (-P). The -q option
206			means be quiet which will reduce the number of messages you'll see in
207			the terminal. If you have modified the code prior to upgrading, CVS
208			will try to merge your changes with the upgrades. If there is a
209			conflict between your modifications and the upgrade, it will report
210			that file with a ``C'' in front, e.g.:
211			\begin{verbatim}
212			C model/src/ini_parms.F
213			\end{verbatim}
214			If the list of conflicts scrolled off the screen, you can re-issue the
215			cvs update command and it will report the conflicts. Conflicts are
216	edhill	1.15	indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
217			``$>>>>>>>$''. For example,
218	edhill	1.17	{\small
219	adcroft	1.12	\begin{verbatim}
220			<<<<<<< ini_parms.F
221			& bottomDragLinear,myOwnBottomDragCoefficient,
222			=======
223			& bottomDragLinear,bottomDragQuadratic,
224			>>>>>>> 1.18
225			\end{verbatim}
226	edhill	1.17	}
227	adcroft	1.12	means that you added ``myOwnBottomDragCoefficient'' to a namelist at
228			the same time and place that we added ``bottomDragQuadratic''. You
229			need to resolve this conflict and in this case the line should be
230			changed to:
231	edhill	1.17	{\small
232	adcroft	1.12	\begin{verbatim}
233			& bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
234			\end{verbatim}
235	edhill	1.17	}
236	edhill	1.15	and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
237	adcroft	1.12	Unless you are making modifications which exactly parallel
238			developments we make, these types of conflicts should be rare.
239
240			\paragraph*{Upgrading to the current pre-release version}
241
242			We don't make a ``release'' for every little patch and bug fix in
243			order to keep the frequency of upgrades to a minimum. However, if you
244			have run into a problem for which ``we have already fixed in the
245			latest code'' and we haven't made a ``tag'' or ``release'' since that
246			patch then you'll need to get the latest code:
247			\begin{verbatim}
248			% cvs -q update -A -d -P
249			\end{verbatim}
250			Unlike, the ``check-out'' and ``update'' procedures above, there is no
251			``tag'' or release name. The -A tells CVS to upgrade to the
252			very latest version. As a rule, we don't recommend this since you
253			might upgrade while we are in the processes of checking in the code so
254			that you may only have part of a patch. Using this method of updating
255			also means we can't tell what version of the code you are working
256			with. So please be sure you understand what you're doing.
257
258	adcroft	1.4	\section{Model and directory structure}
259	adcroft	1.1
260	adcroft	1.12	The ``numerical'' model is contained within a execution environment
261			support wrapper. This wrapper is designed to provide a general
262			framework for grid-point models. MITgcmUV is a specific numerical
263			model that uses the framework. Under this structure the model is split
264			into execution environment support code and conventional numerical
265			model code. The execution environment support code is held under the
266			\textit{eesupp} directory. The grid point model code is held under the
267			\textit{model} directory. Code execution actually starts in the
268			\textit{eesupp} routines and not in the \textit{model} routines. For
269	edhill	1.17	this reason the top-level \textit{MAIN.F} is in the
270			\textit{eesupp/src} directory. In general, end-users should not need
271			to worry about this level. The top-level routine for the numerical
272			part of the code is in \textit{model/src/THE\_MODEL\_MAIN.F}. Here is
273			a brief description of the directory structure of the model under the
274			root tree (a detailed description is given in section 3: Code
275			structure).
276	adcroft	1.1
277			\begin{itemize}
278
279	edhill	1.17	\item \textit{bin}: this directory is initially empty. It is the
280			default directory in which to compile the code.
281
282	adcroft	1.1	\item \textit{diags}: contains the code relative to time-averaged
283	edhill	1.17	diagnostics. It is subdivided into two subdirectories \textit{inc}
284			and \textit{src} that contain include files (*.\textit{h} files) and
285			Fortran subroutines (*.\textit{F} files), respectively.
286	adcroft	1.1
287			\item \textit{doc}: contains brief documentation notes.
288	edhill	1.17
289			\item \textit{eesupp}: contains the execution environment source code.
290			Also subdivided into two subdirectories \textit{inc} and
291			\textit{src}.
292
293			\item \textit{exe}: this directory is initially empty. It is the
294			default directory in which to execute the code.
295
296			\item \textit{model}: this directory contains the main source code.
297			Also subdivided into two subdirectories \textit{inc} and
298			\textit{src}.
299
300			\item \textit{pkg}: contains the source code for the packages. Each
301			package corresponds to a subdirectory. For example, \textit{gmredi}
302			contains the code related to the Gent-McWilliams/Redi scheme,
303			\textit{aim} the code relative to the atmospheric intermediate
304			physics. The packages are described in detail in section 3.
305
306			\item \textit{tools}: this directory contains various useful tools.
307			For example, \textit{genmake2} is a script written in csh (C-shell)
308			that should be used to generate your makefile. The directory
309			\textit{adjoint} contains the makefile specific to the Tangent
310			linear and Adjoint Compiler (TAMC) that generates the adjoint code.
311			The latter is described in details in part V.
312
313	adcroft	1.1	\item \textit{utils}: this directory contains various utilities. The
314	edhill	1.17	subdirectory \textit{knudsen2} contains code and a makefile that
315			compute coefficients of the polynomial approximation to the knudsen
316			formula for an ocean nonlinear equation of state. The
317			\textit{matlab} subdirectory contains matlab scripts for reading
318			model output directly into matlab. \textit{scripts} contains C-shell
319			post-processing scripts for joining processor-based and tiled-based
320			model output.
321
322			\item \textit{verification}: this directory contains the model
323			examples. See section \ref{sect:modelExamples}.
324	adcroft	1.1
325			\end{itemize}
326
327	adcroft	1.4	\section{Example experiments}
328			\label{sect:modelExamples}
329	adcroft	1.1
330	edhill	1.15	%% a set of twenty-four pre-configured numerical experiments
331
332			The MITgcm distribution comes with more than a dozen pre-configured
333			numerical experiments. Some of these example experiments are tests of
334			individual parts of the model code, but many are fully fledged
335			numerical simulations. A few of the examples are used for tutorial
336			documentation in sections \ref{sect:eg-baro} - \ref{sect:eg-global}.
337			The other examples follow the same general structure as the tutorial
338			examples. However, they only include brief instructions in a text file
339			called {\it README}. The examples are located in subdirectories under
340			the directory \textit{verification}. Each example is briefly described
341			below.
342	adcroft	1.1
343	cnh	1.8	\subsection{Full list of model examples}
344	adcroft	1.1
345	cnh	1.8	\begin{enumerate}
346	edhill	1.17
347	adcroft	1.1	\item \textit{exp0} - single layer, ocean double gyre (barotropic with
348	edhill	1.15	free-surface). This experiment is described in detail in section
349			\ref{sect:eg-baro}.
350	adcroft	1.1
351	edhill	1.15	\item \textit{exp1} - Four layer, ocean double gyre. This experiment
352			is described in detail in section \ref{sect:eg-baroc}.
353
354	adcroft	1.1	\item \textit{exp2} - 4x4 degree global ocean simulation with steady
355	edhill	1.15	climatological forcing. This experiment is described in detail in
356			section \ref{sect:eg-global}.
357
358			\item \textit{exp4} - Flow over a Gaussian bump in open-water or
359			channel with open boundaries.
360
361			\item \textit{exp5} - Inhomogenously forced ocean convection in a
362			doubly periodic box.
363	adcroft	1.1
364	cnh	1.8	\item \textit{front\_relax} - Relaxation of an ocean thermal front (test for
365	adcroft	1.1	Gent/McWilliams scheme). 2D (Y-Z).
366
367	edhill	1.15	\item \textit{internal wave} - Ocean internal wave forced by open
368			boundary conditions.
369
370	cnh	1.8	\item \textit{natl\_box} - Eastern subtropical North Atlantic with KPP
371	edhill	1.15	scheme; 1 month integration
372
373			\item \textit{hs94.1x64x5} - Zonal averaged atmosphere using Held and
374			Suarez '94 forcing.
375
376			\item \textit{hs94.128x64x5} - 3D atmosphere dynamics using Held and
377			Suarez '94 forcing.
378
379	adcroft	1.1	\item \textit{hs94.cs-32x32x5} - 3D atmosphere dynamics using Held and
380	edhill	1.15	Suarez '94 forcing on the cubed sphere.
381
382			\item \textit{aim.5l\_zon-ave} - Intermediate Atmospheric physics.
383			Global Zonal Mean configuration, 1x64x5 resolution.
384
385			\item \textit{aim.5l\_XZ\_Equatorial\_Slice} - Intermediate
386			Atmospheric physics, equatorial Slice configuration. 2D (X-Z).
387
388	adcroft	1.1	\item \textit{aim.5l\_Equatorial\_Channel} - Intermediate Atmospheric
389	edhill	1.15	physics. 3D Equatorial Channel configuration.
390
391	cnh	1.8	\item \textit{aim.5l\_LatLon} - Intermediate Atmospheric physics.
392	edhill	1.15	Global configuration, on latitude longitude grid with 128x64x5 grid
393			points ($2.8^\circ{\rm degree}$ resolution).
394
395			\item \textit{adjustment.128x64x1} Barotropic adjustment problem on
396			latitude longitude grid with 128x64 grid points ($2.8^\circ{\rm
397			degree}$ resolution).
398
399			\item \textit{adjustment.cs-32x32x1} Barotropic adjustment problem on
400			cube sphere grid with 32x32 points per face ( roughly $2.8^\circ{\rm
401			degree}$ resolution).
402
403	cnh	1.8	\item \textit{advect\_cs} Two-dimensional passive advection test on
404	edhill	1.15	cube sphere grid.
405
406			\item \textit{advect\_xy} Two-dimensional (horizontal plane) passive
407			advection test on Cartesian grid.
408
409			\item \textit{advect\_yz} Two-dimensional (vertical plane) passive
410			advection test on Cartesian grid.
411
412			\item \textit{carbon} Simple passive tracer experiment. Includes
413			derivative calculation. Described in detail in section
414			\ref{sect:eg-carbon-ad}.
415	cnh	1.8
416			\item \textit{flt\_example} Example of using float package.
417	edhill	1.15
418			\item \textit{global\_ocean.90x40x15} Global circulation with GM, flux
419			boundary conditions and poles.
420	cnh	1.8
421	mlosch	1.13	\item \textit{global\_ocean\_pressure} Global circulation in pressure
422			coordinate (non-Boussinesq ocean model). Described in detail in
423			section \ref{sect:eg-globalpressure}.
424	edhill	1.15
425			\item \textit{solid-body.cs-32x32x1} Solid body rotation test for cube
426			sphere grid.
427	cnh	1.8
428			\end{enumerate}
429	adcroft	1.1
430	adcroft	1.4	\subsection{Directory structure of model examples}
431	adcroft	1.1
432			Each example directory has the following subdirectories:
433
434			\begin{itemize}
435			\item \textit{code}: contains the code particular to the example. At a
436	edhill	1.16	minimum, this directory includes the following files:
437	adcroft	1.1
438	edhill	1.16	\begin{itemize}
439			\item \textit{code/CPP\_EEOPTIONS.h}: declares CPP keys relative to
440			the ``execution environment'' part of the code. The default
441			version is located in \textit{eesupp/inc}.
442
443			\item \textit{code/CPP\_OPTIONS.h}: declares CPP keys relative to
444			the ``numerical model'' part of the code. The default version is
445			located in \textit{model/inc}.
446
447			\item \textit{code/SIZE.h}: declares size of underlying
448			computational grid. The default version is located in
449			\textit{model/inc}.
450			\end{itemize}
451
452			In addition, other include files and subroutines might be present in
453			\textit{code} depending on the particular experiment. See Section 2
454			for more details.
455	edhill	1.15
456			\item \textit{input}: contains the input data files required to run
457			the example. At a minimum, the \textit{input} directory contains the
458			following files:
459	adcroft	1.1
460	edhill	1.16	\begin{itemize}
461			\item \textit{input/data}: this file, written as a namelist,
462			specifies the main parameters for the experiment.
463
464			\item \textit{input/data.pkg}: contains parameters relative to the
465			packages used in the experiment.
466
467			\item \textit{input/eedata}: this file contains ``execution
468			environment'' data. At present, this consists of a specification
469			of the number of threads to use in $X$ and $Y$ under multithreaded
470			execution.
471			\end{itemize}
472	edhill	1.17
473			In addition, you will also find in this directory the forcing and
474			topography files as well as the files describing the initial state
475			of the experiment. This varies from experiment to experiment. See
476			section 2 for more details.
477	edhill	1.16
478			\item \textit{results}: this directory contains the output file
479			\textit{output.txt} produced by the simulation example. This file is
480			useful for comparison with your own output when you run the
481			experiment.
482	adcroft	1.1	\end{itemize}
483
484	edhill	1.17	Once you have chosen the example you want to run, you are ready to
485			compile the code.
486	adcroft	1.1
487	adcroft	1.4	\section{Building the code}
488			\label{sect:buildingCode}
489
490			To compile the code, we use the {\em make} program. This uses a file
491			({\em Makefile}) that allows us to pre-process source files, specify
492			compiler and optimization options and also figures out any file
493	edhill	1.16	dependencies. We supply a script ({\em genmake2}), described in
494			section \ref{sect:genmake}, that automatically creates the {\em
495			Makefile} for you. You then need to build the dependencies and
496			compile the code.
497	adcroft	1.4
498			As an example, let's assume that you want to build and run experiment
499	edhill	1.16	\textit{verification/exp2}. The are multiple ways and places to
500			actually do this but here let's build the code in
501	adcroft	1.4	\textit{verification/exp2/input}:
502			\begin{verbatim}
503			% cd verification/exp2/input
504			\end{verbatim}
505			First, build the {\em Makefile}:
506			\begin{verbatim}
507	edhill	1.16	% ../../../tools/genmake2 -mods=../code
508	adcroft	1.4	\end{verbatim}
509			The command line option tells {\em genmake} to override model source
510			code with any files in the directory {\em ./code/}.
511
512	edhill	1.16	On many systems, the {\em genmake2} program will be able to
513			automatically recognize the hardware, find compilers and other tools
514			within the user's path (``echo \$PATH''), and then choose an
515			appropriate set of options from the files contained in the {\em
516			tools/build\_options} directory. Under some circumstances, a user
517			may have to create a new ``optfile'' in order to specify the exact
518			combination of compiler, compiler flags, libraries, and other options
519			necessary to build a particular configuration of MITgcm. In such
520			cases, it is generally helpful to read the existing ``optfiles'' and
521			mimic their syntax.
522
523			Through the MITgcm-support list, the MITgcm developers are willing to
524			provide help writing or modifing ``optfiles''. And we encourage users
525			to post new ``optfiles'' (particularly ones for new machines or
526	edhill	1.17	architectures) to the
527			\begin{rawhtml} <A href=''mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
528			MITgcm-support@mitgcm.org
529			\begin{rawhtml} </A> \end{rawhtml}
530			list.
531	edhill	1.16
532			To specify an optfile to {\em genmake2}, the syntax is:
533	adcroft	1.4	\begin{verbatim}
534	edhill	1.16	% ../../../tools/genmake2 -mods=../code -of /path/to/optfile
535	adcroft	1.4	\end{verbatim}
536
537	edhill	1.16	Once a {\em Makefile} has been generated, we create the dependencies:
538	adcroft	1.4	\begin{verbatim}
539			% make depend
540			\end{verbatim}
541	edhill	1.16	This modifies the {\em Makefile} by attaching a [long] list of files
542			upon which other files depend. The purpose of this is to reduce
543			re-compilation if and when you start to modify the code. The {\tt make
544			depend} command also creates links from the model source to this
545			directory.
546	adcroft	1.1
547	edhill	1.16	Next compile the code:
548	adcroft	1.4	\begin{verbatim}
549			% make
550			\end{verbatim}
551			The {\tt make} command creates an executable called \textit{mitgcmuv}.
552	edhill	1.16	Additional make ``targets'' are defined within the makefile to aid in
553			the production of adjoint and other versions of MITgcm.
554	adcroft	1.4
555			Now you are ready to run the model. General instructions for doing so are
556			given in section \ref{sect:runModel}. Here, we can run the model with:
557			\begin{verbatim}
558			./mitgcmuv > output.txt
559			\end{verbatim}
560			where we are re-directing the stream of text output to the file {\em
561			output.txt}.
562
563
564			\subsection{Building/compiling the code elsewhere}
565
566			In the example above (section \ref{sect:buildingCode}) we built the
567			executable in the {\em input} directory of the experiment for
568			convenience. You can also configure and compile the code in other
569			locations, for example on a scratch disk with out having to copy the
570			entire source tree. The only requirement to do so is you have {\tt
571	edhill	1.16	genmake2} in your path or you know the absolute path to {\tt
572			genmake2}.
573	adcroft	1.4
574	edhill	1.16	The following sections outline some possible methods of organizing
575			your source and data.
576	adcroft	1.4
577			\subsubsection{Building from the {\em ../code directory}}
578
579			This is just as simple as building in the {\em input/} directory:
580			\begin{verbatim}
581			% cd verification/exp2/code
582	edhill	1.16	% ../../../tools/genmake2
583	adcroft	1.4	% make depend
584			% make
585			\end{verbatim}
586			However, to run the model the executable ({\em mitgcmuv}) and input
587			files must be in the same place. If you only have one calculation to make:
588			\begin{verbatim}
589			% cd ../input
590			% cp ../code/mitgcmuv ./
591			% ./mitgcmuv > output.txt
592			\end{verbatim}
593	cnh	1.9	or if you will be making multiple runs with the same executable:
594	adcroft	1.4	\begin{verbatim}
595			% cd ../
596			% cp -r input run1
597			% cp code/mitgcmuv run1
598			% cd run1
599			% ./mitgcmuv > output.txt
600			\end{verbatim}
601
602			\subsubsection{Building from a new directory}
603
604			Since the {\em input} directory contains input files it is often more
605	cnh	1.9	useful to keep {\em input} pristine and build in a new directory
606	adcroft	1.4	within {\em verification/exp2/}:
607			\begin{verbatim}
608			% cd verification/exp2
609			% mkdir build
610			% cd build
611	edhill	1.16	% ../../../tools/genmake2 -mods=../code
612	adcroft	1.4	% make depend
613			% make
614			\end{verbatim}
615			This builds the code exactly as before but this time you need to copy
616			either the executable or the input files or both in order to run the
617			model. For example,
618			\begin{verbatim}
619			% cp ../input/* ./
620			% ./mitgcmuv > output.txt
621			\end{verbatim}
622			or if you tend to make multiple runs with the same executable then
623			running in a new directory each time might be more appropriate:
624			\begin{verbatim}
625			% cd ../
626			% mkdir run1
627			% cp build/mitgcmuv run1/
628			% cp input/* run1/
629			% cd run1
630			% ./mitgcmuv > output.txt
631			\end{verbatim}
632
633	edhill	1.16	\subsubsection{Building on a scratch disk}
634	adcroft	1.4
635			Model object files and output data can use up large amounts of disk
636			space so it is often the case that you will be operating on a large
637			scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
638			following commands will build the model in {\em /scratch/exp2-run1}:
639			\begin{verbatim}
640			% cd /scratch/exp2-run1
641	edhill	1.16	% ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
642			-mods=~/MITgcm/verification/exp2/code
643	adcroft	1.4	% make depend
644			% make
645			\end{verbatim}
646			To run the model here, you'll need the input files:
647			\begin{verbatim}
648			% cp ~/MITgcm/verification/exp2/input/* ./
649			% ./mitgcmuv > output.txt
650			\end{verbatim}
651
652			As before, you could build in one directory and make multiple runs of
653			the one experiment:
654			\begin{verbatim}
655			% cd /scratch/exp2
656			% mkdir build
657			% cd build
658	edhill	1.16	% ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
659			-mods=~/MITgcm/verification/exp2/code
660	adcroft	1.4	% make depend
661			% make
662			% cd ../
663			% cp -r ~/MITgcm/verification/exp2/input run2
664			% cd run2
665			% ./mitgcmuv > output.txt
666			\end{verbatim}
667
668
669	edhill	1.16	\subsection{Using \textit{genmake2}}
670	adcroft	1.4	\label{sect:genmake}
671	adcroft	1.1
672	edhill	1.16	To compile the code, first use the program \texttt{genmake2} (located
673			in the \textit{tools} directory) to generate a Makefile.
674			\texttt{genmake2} is a shell script written to work with all
675			``sh''--compatible shells including bash v1, bash v2, and Bourne.
676			Internally, \texttt{genmake2} determines the locations of needed
677			files, the compiler, compiler options, libraries, and Unix tools. It
678			relies upon a number of ``optfiles'' located in the {\em
679			tools/build\_options} directory.
680
681			The purpose of the optfiles is to provide all the compilation options
682			for particular ``platforms'' (where ``platform'' roughly means the
683			combination of the hardware and the compiler) and code configurations.
684			Given the combinations of possible compilers and library dependencies
685			({\it eg.} MPI and NetCDF) there may be numerous optfiles available
686			for a single machine. The naming scheme for the majority of the
687			optfiles shipped with the code is
688			\begin{center}
689			{\bf OS\_HARDWARE\_COMPILER }
690			\end{center}
691			where
692			\begin{description}
693			\item[OS] is the name of the operating system (generally the
694			lower-case output of the {\tt 'uname'} command)
695			\item[HARDWARE] is a string that describes the CPU type and
696			corresponds to output from the {\tt 'uname -m'} command:
697			\begin{description}
698			\item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
699			and athlon
700			\item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
701			\item[amd64] is AMD x86\_64 systems
702			\item[ppc] is for Mac PowerPC systems
703			\end{description}
704			\item[COMPILER] is the compiler name (generally, the name of the
705			FORTRAN executable)
706			\end{description}
707
708			In many cases, the default optfiles are sufficient and will result in
709			usable Makefiles. However, for some machines or code configurations,
710			new ``optfiles'' must be written. To create a new optfile, it is
711			generally best to start with one of the defaults and modify it to suit
712			your needs. Like \texttt{genmake2}, the optfiles are all written
713			using a simple ``sh''--compatible syntax. While nearly all variables
714			used within \texttt{genmake2} may be specified in the optfiles, the
715			critical ones that should be defined are:
716
717			\begin{description}
718			\item[FC] the FORTRAN compiler (executable) to use
719			\item[DEFINES] the command-line DEFINE options passed to the compiler
720			\item[CPP] the C pre-processor to use
721			\item[NOOPTFLAGS] options flags for special files that should not be
722			optimized
723			\end{description}
724
725			For example, the optfile for a typical Red Hat Linux machine (``ia32''
726			architecture) using the GCC (g77) compiler is
727			\begin{verbatim}
728			FC=g77
729			DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
730			CPP='cpp -traditional -P'
731			NOOPTFLAGS='-O0'
732			# For IEEE, use the "-ffloat-store" option
733			if test "x$IEEE" = x ; then
734			FFLAGS='-Wimplicit -Wunused -Wuninitialized'
735			FOPTIM='-O3 -malign-double -funroll-loops'
736			else
737			FFLAGS='-Wimplicit -Wunused -ffloat-store'
738			FOPTIM='-O0 -malign-double'
739			fi
740			\end{verbatim}
741
742			If you write an optfile for an unrepresented machine or compiler, you
743			are strongly encouraged to submit the optfile to the MITgcm project
744			for inclusion. Please send the file to the
745			\begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
746			\begin{center}
747			MITgcm-support@mitgcm.org
748			\end{center}
749			\begin{rawhtml} </A> \end{rawhtml}
750			mailing list.
751	adcroft	1.1
752	edhill	1.16	In addition to the optfiles, \texttt{genmake2} supports a number of
753			helpful command-line options. A complete list of these options can be
754			obtained from:
755			\begin{verbatim}
756			% genmake2 -h
757			\end{verbatim}
758
759			The most important command-line options are:
760			\begin{description}
761
762	edhill	1.17	\item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
763			should be used for a particular build.
764	edhill	1.16
765			If no "optfile" is specified (either through the command line or the
766			MITGCM\_OPTFILE environment variable), genmake2 will try to make a
767			reasonable guess from the list provided in {\em
768			tools/build\_options}. The method used for making this guess is
769			to first determine the combination of operating system and hardware
770			(eg. "linux\_ia32") and then find a working FORTRAN compiler within
771			the user's path. When these three items have been identified,
772			genmake2 will try to find an optfile that has a matching name.
773
774	edhill	1.22	\item[\texttt{--pdefault='PKG1 PKG2 PKG3 ...'}] specifies the default
775			set of packages to be used. The normal order of precedence for
776			packages is as follows:
777			\begin{enumerate}
778			\item If available, the command line (\texttt{--pdefault}) settings
779			over-rule any others.
780
781			\item Next, \texttt{genmake2} will look for a file named
782			``\texttt{packages.conf}'' in the local directory or in any of the
783			directories specified with the \texttt{--mods} option.
784
785			\item Finally, if neither of the above are available,
786			\texttt{genmake2} will use the \texttt{/pkg/pkg\_default} file.
787			\end{enumerate}
788
789	edhill	1.17	\item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
790			used for packages.
791	edhill	1.16
792			If not specified, the default dependency file {\em pkg/pkg\_depend}
793			is used. The syntax for this file is parsed on a line-by-line basis
794			where each line containes either a comment ("\#") or a simple
795			"PKGNAME1 (+\|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
796			specifies a "must be used with" or a "must not be used with"
797			relationship, respectively. If no rule is specified, then it is
798			assumed that the two packages are compatible and will function
799			either with or without each other.
800
801	edhill	1.17	\item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
802			automatic differentiation options file to be used. The file is
803			analogous to the ``optfile'' defined above but it specifies
804			information for the AD build process.
805	edhill	1.16
806			The default file is located in {\em
807			tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
808			and "TAMC" compilers. An alternate version is also available at
809			{\em tools/adjoint\_options/adjoint\_staf} that selects the newer
810			"STAF" compiler. As with any compilers, it is helpful to have their
811			directories listed in your {\tt \$PATH} environment variable.
812
813	edhill	1.17	\item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
814			directories containing ``modifications''. These directories contain
815			files with names that may (or may not) exist in the main MITgcm
816			source tree but will be overridden by any identically-named sources
817			within the ``MODS'' directories.
818	edhill	1.16
819			The order of precedence for this "name-hiding" is as follows:
820			\begin{itemize}
821			\item ``MODS'' directories (in the order given)
822			\item Packages either explicitly specified or provided by default
823			(in the order given)
824			\item Packages included due to package dependencies (in the order
825			that that package dependencies are parsed)
826			\item The "standard dirs" (which may have been specified by the
827			``-standarddirs'' option)
828			\end{itemize}
829
830	edhill	1.23	\item[\texttt{--mpi}] This option enables certain MPI features (using
831			CPP \texttt{\#define}s) within the code and is necessary for MPI
832			builds (see Section \ref{sect:mpi-build}).
833
834	edhill	1.17	\item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
835			soft-links and other bugs common with the \texttt{make} versions
836			provided by commercial Unix vendors, GNU \texttt{make} (sometimes
837			called \texttt{gmake}) should be preferred. This option provides a
838			means for specifying the make executable to be used.
839	edhill	1.21
840			\item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
841			machines, the ``bash'' shell is unavailable. To run on these
842			systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
843			a Bourne, POSIX, or compatible) shell. The syntax in these
844			circumstances is:
845			\begin{center}
846	edhill	1.23	\texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
847	edhill	1.21	\end{center}
848			where \texttt{/bin/sh} can be replaced with the full path and name
849			of the desired shell.
850	adcroft	1.1
851	edhill	1.16	\end{description}
852	adcroft	1.1
853
854	edhill	1.23	\subsection{Building with MPI}
855			\label{sect:mpi-build}
856
857			Building MITgcm to use MPI libraries can be complicated due to the
858			variety of different MPI implementations available, their dependencies
859			or interactions with different compilers, and their often ad-hoc
860			locations within file systems. For these reasons, its generally a
861			good idea to start by finding and reading the documentation for your
862			machine(s) and, if necessary, seeking help from your local systems
863			administrator.
864
865			The steps for building MITgcm with MPI support are:
866			\begin{enumerate}
867
868			\item Determine the locations of your MPI-enabled compiler and/or MPI
869			libraries and put them into an options file as described in Section
870			\ref{sect:genmake}. One can start with one of the examples in:
871			\begin{rawhtml} <A
872			href="http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/tools/build_options/">
873			\end{rawhtml}
874			\begin{center}
875			\texttt{MITgcm/tools/build\_options/}
876			\end{center}
877			\begin{rawhtml} </A> \end{rawhtml}
878			such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
879			\texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
880			hand. You may need help from your user guide or local systems
881			administrator to determine the exact location of the MPI libraries.
882			If libraries are not installed, MPI implementations and related
883			tools are available including:
884			\begin{itemize}
885			\item \begin{rawhtml} <A
886			href="http://www-unix.mcs.anl.gov/mpi/mpich/">
887			\end{rawhtml}
888			MPICH
889			\begin{rawhtml} </A> \end{rawhtml}
890
891			\item \begin{rawhtml} <A
892			href="http://www.lam-mpi.org/">
893			\end{rawhtml}
894			LAM/MPI
895			\begin{rawhtml} </A> \end{rawhtml}
896
897			\item \begin{rawhtml} <A
898			href="http://www.osc.edu/~pw/mpiexec/">
899			\end{rawhtml}
900			MPIexec
901			\begin{rawhtml} </A> \end{rawhtml}
902			\end{itemize}
903
904			\item Build the code with the \texttt{genmake2} \texttt{-mpi} option
905			(see Section \ref{sect:genmake}) using commands such as:
906			{\footnotesize \begin{verbatim}
907			% ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
908			% make depend
909			% make
910			\end{verbatim} }
911
912			\item Run the code with the appropriate MPI ``run'' or ``exec''
913			program provided with your particular implementation of MPI.
914			Typical MPI packages such as MPICH will use something like:
915			\begin{verbatim}
916			% mpirun -np 4 -machinefile mf ./mitgcmuv
917			\end{verbatim}
918			Sightly more complicated scripts may be needed for many machines
919			since execution of the code may be controlled by both the MPI
920			library and a job scheduling and queueing system such as PBS,
921			LoadLeveller, Condor, or any of a number of similar tools.
922
923			\end{enumerate}
924
925
926	adcroft	1.1
927	adcroft	1.4	\section{Running the model}
928			\label{sect:runModel}
929
930	edhill	1.23	If compilation finished succesfuully (section \ref{sect:buildingCode})
931			then an executable called \texttt{mitgcmuv} will now exist in the
932			local directory.
933	adcroft	1.1
934	adcroft	1.4	To run the model as a single process (ie. not in parallel) simply
935			type:
936	adcroft	1.1	\begin{verbatim}
937	adcroft	1.4	% ./mitgcmuv
938	adcroft	1.1	\end{verbatim}
939	adcroft	1.4	The ``./'' is a safe-guard to make sure you use the local executable
940			in case you have others that exist in your path (surely odd if you
941			do!). The above command will spew out many lines of text output to
942			your screen. This output contains details such as parameter values as
943			well as diagnostics such as mean Kinetic energy, largest CFL number,
944			etc. It is worth keeping this text output with the binary output so we
945			normally re-direct the {\em stdout} stream as follows:
946	adcroft	1.1	\begin{verbatim}
947	adcroft	1.4	% ./mitgcmuv > output.txt
948	adcroft	1.1	\end{verbatim}
949
950	edhill	1.17	For the example experiments in {\em verification}, an example of the
951	adcroft	1.4	output is kept in {\em results/output.txt} for comparison. You can compare
952			your {\em output.txt} with this one to check that the set-up works.
953	adcroft	1.1
954
955
956	adcroft	1.4	\subsection{Output files}
957	adcroft	1.1
958			The model produces various output files. At a minimum, the instantaneous
959			``state'' of the model is written out, which is made of the following files:
960
961			\begin{itemize}
962			\item \textit{U.00000nIter} - zonal component of velocity field (m/s and $>
963			0 $ eastward).
964
965			\item \textit{V.00000nIter} - meridional component of velocity field (m/s
966			and $> 0$ northward).
967
968			\item \textit{W.00000nIter} - vertical component of velocity field (ocean:
969			m/s and $> 0$ upward, atmosphere: Pa/s and $> 0$ towards increasing pressure
970			i.e. downward).
971
972			\item \textit{T.00000nIter} - potential temperature (ocean: $^{0}$C,
973			atmosphere: $^{0}$K).
974
975			\item \textit{S.00000nIter} - ocean: salinity (psu), atmosphere: water vapor
976			(g/kg).
977
978			\item \textit{Eta.00000nIter} - ocean: surface elevation (m), atmosphere:
979			surface pressure anomaly (Pa).
980			\end{itemize}
981
982			The chain \textit{00000nIter} consists of ten figures that specify the
983			iteration number at which the output is written out. For example, \textit{%
984			U.0000000300} is the zonal velocity at iteration 300.
985
986			In addition, a ``pickup'' or ``checkpoint'' file called:
987
988			\begin{itemize}
989			\item \textit{pickup.00000nIter}
990			\end{itemize}
991
992			is written out. This file represents the state of the model in a condensed
993			form and is used for restarting the integration. If the C-D scheme is used,
994			there is an additional ``pickup'' file:
995
996			\begin{itemize}
997			\item \textit{pickup\_cd.00000nIter}
998			\end{itemize}
999
1000			containing the D-grid velocity data and that has to be written out as well
1001			in order to restart the integration. Rolling checkpoint files are the same
1002			as the pickup files but are named differently. Their name contain the chain
1003			\textit{ckptA} or \textit{ckptB} instead of \textit{00000nIter}. They can be
1004			used to restart the model but are overwritten every other time they are
1005			output to save disk space during long integrations.
1006
1007	adcroft	1.4	\subsection{Looking at the output}
1008	adcroft	1.1
1009			All the model data are written according to a ``meta/data'' file format.
1010			Each variable is associated with two files with suffix names \textit{.data}
1011			and \textit{.meta}. The \textit{.data} file contains the data written in
1012			binary form (big\_endian by default). The \textit{.meta} file is a
1013			``header'' file that contains information about the size and the structure
1014			of the \textit{.data} file. This way of organizing the output is
1015			particularly useful when running multi-processors calculations. The base
1016			version of the model includes a few matlab utilities to read output files
1017			written in this format. The matlab scripts are located in the directory
1018			\textit{utils/matlab} under the root tree. The script \textit{rdmds.m} reads
1019			the data. Look at the comments inside the script to see how to use it.
1020
1021	adcroft	1.4	Some examples of reading and visualizing some output in {\em Matlab}:
1022			\begin{verbatim}
1023			% matlab
1024			>> H=rdmds('Depth');
1025			>> contourf(H');colorbar;
1026			>> title('Depth of fluid as used by model');
1027
1028			>> eta=rdmds('Eta',10);
1029			>> imagesc(eta');axis ij;colorbar;
1030			>> title('Surface height at iter=10');
1031
1032			>> eta=rdmds('Eta',[0:10:100]);
1033			>> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
1034			\end{verbatim}
1035	adcroft	1.1
1036			\section{Doing it yourself: customizing the code}
1037
1038			When you are ready to run the model in the configuration you want, the
1039	edhill	1.17	easiest thing is to use and adapt the setup of the case studies
1040			experiment (described previously) that is the closest to your
1041			configuration. Then, the amount of setup will be minimized. In this
1042			section, we focus on the setup relative to the ``numerical model''
1043			part of the code (the setup relative to the ``execution environment''
1044			part is covered in the parallel implementation section) and on the
1045			variables and parameters that you are likely to change.
1046	adcroft	1.1
1047	adcroft	1.5	\subsection{Configuration and setup}
1048	adcroft	1.4
1049	edhill	1.17	The CPP keys relative to the ``numerical model'' part of the code are
1050			all defined and set in the file \textit{CPP\_OPTIONS.h }in the
1051			directory \textit{ model/inc }or in one of the \textit{code
1052			}directories of the case study experiments under
1053			\textit{verification.} The model parameters are defined and declared
1054			in the file \textit{model/inc/PARAMS.h }and their default values are
1055			set in the routine \textit{model/src/set\_defaults.F. }The default
1056			values can be modified in the namelist file \textit{data }which needs
1057			to be located in the directory where you will run the model. The
1058			parameters are initialized in the routine
1059			\textit{model/src/ini\_parms.F}. Look at this routine to see in what
1060			part of the namelist the parameters are located.
1061
1062			In what follows the parameters are grouped into categories related to
1063			the computational domain, the equations solved in the model, and the
1064			simulation controls.
1065	adcroft	1.1
1066	adcroft	1.4	\subsection{Computational domain, geometry and time-discretization}
1067	adcroft	1.1
1068	edhill	1.17	\begin{description}
1069			\item[dimensions] \
1070
1071	edhill	1.18	The number of points in the x, y, and r directions are represented
1072			by the variables \textbf{sNx}, \textbf{sNy} and \textbf{Nr}
1073			respectively which are declared and set in the file
1074			\textit{model/inc/SIZE.h}. (Again, this assumes a mono-processor
1075			calculation. For multiprocessor calculations see the section on
1076			parallel implementation.)
1077	edhill	1.17
1078			\item[grid] \
1079
1080			Three different grids are available: cartesian, spherical polar, and
1081	edhill	1.18	curvilinear (which includes the cubed sphere). The grid is set
1082			through the logical variables \textbf{usingCartesianGrid},
1083			\textbf{usingSphericalPolarGrid}, and \textbf{usingCurvilinearGrid}.
1084			In the case of spherical and curvilinear grids, the southern
1085			boundary is defined through the variable \textbf{phiMin} which
1086			corresponds to the latitude of the southern most cell face (in
1087			degrees). The resolution along the x and y directions is controlled
1088			by the 1D arrays \textbf{delx} and \textbf{dely} (in meters in the
1089			case of a cartesian grid, in degrees otherwise). The vertical grid
1090			spacing is set through the 1D array \textbf{delz} for the ocean (in
1091			meters) or \textbf{delp} for the atmosphere (in Pa). The variable
1092			\textbf{Ro\_SeaLevel} represents the standard position of Sea-Level
1093			in ``R'' coordinate. This is typically set to 0m for the ocean
1094			(default value) and 10$^{5}$Pa for the atmosphere. For the
1095			atmosphere, also set the logical variable \textbf{groundAtK1} to
1096			\texttt{'.TRUE.'} which puts the first level (k=1) at the lower
1097	edhill	1.17	boundary (ground).
1098
1099			For the cartesian grid case, the Coriolis parameter $f$ is set
1100	edhill	1.18	through the variables \textbf{f0} and \textbf{beta} which correspond
1101			to the reference Coriolis parameter (in s$^{-1}$) and
1102			$\frac{\partial f}{ \partial y}$(in m$^{-1}$s$^{-1}$) respectively.
1103			If \textbf{beta } is set to a nonzero value, \textbf{f0} is the
1104			value of $f$ at the southern edge of the domain.
1105	edhill	1.17
1106			\item[topography - full and partial cells] \
1107
1108			The domain bathymetry is read from a file that contains a 2D (x,y)
1109			map of depths (in m) for the ocean or pressures (in Pa) for the
1110			atmosphere. The file name is represented by the variable
1111	edhill	1.18	\textbf{bathyFile}. The file is assumed to contain binary numbers
1112			giving the depth (pressure) of the model at each grid cell, ordered
1113			with the x coordinate varying fastest. The points are ordered from
1114			low coordinate to high coordinate for both axes. The model code
1115			applies without modification to enclosed, periodic, and double
1116			periodic domains. Periodicity is assumed by default and is
1117	edhill	1.17	suppressed by setting the depths to 0m for the cells at the limits
1118			of the computational domain (note: not sure this is the case for the
1119			atmosphere). The precision with which to read the binary data is
1120	edhill	1.18	controlled by the integer variable \textbf{readBinaryPrec} which can
1121	edhill	1.17	take the value \texttt{32} (single precision) or \texttt{64} (double
1122	edhill	1.18	precision). See the matlab program \textit{gendata.m} in the
1123			\textit{input} directories under \textit{verification} to see how
1124	edhill	1.17	the bathymetry files are generated for the case study experiments.
1125
1126	edhill	1.18	To use the partial cell capability, the variable \textbf{hFacMin}
1127			needs to be set to a value between 0 and 1 (it is set to 1 by
1128			default) corresponding to the minimum fractional size of the cell.
1129			For example if the bottom cell is 500m thick and \textbf{hFacMin} is
1130			set to 0.1, the actual thickness of the cell (i.e. used in the code)
1131			can cover a range of discrete values 50m apart from 50m to 500m
1132			depending on the value of the bottom depth (in \textbf{bathyFile})
1133			at this point.
1134	edhill	1.17
1135			Note that the bottom depths (or pressures) need not coincide with
1136	edhill	1.18	the models levels as deduced from \textbf{delz} or \textbf{delp}.
1137			The model will interpolate the numbers in \textbf{bathyFile} so that
1138			they match the levels obtained from \textbf{delz} or \textbf{delp}
1139			and \textbf{hFacMin}.
1140	edhill	1.17
1141			(Note: the atmospheric case is a bit more complicated than what is
1142			written here I think. To come soon...)
1143
1144			\item[time-discretization] \
1145
1146			The time steps are set through the real variables \textbf{deltaTMom}
1147			and \textbf{deltaTtracer} (in s) which represent the time step for
1148			the momentum and tracer equations, respectively. For synchronous
1149			integrations, simply set the two variables to the same value (or you
1150			can prescribe one time step only through the variable
1151			\textbf{deltaT}). The Adams-Bashforth stabilizing parameter is set
1152			through the variable \textbf{abEps} (dimensionless). The stagger
1153			baroclinic time stepping can be activated by setting the logical
1154	edhill	1.18	variable \textbf{staggerTimeStep} to \texttt{'.TRUE.'}.
1155	adcroft	1.1
1156	edhill	1.17	\end{description}
1157	adcroft	1.1
1158
1159	adcroft	1.4	\subsection{Equation of state}
1160	adcroft	1.1
1161	mlosch	1.13	First, because the model equations are written in terms of
1162			perturbations, a reference thermodynamic state needs to be specified.
1163			This is done through the 1D arrays \textbf{tRef} and \textbf{sRef}.
1164			\textbf{tRef} specifies the reference potential temperature profile
1165			(in $^{o}$C for the ocean and $^{o}$K for the atmosphere) starting
1166			from the level k=1. Similarly, \textbf{sRef} specifies the reference
1167			salinity profile (in ppt) for the ocean or the reference specific
1168			humidity profile (in g/kg) for the atmosphere.
1169
1170			The form of the equation of state is controlled by the character
1171			variables \textbf{buoyancyRelation} and \textbf{eosType}.
1172	edhill	1.18	\textbf{buoyancyRelation} is set to \texttt{'OCEANIC'} by default and
1173			needs to be set to \texttt{'ATMOSPHERIC'} for atmosphere simulations.
1174			In this case, \textbf{eosType} must be set to \texttt{'IDEALGAS'}.
1175	mlosch	1.13	For the ocean, two forms of the equation of state are available:
1176	edhill	1.18	linear (set \textbf{eosType} to \texttt{'LINEAR'}) and a polynomial
1177			approximation to the full nonlinear equation ( set \textbf{eosType} to
1178			\texttt{'POLYNOMIAL'}). In the linear case, you need to specify the
1179			thermal and haline expansion coefficients represented by the variables
1180			\textbf{tAlpha} (in K$^{-1}$) and \textbf{sBeta} (in ppt$^{-1}$). For
1181			the nonlinear case, you need to generate a file of polynomial
1182			coefficients called \textit{POLY3.COEFFS}. To do this, use the program
1183	mlosch	1.13	\textit{utils/knudsen2/knudsen2.f} under the model tree (a Makefile is
1184			available in the same directory and you will need to edit the number
1185			and the values of the vertical levels in \textit{knudsen2.f} so that
1186			they match those of your configuration).
1187
1188			There there are also higher polynomials for the equation of state:
1189			\begin{description}
1190	edhill	1.18	\item[\texttt{'UNESCO'}:] The UNESCO equation of state formula of
1191	mlosch	1.13	Fofonoff and Millard \cite{fofonoff83}. This equation of state
1192	edhill	1.18	assumes in-situ temperature, which is not a model variable; {\em its
1193			use is therefore discouraged, and it is only listed for
1194			completeness}.
1195			\item[\texttt{'JMD95Z'}:] A modified UNESCO formula by Jackett and
1196	mlosch	1.13	McDougall \cite{jackett95}, which uses the model variable potential
1197	edhill	1.18	temperature as input. The \texttt{'Z'} indicates that this equation
1198	mlosch	1.13	of state uses a horizontally and temporally constant pressure
1199			$p_{0}=-g\rho_{0}z$.
1200	edhill	1.18	\item[\texttt{'JMD95P'}:] A modified UNESCO formula by Jackett and
1201	mlosch	1.13	McDougall \cite{jackett95}, which uses the model variable potential
1202	edhill	1.18	temperature as input. The \texttt{'P'} indicates that this equation
1203	mlosch	1.13	of state uses the actual hydrostatic pressure of the last time
1204			step. Lagging the pressure in this way requires an additional pickup
1205			file for restarts.
1206	edhill	1.18	\item[\texttt{'MDJWF'}:] The new, more accurate and less expensive
1207	mlosch	1.13	equation of state by McDougall et~al. \cite{mcdougall03}. It also
1208			requires lagging the pressure and therefore an additional pickup
1209			file for restarts.
1210			\end{description}
1211			For none of these options an reference profile of temperature or
1212			salinity is required.
1213	adcroft	1.1
1214	adcroft	1.4	\subsection{Momentum equations}
1215	adcroft	1.1
1216	edhill	1.18	In this section, we only focus for now on the parameters that you are
1217			likely to change, i.e. the ones relative to forcing and dissipation
1218			for example. The details relevant to the vector-invariant form of the
1219			equations and the various advection schemes are not covered for the
1220			moment. We assume that you use the standard form of the momentum
1221			equations (i.e. the flux-form) with the default advection scheme.
1222			Also, there are a few logical variables that allow you to turn on/off
1223			various terms in the momentum equation. These variables are called
1224			\textbf{momViscosity, momAdvection, momForcing, useCoriolis,
1225			momPressureForcing, momStepping} and \textbf{metricTerms }and are
1226			assumed to be set to \texttt{'.TRUE.'} here. Look at the file
1227			\textit{model/inc/PARAMS.h }for a precise definition of these
1228			variables.
1229	adcroft	1.1
1230	edhill	1.17	\begin{description}
1231			\item[initialization] \
1232
1233			The velocity components are initialized to 0 unless the simulation
1234			is starting from a pickup file (see section on simulation control
1235			parameters).
1236
1237			\item[forcing] \
1238
1239			This section only applies to the ocean. You need to generate
1240	edhill	1.18	wind-stress data into two files \textbf{zonalWindFile} and
1241			\textbf{meridWindFile} corresponding to the zonal and meridional
1242	edhill	1.17	components of the wind stress, respectively (if you want the stress
1243			to be along the direction of only one of the model horizontal axes,
1244			you only need to generate one file). The format of the files is
1245			similar to the bathymetry file. The zonal (meridional) stress data
1246			are assumed to be in Pa and located at U-points (V-points). As for
1247			the bathymetry, the precision with which to read the binary data is
1248	edhill	1.18	controlled by the variable \textbf{readBinaryPrec}. See the matlab
1249			program \textit{gendata.m} in the \textit{input} directories under
1250			\textit{verification} to see how simple analytical wind forcing data
1251			are generated for the case study experiments.
1252	edhill	1.17
1253			There is also the possibility of prescribing time-dependent periodic
1254			forcing. To do this, concatenate the successive time records into a
1255	edhill	1.18	single file (for each stress component) ordered in a (x,y,t) fashion
1256			and set the following variables: \textbf{periodicExternalForcing }to
1257			\texttt{'.TRUE.'}, \textbf{externForcingPeriod }to the period (in s)
1258			of which the forcing varies (typically 1 month), and
1259			\textbf{externForcingCycle} to the repeat time (in s) of the forcing
1260			(typically 1 year -- note: \textbf{ externForcingCycle} must be a
1261			multiple of \textbf{externForcingPeriod}). With these variables set
1262			up, the model will interpolate the forcing linearly at each
1263			iteration.
1264	edhill	1.17
1265			\item[dissipation] \
1266
1267			The lateral eddy viscosity coefficient is specified through the
1268	edhill	1.18	variable \textbf{viscAh} (in m$^{2}$s$^{-1}$). The vertical eddy
1269			viscosity coefficient is specified through the variable
1270			\textbf{viscAz} (in m$^{2}$s$^{-1}$) for the ocean and
1271			\textbf{viscAp} (in Pa$^{2}$s$^{-1}$) for the atmosphere. The
1272			vertical diffusive fluxes can be computed implicitly by setting the
1273			logical variable \textbf{implicitViscosity }to \texttt{'.TRUE.'}.
1274			In addition, biharmonic mixing can be added as well through the
1275			variable \textbf{viscA4} (in m$^{4}$s$^{-1}$). On a spherical polar
1276			grid, you might also need to set the variable \textbf{cosPower}
1277			which is set to 0 by default and which represents the power of
1278			cosine of latitude to multiply viscosity. Slip or no-slip conditions
1279			at lateral and bottom boundaries are specified through the logical
1280			variables \textbf{no\_slip\_sides} and \textbf{no\_slip\_bottom}. If
1281			set to \texttt{'.FALSE.'}, free-slip boundary conditions are
1282			applied. If no-slip boundary conditions are applied at the bottom, a
1283			bottom drag can be applied as well. Two forms are available: linear
1284			(set the variable \textbf{bottomDragLinear} in s$ ^{-1}$) and
1285			quadratic (set the variable \textbf{bottomDragQuadratic} in
1286			m$^{-1}$).
1287	edhill	1.17
1288			The Fourier and Shapiro filters are described elsewhere.
1289
1290			\item[C-D scheme] \
1291
1292			If you run at a sufficiently coarse resolution, you will need the
1293			C-D scheme for the computation of the Coriolis terms. The
1294			variable\textbf{\ tauCD}, which represents the C-D scheme coupling
1295			timescale (in s) needs to be set.
1296
1297			\item[calculation of pressure/geopotential] \
1298
1299			First, to run a non-hydrostatic ocean simulation, set the logical
1300	edhill	1.18	variable \textbf{nonHydrostatic} to \texttt{'.TRUE.'}. The pressure
1301	edhill	1.17	field is then inverted through a 3D elliptic equation. (Note: this
1302			capability is not available for the atmosphere yet.) By default, a
1303			hydrostatic simulation is assumed and a 2D elliptic equation is used
1304			to invert the pressure field. The parameters controlling the
1305			behaviour of the elliptic solvers are the variables
1306	edhill	1.18	\textbf{cg2dMaxIters} and \textbf{cg2dTargetResidual } for
1307			the 2D case and \textbf{cg3dMaxIters} and
1308			\textbf{cg3dTargetResidual} for the 3D case. You probably won't need to
1309	edhill	1.17	alter the default values (are we sure of this?).
1310
1311			For the calculation of the surface pressure (for the ocean) or
1312			surface geopotential (for the atmosphere) you need to set the
1313	edhill	1.18	logical variables \textbf{rigidLid} and \textbf{implicitFreeSurface}
1314			(set one to \texttt{'.TRUE.'} and the other to \texttt{'.FALSE.'}
1315			depending on how you want to deal with the ocean upper or atmosphere
1316			lower boundary).
1317	adcroft	1.1
1318	edhill	1.17	\end{description}
1319	adcroft	1.1
1320	adcroft	1.4	\subsection{Tracer equations}
1321	adcroft	1.1
1322	edhill	1.18	This section covers the tracer equations i.e. the potential
1323			temperature equation and the salinity (for the ocean) or specific
1324			humidity (for the atmosphere) equation. As for the momentum equations,
1325			we only describe for now the parameters that you are likely to change.
1326			The logical variables \textbf{tempDiffusion} \textbf{tempAdvection}
1327			\textbf{tempForcing}, and \textbf{tempStepping} allow you to turn
1328			on/off terms in the temperature equation (same thing for salinity or
1329			specific humidity with variables \textbf{saltDiffusion},
1330			\textbf{saltAdvection} etc.). These variables are all assumed here to
1331			be set to \texttt{'.TRUE.'}. Look at file \textit{model/inc/PARAMS.h}
1332			for a precise definition.
1333	adcroft	1.1
1334	edhill	1.17	\begin{description}
1335			\item[initialization] \
1336
1337			The initial tracer data can be contained in the binary files
1338	edhill	1.18	\textbf{hydrogThetaFile} and \textbf{hydrogSaltFile}. These files
1339			should contain 3D data ordered in an (x,y,r) fashion with k=1 as the
1340			first vertical level. If no file names are provided, the tracers
1341			are then initialized with the values of \textbf{tRef} and
1342			\textbf{sRef} mentioned above (in the equation of state section). In
1343	edhill	1.17	this case, the initial tracer data are uniform in x and y for each
1344			depth level.
1345
1346			\item[forcing] \
1347
1348			This part is more relevant for the ocean, the procedure for the
1349			atmosphere not being completely stabilized at the moment.
1350
1351			A combination of fluxes data and relaxation terms can be used for
1352	edhill	1.18	driving the tracer equations. For potential temperature, heat flux
1353	edhill	1.17	data (in W/m$ ^{2}$) can be stored in the 2D binary file
1354	edhill	1.18	\textbf{surfQfile}. Alternatively or in addition, the forcing can
1355			be specified through a relaxation term. The SST data to which the
1356			model surface temperatures are restored to are supposed to be stored
1357			in the 2D binary file \textbf{thetaClimFile}. The corresponding
1358			relaxation time scale coefficient is set through the variable
1359			\textbf{tauThetaClimRelax} (in s). The same procedure applies for
1360			salinity with the variable names \textbf{EmPmRfile},
1361			\textbf{saltClimFile}, and \textbf{tauSaltClimRelax} for freshwater
1362			flux (in m/s) and surface salinity (in ppt) data files and
1363			relaxation time scale coefficient (in s), respectively. Also for
1364			salinity, if the CPP key \textbf{USE\_NATURAL\_BCS} is turned on,
1365			natural boundary conditions are applied i.e. when computing the
1366			surface salinity tendency, the freshwater flux is multiplied by the
1367			model surface salinity instead of a constant salinity value.
1368	edhill	1.17
1369			As for the other input files, the precision with which to read the
1370			data is controlled by the variable \textbf{readBinaryPrec}.
1371			Time-dependent, periodic forcing can be applied as well following
1372			the same procedure used for the wind forcing data (see above).
1373
1374			\item[dissipation] \
1375
1376			Lateral eddy diffusivities for temperature and salinity/specific
1377	edhill	1.18	humidity are specified through the variables \textbf{diffKhT} and
1378			\textbf{diffKhS} (in m$^{2}$/s). Vertical eddy diffusivities are
1379			specified through the variables \textbf{diffKzT} and
1380			\textbf{diffKzS} (in m$^{2}$/s) for the ocean and \textbf{diffKpT
1381			}and \textbf{diffKpS} (in Pa$^{2}$/s) for the atmosphere. The
1382			vertical diffusive fluxes can be computed implicitly by setting the
1383			logical variable \textbf{implicitDiffusion} to \texttt{'.TRUE.'}.
1384			In addition, biharmonic diffusivities can be specified as well
1385			through the coefficients \textbf{diffK4T} and \textbf{diffK4S} (in
1386	edhill	1.17	m$^{4}$/s). Note that the cosine power scaling (specified through
1387	edhill	1.18	\textbf{cosPower}---see the momentum equations section) is applied to
1388			the tracer diffusivities (Laplacian and biharmonic) as well. The
1389	edhill	1.17	Gent and McWilliams parameterization for oceanic tracers is
1390			described in the package section. Finally, note that tracers can be
1391			also subject to Fourier and Shapiro filtering (see the corresponding
1392			section on these filters).
1393
1394			\item[ocean convection] \
1395
1396			Two options are available to parameterize ocean convection: one is
1397			to use the convective adjustment scheme. In this case, you need to
1398			set the variable \textbf{cadjFreq}, which represents the frequency
1399			(in s) with which the adjustment algorithm is called, to a non-zero
1400			value (if set to a negative value by the user, the model will set it
1401			to the tracer time step). The other option is to parameterize
1402			convection with implicit vertical diffusion. To do this, set the
1403	edhill	1.18	logical variable \textbf{implicitDiffusion} to \texttt{'.TRUE.'}
1404			and the real variable \textbf{ivdc\_kappa} to a value (in m$^{2}$/s)
1405	edhill	1.17	you wish the tracer vertical diffusivities to have when mixing
1406			tracers vertically due to static instabilities. Note that
1407	edhill	1.18	\textbf{cadjFreq} and \textbf{ivdc\_kappa}can not both have non-zero
1408			value.
1409	adcroft	1.1
1410	edhill	1.17	\end{description}
1411	adcroft	1.1
1412	adcroft	1.4	\subsection{Simulation controls}
1413	adcroft	1.1
1414	edhill	1.18	The model ''clock'' is defined by the variable \textbf{deltaTClock}
1415			(in s) which determines the IO frequencies and is used in tagging
1416			output. Typically, you will set it to the tracer time step for
1417			accelerated runs (otherwise it is simply set to the default time step
1418			\textbf{deltaT}). Frequency of checkpointing and dumping of the model
1419			state are referenced to this clock (see below).
1420	adcroft	1.1
1421	edhill	1.17	\begin{description}
1422			\item[run duration] \
1423
1424			The beginning of a simulation is set by specifying a start time (in
1425	edhill	1.18	s) through the real variable \textbf{startTime} or by specifying an
1426	edhill	1.17	initial iteration number through the integer variable
1427			\textbf{nIter0}. If these variables are set to nonzero values, the
1428	edhill	1.18	model will look for a ''pickup'' file \textit{pickup.0000nIter0} to
1429			restart the integration. The end of a simulation is set through the
1430			real variable \textbf{endTime} (in s). Alternatively, you can
1431			specify instead the number of time steps to execute through the
1432			integer variable \textbf{nTimeSteps}.
1433	edhill	1.17
1434			\item[frequency of output] \
1435
1436			Real variables defining frequencies (in s) with which output files
1437	edhill	1.18	are written on disk need to be set up. \textbf{dumpFreq} controls
1438	edhill	1.17	the frequency with which the instantaneous state of the model is
1439	edhill	1.18	saved. \textbf{chkPtFreq} and \textbf{pchkPtFreq} control the output
1440			frequency of rolling and permanent checkpoint files, respectively.
1441			See section 1.5.1 Output files for the definition of model state and
1442			checkpoint files. In addition, time-averaged fields can be written
1443			out by setting the variable \textbf{taveFreq} (in s). The precision
1444			with which to write the binary data is controlled by the integer
1445			variable w\textbf{riteBinaryPrec} (set it to \texttt{32} or
1446			\texttt{64}).
1447	adcroft	1.1
1448	edhill	1.17	\end{description}
1449	adcroft	1.1
1450	mlosch	1.13
1451			%%% Local Variables:
1452			%%% mode: latex
1453			%%% TeX-master: t
1454			%%% End:

	ViewVC Help
Powered by ViewVC 1.1.22

Annotation of /manual/s_getstarted/text/getting_started.tex