/[MITgcm]/manual/s_getstarted/text/getting_started.tex
ViewVC logotype

Contents of /manual/s_getstarted/text/getting_started.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.36 - (show annotations) (download) (as text)
Tue Jun 27 19:08:22 2006 UTC (19 years ago) by molod
Branch: MAIN
Changes since 1.35: +1 -172 lines
File MIME type: application/x-tex
Add cross references between tutorials and verification file system directories

1 % $Header: /u/gcmpack/manual/part3/getting_started.tex,v 1.35 2006/04/20 22:09:08 molod Exp $
2 % $Name: $
3
4 %\section{Getting started}
5
6 In this section, we describe how to use the model. In the first
7 section, we provide enough information to help you get started with
8 the model. We believe the best way to familiarize yourself with the
9 model is to run the case study examples provided with the base
10 version. Information on how to obtain, compile, and run the code is
11 found there as well as a brief description of the model structure
12 directory and the case study examples. The latter and the code
13 structure are described more fully in chapters
14 \ref{chap:discretization} and \ref{chap:sarch}, respectively. Here, in
15 this section, we provide information on how to customize the code when
16 you are ready to try implementing the configuration you have in mind.
17
18 \section{Where to find information}
19 \label{sect:whereToFindInfo}
20 \begin{rawhtml}
21 <!-- CMIREDIR:whereToFindInfo: -->
22 \end{rawhtml}
23
24 A web site is maintained for release 2 (``Pelican'') of MITgcm:
25 \begin{rawhtml} <A href=http://mitgcm.org/pelican/ target="idontexist"> \end{rawhtml}
26 \begin{verbatim}
27 http://mitgcm.org/pelican
28 \end{verbatim}
29 \begin{rawhtml} </A> \end{rawhtml}
30 Here you will find an on-line version of this document, a
31 ``browsable'' copy of the code and a searchable database of the model
32 and site, as well as links for downloading the model and
33 documentation, to data-sources, and other related sites.
34
35 There is also a web-archived support mailing list for the model that
36 you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
37 \begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
38 \begin{verbatim}
39 http://mitgcm.org/mailman/listinfo/mitgcm-support/
40 http://mitgcm.org/pipermail/mitgcm-support/
41 \end{verbatim}
42 \begin{rawhtml} </A> \end{rawhtml}
43 Essentially all of the MITgcm web pages can be searched using a
44 popular web crawler such as Google or through our own search facility:
45 \begin{rawhtml} <A href=http://mitgcm.org/mailman/htdig/ target="idontexist"> \end{rawhtml}
46 \begin{verbatim}
47 http://mitgcm.org/htdig/
48 \end{verbatim}
49 \begin{rawhtml} </A> \end{rawhtml}
50 %%% http://www.google.com/search?q=hydrostatic+site%3Amitgcm.org
51
52
53
54 \section{Obtaining the code}
55 \label{sect:obtainingCode}
56 \begin{rawhtml}
57 <!-- CMIREDIR:obtainingCode: -->
58 \end{rawhtml}
59
60 MITgcm can be downloaded from our system by following
61 the instructions below. As a courtesy we ask that you send e-mail to us at
62 \begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
63 MITgcm-support@mitgcm.org
64 \begin{rawhtml} </A> \end{rawhtml}
65 to enable us to keep track of who's using the model and in what application.
66 You can download the model two ways:
67
68 \begin{enumerate}
69 \item Using CVS software. CVS is a freely available source code management
70 tool. To use CVS you need to have the software installed. Many systems
71 come with CVS pre-installed, otherwise good places to look for
72 the software for a particular platform are
73 \begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
74 cvshome.org
75 \begin{rawhtml} </A> \end{rawhtml}
76 and
77 \begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
78 wincvs.org
79 \begin{rawhtml} </A> \end{rawhtml}
80 .
81
82 \item Using a tar file. This method is simple and does not
83 require any special software. However, this method does not
84 provide easy support for maintenance updates.
85
86 \end{enumerate}
87
88 \subsection{Method 1 - Checkout from CVS}
89 \label{sect:cvs_checkout}
90
91 If CVS is available on your system, we strongly encourage you to use it. CVS
92 provides an efficient and elegant way of organizing your code and keeping
93 track of your changes. If CVS is not available on your machine, you can also
94 download a tar file.
95
96 Before you can use CVS, the following environment variable(s) should
97 be set within your shell. For a csh or tcsh shell, put the following
98 \begin{verbatim}
99 % setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
100 \end{verbatim}
101 in your \texttt{.cshrc} or \texttt{.tcshrc} file. For bash or sh
102 shells, put:
103 \begin{verbatim}
104 % export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
105 \end{verbatim}
106 in your \texttt{.profile} or \texttt{.bashrc} file.
107
108
109 To get MITgcm through CVS, first register with the MITgcm CVS server
110 using command:
111 \begin{verbatim}
112 % cvs login ( CVS password: cvsanon )
113 \end{verbatim}
114 You only need to do a ``cvs login'' once.
115
116 To obtain the latest sources type:
117 \begin{verbatim}
118 % cvs co MITgcm
119 \end{verbatim}
120 or to get a specific release type:
121 \begin{verbatim}
122 % cvs co -P -r checkpoint52i_post MITgcm
123 \end{verbatim}
124 The MITgcm web site contains further directions concerning the source
125 code and CVS. It also contains a web interface to our CVS archive so
126 that one may easily view the state of files, revisions, and other
127 development milestones:
128 \begin{rawhtml} <A href="http://mitgcm.org/download" target="idontexist"> \end{rawhtml}
129 \begin{verbatim}
130 http://mitgcm.org/source_code.html
131 \end{verbatim}
132 \begin{rawhtml} </A> \end{rawhtml}
133
134 As a convenience, the MITgcm CVS server contains aliases which are
135 named subsets of the codebase. These aliases can be especially
136 helpful when used over slow internet connections or on machines with
137 restricted storage space. Table \ref{tab:cvsModules} contains a list
138 of CVS aliases
139 \begin{table}[htb]
140 \centering
141 \begin{tabular}[htb]{|lp{3.25in}|}\hline
142 \textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
143 \texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
144 \texttt{MITgcm\_verif\_basic}
145 & Source code plus a small set of the verification examples
146 (\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
147 \texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
148 \texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
149 \texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
150 \texttt{MITgcm\_verif\_all} & Source code plus all of the
151 verification examples. \\\hline
152 \end{tabular}
153 \caption{MITgcm CVS Modules}
154 \label{tab:cvsModules}
155 \end{table}
156
157 The checkout process creates a directory called \texttt{MITgcm}. If
158 the directory \texttt{MITgcm} exists this command updates your code
159 based on the repository. Each directory in the source tree contains a
160 directory \texttt{CVS}. This information is required by CVS to keep
161 track of your file versions with respect to the repository. Don't edit
162 the files in \texttt{CVS}! You can also use CVS to download code
163 updates. More extensive information on using CVS for maintaining
164 MITgcm code can be found
165 \begin{rawhtml} <A href="http://mitgcm.org/usingcvstoget.html" target="idontexist"> \end{rawhtml}
166 here
167 \begin{rawhtml} </A> \end{rawhtml}
168 .
169 It is important to note that the CVS aliases in Table
170 \ref{tab:cvsModules} cannot be used in conjunction with the CVS
171 \texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
172 they create can be changed to a different name following the check-out:
173 \begin{verbatim}
174 % cvs co MITgcm_verif_basic
175 % mv MITgcm MITgcm_verif_basic
176 \end{verbatim}
177
178
179 \subsection{Method 2 - Tar file download}
180 \label{sect:conventionalDownload}
181
182 If you do not have CVS on your system, you can download the model as a
183 tar file from the web site at:
184 \begin{rawhtml} <A href=http://mitgcm.org/download target="idontexist"> \end{rawhtml}
185 \begin{verbatim}
186 http://mitgcm.org/download/
187 \end{verbatim}
188 \begin{rawhtml} </A> \end{rawhtml}
189 The tar file still contains CVS information which we urge you not to
190 delete; even if you do not use CVS yourself the information can help
191 us if you should need to send us your copy of the code. If a recent
192 tar file does not exist, then please contact the developers through
193 the
194 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
195 MITgcm-support@mitgcm.org
196 \begin{rawhtml} </A> \end{rawhtml}
197 mailing list.
198
199 \subsubsection{Upgrading from an earlier version}
200
201 If you already have an earlier version of the code you can ``upgrade''
202 your copy instead of downloading the entire repository again. First,
203 ``cd'' (change directory) to the top of your working copy:
204 \begin{verbatim}
205 % cd MITgcm
206 \end{verbatim}
207 and then issue the cvs update command such as:
208 \begin{verbatim}
209 % cvs -q update -r checkpoint52i_post -d -P
210 \end{verbatim}
211 This will update the ``tag'' to ``checkpoint52i\_post'', add any new
212 directories (-d) and remove any empty directories (-P). The -q option
213 means be quiet which will reduce the number of messages you'll see in
214 the terminal. If you have modified the code prior to upgrading, CVS
215 will try to merge your changes with the upgrades. If there is a
216 conflict between your modifications and the upgrade, it will report
217 that file with a ``C'' in front, e.g.:
218 \begin{verbatim}
219 C model/src/ini_parms.F
220 \end{verbatim}
221 If the list of conflicts scrolled off the screen, you can re-issue the
222 cvs update command and it will report the conflicts. Conflicts are
223 indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
224 ``$>>>>>>>$''. For example,
225 {\small
226 \begin{verbatim}
227 <<<<<<< ini_parms.F
228 & bottomDragLinear,myOwnBottomDragCoefficient,
229 =======
230 & bottomDragLinear,bottomDragQuadratic,
231 >>>>>>> 1.18
232 \end{verbatim}
233 }
234 means that you added ``myOwnBottomDragCoefficient'' to a namelist at
235 the same time and place that we added ``bottomDragQuadratic''. You
236 need to resolve this conflict and in this case the line should be
237 changed to:
238 {\small
239 \begin{verbatim}
240 & bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
241 \end{verbatim}
242 }
243 and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
244 Unless you are making modifications which exactly parallel
245 developments we make, these types of conflicts should be rare.
246
247 \paragraph*{Upgrading to the current pre-release version}
248
249 We don't make a ``release'' for every little patch and bug fix in
250 order to keep the frequency of upgrades to a minimum. However, if you
251 have run into a problem for which ``we have already fixed in the
252 latest code'' and we haven't made a ``tag'' or ``release'' since that
253 patch then you'll need to get the latest code:
254 \begin{verbatim}
255 % cvs -q update -A -d -P
256 \end{verbatim}
257 Unlike, the ``check-out'' and ``update'' procedures above, there is no
258 ``tag'' or release name. The -A tells CVS to upgrade to the
259 very latest version. As a rule, we don't recommend this since you
260 might upgrade while we are in the processes of checking in the code so
261 that you may only have part of a patch. Using this method of updating
262 also means we can't tell what version of the code you are working
263 with. So please be sure you understand what you're doing.
264
265 \section{Model and directory structure}
266 \begin{rawhtml}
267 <!-- CMIREDIR:directory_structure: -->
268 \end{rawhtml}
269
270 The ``numerical'' model is contained within a execution environment
271 support wrapper. This wrapper is designed to provide a general
272 framework for grid-point models. MITgcmUV is a specific numerical
273 model that uses the framework. Under this structure the model is split
274 into execution environment support code and conventional numerical
275 model code. The execution environment support code is held under the
276 \texttt{eesupp} directory. The grid point model code is held under the
277 \texttt{model} directory. Code execution actually starts in the
278 \texttt{eesupp} routines and not in the \texttt{model} routines. For
279 this reason the top-level \texttt{MAIN.F} is in the
280 \texttt{eesupp/src} directory. In general, end-users should not need
281 to worry about this level. The top-level routine for the numerical
282 part of the code is in \texttt{model/src/THE\_MODEL\_MAIN.F}. Here is
283 a brief description of the directory structure of the model under the
284 root tree (a detailed description is given in section 3: Code
285 structure).
286
287 \begin{itemize}
288
289 \item \texttt{bin}: this directory is initially empty. It is the
290 default directory in which to compile the code.
291
292 \item \texttt{diags}: contains the code relative to time-averaged
293 diagnostics. It is subdivided into two subdirectories \texttt{inc}
294 and \texttt{src} that contain include files (\texttt{*.h} files) and
295 Fortran subroutines (\texttt{*.F} files), respectively.
296
297 \item \texttt{doc}: contains brief documentation notes.
298
299 \item \texttt{eesupp}: contains the execution environment source code.
300 Also subdivided into two subdirectories \texttt{inc} and
301 \texttt{src}.
302
303 \item \texttt{exe}: this directory is initially empty. It is the
304 default directory in which to execute the code.
305
306 \item \texttt{model}: this directory contains the main source code.
307 Also subdivided into two subdirectories \texttt{inc} and
308 \texttt{src}.
309
310 \item \texttt{pkg}: contains the source code for the packages. Each
311 package corresponds to a subdirectory. For example, \texttt{gmredi}
312 contains the code related to the Gent-McWilliams/Redi scheme,
313 \texttt{aim} the code relative to the atmospheric intermediate
314 physics. The packages are described in detail in section 3.
315
316 \item \texttt{tools}: this directory contains various useful tools.
317 For example, \texttt{genmake2} is a script written in csh (C-shell)
318 that should be used to generate your makefile. The directory
319 \texttt{adjoint} contains the makefile specific to the Tangent
320 linear and Adjoint Compiler (TAMC) that generates the adjoint code.
321 The latter is described in details in part V.
322
323 \item \texttt{utils}: this directory contains various utilities. The
324 subdirectory \texttt{knudsen2} contains code and a makefile that
325 compute coefficients of the polynomial approximation to the knudsen
326 formula for an ocean nonlinear equation of state. The
327 \texttt{matlab} subdirectory contains matlab scripts for reading
328 model output directly into matlab. \texttt{scripts} contains C-shell
329 post-processing scripts for joining processor-based and tiled-based
330 model output.
331
332 \item \texttt{verification}: this directory contains the model
333 examples. See section \ref{sect:modelExamples}.
334
335 \end{itemize}
336
337 \section[Building MITgcm]{Building the code}
338 \label{sect:buildingCode}
339 \begin{rawhtml}
340 <!-- CMIREDIR:buildingCode: -->
341 \end{rawhtml}
342
343 To compile the code, we use the \texttt{make} program. This uses a
344 file (\texttt{Makefile}) that allows us to pre-process source files,
345 specify compiler and optimization options and also figures out any
346 file dependencies. We supply a script (\texttt{genmake2}), described
347 in section \ref{sect:genmake}, that automatically creates the
348 \texttt{Makefile} for you. You then need to build the dependencies and
349 compile the code.
350
351 As an example, assume that you want to build and run experiment
352 \texttt{verification/exp2}. The are multiple ways and places to
353 actually do this but here let's build the code in
354 \texttt{verification/exp2/build}:
355 \begin{verbatim}
356 % cd verification/exp2/build
357 \end{verbatim}
358 First, build the \texttt{Makefile}:
359 \begin{verbatim}
360 % ../../../tools/genmake2 -mods=../code
361 \end{verbatim}
362 The command line option tells \texttt{genmake} to override model source
363 code with any files in the directory \texttt{../code/}.
364
365 On many systems, the \texttt{genmake2} program will be able to
366 automatically recognize the hardware, find compilers and other tools
367 within the user's path (``\texttt{echo \$PATH}''), and then choose an
368 appropriate set of options from the files (``optfiles'') contained in
369 the \texttt{tools/build\_options} directory. Under some
370 circumstances, a user may have to create a new ``optfile'' in order to
371 specify the exact combination of compiler, compiler flags, libraries,
372 and other options necessary to build a particular configuration of
373 MITgcm. In such cases, it is generally helpful to read the existing
374 ``optfiles'' and mimic their syntax.
375
376 Through the MITgcm-support list, the MITgcm developers are willing to
377 provide help writing or modifing ``optfiles''. And we encourage users
378 to post new ``optfiles'' (particularly ones for new machines or
379 architectures) to the
380 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
381 MITgcm-support@mitgcm.org
382 \begin{rawhtml} </A> \end{rawhtml}
383 list.
384
385 To specify an optfile to \texttt{genmake2}, the syntax is:
386 \begin{verbatim}
387 % ../../../tools/genmake2 -mods=../code -of /path/to/optfile
388 \end{verbatim}
389
390 Once a \texttt{Makefile} has been generated, we create the
391 dependencies with the command:
392 \begin{verbatim}
393 % make depend
394 \end{verbatim}
395 This modifies the \texttt{Makefile} by attaching a (usually, long)
396 list of files upon which other files depend. The purpose of this is to
397 reduce re-compilation if and when you start to modify the code. The
398 {\tt make depend} command also creates links from the model source to
399 this directory. It is important to note that the {\tt make depend}
400 stage will occasionally produce warnings or errors since the
401 dependency parsing tool is unable to find all of the necessary header
402 files (\textit{eg.} \texttt{netcdf.inc}). In these circumstances, it
403 is usually OK to ignore the warnings/errors and proceed to the next
404 step.
405
406 Next one can compile the code using:
407 \begin{verbatim}
408 % make
409 \end{verbatim}
410 The {\tt make} command creates an executable called \texttt{mitgcmuv}.
411 Additional make ``targets'' are defined within the makefile to aid in
412 the production of adjoint and other versions of MITgcm. On SMP
413 (shared multi-processor) systems, the build process can often be sped
414 up appreciably using the command:
415 \begin{verbatim}
416 % make -j 2
417 \end{verbatim}
418 where the ``2'' can be replaced with a number that corresponds to the
419 number of CPUs available.
420
421 Now you are ready to run the model. General instructions for doing so are
422 given in section \ref{sect:runModel}. Here, we can run the model by
423 first creating links to all the input files:
424 \begin{verbatim}
425 ln -s ../input/* .
426 \end{verbatim}
427 and then calling the executable with:
428 \begin{verbatim}
429 ./mitgcmuv > output.txt
430 \end{verbatim}
431 where we are re-directing the stream of text output to the file
432 \texttt{output.txt}.
433
434 \subsection{Building/compiling the code elsewhere}
435
436 In the example above (section \ref{sect:buildingCode}) we built the
437 executable in the {\em input} directory of the experiment for
438 convenience. You can also configure and compile the code in other
439 locations, for example on a scratch disk with out having to copy the
440 entire source tree. The only requirement to do so is you have {\tt
441 genmake2} in your path or you know the absolute path to {\tt
442 genmake2}.
443
444 The following sections outline some possible methods of organizing
445 your source and data.
446
447 \subsubsection{Building from the {\em ../code directory}}
448
449 This is just as simple as building in the {\em input/} directory:
450 \begin{verbatim}
451 % cd verification/exp2/code
452 % ../../../tools/genmake2
453 % make depend
454 % make
455 \end{verbatim}
456 However, to run the model the executable ({\em mitgcmuv}) and input
457 files must be in the same place. If you only have one calculation to make:
458 \begin{verbatim}
459 % cd ../input
460 % cp ../code/mitgcmuv ./
461 % ./mitgcmuv > output.txt
462 \end{verbatim}
463 or if you will be making multiple runs with the same executable:
464 \begin{verbatim}
465 % cd ../
466 % cp -r input run1
467 % cp code/mitgcmuv run1
468 % cd run1
469 % ./mitgcmuv > output.txt
470 \end{verbatim}
471
472 \subsubsection{Building from a new directory}
473
474 Since the {\em input} directory contains input files it is often more
475 useful to keep {\em input} pristine and build in a new directory
476 within {\em verification/exp2/}:
477 \begin{verbatim}
478 % cd verification/exp2
479 % mkdir build
480 % cd build
481 % ../../../tools/genmake2 -mods=../code
482 % make depend
483 % make
484 \end{verbatim}
485 This builds the code exactly as before but this time you need to copy
486 either the executable or the input files or both in order to run the
487 model. For example,
488 \begin{verbatim}
489 % cp ../input/* ./
490 % ./mitgcmuv > output.txt
491 \end{verbatim}
492 or if you tend to make multiple runs with the same executable then
493 running in a new directory each time might be more appropriate:
494 \begin{verbatim}
495 % cd ../
496 % mkdir run1
497 % cp build/mitgcmuv run1/
498 % cp input/* run1/
499 % cd run1
500 % ./mitgcmuv > output.txt
501 \end{verbatim}
502
503 \subsubsection{Building on a scratch disk}
504
505 Model object files and output data can use up large amounts of disk
506 space so it is often the case that you will be operating on a large
507 scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
508 following commands will build the model in {\em /scratch/exp2-run1}:
509 \begin{verbatim}
510 % cd /scratch/exp2-run1
511 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
512 -mods=~/MITgcm/verification/exp2/code
513 % make depend
514 % make
515 \end{verbatim}
516 To run the model here, you'll need the input files:
517 \begin{verbatim}
518 % cp ~/MITgcm/verification/exp2/input/* ./
519 % ./mitgcmuv > output.txt
520 \end{verbatim}
521
522 As before, you could build in one directory and make multiple runs of
523 the one experiment:
524 \begin{verbatim}
525 % cd /scratch/exp2
526 % mkdir build
527 % cd build
528 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
529 -mods=~/MITgcm/verification/exp2/code
530 % make depend
531 % make
532 % cd ../
533 % cp -r ~/MITgcm/verification/exp2/input run2
534 % cd run2
535 % ./mitgcmuv > output.txt
536 \end{verbatim}
537
538
539 \subsection{Using \texttt{genmake2}}
540 \label{sect:genmake}
541
542 To compile the code, first use the program \texttt{genmake2} (located
543 in the \texttt{tools} directory) to generate a Makefile.
544 \texttt{genmake2} is a shell script written to work with all
545 ``sh''--compatible shells including bash v1, bash v2, and Bourne.
546 Internally, \texttt{genmake2} determines the locations of needed
547 files, the compiler, compiler options, libraries, and Unix tools. It
548 relies upon a number of ``optfiles'' located in the
549 \texttt{tools/build\_options} directory.
550
551 The purpose of the optfiles is to provide all the compilation options
552 for particular ``platforms'' (where ``platform'' roughly means the
553 combination of the hardware and the compiler) and code configurations.
554 Given the combinations of possible compilers and library dependencies
555 ({\it eg.} MPI and NetCDF) there may be numerous optfiles available
556 for a single machine. The naming scheme for the majority of the
557 optfiles shipped with the code is
558 \begin{center}
559 {\bf OS\_HARDWARE\_COMPILER }
560 \end{center}
561 where
562 \begin{description}
563 \item[OS] is the name of the operating system (generally the
564 lower-case output of the {\tt 'uname'} command)
565 \item[HARDWARE] is a string that describes the CPU type and
566 corresponds to output from the {\tt 'uname -m'} command:
567 \begin{description}
568 \item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
569 and athlon
570 \item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
571 \item[amd64] is AMD x86\_64 systems
572 \item[ppc] is for Mac PowerPC systems
573 \end{description}
574 \item[COMPILER] is the compiler name (generally, the name of the
575 FORTRAN executable)
576 \end{description}
577
578 In many cases, the default optfiles are sufficient and will result in
579 usable Makefiles. However, for some machines or code configurations,
580 new ``optfiles'' must be written. To create a new optfile, it is
581 generally best to start with one of the defaults and modify it to suit
582 your needs. Like \texttt{genmake2}, the optfiles are all written
583 using a simple ``sh''--compatible syntax. While nearly all variables
584 used within \texttt{genmake2} may be specified in the optfiles, the
585 critical ones that should be defined are:
586
587 \begin{description}
588 \item[FC] the FORTRAN compiler (executable) to use
589 \item[DEFINES] the command-line DEFINE options passed to the compiler
590 \item[CPP] the C pre-processor to use
591 \item[NOOPTFLAGS] options flags for special files that should not be
592 optimized
593 \end{description}
594
595 For example, the optfile for a typical Red Hat Linux machine (``ia32''
596 architecture) using the GCC (g77) compiler is
597 \begin{verbatim}
598 FC=g77
599 DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
600 CPP='cpp -traditional -P'
601 NOOPTFLAGS='-O0'
602 # For IEEE, use the "-ffloat-store" option
603 if test "x$IEEE" = x ; then
604 FFLAGS='-Wimplicit -Wunused -Wuninitialized'
605 FOPTIM='-O3 -malign-double -funroll-loops'
606 else
607 FFLAGS='-Wimplicit -Wunused -ffloat-store'
608 FOPTIM='-O0 -malign-double'
609 fi
610 \end{verbatim}
611
612 If you write an optfile for an unrepresented machine or compiler, you
613 are strongly encouraged to submit the optfile to the MITgcm project
614 for inclusion. Please send the file to the
615 \begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
616 \begin{center}
617 MITgcm-support@mitgcm.org
618 \end{center}
619 \begin{rawhtml} </A> \end{rawhtml}
620 mailing list.
621
622 In addition to the optfiles, \texttt{genmake2} supports a number of
623 helpful command-line options. A complete list of these options can be
624 obtained from:
625 \begin{verbatim}
626 % genmake2 -h
627 \end{verbatim}
628
629 The most important command-line options are:
630 \begin{description}
631
632 \item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
633 should be used for a particular build.
634
635 If no "optfile" is specified (either through the command line or the
636 MITGCM\_OPTFILE environment variable), genmake2 will try to make a
637 reasonable guess from the list provided in {\em
638 tools/build\_options}. The method used for making this guess is
639 to first determine the combination of operating system and hardware
640 (eg. "linux\_ia32") and then find a working FORTRAN compiler within
641 the user's path. When these three items have been identified,
642 genmake2 will try to find an optfile that has a matching name.
643
644 \item[\texttt{--pdefault='PKG1 PKG2 PKG3 ...'}] specifies the default
645 set of packages to be used. The normal order of precedence for
646 packages is as follows:
647 \begin{enumerate}
648 \item If available, the command line (\texttt{--pdefault}) settings
649 over-rule any others.
650
651 \item Next, \texttt{genmake2} will look for a file named
652 ``\texttt{packages.conf}'' in the local directory or in any of the
653 directories specified with the \texttt{--mods} option.
654
655 \item Finally, if neither of the above are available,
656 \texttt{genmake2} will use the \texttt{/pkg/pkg\_default} file.
657 \end{enumerate}
658
659 \item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
660 used for packages.
661
662 If not specified, the default dependency file {\em pkg/pkg\_depend}
663 is used. The syntax for this file is parsed on a line-by-line basis
664 where each line containes either a comment ("\#") or a simple
665 "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
666 specifies a "must be used with" or a "must not be used with"
667 relationship, respectively. If no rule is specified, then it is
668 assumed that the two packages are compatible and will function
669 either with or without each other.
670
671 \item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
672 automatic differentiation options file to be used. The file is
673 analogous to the ``optfile'' defined above but it specifies
674 information for the AD build process.
675
676 The default file is located in {\em
677 tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
678 and "TAMC" compilers. An alternate version is also available at
679 {\em tools/adjoint\_options/adjoint\_staf} that selects the newer
680 "STAF" compiler. As with any compilers, it is helpful to have their
681 directories listed in your {\tt \$PATH} environment variable.
682
683 \item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
684 directories containing ``modifications''. These directories contain
685 files with names that may (or may not) exist in the main MITgcm
686 source tree but will be overridden by any identically-named sources
687 within the ``MODS'' directories.
688
689 The order of precedence for this "name-hiding" is as follows:
690 \begin{itemize}
691 \item ``MODS'' directories (in the order given)
692 \item Packages either explicitly specified or provided by default
693 (in the order given)
694 \item Packages included due to package dependencies (in the order
695 that that package dependencies are parsed)
696 \item The "standard dirs" (which may have been specified by the
697 ``-standarddirs'' option)
698 \end{itemize}
699
700 \item[\texttt{--mpi}] This option enables certain MPI features (using
701 CPP \texttt{\#define}s) within the code and is necessary for MPI
702 builds (see Section \ref{sect:mpi-build}).
703
704 \item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
705 soft-links and other bugs common with the \texttt{make} versions
706 provided by commercial Unix vendors, GNU \texttt{make} (sometimes
707 called \texttt{gmake}) should be preferred. This option provides a
708 means for specifying the make executable to be used.
709
710 \item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
711 machines, the ``bash'' shell is unavailable. To run on these
712 systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
713 a Bourne, POSIX, or compatible) shell. The syntax in these
714 circumstances is:
715 \begin{center}
716 \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
717 \end{center}
718 where \texttt{/bin/sh} can be replaced with the full path and name
719 of the desired shell.
720
721 \end{description}
722
723
724 \subsection{Building with MPI}
725 \label{sect:mpi-build}
726
727 Building MITgcm to use MPI libraries can be complicated due to the
728 variety of different MPI implementations available, their dependencies
729 or interactions with different compilers, and their often ad-hoc
730 locations within file systems. For these reasons, its generally a
731 good idea to start by finding and reading the documentation for your
732 machine(s) and, if necessary, seeking help from your local systems
733 administrator.
734
735 The steps for building MITgcm with MPI support are:
736 \begin{enumerate}
737
738 \item Determine the locations of your MPI-enabled compiler and/or MPI
739 libraries and put them into an options file as described in Section
740 \ref{sect:genmake}. One can start with one of the examples in:
741 \begin{rawhtml} <A
742 href="http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/tools/build_options/">
743 \end{rawhtml}
744 \begin{center}
745 \texttt{MITgcm/tools/build\_options/}
746 \end{center}
747 \begin{rawhtml} </A> \end{rawhtml}
748 such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
749 \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
750 hand. You may need help from your user guide or local systems
751 administrator to determine the exact location of the MPI libraries.
752 If libraries are not installed, MPI implementations and related
753 tools are available including:
754 \begin{itemize}
755 \item \begin{rawhtml} <A
756 href="http://www-unix.mcs.anl.gov/mpi/mpich/">
757 \end{rawhtml}
758 MPICH
759 \begin{rawhtml} </A> \end{rawhtml}
760
761 \item \begin{rawhtml} <A
762 href="http://www.lam-mpi.org/">
763 \end{rawhtml}
764 LAM/MPI
765 \begin{rawhtml} </A> \end{rawhtml}
766
767 \item \begin{rawhtml} <A
768 href="http://www.osc.edu/~pw/mpiexec/">
769 \end{rawhtml}
770 MPIexec
771 \begin{rawhtml} </A> \end{rawhtml}
772 \end{itemize}
773
774 \item Build the code with the \texttt{genmake2} \texttt{-mpi} option
775 (see Section \ref{sect:genmake}) using commands such as:
776 {\footnotesize \begin{verbatim}
777 % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
778 % make depend
779 % make
780 \end{verbatim} }
781
782 \item Run the code with the appropriate MPI ``run'' or ``exec''
783 program provided with your particular implementation of MPI.
784 Typical MPI packages such as MPICH will use something like:
785 \begin{verbatim}
786 % mpirun -np 4 -machinefile mf ./mitgcmuv
787 \end{verbatim}
788 Sightly more complicated scripts may be needed for many machines
789 since execution of the code may be controlled by both the MPI
790 library and a job scheduling and queueing system such as PBS,
791 LoadLeveller, Condor, or any of a number of similar tools. A few
792 example scripts (those used for our \begin{rawhtml} <A
793 href="http://mitgcm.org/testing.html"> \end{rawhtml}regular
794 verification runs\begin{rawhtml} </A> \end{rawhtml}) are available
795 at:
796 \begin{rawhtml} <A
797 href="http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">
798 \end{rawhtml}
799 {\footnotesize \tt
800 http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm\_contrib/test\_scripts/ }
801 \begin{rawhtml} </A> \end{rawhtml}
802
803 \end{enumerate}
804
805 An example of the above process on the MITgcm cluster (``cg01'') using
806 the GNU g77 compiler and the mpich MPI library is:
807
808 {\footnotesize \begin{verbatim}
809 % cd MITgcm/verification/exp5
810 % mkdir build
811 % cd build
812 % ../../../tools/genmake2 -mpi -mods=../code \
813 -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
814 % make depend
815 % make
816 % cd ../input
817 % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
818 -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
819 \end{verbatim} }
820
821 \section[Running MITgcm]{Running the model in prognostic mode}
822 \label{sect:runModel}
823 \begin{rawhtml}
824 <!-- CMIREDIR:runModel: -->
825 \end{rawhtml}
826
827 If compilation finished succesfully (section \ref{sect:buildingCode})
828 then an executable called \texttt{mitgcmuv} will now exist in the
829 local directory.
830
831 To run the model as a single process (\textit{ie.} not in parallel)
832 simply type:
833 \begin{verbatim}
834 % ./mitgcmuv
835 \end{verbatim}
836 The ``./'' is a safe-guard to make sure you use the local executable
837 in case you have others that exist in your path (surely odd if you
838 do!). The above command will spew out many lines of text output to
839 your screen. This output contains details such as parameter values as
840 well as diagnostics such as mean Kinetic energy, largest CFL number,
841 etc. It is worth keeping this text output with the binary output so we
842 normally re-direct the \texttt{stdout} stream as follows:
843 \begin{verbatim}
844 % ./mitgcmuv > output.txt
845 \end{verbatim}
846 In the event that the model encounters an error and stops, it is very
847 helpful to include the last few line of this \texttt{output.txt} file
848 along with the (\texttt{stderr}) error message within any bug reports.
849
850 For the example experiments in \texttt{verification}, an example of the
851 output is kept in \texttt{results/output.txt} for comparison. You can
852 compare your \texttt{output.txt} with the corresponding one for that
853 experiment to check that the set-up works.
854
855
856
857 \subsection{Output files}
858
859 The model produces various output files and, when using \texttt{mnc},
860 sometimes even directories. Depending upon the I/O package(s)
861 selected at compile time (either \texttt{mdsio} or \texttt{mnc} or
862 both as determined by \texttt{code/packages.conf}) and the run-time
863 flags set (in \texttt{input/data.pkg}), the following output may
864 appear.
865
866
867 \subsubsection{MDSIO output files}
868
869 The ``traditional'' output files are generated by the \texttt{mdsio}
870 package. At a minimum, the instantaneous ``state'' of the model is
871 written out, which is made of the following files:
872
873 \begin{itemize}
874 \item \texttt{U.00000nIter} - zonal component of velocity field (m/s
875 and positive eastward).
876
877 \item \texttt{V.00000nIter} - meridional component of velocity field
878 (m/s and positive northward).
879
880 \item \texttt{W.00000nIter} - vertical component of velocity field
881 (ocean: m/s and positive upward, atmosphere: Pa/s and positive
882 towards increasing pressure i.e. downward).
883
884 \item \texttt{T.00000nIter} - potential temperature (ocean:
885 $^{\circ}\mathrm{C}$, atmosphere: $^{\circ}\mathrm{K}$).
886
887 \item \texttt{S.00000nIter} - ocean: salinity (psu), atmosphere: water
888 vapor (g/kg).
889
890 \item \texttt{Eta.00000nIter} - ocean: surface elevation (m),
891 atmosphere: surface pressure anomaly (Pa).
892 \end{itemize}
893
894 The chain \texttt{00000nIter} consists of ten figures that specify the
895 iteration number at which the output is written out. For example,
896 \texttt{U.0000000300} is the zonal velocity at iteration 300.
897
898 In addition, a ``pickup'' or ``checkpoint'' file called:
899
900 \begin{itemize}
901 \item \texttt{pickup.00000nIter}
902 \end{itemize}
903
904 is written out. This file represents the state of the model in a condensed
905 form and is used for restarting the integration. If the C-D scheme is used,
906 there is an additional ``pickup'' file:
907
908 \begin{itemize}
909 \item \texttt{pickup\_cd.00000nIter}
910 \end{itemize}
911
912 containing the D-grid velocity data and that has to be written out as well
913 in order to restart the integration. Rolling checkpoint files are the same
914 as the pickup files but are named differently. Their name contain the chain
915 \texttt{ckptA} or \texttt{ckptB} instead of \texttt{00000nIter}. They can be
916 used to restart the model but are overwritten every other time they are
917 output to save disk space during long integrations.
918
919
920
921 \subsubsection{MNC output files}
922
923 Unlike the \texttt{mdsio} output, the \texttt{mnc}--generated output
924 is usually (though not necessarily) placed within a subdirectory with
925 a name such as \texttt{mnc\_test\_\${DATE}\_\${SEQ}}. The files
926 within this subdirectory are all in the ``self-describing'' netCDF
927 format and can thus be browsed and/or plotted using tools such as:
928 \begin{itemize}
929 \item \texttt{ncdump} is a utility which is typically included
930 with every netCDF install:
931 \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
932 \begin{verbatim}
933 http://www.unidata.ucar.edu/packages/netcdf/
934 \end{verbatim}
935 \begin{rawhtml} </A> \end{rawhtml} and it converts the netCDF
936 binaries into formatted ASCII text files.
937
938 \item \texttt{ncview} utility is a very convenient and quick way
939 to plot netCDF data and it runs on most OSes:
940 \begin{rawhtml} <A href="http://meteora.ucsd.edu/~pierce/ncview_home_page.html"> \end{rawhtml}
941 \begin{verbatim}
942 http://meteora.ucsd.edu/~pierce/ncview_home_page.html
943 \end{verbatim}
944 \begin{rawhtml} </A> \end{rawhtml}
945
946 \item MatLAB(c) and other common post-processing environments provide
947 various netCDF interfaces including:
948 \begin{rawhtml} <A href="http://mexcdf.sourceforge.net/"> \end{rawhtml}
949 \begin{verbatim}
950 http://mexcdf.sourceforge.net/
951 \end{verbatim}
952 \begin{rawhtml} </A> \end{rawhtml}
953 \begin{rawhtml} <A href="http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html"> \end{rawhtml}
954 \begin{verbatim}
955 http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html
956 \end{verbatim}
957 \begin{rawhtml} </A> \end{rawhtml}
958 \end{itemize}
959
960
961 \subsection{Looking at the output}
962
963 The ``traditional'' or mdsio model data are written according to a
964 ``meta/data'' file format. Each variable is associated with two files
965 with suffix names \texttt{.data} and \texttt{.meta}. The
966 \texttt{.data} file contains the data written in binary form
967 (big\_endian by default). The \texttt{.meta} file is a ``header'' file
968 that contains information about the size and the structure of the
969 \texttt{.data} file. This way of organizing the output is particularly
970 useful when running multi-processors calculations. The base version of
971 the model includes a few matlab utilities to read output files written
972 in this format. The matlab scripts are located in the directory
973 \texttt{utils/matlab} under the root tree. The script \texttt{rdmds.m}
974 reads the data. Look at the comments inside the script to see how to
975 use it.
976
977 Some examples of reading and visualizing some output in {\em Matlab}:
978 \begin{verbatim}
979 % matlab
980 >> H=rdmds('Depth');
981 >> contourf(H');colorbar;
982 >> title('Depth of fluid as used by model');
983
984 >> eta=rdmds('Eta',10);
985 >> imagesc(eta');axis ij;colorbar;
986 >> title('Surface height at iter=10');
987
988 >> eta=rdmds('Eta',[0:10:100]);
989 >> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
990 \end{verbatim}
991
992 Similar scripts for netCDF output (\texttt{rdmnc.m}) are available and
993 they are described in Section \ref{sec:pkg:mnc}.
994

  ViewVC Help
Powered by ViewVC 1.1.22