/[MITgcm]/manual/s_getstarted/text/getting_started.tex
ViewVC logotype

Contents of /manual/s_getstarted/text/getting_started.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.41 - (show annotations) (download) (as text)
Fri Jan 22 00:38:16 2010 UTC (15 years, 5 months ago) by jmc
Branch: MAIN
Changes since 1.40: +10 -3 lines
File MIME type: application/x-tex
fix few other links

1 % $Header: /u/gcmpack/manual/part3/getting_started.tex,v 1.40 2010/01/21 19:20:08 jmc Exp $
2 % $Name: $
3
4 %\section{Getting started}
5
6 We believe the best way to familiarize yourself with the
7 model is to run the case study examples provided with the base
8 version. Information on how to obtain, compile, and run the code is
9 found here as well as a brief description of the model structure
10 directory and the case study examples. Information is also provided
11 here on how to customize the code when you are ready to try implementing
12 the configuration you have in mind. The code and algorithm
13 are described more fully in chapters \ref{chap:discretization} and
14 \ref{chap:sarch}.
15
16 \section{Where to find information}
17 \label{sect:whereToFindInfo}
18 \begin{rawhtml}
19 <!-- CMIREDIR:whereToFindInfo: -->
20 \end{rawhtml}
21
22 There is a web-archived support mailing list for the model that
23 you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
24 \begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
25 \begin{verbatim}
26 http://mitgcm.org/mailman/listinfo/mitgcm-support/
27 http://mitgcm.org/pipermail/mitgcm-support/
28 \end{verbatim}
29 \begin{rawhtml} </A> \end{rawhtml}
30
31 \section{Obtaining the code}
32 \label{sect:obtainingCode}
33 \begin{rawhtml}
34 <!-- CMIREDIR:obtainingCode: -->
35 \end{rawhtml}
36
37 MITgcm can be downloaded from our system by following
38 the instructions below. As a courtesy we ask that you send e-mail to us at
39 \begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
40 MITgcm-support@mitgcm.org
41 \begin{rawhtml} </A> \end{rawhtml}
42 to enable us to keep track of who's using the model and in what application.
43 You can download the model two ways:
44
45 \begin{enumerate}
46 \item Using CVS software. CVS is a freely available source code management
47 tool. To use CVS you need to have the software installed. Many systems
48 come with CVS pre-installed, otherwise good places to look for
49 the software for a particular platform are
50 \begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
51 cvshome.org
52 \begin{rawhtml} </A> \end{rawhtml}
53 and
54 \begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
55 wincvs.org
56 \begin{rawhtml} </A> \end{rawhtml}
57 .
58
59 \item Using a tar file. This method is simple and does not
60 require any special software. However, this method does not
61 provide easy support for maintenance updates.
62
63 \end{enumerate}
64
65 \subsection{Method 1 - Checkout from CVS}
66 \label{sect:cvs_checkout}
67
68 If CVS is available on your system, we strongly encourage you to use it. CVS
69 provides an efficient and elegant way of organizing your code and keeping
70 track of your changes. If CVS is not available on your machine, you can also
71 download a tar file.
72
73 Before you can use CVS, the following environment variable(s) should
74 be set within your shell. For a csh or tcsh shell, put the following
75 \begin{verbatim}
76 % setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
77 \end{verbatim}
78 in your \texttt{.cshrc} or \texttt{.tcshrc} file. For bash or sh
79 shells, put:
80 \begin{verbatim}
81 % export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
82 \end{verbatim}
83 in your \texttt{.profile} or \texttt{.bashrc} file.
84
85
86 To get MITgcm through CVS, first register with the MITgcm CVS server
87 using command:
88 \begin{verbatim}
89 % cvs login ( CVS password: cvsanon )
90 \end{verbatim}
91 You only need to do a ``cvs login'' once.
92
93 To obtain the latest sources type:
94 \begin{verbatim}
95 % cvs co MITgcm
96 \end{verbatim}
97 or to get a specific release type:
98 \begin{verbatim}
99 % cvs co -P -r checkpoint52i_post MITgcm
100 \end{verbatim}
101 The MITgcm web site contains further directions concerning the source
102 code and CVS. It also contains a web interface to our CVS archive so
103 that one may easily view the state of files, revisions, and other
104 development milestones:
105 %\begin{rawhtml} <A href="http://mitgcm.org/download" target="idontexist"> \end{rawhtml}
106 \begin{rawhtml} <A href="http://mitgcm.org/viewvc/MITgcm/MITgcm/" target="idontexist"> \end{rawhtml}
107 \begin{verbatim}
108 http://mitgcm.org/source_code.html
109 \end{verbatim}
110 \begin{rawhtml} </A> \end{rawhtml}
111
112 As a convenience, the MITgcm CVS server contains aliases which are
113 named subsets of the codebase. These aliases can be especially
114 helpful when used over slow internet connections or on machines with
115 restricted storage space. Table \ref{tab:cvsModules} contains a list
116 of CVS aliases
117 \begin{table}[htb]
118 \centering
119 \begin{tabular}[htb]{|lp{3.25in}|}\hline
120 \textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
121 \texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
122 \texttt{MITgcm\_verif\_basic}
123 & Source code plus a small set of the verification examples
124 (\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
125 \texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
126 \texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
127 \texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
128 \texttt{MITgcm\_verif\_all} & Source code plus all of the
129 verification examples. \\\hline
130 \end{tabular}
131 \caption{MITgcm CVS Modules}
132 \label{tab:cvsModules}
133 \end{table}
134
135 The checkout process creates a directory called \texttt{MITgcm}. If
136 the directory \texttt{MITgcm} exists this command updates your code
137 based on the repository. Each directory in the source tree contains a
138 directory \texttt{CVS}. This information is required by CVS to keep
139 track of your file versions with respect to the repository. Don't edit
140 the files in \texttt{CVS}! You can also use CVS to download code
141 updates. More extensive information on using CVS for maintaining
142 MITgcm code can be found
143 \begin{rawhtml} <A href="http://mitgcm.org/public/using_cvs.html" target="idontexist"> \end{rawhtml}
144 here
145 \begin{rawhtml} </A> \end{rawhtml}
146 .
147 It is important to note that the CVS aliases in Table
148 \ref{tab:cvsModules} cannot be used in conjunction with the CVS
149 \texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
150 they create can be changed to a different name following the check-out:
151 \begin{verbatim}
152 % cvs co MITgcm_verif_basic
153 % mv MITgcm MITgcm_verif_basic
154 \end{verbatim}
155
156 \subsubsection{Upgrading from an earlier version}
157
158 If you already have an earlier version of the code you can ``upgrade''
159 your copy instead of downloading the entire repository again. First,
160 ``cd'' (change directory) to the top of your working copy:
161 \begin{verbatim}
162 % cd MITgcm
163 \end{verbatim}
164 and then issue the cvs update command such as:
165 \begin{verbatim}
166 % cvs -q update -r checkpoint52i_post -d -P
167 \end{verbatim}
168 This will update the ``tag'' to ``checkpoint52i\_post'', add any new
169 directories (-d) and remove any empty directories (-P). The -q option
170 means be quiet which will reduce the number of messages you'll see in
171 the terminal. If you have modified the code prior to upgrading, CVS
172 will try to merge your changes with the upgrades. If there is a
173 conflict between your modifications and the upgrade, it will report
174 that file with a ``C'' in front, e.g.:
175 \begin{verbatim}
176 C model/src/ini_parms.F
177 \end{verbatim}
178 If the list of conflicts scrolled off the screen, you can re-issue the
179 cvs update command and it will report the conflicts. Conflicts are
180 indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
181 ``$>>>>>>>$''. For example,
182 {\small
183 \begin{verbatim}
184 <<<<<<< ini_parms.F
185 & bottomDragLinear,myOwnBottomDragCoefficient,
186 =======
187 & bottomDragLinear,bottomDragQuadratic,
188 >>>>>>> 1.18
189 \end{verbatim}
190 }
191 means that you added ``myOwnBottomDragCoefficient'' to a namelist at
192 the same time and place that we added ``bottomDragQuadratic''. You
193 need to resolve this conflict and in this case the line should be
194 changed to:
195 {\small
196 \begin{verbatim}
197 & bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
198 \end{verbatim}
199 }
200 and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
201 Unless you are making modifications which exactly parallel
202 developments we make, these types of conflicts should be rare.
203
204 \paragraph*{Upgrading to the current pre-release version}
205
206 We don't make a ``release'' for every little patch and bug fix in
207 order to keep the frequency of upgrades to a minimum. However, if you
208 have run into a problem for which ``we have already fixed in the
209 latest code'' and we haven't made a ``tag'' or ``release'' since that
210 patch then you'll need to get the latest code:
211 \begin{verbatim}
212 % cvs -q update -A -d -P
213 \end{verbatim}
214 Unlike, the ``check-out'' and ``update'' procedures above, there is no
215 ``tag'' or release name. The -A tells CVS to upgrade to the
216 very latest version. As a rule, we don't recommend this since you
217 might upgrade while we are in the processes of checking in the code so
218 that you may only have part of a patch. Using this method of updating
219 also means we can't tell what version of the code you are working
220 with. So please be sure you understand what you're doing.
221
222 \subsection{Method 2 - Tar file download}
223 \label{sect:conventionalDownload}
224
225 If you do not have CVS on your system, you can download the model as a
226 tar file from the web site at:
227 \begin{rawhtml} <A href=http://mitgcm.org/download/ target="idontexist"> \end{rawhtml}
228 \begin{verbatim}
229 http://mitgcm.org/download/
230 \end{verbatim}
231 \begin{rawhtml} </A> \end{rawhtml}
232 The tar file still contains CVS information which we urge you not to
233 delete; even if you do not use CVS yourself the information can help
234 us if you should need to send us your copy of the code. If a recent
235 tar file does not exist, then please contact the developers through
236 the
237 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
238 MITgcm-support@mitgcm.org
239 \begin{rawhtml} </A> \end{rawhtml}
240 mailing list.
241
242 \section{Model and directory structure}
243 \begin{rawhtml}
244 <!-- CMIREDIR:directory_structure: -->
245 \end{rawhtml}
246
247 The ``numerical'' model is contained within a execution environment
248 support wrapper. This wrapper is designed to provide a general
249 framework for grid-point models. MITgcmUV is a specific numerical
250 model that uses the framework. Under this structure the model is split
251 into execution environment support code and conventional numerical
252 model code. The execution environment support code is held under the
253 \texttt{eesupp} directory. The grid point model code is held under the
254 \texttt{model} directory. Code execution actually starts in the
255 \texttt{eesupp} routines and not in the \texttt{model} routines. For
256 this reason the top-level \texttt{MAIN.F} is in the
257 \texttt{eesupp/src} directory. In general, end-users should not need
258 to worry about this level. The top-level routine for the numerical
259 part of the code is in \texttt{model/src/THE\_MODEL\_MAIN.F}. Here is
260 a brief description of the directory structure of the model under the
261 root tree (a detailed description is given in section 3: Code
262 structure).
263
264 \begin{itemize}
265
266 \item \texttt{doc}: contains brief documentation notes.
267
268 \item \texttt{eesupp}: contains the execution environment source code.
269 Also subdivided into two subdirectories \texttt{inc} and
270 \texttt{src}.
271
272 \item \texttt{model}: this directory contains the main source code.
273 Also subdivided into two subdirectories \texttt{inc} and
274 \texttt{src}.
275
276 \item \texttt{pkg}: contains the source code for the packages. Each
277 package corresponds to a subdirectory. For example, \texttt{gmredi}
278 contains the code related to the Gent-McWilliams/Redi scheme,
279 \texttt{aim} the code relative to the atmospheric intermediate
280 physics. The packages are described in detail in chapter \ref{chap.packagesI}.
281
282 \item \texttt{tools}: this directory contains various useful tools.
283 For example, \texttt{genmake2} is a script written in csh (C-shell)
284 that should be used to generate your makefile. The directory
285 \texttt{adjoint} contains the makefile specific to the Tangent
286 linear and Adjoint Compiler (TAMC) that generates the adjoint code.
287 The latter is described in detail in part \ref{chap.ecco}.
288 This directory also contains the subdirectory build\_options, which
289 contains the `optfiles' with the compiler options for the different
290 compilers and machines that can run MITgcm.
291
292 \item \texttt{utils}: this directory contains various utilities. The
293 subdirectory \texttt{knudsen2} contains code and a makefile that
294 compute coefficients of the polynomial approximation to the knudsen
295 formula for an ocean nonlinear equation of state. The
296 \texttt{matlab} subdirectory contains matlab scripts for reading
297 model output directly into matlab. \texttt{scripts} contains C-shell
298 post-processing scripts for joining processor-based and tiled-based
299 model output. The subdirectory exch2 contains the code needed for
300 the exch2 package to work with different combinations of domain
301 decompositions.
302
303 \item \texttt{verification}: this directory contains the model
304 examples. See section \ref{sect:modelExamples}.
305
306 \item \texttt{jobs}: contains sample job scripts for running MITgcm.
307
308 \item \texttt{lsopt}: Line search code used for optimization.
309
310 \item \texttt{optim}: Interface between MITgcm and line search code.
311
312 \end{itemize}
313
314 \section[Building MITgcm]{Building the code}
315 \label{sect:buildingCode}
316 \begin{rawhtml}
317 <!-- CMIREDIR:buildingCode: -->
318 \end{rawhtml}
319
320 To compile the code, we use the \texttt{make} program. This uses a
321 file (\texttt{Makefile}) that allows us to pre-process source files,
322 specify compiler and optimization options and also figures out any
323 file dependencies. We supply a script (\texttt{genmake2}), described
324 in section \ref{sect:genmake}, that automatically creates the
325 \texttt{Makefile} for you. You then need to build the dependencies and
326 compile the code.
327
328 As an example, assume that you want to build and run experiment
329 \texttt{verification/exp2}. The are multiple ways and places to
330 actually do this but here let's build the code in
331 \texttt{verification/exp2/build}:
332 \begin{verbatim}
333 % cd verification/exp2/build
334 \end{verbatim}
335 First, build the \texttt{Makefile}:
336 \begin{verbatim}
337 % ../../../tools/genmake2 -mods=../code
338 \end{verbatim}
339 The command line option tells \texttt{genmake} to override model source
340 code with any files in the directory \texttt{../code/}.
341
342 On many systems, the \texttt{genmake2} program will be able to
343 automatically recognize the hardware, find compilers and other tools
344 within the user's path (``\texttt{echo \$PATH}''), and then choose an
345 appropriate set of options from the files (``optfiles'') contained in
346 the \texttt{tools/build\_options} directory. Under some
347 circumstances, a user may have to create a new ``optfile'' in order to
348 specify the exact combination of compiler, compiler flags, libraries,
349 and other options necessary to build a particular configuration of
350 MITgcm. In such cases, it is generally helpful to read the existing
351 ``optfiles'' and mimic their syntax.
352
353 Through the MITgcm-support list, the MITgcm developers are willing to
354 provide help writing or modifing ``optfiles''. And we encourage users
355 to post new ``optfiles'' (particularly ones for new machines or
356 architectures) to the
357 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
358 MITgcm-support@mitgcm.org
359 \begin{rawhtml} </A> \end{rawhtml}
360 list.
361
362 To specify an optfile to \texttt{genmake2}, the syntax is:
363 \begin{verbatim}
364 % ../../../tools/genmake2 -mods=../code -of /path/to/optfile
365 \end{verbatim}
366
367 Once a \texttt{Makefile} has been generated, we create the
368 dependencies with the command:
369 \begin{verbatim}
370 % make depend
371 \end{verbatim}
372 This modifies the \texttt{Makefile} by attaching a (usually, long)
373 list of files upon which other files depend. The purpose of this is to
374 reduce re-compilation if and when you start to modify the code. The
375 {\tt make depend} command also creates links from the model source to
376 this directory. It is important to note that the {\tt make depend}
377 stage will occasionally produce warnings or errors since the
378 dependency parsing tool is unable to find all of the necessary header
379 files (\textit{eg.} \texttt{netcdf.inc}). In these circumstances, it
380 is usually OK to ignore the warnings/errors and proceed to the next
381 step.
382
383 Next one can compile the code using:
384 \begin{verbatim}
385 % make
386 \end{verbatim}
387 The {\tt make} command creates an executable called \texttt{mitgcmuv}.
388 Additional make ``targets'' are defined within the makefile to aid in
389 the production of adjoint and other versions of MITgcm. On SMP
390 (shared multi-processor) systems, the build process can often be sped
391 up appreciably using the command:
392 \begin{verbatim}
393 % make -j 2
394 \end{verbatim}
395 where the ``2'' can be replaced with a number that corresponds to the
396 number of CPUs available.
397
398 Now you are ready to run the model. General instructions for doing so are
399 given in section \ref{sect:runModel}. Here, we can run the model by
400 first creating links to all the input files:
401 \begin{verbatim}
402 ln -s ../input/* .
403 \end{verbatim}
404 and then calling the executable with:
405 \begin{verbatim}
406 ./mitgcmuv > output.txt
407 \end{verbatim}
408 where we are re-directing the stream of text output to the file
409 \texttt{output.txt}.
410
411 \subsection{Building/compiling the code elsewhere}
412
413 In the example above (section \ref{sect:buildingCode}) we built the
414 executable in the {\em input} directory of the experiment for
415 convenience. You can also configure and compile the code in other
416 locations, for example on a scratch disk with out having to copy the
417 entire source tree. The only requirement to do so is you have {\tt
418 genmake2} in your path or you know the absolute path to {\tt
419 genmake2}.
420
421 The following sections outline some possible methods of organizing
422 your source and data.
423
424 \subsubsection{Building from the {\em ../code directory}}
425
426 This is just as simple as building in the {\em input/} directory:
427 \begin{verbatim}
428 % cd verification/exp2/code
429 % ../../../tools/genmake2
430 % make depend
431 % make
432 \end{verbatim}
433 However, to run the model the executable ({\em mitgcmuv}) and input
434 files must be in the same place. If you only have one calculation to make:
435 \begin{verbatim}
436 % cd ../input
437 % cp ../code/mitgcmuv ./
438 % ./mitgcmuv > output.txt
439 \end{verbatim}
440 or if you will be making multiple runs with the same executable:
441 \begin{verbatim}
442 % cd ../
443 % cp -r input run1
444 % cp code/mitgcmuv run1
445 % cd run1
446 % ./mitgcmuv > output.txt
447 \end{verbatim}
448
449 \subsubsection{Building from a new directory}
450
451 Since the {\em input} directory contains input files it is often more
452 useful to keep {\em input} pristine and build in a new directory
453 within {\em verification/exp2/}:
454 \begin{verbatim}
455 % cd verification/exp2
456 % mkdir build
457 % cd build
458 % ../../../tools/genmake2 -mods=../code
459 % make depend
460 % make
461 \end{verbatim}
462 This builds the code exactly as before but this time you need to copy
463 either the executable or the input files or both in order to run the
464 model. For example,
465 \begin{verbatim}
466 % cp ../input/* ./
467 % ./mitgcmuv > output.txt
468 \end{verbatim}
469 or if you tend to make multiple runs with the same executable then
470 running in a new directory each time might be more appropriate:
471 \begin{verbatim}
472 % cd ../
473 % mkdir run1
474 % cp build/mitgcmuv run1/
475 % cp input/* run1/
476 % cd run1
477 % ./mitgcmuv > output.txt
478 \end{verbatim}
479
480 \subsubsection{Building on a scratch disk}
481
482 Model object files and output data can use up large amounts of disk
483 space so it is often the case that you will be operating on a large
484 scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
485 following commands will build the model in {\em /scratch/exp2-run1}:
486 \begin{verbatim}
487 % cd /scratch/exp2-run1
488 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
489 -mods=~/MITgcm/verification/exp2/code
490 % make depend
491 % make
492 \end{verbatim}
493 To run the model here, you'll need the input files:
494 \begin{verbatim}
495 % cp ~/MITgcm/verification/exp2/input/* ./
496 % ./mitgcmuv > output.txt
497 \end{verbatim}
498
499 As before, you could build in one directory and make multiple runs of
500 the one experiment:
501 \begin{verbatim}
502 % cd /scratch/exp2
503 % mkdir build
504 % cd build
505 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
506 -mods=~/MITgcm/verification/exp2/code
507 % make depend
508 % make
509 % cd ../
510 % cp -r ~/MITgcm/verification/exp2/input run2
511 % cd run2
512 % ./mitgcmuv > output.txt
513 \end{verbatim}
514
515
516 \subsection{Using \texttt{genmake2}}
517 \label{sect:genmake}
518
519 To compile the code, first use the program \texttt{genmake2} (located
520 in the \texttt{tools} directory) to generate a Makefile.
521 \texttt{genmake2} is a shell script written to work with all
522 ``sh''--compatible shells including bash v1, bash v2, and Bourne.
523 Internally, \texttt{genmake2} determines the locations of needed
524 files, the compiler, compiler options, libraries, and Unix tools. It
525 relies upon a number of ``optfiles'' located in the
526 \texttt{tools/build\_options} directory.
527
528 The purpose of the optfiles is to provide all the compilation options
529 for particular ``platforms'' (where ``platform'' roughly means the
530 combination of the hardware and the compiler) and code configurations.
531 Given the combinations of possible compilers and library dependencies
532 ({\it eg.} MPI and NetCDF) there may be numerous optfiles available
533 for a single machine. The naming scheme for the majority of the
534 optfiles shipped with the code is
535 \begin{center}
536 {\bf OS\_HARDWARE\_COMPILER }
537 \end{center}
538 where
539 \begin{description}
540 \item[OS] is the name of the operating system (generally the
541 lower-case output of the {\tt 'uname'} command)
542 \item[HARDWARE] is a string that describes the CPU type and
543 corresponds to output from the {\tt 'uname -m'} command:
544 \begin{description}
545 \item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
546 and athlon
547 \item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
548 \item[amd64] is AMD x86\_64 systems
549 \item[ppc] is for Mac PowerPC systems
550 \end{description}
551 \item[COMPILER] is the compiler name (generally, the name of the
552 FORTRAN executable)
553 \end{description}
554
555 In many cases, the default optfiles are sufficient and will result in
556 usable Makefiles. However, for some machines or code configurations,
557 new ``optfiles'' must be written. To create a new optfile, it is
558 generally best to start with one of the defaults and modify it to suit
559 your needs. Like \texttt{genmake2}, the optfiles are all written
560 using a simple ``sh''--compatible syntax. While nearly all variables
561 used within \texttt{genmake2} may be specified in the optfiles, the
562 critical ones that should be defined are:
563
564 \begin{description}
565 \item[FC] the FORTRAN compiler (executable) to use
566 \item[DEFINES] the command-line DEFINE options passed to the compiler
567 \item[CPP] the C pre-processor to use
568 \item[NOOPTFLAGS] options flags for special files that should not be
569 optimized
570 \end{description}
571
572 For example, the optfile for a typical Red Hat Linux machine (``ia32''
573 architecture) using the GCC (g77) compiler is
574 \begin{verbatim}
575 FC=g77
576 DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
577 CPP='cpp -traditional -P'
578 NOOPTFLAGS='-O0'
579 # For IEEE, use the "-ffloat-store" option
580 if test "x$IEEE" = x ; then
581 FFLAGS='-Wimplicit -Wunused -Wuninitialized'
582 FOPTIM='-O3 -malign-double -funroll-loops'
583 else
584 FFLAGS='-Wimplicit -Wunused -ffloat-store'
585 FOPTIM='-O0 -malign-double'
586 fi
587 \end{verbatim}
588
589 If you write an optfile for an unrepresented machine or compiler, you
590 are strongly encouraged to submit the optfile to the MITgcm project
591 for inclusion. Please send the file to the
592 \begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
593 \begin{center}
594 MITgcm-support@mitgcm.org
595 \end{center}
596 \begin{rawhtml} </A> \end{rawhtml}
597 mailing list.
598
599 In addition to the optfiles, \texttt{genmake2} supports a number of
600 helpful command-line options. A complete list of these options can be
601 obtained from:
602 \begin{verbatim}
603 % genmake2 -h
604 \end{verbatim}
605
606 The most important command-line options are:
607 \begin{description}
608
609 \item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
610 should be used for a particular build.
611
612 If no "optfile" is specified (either through the command line or the
613 MITGCM\_OPTFILE environment variable), genmake2 will try to make a
614 reasonable guess from the list provided in {\em
615 tools/build\_options}. The method used for making this guess is
616 to first determine the combination of operating system and hardware
617 (eg. "linux\_ia32") and then find a working FORTRAN compiler within
618 the user's path. When these three items have been identified,
619 genmake2 will try to find an optfile that has a matching name.
620
621 \item[\texttt{--pdefault='PKG1 PKG2 PKG3 ...'}] specifies the default
622 set of packages to be used. The normal order of precedence for
623 packages is as follows:
624 \begin{enumerate}
625 \item If available, the command line (\texttt{--pdefault}) settings
626 over-rule any others.
627
628 \item Next, \texttt{genmake2} will look for a file named
629 ``\texttt{packages.conf}'' in the local directory or in any of the
630 directories specified with the \texttt{--mods} option.
631
632 \item Finally, if neither of the above are available,
633 \texttt{genmake2} will use the \texttt{/pkg/pkg\_default} file.
634 \end{enumerate}
635
636 \item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
637 used for packages.
638
639 If not specified, the default dependency file {\em pkg/pkg\_depend}
640 is used. The syntax for this file is parsed on a line-by-line basis
641 where each line containes either a comment ("\#") or a simple
642 "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
643 specifies a "must be used with" or a "must not be used with"
644 relationship, respectively. If no rule is specified, then it is
645 assumed that the two packages are compatible and will function
646 either with or without each other.
647
648 \item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
649 automatic differentiation options file to be used. The file is
650 analogous to the ``optfile'' defined above but it specifies
651 information for the AD build process.
652
653 The default file is located in {\em
654 tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
655 and "TAMC" compilers. An alternate version is also available at
656 {\em tools/adjoint\_options/adjoint\_staf} that selects the newer
657 "STAF" compiler. As with any compilers, it is helpful to have their
658 directories listed in your {\tt \$PATH} environment variable.
659
660 \item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
661 directories containing ``modifications''. These directories contain
662 files with names that may (or may not) exist in the main MITgcm
663 source tree but will be overridden by any identically-named sources
664 within the ``MODS'' directories.
665
666 The order of precedence for this "name-hiding" is as follows:
667 \begin{itemize}
668 \item ``MODS'' directories (in the order given)
669 \item Packages either explicitly specified or provided by default
670 (in the order given)
671 \item Packages included due to package dependencies (in the order
672 that that package dependencies are parsed)
673 \item The "standard dirs" (which may have been specified by the
674 ``-standarddirs'' option)
675 \end{itemize}
676
677 \item[\texttt{--mpi}] This option enables certain MPI features (using
678 CPP \texttt{\#define}s) within the code and is necessary for MPI
679 builds (see Section \ref{sect:mpi-build}).
680
681 \item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
682 soft-links and other bugs common with the \texttt{make} versions
683 provided by commercial Unix vendors, GNU \texttt{make} (sometimes
684 called \texttt{gmake}) should be preferred. This option provides a
685 means for specifying the make executable to be used.
686
687 \item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
688 machines, the ``bash'' shell is unavailable. To run on these
689 systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
690 a Bourne, POSIX, or compatible) shell. The syntax in these
691 circumstances is:
692 \begin{center}
693 \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
694 \end{center}
695 where \texttt{/bin/sh} can be replaced with the full path and name
696 of the desired shell.
697
698 \end{description}
699
700
701 \subsection{Building with MPI}
702 \label{sect:mpi-build}
703
704 Building MITgcm to use MPI libraries can be complicated due to the
705 variety of different MPI implementations available, their dependencies
706 or interactions with different compilers, and their often ad-hoc
707 locations within file systems. For these reasons, its generally a
708 good idea to start by finding and reading the documentation for your
709 machine(s) and, if necessary, seeking help from your local systems
710 administrator.
711
712 The steps for building MITgcm with MPI support are:
713 \begin{enumerate}
714
715 \item Determine the locations of your MPI-enabled compiler and/or MPI
716 libraries and put them into an options file as described in Section
717 \ref{sect:genmake}. One can start with one of the examples in:
718 \begin{rawhtml} <A
719 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/build_options/">
720 \end{rawhtml}
721 \begin{center}
722 \texttt{MITgcm/tools/build\_options/}
723 \end{center}
724 \begin{rawhtml} </A> \end{rawhtml}
725 such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
726 \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
727 hand. You may need help from your user guide or local systems
728 administrator to determine the exact location of the MPI libraries.
729 If libraries are not installed, MPI implementations and related
730 tools are available including:
731 \begin{itemize}
732 \item \begin{rawhtml} <A
733 href="http://www-unix.mcs.anl.gov/mpi/mpich/">
734 \end{rawhtml}
735 MPICH
736 \begin{rawhtml} </A> \end{rawhtml}
737
738 \item \begin{rawhtml} <A
739 href="http://www.lam-mpi.org/">
740 \end{rawhtml}
741 LAM/MPI
742 \begin{rawhtml} </A> \end{rawhtml}
743
744 \item \begin{rawhtml} <A
745 href="http://www.osc.edu/~pw/mpiexec/">
746 \end{rawhtml}
747 MPIexec
748 \begin{rawhtml} </A> \end{rawhtml}
749 \end{itemize}
750
751 \item Build the code with the \texttt{genmake2} \texttt{-mpi} option
752 (see Section \ref{sect:genmake}) using commands such as:
753 {\footnotesize \begin{verbatim}
754 % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
755 % make depend
756 % make
757 \end{verbatim} }
758
759 \item Run the code with the appropriate MPI ``run'' or ``exec''
760 program provided with your particular implementation of MPI.
761 Typical MPI packages such as MPICH will use something like:
762 \begin{verbatim}
763 % mpirun -np 4 -machinefile mf ./mitgcmuv
764 \end{verbatim}
765 Sightly more complicated scripts may be needed for many machines
766 since execution of the code may be controlled by both the MPI
767 library and a job scheduling and queueing system such as PBS,
768 LoadLeveller, Condor, or any of a number of similar tools. A few
769 example scripts (those used for our \begin{rawhtml} <A
770 href="http://mitgcm.org/public/testing.html"> \end{rawhtml}regular
771 verification runs\begin{rawhtml} </A> \end{rawhtml}) are available
772 at:
773 \begin{rawhtml} <A
774 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
775 \end{rawhtml}
776 {\footnotesize \tt
777 http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example\_scripts/ }
778 \begin{rawhtml} </A> \end{rawhtml}
779 or at:
780 \begin{rawhtml} <A
781 href="http://mitgcm.org/viewvc/MITgcm/MITgcm_contrib/test_scripts/">
782 \end{rawhtml}
783 {\footnotesize \tt
784 http://mitgcm.org/viewvc/MITgcm/MITgcm\_contrib/test\_scripts/ }
785 \begin{rawhtml} </A> \end{rawhtml}
786
787 \end{enumerate}
788
789 An example of the above process on the MITgcm cluster (``cg01'') using
790 the GNU g77 compiler and the mpich MPI library is:
791
792 {\footnotesize \begin{verbatim}
793 % cd MITgcm/verification/exp5
794 % mkdir build
795 % cd build
796 % ../../../tools/genmake2 -mpi -mods=../code \
797 -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
798 % make depend
799 % make
800 % cd ../input
801 % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
802 -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
803 \end{verbatim} }
804
805 \section[Running MITgcm]{Running the model in prognostic mode}
806 \label{sect:runModel}
807 \begin{rawhtml}
808 <!-- CMIREDIR:runModel: -->
809 \end{rawhtml}
810
811 If compilation finished succesfully (section \ref{sect:buildingCode})
812 then an executable called \texttt{mitgcmuv} will now exist in the
813 local directory.
814
815 To run the model as a single process (\textit{ie.} not in parallel)
816 simply type:
817 \begin{verbatim}
818 % ./mitgcmuv
819 \end{verbatim}
820 The ``./'' is a safe-guard to make sure you use the local executable
821 in case you have others that exist in your path (surely odd if you
822 do!). The above command will spew out many lines of text output to
823 your screen. This output contains details such as parameter values as
824 well as diagnostics such as mean Kinetic energy, largest CFL number,
825 etc. It is worth keeping this text output with the binary output so we
826 normally re-direct the \texttt{stdout} stream as follows:
827 \begin{verbatim}
828 % ./mitgcmuv > output.txt
829 \end{verbatim}
830 In the event that the model encounters an error and stops, it is very
831 helpful to include the last few line of this \texttt{output.txt} file
832 along with the (\texttt{stderr}) error message within any bug reports.
833
834 For the example experiments in \texttt{verification}, an example of the
835 output is kept in \texttt{results/output.txt} for comparison. You can
836 compare your \texttt{output.txt} with the corresponding one for that
837 experiment to check that the set-up works.
838
839
840
841 \subsection{Output files}
842
843 The model produces various output files and, when using \texttt{mnc},
844 sometimes even directories. Depending upon the I/O package(s)
845 selected at compile time (either \texttt{mdsio} or \texttt{mnc} or
846 both as determined by \texttt{code/packages.conf}) and the run-time
847 flags set (in \texttt{input/data.pkg}), the following output may
848 appear.
849
850
851 \subsubsection{MDSIO output files}
852
853 The ``traditional'' output files are generated by the \texttt{mdsio}
854 package. At a minimum, the instantaneous ``state'' of the model is
855 written out, which is made of the following files:
856
857 \begin{itemize}
858 \item \texttt{U.00000nIter} - zonal component of velocity field (m/s
859 and positive eastward).
860
861 \item \texttt{V.00000nIter} - meridional component of velocity field
862 (m/s and positive northward).
863
864 \item \texttt{W.00000nIter} - vertical component of velocity field
865 (ocean: m/s and positive upward, atmosphere: Pa/s and positive
866 towards increasing pressure i.e. downward).
867
868 \item \texttt{T.00000nIter} - potential temperature (ocean:
869 $^{\circ}\mathrm{C}$, atmosphere: $^{\circ}\mathrm{K}$).
870
871 \item \texttt{S.00000nIter} - ocean: salinity (psu), atmosphere: water
872 vapor (g/kg).
873
874 \item \texttt{Eta.00000nIter} - ocean: surface elevation (m),
875 atmosphere: surface pressure anomaly (Pa).
876 \end{itemize}
877
878 The chain \texttt{00000nIter} consists of ten figures that specify the
879 iteration number at which the output is written out. For example,
880 \texttt{U.0000000300} is the zonal velocity at iteration 300.
881
882 In addition, a ``pickup'' or ``checkpoint'' file called:
883
884 \begin{itemize}
885 \item \texttt{pickup.00000nIter}
886 \end{itemize}
887
888 is written out. This file represents the state of the model in a condensed
889 form and is used for restarting the integration. If the C-D scheme is used,
890 there is an additional ``pickup'' file:
891
892 \begin{itemize}
893 \item \texttt{pickup\_cd.00000nIter}
894 \end{itemize}
895
896 containing the D-grid velocity data and that has to be written out as well
897 in order to restart the integration. Rolling checkpoint files are the same
898 as the pickup files but are named differently. Their name contain the chain
899 \texttt{ckptA} or \texttt{ckptB} instead of \texttt{00000nIter}. They can be
900 used to restart the model but are overwritten every other time they are
901 output to save disk space during long integrations.
902
903 \subsubsection{MNC output files}
904
905 Unlike the \texttt{mdsio} output, the \texttt{mnc}--generated output
906 is usually (though not necessarily) placed within a subdirectory with
907 a name such as \texttt{mnc\_test\_\${DATE}\_\${SEQ}}.
908
909 \subsection{Looking at the output}
910
911 The ``traditional'' or mdsio model data are written according to a
912 ``meta/data'' file format. Each variable is associated with two files
913 with suffix names \texttt{.data} and \texttt{.meta}. The
914 \texttt{.data} file contains the data written in binary form
915 (big\_endian by default). The \texttt{.meta} file is a ``header'' file
916 that contains information about the size and the structure of the
917 \texttt{.data} file. This way of organizing the output is particularly
918 useful when running multi-processors calculations. The base version of
919 the model includes a few matlab utilities to read output files written
920 in this format. The matlab scripts are located in the directory
921 \texttt{utils/matlab} under the root tree. The script \texttt{rdmds.m}
922 reads the data. Look at the comments inside the script to see how to
923 use it.
924
925 Some examples of reading and visualizing some output in {\em Matlab}:
926 \begin{verbatim}
927 % matlab
928 >> H=rdmds('Depth');
929 >> contourf(H');colorbar;
930 >> title('Depth of fluid as used by model');
931
932 >> eta=rdmds('Eta',10);
933 >> imagesc(eta');axis ij;colorbar;
934 >> title('Surface height at iter=10');
935
936 >> eta=rdmds('Eta',[0:10:100]);
937 >> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
938 \end{verbatim}
939
940 Similar scripts for netCDF output (\texttt{rdmnc.m}) are available and
941 they are described in Section \ref{sec:pkg:mnc}.
942
943 The MNC output files are all in the ``self-describing'' netCDF
944 format and can thus be browsed and/or plotted using tools such as:
945 \begin{itemize}
946 \item \texttt{ncdump} is a utility which is typically included
947 with every netCDF install:
948 \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
949 \begin{verbatim}
950 http://www.unidata.ucar.edu/packages/netcdf/
951 \end{verbatim}
952 \begin{rawhtml} </A> \end{rawhtml} and it converts the netCDF
953 binaries into formatted ASCII text files.
954
955 \item \texttt{ncview} utility is a very convenient and quick way
956 to plot netCDF data and it runs on most OSes:
957 \begin{rawhtml} <A href="http://meteora.ucsd.edu/~pierce/ncview_home_page.html"> \end{rawhtml}
958 \begin{verbatim}
959 http://meteora.ucsd.edu/~pierce/ncview_home_page.html
960 \end{verbatim}
961 \begin{rawhtml} </A> \end{rawhtml}
962
963 \item MatLAB(c) and other common post-processing environments provide
964 various netCDF interfaces including:
965 \begin{rawhtml} <A href="http://mexcdf.sourceforge.net/"> \end{rawhtml}
966 \begin{verbatim}
967 http://mexcdf.sourceforge.net/
968 \end{verbatim}
969 \begin{rawhtml} </A> \end{rawhtml}
970 \begin{rawhtml} <A href="http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html"> \end{rawhtml}
971 \begin{verbatim}
972 http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html
973 \end{verbatim}
974 \begin{rawhtml} </A> \end{rawhtml}
975 \end{itemize}
976

  ViewVC Help
Powered by ViewVC 1.1.22