/[MITgcm]/manual/s_getstarted/text/getting_started.tex
ViewVC logotype

Contents of /manual/s_getstarted/text/getting_started.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.42 - (show annotations) (download) (as text)
Fri May 28 02:09:59 2010 UTC (15 years, 1 month ago) by jmc
Branch: MAIN
Changes since 1.41: +45 -36 lines
File MIME type: application/x-tex
update genmake2 documentation

1 % $Header: /u/gcmpack/manual/part3/getting_started.tex,v 1.41 2010/01/22 00:38:16 jmc Exp $
2 % $Name: $
3
4 %\section{Getting started}
5
6 We believe the best way to familiarize yourself with the
7 model is to run the case study examples provided with the base
8 version. Information on how to obtain, compile, and run the code is
9 found here as well as a brief description of the model structure
10 directory and the case study examples. Information is also provided
11 here on how to customize the code when you are ready to try implementing
12 the configuration you have in mind. The code and algorithm
13 are described more fully in chapters \ref{chap:discretization} and
14 \ref{chap:sarch}.
15
16 \section{Where to find information}
17 \label{sect:whereToFindInfo}
18 \begin{rawhtml}
19 <!-- CMIREDIR:whereToFindInfo: -->
20 \end{rawhtml}
21
22 There is a web-archived support mailing list for the model that
23 you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
24 \begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
25 \begin{verbatim}
26 http://mitgcm.org/mailman/listinfo/mitgcm-support/
27 http://mitgcm.org/pipermail/mitgcm-support/
28 \end{verbatim}
29 \begin{rawhtml} </A> \end{rawhtml}
30
31 \section{Obtaining the code}
32 \label{sect:obtainingCode}
33 \begin{rawhtml}
34 <!-- CMIREDIR:obtainingCode: -->
35 \end{rawhtml}
36
37 MITgcm can be downloaded from our system by following
38 the instructions below. As a courtesy we ask that you send e-mail to us at
39 \begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
40 MITgcm-support@mitgcm.org
41 \begin{rawhtml} </A> \end{rawhtml}
42 to enable us to keep track of who's using the model and in what application.
43 You can download the model two ways:
44
45 \begin{enumerate}
46 \item Using CVS software. CVS is a freely available source code management
47 tool. To use CVS you need to have the software installed. Many systems
48 come with CVS pre-installed, otherwise good places to look for
49 the software for a particular platform are
50 \begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
51 cvshome.org
52 \begin{rawhtml} </A> \end{rawhtml}
53 and
54 \begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
55 wincvs.org
56 \begin{rawhtml} </A> \end{rawhtml}
57 .
58
59 \item Using a tar file. This method is simple and does not
60 require any special software. However, this method does not
61 provide easy support for maintenance updates.
62
63 \end{enumerate}
64
65 \subsection{Method 1 - Checkout from CVS}
66 \label{sect:cvs_checkout}
67
68 If CVS is available on your system, we strongly encourage you to use it. CVS
69 provides an efficient and elegant way of organizing your code and keeping
70 track of your changes. If CVS is not available on your machine, you can also
71 download a tar file.
72
73 Before you can use CVS, the following environment variable(s) should
74 be set within your shell. For a csh or tcsh shell, put the following
75 \begin{verbatim}
76 % setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
77 \end{verbatim}
78 in your \texttt{.cshrc} or \texttt{.tcshrc} file. For bash or sh
79 shells, put:
80 \begin{verbatim}
81 % export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
82 \end{verbatim}
83 in your \texttt{.profile} or \texttt{.bashrc} file.
84
85
86 To get MITgcm through CVS, first register with the MITgcm CVS server
87 using command:
88 \begin{verbatim}
89 % cvs login ( CVS password: cvsanon )
90 \end{verbatim}
91 You only need to do a ``cvs login'' once.
92
93 To obtain the latest sources type:
94 \begin{verbatim}
95 % cvs co MITgcm
96 \end{verbatim}
97 or to get a specific release type:
98 \begin{verbatim}
99 % cvs co -P -r checkpoint52i_post MITgcm
100 \end{verbatim}
101 The MITgcm web site contains further directions concerning the source
102 code and CVS. It also contains a web interface to our CVS archive so
103 that one may easily view the state of files, revisions, and other
104 development milestones:
105 %\begin{rawhtml} <A href="http://mitgcm.org/download" target="idontexist"> \end{rawhtml}
106 \begin{rawhtml} <A href="http://mitgcm.org/viewvc/MITgcm/MITgcm/" target="idontexist"> \end{rawhtml}
107 \begin{verbatim}
108 http://mitgcm.org/source_code.html
109 \end{verbatim}
110 \begin{rawhtml} </A> \end{rawhtml}
111
112 As a convenience, the MITgcm CVS server contains aliases which are
113 named subsets of the codebase. These aliases can be especially
114 helpful when used over slow internet connections or on machines with
115 restricted storage space. Table \ref{tab:cvsModules} contains a list
116 of CVS aliases
117 \begin{table}[htb]
118 \centering
119 \begin{tabular}[htb]{|lp{3.25in}|}\hline
120 \textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
121 \texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
122 \texttt{MITgcm\_verif\_basic}
123 & Source code plus a small set of the verification examples
124 (\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
125 \texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
126 \texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
127 \texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
128 \texttt{MITgcm\_verif\_all} & Source code plus all of the
129 verification examples. \\\hline
130 \end{tabular}
131 \caption{MITgcm CVS Modules}
132 \label{tab:cvsModules}
133 \end{table}
134
135 The checkout process creates a directory called \texttt{MITgcm}. If
136 the directory \texttt{MITgcm} exists this command updates your code
137 based on the repository. Each directory in the source tree contains a
138 directory \texttt{CVS}. This information is required by CVS to keep
139 track of your file versions with respect to the repository. Don't edit
140 the files in \texttt{CVS}! You can also use CVS to download code
141 updates. More extensive information on using CVS for maintaining
142 MITgcm code can be found
143 \begin{rawhtml} <A href="http://mitgcm.org/public/using_cvs.html" target="idontexist"> \end{rawhtml}
144 here
145 \begin{rawhtml} </A> \end{rawhtml}
146 .
147 It is important to note that the CVS aliases in Table
148 \ref{tab:cvsModules} cannot be used in conjunction with the CVS
149 \texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
150 they create can be changed to a different name following the check-out:
151 \begin{verbatim}
152 % cvs co MITgcm_verif_basic
153 % mv MITgcm MITgcm_verif_basic
154 \end{verbatim}
155
156 \subsubsection{Upgrading from an earlier version}
157
158 If you already have an earlier version of the code you can ``upgrade''
159 your copy instead of downloading the entire repository again. First,
160 ``cd'' (change directory) to the top of your working copy:
161 \begin{verbatim}
162 % cd MITgcm
163 \end{verbatim}
164 and then issue the cvs update command such as:
165 \begin{verbatim}
166 % cvs -q update -r checkpoint52i_post -d -P
167 \end{verbatim}
168 This will update the ``tag'' to ``checkpoint52i\_post'', add any new
169 directories (-d) and remove any empty directories (-P). The -q option
170 means be quiet which will reduce the number of messages you'll see in
171 the terminal. If you have modified the code prior to upgrading, CVS
172 will try to merge your changes with the upgrades. If there is a
173 conflict between your modifications and the upgrade, it will report
174 that file with a ``C'' in front, e.g.:
175 \begin{verbatim}
176 C model/src/ini_parms.F
177 \end{verbatim}
178 If the list of conflicts scrolled off the screen, you can re-issue the
179 cvs update command and it will report the conflicts. Conflicts are
180 indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
181 ``$>>>>>>>$''. For example,
182 {\small
183 \begin{verbatim}
184 <<<<<<< ini_parms.F
185 & bottomDragLinear,myOwnBottomDragCoefficient,
186 =======
187 & bottomDragLinear,bottomDragQuadratic,
188 >>>>>>> 1.18
189 \end{verbatim}
190 }
191 means that you added ``myOwnBottomDragCoefficient'' to a namelist at
192 the same time and place that we added ``bottomDragQuadratic''. You
193 need to resolve this conflict and in this case the line should be
194 changed to:
195 {\small
196 \begin{verbatim}
197 & bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
198 \end{verbatim}
199 }
200 and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
201 Unless you are making modifications which exactly parallel
202 developments we make, these types of conflicts should be rare.
203
204 \paragraph*{Upgrading to the current pre-release version}
205
206 We don't make a ``release'' for every little patch and bug fix in
207 order to keep the frequency of upgrades to a minimum. However, if you
208 have run into a problem for which ``we have already fixed in the
209 latest code'' and we haven't made a ``tag'' or ``release'' since that
210 patch then you'll need to get the latest code:
211 \begin{verbatim}
212 % cvs -q update -A -d -P
213 \end{verbatim}
214 Unlike, the ``check-out'' and ``update'' procedures above, there is no
215 ``tag'' or release name. The -A tells CVS to upgrade to the
216 very latest version. As a rule, we don't recommend this since you
217 might upgrade while we are in the processes of checking in the code so
218 that you may only have part of a patch. Using this method of updating
219 also means we can't tell what version of the code you are working
220 with. So please be sure you understand what you're doing.
221
222 \subsection{Method 2 - Tar file download}
223 \label{sect:conventionalDownload}
224
225 If you do not have CVS on your system, you can download the model as a
226 tar file from the web site at:
227 \begin{rawhtml} <A href=http://mitgcm.org/download/ target="idontexist"> \end{rawhtml}
228 \begin{verbatim}
229 http://mitgcm.org/download/
230 \end{verbatim}
231 \begin{rawhtml} </A> \end{rawhtml}
232 The tar file still contains CVS information which we urge you not to
233 delete; even if you do not use CVS yourself the information can help
234 us if you should need to send us your copy of the code. If a recent
235 tar file does not exist, then please contact the developers through
236 the
237 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
238 MITgcm-support@mitgcm.org
239 \begin{rawhtml} </A> \end{rawhtml}
240 mailing list.
241
242 \section{Model and directory structure}
243 \begin{rawhtml}
244 <!-- CMIREDIR:directory_structure: -->
245 \end{rawhtml}
246
247 The ``numerical'' model is contained within a execution environment
248 support wrapper. This wrapper is designed to provide a general
249 framework for grid-point models. MITgcmUV is a specific numerical
250 model that uses the framework. Under this structure the model is split
251 into execution environment support code and conventional numerical
252 model code. The execution environment support code is held under the
253 \texttt{eesupp} directory. The grid point model code is held under the
254 \texttt{model} directory. Code execution actually starts in the
255 \texttt{eesupp} routines and not in the \texttt{model} routines. For
256 this reason the top-level \texttt{MAIN.F} is in the
257 \texttt{eesupp/src} directory. In general, end-users should not need
258 to worry about this level. The top-level routine for the numerical
259 part of the code is in \texttt{model/src/THE\_MODEL\_MAIN.F}. Here is
260 a brief description of the directory structure of the model under the
261 root tree (a detailed description is given in section 3: Code
262 structure).
263
264 \begin{itemize}
265
266 \item \texttt{doc}: contains brief documentation notes.
267
268 \item \texttt{eesupp}: contains the execution environment source code.
269 Also subdivided into two subdirectories \texttt{inc} and
270 \texttt{src}.
271
272 \item \texttt{model}: this directory contains the main source code.
273 Also subdivided into two subdirectories \texttt{inc} and
274 \texttt{src}.
275
276 \item \texttt{pkg}: contains the source code for the packages. Each
277 package corresponds to a subdirectory. For example, \texttt{gmredi}
278 contains the code related to the Gent-McWilliams/Redi scheme,
279 \texttt{aim} the code relative to the atmospheric intermediate
280 physics. The packages are described in detail in chapter \ref{chap.packagesI}.
281
282 \item \texttt{tools}: this directory contains various useful tools.
283 For example, \texttt{genmake2} is a script written in csh (C-shell)
284 that should be used to generate your makefile. The directory
285 \texttt{adjoint} contains the makefile specific to the Tangent
286 linear and Adjoint Compiler (TAMC) that generates the adjoint code.
287 The latter is described in detail in part \ref{chap.ecco}.
288 This directory also contains the subdirectory build\_options, which
289 contains the `optfiles' with the compiler options for the different
290 compilers and machines that can run MITgcm.
291
292 \item \texttt{utils}: this directory contains various utilities. The
293 subdirectory \texttt{knudsen2} contains code and a makefile that
294 compute coefficients of the polynomial approximation to the knudsen
295 formula for an ocean nonlinear equation of state. The
296 \texttt{matlab} subdirectory contains matlab scripts for reading
297 model output directly into matlab. \texttt{scripts} contains C-shell
298 post-processing scripts for joining processor-based and tiled-based
299 model output. The subdirectory exch2 contains the code needed for
300 the exch2 package to work with different combinations of domain
301 decompositions.
302
303 \item \texttt{verification}: this directory contains the model
304 examples. See section \ref{sect:modelExamples}.
305
306 \item \texttt{jobs}: contains sample job scripts for running MITgcm.
307
308 \item \texttt{lsopt}: Line search code used for optimization.
309
310 \item \texttt{optim}: Interface between MITgcm and line search code.
311
312 \end{itemize}
313
314 \section[Building MITgcm]{Building the code}
315 \label{sect:buildingCode}
316 \begin{rawhtml}
317 <!-- CMIREDIR:buildingCode: -->
318 \end{rawhtml}
319
320 To compile the code, we use the \texttt{make} program. This uses a
321 file (\texttt{Makefile}) that allows us to pre-process source files,
322 specify compiler and optimization options and also figures out any
323 file dependencies. We supply a script (\texttt{genmake2}), described
324 in section \ref{sect:genmake}, that automatically creates the
325 \texttt{Makefile} for you. You then need to build the dependencies and
326 compile the code.
327
328 As an example, assume that you want to build and run experiment
329 \texttt{verification/exp2}. The are multiple ways and places to
330 actually do this but here let's build the code in
331 \texttt{verification/exp2/build}:
332 \begin{verbatim}
333 % cd verification/exp2/build
334 \end{verbatim}
335 First, build the \texttt{Makefile}:
336 \begin{verbatim}
337 % ../../../tools/genmake2 -mods=../code
338 \end{verbatim}
339 The command line option tells \texttt{genmake} to override model source
340 code with any files in the directory \texttt{../code/}.
341
342 On many systems, the \texttt{genmake2} program will be able to
343 automatically recognize the hardware, find compilers and other tools
344 within the user's path (``\texttt{echo \$PATH}''), and then choose an
345 appropriate set of options from the files (``optfiles'') contained in
346 the \texttt{tools/build\_options} directory. Under some
347 circumstances, a user may have to create a new ``optfile'' in order to
348 specify the exact combination of compiler, compiler flags, libraries,
349 and other options necessary to build a particular configuration of
350 MITgcm. In such cases, it is generally helpful to read the existing
351 ``optfiles'' and mimic their syntax.
352
353 Through the MITgcm-support list, the MITgcm developers are willing to
354 provide help writing or modifing ``optfiles''. And we encourage users
355 to post new ``optfiles'' (particularly ones for new machines or
356 architectures) to the
357 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
358 MITgcm-support@mitgcm.org
359 \begin{rawhtml} </A> \end{rawhtml}
360 list.
361
362 To specify an optfile to \texttt{genmake2}, the syntax is:
363 \begin{verbatim}
364 % ../../../tools/genmake2 -mods=../code -of /path/to/optfile
365 \end{verbatim}
366
367 Once a \texttt{Makefile} has been generated, we create the
368 dependencies with the command:
369 \begin{verbatim}
370 % make depend
371 \end{verbatim}
372 This modifies the \texttt{Makefile} by attaching a (usually, long)
373 list of files upon which other files depend. The purpose of this is to
374 reduce re-compilation if and when you start to modify the code. The
375 {\tt make depend} command also creates links from the model source to
376 this directory. It is important to note that the {\tt make depend}
377 stage will occasionally produce warnings or errors since the
378 dependency parsing tool is unable to find all of the necessary header
379 files (\textit{eg.} \texttt{netcdf.inc}). In these circumstances, it
380 is usually OK to ignore the warnings/errors and proceed to the next
381 step.
382
383 Next one can compile the code using:
384 \begin{verbatim}
385 % make
386 \end{verbatim}
387 The {\tt make} command creates an executable called \texttt{mitgcmuv}.
388 Additional make ``targets'' are defined within the makefile to aid in
389 the production of adjoint and other versions of MITgcm. On SMP
390 (shared multi-processor) systems, the build process can often be sped
391 up appreciably using the command:
392 \begin{verbatim}
393 % make -j 2
394 \end{verbatim}
395 where the ``2'' can be replaced with a number that corresponds to the
396 number of CPUs available.
397
398 Now you are ready to run the model. General instructions for doing so are
399 given in section \ref{sect:runModel}. Here, we can run the model by
400 first creating links to all the input files:
401 \begin{verbatim}
402 ln -s ../input/* .
403 \end{verbatim}
404 and then calling the executable with:
405 \begin{verbatim}
406 ./mitgcmuv > output.txt
407 \end{verbatim}
408 where we are re-directing the stream of text output to the file
409 \texttt{output.txt}.
410
411 \subsection{Building/compiling the code elsewhere}
412
413 In the example above (section \ref{sect:buildingCode}) we built the
414 executable in the {\em input} directory of the experiment for
415 convenience. You can also configure and compile the code in other
416 locations, for example on a scratch disk with out having to copy the
417 entire source tree. The only requirement to do so is you have {\tt
418 genmake2} in your path or you know the absolute path to {\tt
419 genmake2}.
420
421 The following sections outline some possible methods of organizing
422 your source and data.
423
424 \subsubsection{Building from the {\em ../code directory}}
425
426 This is just as simple as building in the {\em input/} directory:
427 \begin{verbatim}
428 % cd verification/exp2/code
429 % ../../../tools/genmake2
430 % make depend
431 % make
432 \end{verbatim}
433 However, to run the model the executable ({\em mitgcmuv}) and input
434 files must be in the same place. If you only have one calculation to make:
435 \begin{verbatim}
436 % cd ../input
437 % cp ../code/mitgcmuv ./
438 % ./mitgcmuv > output.txt
439 \end{verbatim}
440 or if you will be making multiple runs with the same executable:
441 \begin{verbatim}
442 % cd ../
443 % cp -r input run1
444 % cp code/mitgcmuv run1
445 % cd run1
446 % ./mitgcmuv > output.txt
447 \end{verbatim}
448
449 \subsubsection{Building from a new directory}
450
451 Since the {\em input} directory contains input files it is often more
452 useful to keep {\em input} pristine and build in a new directory
453 within {\em verification/exp2/}:
454 \begin{verbatim}
455 % cd verification/exp2
456 % mkdir build
457 % cd build
458 % ../../../tools/genmake2 -mods=../code
459 % make depend
460 % make
461 \end{verbatim}
462 This builds the code exactly as before but this time you need to copy
463 either the executable or the input files or both in order to run the
464 model. For example,
465 \begin{verbatim}
466 % cp ../input/* ./
467 % ./mitgcmuv > output.txt
468 \end{verbatim}
469 or if you tend to make multiple runs with the same executable then
470 running in a new directory each time might be more appropriate:
471 \begin{verbatim}
472 % cd ../
473 % mkdir run1
474 % cp build/mitgcmuv run1/
475 % cp input/* run1/
476 % cd run1
477 % ./mitgcmuv > output.txt
478 \end{verbatim}
479
480 \subsubsection{Building on a scratch disk}
481
482 Model object files and output data can use up large amounts of disk
483 space so it is often the case that you will be operating on a large
484 scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
485 following commands will build the model in {\em /scratch/exp2-run1}:
486 \begin{verbatim}
487 % cd /scratch/exp2-run1
488 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
489 -mods=~/MITgcm/verification/exp2/code
490 % make depend
491 % make
492 \end{verbatim}
493 To run the model here, you'll need the input files:
494 \begin{verbatim}
495 % cp ~/MITgcm/verification/exp2/input/* ./
496 % ./mitgcmuv > output.txt
497 \end{verbatim}
498
499 As before, you could build in one directory and make multiple runs of
500 the one experiment:
501 \begin{verbatim}
502 % cd /scratch/exp2
503 % mkdir build
504 % cd build
505 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
506 -mods=~/MITgcm/verification/exp2/code
507 % make depend
508 % make
509 % cd ../
510 % cp -r ~/MITgcm/verification/exp2/input run2
511 % cd run2
512 % ./mitgcmuv > output.txt
513 \end{verbatim}
514
515
516 \subsection{Using \texttt{genmake2}}
517 \label{sect:genmake}
518
519 To compile the code, first use the program \texttt{genmake2} (located
520 in the \texttt{tools} directory) to generate a Makefile.
521 \texttt{genmake2} is a shell script written to work with all
522 ``sh''--compatible shells including bash v1, bash v2, and Bourne.
523 %Internally, \texttt{genmake2} determines the locations of needed
524 %files, the compiler, compiler options, libraries, and Unix tools. It
525 %relies upon a number of ``optfiles'' located in the
526 %\texttt{tools/build\_options} directory.
527 \texttt{genmake2} parses information from the following sources:
528 \begin{description}
529 \item[-] a {\em gemake\_local} file if one is found in the current
530 directory
531 \item[-] command-line options
532 \item[-] an "options file" as specified by the command-line option
533 \texttt{--optfile=/PATH/FILENAME}
534 \item[-] a {\em packages.conf} file (if one is found) with the
535 specific list of packages to compile. The search path for
536 file {\em packages.conf} is, first, the current directory and
537 then each of the "MODS" directories in the given order (see below).
538 \end{description}
539
540 \subsubsection{Optfiles in \texttt{tools/build\_options} directory:}
541
542 The purpose of the optfiles is to provide all the compilation options
543 for particular ``platforms'' (where ``platform'' roughly means the
544 combination of the hardware and the compiler) and code configurations.
545 Given the combinations of possible compilers and library dependencies
546 ({\it eg.} MPI and NetCDF) there may be numerous optfiles available
547 for a single machine. The naming scheme for the majority of the
548 optfiles shipped with the code is
549 \begin{center}
550 {\bf OS\_HARDWARE\_COMPILER }
551 \end{center}
552 where
553 \begin{description}
554 \item[OS] is the name of the operating system (generally the
555 lower-case output of the {\tt 'uname'} command)
556 \item[HARDWARE] is a string that describes the CPU type and
557 corresponds to output from the {\tt 'uname -m'} command:
558 \begin{description}
559 \item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
560 and athlon
561 \item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
562 \item[amd64] is AMD x86\_64 systems
563 \item[ppc] is for Mac PowerPC systems
564 \end{description}
565 \item[COMPILER] is the compiler name (generally, the name of the
566 FORTRAN executable)
567 \end{description}
568
569 In many cases, the default optfiles are sufficient and will result in
570 usable Makefiles. However, for some machines or code configurations,
571 new ``optfiles'' must be written. To create a new optfile, it is
572 generally best to start with one of the defaults and modify it to suit
573 your needs. Like \texttt{genmake2}, the optfiles are all written
574 using a simple ``sh''--compatible syntax. While nearly all variables
575 used within \texttt{genmake2} may be specified in the optfiles, the
576 critical ones that should be defined are:
577
578 \begin{description}
579 \item[FC] the FORTRAN compiler (executable) to use
580 \item[DEFINES] the command-line DEFINE options passed to the compiler
581 \item[CPP] the C pre-processor to use
582 \item[NOOPTFLAGS] options flags for special files that should not be
583 optimized
584 \end{description}
585
586 For example, the optfile for a typical Red Hat Linux machine (``ia32''
587 architecture) using the GCC (g77) compiler is
588 \begin{verbatim}
589 FC=g77
590 DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
591 CPP='cpp -traditional -P'
592 NOOPTFLAGS='-O0'
593 # For IEEE, use the "-ffloat-store" option
594 if test "x$IEEE" = x ; then
595 FFLAGS='-Wimplicit -Wunused -Wuninitialized'
596 FOPTIM='-O3 -malign-double -funroll-loops'
597 else
598 FFLAGS='-Wimplicit -Wunused -ffloat-store'
599 FOPTIM='-O0 -malign-double'
600 fi
601 \end{verbatim}
602
603 If you write an optfile for an unrepresented machine or compiler, you
604 are strongly encouraged to submit the optfile to the MITgcm project
605 for inclusion. Please send the file to the
606 \begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
607 \begin{center}
608 MITgcm-support@mitgcm.org
609 \end{center}
610 \begin{rawhtml} </A> \end{rawhtml}
611 mailing list.
612
613 \subsubsection{Command-line options:}
614
615 In addition to the optfiles, \texttt{genmake2} supports a number of
616 helpful command-line options. A complete list of these options can be
617 obtained from:
618 \begin{verbatim}
619 % genmake2 -h
620 \end{verbatim}
621
622 The most important command-line options are:
623 \begin{description}
624
625 \item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
626 should be used for a particular build.
627
628 If no "optfile" is specified (either through the command line or the
629 MITGCM\_OPTFILE environment variable), genmake2 will try to make a
630 reasonable guess from the list provided in {\em
631 tools/build\_options}. The method used for making this guess is
632 to first determine the combination of operating system and hardware
633 (eg. "linux\_ia32") and then find a working FORTRAN compiler within
634 the user's path. When these three items have been identified,
635 genmake2 will try to find an optfile that has a matching name.
636
637 \item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
638 directories containing ``modifications''. These directories contain
639 files with names that may (or may not) exist in the main MITgcm
640 source tree but will be overridden by any identically-named sources
641 within the ``MODS'' directories.
642
643 The order of precedence for this "name-hiding" is as follows:
644 \begin{itemize}
645 \item ``MODS'' directories (in the order given)
646 \item Packages either explicitly specified or provided by default
647 (in the order given)
648 \item Packages included due to package dependencies (in the order
649 that that package dependencies are parsed)
650 \item The "standard dirs" (which may have been specified by the
651 ``-standarddirs'' option)
652 \end{itemize}
653
654 \item[\texttt{--pgroups=/PATH/FILENAME}] specifies the file
655 where package groups are defined. If not set, the package-groups
656 definition will be read from {\em pkg/pkg\_groups}.
657 It also contains the default list of packages (defined
658 as the group ``{\it default\_pkg\_list}'' which is used
659 when no specific package list ({\em packages.conf})
660 is found in current directory or in any "MODS" directory.
661
662 \item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
663 used for packages.
664
665 If not specified, the default dependency file {\em pkg/pkg\_depend}
666 is used. The syntax for this file is parsed on a line-by-line basis
667 where each line containes either a comment ("\#") or a simple
668 "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
669 specifies a "must be used with" or a "must not be used with"
670 relationship, respectively. If no rule is specified, then it is
671 assumed that the two packages are compatible and will function
672 either with or without each other.
673
674 \item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
675 automatic differentiation options file to be used. The file is
676 analogous to the ``optfile'' defined above but it specifies
677 information for the AD build process.
678
679 The default file is located in {\em
680 tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
681 and "TAMC" compilers. An alternate version is also available at
682 {\em tools/adjoint\_options/adjoint\_staf} that selects the newer
683 "STAF" compiler. As with any compilers, it is helpful to have their
684 directories listed in your {\tt \$PATH} environment variable.
685
686 \item[\texttt{--mpi}] This option enables certain MPI features (using
687 CPP \texttt{\#define}s) within the code and is necessary for MPI
688 builds (see Section \ref{sect:mpi-build}).
689
690 \item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
691 soft-links and other bugs common with the \texttt{make} versions
692 provided by commercial Unix vendors, GNU \texttt{make} (sometimes
693 called \texttt{gmake}) should be preferred. This option provides a
694 means for specifying the make executable to be used.
695
696 \item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
697 machines, the ``bash'' shell is unavailable. To run on these
698 systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
699 a Bourne, POSIX, or compatible) shell. The syntax in these
700 circumstances is:
701 \begin{center}
702 \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
703 \end{center}
704 where \texttt{/bin/sh} can be replaced with the full path and name
705 of the desired shell.
706
707 \end{description}
708
709
710 \subsection{Building with MPI}
711 \label{sect:mpi-build}
712
713 Building MITgcm to use MPI libraries can be complicated due to the
714 variety of different MPI implementations available, their dependencies
715 or interactions with different compilers, and their often ad-hoc
716 locations within file systems. For these reasons, its generally a
717 good idea to start by finding and reading the documentation for your
718 machine(s) and, if necessary, seeking help from your local systems
719 administrator.
720
721 The steps for building MITgcm with MPI support are:
722 \begin{enumerate}
723
724 \item Determine the locations of your MPI-enabled compiler and/or MPI
725 libraries and put them into an options file as described in Section
726 \ref{sect:genmake}. One can start with one of the examples in:
727 \begin{rawhtml} <A
728 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/build_options/">
729 \end{rawhtml}
730 \begin{center}
731 \texttt{MITgcm/tools/build\_options/}
732 \end{center}
733 \begin{rawhtml} </A> \end{rawhtml}
734 such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
735 \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
736 hand. You may need help from your user guide or local systems
737 administrator to determine the exact location of the MPI libraries.
738 If libraries are not installed, MPI implementations and related
739 tools are available including:
740 \begin{itemize}
741 \item \begin{rawhtml} <A
742 href="http://www-unix.mcs.anl.gov/mpi/mpich/">
743 \end{rawhtml}
744 MPICH
745 \begin{rawhtml} </A> \end{rawhtml}
746
747 \item \begin{rawhtml} <A
748 href="http://www.lam-mpi.org/">
749 \end{rawhtml}
750 LAM/MPI
751 \begin{rawhtml} </A> \end{rawhtml}
752
753 \item \begin{rawhtml} <A
754 href="http://www.osc.edu/~pw/mpiexec/">
755 \end{rawhtml}
756 MPIexec
757 \begin{rawhtml} </A> \end{rawhtml}
758 \end{itemize}
759
760 \item Build the code with the \texttt{genmake2} \texttt{-mpi} option
761 (see Section \ref{sect:genmake}) using commands such as:
762 {\footnotesize \begin{verbatim}
763 % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
764 % make depend
765 % make
766 \end{verbatim} }
767
768 \item Run the code with the appropriate MPI ``run'' or ``exec''
769 program provided with your particular implementation of MPI.
770 Typical MPI packages such as MPICH will use something like:
771 \begin{verbatim}
772 % mpirun -np 4 -machinefile mf ./mitgcmuv
773 \end{verbatim}
774 Sightly more complicated scripts may be needed for many machines
775 since execution of the code may be controlled by both the MPI
776 library and a job scheduling and queueing system such as PBS,
777 LoadLeveller, Condor, or any of a number of similar tools. A few
778 example scripts (those used for our \begin{rawhtml} <A
779 href="http://mitgcm.org/public/testing.html"> \end{rawhtml}regular
780 verification runs\begin{rawhtml} </A> \end{rawhtml}) are available
781 at:
782 \begin{rawhtml} <A
783 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
784 \end{rawhtml}
785 {\footnotesize \tt
786 http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example\_scripts/ }
787 \begin{rawhtml} </A> \end{rawhtml}
788 or at:
789 \begin{rawhtml} <A
790 href="http://mitgcm.org/viewvc/MITgcm/MITgcm_contrib/test_scripts/">
791 \end{rawhtml}
792 {\footnotesize \tt
793 http://mitgcm.org/viewvc/MITgcm/MITgcm\_contrib/test\_scripts/ }
794 \begin{rawhtml} </A> \end{rawhtml}
795
796 \end{enumerate}
797
798 An example of the above process on the MITgcm cluster (``cg01'') using
799 the GNU g77 compiler and the mpich MPI library is:
800
801 {\footnotesize \begin{verbatim}
802 % cd MITgcm/verification/exp5
803 % mkdir build
804 % cd build
805 % ../../../tools/genmake2 -mpi -mods=../code \
806 -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
807 % make depend
808 % make
809 % cd ../input
810 % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
811 -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
812 \end{verbatim} }
813
814 \section[Running MITgcm]{Running the model in prognostic mode}
815 \label{sect:runModel}
816 \begin{rawhtml}
817 <!-- CMIREDIR:runModel: -->
818 \end{rawhtml}
819
820 If compilation finished succesfully (section \ref{sect:buildingCode})
821 then an executable called \texttt{mitgcmuv} will now exist in the
822 local directory.
823
824 To run the model as a single process (\textit{ie.} not in parallel)
825 simply type:
826 \begin{verbatim}
827 % ./mitgcmuv
828 \end{verbatim}
829 The ``./'' is a safe-guard to make sure you use the local executable
830 in case you have others that exist in your path (surely odd if you
831 do!). The above command will spew out many lines of text output to
832 your screen. This output contains details such as parameter values as
833 well as diagnostics such as mean Kinetic energy, largest CFL number,
834 etc. It is worth keeping this text output with the binary output so we
835 normally re-direct the \texttt{stdout} stream as follows:
836 \begin{verbatim}
837 % ./mitgcmuv > output.txt
838 \end{verbatim}
839 In the event that the model encounters an error and stops, it is very
840 helpful to include the last few line of this \texttt{output.txt} file
841 along with the (\texttt{stderr}) error message within any bug reports.
842
843 For the example experiments in \texttt{verification}, an example of the
844 output is kept in \texttt{results/output.txt} for comparison. You can
845 compare your \texttt{output.txt} with the corresponding one for that
846 experiment to check that the set-up works.
847
848
849
850 \subsection{Output files}
851
852 The model produces various output files and, when using \texttt{mnc},
853 sometimes even directories. Depending upon the I/O package(s)
854 selected at compile time (either \texttt{mdsio} or \texttt{mnc} or
855 both as determined by \texttt{code/packages.conf}) and the run-time
856 flags set (in \texttt{input/data.pkg}), the following output may
857 appear.
858
859
860 \subsubsection{MDSIO output files}
861
862 The ``traditional'' output files are generated by the \texttt{mdsio}
863 package. At a minimum, the instantaneous ``state'' of the model is
864 written out, which is made of the following files:
865
866 \begin{itemize}
867 \item \texttt{U.00000nIter} - zonal component of velocity field (m/s
868 and positive eastward).
869
870 \item \texttt{V.00000nIter} - meridional component of velocity field
871 (m/s and positive northward).
872
873 \item \texttt{W.00000nIter} - vertical component of velocity field
874 (ocean: m/s and positive upward, atmosphere: Pa/s and positive
875 towards increasing pressure i.e. downward).
876
877 \item \texttt{T.00000nIter} - potential temperature (ocean:
878 $^{\circ}\mathrm{C}$, atmosphere: $^{\circ}\mathrm{K}$).
879
880 \item \texttt{S.00000nIter} - ocean: salinity (psu), atmosphere: water
881 vapor (g/kg).
882
883 \item \texttt{Eta.00000nIter} - ocean: surface elevation (m),
884 atmosphere: surface pressure anomaly (Pa).
885 \end{itemize}
886
887 The chain \texttt{00000nIter} consists of ten figures that specify the
888 iteration number at which the output is written out. For example,
889 \texttt{U.0000000300} is the zonal velocity at iteration 300.
890
891 In addition, a ``pickup'' or ``checkpoint'' file called:
892
893 \begin{itemize}
894 \item \texttt{pickup.00000nIter}
895 \end{itemize}
896
897 is written out. This file represents the state of the model in a condensed
898 form and is used for restarting the integration. If the C-D scheme is used,
899 there is an additional ``pickup'' file:
900
901 \begin{itemize}
902 \item \texttt{pickup\_cd.00000nIter}
903 \end{itemize}
904
905 containing the D-grid velocity data and that has to be written out as well
906 in order to restart the integration. Rolling checkpoint files are the same
907 as the pickup files but are named differently. Their name contain the chain
908 \texttt{ckptA} or \texttt{ckptB} instead of \texttt{00000nIter}. They can be
909 used to restart the model but are overwritten every other time they are
910 output to save disk space during long integrations.
911
912 \subsubsection{MNC output files}
913
914 Unlike the \texttt{mdsio} output, the \texttt{mnc}--generated output
915 is usually (though not necessarily) placed within a subdirectory with
916 a name such as \texttt{mnc\_test\_\${DATE}\_\${SEQ}}.
917
918 \subsection{Looking at the output}
919
920 The ``traditional'' or mdsio model data are written according to a
921 ``meta/data'' file format. Each variable is associated with two files
922 with suffix names \texttt{.data} and \texttt{.meta}. The
923 \texttt{.data} file contains the data written in binary form
924 (big\_endian by default). The \texttt{.meta} file is a ``header'' file
925 that contains information about the size and the structure of the
926 \texttt{.data} file. This way of organizing the output is particularly
927 useful when running multi-processors calculations. The base version of
928 the model includes a few matlab utilities to read output files written
929 in this format. The matlab scripts are located in the directory
930 \texttt{utils/matlab} under the root tree. The script \texttt{rdmds.m}
931 reads the data. Look at the comments inside the script to see how to
932 use it.
933
934 Some examples of reading and visualizing some output in {\em Matlab}:
935 \begin{verbatim}
936 % matlab
937 >> H=rdmds('Depth');
938 >> contourf(H');colorbar;
939 >> title('Depth of fluid as used by model');
940
941 >> eta=rdmds('Eta',10);
942 >> imagesc(eta');axis ij;colorbar;
943 >> title('Surface height at iter=10');
944
945 >> eta=rdmds('Eta',[0:10:100]);
946 >> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
947 \end{verbatim}
948
949 Similar scripts for netCDF output (\texttt{rdmnc.m}) are available and
950 they are described in Section \ref{sec:pkg:mnc}.
951
952 The MNC output files are all in the ``self-describing'' netCDF
953 format and can thus be browsed and/or plotted using tools such as:
954 \begin{itemize}
955 \item \texttt{ncdump} is a utility which is typically included
956 with every netCDF install:
957 \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
958 \begin{verbatim}
959 http://www.unidata.ucar.edu/packages/netcdf/
960 \end{verbatim}
961 \begin{rawhtml} </A> \end{rawhtml} and it converts the netCDF
962 binaries into formatted ASCII text files.
963
964 \item \texttt{ncview} utility is a very convenient and quick way
965 to plot netCDF data and it runs on most OSes:
966 \begin{rawhtml} <A href="http://meteora.ucsd.edu/~pierce/ncview_home_page.html"> \end{rawhtml}
967 \begin{verbatim}
968 http://meteora.ucsd.edu/~pierce/ncview_home_page.html
969 \end{verbatim}
970 \begin{rawhtml} </A> \end{rawhtml}
971
972 \item MatLAB(c) and other common post-processing environments provide
973 various netCDF interfaces including:
974 \begin{rawhtml} <A href="http://mexcdf.sourceforge.net/"> \end{rawhtml}
975 \begin{verbatim}
976 http://mexcdf.sourceforge.net/
977 \end{verbatim}
978 \begin{rawhtml} </A> \end{rawhtml}
979 \begin{rawhtml} <A href="http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html"> \end{rawhtml}
980 \begin{verbatim}
981 http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html
982 \end{verbatim}
983 \begin{rawhtml} </A> \end{rawhtml}
984 \end{itemize}
985

  ViewVC Help
Powered by ViewVC 1.1.22