/[MITgcm]/manual/s_getstarted/text/getting_started.tex
ViewVC logotype

Contents of /manual/s_getstarted/text/getting_started.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.43 - (show annotations) (download) (as text)
Fri May 28 21:04:18 2010 UTC (15 years, 1 month ago) by jmc
Branch: MAIN
Changes since 1.42: +2 -3 lines
File MIME type: application/x-tex
http link match its description

1 % $Header: /u/gcmpack/manual/part3/getting_started.tex,v 1.42 2010/05/28 02:09:59 jmc Exp $
2 % $Name: $
3
4 %\section{Getting started}
5
6 We believe the best way to familiarize yourself with the
7 model is to run the case study examples provided with the base
8 version. Information on how to obtain, compile, and run the code is
9 found here as well as a brief description of the model structure
10 directory and the case study examples. Information is also provided
11 here on how to customize the code when you are ready to try implementing
12 the configuration you have in mind. The code and algorithm
13 are described more fully in chapters \ref{chap:discretization} and
14 \ref{chap:sarch}.
15
16 \section{Where to find information}
17 \label{sect:whereToFindInfo}
18 \begin{rawhtml}
19 <!-- CMIREDIR:whereToFindInfo: -->
20 \end{rawhtml}
21
22 There is a web-archived support mailing list for the model that
23 you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
24 \begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
25 \begin{verbatim}
26 http://mitgcm.org/mailman/listinfo/mitgcm-support/
27 http://mitgcm.org/pipermail/mitgcm-support/
28 \end{verbatim}
29 \begin{rawhtml} </A> \end{rawhtml}
30
31 \section{Obtaining the code}
32 \label{sect:obtainingCode}
33 \begin{rawhtml}
34 <!-- CMIREDIR:obtainingCode: -->
35 \end{rawhtml}
36
37 MITgcm can be downloaded from our system by following
38 the instructions below. As a courtesy we ask that you send e-mail to us at
39 \begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
40 MITgcm-support@mitgcm.org
41 \begin{rawhtml} </A> \end{rawhtml}
42 to enable us to keep track of who's using the model and in what application.
43 You can download the model two ways:
44
45 \begin{enumerate}
46 \item Using CVS software. CVS is a freely available source code management
47 tool. To use CVS you need to have the software installed. Many systems
48 come with CVS pre-installed, otherwise good places to look for
49 the software for a particular platform are
50 \begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
51 cvshome.org
52 \begin{rawhtml} </A> \end{rawhtml}
53 and
54 \begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
55 wincvs.org
56 \begin{rawhtml} </A> \end{rawhtml}
57 .
58
59 \item Using a tar file. This method is simple and does not
60 require any special software. However, this method does not
61 provide easy support for maintenance updates.
62
63 \end{enumerate}
64
65 \subsection{Method 1 - Checkout from CVS}
66 \label{sect:cvs_checkout}
67
68 If CVS is available on your system, we strongly encourage you to use it. CVS
69 provides an efficient and elegant way of organizing your code and keeping
70 track of your changes. If CVS is not available on your machine, you can also
71 download a tar file.
72
73 Before you can use CVS, the following environment variable(s) should
74 be set within your shell. For a csh or tcsh shell, put the following
75 \begin{verbatim}
76 % setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
77 \end{verbatim}
78 in your \texttt{.cshrc} or \texttt{.tcshrc} file. For bash or sh
79 shells, put:
80 \begin{verbatim}
81 % export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
82 \end{verbatim}
83 in your \texttt{.profile} or \texttt{.bashrc} file.
84
85
86 To get MITgcm through CVS, first register with the MITgcm CVS server
87 using command:
88 \begin{verbatim}
89 % cvs login ( CVS password: cvsanon )
90 \end{verbatim}
91 You only need to do a ``cvs login'' once.
92
93 To obtain the latest sources type:
94 \begin{verbatim}
95 % cvs co MITgcm
96 \end{verbatim}
97 or to get a specific release type:
98 \begin{verbatim}
99 % cvs co -P -r checkpoint52i_post MITgcm
100 \end{verbatim}
101 The MITgcm web site contains further directions concerning the source
102 code and CVS. It also contains a web interface to our CVS archive so
103 that one may easily view the state of files, revisions, and other
104 development milestones:
105 \begin{rawhtml} <A href="http://mitgcm.org/viewvc/MITgcm/MITgcm/" target="idontexist"> \end{rawhtml}
106 \begin{verbatim}
107 http://mitgcm.org/viewvc/MITgcm/MITgcm/
108 \end{verbatim}
109 \begin{rawhtml} </A> \end{rawhtml}
110
111 As a convenience, the MITgcm CVS server contains aliases which are
112 named subsets of the codebase. These aliases can be especially
113 helpful when used over slow internet connections or on machines with
114 restricted storage space. Table \ref{tab:cvsModules} contains a list
115 of CVS aliases
116 \begin{table}[htb]
117 \centering
118 \begin{tabular}[htb]{|lp{3.25in}|}\hline
119 \textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
120 \texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
121 \texttt{MITgcm\_verif\_basic}
122 & Source code plus a small set of the verification examples
123 (\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
124 \texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
125 \texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
126 \texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
127 \texttt{MITgcm\_verif\_all} & Source code plus all of the
128 verification examples. \\\hline
129 \end{tabular}
130 \caption{MITgcm CVS Modules}
131 \label{tab:cvsModules}
132 \end{table}
133
134 The checkout process creates a directory called \texttt{MITgcm}. If
135 the directory \texttt{MITgcm} exists this command updates your code
136 based on the repository. Each directory in the source tree contains a
137 directory \texttt{CVS}. This information is required by CVS to keep
138 track of your file versions with respect to the repository. Don't edit
139 the files in \texttt{CVS}! You can also use CVS to download code
140 updates. More extensive information on using CVS for maintaining
141 MITgcm code can be found
142 \begin{rawhtml} <A href="http://mitgcm.org/public/using_cvs.html" target="idontexist"> \end{rawhtml}
143 here
144 \begin{rawhtml} </A> \end{rawhtml}
145 .
146 It is important to note that the CVS aliases in Table
147 \ref{tab:cvsModules} cannot be used in conjunction with the CVS
148 \texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
149 they create can be changed to a different name following the check-out:
150 \begin{verbatim}
151 % cvs co MITgcm_verif_basic
152 % mv MITgcm MITgcm_verif_basic
153 \end{verbatim}
154
155 \subsubsection{Upgrading from an earlier version}
156
157 If you already have an earlier version of the code you can ``upgrade''
158 your copy instead of downloading the entire repository again. First,
159 ``cd'' (change directory) to the top of your working copy:
160 \begin{verbatim}
161 % cd MITgcm
162 \end{verbatim}
163 and then issue the cvs update command such as:
164 \begin{verbatim}
165 % cvs -q update -r checkpoint52i_post -d -P
166 \end{verbatim}
167 This will update the ``tag'' to ``checkpoint52i\_post'', add any new
168 directories (-d) and remove any empty directories (-P). The -q option
169 means be quiet which will reduce the number of messages you'll see in
170 the terminal. If you have modified the code prior to upgrading, CVS
171 will try to merge your changes with the upgrades. If there is a
172 conflict between your modifications and the upgrade, it will report
173 that file with a ``C'' in front, e.g.:
174 \begin{verbatim}
175 C model/src/ini_parms.F
176 \end{verbatim}
177 If the list of conflicts scrolled off the screen, you can re-issue the
178 cvs update command and it will report the conflicts. Conflicts are
179 indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
180 ``$>>>>>>>$''. For example,
181 {\small
182 \begin{verbatim}
183 <<<<<<< ini_parms.F
184 & bottomDragLinear,myOwnBottomDragCoefficient,
185 =======
186 & bottomDragLinear,bottomDragQuadratic,
187 >>>>>>> 1.18
188 \end{verbatim}
189 }
190 means that you added ``myOwnBottomDragCoefficient'' to a namelist at
191 the same time and place that we added ``bottomDragQuadratic''. You
192 need to resolve this conflict and in this case the line should be
193 changed to:
194 {\small
195 \begin{verbatim}
196 & bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
197 \end{verbatim}
198 }
199 and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
200 Unless you are making modifications which exactly parallel
201 developments we make, these types of conflicts should be rare.
202
203 \paragraph*{Upgrading to the current pre-release version}
204
205 We don't make a ``release'' for every little patch and bug fix in
206 order to keep the frequency of upgrades to a minimum. However, if you
207 have run into a problem for which ``we have already fixed in the
208 latest code'' and we haven't made a ``tag'' or ``release'' since that
209 patch then you'll need to get the latest code:
210 \begin{verbatim}
211 % cvs -q update -A -d -P
212 \end{verbatim}
213 Unlike, the ``check-out'' and ``update'' procedures above, there is no
214 ``tag'' or release name. The -A tells CVS to upgrade to the
215 very latest version. As a rule, we don't recommend this since you
216 might upgrade while we are in the processes of checking in the code so
217 that you may only have part of a patch. Using this method of updating
218 also means we can't tell what version of the code you are working
219 with. So please be sure you understand what you're doing.
220
221 \subsection{Method 2 - Tar file download}
222 \label{sect:conventionalDownload}
223
224 If you do not have CVS on your system, you can download the model as a
225 tar file from the web site at:
226 \begin{rawhtml} <A href=http://mitgcm.org/download/ target="idontexist"> \end{rawhtml}
227 \begin{verbatim}
228 http://mitgcm.org/download/
229 \end{verbatim}
230 \begin{rawhtml} </A> \end{rawhtml}
231 The tar file still contains CVS information which we urge you not to
232 delete; even if you do not use CVS yourself the information can help
233 us if you should need to send us your copy of the code. If a recent
234 tar file does not exist, then please contact the developers through
235 the
236 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
237 MITgcm-support@mitgcm.org
238 \begin{rawhtml} </A> \end{rawhtml}
239 mailing list.
240
241 \section{Model and directory structure}
242 \begin{rawhtml}
243 <!-- CMIREDIR:directory_structure: -->
244 \end{rawhtml}
245
246 The ``numerical'' model is contained within a execution environment
247 support wrapper. This wrapper is designed to provide a general
248 framework for grid-point models. MITgcmUV is a specific numerical
249 model that uses the framework. Under this structure the model is split
250 into execution environment support code and conventional numerical
251 model code. The execution environment support code is held under the
252 \texttt{eesupp} directory. The grid point model code is held under the
253 \texttt{model} directory. Code execution actually starts in the
254 \texttt{eesupp} routines and not in the \texttt{model} routines. For
255 this reason the top-level \texttt{MAIN.F} is in the
256 \texttt{eesupp/src} directory. In general, end-users should not need
257 to worry about this level. The top-level routine for the numerical
258 part of the code is in \texttt{model/src/THE\_MODEL\_MAIN.F}. Here is
259 a brief description of the directory structure of the model under the
260 root tree (a detailed description is given in section 3: Code
261 structure).
262
263 \begin{itemize}
264
265 \item \texttt{doc}: contains brief documentation notes.
266
267 \item \texttt{eesupp}: contains the execution environment source code.
268 Also subdivided into two subdirectories \texttt{inc} and
269 \texttt{src}.
270
271 \item \texttt{model}: this directory contains the main source code.
272 Also subdivided into two subdirectories \texttt{inc} and
273 \texttt{src}.
274
275 \item \texttt{pkg}: contains the source code for the packages. Each
276 package corresponds to a subdirectory. For example, \texttt{gmredi}
277 contains the code related to the Gent-McWilliams/Redi scheme,
278 \texttt{aim} the code relative to the atmospheric intermediate
279 physics. The packages are described in detail in chapter \ref{chap.packagesI}.
280
281 \item \texttt{tools}: this directory contains various useful tools.
282 For example, \texttt{genmake2} is a script written in csh (C-shell)
283 that should be used to generate your makefile. The directory
284 \texttt{adjoint} contains the makefile specific to the Tangent
285 linear and Adjoint Compiler (TAMC) that generates the adjoint code.
286 The latter is described in detail in part \ref{chap.ecco}.
287 This directory also contains the subdirectory build\_options, which
288 contains the `optfiles' with the compiler options for the different
289 compilers and machines that can run MITgcm.
290
291 \item \texttt{utils}: this directory contains various utilities. The
292 subdirectory \texttt{knudsen2} contains code and a makefile that
293 compute coefficients of the polynomial approximation to the knudsen
294 formula for an ocean nonlinear equation of state. The
295 \texttt{matlab} subdirectory contains matlab scripts for reading
296 model output directly into matlab. \texttt{scripts} contains C-shell
297 post-processing scripts for joining processor-based and tiled-based
298 model output. The subdirectory exch2 contains the code needed for
299 the exch2 package to work with different combinations of domain
300 decompositions.
301
302 \item \texttt{verification}: this directory contains the model
303 examples. See section \ref{sect:modelExamples}.
304
305 \item \texttt{jobs}: contains sample job scripts for running MITgcm.
306
307 \item \texttt{lsopt}: Line search code used for optimization.
308
309 \item \texttt{optim}: Interface between MITgcm and line search code.
310
311 \end{itemize}
312
313 \section[Building MITgcm]{Building the code}
314 \label{sect:buildingCode}
315 \begin{rawhtml}
316 <!-- CMIREDIR:buildingCode: -->
317 \end{rawhtml}
318
319 To compile the code, we use the \texttt{make} program. This uses a
320 file (\texttt{Makefile}) that allows us to pre-process source files,
321 specify compiler and optimization options and also figures out any
322 file dependencies. We supply a script (\texttt{genmake2}), described
323 in section \ref{sect:genmake}, that automatically creates the
324 \texttt{Makefile} for you. You then need to build the dependencies and
325 compile the code.
326
327 As an example, assume that you want to build and run experiment
328 \texttt{verification/exp2}. The are multiple ways and places to
329 actually do this but here let's build the code in
330 \texttt{verification/exp2/build}:
331 \begin{verbatim}
332 % cd verification/exp2/build
333 \end{verbatim}
334 First, build the \texttt{Makefile}:
335 \begin{verbatim}
336 % ../../../tools/genmake2 -mods=../code
337 \end{verbatim}
338 The command line option tells \texttt{genmake} to override model source
339 code with any files in the directory \texttt{../code/}.
340
341 On many systems, the \texttt{genmake2} program will be able to
342 automatically recognize the hardware, find compilers and other tools
343 within the user's path (``\texttt{echo \$PATH}''), and then choose an
344 appropriate set of options from the files (``optfiles'') contained in
345 the \texttt{tools/build\_options} directory. Under some
346 circumstances, a user may have to create a new ``optfile'' in order to
347 specify the exact combination of compiler, compiler flags, libraries,
348 and other options necessary to build a particular configuration of
349 MITgcm. In such cases, it is generally helpful to read the existing
350 ``optfiles'' and mimic their syntax.
351
352 Through the MITgcm-support list, the MITgcm developers are willing to
353 provide help writing or modifing ``optfiles''. And we encourage users
354 to post new ``optfiles'' (particularly ones for new machines or
355 architectures) to the
356 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
357 MITgcm-support@mitgcm.org
358 \begin{rawhtml} </A> \end{rawhtml}
359 list.
360
361 To specify an optfile to \texttt{genmake2}, the syntax is:
362 \begin{verbatim}
363 % ../../../tools/genmake2 -mods=../code -of /path/to/optfile
364 \end{verbatim}
365
366 Once a \texttt{Makefile} has been generated, we create the
367 dependencies with the command:
368 \begin{verbatim}
369 % make depend
370 \end{verbatim}
371 This modifies the \texttt{Makefile} by attaching a (usually, long)
372 list of files upon which other files depend. The purpose of this is to
373 reduce re-compilation if and when you start to modify the code. The
374 {\tt make depend} command also creates links from the model source to
375 this directory. It is important to note that the {\tt make depend}
376 stage will occasionally produce warnings or errors since the
377 dependency parsing tool is unable to find all of the necessary header
378 files (\textit{eg.} \texttt{netcdf.inc}). In these circumstances, it
379 is usually OK to ignore the warnings/errors and proceed to the next
380 step.
381
382 Next one can compile the code using:
383 \begin{verbatim}
384 % make
385 \end{verbatim}
386 The {\tt make} command creates an executable called \texttt{mitgcmuv}.
387 Additional make ``targets'' are defined within the makefile to aid in
388 the production of adjoint and other versions of MITgcm. On SMP
389 (shared multi-processor) systems, the build process can often be sped
390 up appreciably using the command:
391 \begin{verbatim}
392 % make -j 2
393 \end{verbatim}
394 where the ``2'' can be replaced with a number that corresponds to the
395 number of CPUs available.
396
397 Now you are ready to run the model. General instructions for doing so are
398 given in section \ref{sect:runModel}. Here, we can run the model by
399 first creating links to all the input files:
400 \begin{verbatim}
401 ln -s ../input/* .
402 \end{verbatim}
403 and then calling the executable with:
404 \begin{verbatim}
405 ./mitgcmuv > output.txt
406 \end{verbatim}
407 where we are re-directing the stream of text output to the file
408 \texttt{output.txt}.
409
410 \subsection{Building/compiling the code elsewhere}
411
412 In the example above (section \ref{sect:buildingCode}) we built the
413 executable in the {\em input} directory of the experiment for
414 convenience. You can also configure and compile the code in other
415 locations, for example on a scratch disk with out having to copy the
416 entire source tree. The only requirement to do so is you have {\tt
417 genmake2} in your path or you know the absolute path to {\tt
418 genmake2}.
419
420 The following sections outline some possible methods of organizing
421 your source and data.
422
423 \subsubsection{Building from the {\em ../code directory}}
424
425 This is just as simple as building in the {\em input/} directory:
426 \begin{verbatim}
427 % cd verification/exp2/code
428 % ../../../tools/genmake2
429 % make depend
430 % make
431 \end{verbatim}
432 However, to run the model the executable ({\em mitgcmuv}) and input
433 files must be in the same place. If you only have one calculation to make:
434 \begin{verbatim}
435 % cd ../input
436 % cp ../code/mitgcmuv ./
437 % ./mitgcmuv > output.txt
438 \end{verbatim}
439 or if you will be making multiple runs with the same executable:
440 \begin{verbatim}
441 % cd ../
442 % cp -r input run1
443 % cp code/mitgcmuv run1
444 % cd run1
445 % ./mitgcmuv > output.txt
446 \end{verbatim}
447
448 \subsubsection{Building from a new directory}
449
450 Since the {\em input} directory contains input files it is often more
451 useful to keep {\em input} pristine and build in a new directory
452 within {\em verification/exp2/}:
453 \begin{verbatim}
454 % cd verification/exp2
455 % mkdir build
456 % cd build
457 % ../../../tools/genmake2 -mods=../code
458 % make depend
459 % make
460 \end{verbatim}
461 This builds the code exactly as before but this time you need to copy
462 either the executable or the input files or both in order to run the
463 model. For example,
464 \begin{verbatim}
465 % cp ../input/* ./
466 % ./mitgcmuv > output.txt
467 \end{verbatim}
468 or if you tend to make multiple runs with the same executable then
469 running in a new directory each time might be more appropriate:
470 \begin{verbatim}
471 % cd ../
472 % mkdir run1
473 % cp build/mitgcmuv run1/
474 % cp input/* run1/
475 % cd run1
476 % ./mitgcmuv > output.txt
477 \end{verbatim}
478
479 \subsubsection{Building on a scratch disk}
480
481 Model object files and output data can use up large amounts of disk
482 space so it is often the case that you will be operating on a large
483 scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
484 following commands will build the model in {\em /scratch/exp2-run1}:
485 \begin{verbatim}
486 % cd /scratch/exp2-run1
487 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
488 -mods=~/MITgcm/verification/exp2/code
489 % make depend
490 % make
491 \end{verbatim}
492 To run the model here, you'll need the input files:
493 \begin{verbatim}
494 % cp ~/MITgcm/verification/exp2/input/* ./
495 % ./mitgcmuv > output.txt
496 \end{verbatim}
497
498 As before, you could build in one directory and make multiple runs of
499 the one experiment:
500 \begin{verbatim}
501 % cd /scratch/exp2
502 % mkdir build
503 % cd build
504 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
505 -mods=~/MITgcm/verification/exp2/code
506 % make depend
507 % make
508 % cd ../
509 % cp -r ~/MITgcm/verification/exp2/input run2
510 % cd run2
511 % ./mitgcmuv > output.txt
512 \end{verbatim}
513
514
515 \subsection{Using \texttt{genmake2}}
516 \label{sect:genmake}
517
518 To compile the code, first use the program \texttt{genmake2} (located
519 in the \texttt{tools} directory) to generate a Makefile.
520 \texttt{genmake2} is a shell script written to work with all
521 ``sh''--compatible shells including bash v1, bash v2, and Bourne.
522 %Internally, \texttt{genmake2} determines the locations of needed
523 %files, the compiler, compiler options, libraries, and Unix tools. It
524 %relies upon a number of ``optfiles'' located in the
525 %\texttt{tools/build\_options} directory.
526 \texttt{genmake2} parses information from the following sources:
527 \begin{description}
528 \item[-] a {\em gemake\_local} file if one is found in the current
529 directory
530 \item[-] command-line options
531 \item[-] an "options file" as specified by the command-line option
532 \texttt{--optfile=/PATH/FILENAME}
533 \item[-] a {\em packages.conf} file (if one is found) with the
534 specific list of packages to compile. The search path for
535 file {\em packages.conf} is, first, the current directory and
536 then each of the "MODS" directories in the given order (see below).
537 \end{description}
538
539 \subsubsection{Optfiles in \texttt{tools/build\_options} directory:}
540
541 The purpose of the optfiles is to provide all the compilation options
542 for particular ``platforms'' (where ``platform'' roughly means the
543 combination of the hardware and the compiler) and code configurations.
544 Given the combinations of possible compilers and library dependencies
545 ({\it eg.} MPI and NetCDF) there may be numerous optfiles available
546 for a single machine. The naming scheme for the majority of the
547 optfiles shipped with the code is
548 \begin{center}
549 {\bf OS\_HARDWARE\_COMPILER }
550 \end{center}
551 where
552 \begin{description}
553 \item[OS] is the name of the operating system (generally the
554 lower-case output of the {\tt 'uname'} command)
555 \item[HARDWARE] is a string that describes the CPU type and
556 corresponds to output from the {\tt 'uname -m'} command:
557 \begin{description}
558 \item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
559 and athlon
560 \item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
561 \item[amd64] is AMD x86\_64 systems
562 \item[ppc] is for Mac PowerPC systems
563 \end{description}
564 \item[COMPILER] is the compiler name (generally, the name of the
565 FORTRAN executable)
566 \end{description}
567
568 In many cases, the default optfiles are sufficient and will result in
569 usable Makefiles. However, for some machines or code configurations,
570 new ``optfiles'' must be written. To create a new optfile, it is
571 generally best to start with one of the defaults and modify it to suit
572 your needs. Like \texttt{genmake2}, the optfiles are all written
573 using a simple ``sh''--compatible syntax. While nearly all variables
574 used within \texttt{genmake2} may be specified in the optfiles, the
575 critical ones that should be defined are:
576
577 \begin{description}
578 \item[FC] the FORTRAN compiler (executable) to use
579 \item[DEFINES] the command-line DEFINE options passed to the compiler
580 \item[CPP] the C pre-processor to use
581 \item[NOOPTFLAGS] options flags for special files that should not be
582 optimized
583 \end{description}
584
585 For example, the optfile for a typical Red Hat Linux machine (``ia32''
586 architecture) using the GCC (g77) compiler is
587 \begin{verbatim}
588 FC=g77
589 DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
590 CPP='cpp -traditional -P'
591 NOOPTFLAGS='-O0'
592 # For IEEE, use the "-ffloat-store" option
593 if test "x$IEEE" = x ; then
594 FFLAGS='-Wimplicit -Wunused -Wuninitialized'
595 FOPTIM='-O3 -malign-double -funroll-loops'
596 else
597 FFLAGS='-Wimplicit -Wunused -ffloat-store'
598 FOPTIM='-O0 -malign-double'
599 fi
600 \end{verbatim}
601
602 If you write an optfile for an unrepresented machine or compiler, you
603 are strongly encouraged to submit the optfile to the MITgcm project
604 for inclusion. Please send the file to the
605 \begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
606 \begin{center}
607 MITgcm-support@mitgcm.org
608 \end{center}
609 \begin{rawhtml} </A> \end{rawhtml}
610 mailing list.
611
612 \subsubsection{Command-line options:}
613
614 In addition to the optfiles, \texttt{genmake2} supports a number of
615 helpful command-line options. A complete list of these options can be
616 obtained from:
617 \begin{verbatim}
618 % genmake2 -h
619 \end{verbatim}
620
621 The most important command-line options are:
622 \begin{description}
623
624 \item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
625 should be used for a particular build.
626
627 If no "optfile" is specified (either through the command line or the
628 MITGCM\_OPTFILE environment variable), genmake2 will try to make a
629 reasonable guess from the list provided in {\em
630 tools/build\_options}. The method used for making this guess is
631 to first determine the combination of operating system and hardware
632 (eg. "linux\_ia32") and then find a working FORTRAN compiler within
633 the user's path. When these three items have been identified,
634 genmake2 will try to find an optfile that has a matching name.
635
636 \item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
637 directories containing ``modifications''. These directories contain
638 files with names that may (or may not) exist in the main MITgcm
639 source tree but will be overridden by any identically-named sources
640 within the ``MODS'' directories.
641
642 The order of precedence for this "name-hiding" is as follows:
643 \begin{itemize}
644 \item ``MODS'' directories (in the order given)
645 \item Packages either explicitly specified or provided by default
646 (in the order given)
647 \item Packages included due to package dependencies (in the order
648 that that package dependencies are parsed)
649 \item The "standard dirs" (which may have been specified by the
650 ``-standarddirs'' option)
651 \end{itemize}
652
653 \item[\texttt{--pgroups=/PATH/FILENAME}] specifies the file
654 where package groups are defined. If not set, the package-groups
655 definition will be read from {\em pkg/pkg\_groups}.
656 It also contains the default list of packages (defined
657 as the group ``{\it default\_pkg\_list}'' which is used
658 when no specific package list ({\em packages.conf})
659 is found in current directory or in any "MODS" directory.
660
661 \item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
662 used for packages.
663
664 If not specified, the default dependency file {\em pkg/pkg\_depend}
665 is used. The syntax for this file is parsed on a line-by-line basis
666 where each line containes either a comment ("\#") or a simple
667 "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
668 specifies a "must be used with" or a "must not be used with"
669 relationship, respectively. If no rule is specified, then it is
670 assumed that the two packages are compatible and will function
671 either with or without each other.
672
673 \item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
674 automatic differentiation options file to be used. The file is
675 analogous to the ``optfile'' defined above but it specifies
676 information for the AD build process.
677
678 The default file is located in {\em
679 tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
680 and "TAMC" compilers. An alternate version is also available at
681 {\em tools/adjoint\_options/adjoint\_staf} that selects the newer
682 "STAF" compiler. As with any compilers, it is helpful to have their
683 directories listed in your {\tt \$PATH} environment variable.
684
685 \item[\texttt{--mpi}] This option enables certain MPI features (using
686 CPP \texttt{\#define}s) within the code and is necessary for MPI
687 builds (see Section \ref{sect:mpi-build}).
688
689 \item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
690 soft-links and other bugs common with the \texttt{make} versions
691 provided by commercial Unix vendors, GNU \texttt{make} (sometimes
692 called \texttt{gmake}) should be preferred. This option provides a
693 means for specifying the make executable to be used.
694
695 \item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
696 machines, the ``bash'' shell is unavailable. To run on these
697 systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
698 a Bourne, POSIX, or compatible) shell. The syntax in these
699 circumstances is:
700 \begin{center}
701 \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
702 \end{center}
703 where \texttt{/bin/sh} can be replaced with the full path and name
704 of the desired shell.
705
706 \end{description}
707
708
709 \subsection{Building with MPI}
710 \label{sect:mpi-build}
711
712 Building MITgcm to use MPI libraries can be complicated due to the
713 variety of different MPI implementations available, their dependencies
714 or interactions with different compilers, and their often ad-hoc
715 locations within file systems. For these reasons, its generally a
716 good idea to start by finding and reading the documentation for your
717 machine(s) and, if necessary, seeking help from your local systems
718 administrator.
719
720 The steps for building MITgcm with MPI support are:
721 \begin{enumerate}
722
723 \item Determine the locations of your MPI-enabled compiler and/or MPI
724 libraries and put them into an options file as described in Section
725 \ref{sect:genmake}. One can start with one of the examples in:
726 \begin{rawhtml} <A
727 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/build_options/">
728 \end{rawhtml}
729 \begin{center}
730 \texttt{MITgcm/tools/build\_options/}
731 \end{center}
732 \begin{rawhtml} </A> \end{rawhtml}
733 such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
734 \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
735 hand. You may need help from your user guide or local systems
736 administrator to determine the exact location of the MPI libraries.
737 If libraries are not installed, MPI implementations and related
738 tools are available including:
739 \begin{itemize}
740 \item \begin{rawhtml} <A
741 href="http://www-unix.mcs.anl.gov/mpi/mpich/">
742 \end{rawhtml}
743 MPICH
744 \begin{rawhtml} </A> \end{rawhtml}
745
746 \item \begin{rawhtml} <A
747 href="http://www.lam-mpi.org/">
748 \end{rawhtml}
749 LAM/MPI
750 \begin{rawhtml} </A> \end{rawhtml}
751
752 \item \begin{rawhtml} <A
753 href="http://www.osc.edu/~pw/mpiexec/">
754 \end{rawhtml}
755 MPIexec
756 \begin{rawhtml} </A> \end{rawhtml}
757 \end{itemize}
758
759 \item Build the code with the \texttt{genmake2} \texttt{-mpi} option
760 (see Section \ref{sect:genmake}) using commands such as:
761 {\footnotesize \begin{verbatim}
762 % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
763 % make depend
764 % make
765 \end{verbatim} }
766
767 \item Run the code with the appropriate MPI ``run'' or ``exec''
768 program provided with your particular implementation of MPI.
769 Typical MPI packages such as MPICH will use something like:
770 \begin{verbatim}
771 % mpirun -np 4 -machinefile mf ./mitgcmuv
772 \end{verbatim}
773 Sightly more complicated scripts may be needed for many machines
774 since execution of the code may be controlled by both the MPI
775 library and a job scheduling and queueing system such as PBS,
776 LoadLeveller, Condor, or any of a number of similar tools. A few
777 example scripts (those used for our \begin{rawhtml} <A
778 href="http://mitgcm.org/public/testing.html"> \end{rawhtml}regular
779 verification runs\begin{rawhtml} </A> \end{rawhtml}) are available
780 at:
781 \begin{rawhtml} <A
782 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
783 \end{rawhtml}
784 {\footnotesize \tt
785 http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example\_scripts/ }
786 \begin{rawhtml} </A> \end{rawhtml}
787 or at:
788 \begin{rawhtml} <A
789 href="http://mitgcm.org/viewvc/MITgcm/MITgcm_contrib/test_scripts/">
790 \end{rawhtml}
791 {\footnotesize \tt
792 http://mitgcm.org/viewvc/MITgcm/MITgcm\_contrib/test\_scripts/ }
793 \begin{rawhtml} </A> \end{rawhtml}
794
795 \end{enumerate}
796
797 An example of the above process on the MITgcm cluster (``cg01'') using
798 the GNU g77 compiler and the mpich MPI library is:
799
800 {\footnotesize \begin{verbatim}
801 % cd MITgcm/verification/exp5
802 % mkdir build
803 % cd build
804 % ../../../tools/genmake2 -mpi -mods=../code \
805 -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
806 % make depend
807 % make
808 % cd ../input
809 % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
810 -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
811 \end{verbatim} }
812
813 \section[Running MITgcm]{Running the model in prognostic mode}
814 \label{sect:runModel}
815 \begin{rawhtml}
816 <!-- CMIREDIR:runModel: -->
817 \end{rawhtml}
818
819 If compilation finished succesfully (section \ref{sect:buildingCode})
820 then an executable called \texttt{mitgcmuv} will now exist in the
821 local directory.
822
823 To run the model as a single process (\textit{ie.} not in parallel)
824 simply type:
825 \begin{verbatim}
826 % ./mitgcmuv
827 \end{verbatim}
828 The ``./'' is a safe-guard to make sure you use the local executable
829 in case you have others that exist in your path (surely odd if you
830 do!). The above command will spew out many lines of text output to
831 your screen. This output contains details such as parameter values as
832 well as diagnostics such as mean Kinetic energy, largest CFL number,
833 etc. It is worth keeping this text output with the binary output so we
834 normally re-direct the \texttt{stdout} stream as follows:
835 \begin{verbatim}
836 % ./mitgcmuv > output.txt
837 \end{verbatim}
838 In the event that the model encounters an error and stops, it is very
839 helpful to include the last few line of this \texttt{output.txt} file
840 along with the (\texttt{stderr}) error message within any bug reports.
841
842 For the example experiments in \texttt{verification}, an example of the
843 output is kept in \texttt{results/output.txt} for comparison. You can
844 compare your \texttt{output.txt} with the corresponding one for that
845 experiment to check that the set-up works.
846
847
848
849 \subsection{Output files}
850
851 The model produces various output files and, when using \texttt{mnc},
852 sometimes even directories. Depending upon the I/O package(s)
853 selected at compile time (either \texttt{mdsio} or \texttt{mnc} or
854 both as determined by \texttt{code/packages.conf}) and the run-time
855 flags set (in \texttt{input/data.pkg}), the following output may
856 appear.
857
858
859 \subsubsection{MDSIO output files}
860
861 The ``traditional'' output files are generated by the \texttt{mdsio}
862 package. At a minimum, the instantaneous ``state'' of the model is
863 written out, which is made of the following files:
864
865 \begin{itemize}
866 \item \texttt{U.00000nIter} - zonal component of velocity field (m/s
867 and positive eastward).
868
869 \item \texttt{V.00000nIter} - meridional component of velocity field
870 (m/s and positive northward).
871
872 \item \texttt{W.00000nIter} - vertical component of velocity field
873 (ocean: m/s and positive upward, atmosphere: Pa/s and positive
874 towards increasing pressure i.e. downward).
875
876 \item \texttt{T.00000nIter} - potential temperature (ocean:
877 $^{\circ}\mathrm{C}$, atmosphere: $^{\circ}\mathrm{K}$).
878
879 \item \texttt{S.00000nIter} - ocean: salinity (psu), atmosphere: water
880 vapor (g/kg).
881
882 \item \texttt{Eta.00000nIter} - ocean: surface elevation (m),
883 atmosphere: surface pressure anomaly (Pa).
884 \end{itemize}
885
886 The chain \texttt{00000nIter} consists of ten figures that specify the
887 iteration number at which the output is written out. For example,
888 \texttt{U.0000000300} is the zonal velocity at iteration 300.
889
890 In addition, a ``pickup'' or ``checkpoint'' file called:
891
892 \begin{itemize}
893 \item \texttt{pickup.00000nIter}
894 \end{itemize}
895
896 is written out. This file represents the state of the model in a condensed
897 form and is used for restarting the integration. If the C-D scheme is used,
898 there is an additional ``pickup'' file:
899
900 \begin{itemize}
901 \item \texttt{pickup\_cd.00000nIter}
902 \end{itemize}
903
904 containing the D-grid velocity data and that has to be written out as well
905 in order to restart the integration. Rolling checkpoint files are the same
906 as the pickup files but are named differently. Their name contain the chain
907 \texttt{ckptA} or \texttt{ckptB} instead of \texttt{00000nIter}. They can be
908 used to restart the model but are overwritten every other time they are
909 output to save disk space during long integrations.
910
911 \subsubsection{MNC output files}
912
913 Unlike the \texttt{mdsio} output, the \texttt{mnc}--generated output
914 is usually (though not necessarily) placed within a subdirectory with
915 a name such as \texttt{mnc\_test\_\${DATE}\_\${SEQ}}.
916
917 \subsection{Looking at the output}
918
919 The ``traditional'' or mdsio model data are written according to a
920 ``meta/data'' file format. Each variable is associated with two files
921 with suffix names \texttt{.data} and \texttt{.meta}. The
922 \texttt{.data} file contains the data written in binary form
923 (big\_endian by default). The \texttt{.meta} file is a ``header'' file
924 that contains information about the size and the structure of the
925 \texttt{.data} file. This way of organizing the output is particularly
926 useful when running multi-processors calculations. The base version of
927 the model includes a few matlab utilities to read output files written
928 in this format. The matlab scripts are located in the directory
929 \texttt{utils/matlab} under the root tree. The script \texttt{rdmds.m}
930 reads the data. Look at the comments inside the script to see how to
931 use it.
932
933 Some examples of reading and visualizing some output in {\em Matlab}:
934 \begin{verbatim}
935 % matlab
936 >> H=rdmds('Depth');
937 >> contourf(H');colorbar;
938 >> title('Depth of fluid as used by model');
939
940 >> eta=rdmds('Eta',10);
941 >> imagesc(eta');axis ij;colorbar;
942 >> title('Surface height at iter=10');
943
944 >> eta=rdmds('Eta',[0:10:100]);
945 >> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
946 \end{verbatim}
947
948 Similar scripts for netCDF output (\texttt{rdmnc.m}) are available and
949 they are described in Section \ref{sec:pkg:mnc}.
950
951 The MNC output files are all in the ``self-describing'' netCDF
952 format and can thus be browsed and/or plotted using tools such as:
953 \begin{itemize}
954 \item \texttt{ncdump} is a utility which is typically included
955 with every netCDF install:
956 \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
957 \begin{verbatim}
958 http://www.unidata.ucar.edu/packages/netcdf/
959 \end{verbatim}
960 \begin{rawhtml} </A> \end{rawhtml} and it converts the netCDF
961 binaries into formatted ASCII text files.
962
963 \item \texttt{ncview} utility is a very convenient and quick way
964 to plot netCDF data and it runs on most OSes:
965 \begin{rawhtml} <A href="http://meteora.ucsd.edu/~pierce/ncview_home_page.html"> \end{rawhtml}
966 \begin{verbatim}
967 http://meteora.ucsd.edu/~pierce/ncview_home_page.html
968 \end{verbatim}
969 \begin{rawhtml} </A> \end{rawhtml}
970
971 \item MatLAB(c) and other common post-processing environments provide
972 various netCDF interfaces including:
973 \begin{rawhtml} <A href="http://mexcdf.sourceforge.net/"> \end{rawhtml}
974 \begin{verbatim}
975 http://mexcdf.sourceforge.net/
976 \end{verbatim}
977 \begin{rawhtml} </A> \end{rawhtml}
978 \begin{rawhtml} <A href="http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html"> \end{rawhtml}
979 \begin{verbatim}
980 http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html
981 \end{verbatim}
982 \begin{rawhtml} </A> \end{rawhtml}
983 \end{itemize}
984

  ViewVC Help
Powered by ViewVC 1.1.22