/[MITgcm]/manual/s_getstarted/text/getting_started.tex
ViewVC logotype

Contents of /manual/s_getstarted/text/getting_started.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.37 - (show annotations) (download) (as text)
Wed Jun 28 16:48:19 2006 UTC (19 years ago) by molod
Branch: MAIN
Changes since 1.36: +36 -58 lines
File MIME type: application/x-tex
Changes in 3.1 through 3.3

1 % $Header: /u/gcmpack/manual/part3/getting_started.tex,v 1.36 2006/06/27 19:08:22 molod Exp $
2 % $Name: $
3
4 %\section{Getting started}
5
6 In this section, we describe how to use the model. In the first
7 section, we provide enough information to help you get started with
8 the model. We believe the best way to familiarize yourself with the
9 model is to run the case study examples provided with the base
10 version. Information on how to obtain, compile, and run the code is
11 found there as well as a brief description of the model structure
12 directory and the case study examples. The latter and the code
13 structure are described more fully in chapters
14 \ref{chap:discretization} and \ref{chap:sarch}, respectively. Here, in
15 this section, we provide information on how to customize the code when
16 you are ready to try implementing the configuration you have in mind.
17
18 \section{Where to find information}
19 \label{sect:whereToFindInfo}
20 \begin{rawhtml}
21 <!-- CMIREDIR:whereToFindInfo: -->
22 \end{rawhtml}
23
24 There is a web-archived support mailing list for the model that
25 you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
26 \begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
27 \begin{verbatim}
28 http://mitgcm.org/mailman/listinfo/mitgcm-support/
29 http://mitgcm.org/pipermail/mitgcm-support/
30 \end{verbatim}
31 \begin{rawhtml} </A> \end{rawhtml}
32
33 \section{Obtaining the code}
34 \label{sect:obtainingCode}
35 \begin{rawhtml}
36 <!-- CMIREDIR:obtainingCode: -->
37 \end{rawhtml}
38
39 MITgcm can be downloaded from our system by following
40 the instructions below. As a courtesy we ask that you send e-mail to us at
41 \begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
42 MITgcm-support@mitgcm.org
43 \begin{rawhtml} </A> \end{rawhtml}
44 to enable us to keep track of who's using the model and in what application.
45 You can download the model two ways:
46
47 \begin{enumerate}
48 \item Using CVS software. CVS is a freely available source code management
49 tool. To use CVS you need to have the software installed. Many systems
50 come with CVS pre-installed, otherwise good places to look for
51 the software for a particular platform are
52 \begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
53 cvshome.org
54 \begin{rawhtml} </A> \end{rawhtml}
55 and
56 \begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
57 wincvs.org
58 \begin{rawhtml} </A> \end{rawhtml}
59 .
60
61 \item Using a tar file. This method is simple and does not
62 require any special software. However, this method does not
63 provide easy support for maintenance updates.
64
65 \end{enumerate}
66
67 \subsection{Method 1 - Checkout from CVS}
68 \label{sect:cvs_checkout}
69
70 If CVS is available on your system, we strongly encourage you to use it. CVS
71 provides an efficient and elegant way of organizing your code and keeping
72 track of your changes. If CVS is not available on your machine, you can also
73 download a tar file.
74
75 Before you can use CVS, the following environment variable(s) should
76 be set within your shell. For a csh or tcsh shell, put the following
77 \begin{verbatim}
78 % setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
79 \end{verbatim}
80 in your \texttt{.cshrc} or \texttt{.tcshrc} file. For bash or sh
81 shells, put:
82 \begin{verbatim}
83 % export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
84 \end{verbatim}
85 in your \texttt{.profile} or \texttt{.bashrc} file.
86
87
88 To get MITgcm through CVS, first register with the MITgcm CVS server
89 using command:
90 \begin{verbatim}
91 % cvs login ( CVS password: cvsanon )
92 \end{verbatim}
93 You only need to do a ``cvs login'' once.
94
95 To obtain the latest sources type:
96 \begin{verbatim}
97 % cvs co MITgcm
98 \end{verbatim}
99 or to get a specific release type:
100 \begin{verbatim}
101 % cvs co -P -r checkpoint52i_post MITgcm
102 \end{verbatim}
103 The MITgcm web site contains further directions concerning the source
104 code and CVS. It also contains a web interface to our CVS archive so
105 that one may easily view the state of files, revisions, and other
106 development milestones:
107 \begin{rawhtml} <A href="http://mitgcm.org/download" target="idontexist"> \end{rawhtml}
108 \begin{verbatim}
109 http://mitgcm.org/source_code.html
110 \end{verbatim}
111 \begin{rawhtml} </A> \end{rawhtml}
112
113 As a convenience, the MITgcm CVS server contains aliases which are
114 named subsets of the codebase. These aliases can be especially
115 helpful when used over slow internet connections or on machines with
116 restricted storage space. Table \ref{tab:cvsModules} contains a list
117 of CVS aliases
118 \begin{table}[htb]
119 \centering
120 \begin{tabular}[htb]{|lp{3.25in}|}\hline
121 \textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
122 \texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
123 \texttt{MITgcm\_verif\_basic}
124 & Source code plus a small set of the verification examples
125 (\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
126 \texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
127 \texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
128 \texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
129 \texttt{MITgcm\_verif\_all} & Source code plus all of the
130 verification examples. \\\hline
131 \end{tabular}
132 \caption{MITgcm CVS Modules}
133 \label{tab:cvsModules}
134 \end{table}
135
136 The checkout process creates a directory called \texttt{MITgcm}. If
137 the directory \texttt{MITgcm} exists this command updates your code
138 based on the repository. Each directory in the source tree contains a
139 directory \texttt{CVS}. This information is required by CVS to keep
140 track of your file versions with respect to the repository. Don't edit
141 the files in \texttt{CVS}! You can also use CVS to download code
142 updates. More extensive information on using CVS for maintaining
143 MITgcm code can be found
144 \begin{rawhtml} <A href="http://mitgcm.org/usingcvstoget.html" target="idontexist"> \end{rawhtml}
145 here
146 \begin{rawhtml} </A> \end{rawhtml}
147 .
148 It is important to note that the CVS aliases in Table
149 \ref{tab:cvsModules} cannot be used in conjunction with the CVS
150 \texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
151 they create can be changed to a different name following the check-out:
152 \begin{verbatim}
153 % cvs co MITgcm_verif_basic
154 % mv MITgcm MITgcm_verif_basic
155 \end{verbatim}
156
157 \subsubsection{Upgrading from an earlier version}
158
159 If you already have an earlier version of the code you can ``upgrade''
160 your copy instead of downloading the entire repository again. First,
161 ``cd'' (change directory) to the top of your working copy:
162 \begin{verbatim}
163 % cd MITgcm
164 \end{verbatim}
165 and then issue the cvs update command such as:
166 \begin{verbatim}
167 % cvs -q update -r checkpoint52i_post -d -P
168 \end{verbatim}
169 This will update the ``tag'' to ``checkpoint52i\_post'', add any new
170 directories (-d) and remove any empty directories (-P). The -q option
171 means be quiet which will reduce the number of messages you'll see in
172 the terminal. If you have modified the code prior to upgrading, CVS
173 will try to merge your changes with the upgrades. If there is a
174 conflict between your modifications and the upgrade, it will report
175 that file with a ``C'' in front, e.g.:
176 \begin{verbatim}
177 C model/src/ini_parms.F
178 \end{verbatim}
179 If the list of conflicts scrolled off the screen, you can re-issue the
180 cvs update command and it will report the conflicts. Conflicts are
181 indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
182 ``$>>>>>>>$''. For example,
183 {\small
184 \begin{verbatim}
185 <<<<<<< ini_parms.F
186 & bottomDragLinear,myOwnBottomDragCoefficient,
187 =======
188 & bottomDragLinear,bottomDragQuadratic,
189 >>>>>>> 1.18
190 \end{verbatim}
191 }
192 means that you added ``myOwnBottomDragCoefficient'' to a namelist at
193 the same time and place that we added ``bottomDragQuadratic''. You
194 need to resolve this conflict and in this case the line should be
195 changed to:
196 {\small
197 \begin{verbatim}
198 & bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
199 \end{verbatim}
200 }
201 and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
202 Unless you are making modifications which exactly parallel
203 developments we make, these types of conflicts should be rare.
204
205 \paragraph*{Upgrading to the current pre-release version}
206
207 We don't make a ``release'' for every little patch and bug fix in
208 order to keep the frequency of upgrades to a minimum. However, if you
209 have run into a problem for which ``we have already fixed in the
210 latest code'' and we haven't made a ``tag'' or ``release'' since that
211 patch then you'll need to get the latest code:
212 \begin{verbatim}
213 % cvs -q update -A -d -P
214 \end{verbatim}
215 Unlike, the ``check-out'' and ``update'' procedures above, there is no
216 ``tag'' or release name. The -A tells CVS to upgrade to the
217 very latest version. As a rule, we don't recommend this since you
218 might upgrade while we are in the processes of checking in the code so
219 that you may only have part of a patch. Using this method of updating
220 also means we can't tell what version of the code you are working
221 with. So please be sure you understand what you're doing.
222
223 \subsection{Method 2 - Tar file download}
224 \label{sect:conventionalDownload}
225
226 If you do not have CVS on your system, you can download the model as a
227 tar file from the web site at:
228 \begin{rawhtml} <A href=http://mitgcm.org/download target="idontexist"> \end{rawhtml}
229 \begin{verbatim}
230 http://mitgcm.org/download/
231 \end{verbatim}
232 \begin{rawhtml} </A> \end{rawhtml}
233 The tar file still contains CVS information which we urge you not to
234 delete; even if you do not use CVS yourself the information can help
235 us if you should need to send us your copy of the code. If a recent
236 tar file does not exist, then please contact the developers through
237 the
238 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
239 MITgcm-support@mitgcm.org
240 \begin{rawhtml} </A> \end{rawhtml}
241 mailing list.
242
243 \section{Model and directory structure}
244 \begin{rawhtml}
245 <!-- CMIREDIR:directory_structure: -->
246 \end{rawhtml}
247
248 The ``numerical'' model is contained within a execution environment
249 support wrapper. This wrapper is designed to provide a general
250 framework for grid-point models. MITgcmUV is a specific numerical
251 model that uses the framework. Under this structure the model is split
252 into execution environment support code and conventional numerical
253 model code. The execution environment support code is held under the
254 \texttt{eesupp} directory. The grid point model code is held under the
255 \texttt{model} directory. Code execution actually starts in the
256 \texttt{eesupp} routines and not in the \texttt{model} routines. For
257 this reason the top-level \texttt{MAIN.F} is in the
258 \texttt{eesupp/src} directory. In general, end-users should not need
259 to worry about this level. The top-level routine for the numerical
260 part of the code is in \texttt{model/src/THE\_MODEL\_MAIN.F}. Here is
261 a brief description of the directory structure of the model under the
262 root tree (a detailed description is given in section 3: Code
263 structure).
264
265 \begin{itemize}
266
267 \item \texttt{doc}: contains brief documentation notes.
268
269 \item \texttt{eesupp}: contains the execution environment source code.
270 Also subdivided into two subdirectories \texttt{inc} and
271 \texttt{src}.
272
273 \item \texttt{model}: this directory contains the main source code.
274 Also subdivided into two subdirectories \texttt{inc} and
275 \texttt{src}.
276
277 \item \texttt{pkg}: contains the source code for the packages. Each
278 package corresponds to a subdirectory. For example, \texttt{gmredi}
279 contains the code related to the Gent-McWilliams/Redi scheme,
280 \texttt{aim} the code relative to the atmospheric intermediate
281 physics. The packages are described in detail in chapter \ref{chap.packagesI}.
282
283 \item \texttt{tools}: this directory contains various useful tools.
284 For example, \texttt{genmake2} is a script written in csh (C-shell)
285 that should be used to generate your makefile. The directory
286 \texttt{adjoint} contains the makefile specific to the Tangent
287 linear and Adjoint Compiler (TAMC) that generates the adjoint code.
288 The latter is described in detail in part \ref{chap.ecco}.
289 This directory also contains the subdirectory build\_options, which
290 contains the `optfiles' with the compiler options for the different
291 compilers and machines that can run MITgcm.
292
293 \item \texttt{utils}: this directory contains various utilities. The
294 subdirectory \texttt{knudsen2} contains code and a makefile that
295 compute coefficients of the polynomial approximation to the knudsen
296 formula for an ocean nonlinear equation of state. The
297 \texttt{matlab} subdirectory contains matlab scripts for reading
298 model output directly into matlab. \texttt{scripts} contains C-shell
299 post-processing scripts for joining processor-based and tiled-based
300 model output. The subdirectory exch2 contains the code needed for
301 the exch2 package to work with different combinations of domain
302 decompositions.
303
304 \item \texttt{verification}: this directory contains the model
305 examples. See section \ref{sect:modelExamples}.
306
307 \item \texttt{jobs}: contains sample job scripts for running MITgcm.
308
309 \item \texttt{lsopt}: Line search code used for optimization.
310
311 \item \texttt{optim}: Interface between MITgcm and line search code.
312
313 \end{itemize}
314
315 \section[Building MITgcm]{Building the code}
316 \label{sect:buildingCode}
317 \begin{rawhtml}
318 <!-- CMIREDIR:buildingCode: -->
319 \end{rawhtml}
320
321 To compile the code, we use the \texttt{make} program. This uses a
322 file (\texttt{Makefile}) that allows us to pre-process source files,
323 specify compiler and optimization options and also figures out any
324 file dependencies. We supply a script (\texttt{genmake2}), described
325 in section \ref{sect:genmake}, that automatically creates the
326 \texttt{Makefile} for you. You then need to build the dependencies and
327 compile the code.
328
329 As an example, assume that you want to build and run experiment
330 \texttt{verification/exp2}. The are multiple ways and places to
331 actually do this but here let's build the code in
332 \texttt{verification/exp2/build}:
333 \begin{verbatim}
334 % cd verification/exp2/build
335 \end{verbatim}
336 First, build the \texttt{Makefile}:
337 \begin{verbatim}
338 % ../../../tools/genmake2 -mods=../code
339 \end{verbatim}
340 The command line option tells \texttt{genmake} to override model source
341 code with any files in the directory \texttt{../code/}.
342
343 On many systems, the \texttt{genmake2} program will be able to
344 automatically recognize the hardware, find compilers and other tools
345 within the user's path (``\texttt{echo \$PATH}''), and then choose an
346 appropriate set of options from the files (``optfiles'') contained in
347 the \texttt{tools/build\_options} directory. Under some
348 circumstances, a user may have to create a new ``optfile'' in order to
349 specify the exact combination of compiler, compiler flags, libraries,
350 and other options necessary to build a particular configuration of
351 MITgcm. In such cases, it is generally helpful to read the existing
352 ``optfiles'' and mimic their syntax.
353
354 Through the MITgcm-support list, the MITgcm developers are willing to
355 provide help writing or modifing ``optfiles''. And we encourage users
356 to post new ``optfiles'' (particularly ones for new machines or
357 architectures) to the
358 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
359 MITgcm-support@mitgcm.org
360 \begin{rawhtml} </A> \end{rawhtml}
361 list.
362
363 To specify an optfile to \texttt{genmake2}, the syntax is:
364 \begin{verbatim}
365 % ../../../tools/genmake2 -mods=../code -of /path/to/optfile
366 \end{verbatim}
367
368 Once a \texttt{Makefile} has been generated, we create the
369 dependencies with the command:
370 \begin{verbatim}
371 % make depend
372 \end{verbatim}
373 This modifies the \texttt{Makefile} by attaching a (usually, long)
374 list of files upon which other files depend. The purpose of this is to
375 reduce re-compilation if and when you start to modify the code. The
376 {\tt make depend} command also creates links from the model source to
377 this directory. It is important to note that the {\tt make depend}
378 stage will occasionally produce warnings or errors since the
379 dependency parsing tool is unable to find all of the necessary header
380 files (\textit{eg.} \texttt{netcdf.inc}). In these circumstances, it
381 is usually OK to ignore the warnings/errors and proceed to the next
382 step.
383
384 Next one can compile the code using:
385 \begin{verbatim}
386 % make
387 \end{verbatim}
388 The {\tt make} command creates an executable called \texttt{mitgcmuv}.
389 Additional make ``targets'' are defined within the makefile to aid in
390 the production of adjoint and other versions of MITgcm. On SMP
391 (shared multi-processor) systems, the build process can often be sped
392 up appreciably using the command:
393 \begin{verbatim}
394 % make -j 2
395 \end{verbatim}
396 where the ``2'' can be replaced with a number that corresponds to the
397 number of CPUs available.
398
399 Now you are ready to run the model. General instructions for doing so are
400 given in section \ref{sect:runModel}. Here, we can run the model by
401 first creating links to all the input files:
402 \begin{verbatim}
403 ln -s ../input/* .
404 \end{verbatim}
405 and then calling the executable with:
406 \begin{verbatim}
407 ./mitgcmuv > output.txt
408 \end{verbatim}
409 where we are re-directing the stream of text output to the file
410 \texttt{output.txt}.
411
412 \subsection{Building/compiling the code elsewhere}
413
414 In the example above (section \ref{sect:buildingCode}) we built the
415 executable in the {\em input} directory of the experiment for
416 convenience. You can also configure and compile the code in other
417 locations, for example on a scratch disk with out having to copy the
418 entire source tree. The only requirement to do so is you have {\tt
419 genmake2} in your path or you know the absolute path to {\tt
420 genmake2}.
421
422 The following sections outline some possible methods of organizing
423 your source and data.
424
425 \subsubsection{Building from the {\em ../code directory}}
426
427 This is just as simple as building in the {\em input/} directory:
428 \begin{verbatim}
429 % cd verification/exp2/code
430 % ../../../tools/genmake2
431 % make depend
432 % make
433 \end{verbatim}
434 However, to run the model the executable ({\em mitgcmuv}) and input
435 files must be in the same place. If you only have one calculation to make:
436 \begin{verbatim}
437 % cd ../input
438 % cp ../code/mitgcmuv ./
439 % ./mitgcmuv > output.txt
440 \end{verbatim}
441 or if you will be making multiple runs with the same executable:
442 \begin{verbatim}
443 % cd ../
444 % cp -r input run1
445 % cp code/mitgcmuv run1
446 % cd run1
447 % ./mitgcmuv > output.txt
448 \end{verbatim}
449
450 \subsubsection{Building from a new directory}
451
452 Since the {\em input} directory contains input files it is often more
453 useful to keep {\em input} pristine and build in a new directory
454 within {\em verification/exp2/}:
455 \begin{verbatim}
456 % cd verification/exp2
457 % mkdir build
458 % cd build
459 % ../../../tools/genmake2 -mods=../code
460 % make depend
461 % make
462 \end{verbatim}
463 This builds the code exactly as before but this time you need to copy
464 either the executable or the input files or both in order to run the
465 model. For example,
466 \begin{verbatim}
467 % cp ../input/* ./
468 % ./mitgcmuv > output.txt
469 \end{verbatim}
470 or if you tend to make multiple runs with the same executable then
471 running in a new directory each time might be more appropriate:
472 \begin{verbatim}
473 % cd ../
474 % mkdir run1
475 % cp build/mitgcmuv run1/
476 % cp input/* run1/
477 % cd run1
478 % ./mitgcmuv > output.txt
479 \end{verbatim}
480
481 \subsubsection{Building on a scratch disk}
482
483 Model object files and output data can use up large amounts of disk
484 space so it is often the case that you will be operating on a large
485 scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
486 following commands will build the model in {\em /scratch/exp2-run1}:
487 \begin{verbatim}
488 % cd /scratch/exp2-run1
489 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
490 -mods=~/MITgcm/verification/exp2/code
491 % make depend
492 % make
493 \end{verbatim}
494 To run the model here, you'll need the input files:
495 \begin{verbatim}
496 % cp ~/MITgcm/verification/exp2/input/* ./
497 % ./mitgcmuv > output.txt
498 \end{verbatim}
499
500 As before, you could build in one directory and make multiple runs of
501 the one experiment:
502 \begin{verbatim}
503 % cd /scratch/exp2
504 % mkdir build
505 % cd build
506 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
507 -mods=~/MITgcm/verification/exp2/code
508 % make depend
509 % make
510 % cd ../
511 % cp -r ~/MITgcm/verification/exp2/input run2
512 % cd run2
513 % ./mitgcmuv > output.txt
514 \end{verbatim}
515
516
517 \subsection{Using \texttt{genmake2}}
518 \label{sect:genmake}
519
520 To compile the code, first use the program \texttt{genmake2} (located
521 in the \texttt{tools} directory) to generate a Makefile.
522 \texttt{genmake2} is a shell script written to work with all
523 ``sh''--compatible shells including bash v1, bash v2, and Bourne.
524 Internally, \texttt{genmake2} determines the locations of needed
525 files, the compiler, compiler options, libraries, and Unix tools. It
526 relies upon a number of ``optfiles'' located in the
527 \texttt{tools/build\_options} directory.
528
529 The purpose of the optfiles is to provide all the compilation options
530 for particular ``platforms'' (where ``platform'' roughly means the
531 combination of the hardware and the compiler) and code configurations.
532 Given the combinations of possible compilers and library dependencies
533 ({\it eg.} MPI and NetCDF) there may be numerous optfiles available
534 for a single machine. The naming scheme for the majority of the
535 optfiles shipped with the code is
536 \begin{center}
537 {\bf OS\_HARDWARE\_COMPILER }
538 \end{center}
539 where
540 \begin{description}
541 \item[OS] is the name of the operating system (generally the
542 lower-case output of the {\tt 'uname'} command)
543 \item[HARDWARE] is a string that describes the CPU type and
544 corresponds to output from the {\tt 'uname -m'} command:
545 \begin{description}
546 \item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
547 and athlon
548 \item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
549 \item[amd64] is AMD x86\_64 systems
550 \item[ppc] is for Mac PowerPC systems
551 \end{description}
552 \item[COMPILER] is the compiler name (generally, the name of the
553 FORTRAN executable)
554 \end{description}
555
556 In many cases, the default optfiles are sufficient and will result in
557 usable Makefiles. However, for some machines or code configurations,
558 new ``optfiles'' must be written. To create a new optfile, it is
559 generally best to start with one of the defaults and modify it to suit
560 your needs. Like \texttt{genmake2}, the optfiles are all written
561 using a simple ``sh''--compatible syntax. While nearly all variables
562 used within \texttt{genmake2} may be specified in the optfiles, the
563 critical ones that should be defined are:
564
565 \begin{description}
566 \item[FC] the FORTRAN compiler (executable) to use
567 \item[DEFINES] the command-line DEFINE options passed to the compiler
568 \item[CPP] the C pre-processor to use
569 \item[NOOPTFLAGS] options flags for special files that should not be
570 optimized
571 \end{description}
572
573 For example, the optfile for a typical Red Hat Linux machine (``ia32''
574 architecture) using the GCC (g77) compiler is
575 \begin{verbatim}
576 FC=g77
577 DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
578 CPP='cpp -traditional -P'
579 NOOPTFLAGS='-O0'
580 # For IEEE, use the "-ffloat-store" option
581 if test "x$IEEE" = x ; then
582 FFLAGS='-Wimplicit -Wunused -Wuninitialized'
583 FOPTIM='-O3 -malign-double -funroll-loops'
584 else
585 FFLAGS='-Wimplicit -Wunused -ffloat-store'
586 FOPTIM='-O0 -malign-double'
587 fi
588 \end{verbatim}
589
590 If you write an optfile for an unrepresented machine or compiler, you
591 are strongly encouraged to submit the optfile to the MITgcm project
592 for inclusion. Please send the file to the
593 \begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
594 \begin{center}
595 MITgcm-support@mitgcm.org
596 \end{center}
597 \begin{rawhtml} </A> \end{rawhtml}
598 mailing list.
599
600 In addition to the optfiles, \texttt{genmake2} supports a number of
601 helpful command-line options. A complete list of these options can be
602 obtained from:
603 \begin{verbatim}
604 % genmake2 -h
605 \end{verbatim}
606
607 The most important command-line options are:
608 \begin{description}
609
610 \item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
611 should be used for a particular build.
612
613 If no "optfile" is specified (either through the command line or the
614 MITGCM\_OPTFILE environment variable), genmake2 will try to make a
615 reasonable guess from the list provided in {\em
616 tools/build\_options}. The method used for making this guess is
617 to first determine the combination of operating system and hardware
618 (eg. "linux\_ia32") and then find a working FORTRAN compiler within
619 the user's path. When these three items have been identified,
620 genmake2 will try to find an optfile that has a matching name.
621
622 \item[\texttt{--pdefault='PKG1 PKG2 PKG3 ...'}] specifies the default
623 set of packages to be used. The normal order of precedence for
624 packages is as follows:
625 \begin{enumerate}
626 \item If available, the command line (\texttt{--pdefault}) settings
627 over-rule any others.
628
629 \item Next, \texttt{genmake2} will look for a file named
630 ``\texttt{packages.conf}'' in the local directory or in any of the
631 directories specified with the \texttt{--mods} option.
632
633 \item Finally, if neither of the above are available,
634 \texttt{genmake2} will use the \texttt{/pkg/pkg\_default} file.
635 \end{enumerate}
636
637 \item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
638 used for packages.
639
640 If not specified, the default dependency file {\em pkg/pkg\_depend}
641 is used. The syntax for this file is parsed on a line-by-line basis
642 where each line containes either a comment ("\#") or a simple
643 "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
644 specifies a "must be used with" or a "must not be used with"
645 relationship, respectively. If no rule is specified, then it is
646 assumed that the two packages are compatible and will function
647 either with or without each other.
648
649 \item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
650 automatic differentiation options file to be used. The file is
651 analogous to the ``optfile'' defined above but it specifies
652 information for the AD build process.
653
654 The default file is located in {\em
655 tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
656 and "TAMC" compilers. An alternate version is also available at
657 {\em tools/adjoint\_options/adjoint\_staf} that selects the newer
658 "STAF" compiler. As with any compilers, it is helpful to have their
659 directories listed in your {\tt \$PATH} environment variable.
660
661 \item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
662 directories containing ``modifications''. These directories contain
663 files with names that may (or may not) exist in the main MITgcm
664 source tree but will be overridden by any identically-named sources
665 within the ``MODS'' directories.
666
667 The order of precedence for this "name-hiding" is as follows:
668 \begin{itemize}
669 \item ``MODS'' directories (in the order given)
670 \item Packages either explicitly specified or provided by default
671 (in the order given)
672 \item Packages included due to package dependencies (in the order
673 that that package dependencies are parsed)
674 \item The "standard dirs" (which may have been specified by the
675 ``-standarddirs'' option)
676 \end{itemize}
677
678 \item[\texttt{--mpi}] This option enables certain MPI features (using
679 CPP \texttt{\#define}s) within the code and is necessary for MPI
680 builds (see Section \ref{sect:mpi-build}).
681
682 \item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
683 soft-links and other bugs common with the \texttt{make} versions
684 provided by commercial Unix vendors, GNU \texttt{make} (sometimes
685 called \texttt{gmake}) should be preferred. This option provides a
686 means for specifying the make executable to be used.
687
688 \item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
689 machines, the ``bash'' shell is unavailable. To run on these
690 systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
691 a Bourne, POSIX, or compatible) shell. The syntax in these
692 circumstances is:
693 \begin{center}
694 \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
695 \end{center}
696 where \texttt{/bin/sh} can be replaced with the full path and name
697 of the desired shell.
698
699 \end{description}
700
701
702 \subsection{Building with MPI}
703 \label{sect:mpi-build}
704
705 Building MITgcm to use MPI libraries can be complicated due to the
706 variety of different MPI implementations available, their dependencies
707 or interactions with different compilers, and their often ad-hoc
708 locations within file systems. For these reasons, its generally a
709 good idea to start by finding and reading the documentation for your
710 machine(s) and, if necessary, seeking help from your local systems
711 administrator.
712
713 The steps for building MITgcm with MPI support are:
714 \begin{enumerate}
715
716 \item Determine the locations of your MPI-enabled compiler and/or MPI
717 libraries and put them into an options file as described in Section
718 \ref{sect:genmake}. One can start with one of the examples in:
719 \begin{rawhtml} <A
720 href="http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/tools/build_options/">
721 \end{rawhtml}
722 \begin{center}
723 \texttt{MITgcm/tools/build\_options/}
724 \end{center}
725 \begin{rawhtml} </A> \end{rawhtml}
726 such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
727 \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
728 hand. You may need help from your user guide or local systems
729 administrator to determine the exact location of the MPI libraries.
730 If libraries are not installed, MPI implementations and related
731 tools are available including:
732 \begin{itemize}
733 \item \begin{rawhtml} <A
734 href="http://www-unix.mcs.anl.gov/mpi/mpich/">
735 \end{rawhtml}
736 MPICH
737 \begin{rawhtml} </A> \end{rawhtml}
738
739 \item \begin{rawhtml} <A
740 href="http://www.lam-mpi.org/">
741 \end{rawhtml}
742 LAM/MPI
743 \begin{rawhtml} </A> \end{rawhtml}
744
745 \item \begin{rawhtml} <A
746 href="http://www.osc.edu/~pw/mpiexec/">
747 \end{rawhtml}
748 MPIexec
749 \begin{rawhtml} </A> \end{rawhtml}
750 \end{itemize}
751
752 \item Build the code with the \texttt{genmake2} \texttt{-mpi} option
753 (see Section \ref{sect:genmake}) using commands such as:
754 {\footnotesize \begin{verbatim}
755 % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
756 % make depend
757 % make
758 \end{verbatim} }
759
760 \item Run the code with the appropriate MPI ``run'' or ``exec''
761 program provided with your particular implementation of MPI.
762 Typical MPI packages such as MPICH will use something like:
763 \begin{verbatim}
764 % mpirun -np 4 -machinefile mf ./mitgcmuv
765 \end{verbatim}
766 Sightly more complicated scripts may be needed for many machines
767 since execution of the code may be controlled by both the MPI
768 library and a job scheduling and queueing system such as PBS,
769 LoadLeveller, Condor, or any of a number of similar tools. A few
770 example scripts (those used for our \begin{rawhtml} <A
771 href="http://mitgcm.org/testing.html"> \end{rawhtml}regular
772 verification runs\begin{rawhtml} </A> \end{rawhtml}) are available
773 at:
774 \begin{rawhtml} <A
775 href="http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">
776 \end{rawhtml}
777 {\footnotesize \tt
778 http://mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm\_contrib/test\_scripts/ }
779 \begin{rawhtml} </A> \end{rawhtml}
780
781 \end{enumerate}
782
783 An example of the above process on the MITgcm cluster (``cg01'') using
784 the GNU g77 compiler and the mpich MPI library is:
785
786 {\footnotesize \begin{verbatim}
787 % cd MITgcm/verification/exp5
788 % mkdir build
789 % cd build
790 % ../../../tools/genmake2 -mpi -mods=../code \
791 -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
792 % make depend
793 % make
794 % cd ../input
795 % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
796 -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
797 \end{verbatim} }
798
799 \section[Running MITgcm]{Running the model in prognostic mode}
800 \label{sect:runModel}
801 \begin{rawhtml}
802 <!-- CMIREDIR:runModel: -->
803 \end{rawhtml}
804
805 If compilation finished succesfully (section \ref{sect:buildingCode})
806 then an executable called \texttt{mitgcmuv} will now exist in the
807 local directory.
808
809 To run the model as a single process (\textit{ie.} not in parallel)
810 simply type:
811 \begin{verbatim}
812 % ./mitgcmuv
813 \end{verbatim}
814 The ``./'' is a safe-guard to make sure you use the local executable
815 in case you have others that exist in your path (surely odd if you
816 do!). The above command will spew out many lines of text output to
817 your screen. This output contains details such as parameter values as
818 well as diagnostics such as mean Kinetic energy, largest CFL number,
819 etc. It is worth keeping this text output with the binary output so we
820 normally re-direct the \texttt{stdout} stream as follows:
821 \begin{verbatim}
822 % ./mitgcmuv > output.txt
823 \end{verbatim}
824 In the event that the model encounters an error and stops, it is very
825 helpful to include the last few line of this \texttt{output.txt} file
826 along with the (\texttt{stderr}) error message within any bug reports.
827
828 For the example experiments in \texttt{verification}, an example of the
829 output is kept in \texttt{results/output.txt} for comparison. You can
830 compare your \texttt{output.txt} with the corresponding one for that
831 experiment to check that the set-up works.
832
833
834
835 \subsection{Output files}
836
837 The model produces various output files and, when using \texttt{mnc},
838 sometimes even directories. Depending upon the I/O package(s)
839 selected at compile time (either \texttt{mdsio} or \texttt{mnc} or
840 both as determined by \texttt{code/packages.conf}) and the run-time
841 flags set (in \texttt{input/data.pkg}), the following output may
842 appear.
843
844
845 \subsubsection{MDSIO output files}
846
847 The ``traditional'' output files are generated by the \texttt{mdsio}
848 package. At a minimum, the instantaneous ``state'' of the model is
849 written out, which is made of the following files:
850
851 \begin{itemize}
852 \item \texttt{U.00000nIter} - zonal component of velocity field (m/s
853 and positive eastward).
854
855 \item \texttt{V.00000nIter} - meridional component of velocity field
856 (m/s and positive northward).
857
858 \item \texttt{W.00000nIter} - vertical component of velocity field
859 (ocean: m/s and positive upward, atmosphere: Pa/s and positive
860 towards increasing pressure i.e. downward).
861
862 \item \texttt{T.00000nIter} - potential temperature (ocean:
863 $^{\circ}\mathrm{C}$, atmosphere: $^{\circ}\mathrm{K}$).
864
865 \item \texttt{S.00000nIter} - ocean: salinity (psu), atmosphere: water
866 vapor (g/kg).
867
868 \item \texttt{Eta.00000nIter} - ocean: surface elevation (m),
869 atmosphere: surface pressure anomaly (Pa).
870 \end{itemize}
871
872 The chain \texttt{00000nIter} consists of ten figures that specify the
873 iteration number at which the output is written out. For example,
874 \texttt{U.0000000300} is the zonal velocity at iteration 300.
875
876 In addition, a ``pickup'' or ``checkpoint'' file called:
877
878 \begin{itemize}
879 \item \texttt{pickup.00000nIter}
880 \end{itemize}
881
882 is written out. This file represents the state of the model in a condensed
883 form and is used for restarting the integration. If the C-D scheme is used,
884 there is an additional ``pickup'' file:
885
886 \begin{itemize}
887 \item \texttt{pickup\_cd.00000nIter}
888 \end{itemize}
889
890 containing the D-grid velocity data and that has to be written out as well
891 in order to restart the integration. Rolling checkpoint files are the same
892 as the pickup files but are named differently. Their name contain the chain
893 \texttt{ckptA} or \texttt{ckptB} instead of \texttt{00000nIter}. They can be
894 used to restart the model but are overwritten every other time they are
895 output to save disk space during long integrations.
896
897
898
899 \subsubsection{MNC output files}
900
901 Unlike the \texttt{mdsio} output, the \texttt{mnc}--generated output
902 is usually (though not necessarily) placed within a subdirectory with
903 a name such as \texttt{mnc\_test\_\${DATE}\_\${SEQ}}. The files
904 within this subdirectory are all in the ``self-describing'' netCDF
905 format and can thus be browsed and/or plotted using tools such as:
906 \begin{itemize}
907 \item \texttt{ncdump} is a utility which is typically included
908 with every netCDF install:
909 \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
910 \begin{verbatim}
911 http://www.unidata.ucar.edu/packages/netcdf/
912 \end{verbatim}
913 \begin{rawhtml} </A> \end{rawhtml} and it converts the netCDF
914 binaries into formatted ASCII text files.
915
916 \item \texttt{ncview} utility is a very convenient and quick way
917 to plot netCDF data and it runs on most OSes:
918 \begin{rawhtml} <A href="http://meteora.ucsd.edu/~pierce/ncview_home_page.html"> \end{rawhtml}
919 \begin{verbatim}
920 http://meteora.ucsd.edu/~pierce/ncview_home_page.html
921 \end{verbatim}
922 \begin{rawhtml} </A> \end{rawhtml}
923
924 \item MatLAB(c) and other common post-processing environments provide
925 various netCDF interfaces including:
926 \begin{rawhtml} <A href="http://mexcdf.sourceforge.net/"> \end{rawhtml}
927 \begin{verbatim}
928 http://mexcdf.sourceforge.net/
929 \end{verbatim}
930 \begin{rawhtml} </A> \end{rawhtml}
931 \begin{rawhtml} <A href="http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html"> \end{rawhtml}
932 \begin{verbatim}
933 http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html
934 \end{verbatim}
935 \begin{rawhtml} </A> \end{rawhtml}
936 \end{itemize}
937
938
939 \subsection{Looking at the output}
940
941 The ``traditional'' or mdsio model data are written according to a
942 ``meta/data'' file format. Each variable is associated with two files
943 with suffix names \texttt{.data} and \texttt{.meta}. The
944 \texttt{.data} file contains the data written in binary form
945 (big\_endian by default). The \texttt{.meta} file is a ``header'' file
946 that contains information about the size and the structure of the
947 \texttt{.data} file. This way of organizing the output is particularly
948 useful when running multi-processors calculations. The base version of
949 the model includes a few matlab utilities to read output files written
950 in this format. The matlab scripts are located in the directory
951 \texttt{utils/matlab} under the root tree. The script \texttt{rdmds.m}
952 reads the data. Look at the comments inside the script to see how to
953 use it.
954
955 Some examples of reading and visualizing some output in {\em Matlab}:
956 \begin{verbatim}
957 % matlab
958 >> H=rdmds('Depth');
959 >> contourf(H');colorbar;
960 >> title('Depth of fluid as used by model');
961
962 >> eta=rdmds('Eta',10);
963 >> imagesc(eta');axis ij;colorbar;
964 >> title('Surface height at iter=10');
965
966 >> eta=rdmds('Eta',[0:10:100]);
967 >> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
968 \end{verbatim}
969
970 Similar scripts for netCDF output (\texttt{rdmnc.m}) are available and
971 they are described in Section \ref{sec:pkg:mnc}.
972

  ViewVC Help
Powered by ViewVC 1.1.22