/[MITgcm]/manual/s_getstarted/text/getting_started.tex
ViewVC logotype

Contents of /manual/s_getstarted/text/getting_started.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.45 - (show annotations) (download) (as text)
Wed May 11 18:58:02 2011 UTC (14 years, 2 months ago) by jmc
Branch: MAIN
Changes since 1.44: +10 -6 lines
File MIME type: application/x-tex
add option -P to "cvs co" instruction

1 % $Header: /u/gcmpack/manual/s_getstarted/text/getting_started.tex,v 1.44 2010/08/30 23:09:20 jmc Exp $
2 % $Name: $
3
4 %\section{Getting started}
5
6 We believe the best way to familiarize yourself with the
7 model is to run the case study examples provided with the base
8 version. Information on how to obtain, compile, and run the code is
9 found here as well as a brief description of the model structure
10 directory and the case study examples. Information is also provided
11 here on how to customize the code when you are ready to try implementing
12 the configuration you have in mind. The code and algorithm
13 are described more fully in chapters \ref{chap:discretization} and
14 \ref{chap:sarch}.
15
16 \section{Where to find information}
17 \label{sec:whereToFindInfo}
18 \begin{rawhtml}
19 <!-- CMIREDIR:whereToFindInfo: -->
20 \end{rawhtml}
21
22 There is a web-archived support mailing list for the model that
23 you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
24 \begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
25 \begin{verbatim}
26 http://mitgcm.org/mailman/listinfo/mitgcm-support/
27 http://mitgcm.org/pipermail/mitgcm-support/
28 \end{verbatim}
29 \begin{rawhtml} </A> \end{rawhtml}
30
31 \section{Obtaining the code}
32 \label{sec:obtainingCode}
33 \begin{rawhtml}
34 <!-- CMIREDIR:obtainingCode: -->
35 \end{rawhtml}
36
37 MITgcm can be downloaded from our system by following
38 the instructions below. As a courtesy we ask that you send e-mail to us at
39 \begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
40 MITgcm-support@mitgcm.org
41 \begin{rawhtml} </A> \end{rawhtml}
42 to enable us to keep track of who's using the model and in what application.
43 You can download the model two ways:
44
45 \begin{enumerate}
46 \item Using CVS software. CVS is a freely available source code management
47 tool. To use CVS you need to have the software installed. Many systems
48 come with CVS pre-installed, otherwise good places to look for
49 the software for a particular platform are
50 \begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
51 cvshome.org
52 \begin{rawhtml} </A> \end{rawhtml}
53 and
54 \begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
55 wincvs.org
56 \begin{rawhtml} </A> \end{rawhtml}
57 .
58
59 \item Using a tar file. This method is simple and does not
60 require any special software. However, this method does not
61 provide easy support for maintenance updates.
62
63 \end{enumerate}
64
65 \subsection{Method 1 - Checkout from CVS}
66 \label{sec:cvs_checkout}
67
68 If CVS is available on your system, we strongly encourage you to use it. CVS
69 provides an efficient and elegant way of organizing your code and keeping
70 track of your changes. If CVS is not available on your machine, you can also
71 download a tar file.
72
73 Before you can use CVS, the following environment variable(s) should
74 be set within your shell. For a csh or tcsh shell, put the following
75 \begin{verbatim}
76 % setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
77 \end{verbatim}
78 in your \texttt{.cshrc} or \texttt{.tcshrc} file. For bash or sh
79 shells, put:
80 \begin{verbatim}
81 % export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
82 \end{verbatim}
83 in your \texttt{.profile} or \texttt{.bashrc} file.
84
85
86 To get MITgcm through CVS, first register with the MITgcm CVS server
87 using command:
88 \begin{verbatim}
89 % cvs login ( CVS password: cvsanon )
90 \end{verbatim}
91 You only need to do a ``cvs login'' once.
92
93 To obtain the latest sources type:
94 \begin{verbatim}
95 % cvs co -P MITgcm
96 \end{verbatim}
97 or to get a specific release type:
98 \begin{verbatim}
99 % cvs co -P -r checkpoint52i_post MITgcm
100 \end{verbatim}
101 The CVS command ``\texttt{cvs co}'' is the abreviation of the full-name
102 ``\texttt{cvs checkout}'' command and using the option ``-P'' (\texttt{cvs co -P})
103 will prevent to download unnecessary empty directories.
104
105 The MITgcm web site contains further directions concerning the source
106 code and CVS. It also contains a web interface to our CVS archive so
107 that one may easily view the state of files, revisions, and other
108 development milestones:
109 \begin{rawhtml} <A href="http://mitgcm.org/viewvc/MITgcm/MITgcm/" target="idontexist"> \end{rawhtml}
110 \begin{verbatim}
111 http://mitgcm.org/viewvc/MITgcm/MITgcm/
112 \end{verbatim}
113 \begin{rawhtml} </A> \end{rawhtml}
114
115 As a convenience, the MITgcm CVS server contains aliases which are
116 named subsets of the codebase. These aliases can be especially
117 helpful when used over slow internet connections or on machines with
118 restricted storage space. Table \ref{tab:cvsModules} contains a list
119 of CVS aliases
120 \begin{table}[htb]
121 \centering
122 \begin{tabular}[htb]{|lp{3.25in}|}\hline
123 \textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
124 \texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
125 \texttt{MITgcm\_verif\_basic}
126 & Source code plus a small set of the verification examples
127 (\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
128 \texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
129 \texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
130 \texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
131 \texttt{MITgcm\_verif\_all} & Source code plus all of the
132 verification examples. \\\hline
133 \end{tabular}
134 \caption{MITgcm CVS Modules}
135 \label{tab:cvsModules}
136 \end{table}
137
138 The checkout process creates a directory called \texttt{MITgcm}. If
139 the directory \texttt{MITgcm} exists this command updates your code
140 based on the repository. Each directory in the source tree contains a
141 directory \texttt{CVS}. This information is required by CVS to keep
142 track of your file versions with respect to the repository. Don't edit
143 the files in \texttt{CVS}! You can also use CVS to download code
144 updates. More extensive information on using CVS for maintaining
145 MITgcm code can be found
146 \begin{rawhtml} <A href="http://mitgcm.org/public/using_cvs.html" target="idontexist"> \end{rawhtml}
147 here
148 \begin{rawhtml} </A> \end{rawhtml}
149 .
150 It is important to note that the CVS aliases in Table
151 \ref{tab:cvsModules} cannot be used in conjunction with the CVS
152 \texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
153 they create can be changed to a different name following the check-out:
154 \begin{verbatim}
155 % cvs co -P MITgcm_verif_basic
156 % mv MITgcm MITgcm_verif_basic
157 \end{verbatim}
158
159 \subsubsection{Upgrading from an earlier version}
160
161 If you already have an earlier version of the code you can ``upgrade''
162 your copy instead of downloading the entire repository again. First,
163 ``cd'' (change directory) to the top of your working copy:
164 \begin{verbatim}
165 % cd MITgcm
166 \end{verbatim}
167 and then issue the cvs update command such as:
168 \begin{verbatim}
169 % cvs -q update -d -P -r checkpoint52i_post
170 \end{verbatim}
171 This will update the ``tag'' to ``checkpoint52i\_post'', add any new
172 directories (-d) and remove any empty directories (-P). The -q option
173 means be quiet which will reduce the number of messages you'll see in
174 the terminal. If you have modified the code prior to upgrading, CVS
175 will try to merge your changes with the upgrades. If there is a
176 conflict between your modifications and the upgrade, it will report
177 that file with a ``C'' in front, e.g.:
178 \begin{verbatim}
179 C model/src/ini_parms.F
180 \end{verbatim}
181 If the list of conflicts scrolled off the screen, you can re-issue the
182 cvs update command and it will report the conflicts. Conflicts are
183 indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
184 ``$>>>>>>>$''. For example,
185 {\small
186 \begin{verbatim}
187 <<<<<<< ini_parms.F
188 & bottomDragLinear,myOwnBottomDragCoefficient,
189 =======
190 & bottomDragLinear,bottomDragQuadratic,
191 >>>>>>> 1.18
192 \end{verbatim}
193 }
194 means that you added ``myOwnBottomDragCoefficient'' to a namelist at
195 the same time and place that we added ``bottomDragQuadratic''. You
196 need to resolve this conflict and in this case the line should be
197 changed to:
198 {\small
199 \begin{verbatim}
200 & bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
201 \end{verbatim}
202 }
203 and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
204 Unless you are making modifications which exactly parallel
205 developments we make, these types of conflicts should be rare.
206
207 \paragraph*{Upgrading to the current pre-release version}
208
209 We don't make a ``release'' for every little patch and bug fix in
210 order to keep the frequency of upgrades to a minimum. However, if you
211 have run into a problem for which ``we have already fixed in the
212 latest code'' and we haven't made a ``tag'' or ``release'' since that
213 patch then you'll need to get the latest code:
214 \begin{verbatim}
215 % cvs -q update -d -P -A
216 \end{verbatim}
217 Unlike, the ``check-out'' and ``update'' procedures above, there is no
218 ``tag'' or release name. The -A tells CVS to upgrade to the
219 very latest version. As a rule, we don't recommend this since you
220 might upgrade while we are in the processes of checking in the code so
221 that you may only have part of a patch. Using this method of updating
222 also means we can't tell what version of the code you are working
223 with. So please be sure you understand what you're doing.
224
225 \subsection{Method 2 - Tar file download}
226 \label{sec:conventionalDownload}
227
228 If you do not have CVS on your system, you can download the model as a
229 tar file from the web site at:
230 \begin{rawhtml} <A href=http://mitgcm.org/download/ target="idontexist"> \end{rawhtml}
231 \begin{verbatim}
232 http://mitgcm.org/download/
233 \end{verbatim}
234 \begin{rawhtml} </A> \end{rawhtml}
235 The tar file still contains CVS information which we urge you not to
236 delete; even if you do not use CVS yourself the information can help
237 us if you should need to send us your copy of the code. If a recent
238 tar file does not exist, then please contact the developers through
239 the
240 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
241 MITgcm-support@mitgcm.org
242 \begin{rawhtml} </A> \end{rawhtml}
243 mailing list.
244
245 \section{Model and directory structure}
246 \begin{rawhtml}
247 <!-- CMIREDIR:directory_structure: -->
248 \end{rawhtml}
249
250 The ``numerical'' model is contained within a execution environment
251 support wrapper. This wrapper is designed to provide a general
252 framework for grid-point models. MITgcmUV is a specific numerical
253 model that uses the framework. Under this structure the model is split
254 into execution environment support code and conventional numerical
255 model code. The execution environment support code is held under the
256 \texttt{eesupp} directory. The grid point model code is held under the
257 \texttt{model} directory. Code execution actually starts in the
258 \texttt{eesupp} routines and not in the \texttt{model} routines. For
259 this reason the top-level \texttt{MAIN.F} is in the
260 \texttt{eesupp/src} directory. In general, end-users should not need
261 to worry about this level. The top-level routine for the numerical
262 part of the code is in \texttt{model/src/THE\_MODEL\_MAIN.F}. Here is
263 a brief description of the directory structure of the model under the
264 root tree (a detailed description is given in section 3: Code
265 structure).
266
267 \begin{itemize}
268
269 \item \texttt{doc}: contains brief documentation notes.
270
271 \item \texttt{eesupp}: contains the execution environment source code.
272 Also subdivided into two subdirectories \texttt{inc} and
273 \texttt{src}.
274
275 \item \texttt{model}: this directory contains the main source code.
276 Also subdivided into two subdirectories \texttt{inc} and
277 \texttt{src}.
278
279 \item \texttt{pkg}: contains the source code for the packages. Each
280 package corresponds to a subdirectory. For example, \texttt{gmredi}
281 contains the code related to the Gent-McWilliams/Redi scheme,
282 \texttt{aim} the code relative to the atmospheric intermediate
283 physics. The packages are described in detail in chapter \ref{chap:packagesI}.
284
285 \item \texttt{tools}: this directory contains various useful tools.
286 For example, \texttt{genmake2} is a script written in csh (C-shell)
287 that should be used to generate your makefile. The directory
288 \texttt{adjoint} contains the makefile specific to the Tangent
289 linear and Adjoint Compiler (TAMC) that generates the adjoint code.
290 The latter is described in detail in part \ref{chap.ecco}.
291 This directory also contains the subdirectory build\_options, which
292 contains the `optfiles' with the compiler options for the different
293 compilers and machines that can run MITgcm.
294
295 \item \texttt{utils}: this directory contains various utilities. The
296 subdirectory \texttt{knudsen2} contains code and a makefile that
297 compute coefficients of the polynomial approximation to the knudsen
298 formula for an ocean nonlinear equation of state. The
299 \texttt{matlab} subdirectory contains matlab scripts for reading
300 model output directly into matlab. \texttt{scripts} contains C-shell
301 post-processing scripts for joining processor-based and tiled-based
302 model output. The subdirectory exch2 contains the code needed for
303 the exch2 package to work with different combinations of domain
304 decompositions.
305
306 \item \texttt{verification}: this directory contains the model
307 examples. See section \ref{sec:modelExamples}.
308
309 \item \texttt{jobs}: contains sample job scripts for running MITgcm.
310
311 \item \texttt{lsopt}: Line search code used for optimization.
312
313 \item \texttt{optim}: Interface between MITgcm and line search code.
314
315 \end{itemize}
316
317 \section[Building MITgcm]{Building the code}
318 \label{sec:buildingCode}
319 \begin{rawhtml}
320 <!-- CMIREDIR:buildingCode: -->
321 \end{rawhtml}
322
323 To compile the code, we use the \texttt{make} program. This uses a
324 file (\texttt{Makefile}) that allows us to pre-process source files,
325 specify compiler and optimization options and also figures out any
326 file dependencies. We supply a script (\texttt{genmake2}), described
327 in section \ref{sec:genmake}, that automatically creates the
328 \texttt{Makefile} for you. You then need to build the dependencies and
329 compile the code.
330
331 As an example, assume that you want to build and run experiment
332 \texttt{verification/exp2}. The are multiple ways and places to
333 actually do this but here let's build the code in
334 \texttt{verification/exp2/build}:
335 \begin{verbatim}
336 % cd verification/exp2/build
337 \end{verbatim}
338 First, build the \texttt{Makefile}:
339 \begin{verbatim}
340 % ../../../tools/genmake2 -mods=../code
341 \end{verbatim}
342 The command line option tells \texttt{genmake} to override model source
343 code with any files in the directory \texttt{../code/}.
344
345 On many systems, the \texttt{genmake2} program will be able to
346 automatically recognize the hardware, find compilers and other tools
347 within the user's path (``\texttt{echo \$PATH}''), and then choose an
348 appropriate set of options from the files (``optfiles'') contained in
349 the \texttt{tools/build\_options} directory. Under some
350 circumstances, a user may have to create a new ``optfile'' in order to
351 specify the exact combination of compiler, compiler flags, libraries,
352 and other options necessary to build a particular configuration of
353 MITgcm. In such cases, it is generally helpful to read the existing
354 ``optfiles'' and mimic their syntax.
355
356 Through the MITgcm-support list, the MITgcm developers are willing to
357 provide help writing or modifing ``optfiles''. And we encourage users
358 to post new ``optfiles'' (particularly ones for new machines or
359 architectures) to the
360 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
361 MITgcm-support@mitgcm.org
362 \begin{rawhtml} </A> \end{rawhtml}
363 list.
364
365 To specify an optfile to \texttt{genmake2}, the syntax is:
366 \begin{verbatim}
367 % ../../../tools/genmake2 -mods=../code -of /path/to/optfile
368 \end{verbatim}
369
370 Once a \texttt{Makefile} has been generated, we create the
371 dependencies with the command:
372 \begin{verbatim}
373 % make depend
374 \end{verbatim}
375 This modifies the \texttt{Makefile} by attaching a (usually, long)
376 list of files upon which other files depend. The purpose of this is to
377 reduce re-compilation if and when you start to modify the code. The
378 {\tt make depend} command also creates links from the model source to
379 this directory. It is important to note that the {\tt make depend}
380 stage will occasionally produce warnings or errors since the
381 dependency parsing tool is unable to find all of the necessary header
382 files (\textit{eg.} \texttt{netcdf.inc}). In these circumstances, it
383 is usually OK to ignore the warnings/errors and proceed to the next
384 step.
385
386 Next one can compile the code using:
387 \begin{verbatim}
388 % make
389 \end{verbatim}
390 The {\tt make} command creates an executable called \texttt{mitgcmuv}.
391 Additional make ``targets'' are defined within the makefile to aid in
392 the production of adjoint and other versions of MITgcm. On SMP
393 (shared multi-processor) systems, the build process can often be sped
394 up appreciably using the command:
395 \begin{verbatim}
396 % make -j 2
397 \end{verbatim}
398 where the ``2'' can be replaced with a number that corresponds to the
399 number of CPUs available.
400
401 Now you are ready to run the model. General instructions for doing so are
402 given in section \ref{sec:runModel}. Here, we can run the model by
403 first creating links to all the input files:
404 \begin{verbatim}
405 ln -s ../input/* .
406 \end{verbatim}
407 and then calling the executable with:
408 \begin{verbatim}
409 ./mitgcmuv > output.txt
410 \end{verbatim}
411 where we are re-directing the stream of text output to the file
412 \texttt{output.txt}.
413
414 \subsection{Building/compiling the code elsewhere}
415
416 In the example above (section \ref{sec:buildingCode}) we built the
417 executable in the {\em input} directory of the experiment for
418 convenience. You can also configure and compile the code in other
419 locations, for example on a scratch disk with out having to copy the
420 entire source tree. The only requirement to do so is you have {\tt
421 genmake2} in your path or you know the absolute path to {\tt
422 genmake2}.
423
424 The following sections outline some possible methods of organizing
425 your source and data.
426
427 \subsubsection{Building from the {\em ../code directory}}
428
429 This is just as simple as building in the {\em input/} directory:
430 \begin{verbatim}
431 % cd verification/exp2/code
432 % ../../../tools/genmake2
433 % make depend
434 % make
435 \end{verbatim}
436 However, to run the model the executable ({\em mitgcmuv}) and input
437 files must be in the same place. If you only have one calculation to make:
438 \begin{verbatim}
439 % cd ../input
440 % cp ../code/mitgcmuv ./
441 % ./mitgcmuv > output.txt
442 \end{verbatim}
443 or if you will be making multiple runs with the same executable:
444 \begin{verbatim}
445 % cd ../
446 % cp -r input run1
447 % cp code/mitgcmuv run1
448 % cd run1
449 % ./mitgcmuv > output.txt
450 \end{verbatim}
451
452 \subsubsection{Building from a new directory}
453
454 Since the {\em input} directory contains input files it is often more
455 useful to keep {\em input} pristine and build in a new directory
456 within {\em verification/exp2/}:
457 \begin{verbatim}
458 % cd verification/exp2
459 % mkdir build
460 % cd build
461 % ../../../tools/genmake2 -mods=../code
462 % make depend
463 % make
464 \end{verbatim}
465 This builds the code exactly as before but this time you need to copy
466 either the executable or the input files or both in order to run the
467 model. For example,
468 \begin{verbatim}
469 % cp ../input/* ./
470 % ./mitgcmuv > output.txt
471 \end{verbatim}
472 or if you tend to make multiple runs with the same executable then
473 running in a new directory each time might be more appropriate:
474 \begin{verbatim}
475 % cd ../
476 % mkdir run1
477 % cp build/mitgcmuv run1/
478 % cp input/* run1/
479 % cd run1
480 % ./mitgcmuv > output.txt
481 \end{verbatim}
482
483 \subsubsection{Building on a scratch disk}
484
485 Model object files and output data can use up large amounts of disk
486 space so it is often the case that you will be operating on a large
487 scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
488 following commands will build the model in {\em /scratch/exp2-run1}:
489 \begin{verbatim}
490 % cd /scratch/exp2-run1
491 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
492 -mods=~/MITgcm/verification/exp2/code
493 % make depend
494 % make
495 \end{verbatim}
496 To run the model here, you'll need the input files:
497 \begin{verbatim}
498 % cp ~/MITgcm/verification/exp2/input/* ./
499 % ./mitgcmuv > output.txt
500 \end{verbatim}
501
502 As before, you could build in one directory and make multiple runs of
503 the one experiment:
504 \begin{verbatim}
505 % cd /scratch/exp2
506 % mkdir build
507 % cd build
508 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
509 -mods=~/MITgcm/verification/exp2/code
510 % make depend
511 % make
512 % cd ../
513 % cp -r ~/MITgcm/verification/exp2/input run2
514 % cd run2
515 % ./mitgcmuv > output.txt
516 \end{verbatim}
517
518
519 \subsection{Using \texttt{genmake2}}
520 \label{sec:genmake}
521
522 To compile the code, first use the program \texttt{genmake2} (located
523 in the \texttt{tools} directory) to generate a Makefile.
524 \texttt{genmake2} is a shell script written to work with all
525 ``sh''--compatible shells including bash v1, bash v2, and Bourne.
526 %Internally, \texttt{genmake2} determines the locations of needed
527 %files, the compiler, compiler options, libraries, and Unix tools. It
528 %relies upon a number of ``optfiles'' located in the
529 %\texttt{tools/build\_options} directory.
530 \texttt{genmake2} parses information from the following sources:
531 \begin{description}
532 \item[-] a {\em gemake\_local} file if one is found in the current
533 directory
534 \item[-] command-line options
535 \item[-] an "options file" as specified by the command-line option
536 \texttt{--optfile=/PATH/FILENAME}
537 \item[-] a {\em packages.conf} file (if one is found) with the
538 specific list of packages to compile. The search path for
539 file {\em packages.conf} is, first, the current directory and
540 then each of the "MODS" directories in the given order (see below).
541 \end{description}
542
543 \subsubsection{Optfiles in \texttt{tools/build\_options} directory:}
544
545 The purpose of the optfiles is to provide all the compilation options
546 for particular ``platforms'' (where ``platform'' roughly means the
547 combination of the hardware and the compiler) and code configurations.
548 Given the combinations of possible compilers and library dependencies
549 ({\it eg.} MPI and NetCDF) there may be numerous optfiles available
550 for a single machine. The naming scheme for the majority of the
551 optfiles shipped with the code is
552 \begin{center}
553 {\bf OS\_HARDWARE\_COMPILER }
554 \end{center}
555 where
556 \begin{description}
557 \item[OS] is the name of the operating system (generally the
558 lower-case output of the {\tt 'uname'} command)
559 \item[HARDWARE] is a string that describes the CPU type and
560 corresponds to output from the {\tt 'uname -m'} command:
561 \begin{description}
562 \item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
563 and athlon
564 \item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
565 \item[amd64] is AMD x86\_64 systems
566 \item[ppc] is for Mac PowerPC systems
567 \end{description}
568 \item[COMPILER] is the compiler name (generally, the name of the
569 FORTRAN executable)
570 \end{description}
571
572 In many cases, the default optfiles are sufficient and will result in
573 usable Makefiles. However, for some machines or code configurations,
574 new ``optfiles'' must be written. To create a new optfile, it is
575 generally best to start with one of the defaults and modify it to suit
576 your needs. Like \texttt{genmake2}, the optfiles are all written
577 using a simple ``sh''--compatible syntax. While nearly all variables
578 used within \texttt{genmake2} may be specified in the optfiles, the
579 critical ones that should be defined are:
580
581 \begin{description}
582 \item[FC] the FORTRAN compiler (executable) to use
583 \item[DEFINES] the command-line DEFINE options passed to the compiler
584 \item[CPP] the C pre-processor to use
585 \item[NOOPTFLAGS] options flags for special files that should not be
586 optimized
587 \end{description}
588
589 For example, the optfile for a typical Red Hat Linux machine (``ia32''
590 architecture) using the GCC (g77) compiler is
591 \begin{verbatim}
592 FC=g77
593 DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
594 CPP='cpp -traditional -P'
595 NOOPTFLAGS='-O0'
596 # For IEEE, use the "-ffloat-store" option
597 if test "x$IEEE" = x ; then
598 FFLAGS='-Wimplicit -Wunused -Wuninitialized'
599 FOPTIM='-O3 -malign-double -funroll-loops'
600 else
601 FFLAGS='-Wimplicit -Wunused -ffloat-store'
602 FOPTIM='-O0 -malign-double'
603 fi
604 \end{verbatim}
605
606 If you write an optfile for an unrepresented machine or compiler, you
607 are strongly encouraged to submit the optfile to the MITgcm project
608 for inclusion. Please send the file to the
609 \begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
610 \begin{center}
611 MITgcm-support@mitgcm.org
612 \end{center}
613 \begin{rawhtml} </A> \end{rawhtml}
614 mailing list.
615
616 \subsubsection{Command-line options:}
617
618 In addition to the optfiles, \texttt{genmake2} supports a number of
619 helpful command-line options. A complete list of these options can be
620 obtained from:
621 \begin{verbatim}
622 % genmake2 -h
623 \end{verbatim}
624
625 The most important command-line options are:
626 \begin{description}
627
628 \item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
629 should be used for a particular build.
630
631 If no "optfile" is specified (either through the command line or the
632 MITGCM\_OPTFILE environment variable), genmake2 will try to make a
633 reasonable guess from the list provided in {\em
634 tools/build\_options}. The method used for making this guess is
635 to first determine the combination of operating system and hardware
636 (eg. "linux\_ia32") and then find a working FORTRAN compiler within
637 the user's path. When these three items have been identified,
638 genmake2 will try to find an optfile that has a matching name.
639
640 \item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
641 directories containing ``modifications''. These directories contain
642 files with names that may (or may not) exist in the main MITgcm
643 source tree but will be overridden by any identically-named sources
644 within the ``MODS'' directories.
645
646 The order of precedence for this "name-hiding" is as follows:
647 \begin{itemize}
648 \item ``MODS'' directories (in the order given)
649 \item Packages either explicitly specified or provided by default
650 (in the order given)
651 \item Packages included due to package dependencies (in the order
652 that that package dependencies are parsed)
653 \item The "standard dirs" (which may have been specified by the
654 ``-standarddirs'' option)
655 \end{itemize}
656
657 \item[\texttt{--pgroups=/PATH/FILENAME}] specifies the file
658 where package groups are defined. If not set, the package-groups
659 definition will be read from {\em pkg/pkg\_groups}.
660 It also contains the default list of packages (defined
661 as the group ``{\it default\_pkg\_list}'' which is used
662 when no specific package list ({\em packages.conf})
663 is found in current directory or in any "MODS" directory.
664
665 \item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
666 used for packages.
667
668 If not specified, the default dependency file {\em pkg/pkg\_depend}
669 is used. The syntax for this file is parsed on a line-by-line basis
670 where each line containes either a comment ("\#") or a simple
671 "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
672 specifies a "must be used with" or a "must not be used with"
673 relationship, respectively. If no rule is specified, then it is
674 assumed that the two packages are compatible and will function
675 either with or without each other.
676
677 \item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
678 automatic differentiation options file to be used. The file is
679 analogous to the ``optfile'' defined above but it specifies
680 information for the AD build process.
681
682 The default file is located in {\em
683 tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
684 and "TAMC" compilers. An alternate version is also available at
685 {\em tools/adjoint\_options/adjoint\_staf} that selects the newer
686 "STAF" compiler. As with any compilers, it is helpful to have their
687 directories listed in your {\tt \$PATH} environment variable.
688
689 \item[\texttt{--mpi}] This option enables certain MPI features (using
690 CPP \texttt{\#define}s) within the code and is necessary for MPI
691 builds (see Section \ref{sec:mpi-build}).
692
693 \item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
694 soft-links and other bugs common with the \texttt{make} versions
695 provided by commercial Unix vendors, GNU \texttt{make} (sometimes
696 called \texttt{gmake}) should be preferred. This option provides a
697 means for specifying the make executable to be used.
698
699 \item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
700 machines, the ``bash'' shell is unavailable. To run on these
701 systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
702 a Bourne, POSIX, or compatible) shell. The syntax in these
703 circumstances is:
704 \begin{center}
705 \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
706 \end{center}
707 where \texttt{/bin/sh} can be replaced with the full path and name
708 of the desired shell.
709
710 \end{description}
711
712
713 \subsection{Building with MPI}
714 \label{sec:mpi-build}
715
716 Building MITgcm to use MPI libraries can be complicated due to the
717 variety of different MPI implementations available, their dependencies
718 or interactions with different compilers, and their often ad-hoc
719 locations within file systems. For these reasons, its generally a
720 good idea to start by finding and reading the documentation for your
721 machine(s) and, if necessary, seeking help from your local systems
722 administrator.
723
724 The steps for building MITgcm with MPI support are:
725 \begin{enumerate}
726
727 \item Determine the locations of your MPI-enabled compiler and/or MPI
728 libraries and put them into an options file as described in Section
729 \ref{sec:genmake}. One can start with one of the examples in:
730 \begin{rawhtml} <A
731 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/build_options/">
732 \end{rawhtml}
733 \begin{center}
734 \texttt{MITgcm/tools/build\_options/}
735 \end{center}
736 \begin{rawhtml} </A> \end{rawhtml}
737 such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
738 \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
739 hand. You may need help from your user guide or local systems
740 administrator to determine the exact location of the MPI libraries.
741 If libraries are not installed, MPI implementations and related
742 tools are available including:
743 \begin{itemize}
744 \item \begin{rawhtml} <A
745 href="http://www-unix.mcs.anl.gov/mpi/mpich/">
746 \end{rawhtml}
747 MPICH
748 \begin{rawhtml} </A> \end{rawhtml}
749
750 \item \begin{rawhtml} <A
751 href="http://www.lam-mpi.org/">
752 \end{rawhtml}
753 LAM/MPI
754 \begin{rawhtml} </A> \end{rawhtml}
755
756 \item \begin{rawhtml} <A
757 href="http://www.osc.edu/~pw/mpiexec/">
758 \end{rawhtml}
759 MPIexec
760 \begin{rawhtml} </A> \end{rawhtml}
761 \end{itemize}
762
763 \item Build the code with the \texttt{genmake2} \texttt{-mpi} option
764 (see Section \ref{sec:genmake}) using commands such as:
765 {\footnotesize \begin{verbatim}
766 % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
767 % make depend
768 % make
769 \end{verbatim} }
770
771 \item Run the code with the appropriate MPI ``run'' or ``exec''
772 program provided with your particular implementation of MPI.
773 Typical MPI packages such as MPICH will use something like:
774 \begin{verbatim}
775 % mpirun -np 4 -machinefile mf ./mitgcmuv
776 \end{verbatim}
777 Sightly more complicated scripts may be needed for many machines
778 since execution of the code may be controlled by both the MPI
779 library and a job scheduling and queueing system such as PBS,
780 LoadLeveller, Condor, or any of a number of similar tools. A few
781 example scripts (those used for our \begin{rawhtml} <A
782 href="http://mitgcm.org/public/testing.html"> \end{rawhtml}regular
783 verification runs\begin{rawhtml} </A> \end{rawhtml}) are available
784 at:
785 \begin{rawhtml} <A
786 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
787 \end{rawhtml}
788 {\footnotesize \tt
789 http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example\_scripts/ }
790 \begin{rawhtml} </A> \end{rawhtml}
791 or at:
792 \begin{rawhtml} <A
793 href="http://mitgcm.org/viewvc/MITgcm/MITgcm_contrib/test_scripts/">
794 \end{rawhtml}
795 {\footnotesize \tt
796 http://mitgcm.org/viewvc/MITgcm/MITgcm\_contrib/test\_scripts/ }
797 \begin{rawhtml} </A> \end{rawhtml}
798
799 \end{enumerate}
800
801 An example of the above process on the MITgcm cluster (``cg01'') using
802 the GNU g77 compiler and the mpich MPI library is:
803
804 {\footnotesize \begin{verbatim}
805 % cd MITgcm/verification/exp5
806 % mkdir build
807 % cd build
808 % ../../../tools/genmake2 -mpi -mods=../code \
809 -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
810 % make depend
811 % make
812 % cd ../input
813 % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
814 -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
815 \end{verbatim} }
816
817 \section[Running MITgcm]{Running the model in prognostic mode}
818 \label{sec:runModel}
819 \begin{rawhtml}
820 <!-- CMIREDIR:runModel: -->
821 \end{rawhtml}
822
823 If compilation finished succesfully (section \ref{sec:buildingCode})
824 then an executable called \texttt{mitgcmuv} will now exist in the
825 local directory.
826
827 To run the model as a single process (\textit{ie.} not in parallel)
828 simply type:
829 \begin{verbatim}
830 % ./mitgcmuv
831 \end{verbatim}
832 The ``./'' is a safe-guard to make sure you use the local executable
833 in case you have others that exist in your path (surely odd if you
834 do!). The above command will spew out many lines of text output to
835 your screen. This output contains details such as parameter values as
836 well as diagnostics such as mean Kinetic energy, largest CFL number,
837 etc. It is worth keeping this text output with the binary output so we
838 normally re-direct the \texttt{stdout} stream as follows:
839 \begin{verbatim}
840 % ./mitgcmuv > output.txt
841 \end{verbatim}
842 In the event that the model encounters an error and stops, it is very
843 helpful to include the last few line of this \texttt{output.txt} file
844 along with the (\texttt{stderr}) error message within any bug reports.
845
846 For the example experiments in \texttt{verification}, an example of the
847 output is kept in \texttt{results/output.txt} for comparison. You can
848 compare your \texttt{output.txt} with the corresponding one for that
849 experiment to check that the set-up works.
850
851
852
853 \subsection{Output files}
854
855 The model produces various output files and, when using \texttt{mnc},
856 sometimes even directories. Depending upon the I/O package(s)
857 selected at compile time (either \texttt{mdsio} or \texttt{mnc} or
858 both as determined by \texttt{code/packages.conf}) and the run-time
859 flags set (in \texttt{input/data.pkg}), the following output may
860 appear.
861
862
863 \subsubsection{MDSIO output files}
864
865 The ``traditional'' output files are generated by the \texttt{mdsio}
866 package. At a minimum, the instantaneous ``state'' of the model is
867 written out, which is made of the following files:
868
869 \begin{itemize}
870 \item \texttt{U.00000nIter} - zonal component of velocity field (m/s
871 and positive eastward).
872
873 \item \texttt{V.00000nIter} - meridional component of velocity field
874 (m/s and positive northward).
875
876 \item \texttt{W.00000nIter} - vertical component of velocity field
877 (ocean: m/s and positive upward, atmosphere: Pa/s and positive
878 towards increasing pressure i.e. downward).
879
880 \item \texttt{T.00000nIter} - potential temperature (ocean:
881 $^{\circ}\mathrm{C}$, atmosphere: $^{\circ}\mathrm{K}$).
882
883 \item \texttt{S.00000nIter} - ocean: salinity (psu), atmosphere: water
884 vapor (g/kg).
885
886 \item \texttt{Eta.00000nIter} - ocean: surface elevation (m),
887 atmosphere: surface pressure anomaly (Pa).
888 \end{itemize}
889
890 The chain \texttt{00000nIter} consists of ten figures that specify the
891 iteration number at which the output is written out. For example,
892 \texttt{U.0000000300} is the zonal velocity at iteration 300.
893
894 In addition, a ``pickup'' or ``checkpoint'' file called:
895
896 \begin{itemize}
897 \item \texttt{pickup.00000nIter}
898 \end{itemize}
899
900 is written out. This file represents the state of the model in a condensed
901 form and is used for restarting the integration. If the C-D scheme is used,
902 there is an additional ``pickup'' file:
903
904 \begin{itemize}
905 \item \texttt{pickup\_cd.00000nIter}
906 \end{itemize}
907
908 containing the D-grid velocity data and that has to be written out as well
909 in order to restart the integration. Rolling checkpoint files are the same
910 as the pickup files but are named differently. Their name contain the chain
911 \texttt{ckptA} or \texttt{ckptB} instead of \texttt{00000nIter}. They can be
912 used to restart the model but are overwritten every other time they are
913 output to save disk space during long integrations.
914
915 \subsubsection{MNC output files}
916
917 Unlike the \texttt{mdsio} output, the \texttt{mnc}--generated output
918 is usually (though not necessarily) placed within a subdirectory with
919 a name such as \texttt{mnc\_test\_\${DATE}\_\${SEQ}}.
920
921 \subsection{Looking at the output}
922
923 The ``traditional'' or mdsio model data are written according to a
924 ``meta/data'' file format. Each variable is associated with two files
925 with suffix names \texttt{.data} and \texttt{.meta}. The
926 \texttt{.data} file contains the data written in binary form
927 (big\_endian by default). The \texttt{.meta} file is a ``header'' file
928 that contains information about the size and the structure of the
929 \texttt{.data} file. This way of organizing the output is particularly
930 useful when running multi-processors calculations. The base version of
931 the model includes a few matlab utilities to read output files written
932 in this format. The matlab scripts are located in the directory
933 \texttt{utils/matlab} under the root tree. The script \texttt{rdmds.m}
934 reads the data. Look at the comments inside the script to see how to
935 use it.
936
937 Some examples of reading and visualizing some output in {\em Matlab}:
938 \begin{verbatim}
939 % matlab
940 >> H=rdmds('Depth');
941 >> contourf(H');colorbar;
942 >> title('Depth of fluid as used by model');
943
944 >> eta=rdmds('Eta',10);
945 >> imagesc(eta');axis ij;colorbar;
946 >> title('Surface height at iter=10');
947
948 >> eta=rdmds('Eta',[0:10:100]);
949 >> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
950 \end{verbatim}
951
952 Similar scripts for netCDF output (\texttt{rdmnc.m}) are available and
953 they are described in Section \ref{sec:pkg:mnc}.
954
955 The MNC output files are all in the ``self-describing'' netCDF
956 format and can thus be browsed and/or plotted using tools such as:
957 \begin{itemize}
958 \item \texttt{ncdump} is a utility which is typically included
959 with every netCDF install:
960 \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
961 \begin{verbatim}
962 http://www.unidata.ucar.edu/packages/netcdf/
963 \end{verbatim}
964 \begin{rawhtml} </A> \end{rawhtml} and it converts the netCDF
965 binaries into formatted ASCII text files.
966
967 \item \texttt{ncview} utility is a very convenient and quick way
968 to plot netCDF data and it runs on most OSes:
969 \begin{rawhtml} <A href="http://meteora.ucsd.edu/~pierce/ncview_home_page.html"> \end{rawhtml}
970 \begin{verbatim}
971 http://meteora.ucsd.edu/~pierce/ncview_home_page.html
972 \end{verbatim}
973 \begin{rawhtml} </A> \end{rawhtml}
974
975 \item MatLAB(c) and other common post-processing environments provide
976 various netCDF interfaces including:
977 \begin{rawhtml} <A href="http://mexcdf.sourceforge.net/"> \end{rawhtml}
978 \begin{verbatim}
979 http://mexcdf.sourceforge.net/
980 \end{verbatim}
981 \begin{rawhtml} </A> \end{rawhtml}
982 \begin{rawhtml} <A href="http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html"> \end{rawhtml}
983 \begin{verbatim}
984 http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html
985 \end{verbatim}
986 \begin{rawhtml} </A> \end{rawhtml}
987 \end{itemize}
988

  ViewVC Help
Powered by ViewVC 1.1.22