/[MITgcm]/manual/s_getstarted/text/getting_started.tex
ViewVC logotype

Contents of /manual/s_getstarted/text/getting_started.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.46 - (show annotations) (download) (as text)
Sat Nov 21 03:19:54 2015 UTC (9 years, 7 months ago) by dimitri
Branch: MAIN
Changes since 1.45: +9 -3 lines
File MIME type: application/x-tex
adding 1-line, no-login, no-shell-variable CVS command

1 % $Header: /u/gcmpack/manual/s_getstarted/text/getting_started.tex,v 1.45 2011/05/11 18:58:02 jmc Exp $
2 % $Name: $
3
4 %\section{Getting started}
5
6 We believe the best way to familiarize yourself with the
7 model is to run the case study examples provided with the base
8 version. Information on how to obtain, compile, and run the code is
9 found here as well as a brief description of the model structure
10 directory and the case study examples. Information is also provided
11 here on how to customize the code when you are ready to try implementing
12 the configuration you have in mind. The code and algorithm
13 are described more fully in chapters \ref{chap:discretization} and
14 \ref{chap:sarch}.
15
16 \section{Where to find information}
17 \label{sec:whereToFindInfo}
18 \begin{rawhtml}
19 <!-- CMIREDIR:whereToFindInfo: -->
20 \end{rawhtml}
21
22 There is a web-archived support mailing list for the model that
23 you can email at \texttt{MITgcm-support@mitgcm.org} or browse at:
24 \begin{rawhtml} <A href=http://mitgcm.org/mailman/listinfo/mitgcm-support/ target="idontexist"> \end{rawhtml}
25 \begin{verbatim}
26 http://mitgcm.org/mailman/listinfo/mitgcm-support/
27 http://mitgcm.org/pipermail/mitgcm-support/
28 \end{verbatim}
29 \begin{rawhtml} </A> \end{rawhtml}
30
31 \section{Obtaining the code}
32 \label{sec:obtainingCode}
33 \begin{rawhtml}
34 <!-- CMIREDIR:obtainingCode: -->
35 \end{rawhtml}
36
37 MITgcm can be downloaded from our system by following
38 the instructions below. As a courtesy we ask that you send e-mail to us at
39 \begin{rawhtml} <A href=mailto:MITgcm-support@mitgcm.org> \end{rawhtml}
40 MITgcm-support@mitgcm.org
41 \begin{rawhtml} </A> \end{rawhtml}
42 to enable us to keep track of who's using the model and in what application.
43 You can download the model two ways:
44
45 \begin{enumerate}
46 \item Using CVS software. CVS is a freely available source code management
47 tool. To use CVS you need to have the software installed. Many systems
48 come with CVS pre-installed, otherwise good places to look for
49 the software for a particular platform are
50 \begin{rawhtml} <A href=http://www.cvshome.org/ target="idontexist"> \end{rawhtml}
51 cvshome.org
52 \begin{rawhtml} </A> \end{rawhtml}
53 and
54 \begin{rawhtml} <A href=http://www.wincvs.org/ target="idontexist"> \end{rawhtml}
55 wincvs.org
56 \begin{rawhtml} </A> \end{rawhtml}
57 .
58
59 \item Using a tar file. This method is simple and does not
60 require any special software. However, this method does not
61 provide easy support for maintenance updates.
62
63 \end{enumerate}
64
65 \subsection{Method 1 - Checkout from CVS}
66 \label{sec:cvs_checkout}
67
68 If CVS is available on your system, we strongly encourage you to use it. CVS
69 provides an efficient and elegant way of organizing your code and keeping
70 track of your changes. If CVS is not available on your machine, you can also
71 download a tar file.
72
73 Before you can use CVS, the following environment variable(s) should
74 be set within your shell. For a csh or tcsh shell, put the following
75 \begin{verbatim}
76 % setenv CVSROOT :pserver:cvsanon@mitgcm.org:/u/gcmpack
77 \end{verbatim}
78 in your \texttt{.cshrc} or \texttt{.tcshrc} file. For bash or sh
79 shells, put:
80 \begin{verbatim}
81 % export CVSROOT=':pserver:cvsanon@mitgcm.org:/u/gcmpack'
82 \end{verbatim}
83 in your \texttt{.profile} or \texttt{.bashrc} file.
84
85
86 To get MITgcm through CVS, first register with the MITgcm CVS server
87 using command:
88 \begin{verbatim}
89 % cvs login ( CVS password: cvsanon )
90 \end{verbatim}
91 You only need to do a ``cvs login'' once.
92
93 To obtain the latest sources type:
94 \begin{verbatim}
95 % cvs co -P MITgcm
96 \end{verbatim}
97 or to get a specific release type:
98 \begin{verbatim}
99 % cvs co -P -r checkpoint52i_post MITgcm
100 \end{verbatim}
101 The CVS command ``\texttt{cvs co}'' is the abreviation of the full-name
102 ``\texttt{cvs checkout}'' command and using the option ``-P'' (\texttt{cvs co -P})
103 will prevent to download unnecessary empty directories.
104
105 The MITgcm web site contains further directions concerning the source
106 code and CVS. It also contains a web interface to our CVS archive so
107 that one may easily view the state of files, revisions, and other
108 development milestones:
109 \begin{rawhtml} <A href="http://mitgcm.org/viewvc/MITgcm/MITgcm/" target="idontexist"> \end{rawhtml}
110 \begin{verbatim}
111 http://mitgcm.org/viewvc/MITgcm/MITgcm/
112 \end{verbatim}
113 \begin{rawhtml} </A> \end{rawhtml}
114
115 As a convenience, the MITgcm CVS server contains aliases which are
116 named subsets of the codebase. These aliases can be especially
117 helpful when used over slow internet connections or on machines with
118 restricted storage space. Table \ref{tab:cvsModules} contains a list
119 of CVS aliases
120 \begin{table}[htb]
121 \centering
122 \begin{tabular}[htb]{|lp{3.25in}|}\hline
123 \textbf{Alias Name} & \textbf{Information (directories) Contained} \\\hline
124 \texttt{MITgcm\_code} & Only the source code -- none of the verification examples. \\
125 \texttt{MITgcm\_verif\_basic}
126 & Source code plus a small set of the verification examples
127 (\texttt{global\_ocean.90x40x15}, \texttt{aim.5l\_cs}, \texttt{hs94.128x64x5},
128 \texttt{front\_relax}, and \texttt{plume\_on\_slope}). \\
129 \texttt{MITgcm\_verif\_atmos} & Source code plus all of the atmospheric examples. \\
130 \texttt{MITgcm\_verif\_ocean} & Source code plus all of the oceanic examples. \\
131 \texttt{MITgcm\_verif\_all} & Source code plus all of the
132 verification examples. \\\hline
133 \end{tabular}
134 \caption{MITgcm CVS Modules}
135 \label{tab:cvsModules}
136 \end{table}
137
138 The checkout process creates a directory called \texttt{MITgcm}. If
139 the directory \texttt{MITgcm} exists this command updates your code
140 based on the repository. Each directory in the source tree contains a
141 directory \texttt{CVS}. This information is required by CVS to keep
142 track of your file versions with respect to the repository. Don't edit
143 the files in \texttt{CVS}! You can also use CVS to download code
144 updates. More extensive information on using CVS for maintaining
145 MITgcm code can be found
146 \begin{rawhtml} <A href="http://mitgcm.org/public/using_cvs.html" target="idontexist"> \end{rawhtml}
147 here
148 \begin{rawhtml} </A> \end{rawhtml}.
149 It is important to note that the CVS aliases in Table
150 \ref{tab:cvsModules} cannot be used in conjunction with the CVS
151 \texttt{-d DIRNAME} option. However, the \texttt{MITgcm} directories
152 they create can be changed to a different name following the check-out:
153 \begin{verbatim}
154 % cvs co -P MITgcm_verif_basic
155 % mv MITgcm MITgcm_verif_basic
156 \end{verbatim}
157
158 Note that it is possible to checkout code without ``cvs login'' and without
159 setting any shell environment variables by specifying the pserver name and
160 password in one line, for example:
161 \begin{verbatim}
162 % cvs -d :pserver:cvsanon:cvsanon@mitgcm.org:/u/gcmpack co -P MITgcm
163 \end{verbatim}
164
165 \subsubsection{Upgrading from an earlier version}
166
167 If you already have an earlier version of the code you can ``upgrade''
168 your copy instead of downloading the entire repository again. First,
169 ``cd'' (change directory) to the top of your working copy:
170 \begin{verbatim}
171 % cd MITgcm
172 \end{verbatim}
173 and then issue the cvs update command such as:
174 \begin{verbatim}
175 % cvs -q update -d -P -r checkpoint52i_post
176 \end{verbatim}
177 This will update the ``tag'' to ``checkpoint52i\_post'', add any new
178 directories (-d) and remove any empty directories (-P). The -q option
179 means be quiet which will reduce the number of messages you'll see in
180 the terminal. If you have modified the code prior to upgrading, CVS
181 will try to merge your changes with the upgrades. If there is a
182 conflict between your modifications and the upgrade, it will report
183 that file with a ``C'' in front, e.g.:
184 \begin{verbatim}
185 C model/src/ini_parms.F
186 \end{verbatim}
187 If the list of conflicts scrolled off the screen, you can re-issue the
188 cvs update command and it will report the conflicts. Conflicts are
189 indicated in the code by the delimites ``$<<<<<<<$'', ``======='' and
190 ``$>>>>>>>$''. For example,
191 {\small
192 \begin{verbatim}
193 <<<<<<< ini_parms.F
194 & bottomDragLinear,myOwnBottomDragCoefficient,
195 =======
196 & bottomDragLinear,bottomDragQuadratic,
197 >>>>>>> 1.18
198 \end{verbatim}
199 }
200 means that you added ``myOwnBottomDragCoefficient'' to a namelist at
201 the same time and place that we added ``bottomDragQuadratic''. You
202 need to resolve this conflict and in this case the line should be
203 changed to:
204 {\small
205 \begin{verbatim}
206 & bottomDragLinear,bottomDragQuadratic,myOwnBottomDragCoefficient,
207 \end{verbatim}
208 }
209 and the lines with the delimiters ($<<<<<<$,======,$>>>>>>$) be deleted.
210 Unless you are making modifications which exactly parallel
211 developments we make, these types of conflicts should be rare.
212
213 \paragraph*{Upgrading to the current pre-release version}
214
215 We don't make a ``release'' for every little patch and bug fix in
216 order to keep the frequency of upgrades to a minimum. However, if you
217 have run into a problem for which ``we have already fixed in the
218 latest code'' and we haven't made a ``tag'' or ``release'' since that
219 patch then you'll need to get the latest code:
220 \begin{verbatim}
221 % cvs -q update -d -P -A
222 \end{verbatim}
223 Unlike, the ``check-out'' and ``update'' procedures above, there is no
224 ``tag'' or release name. The -A tells CVS to upgrade to the
225 very latest version. As a rule, we don't recommend this since you
226 might upgrade while we are in the processes of checking in the code so
227 that you may only have part of a patch. Using this method of updating
228 also means we can't tell what version of the code you are working
229 with. So please be sure you understand what you're doing.
230
231 \subsection{Method 2 - Tar file download}
232 \label{sec:conventionalDownload}
233
234 If you do not have CVS on your system, you can download the model as a
235 tar file from the web site at:
236 \begin{rawhtml} <A href=http://mitgcm.org/download/ target="idontexist"> \end{rawhtml}
237 \begin{verbatim}
238 http://mitgcm.org/download/
239 \end{verbatim}
240 \begin{rawhtml} </A> \end{rawhtml}
241 The tar file still contains CVS information which we urge you not to
242 delete; even if you do not use CVS yourself the information can help
243 us if you should need to send us your copy of the code. If a recent
244 tar file does not exist, then please contact the developers through
245 the
246 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
247 MITgcm-support@mitgcm.org
248 \begin{rawhtml} </A> \end{rawhtml}
249 mailing list.
250
251 \section{Model and directory structure}
252 \begin{rawhtml}
253 <!-- CMIREDIR:directory_structure: -->
254 \end{rawhtml}
255
256 The ``numerical'' model is contained within a execution environment
257 support wrapper. This wrapper is designed to provide a general
258 framework for grid-point models. MITgcmUV is a specific numerical
259 model that uses the framework. Under this structure the model is split
260 into execution environment support code and conventional numerical
261 model code. The execution environment support code is held under the
262 \texttt{eesupp} directory. The grid point model code is held under the
263 \texttt{model} directory. Code execution actually starts in the
264 \texttt{eesupp} routines and not in the \texttt{model} routines. For
265 this reason the top-level \texttt{MAIN.F} is in the
266 \texttt{eesupp/src} directory. In general, end-users should not need
267 to worry about this level. The top-level routine for the numerical
268 part of the code is in \texttt{model/src/THE\_MODEL\_MAIN.F}. Here is
269 a brief description of the directory structure of the model under the
270 root tree (a detailed description is given in section 3: Code
271 structure).
272
273 \begin{itemize}
274
275 \item \texttt{doc}: contains brief documentation notes.
276
277 \item \texttt{eesupp}: contains the execution environment source code.
278 Also subdivided into two subdirectories \texttt{inc} and
279 \texttt{src}.
280
281 \item \texttt{model}: this directory contains the main source code.
282 Also subdivided into two subdirectories \texttt{inc} and
283 \texttt{src}.
284
285 \item \texttt{pkg}: contains the source code for the packages. Each
286 package corresponds to a subdirectory. For example, \texttt{gmredi}
287 contains the code related to the Gent-McWilliams/Redi scheme,
288 \texttt{aim} the code relative to the atmospheric intermediate
289 physics. The packages are described in detail in chapter \ref{chap:packagesI}.
290
291 \item \texttt{tools}: this directory contains various useful tools.
292 For example, \texttt{genmake2} is a script written in csh (C-shell)
293 that should be used to generate your makefile. The directory
294 \texttt{adjoint} contains the makefile specific to the Tangent
295 linear and Adjoint Compiler (TAMC) that generates the adjoint code.
296 The latter is described in detail in part \ref{chap.ecco}.
297 This directory also contains the subdirectory build\_options, which
298 contains the `optfiles' with the compiler options for the different
299 compilers and machines that can run MITgcm.
300
301 \item \texttt{utils}: this directory contains various utilities. The
302 subdirectory \texttt{knudsen2} contains code and a makefile that
303 compute coefficients of the polynomial approximation to the knudsen
304 formula for an ocean nonlinear equation of state. The
305 \texttt{matlab} subdirectory contains matlab scripts for reading
306 model output directly into matlab. \texttt{scripts} contains C-shell
307 post-processing scripts for joining processor-based and tiled-based
308 model output. The subdirectory exch2 contains the code needed for
309 the exch2 package to work with different combinations of domain
310 decompositions.
311
312 \item \texttt{verification}: this directory contains the model
313 examples. See section \ref{sec:modelExamples}.
314
315 \item \texttt{jobs}: contains sample job scripts for running MITgcm.
316
317 \item \texttt{lsopt}: Line search code used for optimization.
318
319 \item \texttt{optim}: Interface between MITgcm and line search code.
320
321 \end{itemize}
322
323 \section[Building MITgcm]{Building the code}
324 \label{sec:buildingCode}
325 \begin{rawhtml}
326 <!-- CMIREDIR:buildingCode: -->
327 \end{rawhtml}
328
329 To compile the code, we use the \texttt{make} program. This uses a
330 file (\texttt{Makefile}) that allows us to pre-process source files,
331 specify compiler and optimization options and also figures out any
332 file dependencies. We supply a script (\texttt{genmake2}), described
333 in section \ref{sec:genmake}, that automatically creates the
334 \texttt{Makefile} for you. You then need to build the dependencies and
335 compile the code.
336
337 As an example, assume that you want to build and run experiment
338 \texttt{verification/exp2}. The are multiple ways and places to
339 actually do this but here let's build the code in
340 \texttt{verification/exp2/build}:
341 \begin{verbatim}
342 % cd verification/exp2/build
343 \end{verbatim}
344 First, build the \texttt{Makefile}:
345 \begin{verbatim}
346 % ../../../tools/genmake2 -mods=../code
347 \end{verbatim}
348 The command line option tells \texttt{genmake} to override model source
349 code with any files in the directory \texttt{../code/}.
350
351 On many systems, the \texttt{genmake2} program will be able to
352 automatically recognize the hardware, find compilers and other tools
353 within the user's path (``\texttt{echo \$PATH}''), and then choose an
354 appropriate set of options from the files (``optfiles'') contained in
355 the \texttt{tools/build\_options} directory. Under some
356 circumstances, a user may have to create a new ``optfile'' in order to
357 specify the exact combination of compiler, compiler flags, libraries,
358 and other options necessary to build a particular configuration of
359 MITgcm. In such cases, it is generally helpful to read the existing
360 ``optfiles'' and mimic their syntax.
361
362 Through the MITgcm-support list, the MITgcm developers are willing to
363 provide help writing or modifing ``optfiles''. And we encourage users
364 to post new ``optfiles'' (particularly ones for new machines or
365 architectures) to the
366 \begin{rawhtml} <A href="mailto:MITgcm-support@mitgcm.org"> \end{rawhtml}
367 MITgcm-support@mitgcm.org
368 \begin{rawhtml} </A> \end{rawhtml}
369 list.
370
371 To specify an optfile to \texttt{genmake2}, the syntax is:
372 \begin{verbatim}
373 % ../../../tools/genmake2 -mods=../code -of /path/to/optfile
374 \end{verbatim}
375
376 Once a \texttt{Makefile} has been generated, we create the
377 dependencies with the command:
378 \begin{verbatim}
379 % make depend
380 \end{verbatim}
381 This modifies the \texttt{Makefile} by attaching a (usually, long)
382 list of files upon which other files depend. The purpose of this is to
383 reduce re-compilation if and when you start to modify the code. The
384 {\tt make depend} command also creates links from the model source to
385 this directory. It is important to note that the {\tt make depend}
386 stage will occasionally produce warnings or errors since the
387 dependency parsing tool is unable to find all of the necessary header
388 files (\textit{eg.} \texttt{netcdf.inc}). In these circumstances, it
389 is usually OK to ignore the warnings/errors and proceed to the next
390 step.
391
392 Next one can compile the code using:
393 \begin{verbatim}
394 % make
395 \end{verbatim}
396 The {\tt make} command creates an executable called \texttt{mitgcmuv}.
397 Additional make ``targets'' are defined within the makefile to aid in
398 the production of adjoint and other versions of MITgcm. On SMP
399 (shared multi-processor) systems, the build process can often be sped
400 up appreciably using the command:
401 \begin{verbatim}
402 % make -j 2
403 \end{verbatim}
404 where the ``2'' can be replaced with a number that corresponds to the
405 number of CPUs available.
406
407 Now you are ready to run the model. General instructions for doing so are
408 given in section \ref{sec:runModel}. Here, we can run the model by
409 first creating links to all the input files:
410 \begin{verbatim}
411 ln -s ../input/* .
412 \end{verbatim}
413 and then calling the executable with:
414 \begin{verbatim}
415 ./mitgcmuv > output.txt
416 \end{verbatim}
417 where we are re-directing the stream of text output to the file
418 \texttt{output.txt}.
419
420 \subsection{Building/compiling the code elsewhere}
421
422 In the example above (section \ref{sec:buildingCode}) we built the
423 executable in the {\em input} directory of the experiment for
424 convenience. You can also configure and compile the code in other
425 locations, for example on a scratch disk with out having to copy the
426 entire source tree. The only requirement to do so is you have {\tt
427 genmake2} in your path or you know the absolute path to {\tt
428 genmake2}.
429
430 The following sections outline some possible methods of organizing
431 your source and data.
432
433 \subsubsection{Building from the {\em ../code directory}}
434
435 This is just as simple as building in the {\em input/} directory:
436 \begin{verbatim}
437 % cd verification/exp2/code
438 % ../../../tools/genmake2
439 % make depend
440 % make
441 \end{verbatim}
442 However, to run the model the executable ({\em mitgcmuv}) and input
443 files must be in the same place. If you only have one calculation to make:
444 \begin{verbatim}
445 % cd ../input
446 % cp ../code/mitgcmuv ./
447 % ./mitgcmuv > output.txt
448 \end{verbatim}
449 or if you will be making multiple runs with the same executable:
450 \begin{verbatim}
451 % cd ../
452 % cp -r input run1
453 % cp code/mitgcmuv run1
454 % cd run1
455 % ./mitgcmuv > output.txt
456 \end{verbatim}
457
458 \subsubsection{Building from a new directory}
459
460 Since the {\em input} directory contains input files it is often more
461 useful to keep {\em input} pristine and build in a new directory
462 within {\em verification/exp2/}:
463 \begin{verbatim}
464 % cd verification/exp2
465 % mkdir build
466 % cd build
467 % ../../../tools/genmake2 -mods=../code
468 % make depend
469 % make
470 \end{verbatim}
471 This builds the code exactly as before but this time you need to copy
472 either the executable or the input files or both in order to run the
473 model. For example,
474 \begin{verbatim}
475 % cp ../input/* ./
476 % ./mitgcmuv > output.txt
477 \end{verbatim}
478 or if you tend to make multiple runs with the same executable then
479 running in a new directory each time might be more appropriate:
480 \begin{verbatim}
481 % cd ../
482 % mkdir run1
483 % cp build/mitgcmuv run1/
484 % cp input/* run1/
485 % cd run1
486 % ./mitgcmuv > output.txt
487 \end{verbatim}
488
489 \subsubsection{Building on a scratch disk}
490
491 Model object files and output data can use up large amounts of disk
492 space so it is often the case that you will be operating on a large
493 scratch disk. Assuming the model source is in {\em ~/MITgcm} then the
494 following commands will build the model in {\em /scratch/exp2-run1}:
495 \begin{verbatim}
496 % cd /scratch/exp2-run1
497 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
498 -mods=~/MITgcm/verification/exp2/code
499 % make depend
500 % make
501 \end{verbatim}
502 To run the model here, you'll need the input files:
503 \begin{verbatim}
504 % cp ~/MITgcm/verification/exp2/input/* ./
505 % ./mitgcmuv > output.txt
506 \end{verbatim}
507
508 As before, you could build in one directory and make multiple runs of
509 the one experiment:
510 \begin{verbatim}
511 % cd /scratch/exp2
512 % mkdir build
513 % cd build
514 % ~/MITgcm/tools/genmake2 -rootdir=~/MITgcm \
515 -mods=~/MITgcm/verification/exp2/code
516 % make depend
517 % make
518 % cd ../
519 % cp -r ~/MITgcm/verification/exp2/input run2
520 % cd run2
521 % ./mitgcmuv > output.txt
522 \end{verbatim}
523
524
525 \subsection{Using \texttt{genmake2}}
526 \label{sec:genmake}
527
528 To compile the code, first use the program \texttt{genmake2} (located
529 in the \texttt{tools} directory) to generate a Makefile.
530 \texttt{genmake2} is a shell script written to work with all
531 ``sh''--compatible shells including bash v1, bash v2, and Bourne.
532 %Internally, \texttt{genmake2} determines the locations of needed
533 %files, the compiler, compiler options, libraries, and Unix tools. It
534 %relies upon a number of ``optfiles'' located in the
535 %\texttt{tools/build\_options} directory.
536 \texttt{genmake2} parses information from the following sources:
537 \begin{description}
538 \item[-] a {\em gemake\_local} file if one is found in the current
539 directory
540 \item[-] command-line options
541 \item[-] an "options file" as specified by the command-line option
542 \texttt{--optfile=/PATH/FILENAME}
543 \item[-] a {\em packages.conf} file (if one is found) with the
544 specific list of packages to compile. The search path for
545 file {\em packages.conf} is, first, the current directory and
546 then each of the "MODS" directories in the given order (see below).
547 \end{description}
548
549 \subsubsection{Optfiles in \texttt{tools/build\_options} directory:}
550
551 The purpose of the optfiles is to provide all the compilation options
552 for particular ``platforms'' (where ``platform'' roughly means the
553 combination of the hardware and the compiler) and code configurations.
554 Given the combinations of possible compilers and library dependencies
555 ({\it eg.} MPI and NetCDF) there may be numerous optfiles available
556 for a single machine. The naming scheme for the majority of the
557 optfiles shipped with the code is
558 \begin{center}
559 {\bf OS\_HARDWARE\_COMPILER }
560 \end{center}
561 where
562 \begin{description}
563 \item[OS] is the name of the operating system (generally the
564 lower-case output of the {\tt 'uname'} command)
565 \item[HARDWARE] is a string that describes the CPU type and
566 corresponds to output from the {\tt 'uname -m'} command:
567 \begin{description}
568 \item[ia32] is for ``x86'' machines such as i386, i486, i586, i686,
569 and athlon
570 \item[ia64] is for Intel IA64 systems (eg. Itanium, Itanium2)
571 \item[amd64] is AMD x86\_64 systems
572 \item[ppc] is for Mac PowerPC systems
573 \end{description}
574 \item[COMPILER] is the compiler name (generally, the name of the
575 FORTRAN executable)
576 \end{description}
577
578 In many cases, the default optfiles are sufficient and will result in
579 usable Makefiles. However, for some machines or code configurations,
580 new ``optfiles'' must be written. To create a new optfile, it is
581 generally best to start with one of the defaults and modify it to suit
582 your needs. Like \texttt{genmake2}, the optfiles are all written
583 using a simple ``sh''--compatible syntax. While nearly all variables
584 used within \texttt{genmake2} may be specified in the optfiles, the
585 critical ones that should be defined are:
586
587 \begin{description}
588 \item[FC] the FORTRAN compiler (executable) to use
589 \item[DEFINES] the command-line DEFINE options passed to the compiler
590 \item[CPP] the C pre-processor to use
591 \item[NOOPTFLAGS] options flags for special files that should not be
592 optimized
593 \end{description}
594
595 For example, the optfile for a typical Red Hat Linux machine (``ia32''
596 architecture) using the GCC (g77) compiler is
597 \begin{verbatim}
598 FC=g77
599 DEFINES='-D_BYTESWAPIO -DWORDLENGTH=4'
600 CPP='cpp -traditional -P'
601 NOOPTFLAGS='-O0'
602 # For IEEE, use the "-ffloat-store" option
603 if test "x$IEEE" = x ; then
604 FFLAGS='-Wimplicit -Wunused -Wuninitialized'
605 FOPTIM='-O3 -malign-double -funroll-loops'
606 else
607 FFLAGS='-Wimplicit -Wunused -ffloat-store'
608 FOPTIM='-O0 -malign-double'
609 fi
610 \end{verbatim}
611
612 If you write an optfile for an unrepresented machine or compiler, you
613 are strongly encouraged to submit the optfile to the MITgcm project
614 for inclusion. Please send the file to the
615 \begin{rawhtml} <A href="mail-to:MITgcm-support@mitgcm.org"> \end{rawhtml}
616 \begin{center}
617 MITgcm-support@mitgcm.org
618 \end{center}
619 \begin{rawhtml} </A> \end{rawhtml}
620 mailing list.
621
622 \subsubsection{Command-line options:}
623
624 In addition to the optfiles, \texttt{genmake2} supports a number of
625 helpful command-line options. A complete list of these options can be
626 obtained from:
627 \begin{verbatim}
628 % genmake2 -h
629 \end{verbatim}
630
631 The most important command-line options are:
632 \begin{description}
633
634 \item[\texttt{--optfile=/PATH/FILENAME}] specifies the optfile that
635 should be used for a particular build.
636
637 If no "optfile" is specified (either through the command line or the
638 MITGCM\_OPTFILE environment variable), genmake2 will try to make a
639 reasonable guess from the list provided in {\em
640 tools/build\_options}. The method used for making this guess is
641 to first determine the combination of operating system and hardware
642 (eg. "linux\_ia32") and then find a working FORTRAN compiler within
643 the user's path. When these three items have been identified,
644 genmake2 will try to find an optfile that has a matching name.
645
646 \item[\texttt{--mods='DIR1 DIR2 DIR3 ...'}] specifies a list of
647 directories containing ``modifications''. These directories contain
648 files with names that may (or may not) exist in the main MITgcm
649 source tree but will be overridden by any identically-named sources
650 within the ``MODS'' directories.
651
652 The order of precedence for this "name-hiding" is as follows:
653 \begin{itemize}
654 \item ``MODS'' directories (in the order given)
655 \item Packages either explicitly specified or provided by default
656 (in the order given)
657 \item Packages included due to package dependencies (in the order
658 that that package dependencies are parsed)
659 \item The "standard dirs" (which may have been specified by the
660 ``-standarddirs'' option)
661 \end{itemize}
662
663 \item[\texttt{--pgroups=/PATH/FILENAME}] specifies the file
664 where package groups are defined. If not set, the package-groups
665 definition will be read from {\em pkg/pkg\_groups}.
666 It also contains the default list of packages (defined
667 as the group ``{\it default\_pkg\_list}'' which is used
668 when no specific package list ({\em packages.conf})
669 is found in current directory or in any "MODS" directory.
670
671 \item[\texttt{--pdepend=/PATH/FILENAME}] specifies the dependency file
672 used for packages.
673
674 If not specified, the default dependency file {\em pkg/pkg\_depend}
675 is used. The syntax for this file is parsed on a line-by-line basis
676 where each line containes either a comment ("\#") or a simple
677 "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or "-" symbol
678 specifies a "must be used with" or a "must not be used with"
679 relationship, respectively. If no rule is specified, then it is
680 assumed that the two packages are compatible and will function
681 either with or without each other.
682
683 \item[\texttt{--adof=/path/to/file}] specifies the "adjoint" or
684 automatic differentiation options file to be used. The file is
685 analogous to the ``optfile'' defined above but it specifies
686 information for the AD build process.
687
688 The default file is located in {\em
689 tools/adjoint\_options/adjoint\_default} and it defines the "TAF"
690 and "TAMC" compilers. An alternate version is also available at
691 {\em tools/adjoint\_options/adjoint\_staf} that selects the newer
692 "STAF" compiler. As with any compilers, it is helpful to have their
693 directories listed in your {\tt \$PATH} environment variable.
694
695 \item[\texttt{--mpi}] This option enables certain MPI features (using
696 CPP \texttt{\#define}s) within the code and is necessary for MPI
697 builds (see Section \ref{sec:mpi-build}).
698
699 \item[\texttt{--make=/path/to/gmake}] Due to the poor handling of
700 soft-links and other bugs common with the \texttt{make} versions
701 provided by commercial Unix vendors, GNU \texttt{make} (sometimes
702 called \texttt{gmake}) should be preferred. This option provides a
703 means for specifying the make executable to be used.
704
705 \item[\texttt{--bash=/path/to/sh}] On some (usually older UNIX)
706 machines, the ``bash'' shell is unavailable. To run on these
707 systems, \texttt{genmake2} can be invoked using an ``sh'' (that is,
708 a Bourne, POSIX, or compatible) shell. The syntax in these
709 circumstances is:
710 \begin{center}
711 \texttt{\% /bin/sh genmake2 -bash=/bin/sh [...options...]}
712 \end{center}
713 where \texttt{/bin/sh} can be replaced with the full path and name
714 of the desired shell.
715
716 \end{description}
717
718
719 \subsection{Building with MPI}
720 \label{sec:mpi-build}
721
722 Building MITgcm to use MPI libraries can be complicated due to the
723 variety of different MPI implementations available, their dependencies
724 or interactions with different compilers, and their often ad-hoc
725 locations within file systems. For these reasons, its generally a
726 good idea to start by finding and reading the documentation for your
727 machine(s) and, if necessary, seeking help from your local systems
728 administrator.
729
730 The steps for building MITgcm with MPI support are:
731 \begin{enumerate}
732
733 \item Determine the locations of your MPI-enabled compiler and/or MPI
734 libraries and put them into an options file as described in Section
735 \ref{sec:genmake}. One can start with one of the examples in:
736 \begin{rawhtml} <A
737 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/build_options/">
738 \end{rawhtml}
739 \begin{center}
740 \texttt{MITgcm/tools/build\_options/}
741 \end{center}
742 \begin{rawhtml} </A> \end{rawhtml}
743 such as \texttt{linux\_ia32\_g77+mpi\_cg01} or
744 \texttt{linux\_ia64\_efc+mpi} and then edit it to suit the machine at
745 hand. You may need help from your user guide or local systems
746 administrator to determine the exact location of the MPI libraries.
747 If libraries are not installed, MPI implementations and related
748 tools are available including:
749 \begin{itemize}
750 \item \begin{rawhtml} <A
751 href="http://www-unix.mcs.anl.gov/mpi/mpich/">
752 \end{rawhtml}
753 MPICH
754 \begin{rawhtml} </A> \end{rawhtml}
755
756 \item \begin{rawhtml} <A
757 href="http://www.lam-mpi.org/">
758 \end{rawhtml}
759 LAM/MPI
760 \begin{rawhtml} </A> \end{rawhtml}
761
762 \item \begin{rawhtml} <A
763 href="http://www.osc.edu/~pw/mpiexec/">
764 \end{rawhtml}
765 MPIexec
766 \begin{rawhtml} </A> \end{rawhtml}
767 \end{itemize}
768
769 \item Build the code with the \texttt{genmake2} \texttt{-mpi} option
770 (see Section \ref{sec:genmake}) using commands such as:
771 {\footnotesize \begin{verbatim}
772 % ../../../tools/genmake2 -mods=../code -mpi -of=YOUR_OPTFILE
773 % make depend
774 % make
775 \end{verbatim} }
776
777 \item Run the code with the appropriate MPI ``run'' or ``exec''
778 program provided with your particular implementation of MPI.
779 Typical MPI packages such as MPICH will use something like:
780 \begin{verbatim}
781 % mpirun -np 4 -machinefile mf ./mitgcmuv
782 \end{verbatim}
783 Sightly more complicated scripts may be needed for many machines
784 since execution of the code may be controlled by both the MPI
785 library and a job scheduling and queueing system such as PBS,
786 LoadLeveller, Condor, or any of a number of similar tools. A few
787 example scripts (those used for our \begin{rawhtml} <A
788 href="http://mitgcm.org/public/testing.html"> \end{rawhtml}regular
789 verification runs\begin{rawhtml} </A> \end{rawhtml}) are available
790 at:
791 \begin{rawhtml} <A
792 href="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
793 \end{rawhtml}
794 {\footnotesize \tt
795 http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example\_scripts/ }
796 \begin{rawhtml} </A> \end{rawhtml}
797 or at:
798 \begin{rawhtml} <A
799 href="http://mitgcm.org/viewvc/MITgcm/MITgcm_contrib/test_scripts/">
800 \end{rawhtml}
801 {\footnotesize \tt
802 http://mitgcm.org/viewvc/MITgcm/MITgcm\_contrib/test\_scripts/ }
803 \begin{rawhtml} </A> \end{rawhtml}
804
805 \end{enumerate}
806
807 An example of the above process on the MITgcm cluster (``cg01'') using
808 the GNU g77 compiler and the mpich MPI library is:
809
810 {\footnotesize \begin{verbatim}
811 % cd MITgcm/verification/exp5
812 % mkdir build
813 % cd build
814 % ../../../tools/genmake2 -mpi -mods=../code \
815 -of=../../../tools/build_options/linux_ia32_g77+mpi_cg01
816 % make depend
817 % make
818 % cd ../input
819 % /usr/local/pkg/mpi/mpi-1.2.4..8a-gm-1.5/g77/bin/mpirun.ch_gm \
820 -machinefile mf --gm-kill 5 -v -np 2 ../build/mitgcmuv
821 \end{verbatim} }
822
823 \section[Running MITgcm]{Running the model in prognostic mode}
824 \label{sec:runModel}
825 \begin{rawhtml}
826 <!-- CMIREDIR:runModel: -->
827 \end{rawhtml}
828
829 If compilation finished succesfully (section \ref{sec:buildingCode})
830 then an executable called \texttt{mitgcmuv} will now exist in the
831 local directory.
832
833 To run the model as a single process (\textit{ie.} not in parallel)
834 simply type:
835 \begin{verbatim}
836 % ./mitgcmuv
837 \end{verbatim}
838 The ``./'' is a safe-guard to make sure you use the local executable
839 in case you have others that exist in your path (surely odd if you
840 do!). The above command will spew out many lines of text output to
841 your screen. This output contains details such as parameter values as
842 well as diagnostics such as mean Kinetic energy, largest CFL number,
843 etc. It is worth keeping this text output with the binary output so we
844 normally re-direct the \texttt{stdout} stream as follows:
845 \begin{verbatim}
846 % ./mitgcmuv > output.txt
847 \end{verbatim}
848 In the event that the model encounters an error and stops, it is very
849 helpful to include the last few line of this \texttt{output.txt} file
850 along with the (\texttt{stderr}) error message within any bug reports.
851
852 For the example experiments in \texttt{verification}, an example of the
853 output is kept in \texttt{results/output.txt} for comparison. You can
854 compare your \texttt{output.txt} with the corresponding one for that
855 experiment to check that the set-up works.
856
857
858
859 \subsection{Output files}
860
861 The model produces various output files and, when using \texttt{mnc},
862 sometimes even directories. Depending upon the I/O package(s)
863 selected at compile time (either \texttt{mdsio} or \texttt{mnc} or
864 both as determined by \texttt{code/packages.conf}) and the run-time
865 flags set (in \texttt{input/data.pkg}), the following output may
866 appear.
867
868
869 \subsubsection{MDSIO output files}
870
871 The ``traditional'' output files are generated by the \texttt{mdsio}
872 package. At a minimum, the instantaneous ``state'' of the model is
873 written out, which is made of the following files:
874
875 \begin{itemize}
876 \item \texttt{U.00000nIter} - zonal component of velocity field (m/s
877 and positive eastward).
878
879 \item \texttt{V.00000nIter} - meridional component of velocity field
880 (m/s and positive northward).
881
882 \item \texttt{W.00000nIter} - vertical component of velocity field
883 (ocean: m/s and positive upward, atmosphere: Pa/s and positive
884 towards increasing pressure i.e. downward).
885
886 \item \texttt{T.00000nIter} - potential temperature (ocean:
887 $^{\circ}\mathrm{C}$, atmosphere: $^{\circ}\mathrm{K}$).
888
889 \item \texttt{S.00000nIter} - ocean: salinity (psu), atmosphere: water
890 vapor (g/kg).
891
892 \item \texttt{Eta.00000nIter} - ocean: surface elevation (m),
893 atmosphere: surface pressure anomaly (Pa).
894 \end{itemize}
895
896 The chain \texttt{00000nIter} consists of ten figures that specify the
897 iteration number at which the output is written out. For example,
898 \texttt{U.0000000300} is the zonal velocity at iteration 300.
899
900 In addition, a ``pickup'' or ``checkpoint'' file called:
901
902 \begin{itemize}
903 \item \texttt{pickup.00000nIter}
904 \end{itemize}
905
906 is written out. This file represents the state of the model in a condensed
907 form and is used for restarting the integration. If the C-D scheme is used,
908 there is an additional ``pickup'' file:
909
910 \begin{itemize}
911 \item \texttt{pickup\_cd.00000nIter}
912 \end{itemize}
913
914 containing the D-grid velocity data and that has to be written out as well
915 in order to restart the integration. Rolling checkpoint files are the same
916 as the pickup files but are named differently. Their name contain the chain
917 \texttt{ckptA} or \texttt{ckptB} instead of \texttt{00000nIter}. They can be
918 used to restart the model but are overwritten every other time they are
919 output to save disk space during long integrations.
920
921 \subsubsection{MNC output files}
922
923 Unlike the \texttt{mdsio} output, the \texttt{mnc}--generated output
924 is usually (though not necessarily) placed within a subdirectory with
925 a name such as \texttt{mnc\_test\_\${DATE}\_\${SEQ}}.
926
927 \subsection{Looking at the output}
928
929 The ``traditional'' or mdsio model data are written according to a
930 ``meta/data'' file format. Each variable is associated with two files
931 with suffix names \texttt{.data} and \texttt{.meta}. The
932 \texttt{.data} file contains the data written in binary form
933 (big\_endian by default). The \texttt{.meta} file is a ``header'' file
934 that contains information about the size and the structure of the
935 \texttt{.data} file. This way of organizing the output is particularly
936 useful when running multi-processors calculations. The base version of
937 the model includes a few matlab utilities to read output files written
938 in this format. The matlab scripts are located in the directory
939 \texttt{utils/matlab} under the root tree. The script \texttt{rdmds.m}
940 reads the data. Look at the comments inside the script to see how to
941 use it.
942
943 Some examples of reading and visualizing some output in {\em Matlab}:
944 \begin{verbatim}
945 % matlab
946 >> H=rdmds('Depth');
947 >> contourf(H');colorbar;
948 >> title('Depth of fluid as used by model');
949
950 >> eta=rdmds('Eta',10);
951 >> imagesc(eta');axis ij;colorbar;
952 >> title('Surface height at iter=10');
953
954 >> eta=rdmds('Eta',[0:10:100]);
955 >> for n=1:11; imagesc(eta(:,:,n)');axis ij;colorbar;pause(.5);end
956 \end{verbatim}
957
958 Similar scripts for netCDF output (\texttt{rdmnc.m}) are available and
959 they are described in Section \ref{sec:pkg:mnc}.
960
961 The MNC output files are all in the ``self-describing'' netCDF
962 format and can thus be browsed and/or plotted using tools such as:
963 \begin{itemize}
964 \item \texttt{ncdump} is a utility which is typically included
965 with every netCDF install:
966 \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
967 \begin{verbatim}
968 http://www.unidata.ucar.edu/packages/netcdf/
969 \end{verbatim}
970 \begin{rawhtml} </A> \end{rawhtml} and it converts the netCDF
971 binaries into formatted ASCII text files.
972
973 \item \texttt{ncview} utility is a very convenient and quick way
974 to plot netCDF data and it runs on most OSes:
975 \begin{rawhtml} <A href="http://meteora.ucsd.edu/~pierce/ncview_home_page.html"> \end{rawhtml}
976 \begin{verbatim}
977 http://meteora.ucsd.edu/~pierce/ncview_home_page.html
978 \end{verbatim}
979 \begin{rawhtml} </A> \end{rawhtml}
980
981 \item MatLAB(c) and other common post-processing environments provide
982 various netCDF interfaces including:
983 \begin{rawhtml} <A href="http://mexcdf.sourceforge.net/"> \end{rawhtml}
984 \begin{verbatim}
985 http://mexcdf.sourceforge.net/
986 \end{verbatim}
987 \begin{rawhtml} </A> \end{rawhtml}
988 \begin{rawhtml} <A href="http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html"> \end{rawhtml}
989 \begin{verbatim}
990 http://woodshole.er.usgs.gov/staffpages/cdenham/public_html/MexCDF/nc4ml5.html
991 \end{verbatim}
992 \begin{rawhtml} </A> \end{rawhtml}
993 \end{itemize}
994

  ViewVC Help
Powered by ViewVC 1.1.22