/[MITgcm]/MITgcm/doc/devel_HOWTO.sgml
ViewVC logotype

Contents of /MITgcm/doc/devel_HOWTO.sgml

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.5 - (show annotations) (download) (as text)
Wed Dec 10 20:44:39 2003 UTC (20 years, 3 months ago) by edhill
Branch: MAIN
CVS Tags: checkpoint52d_post
Branch point for: netcdf-sm0
Changes since 1.4: +82 -43 lines
File MIME type: text/x-sgml
 o "make install" now builds and installs all four formats of the HOWTO
   on the web server
 o minor edits to the HOWTO

1 <!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
2
3 <article id="MITgcm-Development-HOWTO">
4
5 <articleinfo>
6 <title>MITgcm Development HOWTO</title>
7
8 <author>
9 <firstname>Ed</firstname>
10 <surname>Hill III</surname>
11 <affiliation>
12 <address><email>eh3@mit.edu</email></address>
13 </affiliation>
14 </author>
15
16 <revhistory>
17 <revision>
18 <revnumber>0.01</revnumber>
19 <date>2003-08-07</date>
20 <authorinitials>eh3</authorinitials>
21 <revremark>
22 Initial version.
23 </revremark>
24 </revision>
25 </revhistory>
26
27 <abstract>
28 <para>This document describes how to develop software for the
29 MITgcm project.</para>
30 </abstract>
31 </articleinfo>
32
33 <sect1 id="intro">
34 <title>Introduction</title> <para>The purpose of this document is
35 to help new developers get "up to speed" with MITgcm
36 development.</para>
37 <sect2>
38 <title>New Versions of This Document</title> <para>You can
39 obtain the latest version of this document <ulink
40 url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in
41 various formats.</para>
42 </sect2>
43 <sect2>
44 <title>Feedback and corrections</title> <para>If you have
45 questions or comments about this document, please feel free to
46 <ulink url="mailto:MITgcm-support@mitgcm.org">contact the
47 authors</ulink>.
48 </para>
49 </sect2>
50 </sect1>
51
52 <sect1 id="background">
53 <title>Background</title>
54
55 <sect2>
56 <title>User Manual</title>
57
58 <para>Before jumping into development, please familiarize yourself with
59 the <ulink url="http://mitgcm.org/docs.html"> MITgcm user manual
60 </ulink>. This document contains volumes of useful information and is
61 included here by reference.</para>
62
63 <!--
64 <para>Also, a "snapshot" or <ulink
65 url="http://mitgcm.org/dev_docs/">development version</ulink> of
66 the user manual may be available, though this is only put on the
67 web for testing purposes.</para>
68 -->
69 </sect2>
70
71 <sect2>
72 <title>Prerequisites</title> <para>To develop for MITgcm project
73 you will need a UNIX or UNIX-like set of build tools including
74 the following:</para>
75 <blockquote>
76 <simplelist type="inline">
77 <member>CVS client</member>
78 <member>make or (preferably) GNU make</member>
79 <member>FORTRAN compiler</member>
80 <member>C compiler</member>
81 <member>[ba]sh and [t]csh shells</member>
82 <member>PERL</member>
83 <member>LaTeX and LaTeX2HTML</member>
84 </simplelist>
85 </blockquote>
86 <para>Essentially all of the work described here has been tested
87 on recent versions of Red Hat Linux (eg. 7.3 through 9). Except
88 where noted, all shell commands will be provided using bash
89 syntax.
90 </para>
91 </sect2>
92
93 </sect1>
94
95 <sect1 id="cvs">
96 <title>CVS Repository</title>
97 <sect2>
98 <title>Layout</title>
99
100 <para>Unlike many open source projects, the MITgcm CVS tree does
101 not follow a simple "src", "docs", "share", and "test" directory
102 layout. Instead, there are multiple higher-level directories
103 that each, to some extent, depend upon the presence of the
104 others. The tree currently resembles:</para>
105
106 <programlisting>gcmpack/
107 MITgcm-contrib contributed code
108 CS-regrid goes into utils
109 cvspolicy.html -save-
110 CVSROOT -save-
111 development experimental stuff
112 manual -save-
113 misc -?-
114
115 MITgcm code
116 adjoint fold into genmake
117 bin stub for ecco build
118 compare01 old from 20th century
119 diags timeave f77 in pkgs now
120 doc tags -- connect to real docs?
121 eesupp cnh?
122 exe ecco user build
123 ,- jobs runtime shell scripts for
124 | various platforms
125 | lsopt line search
126 m| model main dynamics (core)
127 e| optimization_drivers ?
128 r| optim line search interface
129 g| pkg alternate and optional numerics, etc.
130 e|- tools
131 ?| tutorial_examples documented tests
132 | only populated on release1 branch
133 | and not validated during "testscript"
134 '- utils
135 verification std tests
136
137
138 mitgcmdoc -> manual -remove-
139 mitgcm.org build web site
140 models -?-
141 packages -?-
142 preprocess -?-
143 tmp -?-
144 </programlisting>
145
146 <para>Efforts are underway to reduce the complexity.</para>
147
148 </sect2>
149
150 <!--
151 <sect2>
152 <title>Releases</title> <para>Currently, there are two main
153 branches:</para>
154 <itemizedlist mark="bullet">
155 <listitem>
156 <para>Development</para>
157 <itemizedlist mark="bullet">
158 <listitem>
159 <para>MAIN</para>
160 </listitem>
161 <listitem>
162 <para>ecco-branch</para>
163 </listitem>
164 </itemizedlist>
165 </listitem>
166 <listitem>
167 <para>Production</para>
168 <itemizedlist mark="bullet">
169 <listitem>
170 <para>Release1</para>
171 </listitem>
172 <listitem>
173 <para>Release2</para>
174 </listitem>
175 </itemizedlist>
176 </listitem>
177 </itemizedlist>
178 </sect2>
179 -->
180
181 <sect2>
182 <title>Branches</title>
183
184 <para>As shown in the online <ulink
185 url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated
186 tree</ulink>, the MITgcm codebase is split into to two branches
187 or "lines" under which development proceeds. These two lines
188 are referred to as the "MAIN" and "ecco" versions of the code.
189 While not identical, the bulk of the MAIN and ecco lines are
190 composed of files from the same codebase.
191 </para>
192
193 <para>Periodically, a "Release" branch is formed from the "MAIN"
194 development branch. This is done in order to create a
195 relatively stable reference point for both users and developers.
196 The intent is that once a relese branch has been created, only
197 bug-fixes will be added to it. Meanwhile, development (which
198 might "break" or otherwise render invalid the documentation,
199 tutorials, and/or examples contained within a release branch) is
200 allowed to continue along the MAIN and ecco lines.</para>
201 </sect2>
202
203 <sect2>
204 <title>Tagging</title>
205
206 <para>The intent of tagging is to create "known-good"
207 checkpoints that developers can use as references.
208 Traditionally, MITgcm tagging has maintained the following
209 conventions:</para>
210
211 <orderedlist>
212 <listitem>
213 <para>Developer checks out code into a local CVS-managed
214 directory, makes various changes/additions, tests these
215 edits, and eventually reaches a point where (s)he is
216 satisfied that the changes form a new "useful" point in the
217 evolution of the code.</para>
218 </listitem>
219
220 <listitem>
221 <para>The developer then runs the <ulink
222 url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink>
223 shell script to see if any problems are introduced. While
224 not intended to be exhaustive, the test cases within the
225 verification directory do provide some indication whether
226 gross errors have been introduced.
227 </para>
228 </listitem>
229
230 <listitem>
231 <para>Having satisfied him- or herself that the changes are
232 ready to be committed to the CVS repository, the developer
233 then:</para>
234 <orderedlist>
235 <listitem>
236 <para>adds a "checkpointXY_pre" comment (where X is a
237 checkpoint number and Y is a letter) to the <ulink
238 url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink>
239 file and checks it into the CVS repository</para>
240 </listitem>
241 <listitem>
242 <para>submits the set of changes to the CVS repository
243 and adds comments to <filename>tag-index</filename>
244 describing what the changes are along with a matching
245 "checkpointXY_post" entry</para>
246 </listitem>
247 </orderedlist>
248 </listitem>
249 </orderedlist>
250
251 <para>The result of this tagging procedure is a sequence of
252 development checkpoints with comments which resembles:</para>
253
254 <programlisting>
255 checkpoint50e_post
256 o make KPP work with PTRACERS
257 - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is
258 nearly a copy of kpp_transport_s
259 - there is no analogue to SurfaceTendencyS, so I have to use
260 gPtr(of the surface layer) instead
261 o add a new platform SunFire+mpi (SunFire 15000) to genmake
262 checkpoint50e_pre
263
264 checkpoint50d_post
265 o change kpp output from multiple-record state files to single-record state
266 files analogous to write_state.F
267 o reduce the output frequency of cg3d-related stuff to the monitor frequency,
268 analogous to the cg2d-related output.
269 o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,
270 so that writing to internal file fn (with length 512) fails.
271 checkpoint50d_pre
272 </programlisting>
273
274 <para>This information can be used to refer to various stages of
275 the code development. For example, bugs can be traced to
276 individual sets of CVS checkins based upon their first
277 appearance when comparing the results from different
278 checkpoints.</para>
279
280 </sect2>
281 </sect1>
282
283
284 <sect1 id="documentation">
285 <title>Editing the Documentation</title>
286
287 <sect2 id="documentation_getting">
288 <title>Getting the Docs and Code</title>
289
290 <para>The first step towards editing the documentation is to
291 checkout a copy of code, docs, and build scripts from the CVS
292 server using:</para>
293
294 <screen>
295 $ export CVS_RSH=ssh
296 $ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack'
297 $ mkdir scratch
298 $ cvs co MITgcm manual mitgcm.org
299 </screen>
300
301 <para>These commands extract the necessary information from the
302 CVS server and create a temporary (called
303 <filename>scratch</filename>) directory for the storage of the
304 HTML and other files that will be created. Please note that you
305 must either create <filename>scratch</filename> as shown or edit
306 the various <filename>Makefile</filename>s and scripts used to
307 create the documentation.</para>
308 </sect2>
309
310 <sect2>
311 <title>Editing the Documentation</title>
312
313 <para>The documentation is contained in the
314 <filename>manual</filename> directory in a raw LaTeX format.
315 The main document is <filename>manual.tex</filename> and it uses
316 <command>\input{}</command>s to include the chapters and
317 subsections.</para>
318
319 <para>Since the same LaTeX source is used to produce PostScript,
320 PDF, and HTML output, care should be taken to follow certain
321 conventions. Two of the most important are the usage of the
322 <command>\filelink{}{}</command> and
323 <command>\varlink{}{}</command> commands. Both of these
324 commands have been defined to simplify the connection between
325 the automatically generated ("code browser") HTML and the HTML
326 version of the manual produced by LaTeX2HTML. They each take
327 two arguments (corresponding to the contents of the two sets of
328 curly braces) which are the text that the author wishes to be
329 "wrapped" within the link, and a specially formatted link thats
330 relative to the <filename>MITgcm</filename> directory within the
331 CVS tree.</para>
332
333 <para>The result is a command that resembles either</para>
334
335 <orderedlist>
336 <listitem>
337 <para>a reference to a variable or subroutine name such as
338 <command>\varlink{tRef}{tRef}</command>, or </para>
339 </listitem>
340
341 <listitem>
342 <para>a reference to a file such as
343 <command>\varlink{tRef}{path-to-the-file_name.F}</command>
344 where the absolute path to the file is of the form
345 <filename>/foo/MITgcm/path/to/the/file_name.F</filename></para>
346 <para>(please note how the leading "/foo/MITgcm"
347 component of the path is dropped leaving the path
348 <emphasis>relative</emphasis> to the head of the code
349 directory and each directory separator "/" is turned
350 into a "-")</para>
351 </listitem>
352 </orderedlist>
353
354
355
356 </sect2>
357
358 <sect2>
359 <title>Building the Documentation</title>
360
361 <para>Given the directory structure of <xref
362 linkend="documentation_getting">, the entire documentation for the web
363 site can be built using:</para>
364
365 <screen>
366 $ cd mitgcm.org/devel/buildweb
367 $ make All
368 </screen>
369
370 <para>Which builds the PDF from the LaTeX source, creates the
371 HTML output from the LaTeX source, parses the FORTRAN code base
372 to produce a hyperlinked HTML version of the source, and then
373 determines the cross-linking between the various HTML
374 components.</para>
375
376 <para>If there are no errors, the result of the build process
377 (which can take 30+ minutes on a P4/2.5Ghz) will be contained
378 within a single directory called
379 <filename>scratch/dev_docs</filename>. This is a freshly built
380 version of the entire on-line users manual. If you have the
381 correct permissions, it can be directly copied to the web server
382 area:</para>
383
384 <screen>
385 $ mv scratch/dev_docs /u/u0/httpd/html
386 </screen>
387
388 <para>and the update is complete.</para>
389
390 </sect2>
391
392 </sect1>
393
394 <sect1 id="coding">
395 <title>Coding for MITgcm</title>
396
397 <sect2 id="build_tools">
398 <title>Build Tools</title>
399
400 <para>Many Open Source projects use the "GNU Autotools" to help streamline
401 the build process for various Unix and Unix-like architectures. For a
402 user, the result is the common "configure" (that is,
403 "<filename>./configure && make && make install</filename>") commands.
404 For MITgcm, the process is similar. Typical commands are:</para>
405
406 <screen>
407 $ genmake -mods=../code
408 $ make depend
409 $ make
410 </screen>
411
412 <para>The following sections describe the individual steps in the build
413 process.</para>
414
415 <sect3 id="genmake">
416 <title>The <filename>genmake2</> Utility</title>
417
418 <para><emphasis>Please note that the older <filename>genmake</> is
419 deprecated and will eventually be replaced by <filename>genmake2</>.
420 This HOWTO only describes the newer tool.</emphasis></para>
421
422 <para>The first step in any MITgcm build is to create a Unix-style
423 <filename>Makefile</filename> which will be parsed by
424 <filename>make</filename> to specify how to compile the MITgcm source
425 files. For more detailed descriptions of what the make tools are and
426 how they are used, please see:</para>
427
428 <itemizedlist>
429 <listitem>
430 <para><ulink url="http://www.gnu.org/software/make/make.html">
431 http://www.gnu.org/software/make/make.html</></para>
432 </listitem>
433 <listitem>
434 <para><ulink url="http://www.oreilly.com/catalog/make2/">
435 http://www.oreilly.com/catalog/make2/</></para>
436 </listitem>
437 </itemizedlist>
438
439 <para>Genmake can often be invoked successfully with a command line as
440 simple as:</para>
441
442 <screen>
443 $ genmake2 -mods=../code
444 </screen>
445
446 <para>However, some systems (particularly commercial Unixes that lack a
447 more modern "/bin/sh" implementation or that have shells installed in
448 odd locations) may require an explicit shell invocation such as one of
449 the following: </para>
450
451 <screen>
452 $ /usr/bin/sh genmake2 -make=gmake -mods=../code
453 $ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code
454 </screen>
455
456 <para>The genmake2 code has been written in a Bourne and BASH (v1)
457 compatible syntax so it should work with most "sh" and all recent "bash"
458 implementations.</para>
459
460 <para>As the name implies, <filename>genmake2</filename> generates a
461 <filename>Makefile</filename>. It does so by first parsing the
462 information supplied from the following sources</para>
463
464 <orderedlist>
465 <listitem>
466 <para>a <filename>gm_local</filename> file in the current
467 directory</para>
468 </listitem>
469 <listitem>
470 <para>directly from command-line options</para>
471 </listitem>
472 <listitem>
473 <para>an "options file" as specified by the command-line option
474 <filename>-optfile='FILENAME'</filename></para>
475 </listitem>
476 </orderedlist>
477
478 <para>then checking certain dependency rules (the package dependencies),
479 and finally writing a <filename>Makefile</filename> based upon the
480 source code that it finds. For convenience within various Unix
481 shells, <filename>genmake2</> supports both "long"- and "short"-style
482 options. A complete list of the available options can be obtained
483 from:</para>
484
485 <screen>
486 $ genmake2 -help
487 </screen>
488
489 <para>The most important options for <filename>genmake2</> are:</para>
490
491 <variablelist>
492
493 <varlistentry>
494 <term><filename>--optfile=/PATH/FILENAME</></term>
495
496 <listitem>
497 <para>This specifies the "options file" that should be used for a
498 particular build. The options file is a convenient and
499 machine-indepenent way of specifying parameters such as the
500 FORTRAN compiler (<filename>FC=</>), FORTRAN compiler
501 optimization flags (<filename>FFLAGS=</>), and the locations of
502 various platform- and/or machine-specific tools
503 (eg. <filename>MAKEDEPEND=</>). As with <filename>genmake2</>,
504 all options files should be written to be compatible with
505 Bourne--shell ("sh" or "BASH v1") syntax. Examples of various
506 options files can be found in
507 <filename>$ROOTDIR/tools/build_options</>.</para>
508
509 <para>If no "optfile" is specified (either through the command lin
510 or the environment variable), genmake2 will try to make a
511 reasonable guess from the list provided in
512 <filename>$ROOTDIR/tools/build_options</>. The method used for
513 making this guess is to first determine the combination of
514 operating system and hardware (eg. "linux_ia32") and then find a
515 working Fortran compiler within the user's path. When these
516 three items have been identified, genmake2 will try to find an
517 optfile that has a matching name. </para>
518
519 <para>Everyone is encouraged to submit their options files to the
520 MITgcm project for inclusion (please send to
521 <email>MITgcm-support@mitgcm.org</email>). We are particularly
522 grateful for options files tested on new or unique
523 platforms!</para>
524 </listitem>
525
526 </varlistentry>
527
528 <varlistentry>
529 <term><filename>-pdepend=/PATH/FILENAME</></term>
530
531 <listitem>
532 <para>This specifies the dependency file used for packages. If
533 not specified, the default dependency file is
534 <filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for this file is
535 parsed on a line-by-line basis where each line containes either a
536 comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule
537 where the "+" or "-" symbol specifies a "must be used with" or a
538 "must not be used with" relationship, respectively. If no rule is
539 specified, then it is assumed that the two packages are compatible
540 and will function either with or without each other.</para>
541 </listitem>
542 </varlistentry>
543
544 <varlistentry>
545 <term><filename>-pdefault=PKG</></term>
546 <term><filename>-pdefault='PKG1 [PKG2 PKG3 ...]'</></term>
547 <listitem>
548 <para>This option specifies the default set of packages
549 to be used. If not set, the default package list will
550 be read from
551 <filename>$ROOTDIR/pkg/pkg_default</>.</para>
552 </listitem>
553 </varlistentry>
554
555 <varlistentry>
556 <term><filename>-adof=/path/to/file</></term>
557 <term><filename>-adoptfile=/path/to/file</></term>
558 <listitem>
559 <para>This option specifies the "adjoint" or automatic
560 differentiation options file to be used. The file is analogous
561 to the "optfile" defined above but it specifies information for
562 the AD build process. The default file is located in <filename>
563 $ROOTDIR/tools/adjoint_options/adjoint_default </> and it
564 defines the "TAF" and "TAMC" compilers. An alternate version is
565 also available at <filename>
566 $ROOTDIR/tools/adjoint_options/adjoint_staf </> that selects the
567 newer "STAF" compiler. As with any compilers, it is helpful to
568 have their directories listed in your $PATH environment
569 variable.</para>
570 </listitem>
571 </varlistentry>
572
573 <varlistentry>
574 <term><filename>-mods=DIR</></term>
575 <term><filename>-mods='DIR1 [DIR2 ...]'</></term>
576 <listitem>
577 <para>This option specifies a list of directories containing
578 "modifications". These directories contain files with names
579 that may (or may not) exist in the main MITgcm source tree but
580 will be overridden by any identically-named sources within the
581 "MODS" directories. The order of precedence for this
582 "name-hiding" is as follows:</para>
583
584 <itemizedlist>
585 <listitem><para>"MODS" directories (in the order given)
586 </para></listitem>
587 <listitem><para>Packages either explicitly specified or
588 provided by default (in the order given)</para></listitem>
589 <listitem><para>Packages included due to package dependencies
590 (in the order that that package dependencies are
591 parsed)</para></listitem>
592 <listitem><para>The "standard dirs" (which may have been
593 specified by the "-standarddirs" option)</para></listitem>
594 </itemizedlist>
595
596 </listitem>
597 </varlistentry>
598
599 <varlistentry>
600 <term><filename>-make=/path/to/gmake</></term>
601 <listitem>
602 <para>Due to the poor handling of soft-links and other bugs common
603 with the <filename>make</> versions provided by commercial Unix
604 vendors, GNU <filename>make</filename> (sometimes called
605 <filename>gmake</filename>) should be preferred. This option
606 provides a means for specifying the make program to be
607 used.</para>
608 </listitem>
609 </varlistentry>
610
611 </variablelist>
612
613 <para>A successful run of <filename>genmake2</> will produce a
614 <filename>Makefile</>, a <filename>PACKAGES_CONFIG.h</> file, and
615 various convenience files used for the automatic differentiation
616 process.</para>
617
618 <para>In general, it is best to use <filename>genmake2</> on a "clean"
619 directory that is free of all source (*.[F,f],*.[F,f]90) and header
620 (*.h,*.inc) files. Generally, this can be accomplished in an
621 "un-clean" directory by running "make CLEAN" followed by "make
622 makefile".</para>
623
624 </sect3>
625
626 <sect3 id="makefile_use">
627 <title>Using the <filename>Makefile</></title>
628
629 <para>Once a <filename>Makefile</> has been created using
630 <filename>genmake2</>, one can build a "standard" (forward
631 simulator) executable using:</para>
632
633 <screen>
634 $ make CLEAN
635 $ make depend
636 $ make
637 </screen>
638
639 <para>The "make CLEAN" step will remove any stale source files, include
640 files, and links. It is strongly recommended for "un-clean"
641 directories which may contain the (perhaps partial) results of
642 previous builds. Such "debris" can interfere with the next stage of
643 the build.</para>
644
645 <para>The "make depend" step will create a large number of symbolic
646 links from the local directory to the source file locations. It also
647 parses these files and creates an extensive list of dependencies
648 within the <filename>Makefile</> itself. The links that exist at this
649 stage are mostly "large F" files (*.F and *.F90) that need to be
650 processed by a C preprocessor ("CPP"). Since "make depend" edits the
651 <filename>Makefile</>, it is important not to skip this step!</para>
652
653 <para>The final "make" invokes the C preprocessor to produce the "little
654 f" files (*.f and *.f90) and then compiles them to object code using
655 the specified FORTRAN compiler and options. An intermediate script is
656 often used during this stage to further process (usually, make simple
657 substitutions) custom definitions such as variable types within the
658 source files. This additional stage is necessary in order to overcome
659 some of the inconsistencies in the sizes of objects (bytes) between
660 different compilers. The result of the build process is an executable
661 with the name <filename>mitgcmuv</>.</para>
662
663 <para>In addition to the forward simulator described above, the
664 <filename>Makefile</> also has a number of targets that can be used to
665 produce various adjoint and tangent-linear builds for optimization and
666 other parameter-sensitivity problems. The additional targets within
667 the <filename>Makefile</> are:</para>
668
669 <variablelist>
670
671 <varlistentry>
672 <term><filename>make adall</></term>
673 <listitem>
674 <para>This target produces an <filename>mitgcmuv_ad</> executable
675 using the <filename>taf</> or <filename>staf</> adjoint
676 compiler. See the <filename>genmake2</> "-adof" option for
677 compiler selection.</para>
678 </listitem>
679 </varlistentry>
680
681 <varlistentry>
682 <term><filename>make ftlall</></term>
683 <listitem>
684 <para>Similar to <filename>make adall</> above, this
685 produces...</para>
686 </listitem>
687 </varlistentry>
688
689 </variablelist>
690
691 <para>Please report any compilation failures or other build problems to
692 the <email>MITgcm-support@mitgcm.org</email> list.</para>
693
694 </sect3>
695
696 </sect2>
697
698 <sect2 id="verification">
699 <title>The Verification Suite</title>
700
701 <para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</>
702 directory) includes more than a dozen examples intended for regression
703 testing. Each one of these example directories contains "known-good"
704 output files along with all the input (including both code and data
705 files) required for their re-calculation. These example directories are
706 further broken down into sets of subdirectories
707 (eg. <filename>/input</>, <filename>/code</>) intended to expedite the
708 testing process.</para>
709
710 <sect3 id="testreport">
711 <title>The <filename>testreport</> Utility</title>
712
713 <para>Also included in <filename>$ROOTDIR/verification/</> are shell
714 scripts for automated testing. The newest script (which was written
715 to work with <filename>genmake2</>) is called <filename>testreport</>.
716 This script can be used to build different versions of the MITgcm
717 code, run the various examples, compare the output, and (if specified)
718 email the results of each one of these tests to a central
719 repository.</para>
720
721 <para>On some systems, the testreport script can be run with a command
722 line as simple as:</para>
723
724 <screen>
725 $ cd verification
726 $ ./testreport -ieee
727 </screen>
728
729 <para>However, some systems (those lacking or wiht a broken "/bin/sh")
730 may require an explicit shell invocation such as:</para>
731
732 <screen>
733 $ sh ./testreport -ieee -t 'exp0 exp4'
734 $ /some/path/to/bash ./testreport -ieee -t 'ideal_2D_oce lab_sea natl_box'
735 </screen>
736
737 <para>The <filename>testreport</> script accepts a number of
738 command-line options which can be listed using the <filename>-help</>
739 option. The most important ones are:</para>
740
741 <variablelist>
742
743 <varlistentry>
744 <term><filename>-ieee</></term>
745 <listitem>
746 <para>If allowed by the compiler (as defined in the "optfile"),
747 use IEEE arithmetic. This option, along with the GCC compiler,
748 is how the standard results were produced.</para>
749 </listitem>
750 </varlistentry>
751
752 <varlistentry>
753 <term><filename>-tdir TESTDIR</></term>
754 <term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term>
755 <listitem>
756 <para>This option specifies the test directory or list of test
757 directories that should be used. Each of these entries should
758 exactly (note: they are case sensitive!) match the names of
759 directries in <filename>$ROOTDIR/verification/</>. If this
760 option is omitted, then all directories that are properly
761 formatted (that is, containing an <filename>input</>
762 sub-directory and a <filename>results/output.txt</> file) will
763 be used.</para>
764 </listitem>
765 </varlistentry>
766
767 <varlistentry>
768 <term><filename>-optfile=/PATH/FILENAME</></term>
769 <term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>
770 <listitem>
771 <para>This specifies a list of "options files" that will be passed
772 to <filename>genmake2</>. If multiple options files are used
773 (say, to test different compilers or different sets of options
774 for the same compiler), then each options file will be used with
775 each of the test directories.</para>
776 </listitem>
777 </varlistentry>
778
779 <varlistentry>
780 <term><filename>-addr EMAIL</></term>
781 <term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>
782 <listitem>
783 <para>Send the results (namely, <filename>output.txt</>,
784 <filename>genmake_local</>, <filename>genmake_state</>, and
785 <filename>Makefile</>) to the specified email addresses. The
786 results are gzipped, placed in a tar file, MIME encoded, and
787 sent to the specified address. If no email addresses are
788 specified, no mail is sent.</para>
789 </listitem>
790 </varlistentry>
791
792 <varlistentry>
793 <term><filename>-mpi</></term>
794 <listitem>
795 <para>If the necessary files
796 (<filename>TESTDIR/code/CPP_EEOPTIONS.h_mpi</> and
797 <filename>TESTDIR/code/SIZE.h_mpi</>) exist, then use them for an
798 MPI--enabled run. Note that the use of MPI typically requires a
799 special command option (see "-command" below) to invoke the MPI
800 executable. Examples of PBS scripts using MPI with testreport can be
801 found in the <ulink
802 url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">
803 MITgcm-contrib area</ulink></para>
804 </listitem>
805 </varlistentry>
806
807 <varlistentry>
808 <term><filename>-command='some command to run'</></term>
809 <listitem>
810 <para>For some tests, particularly MPI runs, the default "make
811 output.txt" is not sufficient. This option allows a more general
812 command (or shell script) to be invoked. Examples of PBS scripts
813 using MPI with testreport can be found in the <ulink
814 url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">
815 MITgcm-contrib area</ulink></para>
816 </listitem>
817 </varlistentry>
818
819 </variablelist>
820
821 <para>The <filename>testreport</> script will write progress to the
822 screen (stdout) as it runs. In addition, it will create a
823 <filename>tr_out.txt</> file that contains a brief comparison of the
824 current output with the "known-good" output.</para>
825
826 </sect3>
827
828 </sect2>
829
830
831 <sect2 id="packages">
832 <title>Creating MITgcm Packages</title>
833
834 <para>Optional parts of code have been separated from the MITgcmUV core
835 driver code and organised into packages. The packaging structure
836 provides a mechanism for maintaining suites of code, specific to
837 particular classes of problems, in a way that is cleanly separated from
838 the generic fluid dynamical engine.</para>
839
840 <para>The MITgcmUV packaging structure is described below using generic
841 package names ${pkg}. A concrete examples of a package is the code for
842 implementing GM/Redi mixing. This code uses the package name</para>
843
844 </sect2>
845
846 </sect1>
847
848 <sect1>
849 <title>Chris's Notes...</title>
850
851 <programlisting>
852 MITgcmUV Packages
853 =================
854
855 Optional parts of code are separated from
856 the MITgcmUV core driver code and organised into
857 packages. The packaging structure provides a mechanism for
858 maintaining suites of code, specific to particular
859 classes of problem, in a way that is cleanly
860 separated from the generic fluid dynamical engine.
861
862 The MITgcmUV packaging structure is describe
863 below using generic package names ${pkg}.
864 A concrete examples of a package is the code
865 for implementing GM/Redi mixing. This code uses
866 the package name
867 * ${PKG} = GMREDI
868 * ${pkg} = gmredi
869 * ${Pkg} = gmRedi
870
871 Package states
872 ==============
873
874 Packages can be any one of four states, included,
875 excluded, enabled, disabled as follows:
876
877 included(excluded) compile time state which
878 includes(excludes) package
879 code and routine calls from
880 compilation/linking etc...
881
882 enabled(disabled) run-time state which
883 enables(disables) package code
884 execution.
885
886 Every call to a ${pkg}_... routine from outside the package
887 should be placed within both a
888 #ifdef ALLOW_${PKG} ... block and a
889 if ( use${Pkg} ) ... then block.
890 Package states are generally not expected to change during
891 a model run.
892
893 Package structure
894 =================
895
896 o Each package gets its runtime configuration
897 parameters from a file named "data.${pkg}"
898 Package runtime config. options are imported
899 into a common block held in a header file
900 called "${PKG}.h".
901
902 o The core driver part of the model can check
903 for runtime enabling or disabling of individual packages
904 through logical flags use${Pkg}.
905 The information is loaded from a
906 global package setup file called "data.pkg".
907 The use${Pkg} flags are not used within
908 individual packages.
909
910 o Included in "${PKG}.h" is a logical flag
911 called ${Pkg}IsOn. The "${PKG}.h" header file can be imported
912 by other packages to check dependencies and requirements
913 from other packages ( see "Package Boot Sequence" section).
914 NOTE: This procedure is not presently implemented,
915 ----- neither for kpp nor for gmRedi.
916
917 CPP Flags
918 =========
919
920 1. Within the core driver code flags of the form
921 ALLOW_${PKG} are used to include or exclude
922 whole packages. The ALLOW_${PKG} flags are included
923 from a PKG_CPP_OPTIONS block which is currently
924 held in-line in the CPP_OPTIONS.h header file.
925 e.g.
926
927 Core model code .....
928
929 #include "CPP_OPTIONS.h"
930 :
931 :
932 :
933
934 #ifdef ALLOW_${PKG}
935 if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...)
936 #endif
937
938 2. Within an individual package a header file,
939 "${PKG}_OPTIONS.h", is used to set CPP flags
940 specific to that package. It is not recommended
941 to include this file in "CPP_OPTIONS.h".
942
943
944 Package Boot Sequence
945 =====================
946
947 Calls to package routines within the core code timestepping
948 loop can vary. However, all packages follow a required
949 "boot" sequence outlined here:
950
951 1. S/R PACKAGES_BOOT()
952 :
953 CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
954
955
956 2. S/R PACKAGES_READPARMS()
957 :
958 #ifdef ALLOW_${PKG}
959 if ( use${Pkg} )
960 & CALL ${PKG}_READPARMS( retCode )
961 #endif
962
963 2. S/R PACKAGES_CHECK()
964 :
965 #ifdef ALLOW_${PKG}
966 if ( use${Pkg} )
967 & CALL ${PKG}_CHECK( retCode )
968 #else
969 if ( use${Pkg} )
970 & CALL PACKAGES_CHECK_ERROR('${PKG}')
971 #endif
972
973 3. S/R PACKAGES_INIT()
974 :
975 #ifdef ALLOW_${PKG}
976 if ( use${Pkg} )
977 & CALL ${PKG}_INIT( retCode )
978 #endif
979
980
981 Description
982 ===========
983
984 - ${PKG}_READPARMS()
985 is responsible for reading
986 in the package parameters file data.${pkg}, and storing
987 the package parameters in "${PKG}.h".
988 -> called in INITIALISE_FIXED
989
990 - ${PKG}_CHECK()
991 is responsible for validating
992 basic package setup and inter-package dependencies.
993 ${PKG}_CHECK can import other package parameters it may
994 need to check. This is done through header files "${PKG}.h".
995 It is assumed that parameters owned by other packages
996 will not be reset during ${PKG}_CHECK().
997 -> called in INITIALISE_FIXED
998
999 - ${PKG}_INIT()
1000 is responsible for completing the
1001 internal setup of a package. This routine is called after
1002 the core model state has been completely initialised
1003 but before the core model timestepping starts.
1004 -> called in INITIALISE_VARIA
1005
1006 Summary
1007 =======
1008
1009 - CPP options:
1010 -----------------------
1011 * ALLOW_${PKG} include/exclude package for compilation
1012
1013 - FORTRAN logical:
1014 -----------------------
1015 * use${Pkg} enable package for execution at runtime
1016 -> declared in PARAMS.h
1017 * ${Pkg}IsOn for package cross-dependency check
1018 -> declared in ${PKG}.h
1019 N.B.: Not presently used!
1020
1021 - header files
1022 -----------------------
1023 * ${PKG}_OPTIONS.h has further package-specific CPP options
1024 * ${PKG}.h package-specific common block variables, fields
1025
1026 - FORTRAN source files
1027 -----------------------
1028 * ${pkg}_readparms.F reads parameters from file data.${pkg}
1029 * ${pkg}_check.F checks package dependencies and consistencies
1030 * ${pkg}_init.F initialises package-related fields
1031 * ${pkg}_... .F package source code
1032
1033 - parameter file
1034 -----------------------
1035 * data.${pkg} parameter file
1036 </programlisting>
1037
1038 </sect1>
1039
1040
1041 </article>

  ViewVC Help
Powered by ViewVC 1.1.22