/[MITgcm]/MITgcm/doc/devel_HOWTO.sgml
ViewVC logotype

Diff of /MITgcm/doc/devel_HOWTO.sgml

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.1 by edhill, Mon Aug 4 21:09:05 2003 UTC revision 1.19 by jmc, Fri Jan 5 20:47:02 2018 UTC
# Line 1  Line 1 
1  <!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">  <!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
   
 <article id="MITgcm-Development-HOWTO">  
   
2  <!--  <!--
3  Build commands:   $Header$
4    db2pdf -d ldp.dsl devel_HOWTO.sgml   $Name$
   db2html -d ./ldp.dsl devel_HOWTO.sgml  
5  -->  -->
6    
7    <article id="MITgcm-Development-HOWTO">
8    
9    <articleinfo>    <articleinfo>
10      <title>MITgcm Development HOWTO</title>      <title>MITgcm Development HOWTO</title>
11    
# Line 22  Build commands: Line 20  Build commands:
20      <revhistory>      <revhistory>
21        <revision>        <revision>
22          <revnumber>0.01</revnumber>          <revnumber>0.01</revnumber>
23          <date>2003-0-07</date>          <date>2003-08-07</date>
24          <authorinitials>eh3</authorinitials>          <authorinitials>eh3</authorinitials>
25          <revremark>          <revremark>
26            Initial version.            Initial version.
27          </revremark>          </revremark>
28        </revision>        </revision>
29          <revision>
30            <revnumber>0.02</revnumber>
31            <date>2010-01-21</date>
32            <authorinitials>jmc</authorinitials>
33            <revremark>
34              Update links.
35            </revremark>
36          </revision>
37          <revision>
38            <revnumber>0.03</revnumber>
39            <date>2010-04-25</date>
40            <authorinitials>jmc</authorinitials>
41            <revremark>
42              Add subsection "Developer settings" (under CVS Repository).
43            </revremark>
44          </revision>
45          <revision>
46            <revnumber>0.04</revnumber>
47            <date>2011-04-24</date>
48            <authorinitials>jmc</authorinitials>
49            <revremark>
50              Update subsection "The verification suite".
51            </revremark>
52          </revision>
53      </revhistory>      </revhistory>
54    
55      <abstract>      <abstract>
# Line 43  Build commands: Line 65  Build commands:
65      <sect2>      <sect2>
66        <title>New Versions of This Document</title> <para>You can        <title>New Versions of This Document</title> <para>You can
67        obtain the latest version of this document <ulink        obtain the latest version of this document <ulink
68        url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in        url="http://mitgcm.org/public/docs.html">online</ulink> in
69        various formats.</para>        various formats.</para>
70      </sect2>      </sect2>
71      <sect2>      <sect2>
# Line 61  Build commands: Line 83  Build commands:
83      <sect2>      <sect2>
84        <title>User Manual</title>        <title>User Manual</title>
85    
86        <para>Before jumping into        <para>Before jumping into development, please familiarize yourself with
87        development, please familiarize yourself with the MITgcm user          the <ulink url="http://mitgcm.org/public/docs.html"> MITgcm user manual
88        manual which is available <ulink          </ulink>.  This document contains volumes of useful information and is
89        url="http://mitgcm.org/">on the main web page</ulink>.  This          included here by reference.</para>
       document contains volumes of useful information and is included  
       here by reference.</para>  
90    
91        <para>Also, a "snapshot" or<ulink        <!--
92          <para>Also, a "snapshot" or <ulink
93        url="http://mitgcm.org/dev_docs/">development version</ulink> of        url="http://mitgcm.org/dev_docs/">development version</ulink> of
94        the user manual may be available, though this is only put on the        the user manual may be available, though this is only put on the
95        web for testing purposes.</para>        web for testing purposes.</para>
96          -->
97      </sect2>      </sect2>
98    
99      <sect2>      <sect2>
# Line 100  Build commands: Line 122  Build commands:
122    
123    <sect1 id="cvs">    <sect1 id="cvs">
124      <title>CVS Repository</title>      <title>CVS Repository</title>
125    
126      <sect2>      <sect2>
127        <title>Layout</title>        <title>Layout</title>
128    
# Line 110  Build commands: Line 133  Build commands:
133        others.  The tree currently resembles:</para>        others.  The tree currently resembles:</para>
134    
135  <programlisting>gcmpack/  <programlisting>gcmpack/
136    MITgcm-contrib        contributed code    CVSROOT               -hidden-
   CS-regrid             goes into utils  
   cvspolicy.html        -save-  
   CVSROOT               -save-  
   development           experimental stuff  
   manual                -save-  
   misc                  -?-  
137    
138    MITgcm                code    MITgcm                code
139         adjoint                  fold into genmake         bin                      empty
140         bin                      stub for ecco build         doc                      basic developpment documentation
141         compare01                old from 20th century         eesupp                   execution environment support code (wrapper)
142         diags                    timeave f77 in pkgs now         exe                      empty
143         doc                      tags -- connect to real docs?         jobs                     runtime shell scripts for
144         eesupp                   cnh?                                    various platforms (not maintained)
145         exe                      ecco user build         lsopt                    line search
146      *- jobs                     runtime shell scripts for         model                    main dynamics (core)
147      |                             various platforms         optim                    line search interface
148      |  lsopt                    line search         pkg                      alternate and optional numerics, etc.
149     m|  model                    main dynamics (core)         tools                    scripts to build (and test)
150     e|    optimization_drivers   ?         utils                    pre/post processing tools (matlab, ..)
151     r|  optim                    line search interface         verification             standard regression tests + examples
152     g|  pkg                      alternate and optional numerics, etc.                                        + documented examples (tutorials)
153     e*- tools         tutorial_examples        (only in release1 branch)
    ?|  tutorial_examples        documented tests  
     |                             only populated on release1 branch  
     |                             and not validated during "testscript"  
     *- utils  
        verification             std tests  
154    
155      MITgcm_contrib        contributed code
156    
157    mitgcmdoc -> manual   -remove-    acesgrid.org          build acesgrid web site
158      development           experimental stuff
159      gfd_lab               -?-
160      manual                source of MITgcm documentation
161    mitgcm.org            build web site    mitgcm.org            build web site
162    models                -?-    old_develop           old and early development source
163    packages              -?-         misc                 -?-
164    preprocess            -?-         models               -?-
165    tmp                   -?-         packages             -?-
166           preprocess           -?-
167      pdfs                  some pdfs
168      planetinabottle.org   unfinished web site
169      www.ecco-group.org    build ecco web site ?
170  </programlisting>  </programlisting>
171    
172     <!--
173        <para>Efforts are underway to reduce the complexity.</para>        <para>Efforts are underway to reduce the complexity.</para>
174    -->
175    
176      </sect2>      </sect2>
177    
# Line 188  Build commands: Line 210  Build commands:
210        <title>Branches</title>        <title>Branches</title>
211    
212        <para>As shown in the online <ulink        <para>As shown in the online <ulink
213        url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated        url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph">
214        tree</ulink>, the MITgcm codebase is split into to two branches        ViewCVS-generated tree</ulink>, the MITgcm codebase is split into
215        or "lines" under which development proceeds.  These two lines        branches or "lines" under which development proceeds.  The main line
216        are referred to as the "MAIN" and "ecco" versions of the code.        of development is referred to as the "MAIN" version of the code.
       While not identical, the bulk of the MAIN and ecco lines are  
       composed of files from the same codebase.  
217        </para>        </para>
218    
219        <para>Periodically, a "Release" branch is formed from the "MAIN"        <para>Periodically, a "Release" branch is formed from the "MAIN"
220        development branch.  This is done in order to create a        development branch.  This is done in order to create a relatively stable
221        relatively stable reference point for both users and developers.        reference point for both users and developers.  The intent is that once a
222        The intent is that once a relese branch has been created, only        release branch has been created, only bug-fixes will be added to it.
223        bug-fixes will be added to it.  Meanwhile, development (which        Meanwhile, development (which might "break" or otherwise render invalid
224        might "break" or otherwise render invalid the documentation,        the documentation, tutorials, and/or examples contained within a release
225        tutorials, and/or examples contained within a release branch) is        branch) is allowed to continue along the MAIN line.</para>
226        allowed to continue along the MAIN and ecco lines.</para>      </sect2>
227    
228        <sect2>
229          <title> Developer settings </title>
230    
231          <para>CVS is a convenient tool to keep up-to-date a personal copy of the
232          MITgcm code (see: <ulink url="http://mitgcm.org/public/using_cvs.html">
233          using CVS </ulink>). The same tool is used by developers to
234          incorporate any change into the repository. However, this later
235          function requires specific settings, as detailed here after</para>
236          <orderedlist>
237            <listitem>
238              <para> You will need an account (login access) to the server
239               "mitgcm.org" (curently: <filename>mitgcmcvs.mit.edu</filename>)
240               with the proper group setting (e.g., group "gcmctrb" to add/modify
241               code into MITgcm_contrib).
242               This kind of account is granted only upon well motivated request
243               (we recommend to ask your senior MITgcm-collaborator to send such
244               request to marshall-admin at techsquare.com with Cc to Chris Hill
245               for approval).</para>
246              <para> The access to the server <filename>mitgcm.org</filename> is
247               through ssh-key authorization which will need to be set properly on
248               both side (on your local machine and on your server account).
249               You need to be able to ssh to <filename>mitgcm.org</filename>
250               (or <filename>ssh MY_USER_NAME@mitgcm.org</filename>
251               in case of different user-name on both sides) to proceed further.</para>
252            </listitem>
253    
254            <listitem>
255              <para> You need to register to the
256            <ulink url="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-cvs">
257           mitgcm-cvs </ulink> mailing list.
258              This ensures that other developers will receive email notification
259               when you make changes; you will also receive such email
260               when others make changes to the repository.
261              </para>
262            </listitem>
263    
264            <listitem>
265              <para> It is highly recommended that you register also to the
266            <ulink url="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel">
267           mitgcm-devel </ulink> mailing list (expect a short delay for
268           this request to be processed).
269              This list is intended for developer discussions.
270              </para>
271            </listitem>
272    
273            <listitem>
274              <para> The standard CVS-anonymous mode (using "cvsanon",
275            as mentionned <ulink url="http://mitgcm.org/public/source_code.html">
276            here </ulink>) does not provide check-in ("cvs commit") permission.
277             Instead, you will need to set our CVS environment as follow:</para>
278    <screen>
279      $ export CVS_RSH=ssh
280      $ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack'
281    </screen>
282              <para> The reason for such limitation is that when downloading a directory,
283                e.g.: <filename>myCopy</filename>, from the CVS repository (e.g.,
284                <filename>MITgcm_contrib/thisPart</filename>) using the command:</para>
285    <screen>
286      $ cvs co -P -d myCopy MITgcm_contrib/thisPart
287    </screen>
288              <para> the type of CVS environment which has been used
289               is stored in the file <filename>myCopy/CVS/Root</filename>.
290               This prevent to re-use, for cvs-commit purpose,
291               a cvs local copy (<filename>myCopy</filename>) which was obtained
292               using the CVS anonymous mode.</para>
293            </listitem>
294    
295            <listitem>
296              <para> At this stage, you should be able to send your modified source
297              file (e.g., <filename>src_file</filename>) from your local copy directory
298              (<filename>myCopy</filename>) to the CVS repository
299              (<filename>MITgcm_contrib/thisPart</filename>) using the command
300              "cvs commit":</para>
301    <screen>
302      $ cd myCopy
303      $ cvs -n update        (optional; check if new changes have been made)
304      $ cvs diff src_file    (optional; list your changes)
305      $ cvs commit src_file
306    </screen>
307              <para> It is essential that you provide a short description of the
308              changes you made to <filename>src_file</filename> as you check-in
309              this file (the "cvs commit" command automatically opens your standard
310              editor for this purpose).</para>
311              <para> Note:
312               Please ignore the following warnings that the "cvs commit" command
313               produces if you are not part of the "gcmpack" group:
314    <screen>
315       cvs commit: failed to create lock directory for `/u/gcmpack/CVSROOT'
316       (/u/gcmpack/CVSROOT/#cvs.history.lock): Permission denied
317       cvs commit: failed to obtain history lock in repository `/u/gcmpack'
318    </screen>
319              These warnings are not affecting the changes you made to the CVS repository.
320              </para>
321            </listitem>
322    
323          </orderedlist>
324    
325      </sect2>      </sect2>
326    
327      <sect2>      <sect2>
328        <title>Tagging</title>        <title>Main code development</title>
329          <para><emphasis>(formerly named "Tagging" ; this section needs an update)
330        <para>The intent of tagging is to create "known-good"          </emphasis></para>
331        checkpoints that developers can use as references.  
332        Traditionally, MITgcm tagging has maintained the following        <para>The intent of tagging is to create "known-good" checkpoints that
333        conventions:</para>        developers can use as references.  Traditionally, MITgcm tagging has
334          maintained the following conventions:</para>
335    
336        <orderedlist>        <orderedlist>
337          <listitem>          <listitem>
338            <para>Developer checks out code into a local CVS-managed            <para>Developer checks out code into a local CVS-managed directory,
339            directory, makes various changes/additions, tests these            makes various changes/additions, tests these edits, and eventually
340            edits, and eventually reaches a point where (s)he is            reaches a point where (s)he is satisfied that the changes form a new
341            satisfied that the changes form a new "useful" point in the            "useful" point in the evolution of the code.</para>
           evolution of the code.</para>  
342          </listitem>          </listitem>
343    
344          <listitem>          <listitem>
345            <para>The developer then runs the <ulink            <para>The developer then runs the <ulink
346            url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink>            url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport">
347            shell script to see if any problems are introduced.  While            testreport</ulink> shell script to see if any problems are introduced.
348            not intended to be exhaustive, the test cases within the            While not intended to be exhaustive, the test cases within the
349            verification directory do provide some indication whether            verification directory do provide some indication whether gross errors
350            gross errors have been introduced.            have been introduced.
351            </para>            </para>
352          </listitem>          </listitem>
353    
# Line 239  Build commands: Line 357  Build commands:
357            then:</para>            then:</para>
358            <orderedlist>            <orderedlist>
359              <listitem>              <listitem>
360                <para>adds a "checkpointXY_pre" comment (where X is a                <para>adds a "checkpointXY_pre" comment (where X is a checkpoint
361                checkpoint number and Y is a letter) to the <ulink                number and Y is a letter) to the <ulink
362                url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink>                url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index">
363                file and checks it into the CVS repository</para>                tag-index</ulink> file and checks it into the CVS
364                  repository</para>
365              </listitem>              </listitem>
366              <listitem>              <listitem>
367                <para>submits the set of changes to the CVS repository                <para>submits the set of changes to the CVS repository and adds
368                and adds comments to <filename>tag-index</filename>                comments to <filename>tag-index</filename> describing what the
369                describing what the changes are along with a matching                changes are along with a matching "checkpointXY_post" entry</para>
               "checkpointXY_post" entry</para>  
370              </listitem>              </listitem>
371            </orderedlist>            </orderedlist>
372          </listitem>          </listitem>
373        </orderedlist>        </orderedlist>
374    
375        <para>The result of this tagging procedure is a sequence of        <para>The result of this tagging procedure is a sequence of development
376        development checkpoints with comments which resembles:</para>        checkpoints with comments which resembles:</para>
377    
378  <programlisting>  <programlisting>
379  checkpoint50e_post  checkpoint50e_post
380  o make KPP work with PTRACERS  o make KPP work with PTRACERS
381   - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is   - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is
382     nearly a copy of kpp_transport_s     nearly a copy of kpp_transport_s
383   - there is no analogue to SurfaceTendencyS, so I have to use   - there is no analogue to SurfaceTendencyS, so I have to use
384     gPtr(of the surface layer) instead     gPtr(of the surface layer) instead
385  o add a new platform SunFire+mpi (SunFire 15000) to genmake  o add a new platform SunFire+mpi (SunFire 15000) to genmake
386  checkpoint50e_pre  checkpoint50e_pre
387    
388  checkpoint50d_post  checkpoint50d_post
389  o change kpp output from multiple-record state files to single-record state  o change kpp output from multiple-record state files to single-record state
390    files analogous to write_state.F    files analogous to write_state.F
391  o reduce the output frequency of cg3d-related stuff to the monitor frequency,  o reduce the output frequency of cg3d-related stuff to the monitor frequency,
392    analogous to the cg2d-related output.    analogous to the cg2d-related output.
393  o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,  o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,
394    so that writing to internal file fn (with length 512) fails.    so that writing to internal file fn (with length 512) fails.
395  checkpoint50d_pre  checkpoint50d_pre
396  </programlisting>  </programlisting>
397    
398        <para>This information can be used to refer to various stages of        <para>This information can be used to refer to various stages of the code
399        the code development.  For example, bugs can be traced to        development.  For example, bugs can be traced to individual sets of CVS
400        individual sets of CVS checkins based upon their first        checkins based upon their first appearance when comparing the results from
401        appearance when comparing the results from different        different checkpoints.</para>
       checkpoints.</para>  
402    
403      </sect2>      </sect2>
404    </sect1>    </sect1>
405    
406      <sect1 id="coding">
407        <title>Coding for MITgcm</title>
408    
409        <sect2 id="build_tools">
410          <title>Build Tools</title>
411    
412          <para>Many Open Source projects use the "GNU Autotools" to help streamline
413            the build process for various Unix and Unix-like architectures.  For a
414            user, the result is the common "configure" (that is,
415            "<filename>./configure && make && make install</filename>") commands.
416            For MITgcm, the process is similar.  Typical commands are:</para>
417    
418    <screen>
419      $ genmake2 -mods=../code
420      $ make depend
421      $ make
422    </screen>
423    
424          <para>The following sections describe the individual steps in the build
425            process.</para>
426    
427          <sect3 id="genmake">
428            <title>The <filename>genmake2</> Utility</title>
429    
430            <para><emphasis>(Note: the older <filename>genmake</>
431                has been replaced by <filename>genmake2</>)</emphasis></para>
432    
433            <para>The first step in any MITgcm build is to create a Unix-style
434              <filename>Makefile</filename> which will be parsed by
435              <filename>make</filename> to specify how to compile the MITgcm source
436              files.  For more detailed descriptions of what the make tools are and
437              how they are used, please see:</para>
438    
439            <itemizedlist>
440              <listitem>
441                <para><ulink url="http://www.gnu.org/software/make/make.html">
442                    http://www.gnu.org/software/make/make.html</></para>
443              </listitem>
444              <listitem>
445                <para><ulink url="http://www.oreilly.com/catalog/make2/">
446                    http://www.oreilly.com/catalog/make2/</></para>
447              </listitem>
448            </itemizedlist>
449    
450            <para>Genmake can often be invoked successfully with a command line as
451            simple as:</para>
452    
453    <screen>
454      $ genmake2 -mods=../code
455    </screen>
456    
457            <para>However, some systems (particularly commercial Unixes that lack a
458              more modern "/bin/sh" implementation or that have shells installed in
459              odd locations) may require an explicit shell invocation such as one of
460              the following: </para>
461    
462    <screen>
463      $ /usr/bin/sh genmake2 -make=gmake  -mods=../code
464      $ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code
465    </screen>
466    
467            <para>The genmake2 code has been written in a Bourne and BASH (v1)
468            compatible syntax so it should work with most "sh" and all recent "bash"
469            implementations.</para>
470    
471            <para>As the name implies, <filename>genmake2</filename> generates a
472              <filename>Makefile</filename>.  It does so by first parsing the
473              information supplied from the following sources</para>
474    
475            <orderedlist>
476              <listitem>
477                <para>a <filename>gemake_local</filename> file in the current
478                  directory</para>
479              </listitem>
480              <listitem>
481                <para>directly from command-line options</para>
482              </listitem>
483              <listitem>
484                <para>an "options file" as specified by the command-line option
485                  <filename>-optfile='FILENAME'</filename></para>
486              </listitem>
487              <listitem>
488                <para> a <filename>packages.conf</filename> file (in the current
489                  directory or in one of the "MODS" directories, see below)
490                  which contains the specific list of packages to compile
491                </para>
492              </listitem>
493            </orderedlist>
494    
495            <para>then checking certain dependency rules (the package dependencies),
496              and finally writing a <filename>Makefile</filename> based upon the
497              source code that it finds.  For convenience within various Unix
498              shells, <filename>genmake2</> supports both "long"- and "short"-style
499              options.  A complete list of the available options can be obtained
500              from:</para>
501    
502    <screen>
503      $ genmake2 -help
504    </screen>
505    
506            <para>The most important options for <filename>genmake2</> are:</para>
507    
508            <variablelist>
509    
510              <varlistentry>
511                <term><filename>--optfile=/PATH/FILENAME</></term>
512    
513                <listitem>
514                  <para>This specifies the "options file" that should be used for a
515                    particular build.  The options file is a convenient and
516                    machine-indepenent way of specifying parameters such as the
517                    FORTRAN compiler (<filename>FC=</>), FORTRAN compiler
518                    optimization flags (<filename>FFLAGS=</>), and the locations of
519                    various platform- and/or machine-specific tools
520                    (eg. <filename>MAKEDEPEND=</>).  As with <filename>genmake2</>,
521                    all options files should be written to be compatible with
522                    Bourne--shell ("sh" or "BASH v1") syntax.  Examples of various
523                    options files can be found in
524                    <filename>$ROOTDIR/tools/build_options</>.</para>
525    
526                  <para>If no "optfile" is specified (either through the command lin
527                    or the environment variable), genmake2 will try to make a
528                    reasonable guess from the list provided in
529                    <filename>$ROOTDIR/tools/build_options</>.  The method used for
530                    making this guess is to first determine the combination of
531                    operating system and hardware (eg. "linux_ia32") and then find a
532                    working Fortran compiler within the user's path.  When these
533                    three items have been identified, genmake2 will try to find an
534                    optfile that has a matching name. </para>
535    
536                  <para>Everyone is encouraged to submit their options files to the
537                    MITgcm project for inclusion (please send to
538                    <email>MITgcm-support@mitgcm.org</email>).  We are particularly
539                    grateful for options files tested on new or unique
540                    platforms!</para>
541                </listitem>
542    
543              </varlistentry>
544    
545              <varlistentry>
546                <term><filename>-adof=/path/to/file</></term>
547                <term><filename>-adoptfile=/path/to/file</></term>
548                <listitem>
549                  <para>This option specifies the "adjoint" or automatic
550                    differentiation options file to be used.  The file is analogous
551                    to the "optfile" defined above but it specifies information for
552                    the AD build process.  The default file is located in <filename>
553                    $ROOTDIR/tools/adjoint_options/adjoint_default </> and it
554                    defines the "TAF" and "TAMC" compilers.  An alternate version is
555                    also available at <filename>
556                    $ROOTDIR/tools/adjoint_options/adjoint_staf </> that selects the
557                    newer "STAF" compiler.  As with any compilers, it is helpful to
558                    have their directories listed in your $PATH environment
559                    variable.</para>
560                </listitem>
561              </varlistentry>
562    
563              <varlistentry>
564                <term><filename>-mods=DIR</></term>
565                <term><filename>-mods='DIR1 [DIR2 ...]'</></term>
566                <listitem>
567                  <para>This option specifies a list of directories containing
568                    "modifications".  These directories contain files with names
569                    that may (or may not) exist in the main MITgcm source tree but
570                    will be overridden by any identically-named sources within the
571                    "MODS" directories.  The order of precedence for this
572                    "name-hiding" is as follows:</para>
573                  <itemizedlist>
574                    <listitem><para>"MODS" directories (in the order given)
575                      </para></listitem>
576                    <listitem><para>Packages either explicitly specified or
577                        provided by default (in the order given)</para></listitem>
578                    <listitem><para>Packages included due to package dependencies
579                        (in the order that that package dependencies are
580                        parsed)</para></listitem>
581                    <listitem><para>The "standard dirs" (which may have been
582                        specified by the "-standarddirs" option)</para></listitem>
583                  </itemizedlist>
584                </listitem>
585              </varlistentry>
586    
587              <varlistentry>
588                <term><filename>-pgroups=/PATH/FILENAME</></term>
589                <listitem>
590                  <para>This option specifies the file where package groups are
591                  defined.  If not set, the package-groups definition will
592                  be read from
593                  <filename>$ROOTDIR/pkg/pkg_groups</>.</para>
594                  <para>
595                  It also contains the default list of packages (defined
596                  as the group <filename>"default_pkg_list"</>) which is used
597                  when no specific package list (file: <filename>packages.conf</>)
598                  is found in current directory or in any "MODS" directory.
599                  </para>
600                </listitem>
601              </varlistentry>
602    
603              <varlistentry>
604                <term><filename>-pdepend=/PATH/FILENAME</></term>
605    
606                <listitem>
607                  <para>This specifies the dependency file used for packages.  If
608                  not specified, the default dependency file is
609                  <filename>$ROOTDIR/pkg/pkg_depend</>.  The syntax for this file is
610                  parsed on a line-by-line basis where each line containes either a
611                  comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule
612                  where the "+" or "-" symbol specifies a "must be used with" or a
613                  "must not be used with" relationship, respectively.  If no rule is
614                  specified, then it is assumed that the two packages are compatible
615                  and will function either with or without each other.</para>
616                </listitem>
617              </varlistentry>
618    
619              <varlistentry>
620                <term><filename>-make=/path/to/gmake</></term>
621                <listitem>
622                  <para>Due to the poor handling of soft-links and other bugs common
623                    with the <filename>make</> versions provided by commercial Unix
624                    vendors, GNU <filename>make</filename> (sometimes called
625                    <filename>gmake</filename>) should be preferred.  This option
626                    provides a means for specifying the make program to be
627                    used.</para>
628                </listitem>
629              </varlistentry>
630    
631            </variablelist>
632            
633            <para>A successful run of <filename>genmake2</> will produce a
634              <filename>Makefile</>, a <filename>PACKAGES_CONFIG.h</> file, and
635              various convenience files used for the automatic differentiation
636              process.</para>
637    
638            <para>In general, it is best to use <filename>genmake2</> on a "clean"
639              directory that is free of all source (*.[F,f],*.[F,f]90) and header
640              (*.h,*.inc) files.  Generally, this can be accomplished in an
641              "un-clean" directory by running "make Clean" followed by "make
642              makefile".</para>
643    
644          </sect3>
645    
646          <sect3 id="makefile_use">
647            <title>Using the <filename>Makefile</></title>
648    
649            <para>Once a <filename>Makefile</> has been created using
650              <filename>genmake2</>, one can build a "standard" (forward
651              simulator) executable using:</para>
652    
653    <screen>
654      $ make Clean
655      $ make depend
656      $ make
657    </screen>
658    
659            <para>The "make Clean" step will remove any stale source files, include
660              files, and links.  It is strongly recommended for "un-clean"
661              directories which may contain the (perhaps partial) results of
662              previous builds. Such "debris" can interfere with the next stage of
663              the build.
664              A more agressive cleaning option, "make CLEAN", can be used to also
665              remove the executable and output files from a previous run.</para>
666    
667            <para>The "make depend" step will create a large number of symbolic
668              links from the local directory to the source file locations.  It also
669              parses these files and creates an extensive list of dependencies
670              within the <filename>Makefile</> itself.  The links that exist at this
671              stage are mostly "large F" files (*.F and *.F90) that need to be
672              processed by a C preprocessor ("CPP").  Since "make depend" edits the
673              <filename>Makefile</>, it is important not to skip this step!</para>
674    
675            <para>The final "make" invokes the C preprocessor to produce the "little
676              f" files (*.f and *.f90) and then compiles them to object code using
677              the specified FORTRAN compiler and options.  An intermediate script is
678              often used during this stage to further process (usually, make simple
679              substitutions) custom definitions such as variable types within the
680              source files.  This additional stage is necessary in order to overcome
681              some of the inconsistencies in the sizes of objects (bytes) between
682              different compilers. The result of the build process is an executable
683              with the name <filename>mitgcmuv</>.</para>
684    
685            <para>In addition to the forward simulator described above, the
686              <filename>Makefile</> also has a number of targets that can be used to
687              produce various adjoint and tangent-linear builds for optimization and
688              other parameter-sensitivity problems.  The additional targets within
689              the <filename>Makefile</> are:</para>
690    
691            <variablelist>
692    
693              <varlistentry>
694                <term><filename>make adall</></term>
695                <listitem>
696                  <para>This target produces an <filename>mitgcmuv_ad</> executable
697                    using the <filename>taf</> or <filename>staf</> adjoint
698                    compiler.  See the <filename>genmake2</> "-adof" option for
699                    compiler selection.</para>
700                </listitem>
701              </varlistentry>
702    
703              <varlistentry>
704                <term><filename>make ftlall</></term>
705                <listitem>
706                  <para>Similar to <filename>make adall</> above, this
707                    produces...</para>
708                </listitem>
709              </varlistentry>
710    
711            </variablelist>
712    
713            <para>Please report any compilation failures or other build problems to
714              the <email>MITgcm-support@mitgcm.org</email> list.</para>
715    
716          </sect3>
717    
718        </sect2>
719    
720        <sect2 id="verification">
721          <title>The Verification Suite</title>
722    
723          <para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</>
724            directory) includes many (> 90) examples intended for regression
725            testing.  Each one of these test-experiment directories contains "known-good"
726            output files along with all the input (including both code and data
727            files) required for their re-calculation.
728            Also included in <filename>$ROOTDIR/verification/</> is the shell
729            script <filename>testreport</> to perform regression tests.</para>
730    
731          <sect3 id="test-experiment">
732            <title>Test-experiment Directory Content</title>
733    
734              <para> Each test-experiment directory (<filename>TESTDIR</>) contains
735              several standard subdirectories and files which <filename>testreport</>
736              recognizes and uses when running a regression test.
737              The directories/files that <filename>testreport</> uses are different
738              for a forward test and an adjoint test (<filename>testreport -adm</>)
739              and some test-experiments are set-up for only one type of regression
740              test whereas others allow both types of tests (forward and adjoint).</para>
741              <para>Also some test-experiment allows, using the same MITgcm executable,
742              to perform multiple tests using different parameters and input files,
743              with a primary input set-up
744              (<filename>input/ </> or <filename>input_ad/ </>)
745              and corresponding results (<filename>results/output.txt</> or
746              <filename>results/output_adm.txt</>) and with one or several secondary
747              inputs (<filename>input.OTHER/ </> or <filename>input_ad.OTHER/ </>)
748              and corresponding results (<filename>results/output.OTHER.txt </>
749              or <filename>results/output_adm.OTHER.txt</>).</para>
750            <variablelist>
751    
752              <varlistentry>
753                <term>directory <filename>TESTDIR/results/</></term>
754                <listitem>
755                  <para>contains reference standard output used for test comparison.
756                    <filename>results/output.txt</> and
757                    <filename>results/output_adm.txt</>
758                    correspond respectively to primary forward and adjoint test
759                    run on the reference platform (currently
760                    <filename>baudelaire.csail.mit.edu</>)
761                    on one processor (no MPI, single thread)
762                    using the reference compiler (curently the GNU fortran
763                    compiler <filename>gfortran</>).
764                    The presence of these files determines whether or not
765                    <filename>testreport</> is testing or skipping
766                    this test-experiment.
767                    Reference standard output for secondary tests
768                    (<filename>results/output.OTHER.txt</>
769                    or <filename>results/output_adm.OTHER.txt</>) are
770                    also expected here.</para>
771                  <para>The test comparison involves few model variables output, which are,
772                    by default and for a forward test, the 2-D solver initial residual
773                    (<filename>cg2d_init_res</>) and 3-D state variables (T,S,U,V)
774                    monitor output, and, by default and for an adjoint test, the
775                    cost-function and gradient-check. However, some test-experiments
776                    use some package-specific variable/monitor output according to
777                    the file <filename>TESTDIR/input[_ad][.OTHER]/tr_checklist</>
778                    specification.</para>
779                </listitem>
780              </varlistentry>
781    
782              <varlistentry>
783                <term>directory <filename>TESTDIR/build/</></term>
784                <listitem>
785                  <para> initially empty directory where <filename>testreport</>
786                    will build the MITgcm executable for forward and adjoint test.
787                    It might contains an experiment specific
788                    <filename>genmake_local</> file (see <xref linkend="genmake">).
789                    </para>
790                  <para> Note that the original <filename>code[_ad]/SIZE.h_mpi</>
791                    is not directly used as "SIZE.h" to build an MPI-executable ;
792                    instead, a local copy <filename>build/SIZE.h.mpi</> is derived
793                    from <filename>code[_ad]/SIZE.h_mpi</>
794                    by adjusting the number of processors (nPx,nPy) according to
795                    <filename>NUMBER_OF_PROCS</> (see <xref linkend="testreport">,
796                    <filename>testreport -MPI</>) ; then it is linked to "SIZE.h"
797                    (<filename> ln -s SIZE.h.mpi SIZE.h </>) before building
798                    the MPI-executable.</para>
799                </listitem>
800              </varlistentry>
801    
802              <varlistentry>
803                <term>directory <filename>TESTDIR/code/</></term>
804                <listitem>
805                  <para>contains the test-experiment specific source code
806                    used to build the MITgcm executable (<filename>mitgcmuv</>)
807                    for forward-test (using <filename>genmake2 -mods=../code</>).
808                    </para>
809                    <para> It can also contain specific source files with the suffix
810                    "<filename>_mpi</>" to be used
811                    <!--
812                    in the <filename>TESTDIR/build/</> directory
813                    -->
814                    in place of the corresponding file (without suffix)
815                    for an MPI test (see <xref linkend="testreport">).
816                    The presence or absence of <filename>SIZE.h_mpi</>
817                    determines whether or not an MPI test on this
818                    test-experiment is performed or skipped.</para>
819                </listitem>
820              </varlistentry>
821    
822              <varlistentry>
823                <term>directory <filename>TESTDIR/code_ad/</></term>
824                <listitem>
825                  <para>contains the test-experiment specific source code
826                    used to build the MITgcm executable (<filename>mitgcmuv_ad</>)
827                    for adjoint-test (using <filename>genmake2 -mods=../code_ad</>).
828                    It can also contain specific source files with the suffix
829                    "<filename>_mpi</>" (see above).</para>
830                </listitem>
831              </varlistentry>
832    
833              <varlistentry>
834                <term>directory <filename>TESTDIR/input/</></term>
835                <listitem>
836                  <para>contains the input and parameter files
837                    used to run the primary forward test of this test-experiment.
838                    </para>
839                  <para>It can also contain specific parameter files with the suffix
840                    "<filename>.mpi</>" to be used
841                    <!--
842                    in the <filename>TESTDIR/run/</> directory
843                    -->
844                    in place of the corresponding file (without suffix) for MPI
845                    test, or with suffix "<filename>.mth</>" to be used for
846                    multi-threaded test (see <xref linkend="testreport">).
847                    The presence or absence of <filename>eedata.mth</>
848                    determines whether or not a multi-threaded test on this
849                    test-experiment is performed or skipped.</para>
850                  <para>To save disk space and reduce downloading time, multiple
851                    copies of the same input file is avoided by using a shell
852                    script <filename>prepare_run</>.
853                    When such a script is found in <filename>TESTDIR/input/ </>,
854                    <filename>testreport</> run this script in directory
855                    <filename>TESTDIR/run/ </> after linking all the input file
856                    from <filename>TESTDIR/input/ </>.
857                    </para>
858                </listitem>
859              </varlistentry>
860    
861              <varlistentry>
862                <term>directory <filename>TESTDIR/input_ad/</></term>
863                <listitem>
864                  <para>contains the input and parameter files
865                    used to run the primary adjoint test of this test-experiment.
866                    It can also contain specific parameter files with the suffix
867                    "<filename>.mpi</>" and shell script <filename>prepare_run</>
868                    as described above.</para>
869                </listitem>
870              </varlistentry>
871    
872              <varlistentry>
873                <term>directory <filename>TESTDIR/input.OTHER/</></term>
874                <listitem>
875                  <para>contains the input and parameter files
876                    used to run the secondary OTHER forward test of this test-experiment.
877                    It can also contain specific parameter files with suffix
878                    "<filename>.mpi</>" or "<filename>.mth</>" and shell script
879                    <filename>prepare_run</> (see above).</para>
880                    <para> The presence or absence the file <filename>eedata.mth</>
881                    determines whether or not a secondary multi-threaded test on this
882                    test-experiment is performed or skipped.</para>
883                </listitem>
884              </varlistentry>
885    
886              <varlistentry>
887                <term>directory <filename>TESTDIR/input_ad.OTHER/</></term>
888                <listitem>
889                  <para>contains the input and parameter files
890                    used to run the secondary OTHER adjoint test of this test-experiment.
891                    It can also contain specific parameter files with the suffix
892                    "<filename>.mpi</>" and shell script <filename>prepare_run</>
893                    (see above).</para>
894                </listitem>
895              </varlistentry>
896                    <!--
897                    -->
898              <varlistentry>
899                <term>directory <filename>TESTDIR/run/</></term>
900                <listitem>
901                  <para> initially empty directory where <filename>testreport</>
902                    will run the MITgcm executable for primary forward and adjoint
903                    test.</para>
904                  <para>Symbolic links (using command "<filename>ln -s</>") are made
905                    for input and parameter files (from <filename>../input/ </>
906                    or from <filename>../input_ad/ </>) and for MITgcm executable
907                    (from <filename>../build/ </>) before the run proceeds.
908                    The sequence of links (function <filename>linkdata</> within shell
909                    script <filename>testreport</>) for a forward test is:</para>
910    <screen>
911    * link+rename or remove links
912           to special files with suffix ".mpi" or ".mth" from ../input/
913    * link files from ../input/
914    * execute ../input/prepare_run (if it exists)
915    </screen>
916                  <para>The sequence for an ajoint test is similar, with
917                    <filename>../input_ad/ </> replacing <filename>../input/ </>.
918                    </para>
919                </listitem>
920              </varlistentry>
921    
922              <varlistentry>
923                <term>directory <filename>TESTDIR/tr_run.OTHER/</></term>
924                <listitem>
925                  <para> directory created by <filename>testreport</>
926                    to run the MITgcm executable for secondary "OTHER" forward
927                    or adjoint test.</para>
928                  <para> The sequence of links for a forward secondary test is:</para>
929    <screen>
930    * link+rename or remove links
931           to special files with suffix ".mpi" or ".mth" from ../input.OTHER/
932    * link files from ../input.OTHER/
933    * execute ../input.OTHER/prepare_run (if it exists)
934    * link files from ../input/
935    * execute ../input/prepare_run (if it exists)
936    </screen>
937                  <para>The sequence for an ajoint test is similar, with
938                    <filename>../input_ad.OTHER/ </> and <filename>../input_ad/ </>
939                    replacing <filename>../input.OTHER/ </> and <filename>../input/ </>.
940                    </para>
941                </listitem>
942              </varlistentry>
943    
944            </variablelist>
945          </sect3>
946    
947          <sect3 id="testreport">
948            <title>The <filename>testreport</> Utility</title>
949    
950            <para> The shell script  <filename>testreport</> (in
951              <filename>$ROOTDIR/verification/</>), which was written to work with
952              <filename>genmake2</>, can be used to build different versions
953              of the MITgcm code, run the various examples, compare the output,
954              and (if specified) email the results of each one of these tests to a
955              central repository.</para>
956    
957            <para>On some systems, the testreport script can be run with a command
958            line as simple as:</para>
959    
960    <screen>
961      $ cd verification
962      $ ./testreport
963    </screen>
964    
965            <para>However, some systems (those lacking or wiht a broken "/bin/sh")
966              may require an explicit shell invocation such as:</para>
967    
968    <screen>
969      $ sh ./testreport -t 'exp2 exp4'
970      $ /some/path/to/bash ./testreport -t 'ideal_2D_oce lab_sea natl_box'
971    </screen>
972    
973            <para>The <filename>testreport</> script accepts a number of
974              command-line options which can be listed using the <filename>-help</>
975              option.  The most important ones are:</para>
976    
977            <variablelist>
978    
979              <varlistentry>
980                <term><filename>-ieee</> (default) / <filename>-noieee</></term>
981                <listitem>
982                  <para>If allowed by the compiler (as defined in the "optfile"),
983                    use IEEE arithmetic (<filename>genmake2 -ieee</>).
984                    This option, along with the gfortran / gcc compiler,
985                    is how the standard results are produced.</para>
986                </listitem>
987              </varlistentry>
988    
989              <varlistentry>
990                <term><filename>-optfile=/PATH/FILENAME</></term>
991                <term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>
992                <listitem>
993                  <para>This specifies a list of "options files" that will be passed
994                    to <filename>genmake2</>.  If multiple options files are used
995                    (say, to test different compilers or different sets of options
996                    for the same compiler), then each options file will be used with
997                    each of the test directories.</para>
998                </listitem>
999              </varlistentry>
1000    
1001              <varlistentry>
1002                <term><filename>-tdir TESTDIR</></term>
1003                <term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term>
1004                <listitem>
1005                  <para>This option specifies the test directory or list of test
1006                    directories that should be used.  Each of these entries should
1007                    exactly (note: they are case sensitive!) match the names of
1008                    directories in <filename>$ROOTDIR/verification/</>.  If this
1009                    option is omitted, then all directories that are properly
1010                    formatted (that is, containing an <filename>input</>
1011                    sub-directory and a <filename>results/output.txt</> file) will
1012                    be used.</para>
1013                </listitem>
1014              </varlistentry>
1015    
1016              <varlistentry>
1017                <term><filename>-addr EMAIL</></term>
1018                <term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>
1019                <listitem>
1020                  <para>Send the results (namely, <filename>output.txt</>,
1021                    <filename>genmake_local</>, <filename>genmake_state</>, and
1022                    <filename>Makefile</>) to the specified email addresses.  The
1023                    results are gzipped, placed in a tar file, MIME encoded, and
1024                    sent to the specified address.  If no email addresses are
1025                    specified, no mail is sent.</para>
1026                </listitem>
1027              </varlistentry>
1028    
1029              <varlistentry>
1030                <term><filename>-MPI NUMBER_OF_PROCS</></term>
1031                <term><filename>-mpi</></term>
1032                <listitem>
1033                  <para>If the necessary file (<filename>TESTDIR/code/SIZE.h_mpi</>)
1034                  exists, then use it (and all <filename>TESTDIR/code/*_mpi</> files)
1035                  for an MPI--enabled run.  The new option (<filename>-MPI</> followed
1036                  by the maximum number of processors) enable to build and run each
1037                  test-experiment using variable number of MPI processors (multiple
1038                  of <filename>nPx*nPy</> from <filename>TESTDIR/code/SIZE.h_mpi</>
1039                  and not larger than <filename>NUMBER_OF_PROCS</>).
1040                  The short option ("-mpi") can only be used to build and run on 2 MPI
1041                  processors (equivalent to "<filename>-MPI 2</>").</para>
1042                  <para>Note that the use of MPI typically requires a
1043                  special command option (see "-command" below) to invoke the MPI
1044                  executable.  Examples of PBS scripts using testreport with MPI can be
1045                  found in the <ulink
1046                  url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
1047                  tools/example_scripts directory</ulink>.</para>
1048                </listitem>
1049              </varlistentry>
1050    
1051              <varlistentry>
1052                <term><filename>-command='some command to run'</></term>
1053                <listitem>
1054                  <para>For some tests, particularly MPI runs, a specific command
1055                  might be needed to run the executable. This option allows a more general
1056                  command (or shell script) to be invoked.  Examples of PBS scripts
1057                  using testreport with MPI can be found in the <ulink
1058                  url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
1059                  tools/example_scripts directory</ulink>.</para>
1060                  <para>
1061                    For the case where the number of MPI processors varies according
1062                    to each test-experiment, some key-words within the command-to-run
1063                    argument will be replaced by their effective value:</para>
1064                  <para>
1065                    <filename>TR_NPROC </> will be replaced by the actual number
1066                    of MPI processors needed to run the current test-experiment.</para>
1067                  <para>
1068                    <filename>TR_MFILE </> will be replaced by the name of local-file
1069                    that testreport creates from the full list of machines which
1070                    "<filename>testreport -mf MACHINE_FILE</>" provides, but truncated
1071                    to the exact number of machines.</para>
1072                </listitem>
1073              </varlistentry>
1074    
1075              <varlistentry>
1076                <term><filename>-mf MACHINE_FILE</></term>
1077                <listitem>
1078                  <para>
1079                    To use with <filename>-MPI NUMBER_OF_PROCS</> option, to specify
1080                    the file containing the full list of <filename>NUMBER_OF_PROCS</>
1081                    machines to use for the MPI runs.</para>
1082                </listitem>
1083              </varlistentry>
1084    
1085              <varlistentry>
1086                <term><filename>-mth</></term>
1087                <listitem>
1088                  <para>compile (with <filename>genmake2 -omp</>) and run with
1089                    multiple threads (using eedata.mth).</para>
1090                </listitem>
1091              </varlistentry>
1092    
1093            </variablelist>
1094    
1095            <para>The <filename>testreport</> script will create an output directory
1096              <filename>tr_NAME_DATE_N/ </>, with <filename>hostname</> as default
1097              <filename>NAME</>, <filename>DATE</> the current date followed by
1098              a suffix number "N" to distinguish from previous
1099              <filename>testreport</> output directories.
1100              <filename>testreport</> writes progress to the screen (stdout)
1101              and reports into the ouput directory as it runs.
1102              In particular, one can find, in the ouput directory,
1103              the <filename>summary.txt</> file that contains a brief comparison
1104              of the current output with the "known-good" output.
1105              At the end of the testing process, the <filename>tr_out.txt</> file
1106              is generated in <filename>$ROOTDIR/verification/ </>
1107              as a compact version of <filename>summry.txt</> file.</para>
1108    
1109          </sect3>
1110    
1111          <sect3 id="tst_2plus2">
1112            <title>The <filename>do_tst_2+2</> Utility</title>
1113            <para> The shell script  <filename>do_tst_2+2</> (in
1114              <filename>$ROOTDIR/tools/ </>)
1115              can be used to check the accuracy of the restart procedure.
1116            </para>
1117          </sect3>
1118    
1119        </sect2>
1120    
1121        <sect2 id="packages">
1122          <title>Creating MITgcm Packages</title>
1123    
1124          <para>Optional parts of code have been separated from the MITgcmUV core
1125            driver code and organised into packages.  The packaging structure
1126            provides a mechanism for maintaining suites of code, specific to
1127            particular classes of problems, in a way that is cleanly separated from
1128            the generic fluid dynamical engine.</para>
1129    
1130          <para>The MITgcmUV packaging structure is described below using generic
1131            package names ${pkg}. A concrete examples of a package is the code for
1132            implementing GM/Redi mixing. This code uses the package name</para>
1133    
1134        </sect2>
1135    
1136      </sect1>
1137    
1138      <sect1>
1139        <title>Chris's Notes...</title>
1140    
1141    <programlisting>
1142               MITgcmUV Packages
1143               =================
1144    
1145      Optional parts of code are separated from
1146    the MITgcmUV core driver code and organised into
1147    packages. The packaging structure provides a mechanism for
1148    maintaining suites of code, specific to particular
1149    classes of problem, in a way that is cleanly
1150    separated from the generic fluid dynamical engine.
1151    
1152     The MITgcmUV packaging structure is describe
1153    below using generic package names ${pkg}.
1154    A concrete examples of a package is the code
1155    for implementing GM/Redi mixing. This code uses
1156    the package name
1157    *   ${PKG} = GMREDI
1158    *   ${pkg} = gmredi
1159    *   ${Pkg} = gmRedi
1160    
1161    Package states
1162    ==============
1163    
1164     Packages can be any one of four states, included,
1165     excluded, enabled, disabled as follows:
1166    
1167     included(excluded) compile time state which
1168                        includes(excludes) package
1169                        code and routine calls from
1170                        compilation/linking etc...
1171    
1172     enabled(disabled)  run-time state which
1173                        enables(disables) package code
1174                        execution.
1175    
1176     Every call to a ${pkg}_... routine from outside the package
1177     should be placed within both a
1178     #ifdef ALLOW_${PKG} ... block and a
1179     if ( use${Pkg} ) ... then block.
1180     Package states are generally not expected to change during
1181     a model run.
1182    
1183    Package structure
1184    =================
1185    
1186    o  Each package gets its runtime configuration
1187       parameters from a file named "data.${pkg}"
1188       Package runtime config. options are imported
1189       into a common block held in a header file
1190       called "${PKG}.h".
1191       Note: In some packages, the header file "${PKG}.h" is splitted
1192       into "${PKG}_PARAMS.h" that contains the package parameters and
1193       ${PKG}_VARS.h" for the field arrays.
1194    
1195    o  The core driver part of the model can check
1196       for runtime enabling or disabling of individual packages
1197       through logical flags use${Pkg}.
1198       The information is loaded from a
1199       global package setup file called "data.pkg".
1200       The use${Pkg} flags are not used within
1201       individual packages.
1202    
1203    o  Included in "${PKG}.h" is a logical flag
1204       called ${Pkg}IsOn. The "${PKG}.h" header file can be imported
1205       by other packages to check dependencies and requirements
1206       from other packages ( see "Package Boot Sequence" section).
1207       NOTE: This procedure is not presently implemented,
1208       ----- neither for kpp nor for gmRedi.
1209    
1210    CPP Flags
1211    =========
1212    
1213        1. Within the core driver code flags of the form
1214           ALLOW_${PKG} are used to include or exclude
1215           whole packages. The ALLOW_${PKG} flags are included
1216           from a PACKAGES_CONFIG.h file that is automatically
1217           generated by genmake2 (see genmake2 section).
1218           held in-line in the CPP_OPTIONS.h header file.
1219           e.g.
1220    
1221           Core model code .....
1222    
1223           #include "PACKAGES_CONFIG.h"
1224           #include "CPP_OPTIONS.h"
1225             :
1226             :
1227             :
1228    
1229           #ifdef ALLOW_${PKG}
1230             if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...)
1231           #endif
1232    
1233        2. Within an individual package a header file,
1234           "${PKG}_OPTIONS.h", is used to set CPP flags
1235           specific to that package. It also includes
1236           "PACKAGES_CONFIG.h" and "CPP_OPTIONS.h".
1237    
1238    Package Boot Sequence
1239    =====================
1240    
1241        Calls to package routines within the core code timestepping
1242        loop can vary. However, all packages follow a required
1243        "boot" sequence outlined here:
1244    
1245        1. S/R PACKAGES_BOOT()
1246                :
1247            CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
1248    
1249        2. S/R PACKAGES_READPARMS()
1250                :
1251            #ifdef ALLOW_${PKG}
1252              if ( use${Pkg} )
1253         &       CALL ${PKG}_READPARMS( retCode )
1254            #endif
1255    
1256        3. S/R PACKAGES_INIT_FIXED()
1257                :
1258            #ifdef ALLOW_${PKG}
1259              if ( use${Pkg} )
1260         &       CALL ${PKG}_INIT_FIXED( retCode )
1261            #endif
1262    
1263        4. S/R PACKAGES_CHECK()
1264                :
1265            #ifdef ALLOW_${PKG}
1266              if ( use${Pkg} )
1267         &       CALL ${PKG}_CHECK( retCode )
1268            #else
1269              if ( use${Pkg} )
1270         &       CALL PACKAGES_CHECK_ERROR('${PKG}')
1271            #endif
1272    
1273        5. S/R PACKAGES_INIT_VARIABLES()
1274                :
1275            #ifdef ALLOW_${PKG}
1276              if ( use${Pkg} )
1277         &       CALL ${PKG}_INIT_VARIA( )
1278            #endif
1279    
1280    Package Output
1281    ==============
1282         6. S/R DO_THE_MODEL_IO
1283    
1284            #ifdef ALLOW_${PKG}
1285              if ( use${Pkg} )
1286         &       CALL ${PKG}_OUTPUT( )
1287            #endif
1288    
1289         7. S/R PACKAGES_WRITE_PICKUP()
1290    
1291            #ifdef ALLOW_${PKG}
1292              if ( use${Pkg} )
1293         &       CALL ${PKG}_WRITE_PICKUP( )
1294            #endif
1295    
1296    Description
1297    ===========
1298    
1299          - ${PKG}_READPARMS()
1300        is responsible for reading
1301        in the package parameters file data.${pkg}, and storing
1302        the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h").
1303        -> called from INITIALISE_FIXED in PACKAGES_READPARMS
1304    
1305         - ${PKG}_INIT_FIXED()
1306        is responsible for completing the internal setup of a package.
1307        -> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED
1308        note: 1) some pkg use instead:
1309                 CALL ${PKG}_INITIALISE  ( or the old form CALL ${PKG}_INIT )
1310              2) for simple pkg setup, this part is done inside ${PKG}_READPARMS
1311    
1312         - ${PKG}_CHECK()
1313        is responsible for validating
1314        basic package setup and inter-package dependencies.
1315        ${PKG}_CHECK can import other package parameters it may
1316        need to check. This is done through header files "${PKG}.h".
1317        It is assumed that parameters owned by other packages
1318        will not be reset during ${PKG}_CHECK().
1319        -> called from INITIALISE_FIXED in PACKAGES_CHECK
1320    
1321         - ${PKG}_INIT_VARIA()
1322        is responsible for fill-in all package variables with an initial value.
1323        Contains eventually a call to ${PKG}_READ_PICKUP that will read
1324        from a pickup file the package variables required to restart the model.
1325        This routine is called after the core model state has been completely
1326        initialised but before the core model timestepping starts.
1327        -> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES
1328        note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg
1329         use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old
1330         form ${PKG}_INIT
1331    
1332         - ${PKG}_OUTPUT( )
1333         is responsible for writing time-average fields to output files
1334         (but the cumulating step is done within the package main S/R).
1335         Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR)
1336         and write snap-shot fields that are hold in common blocks. Other
1337         temporary fields are directly dump to file where they are available.
1338         NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages
1339                  but is beeing replaced by ${PKG}_OUTPUT
1340                  to avoid confusion with pkg/diagnostics functionality.
1341               2) the output part is not yet in a standard form and might still
1342                  evolve a lot.
1343        -> called within DO_THE_MODEL_IO
1344    
1345         - ${PKG}_WRITE_PICKUP()
1346         is responsible for writing a package pickup file when necessary for
1347         a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT )
1348        -> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP
1349    
1350    Summary
1351    =======
1352    
1353    - CPP options:
1354      -----------------------
1355      * ALLOW_${PKG}         include/exclude package for compilation
1356    
1357    - FORTRAN logical:
1358      -----------------------
1359      * use${Pkg}            enable package for execution at runtime
1360                             -> declared in PARAMS.h
1361      * ${Pkg}IsOn           for package cross-dependency check
1362                             -> declared in ${PKG}.h
1363                             N.B.: Not presently used!
1364    
1365    - header files
1366      -----------------------
1367      * ${PKG}_OPTIONS.h     has further package-specific CPP options
1368      * ${PKG}.h             package-specific common block variables, fields
1369       or  ${PKG}_PARAMS.h   package-specific common block parameters
1370       and ${PKG}_VARS.h     package-specific common block fields
1371    
1372    - FORTRAN source files
1373      -----------------------
1374      * ${pkg}_readparms.F    reads parameters from file data.${pkg}
1375      * ${pkg}_init_fixed.F   complete the package setup
1376      * ${pkg}_check.F        checks package dependencies and consistencies
1377      * ${pkg}_init_varia.F   initialises package-related fields
1378      * ${pkg}_... .F         package source code
1379      * ${pkg}_output.F       write output to file.
1380      * ${pkg}_write_pickup.F write a package pickup file to restart the model
1381    
1382      New: Subroutine in one package (pkgA) that only contains code which
1383           is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F)
1384           will be named: pkgA_pkgB_something.F
1385    
1386    - parameter file
1387      -----------------------
1388      * data.${pkg}          parameter file
1389    </programlisting>
1390    
1391      </sect1>
1392    
1393    <sect1 id="documentation">    <sect1 id="documentation">
1394      <title>Editing the Documentation</title>      <title>Editing the Documentation</title>
# Line 293  checkpoint50d_pre Line 1396  checkpoint50d_pre
1396      <sect2 id="documentation_getting">      <sect2 id="documentation_getting">
1397        <title>Getting the Docs and Code</title>        <title>Getting the Docs and Code</title>
1398    
1399        <para>The first step towards editing the documentation is to        <para>The first step towards editing the documentation is to checkout a
1400        checkout a copy of code, docs, and build scripts from the CVS        copy of code, docs, and build scripts from the CVS server using:</para>
1401        server using:</para>  
1402    <screen>
1403  <screen>    $ export CVS_RSH=ssh
1404  $ export CVS_RSH=ssh    $ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack'
1405  $ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack'    $ mkdir scratch
1406  $ mkdir scratch    $ cvs co -P MITgcm manual mitgcm.org
1407  $ cvs co MITgcm manual mitgcm.org  </screen>
1408  </screen>  
1409          <para>These commands extract the necessary information from the CVS server
1410        <para>These commands extract the necessary information from the        and create a temporary (called <filename>scratch</filename>) directory for
1411        CVS server and create a temporary (called        the storage of the HTML and other files that will be created.  Please note
1412        <filename>scratch</filename>) directory for the storage of the        that you must either create <filename>scratch</filename> as shown or edit
1413        HTML and other files that will be created.  Please note that you        the various <filename>Makefile</filename>s and scripts used to create the
1414        must either create <filename>scratch</filename> as shown or edit        documentation.</para>
       the various <filename>Makefile</filename>s and scripts used to  
       create the documentation.</para>  
1415      </sect2>      </sect2>
1416    
1417      <sect2>      <sect2>
1418        <title>Editing</title>        <title>Editing the Documentation</title>
1419    
1420        <para>The documentation is contained in the        <para>The documentation is contained in the <filename>manual</filename>
1421        <filename>manual</filename> directory in a raw LaTeX format.        directory in a raw LaTeX format.  The main document is
1422        The main document is <filename>manual.tex</filename> and it uses        <filename>manual.tex</filename> and it uses <command>\input{}</command>s
1423        <command>\input{}</command>s to include the chapters and        to include the chapters and subsections.</para>
1424        subsections.</para>  
1425          <para>Since the same LaTeX source is used to produce PostScript, PDF, and
1426        <para>Since the same LaTeX source is used to produce PostScript,        HTML output, care should be taken to follow certain conventions.  Two of
1427        PDF, and HTML output, care should be taken to follow certain        the most important are the usage of the <command>\filelink{}{}</command>
1428        conventions.  Two of the most important are the usage of the        and <command>\varlink{}{}</command> commands.  Both of these commands have
1429        <command>\filelink{}{}</command> and        been defined to simplify the connection between the automatically
1430        <command>\varlink{}{}</command> commands.  Both of these        generated ("code browser") HTML and the HTML version of the manual
1431        commands have been defined to simplify the connection between        produced by LaTeX2HTML.  They each take two arguments (corresponding to
1432        the automatically generated ("code browser") HTML and the HTML        the contents of the two sets of curly braces) which are the text that the
1433        version of the manual produced by LaTeX2HTML.  They each take        author wishes to be "wrapped" within the link, and a specially formatted
1434        two arguments (corresponding to the contents of the two sets of        link thats relative to the <filename>MITgcm</filename> directory within
1435        curly braces) which are the text that the author wishes to be        the CVS tree.</para>
       "wrapped" within the link, and a specially formatted link thats  
       relative to the <filename>MITgcm</filename> directory within the  
       CVS tree.</para>  
1436    
1437        <para>The result is a command that resembles either</para>        <para>The result is a command that resembles either</para>
1438          
1439        <orderedlist>        <orderedlist>
1440          <listitem>          <listitem>
1441            <para>a reference to a variable or subroutine name such as            <para>a reference to a variable or subroutine name such as
# Line 356  $ cvs co MITgcm manual mitgcm.org Line 1454  $ cvs co MITgcm manual mitgcm.org
1454                into a "-")</para>                into a "-")</para>
1455          </listitem>          </listitem>
1456        </orderedlist>        </orderedlist>
1457                      
   
1458    
1459      </sect2>      </sect2>
1460    
1461      <sect2>      <sect2>
1462        <title>Building</title> <para>Given the directory structure of        <title>Building the Documentation</title>
1463        <xref linkend="documentation_getting">, the entire documentation  
1464        for the web site can be built using:</para>        <para>Given the directory structure of <xref
1465          linkend="documentation_getting">, the entire documentation for the web
1466          site can be built using:</para>
1467    
1468  <screen>  <screen>
1469  $ cd mitgcm.org/devel/buildweb    $ cd mitgcm.org/devel/buildweb
1470  $ make All    $ make All
1471  </screen>  </screen>
1472    
1473        <para>Which builds the PDF from the LaTeX source, creates the        <para>Which builds the PDF from the LaTeX source, creates the HTML output
1474        HTML output from the LaTeX source, parses the FORTRAN code base        from the LaTeX source, parses the FORTRAN code base to produce a
1475        to produce a hyperlinked HTML version of the source, and then        hyperlinked HTML version of the source, and then determines the
1476        determines the cross-linking between the various HTML        cross-linking between the various HTML components.</para>
1477        components.</para>  
1478          <para>If there are no errors, the result of the build process (which can
1479        <para>If there are no errors, the result of the build process        take 30+ minutes on a P4/2.5Ghz) will be contained within a single
1480        (which can take 30+ minutes on a P4/2.5Ghz) will be contained        directory called <filename>scratch/dev_docs</filename>.  This is a freshly
1481        within a single directory called        built version of the entire on-line users manual.  If you have the correct
1482        <filename>scratch/dev_docs</filename>.  This is a freshly built        permissions, it can be directly copied to the web server area:</para>
       version of the entire on-line users manual.  If you have the  
       correct permissions, it can be directly copied to the web server  
       area:</para>  
1483    
1484  <screen>  <screen>
1485  $ mv scratch/dev_docs /u/u0/httpd/html    $ mv scratch/dev_docs /u/u0/httpd/html
1486  </screen>  </screen>
1487    
1488        <para>and the update is complete.</para>        <para>and the update is complete.</para>
# Line 395  $ mv scratch/dev_docs /u/u0/httpd/html Line 1491  $ mv scratch/dev_docs /u/u0/httpd/html
1491    
1492    </sect1>    </sect1>
1493    
   
   
1494  </article>  </article>
1495    
   
Legend:
Removed from v.1.1  
changed lines
  Added in v.1.19
  ViewVC Help
Powered by ViewVC 1.1.22