/[MITgcm]/MITgcm/doc/devel_HOWTO.sgml
ViewVC logotype

Diff of /MITgcm/doc/devel_HOWTO.sgml

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.8 by jmc, Wed Jan 5 03:45:24 2005 UTC revision 1.18 by jmc, Tue Dec 19 18:38:40 2017 UTC
# Line 1  Line 1 
1  <!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">  <!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
2    <!--
3     $Header$
4     $Name$
5    -->
6    
7  <article id="MITgcm-Development-HOWTO">  <article id="MITgcm-Development-HOWTO">
8    
# Line 22  Line 26 
26            Initial version.            Initial version.
27          </revremark>          </revremark>
28        </revision>        </revision>
29          <revision>
30            <revnumber>0.02</revnumber>
31            <date>2010-01-21</date>
32            <authorinitials>jmc</authorinitials>
33            <revremark>
34              Update links.
35            </revremark>
36          </revision>
37          <revision>
38            <revnumber>0.03</revnumber>
39            <date>2010-04-25</date>
40            <authorinitials>jmc</authorinitials>
41            <revremark>
42              Add subsection "Developer settings" (under CVS Repository).
43            </revremark>
44          </revision>
45          <revision>
46            <revnumber>0.04</revnumber>
47            <date>2011-04-24</date>
48            <authorinitials>jmc</authorinitials>
49            <revremark>
50              Update subsection "The verification suite".
51            </revremark>
52          </revision>
53      </revhistory>      </revhistory>
54    
55      <abstract>      <abstract>
# Line 37  Line 65 
65      <sect2>      <sect2>
66        <title>New Versions of This Document</title> <para>You can        <title>New Versions of This Document</title> <para>You can
67        obtain the latest version of this document <ulink        obtain the latest version of this document <ulink
68        url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in        url="http://mitgcm.org/public/docs.html">online</ulink> in
69        various formats.</para>        various formats.</para>
70      </sect2>      </sect2>
71      <sect2>      <sect2>
# Line 56  Line 84 
84        <title>User Manual</title>        <title>User Manual</title>
85    
86        <para>Before jumping into development, please familiarize yourself with        <para>Before jumping into development, please familiarize yourself with
87          the <ulink url="http://mitgcm.org/docs.html"> MITgcm user manual          the <ulink url="http://mitgcm.org/public/docs.html"> MITgcm user manual
88          </ulink>.  This document contains volumes of useful information and is          </ulink>.  This document contains volumes of useful information and is
89          included here by reference.</para>          included here by reference.</para>
90    
# Line 94  Line 122 
122    
123    <sect1 id="cvs">    <sect1 id="cvs">
124      <title>CVS Repository</title>      <title>CVS Repository</title>
125    
126      <sect2>      <sect2>
127        <title>Layout</title>        <title>Layout</title>
128    
# Line 104  Line 133 
133        others.  The tree currently resembles:</para>        others.  The tree currently resembles:</para>
134    
135  <programlisting>gcmpack/  <programlisting>gcmpack/
136    MITgcm-contrib        contributed code    CVSROOT               -hidden-
   CS-regrid             goes into utils  
   cvspolicy.html        -save-  
   CVSROOT               -save-  
   development           experimental stuff  
   manual                -save-  
   misc                  -?-  
137    
138    MITgcm                code    MITgcm                code
139         adjoint                  fold into genmake         bin                      empty
140         bin                      stub for ecco build         doc                      basic developpment documentation
141         compare01                old from 20th century         eesupp                   execution environment support code (wrapper)
142         diags                    timeave f77 in pkgs now         exe                      empty
143         doc                      tags -- connect to real docs?         jobs                     runtime shell scripts for
144         eesupp                   cnh?                                    various platforms (not maintained)
145         exe                      ecco user build         lsopt                    line search
146      ,- jobs                     runtime shell scripts for         model                    main dynamics (core)
147      |                             various platforms         optim                    line search interface
148      |  lsopt                    line search         pkg                      alternate and optional numerics, etc.
149     m|  model                    main dynamics (core)         tools                    scripts to build (and test)
150     e|    optimization_drivers   ?         utils                    pre/post processing tools (matlab, ..)
151     r|  optim                    line search interface         verification             standard regression tests + examples
152     g|  pkg                      alternate and optional numerics, etc.                                        + documented examples (tutorials)
153     e|- tools         tutorial_examples        (only in release1 branch)
    ?|  tutorial_examples        documented tests  
     |                             only populated on release1 branch  
     |                             and not validated during "testscript"  
     '- utils  
        verification             std tests  
154    
155      MITgcm_contrib        contributed code
156    
157    mitgcmdoc -> manual   -remove-    acesgrid.org          build acesgrid web site
158      development           experimental stuff
159      gfd_lab               -?-
160      manual                source of MITgcm documentation
161    mitgcm.org            build web site    mitgcm.org            build web site
162    models                -?-    old_develop           old and early development source
163    packages              -?-         misc                 -?-
164    preprocess            -?-         models               -?-
165    tmp                   -?-         packages             -?-
166           preprocess           -?-
167      pdfs                  some pdfs
168      planetinabottle.org   unfinished web site
169      www.ecco-group.org    build ecco web site ?
170  </programlisting>  </programlisting>
171    
172     <!--
173        <para>Efforts are underway to reduce the complexity.</para>        <para>Efforts are underway to reduce the complexity.</para>
174    -->
175    
176      </sect2>      </sect2>
177    
# Line 182  Line 210 
210        <title>Branches</title>        <title>Branches</title>
211    
212        <para>As shown in the online <ulink        <para>As shown in the online <ulink
213        url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">        url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph">
214        ViewCVS-generated tree</ulink>, the MITgcm codebase is split into to two        ViewCVS-generated tree</ulink>, the MITgcm codebase is split into
215        branches or "lines" under which development proceeds.  These two lines are        branches or "lines" under which development proceeds.  The main line
216        referred to as the "MAIN" and "ecco" versions of the code.  While not        of development is referred to as the "MAIN" version of the code.
       identical, the bulk of the MAIN and ecco lines are composed of files from  
       the same codebase.  
217        </para>        </para>
218    
219        <para>Periodically, a "Release" branch is formed from the "MAIN"        <para>Periodically, a "Release" branch is formed from the "MAIN"
220        development branch.  This is done in order to create a relatively stable        development branch.  This is done in order to create a relatively stable
221        reference point for both users and developers.  The intent is that once a        reference point for both users and developers.  The intent is that once a
222        relese branch has been created, only bug-fixes will be added to it.        release branch has been created, only bug-fixes will be added to it.
223        Meanwhile, development (which might "break" or otherwise render invalid        Meanwhile, development (which might "break" or otherwise render invalid
224        the documentation, tutorials, and/or examples contained within a release        the documentation, tutorials, and/or examples contained within a release
225        branch) is allowed to continue along the MAIN and ecco lines.</para>        branch) is allowed to continue along the MAIN line.</para>
226        </sect2>
227    
228        <sect2>
229          <title> Developer settings </title>
230    
231          <para>CVS is a convenient tool to keep up-to-date a personal copy of the
232          MITgcm code (see: <ulink url="http://mitgcm.org/public/using_cvs.html">
233          using CVS </ulink>). The same tool is used by developers to
234          incorporate any change into the repository. However, this later
235          function requires specific settings, as detailed here after</para>
236          <orderedlist>
237            <listitem>
238              <para> You will need an account (login access) to the server
239               "mitgcm.org" (curently: <filename>forge.csail.mit.edu</filename>)
240               with the proper group setting (e.g., group "gcmctrb" to add/modify
241               code into MITgcm_contrib).
242               This kind of account is granted only upon well motivated request
243               (we recommend to ask your senior MITgcm-collaborator to send such
244               request to marshall-admin at techsquare.com with Cc to Chris Hill
245               for approval).</para>
246              <para> The access to the server <filename>mitgcm.org</filename> is
247               through ssh-key authorization which will need to be set properly on
248               both side (on your local machine and on your server account).
249               You need to be able to ssh to <filename>mitgcm.org</filename>
250               (or <filename>ssh MY_USER_NAME@mitgcm.org</filename>
251               in case of different user-name on both sides) to proceed further.</para>
252            </listitem>
253    
254            <listitem>
255              <para> You need to register to the
256            <ulink url="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-cvs">
257           mitgcm-cvs </ulink> mailing list.
258              This ensures that other developers will receive email notification
259               when you make changes; you will also receive such email
260               when others make changes to the repository.
261              </para>
262            </listitem>
263    
264            <listitem>
265              <para> It is highly recommended that you register also to the
266            <ulink url="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel">
267           mitgcm-devel </ulink> mailing list (expect a short delay for
268           this request to be processed).
269              This list is intended for developer discussions.
270              </para>
271            </listitem>
272    
273            <listitem>
274              <para> The standard CVS-anonymous mode (using "cvsanon",
275            as mentionned <ulink url="http://mitgcm.org/public/source_code.html">
276            here </ulink>) does not provide check-in ("cvs commit") permission.
277             Instead, you will need to set our CVS environment as follow:</para>
278    <screen>
279      $ export CVS_RSH=ssh
280      $ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack'
281    </screen>
282              <para> The reason for such limitation is that when downloading a directory,
283                e.g.: <filename>myCopy</filename>, from the CVS repository (e.g.,
284                <filename>MITgcm_contrib/thisPart</filename>) using the command:</para>
285    <screen>
286      $ cvs co -P -d myCopy MITgcm_contrib/thisPart
287    </screen>
288              <para> the type of CVS environment which has been used
289               is stored in the file <filename>myCopy/CVS/Root</filename>.
290               This prevent to re-use, for cvs-commit purpose,
291               a cvs local copy (<filename>myCopy</filename>) which was obtained
292               using the CVS anonymous mode.</para>
293            </listitem>
294    
295            <listitem>
296              <para> At this stage, you should be able to send your modified source
297              file (e.g., <filename>src_file</filename>) from your local copy directory
298              (<filename>myCopy</filename>) to the CVS repository
299              (<filename>MITgcm_contrib/thisPart</filename>) using the command
300              "cvs commit":</para>
301    <screen>
302      $ cd myCopy
303      $ cvs -n update        (optional; check if new changes have been made)
304      $ cvs diff src_file    (optional; list your changes)
305      $ cvs commit src_file
306    </screen>
307              <para> It is essential that you provide a short description of the
308              changes you made to <filename>src_file</filename> as you check-in
309              this file (the "cvs commit" command automatically opens your standard
310              editor for this purpose).</para>
311              <para> Note:
312               Please ignore the following warnings that the "cvs commit" command
313               produces if you are not part of the "gcmpack" group:
314    <screen>
315       cvs commit: failed to create lock directory for `/u/gcmpack/CVSROOT'
316       (/u/gcmpack/CVSROOT/#cvs.history.lock): Permission denied
317       cvs commit: failed to obtain history lock in repository `/u/gcmpack'
318    </screen>
319              These warnings are not affecting the changes you made to the CVS repository.
320              </para>
321            </listitem>
322    
323          </orderedlist>
324    
325      </sect2>      </sect2>
326    
327      <sect2>      <sect2>
328        <title>Tagging</title>        <title>Main code development</title>
329          <para><emphasis>(formerly named "Tagging" ; this section needs an update)
330            </emphasis></para>
331    
332        <para>The intent of tagging is to create "known-good" checkpoints that        <para>The intent of tagging is to create "known-good" checkpoints that
333        developers can use as references.  Traditionally, MITgcm tagging has        developers can use as references.  Traditionally, MITgcm tagging has
# Line 216  Line 343 
343    
344          <listitem>          <listitem>
345            <para>The developer then runs the <ulink            <para>The developer then runs the <ulink
346            url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">            url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport">
347            testscript</ulink> shell script to see if any problems are introduced.            testreport</ulink> shell script to see if any problems are introduced.
348            While not intended to be exhaustive, the test cases within the            While not intended to be exhaustive, the test cases within the
349            verification directory do provide some indication whether gross errors            verification directory do provide some indication whether gross errors
350            have been introduced.            have been introduced.
# Line 232  Line 359 
359              <listitem>              <listitem>
360                <para>adds a "checkpointXY_pre" comment (where X is a checkpoint                <para>adds a "checkpointXY_pre" comment (where X is a checkpoint
361                number and Y is a letter) to the <ulink                number and Y is a letter) to the <ulink
362                url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">                url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index">
363                tag-index</ulink> file and checks it into the CVS                tag-index</ulink> file and checks it into the CVS
364                repository</para>                repository</para>
365              </listitem>              </listitem>
# Line 253  checkpoint50e_post Line 380  checkpoint50e_post
380  o make KPP work with PTRACERS  o make KPP work with PTRACERS
381   - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is   - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is
382     nearly a copy of kpp_transport_s     nearly a copy of kpp_transport_s
383   - there is no analogue to SurfaceTendencyS, so I have to use   - there is no analogue to SurfaceTendencyS, so I have to use
384     gPtr(of the surface layer) instead     gPtr(of the surface layer) instead
385  o add a new platform SunFire+mpi (SunFire 15000) to genmake  o add a new platform SunFire+mpi (SunFire 15000) to genmake
386  checkpoint50e_pre  checkpoint50e_pre
387    
388  checkpoint50d_post  checkpoint50d_post
389  o change kpp output from multiple-record state files to single-record state  o change kpp output from multiple-record state files to single-record state
390    files analogous to write_state.F    files analogous to write_state.F
391  o reduce the output frequency of cg3d-related stuff to the monitor frequency,  o reduce the output frequency of cg3d-related stuff to the monitor frequency,
392    analogous to the cg2d-related output.    analogous to the cg2d-related output.
393  o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,  o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,
394    so that writing to internal file fn (with length 512) fails.    so that writing to internal file fn (with length 512) fails.
395  checkpoint50d_pre  checkpoint50d_pre
396  </programlisting>  </programlisting>
# Line 277  checkpoint50d_pre Line 404  checkpoint50d_pre
404    </sect1>    </sect1>
405    
406    
   <sect1 id="documentation">  
     <title>Editing the Documentation</title>  
   
     <sect2 id="documentation_getting">  
       <title>Getting the Docs and Code</title>  
   
       <para>The first step towards editing the documentation is to checkout a  
       copy of code, docs, and build scripts from the CVS server using:</para>  
   
 <screen>  
   $ export CVS_RSH=ssh  
   $ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack'  
   $ mkdir scratch  
   $ cvs co MITgcm manual mitgcm.org  
 </screen>  
   
       <para>These commands extract the necessary information from the CVS server  
       and create a temporary (called <filename>scratch</filename>) directory for  
       the storage of the HTML and other files that will be created.  Please note  
       that you must either create <filename>scratch</filename> as shown or edit  
       the various <filename>Makefile</filename>s and scripts used to create the  
       documentation.</para>  
     </sect2>  
   
     <sect2>  
       <title>Editing the Documentation</title>  
   
       <para>The documentation is contained in the <filename>manual</filename>  
       directory in a raw LaTeX format.  The main document is  
       <filename>manual.tex</filename> and it uses <command>\input{}</command>s  
       to include the chapters and subsections.</para>  
   
       <para>Since the same LaTeX source is used to produce PostScript, PDF, and  
       HTML output, care should be taken to follow certain conventions.  Two of  
       the most important are the usage of the <command>\filelink{}{}</command>  
       and <command>\varlink{}{}</command> commands.  Both of these commands have  
       been defined to simplify the connection between the automatically  
       generated ("code browser") HTML and the HTML version of the manual  
       produced by LaTeX2HTML.  They each take two arguments (corresponding to  
       the contents of the two sets of curly braces) which are the text that the  
       author wishes to be "wrapped" within the link, and a specially formatted  
       link thats relative to the <filename>MITgcm</filename> directory within  
       the CVS tree.</para>  
   
       <para>The result is a command that resembles either</para>  
         
       <orderedlist>  
         <listitem>  
           <para>a reference to a variable or subroutine name such as  
           <command>\varlink{tRef}{tRef}</command>, or </para>  
         </listitem>  
   
         <listitem>  
           <para>a reference to a file such as  
               <command>\varlink{tRef}{path-to-the-file_name.F}</command>  
               where the absolute path to the file is of the form  
               <filename>/foo/MITgcm/path/to/the/file_name.F</filename></para>  
               <para>(please note how the leading "/foo/MITgcm"  
               component of the path is dropped leaving the path  
               <emphasis>relative</emphasis> to the head of the code  
               directory and each directory separator "/" is turned  
               into a "-")</para>  
         </listitem>  
       </orderedlist>  
             
   
   
     </sect2>  
   
     <sect2>  
       <title>Building the Documentation</title>  
         
       <para>Given the directory structure of <xref  
       linkend="documentation_getting">, the entire documentation for the web  
       site can be built using:</para>  
   
 <screen>  
   $ cd mitgcm.org/devel/buildweb  
   $ make All  
 </screen>  
   
       <para>Which builds the PDF from the LaTeX source, creates the HTML output  
       from the LaTeX source, parses the FORTRAN code base to produce a  
       hyperlinked HTML version of the source, and then determines the  
       cross-linking between the various HTML components.</para>  
   
       <para>If there are no errors, the result of the build process (which can  
       take 30+ minutes on a P4/2.5Ghz) will be contained within a single  
       directory called <filename>scratch/dev_docs</filename>.  This is a freshly  
       built version of the entire on-line users manual.  If you have the correct  
       permissions, it can be directly copied to the web server area:</para>  
   
 <screen>  
   $ mv scratch/dev_docs /u/u0/httpd/html  
 </screen>  
   
       <para>and the update is complete.</para>  
   
     </sect2>  
   
   </sect1>  
   
407    <sect1 id="coding">    <sect1 id="coding">
408      <title>Coding for MITgcm</title>      <title>Coding for MITgcm</title>
409    
# Line 392  checkpoint50d_pre Line 417  checkpoint50d_pre
417          For MITgcm, the process is similar.  Typical commands are:</para>          For MITgcm, the process is similar.  Typical commands are:</para>
418    
419  <screen>  <screen>
420    $ genmake -mods=../code    $ genmake2 -mods=../code
421    $ make depend    $ make depend
422    $ make    $ make
423  </screen>  </screen>
424    
425        <para>The following sections describe the individual steps in the build        <para>The following sections describe the individual steps in the build
426          process.</para>          process.</para>
427          
428        <sect3 id="genmake">        <sect3 id="genmake">
429          <title>The <filename>genmake2</> Utility</title>          <title>The <filename>genmake2</> Utility</title>
430    
431          <para><emphasis>Please note that the older <filename>genmake</> is          <para><emphasis>(Note: the older <filename>genmake</>
432              deprecated and will eventually be replaced by <filename>genmake2</>.              has been replaced by <filename>genmake2</>)</emphasis></para>
             This HOWTO only describes the newer tool.</emphasis></para>  
433    
434          <para>The first step in any MITgcm build is to create a Unix-style          <para>The first step in any MITgcm build is to create a Unix-style
435            <filename>Makefile</filename> which will be parsed by            <filename>Makefile</filename> which will be parsed by
# Line 461  checkpoint50d_pre Line 485  checkpoint50d_pre
485              <para>an "options file" as specified by the command-line option              <para>an "options file" as specified by the command-line option
486                <filename>-optfile='FILENAME'</filename></para>                <filename>-optfile='FILENAME'</filename></para>
487            </listitem>            </listitem>
488              <listitem>
489                <para> a <filename>packages.conf</filename> file (in the current
490                  directory or in one of the "MODS" directories, see below)
491                  which contains the specific list of packages to compile
492                </para>
493              </listitem>
494          </orderedlist>          </orderedlist>
495    
496          <para>then checking certain dependency rules (the package dependencies),          <para>then checking certain dependency rules (the package dependencies),
# Line 514  checkpoint50d_pre Line 544  checkpoint50d_pre
544            </varlistentry>            </varlistentry>
545    
546            <varlistentry>            <varlistentry>
             <term><filename>-pdepend=/PATH/FILENAME</></term>  
   
             <listitem>  
               <para>This specifies the dependency file used for packages.  If  
               not specified, the default dependency file is  
               <filename>$ROOTDIR/pkg/pkg_depend</>.  The syntax for this file is  
               parsed on a line-by-line basis where each line containes either a  
               comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule  
               where the "+" or "-" symbol specifies a "must be used with" or a  
               "must not be used with" relationship, respectively.  If no rule is  
               specified, then it is assumed that the two packages are compatible  
               and will function either with or without each other.</para>  
             </listitem>  
           </varlistentry>  
   
           <varlistentry>  
             <term><filename>-pdefault=PKG</></term>  
             <term><filename>-pdefault='PKG1 [PKG2 PKG3 ...]'</></term>  
             <listitem>  
               <para>This option specifies the default set of packages  
               to be used.  If not set, the default package list will  
               be read from  
               <filename>$ROOTDIR/pkg/pkg_default</>.</para>  
             </listitem>  
           </varlistentry>  
   
           <varlistentry>  
547              <term><filename>-adof=/path/to/file</></term>              <term><filename>-adof=/path/to/file</></term>
548              <term><filename>-adoptfile=/path/to/file</></term>              <term><filename>-adoptfile=/path/to/file</></term>
549              <listitem>              <listitem>
# Line 568  checkpoint50d_pre Line 571  checkpoint50d_pre
571                  will be overridden by any identically-named sources within the                  will be overridden by any identically-named sources within the
572                  "MODS" directories.  The order of precedence for this                  "MODS" directories.  The order of precedence for this
573                  "name-hiding" is as follows:</para>                  "name-hiding" is as follows:</para>
                 
574                <itemizedlist>                <itemizedlist>
575                  <listitem><para>"MODS" directories (in the order given)                  <listitem><para>"MODS" directories (in the order given)
576                    </para></listitem>                    </para></listitem>
577                  <listitem><para>Packages either explicitly specified or                  <listitem><para>Packages either explicitly specified or
578                      provided by default (in the order given)</para></listitem>                      provided by default (in the order given)</para></listitem>
579                  <listitem><para>Packages included due to package dependencies                  <listitem><para>Packages included due to package dependencies
580                      (in the order that that package dependencies are                      (in the order that that package dependencies are
581                      parsed)</para></listitem>                      parsed)</para></listitem>
582                  <listitem><para>The "standard dirs" (which may have been                  <listitem><para>The "standard dirs" (which may have been
583                      specified by the "-standarddirs" option)</para></listitem>                      specified by the "-standarddirs" option)</para></listitem>
584                </itemizedlist>                </itemizedlist>
585                </listitem>
586              </varlistentry>
587    
588              <varlistentry>
589                <term><filename>-pgroups=/PATH/FILENAME</></term>
590                <listitem>
591                  <para>This option specifies the file where package groups are
592                  defined.  If not set, the package-groups definition will
593                  be read from
594                  <filename>$ROOTDIR/pkg/pkg_groups</>.</para>
595                  <para>
596                  It also contains the default list of packages (defined
597                  as the group <filename>"default_pkg_list"</>) which is used
598                  when no specific package list (file: <filename>packages.conf</>)
599                  is found in current directory or in any "MODS" directory.
600                  </para>
601                </listitem>
602              </varlistentry>
603    
604              <varlistentry>
605                <term><filename>-pdepend=/PATH/FILENAME</></term>
606    
607                <listitem>
608                  <para>This specifies the dependency file used for packages.  If
609                  not specified, the default dependency file is
610                  <filename>$ROOTDIR/pkg/pkg_depend</>.  The syntax for this file is
611                  parsed on a line-by-line basis where each line containes either a
612                  comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule
613                  where the "+" or "-" symbol specifies a "must be used with" or a
614                  "must not be used with" relationship, respectively.  If no rule is
615                  specified, then it is assumed that the two packages are compatible
616                  and will function either with or without each other.</para>
617              </listitem>              </listitem>
618            </varlistentry>            </varlistentry>
619    
# Line 606  checkpoint50d_pre Line 639  checkpoint50d_pre
639          <para>In general, it is best to use <filename>genmake2</> on a "clean"          <para>In general, it is best to use <filename>genmake2</> on a "clean"
640            directory that is free of all source (*.[F,f],*.[F,f]90) and header            directory that is free of all source (*.[F,f],*.[F,f]90) and header
641            (*.h,*.inc) files.  Generally, this can be accomplished in an            (*.h,*.inc) files.  Generally, this can be accomplished in an
642            "un-clean" directory by running "make CLEAN" followed by "make            "un-clean" directory by running "make Clean" followed by "make
643            makefile".</para>            makefile".</para>
644    
645        </sect3>        </sect3>
# Line 619  checkpoint50d_pre Line 652  checkpoint50d_pre
652            simulator) executable using:</para>            simulator) executable using:</para>
653    
654  <screen>  <screen>
655    $ make CLEAN    $ make Clean
656    $ make depend    $ make depend
657    $ make    $ make
658  </screen>  </screen>
659    
660          <para>The "make CLEAN" step will remove any stale source files, include          <para>The "make Clean" step will remove any stale source files, include
661            files, and links.  It is strongly recommended for "un-clean"            files, and links.  It is strongly recommended for "un-clean"
662            directories which may contain the (perhaps partial) results of            directories which may contain the (perhaps partial) results of
663            previous builds.  Such "debris" can interfere with the next stage of            previous builds. Such "debris" can interfere with the next stage of
664            the build.</para>            the build.
665              A more agressive cleaning option, "make CLEAN", can be used to also
666              remove the executable and output files from a previous run.</para>
667    
668          <para>The "make depend" step will create a large number of symbolic          <para>The "make depend" step will create a large number of symbolic
669            links from the local directory to the source file locations.  It also            links from the local directory to the source file locations.  It also
# Line 661  checkpoint50d_pre Line 696  checkpoint50d_pre
696              <listitem>              <listitem>
697                <para>This target produces an <filename>mitgcmuv_ad</> executable                <para>This target produces an <filename>mitgcmuv_ad</> executable
698                  using the <filename>taf</> or <filename>staf</> adjoint                  using the <filename>taf</> or <filename>staf</> adjoint
699                  compiler.  See the <filename>genmake2</> "-adof" option for                  compiler.  See the <filename>genmake2</> "-adof" option for
700                  compiler selection.</para>                  compiler selection.</para>
701              </listitem>              </listitem>
702            </varlistentry>            </varlistentry>
# Line 687  checkpoint50d_pre Line 722  checkpoint50d_pre
722        <title>The Verification Suite</title>        <title>The Verification Suite</title>
723    
724        <para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</>        <para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</>
725          directory) includes more than a dozen examples intended for regression          directory) includes many (> 90) examples intended for regression
726          testing.  Each one of these example directories contains "known-good"          testing.  Each one of these test-experiment directories contains "known-good"
727          output files along with all the input (including both code and data          output files along with all the input (including both code and data
728          files) required for their re-calculation.  These example directories are          files) required for their re-calculation.
729          further broken down into sets of subdirectories          Also included in <filename>$ROOTDIR/verification/</> is the shell
730          (eg. <filename>/input</>, <filename>/code</>) intended to expedite the          script <filename>testreport</> to perform regression tests.</para>
731          testing process.</para>  
732          <sect3 id="test-experiment">
733            <title>Test-experiment Directory Content</title>
734    
735              <para> Each test-experiment directory (<filename>TESTDIR</>) contains
736              several standard subdirectories and files which <filename>testreport</>
737              recognizes and uses when running a regression test.
738              The directories/files that <filename>testreport</> uses are different
739              for a forward test and an adjoint test (<filename>testreport -adm</>)
740              and some test-experiments are set-up for only one type of regression
741              test whereas others allow both types of tests (forward and adjoint).</para>
742              <para>Also some test-experiment allows, using the same MITgcm executable,
743              to perform multiple tests using different parameters and input files,
744              with a primary input set-up
745              (<filename>input/ </> or <filename>input_ad/ </>)
746              and corresponding results (<filename>results/output.txt</> or
747              <filename>results/output_adm.txt</>) and with one or several secondary
748              inputs (<filename>input.OTHER/ </> or <filename>input_ad.OTHER/ </>)
749              and corresponding results (<filename>results/output.OTHER.txt </>
750              or <filename>results/output_adm.OTHER.txt</>).</para>
751            <variablelist>
752    
753              <varlistentry>
754                <term>directory <filename>TESTDIR/results/</></term>
755                <listitem>
756                  <para>contains reference standard output used for test comparison.
757                    <filename>results/output.txt</> and
758                    <filename>results/output_adm.txt</>
759                    correspond respectively to primary forward and adjoint test
760                    run on the reference platform (currently
761                    <filename>baudelaire.csail.mit.edu</>)
762                    on one processor (no MPI, single thread)
763                    using the reference compiler (curently the GNU fortran
764                    compiler <filename>gfortran</>).
765                    The presence of these files determines whether or not
766                    <filename>testreport</> is testing or skipping
767                    this test-experiment.
768                    Reference standard output for secondary tests
769                    (<filename>results/output.OTHER.txt</>
770                    or <filename>results/output_adm.OTHER.txt</>) are
771                    also expected here.</para>
772                  <para>The test comparison involves few model variables output, which are,
773                    by default and for a forward test, the 2-D solver initial residual
774                    (<filename>cg2d_init_res</>) and 3-D state variables (T,S,U,V)
775                    monitor output, and, by default and for an adjoint test, the
776                    cost-function and gradient-check. However, some test-experiments
777                    use some package-specific variable/monitor output according to
778                    the file <filename>TESTDIR/input[_ad][.OTHER]/tr_checklist</>
779                    specification.</para>
780                </listitem>
781              </varlistentry>
782    
783              <varlistentry>
784                <term>directory <filename>TESTDIR/build/</></term>
785                <listitem>
786                  <para> initially empty directory where <filename>testreport</>
787                    will build the MITgcm executable for forward and adjoint test.
788                    It might contains an experiment specific
789                    <filename>genmake_local</> file (see <xref linkend="genmake">).
790                    </para>
791                  <para> Note that the original <filename>code[_ad]/SIZE.h_mpi</>
792                    is not directly used as "SIZE.h" to build an MPI-executable ;
793                    instead, a local copy <filename>build/SIZE.h.mpi</> is derived
794                    from <filename>code[_ad]/SIZE.h_mpi</>
795                    by adjusting the number of processors (nPx,nPy) according to
796                    <filename>NUMBER_OF_PROCS</> (see <xref linkend="testreport">,
797                    <filename>testreport -MPI</>) ; then it is linked to "SIZE.h"
798                    (<filename> ln -s SIZE.h.mpi SIZE.h </>) before building
799                    the MPI-executable.</para>
800                </listitem>
801              </varlistentry>
802    
803              <varlistentry>
804                <term>directory <filename>TESTDIR/code/</></term>
805                <listitem>
806                  <para>contains the test-experiment specific source code
807                    used to build the MITgcm executable (<filename>mitgcmuv</>)
808                    for forward-test (using <filename>genmake2 -mods=../code</>).
809                    </para>
810                    <para> It can also contain specific source files with the suffix
811                    "<filename>_mpi</>" to be used
812                    <!--
813                    in the <filename>TESTDIR/build/</> directory
814                    -->
815                    in place of the corresponding file (without suffix)
816                    for an MPI test (see <xref linkend="testreport">).
817                    The presence or absence of <filename>SIZE.h_mpi</>
818                    determines whether or not an MPI test on this
819                    test-experiment is performed or skipped.</para>
820                </listitem>
821              </varlistentry>
822    
823              <varlistentry>
824                <term>directory <filename>TESTDIR/code_ad/</></term>
825                <listitem>
826                  <para>contains the test-experiment specific source code
827                    used to build the MITgcm executable (<filename>mitgcmuv_ad</>)
828                    for adjoint-test (using <filename>genmake2 -mods=../code_ad</>).
829                    It can also contain specific source files with the suffix
830                    "<filename>_mpi</>" (see above).</para>
831                </listitem>
832              </varlistentry>
833    
834              <varlistentry>
835                <term>directory <filename>TESTDIR/input/</></term>
836                <listitem>
837                  <para>contains the input and parameter files
838                    used to run the primary forward test of this test-experiment.
839                    </para>
840                  <para>It can also contain specific parameter files with the suffix
841                    "<filename>.mpi</>" to be used
842                    <!--
843                    in the <filename>TESTDIR/run/</> directory
844                    -->
845                    in place of the corresponding file (without suffix) for MPI
846                    test, or with suffix "<filename>.mth</>" to be used for
847                    multi-threaded test (see <xref linkend="testreport">).
848                    The presence or absence of <filename>eedata.mth</>
849                    determines whether or not a multi-threaded test on this
850                    test-experiment is performed or skipped.</para>
851                  <para>To save disk space and reduce downloading time, multiple
852                    copies of the same input file is avoided by using a shell
853                    script <filename>prepare_run</>.
854                    When such a script is found in <filename>TESTDIR/input/ </>,
855                    <filename>testreport</> run this script in directory
856                    <filename>TESTDIR/run/ </> after linking all the input file
857                    from <filename>TESTDIR/input/ </>.
858                    </para>
859                </listitem>
860              </varlistentry>
861    
862              <varlistentry>
863                <term>directory <filename>TESTDIR/input_ad/</></term>
864                <listitem>
865                  <para>contains the input and parameter files
866                    used to run the primary adjoint test of this test-experiment.
867                    It can also contain specific parameter files with the suffix
868                    "<filename>.mpi</>" and shell script <filename>prepare_run</>
869                    as described above.</para>
870                </listitem>
871              </varlistentry>
872    
873              <varlistentry>
874                <term>directory <filename>TESTDIR/input.OTHER/</></term>
875                <listitem>
876                  <para>contains the input and parameter files
877                    used to run the secondary OTHER forward test of this test-experiment.
878                    It can also contain specific parameter files with suffix
879                    "<filename>.mpi</>" or "<filename>.mth</>" and shell script
880                    <filename>prepare_run</> (see above).</para>
881                    <para> The presence or absence the file <filename>eedata.mth</>
882                    determines whether or not a secondary multi-threaded test on this
883                    test-experiment is performed or skipped.</para>
884                </listitem>
885              </varlistentry>
886    
887              <varlistentry>
888                <term>directory <filename>TESTDIR/input_ad.OTHER/</></term>
889                <listitem>
890                  <para>contains the input and parameter files
891                    used to run the secondary OTHER adjoint test of this test-experiment.
892                    It can also contain specific parameter files with the suffix
893                    "<filename>.mpi</>" and shell script <filename>prepare_run</>
894                    (see above).</para>
895                </listitem>
896              </varlistentry>
897                    <!--
898                    -->
899              <varlistentry>
900                <term>directory <filename>TESTDIR/run/</></term>
901                <listitem>
902                  <para> initially empty directory where <filename>testreport</>
903                    will run the MITgcm executable for primary forward and adjoint
904                    test.</para>
905                  <para>Symbolic links (using command "<filename>ln -s</>") are made
906                    for input and parameter files (from <filename>../input/ </>
907                    or from <filename>../input_ad/ </>) and for MITgcm executable
908                    (from <filename>../build/ </>) before the run proceeds.
909                    The sequence of links (function <filename>linkdata</> within shell
910                    script <filename>testreport</>) for a forward test is:</para>
911    <screen>
912    * link+rename or remove links
913           to special files with suffix ".mpi" or ".mth" from ../input/
914    * link files from ../input/
915    * execute ../input/prepare_run (if it exists)
916    </screen>
917                  <para>The sequence for an ajoint test is similar, with
918                    <filename>../input_ad/ </> replacing <filename>../input/ </>.
919                    </para>
920                </listitem>
921              </varlistentry>
922    
923              <varlistentry>
924                <term>directory <filename>TESTDIR/tr_run.OTHER/</></term>
925                <listitem>
926                  <para> directory created by <filename>testreport</>
927                    to run the MITgcm executable for secondary "OTHER" forward
928                    or adjoint test.</para>
929                  <para> The sequence of links for a forward secondary test is:</para>
930    <screen>
931    * link+rename or remove links
932           to special files with suffix ".mpi" or ".mth" from ../input.OTHER/
933    * link files from ../input.OTHER/
934    * execute ../input.OTHER/prepare_run (if it exists)
935    * link files from ../input/
936    * execute ../input/prepare_run (if it exists)
937    </screen>
938                  <para>The sequence for an ajoint test is similar, with
939                    <filename>../input_ad.OTHER/ </> and <filename>../input_ad/ </>
940                    replacing <filename>../input.OTHER/ </> and <filename>../input/ </>.
941                    </para>
942                </listitem>
943              </varlistentry>
944    
945            </variablelist>
946          </sect3>
947    
948        <sect3 id="testreport">        <sect3 id="testreport">
949          <title>The <filename>testreport</> Utility</title>          <title>The <filename>testreport</> Utility</title>
950    
951          <para>Also included in <filename>$ROOTDIR/verification/</> are shell          <para> The shell script  <filename>testreport</> (in
952            scripts for automated testing.  The newest script (which was written            <filename>$ROOTDIR/verification/</>), which was written to work with
953            to work with <filename>genmake2</>) is called <filename>testreport</>.            <filename>genmake2</>, can be used to build different versions
954            This script can be used to build different versions of the MITgcm            of the MITgcm code, run the various examples, compare the output,
955            code, run the various examples, compare the output, and (if specified)            and (if specified) email the results of each one of these tests to a
956            email the results of each one of these tests to a central            central repository.</para>
           repository.</para>  
957    
958          <para>On some systems, the testreport script can be run with a command          <para>On some systems, the testreport script can be run with a command
959          line as simple as:</para>          line as simple as:</para>
960    
961  <screen>  <screen>
962    $ cd verification    $ cd verification
963    $ ./testreport -ieee    $ ./testreport
964  </screen>  </screen>
965    
966          <para>However, some systems (those lacking or wiht a broken "/bin/sh")          <para>However, some systems (those lacking or wiht a broken "/bin/sh")
967            may require an explicit shell invocation such as:</para>            may require an explicit shell invocation such as:</para>
968    
969  <screen>  <screen>
970    $ sh ./testreport -ieee -t 'exp0 exp4'    $ sh ./testreport -t 'exp2 exp4'
971    $ /some/path/to/bash ./testreport -ieee -t 'ideal_2D_oce lab_sea natl_box'    $ /some/path/to/bash ./testreport -t 'ideal_2D_oce lab_sea natl_box'
972  </screen>  </screen>
973    
974          <para>The <filename>testreport</> script accepts a number of          <para>The <filename>testreport</> script accepts a number of
# Line 729  checkpoint50d_pre Line 978  checkpoint50d_pre
978          <variablelist>          <variablelist>
979    
980            <varlistentry>            <varlistentry>
981              <term><filename>-ieee</></term>              <term><filename>-ieee</> (default) / <filename>-noieee</></term>
982              <listitem>              <listitem>
983                <para>If allowed by the compiler (as defined in the "optfile"),                <para>If allowed by the compiler (as defined in the "optfile"),
984                  use IEEE arithmetic.  This option, along with the GCC compiler,                  use IEEE arithmetic (<filename>genmake2 -ieee</>).
985                  is how the standard results were produced.</para>                  This option, along with the gfortran / gcc compiler,
986                    is how the standard results are produced.</para>
987                </listitem>
988              </varlistentry>
989    
990              <varlistentry>
991                <term><filename>-optfile=/PATH/FILENAME</></term>
992                <term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>
993                <listitem>
994                  <para>This specifies a list of "options files" that will be passed
995                    to <filename>genmake2</>.  If multiple options files are used
996                    (say, to test different compilers or different sets of options
997                    for the same compiler), then each options file will be used with
998                    each of the test directories.</para>
999              </listitem>              </listitem>
1000            </varlistentry>            </varlistentry>
1001    
# Line 744  checkpoint50d_pre Line 1006  checkpoint50d_pre
1006                <para>This option specifies the test directory or list of test                <para>This option specifies the test directory or list of test
1007                  directories that should be used.  Each of these entries should                  directories that should be used.  Each of these entries should
1008                  exactly (note: they are case sensitive!) match the names of                  exactly (note: they are case sensitive!) match the names of
1009                  directries in <filename>$ROOTDIR/verification/</>.  If this                  directories in <filename>$ROOTDIR/verification/</>.  If this
1010                  option is omitted, then all directories that are properly                  option is omitted, then all directories that are properly
1011                  formatted (that is, containing an <filename>input</>                  formatted (that is, containing an <filename>input</>
1012                  sub-directory and a <filename>results/output.txt</> file) will                  sub-directory and a <filename>results/output.txt</> file) will
# Line 753  checkpoint50d_pre Line 1015  checkpoint50d_pre
1015            </varlistentry>            </varlistentry>
1016    
1017            <varlistentry>            <varlistentry>
             <term><filename>-optfile=/PATH/FILENAME</></term>  
             <term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>  
             <listitem>  
               <para>This specifies a list of "options files" that will be passed  
                 to <filename>genmake2</>.  If multiple options files are used  
                 (say, to test different compilers or different sets of options  
                 for the same compiler), then each options file will be used with  
                 each of the test directories.</para>  
             </listitem>  
           </varlistentry>  
   
           <varlistentry>  
1018              <term><filename>-addr EMAIL</></term>              <term><filename>-addr EMAIL</></term>
1019              <term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>              <term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>
1020              <listitem>              <listitem>
# Line 778  checkpoint50d_pre Line 1028  checkpoint50d_pre
1028            </varlistentry>            </varlistentry>
1029    
1030            <varlistentry>            <varlistentry>
1031                <term><filename>-MPI NUMBER_OF_PROCS</></term>
1032              <term><filename>-mpi</></term>              <term><filename>-mpi</></term>
1033              <listitem>              <listitem>
1034                <para>If the necessary files                <para>If the necessary file (<filename>TESTDIR/code/SIZE.h_mpi</>)
1035                (<filename>TESTDIR/code/CPP_EEOPTIONS.h_mpi</> and                exists, then use it (and all <filename>TESTDIR/code/*_mpi</> files)
1036                <filename>TESTDIR/code/SIZE.h_mpi</>) exist, then use them for an                for an MPI--enabled run.  The new option (<filename>-MPI</> followed
1037                MPI--enabled run.  Note that the use of MPI typically requires a                by the maximum number of processors) enable to build and run each
1038                  test-experiment using variable number of MPI processors (multiple
1039                  of <filename>nPx*nPy</> from <filename>TESTDIR/code/SIZE.h_mpi</>
1040                  and not larger than <filename>NUMBER_OF_PROCS</>).
1041                  The short option ("-mpi") can only be used to build and run on 2 MPI
1042                  processors (equivalent to "<filename>-MPI 2</>").</para>
1043                  <para>Note that the use of MPI typically requires a
1044                special command option (see "-command" below) to invoke the MPI                special command option (see "-command" below) to invoke the MPI
1045                executable.  Examples of PBS scripts using MPI with testreport can be                executable.  Examples of PBS scripts using testreport with MPI can be
1046                found in the <ulink                found in the <ulink
1047                url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">                url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
1048                MITgcm-contrib area</ulink></para>                tools/example_scripts directory</ulink>.</para>
1049              </listitem>              </listitem>
1050            </varlistentry>            </varlistentry>
1051    
1052            <varlistentry>            <varlistentry>
1053              <term><filename>-command='some command to run'</></term>              <term><filename>-command='some command to run'</></term>
1054              <listitem>              <listitem>
1055                <para>For some tests, particularly MPI runs, the default "make                <para>For some tests, particularly MPI runs, a specific command
1056                output.txt" is not sufficient.  This option allows a more general                might be needed to run the executable. This option allows a more general
1057                command (or shell script) to be invoked.  Examples of PBS scripts                command (or shell script) to be invoked.  Examples of PBS scripts
1058                using MPI with testreport can be found in the <ulink                using testreport with MPI can be found in the <ulink
1059                url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">                url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
1060                MITgcm-contrib area</ulink></para>                tools/example_scripts directory</ulink>.</para>
1061                  <para>
1062                    For the case where the number of MPI processors varies according
1063                    to each test-experiment, some key-words within the command-to-run
1064                    argument will be replaced by their effective value:</para>
1065                  <para>
1066                    <filename>TR_NPROC </> will be replaced by the actual number
1067                    of MPI processors needed to run the current test-experiment.</para>
1068                  <para>
1069                    <filename>TR_MFILE </> will be replaced by the name of local-file
1070                    that testreport creates from the full list of machines which
1071                    "<filename>testreport -mf MACHINE_FILE</>" provides, but truncated
1072                    to the exact number of machines.</para>
1073                </listitem>
1074              </varlistentry>
1075    
1076              <varlistentry>
1077                <term><filename>-mf MACHINE_FILE</></term>
1078                <listitem>
1079                  <para>
1080                    To use with <filename>-MPI NUMBER_OF_PROCS</> option, to specify
1081                    the file containing the full list of <filename>NUMBER_OF_PROCS</>
1082                    machines to use for the MPI runs.</para>
1083                </listitem>
1084              </varlistentry>
1085    
1086              <varlistentry>
1087                <term><filename>-mth</></term>
1088                <listitem>
1089                  <para>compile (with <filename>genmake2 -omp</>) and run with
1090                    multiple threads (using eedata.mth).</para>
1091              </listitem>              </listitem>
1092            </varlistentry>            </varlistentry>
1093    
1094          </variablelist>          </variablelist>
1095    
1096          <para>The <filename>testreport</> script will write progress to the          <para>The <filename>testreport</> script will create an output directory
1097            screen (stdout) as it runs.  In addition, it will create a            <filename>tr_NAME_DATE_N/ </>, with <filename>hostname</> as default
1098            <filename>tr_out.txt</> file that contains a brief comparison of the            <filename>NAME</>, <filename>DATE</> the current date followed by
1099            current output with the "known-good" output.</para>            a suffix number "N" to distinguish from previous
1100              <filename>testreport</> output directories.
1101              <filename>testreport</> writes progress to the screen (stdout)
1102              and reports into the ouput directory as it runs.
1103              In particular, one can find, in the ouput directory,
1104              the <filename>summary.txt</> file that contains a brief comparison
1105              of the current output with the "known-good" output.
1106              At the end of the testing process, the <filename>tr_out.txt</> file
1107              is generated in <filename>$ROOTDIR/verification/ </>
1108              as a compact version of <filename>summry.txt</> file.</para>
1109    
1110        </sect3>        </sect3>
1111    
1112          <sect3 id="tst_2plus2">
1113            <title>The <filename>do_tst_2+2</> Utility</title>
1114            <para> The shell script  <filename>do_tst_2+2</> (in
1115              <filename>$ROOTDIR/tools/ </>)
1116              can be used to check the accuracy of the restart procedure.
1117            </para>
1118          </sect3>
1119    
1120      </sect2>      </sect2>
1121    
1122    
# Line 826  checkpoint50d_pre Line 1130  checkpoint50d_pre
1130          the generic fluid dynamical engine.</para>          the generic fluid dynamical engine.</para>
1131    
1132        <para>The MITgcmUV packaging structure is described below using generic        <para>The MITgcmUV packaging structure is described below using generic
1133          package names ${pkg}.  A concrete examples of a package is the code for          package names ${pkg}. A concrete examples of a package is the code for
1134          implementing GM/Redi mixing. This code uses the package name</para>          implementing GM/Redi mixing. This code uses the package name</para>
1135    
1136      </sect2>      </sect2>
# Line 851  separated from the generic fluid dynamic Line 1155  separated from the generic fluid dynamic
1155  below using generic package names ${pkg}.  below using generic package names ${pkg}.
1156  A concrete examples of a package is the code  A concrete examples of a package is the code
1157  for implementing GM/Redi mixing. This code uses  for implementing GM/Redi mixing. This code uses
1158  the package name  the package name
1159  *   ${PKG} = GMREDI  *   ${PKG} = GMREDI
1160  *   ${pkg} = gmredi  *   ${pkg} = gmredi
1161  *   ${Pkg} = gmRedi  *   ${Pkg} = gmRedi
# Line 886  o  Each package gets its runtime configu Line 1190  o  Each package gets its runtime configu
1190     Package runtime config. options are imported     Package runtime config. options are imported
1191     into a common block held in a header file     into a common block held in a header file
1192     called "${PKG}.h".     called "${PKG}.h".
1193     Note: In some packages, the header file "${PKG}.h" is splitted     Note: In some packages, the header file "${PKG}.h" is splitted
1194     into "${PKG}_PARAMS.h" that contains the package parameters and     into "${PKG}_PARAMS.h" that contains the package parameters and
1195     ${PKG}_VARS.h" for the field arrays.     ${PKG}_VARS.h" for the field arrays.
1196    
1197  o  The core driver part of the model can check  o  The core driver part of the model can check
# Line 911  CPP Flags Line 1215  CPP Flags
1215      1. Within the core driver code flags of the form      1. Within the core driver code flags of the form
1216         ALLOW_${PKG} are used to include or exclude         ALLOW_${PKG} are used to include or exclude
1217         whole packages. The ALLOW_${PKG} flags are included         whole packages. The ALLOW_${PKG} flags are included
1218         from a PACKAGES_CONFIG.h file that is automatically         from a PACKAGES_CONFIG.h file that is automatically
1219         generated by genmake2 (see genmake2 section).         generated by genmake2 (see genmake2 section).
1220         held in-line in the CPP_OPTIONS.h header file.         held in-line in the CPP_OPTIONS.h header file.
1221         e.g.         e.g.
# Line 930  CPP Flags Line 1234  CPP Flags
1234    
1235      2. Within an individual package a header file,      2. Within an individual package a header file,
1236         "${PKG}_OPTIONS.h", is used to set CPP flags         "${PKG}_OPTIONS.h", is used to set CPP flags
1237         specific to that package. It also includes         specific to that package. It also includes
1238         "PACKAGES_CONFIG.h" and "CPP_OPTIONS.h".         "PACKAGES_CONFIG.h" and "CPP_OPTIONS.h".
1239    
1240    
# Line 944  Package Boot Sequence Line 1248  Package Boot Sequence
1248      1. S/R PACKAGES_BOOT()      1. S/R PACKAGES_BOOT()
1249              :              :
1250          CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )          CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
1251    
1252    
1253      2. S/R PACKAGES_READPARMS()      2. S/R PACKAGES_READPARMS()
1254              :              :
# Line 983  Package Output Line 1287  Package Output
1287    
1288          #ifdef ALLOW_${PKG}          #ifdef ALLOW_${PKG}
1289            if ( use${Pkg} )            if ( use${Pkg} )
1290       &       CALL ${PKG}_DIAGS( )     [ or CALL ${PKG}_OUTPUT( ) ]       &       CALL ${PKG}_OUTPUT( )
1291          #endif          #endif
1292    
1293       7. S/R PACKAGES_WRITE_PICKUP()       7. S/R PACKAGES_WRITE_PICKUP()
# Line 996  Package Output Line 1300  Package Output
1300  Description  Description
1301  ===========  ===========
1302    
1303        - ${PKG}_READPARMS()        - ${PKG}_READPARMS()
1304      is responsible for reading      is responsible for reading
1305      in the package parameters file data.${pkg}, and storing      in the package parameters file data.${pkg}, and storing
1306      the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h").      the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h").
1307      -> called from INITIALISE_FIXED in PACKAGES_READPARMS      -> called from INITIALISE_FIXED in PACKAGES_READPARMS
1308    
1309       - ${PKG}_INIT_FIXED()       - ${PKG}_INIT_FIXED()
1310      is responsible for completing the internal setup of a package.      is responsible for completing the internal setup of a package.
1311      -> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED      -> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED
1312      note: 1) some pkg use instead:      note: 1) some pkg use instead:
1313               CALL ${PKG}_INITIALISE  ( or the old form CALL ${PKG}_INIT )               CALL ${PKG}_INITIALISE  ( or the old form CALL ${PKG}_INIT )
1314            2) for simple pkg setup, this part is done inside ${PKG}_READPARMS            2) for simple pkg setup, this part is done inside ${PKG}_READPARMS
1315    
1316       - ${PKG}_CHECK()       - ${PKG}_CHECK()
1317      is responsible for validating      is responsible for validating
1318      basic package setup and inter-package dependencies.      basic package setup and inter-package dependencies.
1319      ${PKG}_CHECK can import other package parameters it may      ${PKG}_CHECK can import other package parameters it may
# Line 1018  Description Line 1322  Description
1322      will not be reset during ${PKG}_CHECK().      will not be reset during ${PKG}_CHECK().
1323      -> called from INITIALISE_FIXED in PACKAGES_CHECK      -> called from INITIALISE_FIXED in PACKAGES_CHECK
1324    
1325       - ${PKG}_INIT_VARIA()       - ${PKG}_INIT_VARIA()
1326      is responsible for fill-in all package variables with an initial value.      is responsible for fill-in all package variables with an initial value.
1327      Contains eventually a call to ${PKG}_READ_PICKUP that will read      Contains eventually a call to ${PKG}_READ_PICKUP that will read
1328      from a pickup file the package variables required to restart the model.      from a pickup file the package variables required to restart the model.
1329      This routine is called after the core model state has been completely      This routine is called after the core model state has been completely
1330      initialised but before the core model timestepping starts.      initialised but before the core model timestepping starts.
1331      -> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES      -> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES
1332      note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg      note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg
1333       use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old       use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old
1334       form ${PKG}_INIT       form ${PKG}_INIT
1335    
1336       - ${PKG}_DIAGS()      [or ${PKG}_OUTPUT( ) ]       - ${PKG}_OUTPUT( )
1337       is responsible for writing time-average fields to output files       is responsible for writing time-average fields to output files
1338       (but the cumulating step is done within the package main S/R).       (but the cumulating step is done within the package main S/R).
1339       Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR)       Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR)
1340       and write snap-shot fields that are hold in common blocks. Other       and write snap-shot fields that are hold in common blocks. Other
1341       temporary fields are directly dump to file where they are available.       temporary fields are directly dump to file where they are available.
1342       NOTE: 1) ${PKG}_OUTPUT is progressively replacing ${PKG}_DIAGS()       NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages
1343                  but is beeing replaced by ${PKG}_OUTPUT
1344                to avoid confusion with pkg/diagnostics functionality.                to avoid confusion with pkg/diagnostics functionality.
1345             2) the output part is not yet in a standard form and might still             2) the output part is not yet in a standard form and might still
1346                evolve a lot.                evolve a lot.
1347      -> called within DO_THE_MODEL_IO      -> called within DO_THE_MODEL_IO
1348    
1349       - ${PKG}_WRITE_PICKUP()       - ${PKG}_WRITE_PICKUP()
1350       is responsible for writing a package pickup file when necessary for       is responsible for writing a package pickup file when necessary for
1351       a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT )       a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT )
1352      -> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP      -> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP
1353    
# Line 1075  Summary Line 1380  Summary
1380    * ${pkg}_check.F        checks package dependencies and consistencies    * ${pkg}_check.F        checks package dependencies and consistencies
1381    * ${pkg}_init_varia.F   initialises package-related fields    * ${pkg}_init_varia.F   initialises package-related fields
1382    * ${pkg}_... .F         package source code    * ${pkg}_... .F         package source code
1383    * ${pkg}_diags.F        write output to file.    * ${pkg}_output.F       write output to file.
    or ${pkg}_output.F     write output to file.  
1384    * ${pkg}_write_pickup.F write a package pickup file to restart the model    * ${pkg}_write_pickup.F write a package pickup file to restart the model
1385    
1386    New: Subroutine in one package (pkgA) that only contains code which    New: Subroutine in one package (pkgA) that only contains code which
1387         is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F)         is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F)
1388         will be named: pkgA_pkgB_something.F         will be named: pkgA_pkgB_something.F
1389    
# Line 1091  Summary Line 1395  Summary
1395    </sect1>    </sect1>
1396    
1397    
1398      <sect1 id="documentation">
1399        <title>Editing the Documentation</title>
1400    
1401        <sect2 id="documentation_getting">
1402          <title>Getting the Docs and Code</title>
1403    
1404          <para>The first step towards editing the documentation is to checkout a
1405          copy of code, docs, and build scripts from the CVS server using:</para>
1406    
1407    <screen>
1408      $ export CVS_RSH=ssh
1409      $ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack'
1410      $ mkdir scratch
1411      $ cvs co -P MITgcm manual mitgcm.org
1412    </screen>
1413    
1414          <para>These commands extract the necessary information from the CVS server
1415          and create a temporary (called <filename>scratch</filename>) directory for
1416          the storage of the HTML and other files that will be created.  Please note
1417          that you must either create <filename>scratch</filename> as shown or edit
1418          the various <filename>Makefile</filename>s and scripts used to create the
1419          documentation.</para>
1420        </sect2>
1421    
1422        <sect2>
1423          <title>Editing the Documentation</title>
1424    
1425          <para>The documentation is contained in the <filename>manual</filename>
1426          directory in a raw LaTeX format.  The main document is
1427          <filename>manual.tex</filename> and it uses <command>\input{}</command>s
1428          to include the chapters and subsections.</para>
1429    
1430          <para>Since the same LaTeX source is used to produce PostScript, PDF, and
1431          HTML output, care should be taken to follow certain conventions.  Two of
1432          the most important are the usage of the <command>\filelink{}{}</command>
1433          and <command>\varlink{}{}</command> commands.  Both of these commands have
1434          been defined to simplify the connection between the automatically
1435          generated ("code browser") HTML and the HTML version of the manual
1436          produced by LaTeX2HTML.  They each take two arguments (corresponding to
1437          the contents of the two sets of curly braces) which are the text that the
1438          author wishes to be "wrapped" within the link, and a specially formatted
1439          link thats relative to the <filename>MITgcm</filename> directory within
1440          the CVS tree.</para>
1441    
1442          <para>The result is a command that resembles either</para>
1443    
1444          <orderedlist>
1445            <listitem>
1446              <para>a reference to a variable or subroutine name such as
1447              <command>\varlink{tRef}{tRef}</command>, or </para>
1448            </listitem>
1449    
1450            <listitem>
1451              <para>a reference to a file such as
1452                  <command>\varlink{tRef}{path-to-the-file_name.F}</command>
1453                  where the absolute path to the file is of the form
1454                  <filename>/foo/MITgcm/path/to/the/file_name.F</filename></para>
1455                  <para>(please note how the leading "/foo/MITgcm"
1456                  component of the path is dropped leaving the path
1457                  <emphasis>relative</emphasis> to the head of the code
1458                  directory and each directory separator "/" is turned
1459                  into a "-")</para>
1460            </listitem>
1461          </orderedlist>
1462            
1463    
1464    
1465        </sect2>
1466    
1467        <sect2>
1468          <title>Building the Documentation</title>
1469    
1470          <para>Given the directory structure of <xref
1471          linkend="documentation_getting">, the entire documentation for the web
1472          site can be built using:</para>
1473    
1474    <screen>
1475      $ cd mitgcm.org/devel/buildweb
1476      $ make All
1477    </screen>
1478    
1479          <para>Which builds the PDF from the LaTeX source, creates the HTML output
1480          from the LaTeX source, parses the FORTRAN code base to produce a
1481          hyperlinked HTML version of the source, and then determines the
1482          cross-linking between the various HTML components.</para>
1483    
1484          <para>If there are no errors, the result of the build process (which can
1485          take 30+ minutes on a P4/2.5Ghz) will be contained within a single
1486          directory called <filename>scratch/dev_docs</filename>.  This is a freshly
1487          built version of the entire on-line users manual.  If you have the correct
1488          permissions, it can be directly copied to the web server area:</para>
1489    
1490    <screen>
1491      $ mv scratch/dev_docs /u/u0/httpd/html
1492    </screen>
1493    
1494          <para>and the update is complete.</para>
1495    
1496        </sect2>
1497    
1498      </sect1>
1499    
1500  </article>  </article>
1501    
1502    

Legend:
Removed from v.1.8  
changed lines
  Added in v.1.18

  ViewVC Help
Powered by ViewVC 1.1.22