/[MITgcm]/MITgcm/doc/devel_HOWTO.sgml

Diff of /MITgcm/doc/devel_HOWTO.sgml

Parent Directory | Revision Log | View Revision Graph Revision Graph | View Patch Patch

-revision 1.2 by edhill,
Wed Aug 20 19:47:14 2003 UTC
+revision 1.9 by jmc,
Tue May  9 21:10:19 2006 UTC
 Line 2
  <article id="MITgcm-Development-HOWTO">
- <!--
- Build commands:
-   db2ps -d ldp.dsl devel_HOWTO.sgml
-   db2pdf -d ldp.dsl devel_HOWTO.sgml
-   db2html -d ./ldp.dsl devel_HOWTO.sgml
-   db2html -u -d ./ldp.dsl devel_HOWTO.sgml
- -->
    <articleinfo>
      <title>MITgcm Development HOWTO</title>
-Line 63 
 Build commands:
+Line 55 
 Build commands:
      <sect2>
        <title>User Manual</title>
-       <para>Before jumping into
+       <para>Before jumping into development, please familiarize yourself with
-       development, please familiarize yourself with the MITgcm user
+         the <ulink url="http://mitgcm.org/docs.html"> MITgcm user manual
-       manual which is available <ulink
+         </ulink>.  This document contains volumes of useful information and is
-       url="http://mitgcm.org/">on the main web page</ulink>.  This
+         included here by reference.</para>
-       document contains volumes of useful information and is included
-       here by reference.</para>
-       <para>Also, a "snapshot" or<ulink
+       <!--
+       <para>Also, a "snapshot" or <ulink
        url="http://mitgcm.org/dev_docs/">development version</ulink> of
        the user manual may be available, though this is only put on the
        web for testing purposes.</para>
+       -->
      </sect2>
      <sect2>
-Line 128 
 Build commands:
+Line 120 
 Build commands:
         doc                      tags -- connect to real docs?
         eesupp                   cnh?
         exe                      ecco user build
-     *- jobs                     runtime shell scripts for
+     ,- jobs                     runtime shell scripts for
      |                             various platforms
      |  lsopt                    line search
     m|  model                    main dynamics (core)
     e|    optimization_drivers   ?
     r|  optim                    line search interface
     g|  pkg                      alternate and optional numerics, etc.
-    e*- tools
+    e|- tools
     ?|  tutorial_examples        documented tests
      |                             only populated on release1 branch
      |                             and not validated during "testscript"
-     *- utils
+     '- utils
         verification             std tests
-Line 190 
 Build commands:
+Line 182 
 Build commands:
        <title>Branches</title>
        <para>As shown in the online <ulink
-       url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated
+       url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">
-       tree</ulink>, the MITgcm codebase is split into to two branches
+       ViewCVS-generated tree</ulink>, the MITgcm codebase is split into to two
-       or "lines" under which development proceeds.  These two lines
+       branches or "lines" under which development proceeds.  These two lines are
-       are referred to as the "MAIN" and "ecco" versions of the code.
+       referred to as the "MAIN" and "ecco" versions of the code.  While not
-       While not identical, the bulk of the MAIN and ecco lines are
+       identical, the bulk of the MAIN and ecco lines are composed of files from
-       composed of files from the same codebase.
+       the same codebase.
        </para>
        <para>Periodically, a "Release" branch is formed from the "MAIN"
-       development branch.  This is done in order to create a
+       development branch.  This is done in order to create a relatively stable
-       relatively stable reference point for both users and developers.
+       reference point for both users and developers.  The intent is that once a
-       The intent is that once a relese branch has been created, only
+       relese branch has been created, only bug-fixes will be added to it.
-       bug-fixes will be added to it.  Meanwhile, development (which
+       Meanwhile, development (which might "break" or otherwise render invalid
-       might "break" or otherwise render invalid the documentation,
+       the documentation, tutorials, and/or examples contained within a release
-       tutorials, and/or examples contained within a release branch) is
+       branch) is allowed to continue along the MAIN and ecco lines.</para>
-       allowed to continue along the MAIN and ecco lines.</para>
      </sect2>
      <sect2>
        <title>Tagging</title>
-       <para>The intent of tagging is to create "known-good"
+       <para>The intent of tagging is to create "known-good" checkpoints that
-       checkpoints that developers can use as references.
+       developers can use as references.  Traditionally, MITgcm tagging has
-       Traditionally, MITgcm tagging has maintained the following
+       maintained the following conventions:</para>
-       conventions:</para>
        <orderedlist>
          <listitem>
-           <para>Developer checks out code into a local CVS-managed
+           <para>Developer checks out code into a local CVS-managed directory,
-           directory, makes various changes/additions, tests these
+           makes various changes/additions, tests these edits, and eventually
-           edits, and eventually reaches a point where (s)he is
+           reaches a point where (s)he is satisfied that the changes form a new
-           satisfied that the changes form a new "useful" point in the
+           "useful" point in the evolution of the code.</para>
-           evolution of the code.</para>
          </listitem>
          <listitem>
            <para>The developer then runs the <ulink
-           url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink>
+           url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">
-           shell script to see if any problems are introduced.  While
+           testscript</ulink> shell script to see if any problems are introduced.
-           not intended to be exhaustive, the test cases within the
+           While not intended to be exhaustive, the test cases within the
-           verification directory do provide some indication whether
+           verification directory do provide some indication whether gross errors
-           gross errors have been introduced.
+           have been introduced.
            </para>
          </listitem>
-Line 241 
 Build commands:
+Line 230 
 Build commands:
            then:</para>
            <orderedlist>
              <listitem>
-               <para>adds a "checkpointXY_pre" comment (where X is a
+               <para>adds a "checkpointXY_pre" comment (where X is a checkpoint
-               checkpoint number and Y is a letter) to the <ulink
+               number and Y is a letter) to the <ulink
-               url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink>
+               url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">
-               file and checks it into the CVS repository</para>
+               tag-index</ulink> file and checks it into the CVS
+               repository</para>
              </listitem>
              <listitem>
-               <para>submits the set of changes to the CVS repository
+               <para>submits the set of changes to the CVS repository and adds
-               and adds comments to <filename>tag-index</filename>
+               comments to <filename>tag-index</filename> describing what the
-               describing what the changes are along with a matching
+               changes are along with a matching "checkpointXY_post" entry</para>
-               "checkpointXY_post" entry</para>
              </listitem>
            </orderedlist>
          </listitem>
        </orderedlist>
-       <para>The result of this tagging procedure is a sequence of
+       <para>The result of this tagging procedure is a sequence of development
-       development checkpoints with comments which resembles:</para>
+       checkpoints with comments which resembles:</para>
  <programlisting>
  checkpoint50e_post
-Line 279 
 o fix small problem with in ptracers_wri
+Line 268 
 o fix small problem with in ptracers_wri
  checkpoint50d_pre
  </programlisting>
-       <para>This information can be used to refer to various stages of
+       <para>This information can be used to refer to various stages of the code
-       the code development.  For example, bugs can be traced to
+       development.  For example, bugs can be traced to individual sets of CVS
-       individual sets of CVS checkins based upon their first
+       checkins based upon their first appearance when comparing the results from
-       appearance when comparing the results from different
+       different checkpoints.</para>
-       checkpoints.</para>
      </sect2>
    </sect1>
-Line 295 
 checkpoint50d_pre
+Line 283 
 checkpoint50d_pre
      <sect2 id="documentation_getting">
        <title>Getting the Docs and Code</title>
-       <para>The first step towards editing the documentation is to
+       <para>The first step towards editing the documentation is to checkout a
-       checkout a copy of code, docs, and build scripts from the CVS
+       copy of code, docs, and build scripts from the CVS server using:</para>
-       server using:</para>
  <screen>
- $ export CVS_RSH=ssh
+   $ export CVS_RSH=ssh
- $ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack'
+   $ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack'
- $ mkdir scratch
+   $ mkdir scratch
- $ cvs co MITgcm manual mitgcm.org
+   $ cvs co -P MITgcm manual mitgcm.org
  </screen>
-       <para>These commands extract the necessary information from the
+       <para>These commands extract the necessary information from the CVS server
-       CVS server and create a temporary (called
+       and create a temporary (called <filename>scratch</filename>) directory for
-       <filename>scratch</filename>) directory for the storage of the
+       the storage of the HTML and other files that will be created.  Please note
-       HTML and other files that will be created.  Please note that you
+       that you must either create <filename>scratch</filename> as shown or edit
-       must either create <filename>scratch</filename> as shown or edit
+       the various <filename>Makefile</filename>s and scripts used to create the
-       the various <filename>Makefile</filename>s and scripts used to
+       documentation.</para>
-       create the documentation.</para>
      </sect2>
      <sect2>
-       <title>Editing</title>
+       <title>Editing the Documentation</title>
-       <para>The documentation is contained in the
+       <para>The documentation is contained in the <filename>manual</filename>
-       <filename>manual</filename> directory in a raw LaTeX format.
+       directory in a raw LaTeX format.  The main document is
-       The main document is <filename>manual.tex</filename> and it uses
+       <filename>manual.tex</filename> and it uses <command>\input{}</command>s
-       <command>\input{}</command>s to include the chapters and
+       to include the chapters and subsections.</para>
-       subsections.</para>
+       <para>Since the same LaTeX source is used to produce PostScript, PDF, and
-       <para>Since the same LaTeX source is used to produce PostScript,
+       HTML output, care should be taken to follow certain conventions.  Two of
-       PDF, and HTML output, care should be taken to follow certain
+       the most important are the usage of the <command>\filelink{}{}</command>
-       conventions.  Two of the most important are the usage of the
+       and <command>\varlink{}{}</command> commands.  Both of these commands have
-       <command>\filelink{}{}</command> and
+       been defined to simplify the connection between the automatically
-       <command>\varlink{}{}</command> commands.  Both of these
+       generated ("code browser") HTML and the HTML version of the manual
-       commands have been defined to simplify the connection between
+       produced by LaTeX2HTML.  They each take two arguments (corresponding to
-       the automatically generated ("code browser") HTML and the HTML
+       the contents of the two sets of curly braces) which are the text that the
-       version of the manual produced by LaTeX2HTML.  They each take
+       author wishes to be "wrapped" within the link, and a specially formatted
-       two arguments (corresponding to the contents of the two sets of
+       link thats relative to the <filename>MITgcm</filename> directory within
-       curly braces) which are the text that the author wishes to be
+       the CVS tree.</para>
-       "wrapped" within the link, and a specially formatted link thats
-       relative to the <filename>MITgcm</filename> directory within the
-       CVS tree.</para>
        <para>The result is a command that resembles either</para>
-Line 364 
 $ cvs co MITgcm manual mitgcm.org
+Line 347 
 $ cvs co MITgcm manual mitgcm.org
      </sect2>
      <sect2>
-       <title>Building</title> <para>Given the directory structure of
+       <title>Building the Documentation</title>
-       <xref linkend="documentation_getting">, the entire documentation
-       for the web site can be built using:</para>
+       <para>Given the directory structure of <xref
+       linkend="documentation_getting">, the entire documentation for the web
+       site can be built using:</para>
  <screen>
- $ cd mitgcm.org/devel/buildweb
+   $ cd mitgcm.org/devel/buildweb
- $ make All
+   $ make All
  </screen>
-       <para>Which builds the PDF from the LaTeX source, creates the
+       <para>Which builds the PDF from the LaTeX source, creates the HTML output
-       HTML output from the LaTeX source, parses the FORTRAN code base
+       from the LaTeX source, parses the FORTRAN code base to produce a
-       to produce a hyperlinked HTML version of the source, and then
+       hyperlinked HTML version of the source, and then determines the
-       determines the cross-linking between the various HTML
+       cross-linking between the various HTML components.</para>
-       components.</para>
+       <para>If there are no errors, the result of the build process (which can
-       <para>If there are no errors, the result of the build process
+       take 30+ minutes on a P4/2.5Ghz) will be contained within a single
-       (which can take 30+ minutes on a P4/2.5Ghz) will be contained
+       directory called <filename>scratch/dev_docs</filename>.  This is a freshly
-       within a single directory called
+       built version of the entire on-line users manual.  If you have the correct
-       <filename>scratch/dev_docs</filename>.  This is a freshly built
+       permissions, it can be directly copied to the web server area:</para>
-       version of the entire on-line users manual.  If you have the
-       correct permissions, it can be directly copied to the web server
-       area:</para>
  <screen>
- $ mv scratch/dev_docs /u/u0/httpd/html
+   $ mv scratch/dev_docs /u/u0/httpd/html
  </screen>
        <para>and the update is complete.</para>
-Line 398 
 $ mv scratch/dev_docs /u/u0/httpd/html
+Line 380 
 $ mv scratch/dev_docs /u/u0/httpd/html
    </sect1>
    <sect1 id="coding">
-     <title>Coding</title>
+     <title>Coding for MITgcm</title>
+     <sect2 id="build_tools">
+       <title>Build Tools</title>
+       <para>Many Open Source projects use the "GNU Autotools" to help streamline
+         the build process for various Unix and Unix-like architectures.  For a
+         user, the result is the common "configure" (that is,
+         "<filename>./configure && make && make install</filename>") commands.
+         For MITgcm, the process is similar.  Typical commands are:</para>
+ <screen>
+   $ genmake -mods=../code
+   $ make depend
+   $ make
+ </screen>
+       <para>The following sections describe the individual steps in the build
+         process.</para>
+       <sect3 id="genmake">
+         <title>The <filename>genmake2</> Utility</title>
+         <para><emphasis>Please note that the older <filename>genmake</> is
+             deprecated and will eventually be replaced by <filename>genmake2</>.
+             This HOWTO only describes the newer tool.</emphasis></para>
+         <para>The first step in any MITgcm build is to create a Unix-style
+           <filename>Makefile</filename> which will be parsed by
+           <filename>make</filename> to specify how to compile the MITgcm source
+           files.  For more detailed descriptions of what the make tools are and
+           how they are used, please see:</para>
+         <itemizedlist>
+           <listitem>
+             <para><ulink url="http://www.gnu.org/software/make/make.html">
+                 http://www.gnu.org/software/make/make.html</></para>
+           </listitem>
+           <listitem>
+             <para><ulink url="http://www.oreilly.com/catalog/make2/">
+                 http://www.oreilly.com/catalog/make2/</></para>
+           </listitem>
+         </itemizedlist>
+         <para>Genmake can often be invoked successfully with a command line as
+         simple as:</para>
+ <screen>
+   $ genmake2 -mods=../code
+ </screen>
+         <para>However, some systems (particularly commercial Unixes that lack a
+           more modern "/bin/sh" implementation or that have shells installed in
+           odd locations) may require an explicit shell invocation such as one of
+           the following: </para>
+ <screen>
+   $ /usr/bin/sh genmake2 -make=gmake  -mods=../code
+   $ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code
+ </screen>
+         <para>The genmake2 code has been written in a Bourne and BASH (v1)
+         compatible syntax so it should work with most "sh" and all recent "bash"
+         implementations.</para>
+         <para>As the name implies, <filename>genmake2</filename> generates a
+           <filename>Makefile</filename>.  It does so by first parsing the
+           information supplied from the following sources</para>
+         <orderedlist>
+           <listitem>
+             <para>a <filename>gemake_local</filename> file in the current
+               directory</para>
+           </listitem>
+           <listitem>
+             <para>directly from command-line options</para>
+           </listitem>
+           <listitem>
+             <para>an "options file" as specified by the command-line option
+               <filename>-optfile='FILENAME'</filename></para>
+           </listitem>
+         </orderedlist>
+         <para>then checking certain dependency rules (the package dependencies),
+           and finally writing a <filename>Makefile</filename> based upon the
+           source code that it finds.  For convenience within various Unix
+           shells, <filename>genmake2</> supports both "long"- and "short"-style
+           options.  A complete list of the available options can be obtained
+           from:</para>
+ <screen>
+   $ genmake2 -help
+ </screen>
+         <para>The most important options for <filename>genmake2</> are:</para>
+         <variablelist>
+           <varlistentry>
+             <term><filename>--optfile=/PATH/FILENAME</></term>
+             <listitem>
+               <para>This specifies the "options file" that should be used for a
+                 particular build.  The options file is a convenient and
+                 machine-indepenent way of specifying parameters such as the
+                 FORTRAN compiler (<filename>FC=</>), FORTRAN compiler
+                 optimization flags (<filename>FFLAGS=</>), and the locations of
+                 various platform- and/or machine-specific tools
+                 (eg. <filename>MAKEDEPEND=</>).  As with <filename>genmake2</>,
+                 all options files should be written to be compatible with
+                 Bourne--shell ("sh" or "BASH v1") syntax.  Examples of various
+                 options files can be found in
+                 <filename>$ROOTDIR/tools/build_options</>.</para>
+               <para>If no "optfile" is specified (either through the command lin
+                 or the environment variable), genmake2 will try to make a
+                 reasonable guess from the list provided in
+                 <filename>$ROOTDIR/tools/build_options</>.  The method used for
+                 making this guess is to first determine the combination of
+                 operating system and hardware (eg. "linux_ia32") and then find a
+                 working Fortran compiler within the user's path.  When these
+                 three items have been identified, genmake2 will try to find an
+                 optfile that has a matching name. </para>
+               <para>Everyone is encouraged to submit their options files to the
+                 MITgcm project for inclusion (please send to
+                 <email>MITgcm-support@mitgcm.org</email>).  We are particularly
+                 grateful for options files tested on new or unique
+                 platforms!</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-pdepend=/PATH/FILENAME</></term>
+             <listitem>
+               <para>This specifies the dependency file used for packages.  If
+               not specified, the default dependency file is
+               <filename>$ROOTDIR/pkg/pkg_depend</>.  The syntax for this file is
+               parsed on a line-by-line basis where each line containes either a
+               comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule
+               where the "+" or "-" symbol specifies a "must be used with" or a
+               "must not be used with" relationship, respectively.  If no rule is
+               specified, then it is assumed that the two packages are compatible
+               and will function either with or without each other.</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-pdefault=PKG</></term>
+             <term><filename>-pdefault='PKG1 [PKG2 PKG3 ...]'</></term>
+             <listitem>
+               <para>This option specifies the default set of packages
+               to be used.  If not set, the default package list will
+               be read from
+               <filename>$ROOTDIR/pkg/pkg_default</>.</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-adof=/path/to/file</></term>
+             <term><filename>-adoptfile=/path/to/file</></term>
+             <listitem>
+               <para>This option specifies the "adjoint" or automatic
+                 differentiation options file to be used.  The file is analogous
+                 to the "optfile" defined above but it specifies information for
+                 the AD build process.  The default file is located in <filename>
+                 $ROOTDIR/tools/adjoint_options/adjoint_default </> and it
+                 defines the "TAF" and "TAMC" compilers.  An alternate version is
+                 also available at <filename>
+                 $ROOTDIR/tools/adjoint_options/adjoint_staf </> that selects the
+                 newer "STAF" compiler.  As with any compilers, it is helpful to
+                 have their directories listed in your $PATH environment
+                 variable.</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-mods=DIR</></term>
+             <term><filename>-mods='DIR1 [DIR2 ...]'</></term>
+             <listitem>
+               <para>This option specifies a list of directories containing
+                 "modifications".  These directories contain files with names
+                 that may (or may not) exist in the main MITgcm source tree but
+                 will be overridden by any identically-named sources within the
+                 "MODS" directories.  The order of precedence for this
+                 "name-hiding" is as follows:</para>
+               <itemizedlist>
+                 <listitem><para>"MODS" directories (in the order given)
+                   </para></listitem>
+                 <listitem><para>Packages either explicitly specified or
+                     provided by default (in the order given)</para></listitem>
+                 <listitem><para>Packages included due to package dependencies
+                     (in the order that that package dependencies are
+                     parsed)</para></listitem>
+                 <listitem><para>The "standard dirs" (which may have been
+                     specified by the "-standarddirs" option)</para></listitem>
+               </itemizedlist>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-make=/path/to/gmake</></term>
+             <listitem>
+               <para>Due to the poor handling of soft-links and other bugs common
+                 with the <filename>make</> versions provided by commercial Unix
+                 vendors, GNU <filename>make</filename> (sometimes called
+                 <filename>gmake</filename>) should be preferred.  This option
+                 provides a means for specifying the make program to be
+                 used.</para>
+             </listitem>
+           </varlistentry>
+         </variablelist>
+         <para>A successful run of <filename>genmake2</> will produce a
+           <filename>Makefile</>, a <filename>PACKAGES_CONFIG.h</> file, and
+           various convenience files used for the automatic differentiation
+           process.</para>
+         <para>In general, it is best to use <filename>genmake2</> on a "clean"
+           directory that is free of all source (*.[F,f],*.[F,f]90) and header
+           (*.h,*.inc) files.  Generally, this can be accomplished in an
+           "un-clean" directory by running "make CLEAN" followed by "make
+           makefile".</para>
+       </sect3>
+       <sect3 id="makefile_use">
+         <title>Using the <filename>Makefile</></title>
+         <para>Once a <filename>Makefile</> has been created using
+           <filename>genmake2</>, one can build a "standard" (forward
+           simulator) executable using:</para>
+ <screen>
+   $ make CLEAN
+   $ make depend
+   $ make
+ </screen>
+         <para>The "make CLEAN" step will remove any stale source files, include
+           files, and links.  It is strongly recommended for "un-clean"
+           directories which may contain the (perhaps partial) results of
+           previous builds.  Such "debris" can interfere with the next stage of
+           the build.</para>
+         <para>The "make depend" step will create a large number of symbolic
+           links from the local directory to the source file locations.  It also
+           parses these files and creates an extensive list of dependencies
+           within the <filename>Makefile</> itself.  The links that exist at this
+           stage are mostly "large F" files (*.F and *.F90) that need to be
+           processed by a C preprocessor ("CPP").  Since "make depend" edits the
+           <filename>Makefile</>, it is important not to skip this step!</para>
+         <para>The final "make" invokes the C preprocessor to produce the "little
+           f" files (*.f and *.f90) and then compiles them to object code using
+           the specified FORTRAN compiler and options.  An intermediate script is
+           often used during this stage to further process (usually, make simple
+           substitutions) custom definitions such as variable types within the
+           source files.  This additional stage is necessary in order to overcome
+           some of the inconsistencies in the sizes of objects (bytes) between
+           different compilers. The result of the build process is an executable
+           with the name <filename>mitgcmuv</>.</para>
+         <para>In addition to the forward simulator described above, the
+           <filename>Makefile</> also has a number of targets that can be used to
+           produce various adjoint and tangent-linear builds for optimization and
+           other parameter-sensitivity problems.  The additional targets within
+           the <filename>Makefile</> are:</para>
+         <variablelist>
+           <varlistentry>
+             <term><filename>make adall</></term>
+             <listitem>
+               <para>This target produces an <filename>mitgcmuv_ad</> executable
+                 using the <filename>taf</> or <filename>staf</> adjoint
+                 compiler.  See the <filename>genmake2</> "-adof" option for
+                 compiler selection.</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>make ftlall</></term>
+             <listitem>
+               <para>Similar to <filename>make adall</> above, this
+                 produces...</para>
+             </listitem>
+           </varlistentry>
+         </variablelist>
+         <para>Please report any compilation failures or other build problems to
+           the <email>MITgcm-support@mitgcm.org</email> list.</para>
+       </sect3>
+     </sect2>
+     <sect2 id="verification">
+       <title>The Verification Suite</title>
+       <para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</>
+         directory) includes more than a dozen examples intended for regression
+         testing.  Each one of these example directories contains "known-good"
+         output files along with all the input (including both code and data
+         files) required for their re-calculation.  These example directories are
+         further broken down into sets of subdirectories
+         (eg. <filename>/input</>, <filename>/code</>) intended to expedite the
+         testing process.</para>
+       <sect3 id="testreport">
+         <title>The <filename>testreport</> Utility</title>
+         <para>Also included in <filename>$ROOTDIR/verification/</> are shell
+           scripts for automated testing.  The newest script (which was written
+           to work with <filename>genmake2</>) is called <filename>testreport</>.
+           This script can be used to build different versions of the MITgcm
+           code, run the various examples, compare the output, and (if specified)
+           email the results of each one of these tests to a central
+           repository.</para>
+         <para>On some systems, the testreport script can be run with a command
+         line as simple as:</para>
+ <screen>
+   $ cd verification
+   $ ./testreport -ieee
+ </screen>
+         <para>However, some systems (those lacking or wiht a broken "/bin/sh")
+           may require an explicit shell invocation such as:</para>
+ <screen>
+   $ sh ./testreport -ieee -t 'exp0 exp4'
+   $ /some/path/to/bash ./testreport -ieee -t 'ideal_2D_oce lab_sea natl_box'
+ </screen>
+         <para>The <filename>testreport</> script accepts a number of
+           command-line options which can be listed using the <filename>-help</>
+           option.  The most important ones are:</para>
+         <variablelist>
+           <varlistentry>
+             <term><filename>-ieee</></term>
+             <listitem>
+               <para>If allowed by the compiler (as defined in the "optfile"),
+                 use IEEE arithmetic.  This option, along with the GCC compiler,
+                 is how the standard results were produced.</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-tdir TESTDIR</></term>
+             <term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term>
+             <listitem>
+               <para>This option specifies the test directory or list of test
+                 directories that should be used.  Each of these entries should
+                 exactly (note: they are case sensitive!) match the names of
+                 directries in <filename>$ROOTDIR/verification/</>.  If this
+                 option is omitted, then all directories that are properly
+                 formatted (that is, containing an <filename>input</>
+                 sub-directory and a <filename>results/output.txt</> file) will
+                 be used.</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-optfile=/PATH/FILENAME</></term>
+             <term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>
+             <listitem>
+               <para>This specifies a list of "options files" that will be passed
+                 to <filename>genmake2</>.  If multiple options files are used
+                 (say, to test different compilers or different sets of options
+                 for the same compiler), then each options file will be used with
+                 each of the test directories.</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-addr EMAIL</></term>
+             <term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>
+             <listitem>
+               <para>Send the results (namely, <filename>output.txt</>,
+                 <filename>genmake_local</>, <filename>genmake_state</>, and
+                 <filename>Makefile</>) to the specified email addresses.  The
+                 results are gzipped, placed in a tar file, MIME encoded, and
+                 sent to the specified address.  If no email addresses are
+                 specified, no mail is sent.</para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-mpi</></term>
+             <listitem>
+               <para>If the necessary files
+               (<filename>TESTDIR/code/CPP_EEOPTIONS.h_mpi</> and
+               <filename>TESTDIR/code/SIZE.h_mpi</>) exist, then use them for an
+               MPI--enabled run.  Note that the use of MPI typically requires a
+               special command option (see "-command" below) to invoke the MPI
+               executable.  Examples of PBS scripts using MPI with testreport can be
+               found in the <ulink
+               url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">
+               MITgcm-contrib area</ulink></para>
+             </listitem>
+           </varlistentry>
+           <varlistentry>
+             <term><filename>-command='some command to run'</></term>
+             <listitem>
+               <para>For some tests, particularly MPI runs, the default "make
+               output.txt" is not sufficient.  This option allows a more general
+               command (or shell script) to be invoked.  Examples of PBS scripts
+               using MPI with testreport can be found in the <ulink
+               url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/">
+               MITgcm-contrib area</ulink></para>
+             </listitem>
+           </varlistentry>
+         </variablelist>
+         <para>The <filename>testreport</> script will write progress to the
+           screen (stdout) as it runs.  In addition, it will create a
+           <filename>tr_out.txt</> file that contains a brief comparison of the
+           current output with the "known-good" output.</para>
+       </sect3>
+     </sect2>
      <sect2 id="packages">
-       <title>Coding Packages</title>
+       <title>Creating MITgcm Packages</title>
-       <para>Optional parts of code have been separated from the MITgcmUV
+       <para>Optional parts of code have been separated from the MITgcmUV core
-       core driver code and organised into packages. The packaging
+         driver code and organised into packages.  The packaging structure
-       structure provides a mechanism for maintaining suites of code,
+         provides a mechanism for maintaining suites of code, specific to
-       specific to particular classes of problems, in a way that is
+         particular classes of problems, in a way that is cleanly separated from
-       cleanly separated from the generic fluid dynamical
+         the generic fluid dynamical engine.</para>
-       engine.</para>
+       <para>The MITgcmUV packaging structure is described below using generic
-       <para>The MITgcmUV packaging structure is described below using
+         package names ${pkg}.  A concrete examples of a package is the code for
-       generic package names ${pkg}.  A concrete examples of a package
+         implementing GM/Redi mixing. This code uses the package name</para>
-       is the code for implementing GM/Redi mixing. This code uses the
-       package name</para>
      </sect2>
-Line 472 
 o  Each package gets its runtime configu
+Line 886 
 o  Each package gets its runtime configu
     Package runtime config. options are imported
     into a common block held in a header file
     called "${PKG}.h".
+    Note: In some packages, the header file "${PKG}.h" is splitted
+    into "${PKG}_PARAMS.h" that contains the package parameters and
+    ${PKG}_VARS.h" for the field arrays.
  o  The core driver part of the model can check
     for runtime enabling or disabling of individual packages
-Line 494 
 CPP Flags
+Line 911 
 CPP Flags
 . Within the core driver code flags of the form
         ALLOW_${PKG} are used to include or exclude
         whole packages. The ALLOW_${PKG} flags are included
-        from a PKG_CPP_OPTIONS block which is currently
+        from a PACKAGES_CONFIG.h file that is automatically
+        generated by genmake2 (see genmake2 section).
         held in-line in the CPP_OPTIONS.h header file.
         e.g.
         Core model code .....
+        #include "PACKAGES_CONFIG.h"
         #include "CPP_OPTIONS.h"
           :
           :
-Line 511 
 CPP Flags
+Line 930 
 CPP Flags
 . Within an individual package a header file,
         "${PKG}_OPTIONS.h", is used to set CPP flags
-        specific to that package. It is not recommended
+        specific to that package. It also includes
-        to include this file in "CPP_OPTIONS.h".
+        "PACKAGES_CONFIG.h" and "CPP_OPTIONS.h".
  Package Boot Sequence
-Line 534 
 Package Boot Sequence
+Line 953 
 Package Boot Sequence
       &       CALL ${PKG}_READPARMS( retCode )
          #endif
-. S/R PACKAGES_CHECK()
+. S/R PACKAGES_INIT_FIXED()
+             :
+         #ifdef ALLOW_${PKG}
+           if ( use${Pkg} )
+      &       CALL ${PKG}_INIT_FIXED( retCode )
+         #endif
+. S/R PACKAGES_CHECK()
              :
          #ifdef ALLOW_${PKG}
            if ( use${Pkg} )
-Line 544 
 Package Boot Sequence
+Line 970 
 Package Boot Sequence
       &       CALL PACKAGES_CHECK_ERROR('${PKG}')
          #endif
-. S/R PACKAGES_INIT()
+. S/R PACKAGES_INIT_VARIABLES()
              :
          #ifdef ALLOW_${PKG}
            if ( use${Pkg} )
-      &       CALL ${PKG}_INIT( retCode )
+      &       CALL ${PKG}_INIT_VARIA( )
+         #endif
+ Package Output
+ ==============
+. S/R DO_THE_MODEL_IO
+         #ifdef ALLOW_${PKG}
+           if ( use${Pkg} )
+      &       CALL ${PKG}_OUTPUT( )
          #endif
+. S/R PACKAGES_WRITE_PICKUP()
+         #ifdef ALLOW_${PKG}
+           if ( use${Pkg} )
+      &       CALL ${PKG}_WRITE_PICKUP( )
+         #endif
  Description
  ===========
-Line 558 
 Description
+Line 999 
 Description
        - ${PKG}_READPARMS()
      is responsible for reading
      in the package parameters file data.${pkg}, and storing
-     the package parameters in "${PKG}.h".
+     the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h").
-     -> called in INITIALISE_FIXED
+     -> called from INITIALISE_FIXED in PACKAGES_READPARMS
+      - ${PKG}_INIT_FIXED()
+     is responsible for completing the internal setup of a package.
+     -> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED
+     note: 1) some pkg use instead:
+              CALL ${PKG}_INITIALISE  ( or the old form CALL ${PKG}_INIT )
+) for simple pkg setup, this part is done inside ${PKG}_READPARMS
       - ${PKG}_CHECK()
      is responsible for validating
-Line 568 
 Description
+Line 1016 
 Description
      need to check. This is done through header files "${PKG}.h".
      It is assumed that parameters owned by other packages
      will not be reset during ${PKG}_CHECK().
-     -> called in INITIALISE_FIXED
+     -> called from INITIALISE_FIXED in PACKAGES_CHECK
-      - ${PKG}_INIT()
+      - ${PKG}_INIT_VARIA()
-     is responsible for completing the
+     is responsible for fill-in all package variables with an initial value.
-     internal setup of a package. This routine is called after
+     Contains eventually a call to ${PKG}_READ_PICKUP that will read
-     the core model state has been completely initialised
+     from a pickup file the package variables required to restart the model.
-     but before the core model timestepping starts.
+     This routine is called after the core model state has been completely
-     -> called in INITIALISE_VARIA
+     initialised but before the core model timestepping starts.
+     -> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES
+     note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg
+      use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old
+      form ${PKG}_INIT
+      - ${PKG}_OUTPUT( )
+      is responsible for writing time-average fields to output files
+      (but the cumulating step is done within the package main S/R).
+      Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR)
+      and write snap-shot fields that are hold in common blocks. Other
+      temporary fields are directly dump to file where they are available.
+      NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages
+               but is beeing replaced by ${PKG}_OUTPUT
+               to avoid confusion with pkg/diagnostics functionality.
+) the output part is not yet in a standard form and might still
+               evolve a lot.
+     -> called within DO_THE_MODEL_IO
+      - ${PKG}_WRITE_PICKUP()
+      is responsible for writing a package pickup file when necessary for
+      a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT )
+     -> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP
  Summary
  =======
-Line 596 
 Summary
+Line 1066 
 Summary
    -----------------------
    * ${PKG}_OPTIONS.h     has further package-specific CPP options
    * ${PKG}.h             package-specific common block variables, fields
+    or  ${PKG}_PARAMS.h   package-specific common block parameters
+    and ${PKG}_VARS.h     package-specific common block fields
  - FORTRAN source files
    -----------------------
-   * ${pkg}_readparms.F   reads parameters from file data.${pkg}
+   * ${pkg}_readparms.F    reads parameters from file data.${pkg}
-   * ${pkg}_check.F       checks package dependencies and consistencies
+   * ${pkg}_init_fixed.F   complete the package setup
-   * ${pkg}_init.F        initialises package-related fields
+   * ${pkg}_check.F        checks package dependencies and consistencies
-   * ${pkg}_... .F        package source code
+   * ${pkg}_init_varia.F   initialises package-related fields
+   * ${pkg}_... .F         package source code
+   * ${pkg}_output.F       write output to file.
+   * ${pkg}_write_pickup.F write a package pickup file to restart the model
+   New: Subroutine in one package (pkgA) that only contains code which
+        is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F)
+        will be named: pkgA_pkgB_something.F
  - parameter file
    -----------------------

 Legend:



Removed from v.1.2
 


changed lines


 
Added in v.1.9
 Legend:



Removed from v.1.2
 


changed lines


 
Added in v.1.9
-Removed from v.1.2
+Added in v.1.9

	ViewVC Help
Powered by ViewVC 1.1.22