/[MITgcm]/MITgcm/doc/devel_HOWTO.sgml
ViewVC logotype

Diff of /MITgcm/doc/devel_HOWTO.sgml

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.1 by edhill, Mon Aug 4 21:09:05 2003 UTC revision 1.3 by edhill, Thu Aug 28 22:44:00 2003 UTC
# Line 4  Line 4 
4    
5  <!--  <!--
6  Build commands:  Build commands:
7      db2ps -d ldp.dsl devel_HOWTO.sgml
8    db2pdf -d ldp.dsl devel_HOWTO.sgml    db2pdf -d ldp.dsl devel_HOWTO.sgml
9    db2html -d ./ldp.dsl devel_HOWTO.sgml    db2html -d ./ldp.dsl devel_HOWTO.sgml
10      db2html -u -d ./ldp.dsl devel_HOWTO.sgml
11  -->  -->
12    
13    <articleinfo>    <articleinfo>
# Line 22  Build commands: Line 24  Build commands:
24      <revhistory>      <revhistory>
25        <revision>        <revision>
26          <revnumber>0.01</revnumber>          <revnumber>0.01</revnumber>
27          <date>2003-0-07</date>          <date>2003-08-07</date>
28          <authorinitials>eh3</authorinitials>          <authorinitials>eh3</authorinitials>
29          <revremark>          <revremark>
30            Initial version.            Initial version.
# Line 395  $ mv scratch/dev_docs /u/u0/httpd/html Line 397  $ mv scratch/dev_docs /u/u0/httpd/html
397    
398    </sect1>    </sect1>
399    
400      <sect1 id="coding">
401        <title>Coding for MITgcm</title>
402    
403        <sect2 id="build_tools">
404          <title>Build Tools</title>
405    
406          <para>Many Open Source projects use the "GNU Autotools" to help
407          streamline the build process for various Unix and Unix-like
408          architectures.  For a user, the result is the common "configure"
409          (that is, "<filename>./configure && make && make
410          install</filename>") commands.  For MITgcm, the process is
411          similar.  Typical commands are:</para>
412    
413    <screen>
414    $ genmake -mods=../code
415    $ make depend
416    $ make
417    </screen>
418    
419          <para>The following sections describe the individual steps in
420          the build process.</para>
421    
422          <sect3 id="genmake">
423            <title>The <filename>genmake2</> Utility</title>
424    
425            <para><emphasis>Please note that the older
426            <filename>genmake</> is deprecated and will eventually
427            be replaced by <filename>genmake2</>.  This HOWTO only
428            describes the newer tool.</emphasis></para>
429    
430            <para>The first step in any MITgcm build is to create a
431            Unix-style <filename>Makefile</filename> which will be parsed
432            by <filename>make</filename> to specify how to compile the
433            MITgcm source files.  For more detailed descriptions of what
434            the make tools are and how they are used, please see:</para>
435    
436            <itemizedlist>
437              <listitem>
438                <para><ulink url="http://www.gnu.org/software/make/make.html">http://www.gnu.org/software/make/make.html</></para>
439              </listitem>
440              <listitem>
441                <para><ulink url="http://www.oreilly.com/catalog/make2/">http://www.oreilly.com/catalog/make2/</></para>
442              </listitem>
443            </itemizedlist>
444    
445            <para>Due to the poor handling of soft-links and other bugs
446            common with the <filename>make</filename> versions provided by
447            commercial Unix vendors, GNU <filename>make</filename>
448            (sometimes called <filename>gmake</filename>) should be
449            preferred.</para>
450    
451            <para>As the name implies, <filename>genmake2</filename>
452            generates a <filename>Makefile</filename>.  It does so by
453            first parsing the information supplied from the following
454            sources</para>
455    
456            <orderedlist>
457              <listitem>
458                <para>a <filename>gm_local</filename> file in the current
459                directory</para>
460              </listitem>
461              <listitem>
462                <para>directly from command-line options</para>
463              </listitem>
464              <listitem>
465                <para>an "options file" as specified by the command-line
466                option <filename>-optfile='FILENAME'</filename></para>
467              </listitem>
468            </orderedlist>
469    
470            <para>then checking certain dependency rules (the package
471            dependencies), and then writing a
472            <filename>Makefile</filename> based upon the source code that
473            it finds.  For convenience with the various Unix shells,
474            <filename>genmake2</> supports both "long"- and "shor"-style
475            options.  A complete list of the available options can be
476            obtained from:</para>
477    
478    <screen>
479    $ genmake2 -help
480    </screen>
481    
482            <para>The most important options for <filename>genmake2</>
483            are:</para>
484    
485            <variablelist>
486    
487              <varlistentry>
488                <term><filename>--optfile=/PATH/FILENAME</></term>
489                <listitem>
490                  <para>This specifies the "options file" that should be
491                  used for a particular build.  The options file is a
492                  convenient and machine-indepenent way of specifying
493                  parameters such as the FORTRAN compiler
494                  (<filename>FC=</>), FORTRAN compiler optimization flags
495                  (<filename>FFLAGS=</>), and the locations of various
496                  platform- and/or machine-specific tools
497                  (eg. <filename>MAKEDEPEND=</>).  As with
498                  <filename>genmake2</>, all options files should be
499                  written a BASH v1-compatible syntax.  Examples of
500                  various options files can be found in
501                  <filename>$ROOTDIR/tools/build_options</>.  Everyone is
502                  encouraged to submit their options files to the MITgcm
503                  project for inclusion (please send to
504                  <email>MITgcm-support@mitgcm.org</email>).  We are
505                  particularly grateful for options files tested on new or
506                  unique platforms!</para>
507                </listitem>
508              </varlistentry>
509    
510              <varlistentry>
511                <term><filename>-pdepend=/PATH/FILENAME</></term>
512                <listitem>
513                  <para>This specifies the dependency file used for
514                  packages.  If not specified, the default dependency file
515                  is <filename>$ROOTDIR/pkg/pkg_depend</>.  The syntax for
516                  this file is parsed on a line-by-line basis where each
517                  line containes either a comment ("#") or a simple
518                  "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or
519                  "-" symbol specifies a "must be used with" or a "must
520                  not be used with" relationship, respectively.  If no
521                  rule is specified, then it is assumed that the two
522                  packages are compatible and will function either with or
523                  without each other.</para>
524                </listitem>
525              </varlistentry>
526    
527              <varlistentry>
528                <term><filename>-pdefault=PKG</></term>
529                <term><filename>-pdefault='PKG1 [PKG2 PKG3 ...]'</></term>
530                <listitem>
531                  <para>This option specifies the default set of packages
532                  to be used.  If not set, the default package list will
533                  be read from
534                  <filename>$ROOTDIR/pkg/pkg_default</>.</para>
535                </listitem>
536              </varlistentry>
537    
538              <varlistentry>
539                <term><filename></>-mods=DIR</term>
540                <term><filename></>-mods='DIR1 [DIR2 ...]'</term>
541                <listitem>
542                  <para>This option specifies a list of directories
543                  containing "modifications".  These are files that may
544                  (or may not) exist in the main MITgcm source tree but
545                  will be overridden by any identically-named sources
546                  within the "MODS" directories.</para>
547                </listitem>
548              </varlistentry>
549    
550            </variablelist>
551            
552            <para>A successful run of <filename>genmake2</> will produce
553            both a <filename>Makefile</> and a locally modified copy of
554            the specified <filename>CPP_OPTIONS.h</> file.  The local copy
555            of <filename>CPP_OPTIONS.h</> will contain a list of
556            <filename>genmake2</>-created #DEFINE and #UNDEF statements
557            that reflect the list of packages that will be compiled into
558            the code (either directly through enable/disable/defaults
559            options or indirectly through dependencies).</para>
560    
561            <para>In general, it is best to use <filename>genmake2</> on a
562            "clean" directory that is free of all source
563            (*.[F,f],*.[F,f]90) and header (*.h,*.inc) files.  Generally,
564            this can be accomplished in an "un-clean" directory by running
565            "make CLEAN" followed by "make makefile".</para>
566    
567          </sect3>
568    
569          <sect3 id="makefile_use">
570            <title>Using <filename>Makefile</></title>
571    
572            <para>Once a <filename>Makefile</> has been created, one can
573            build the executable using:</para>
574    
575    <screen>
576    $ make CLEAN
577    $ make depend
578    $ make
579    </screen>
580    
581            <para>The "make CLEAN" step will remove any local source
582            files, include files, and links.  It is strongly recommended
583            for "un-clean" directories which may contain the (partial?)
584            results of previous builds.  Such "debris" can interfere with
585            the next stage of the build.</para>
586    
587            <para>The "make depend" step will create a large number of
588            symbolic links from the local directory to the source file
589            locations.  It also parses these files and creates an
590            extensive list of dependencies within the
591            <filename>Makefile</> itself.  The links that exist at this
592            stage are mostly "large F" files (*.F and *.F90) that need to
593            be processed by a C preprocessor ("CPP").
594            </para>
595    
596            <para>The final "make" invokes the C preprocessor to produce
597            the "little f" files (*.f and *.f90) and then compiles them to
598            object code using the specified FORTRAN compiler and options.
599            An intermediate script is often used during this stage to
600            further process (usually, make simple substitutions) custom
601            definitions such as variable types within the source files.
602            This additional stage is necessary in order to overcome some
603            of the inconsistencies in the sizes of objects (bytes) between
604            different compilers.</para>
605    
606            <para>Please report compilation failures or other problems to
607            <email>MITgcm-support@mitgcm.org</email>.</para>
608    
609          </sect3>
610    
611        </sect2>
612    
613        <sect2 id="verification">
614          <title>The Verification Suite</title>
615    
616          <para>The MITgcm CVS tree (within the
617          <filename>$ROOTDIR/verification/</> directory) includes more
618          than a dozen examples intended for regression testing.  Each one
619          of these example directories contains "known-good" output files
620          along with all the input (including both code and data files)
621          required for their re-calculation.  These example directories
622          are further broken down into sets of subdirectories
623          (eg. <filename>/input</>, <filename>/code</>) intended to
624          expedite the testing process.</para>
625    
626          <sect3 id="testreport">
627            <title>The <filename>testreport</> Utility</title>
628    
629            <para>Also included in <filename>$ROOTDIR/verification/</> are
630            shell scripts for automated testing.  The newest script (which
631            was written to work with <filename>genmake2</>) is called
632            <filename>testreport</>.  Ths script can be used to build the
633            different versions of the MITgcm code, run the various
634            examples, compare the output, and (if specified) email the
635            results of each one of these tests to a central
636            repository.</para>
637    
638            <para>The <filename>testreport</> script accepts a number of
639            command-line options which can be listed using the
640            <filename>-help</> option.  The most important ones are:</para>
641    
642            <variablelist>
643    
644              <varlistentry>
645                <term><filename>-tdir TESTDIR</></term>
646                <term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term>
647                <listitem>
648                  <para>This option specifies the test directory or list
649                  of test directories that should be used.  Each of these
650                  entries should exactly (note: they're case sensitive!)
651                  match the names of directries in
652                  <filename>$ROOTDIR/verification/</>.  If this option is
653                  omitted, then all directories that are properly
654                  formatted (that is, containing an <filename>input</>
655                  sub-directory and example output) will be used.</para>
656                </listitem>
657              </varlistentry>
658    
659              <varlistentry>
660                <term><filename>-optfile=/PATH/FILENAME</></term>
661                <term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>
662                <listitem>
663                  <para>This specifies a list of "options files" that will
664                  be passed to <filename>genmake2</>.  If multiple options
665                  files are used (say, to test different compilers or
666                  different sets of options for the same compiler), then
667                  each options file will be used with each of the test
668                  directories.</para>
669                </listitem>
670              </varlistentry>
671    
672              <varlistentry>
673                <term><filename>-addr EMAIL</></term>
674                <term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>
675                <listitem>
676                  <para>Send the results (namely, <filename>output.txt</>,
677                  <filename>gm_local</>, <filename>gm_state</>, and
678                  <filename>Makefile</>) to the specified email addresses.
679                  The results are gzipped, placed in a tar file, MIME
680                  encoded, and sent to an @mitgcm.org address.  If no
681                  email addresses are specified, no mail is sent.</para>
682                </listitem>
683              </varlistentry>
684    
685            </variablelist>
686    
687            <para>The <filename>testreport</> script will write progress
688            to the screen (stdout) as it runs.  In addition, it will
689            create a <filename>summary.txt</> file that contains a brief
690            comparison of the current output with the "known-good"
691            output.</para>
692    
693          </sect3>
694    
695        </sect2>
696    
697        <sect2 id="packages">
698          <title>Creating MITgcm Packages</title>
699    
700          <para>Optional parts of code have been separated from the MITgcmUV
701          core driver code and organised into packages. The packaging
702          structure provides a mechanism for maintaining suites of code,
703          specific to particular classes of problems, in a way that is
704          cleanly separated from the generic fluid dynamical
705          engine.</para>
706    
707          <para>The MITgcmUV packaging structure is described below using
708          generic package names ${pkg}.  A concrete examples of a package
709          is the code for implementing GM/Redi mixing. This code uses the
710          package name</para>
711    
712        </sect2>
713    
714      </sect1>
715    
716      <sect1>
717        <title>Chris's Notes...</title>
718    
719    <programlisting>
720               MITgcmUV Packages
721               =================
722    
723      Optional parts of code are separated from
724    the MITgcmUV core driver code and organised into
725    packages. The packaging structure provides a mechanism for
726    maintaining suites of code, specific to particular
727    classes of problem, in a way that is cleanly
728    separated from the generic fluid dynamical engine.
729    
730     The MITgcmUV packaging structure is describe
731    below using generic package names ${pkg}.
732    A concrete examples of a package is the code
733    for implementing GM/Redi mixing. This code uses
734    the package name
735    *   ${PKG} = GMREDI
736    *   ${pkg} = gmredi
737    *   ${Pkg} = gmRedi
738    
739    Package states
740    ==============
741    
742     Packages can be any one of four states, included,
743     excluded, enabled, disabled as follows:
744    
745     included(excluded) compile time state which
746                        includes(excludes) package
747                        code and routine calls from
748                        compilation/linking etc...
749    
750     enabled(disabled)  run-time state which
751                        enables(disables) package code
752                        execution.
753    
754     Every call to a ${pkg}_... routine from outside the package
755     should be placed within both a
756     #ifdef ALLOW_${PKG} ... block and a
757     if ( use${Pkg} ) ... then block.
758     Package states are generally not expected to change during
759     a model run.
760    
761    Package structure
762    =================
763    
764    o  Each package gets its runtime configuration
765       parameters from a file named "data.${pkg}"
766       Package runtime config. options are imported
767       into a common block held in a header file
768       called "${PKG}.h".
769    
770    o  The core driver part of the model can check
771       for runtime enabling or disabling of individual packages
772       through logical flags use${Pkg}.
773       The information is loaded from a
774       global package setup file called "data.pkg".
775       The use${Pkg} flags are not used within
776       individual packages.
777    
778    o  Included in "${PKG}.h" is a logical flag
779       called ${Pkg}IsOn. The "${PKG}.h" header file can be imported
780       by other packages to check dependencies and requirements
781       from other packages ( see "Package Boot Sequence" section).
782       NOTE: This procedure is not presently implemented,
783       ----- neither for kpp nor for gmRedi.
784    
785    CPP Flags
786    =========
787    
788        1. Within the core driver code flags of the form
789           ALLOW_${PKG} are used to include or exclude
790           whole packages. The ALLOW_${PKG} flags are included
791           from a PKG_CPP_OPTIONS block which is currently
792           held in-line in the CPP_OPTIONS.h header file.
793           e.g.
794    
795           Core model code .....
796    
797           #include "CPP_OPTIONS.h"
798             :
799             :
800             :
801    
802           #ifdef ALLOW_${PKG}
803             if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...)
804           #endif
805    
806        2. Within an individual package a header file,
807           "${PKG}_OPTIONS.h", is used to set CPP flags
808           specific to that package. It is not recommended
809           to include this file in "CPP_OPTIONS.h".
810    
811    
812    Package Boot Sequence
813    =====================
814    
815        Calls to package routines within the core code timestepping
816        loop can vary. However, all packages follow a required
817        "boot" sequence outlined here:
818    
819        1. S/R PACKAGES_BOOT()
820                :
821            CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
822    
823    
824        2. S/R PACKAGES_READPARMS()
825                :
826            #ifdef ALLOW_${PKG}
827              if ( use${Pkg} )
828         &       CALL ${PKG}_READPARMS( retCode )
829            #endif
830    
831        2. S/R PACKAGES_CHECK()
832                :
833            #ifdef ALLOW_${PKG}
834              if ( use${Pkg} )
835         &       CALL ${PKG}_CHECK( retCode )
836            #else
837              if ( use${Pkg} )
838         &       CALL PACKAGES_CHECK_ERROR('${PKG}')
839            #endif
840    
841        3. S/R PACKAGES_INIT()
842                :
843            #ifdef ALLOW_${PKG}
844              if ( use${Pkg} )
845         &       CALL ${PKG}_INIT( retCode )
846            #endif
847    
848    
849    Description
850    ===========
851    
852          - ${PKG}_READPARMS()
853        is responsible for reading
854        in the package parameters file data.${pkg}, and storing
855        the package parameters in "${PKG}.h".
856        -> called in INITIALISE_FIXED
857    
858         - ${PKG}_CHECK()
859        is responsible for validating
860        basic package setup and inter-package dependencies.
861        ${PKG}_CHECK can import other package parameters it may
862        need to check. This is done through header files "${PKG}.h".
863        It is assumed that parameters owned by other packages
864        will not be reset during ${PKG}_CHECK().
865        -> called in INITIALISE_FIXED
866    
867         - ${PKG}_INIT()
868        is responsible for completing the
869        internal setup of a package. This routine is called after
870        the core model state has been completely initialised
871        but before the core model timestepping starts.
872        -> called in INITIALISE_VARIA
873    
874    Summary
875    =======
876    
877    - CPP options:
878      -----------------------
879      * ALLOW_${PKG}         include/exclude package for compilation
880    
881    - FORTRAN logical:
882      -----------------------
883      * use${Pkg}            enable package for execution at runtime
884                             -> declared in PARAMS.h
885      * ${Pkg}IsOn           for package cross-dependency check
886                             -> declared in ${PKG}.h
887                             N.B.: Not presently used!
888    
889    - header files
890      -----------------------
891      * ${PKG}_OPTIONS.h     has further package-specific CPP options
892      * ${PKG}.h             package-specific common block variables, fields
893    
894    - FORTRAN source files
895      -----------------------
896      * ${pkg}_readparms.F   reads parameters from file data.${pkg}
897      * ${pkg}_check.F       checks package dependencies and consistencies
898      * ${pkg}_init.F        initialises package-related fields
899      * ${pkg}_... .F        package source code
900    
901    - parameter file
902      -----------------------
903      * data.${pkg}          parameter file
904    </programlisting>
905    
906      </sect1>
907    
908    
909  </article>  </article>

Legend:
Removed from v.1.1  
changed lines
  Added in v.1.3

  ViewVC Help
Powered by ViewVC 1.1.22