--- MITgcm/doc/devel_HOWTO.sgml 2003/08/28 22:44:00 1.3
+++ MITgcm/doc/devel_HOWTO.sgml 2003/11/20 00:01:42 1.4
@@ -128,18 +128,18 @@
doc tags -- connect to real docs?
eesupp cnh?
exe ecco user build
- *- jobs runtime shell scripts for
+ ,- jobs runtime shell scripts for
| various platforms
| lsopt line search
m| model main dynamics (core)
e| optimization_drivers ?
r| optim line search interface
g| pkg alternate and optional numerics, etc.
- e*- tools
+ e|- tools
?| tutorial_examples documented tests
| only populated on release1 branch
| and not validated during "testscript"
- *- utils
+ '- utils
verification std tests
@@ -316,7 +316,7 @@
- Editing
+ Editing the Documentation
The documentation is contained in the
manual directory in a raw LaTeX format.
@@ -364,9 +364,11 @@
- Building Given the directory structure of
- , the entire documentation
- for the web site can be built using:
+ Building the Documentation
+
+ Given the directory structure of , the entire documentation for the web
+ site can be built using:
$ cd mitgcm.org/devel/buildweb
@@ -403,12 +405,11 @@
Build Tools
- Many Open Source projects use the "GNU Autotools" to help
- streamline the build process for various Unix and Unix-like
- architectures. For a user, the result is the common "configure"
- (that is, "./configure && make && make
- install") commands. For MITgcm, the process is
- similar. Typical commands are:
+ Many Open Source projects use the "GNU Autotools" to help streamline
+ the build process for various Unix and Unix-like architectures. For a
+ user, the result is the common "configure" (that is,
+ "./configure && make && make install") commands.
+ For MITgcm, the process is similar. Typical commands are:
$ genmake -mods=../code
@@ -416,112 +417,137 @@
$ make
- The following sections describe the individual steps in
- the build process.
-
+ The following sections describe the individual steps in the build
+ process.
+
The genmake2> Utility
- Please note that the older
- genmake> is deprecated and will eventually
- be replaced by genmake2>. This HOWTO only
- describes the newer tool.
-
- The first step in any MITgcm build is to create a
- Unix-style Makefile which will be parsed
- by make to specify how to compile the
- MITgcm source files. For more detailed descriptions of what
- the make tools are and how they are used, please see:
+ Please note that the older genmake> is
+ deprecated and will eventually be replaced by genmake2>.
+ This HOWTO only describes the newer tool.
+
+ The first step in any MITgcm build is to create a Unix-style
+ Makefile which will be parsed by
+ make to specify how to compile the MITgcm source
+ files. For more detailed descriptions of what the make tools are and
+ how they are used, please see:
- http://www.gnu.org/software/make/make.html>
+
+ http://www.gnu.org/software/make/make.html>
- http://www.oreilly.com/catalog/make2/>
+
+ http://www.oreilly.com/catalog/make2/>
- Due to the poor handling of soft-links and other bugs
- common with the make versions provided by
- commercial Unix vendors, GNU make
- (sometimes called gmake) should be
- preferred.
-
- As the name implies, genmake2
- generates a Makefile. It does so by
- first parsing the information supplied from the following
- sources
+ Genmake can often be invoked successfully with a command line as
+ simple as:
+
+
+$ genmake2 -mods=../code
+
+
+ However, some systems (particularly commercial Unixes that lack a
+ more modern "/bin/sh" implementation or that have shells installed in
+ odd locations) may require an explicit shell invocation such as one of
+ the following:
+
+
+$ /usr/bin/sh genmake2 -make=gmake -mods=../code
+$ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code
+
+
+ The genmake2 code has been written in a Bourne and BASH (v1)
+ compatible syntax so it should work with most "sh" and all recent "bash"
+ implementations.
+
+ As the name implies, genmake2 generates a
+ Makefile. It does so by first parsing the
+ information supplied from the following sources
a gm_local file in the current
- directory
+ directory
directly from command-line options
- an "options file" as specified by the command-line
- option -optfile='FILENAME'
+ an "options file" as specified by the command-line option
+ -optfile='FILENAME'
- then checking certain dependency rules (the package
- dependencies), and then writing a
- Makefile based upon the source code that
- it finds. For convenience with the various Unix shells,
- genmake2> supports both "long"- and "shor"-style
- options. A complete list of the available options can be
- obtained from:
+ then checking certain dependency rules (the package dependencies),
+ and finally writing a Makefile based upon the
+ source code that it finds. For convenience within various Unix
+ shells, genmake2> supports both "long"- and "short"-style
+ options. A complete list of the available options can be obtained
+ from:
$ genmake2 -help
- The most important options for genmake2>
- are:
+ The most important options for genmake2> are:
--optfile=/PATH/FILENAME>
+
- This specifies the "options file" that should be
- used for a particular build. The options file is a
- convenient and machine-indepenent way of specifying
- parameters such as the FORTRAN compiler
- (FC=>), FORTRAN compiler optimization flags
- (FFLAGS=>), and the locations of various
- platform- and/or machine-specific tools
- (eg. MAKEDEPEND=>). As with
- genmake2>, all options files should be
- written a BASH v1-compatible syntax. Examples of
- various options files can be found in
- $ROOTDIR/tools/build_options>. Everyone is
- encouraged to submit their options files to the MITgcm
- project for inclusion (please send to
- MITgcm-support@mitgcm.org). We are
- particularly grateful for options files tested on new or
- unique platforms!
+ This specifies the "options file" that should be used for a
+ particular build. The options file is a convenient and
+ machine-indepenent way of specifying parameters such as the
+ FORTRAN compiler (FC=>), FORTRAN compiler
+ optimization flags (FFLAGS=>), and the locations of
+ various platform- and/or machine-specific tools
+ (eg. MAKEDEPEND=>). As with genmake2>,
+ all options files should be written to be compatible with
+ Bourne--shell ("sh" or "BASH v1") syntax. Examples of various
+ options files can be found in
+ $ROOTDIR/tools/build_options>.
+
+ If no "optfile" is specified (either through the command lin
+ or the environment variable), genmake2 will try to make a
+ reasonable guess from the list provided in
+ $ROOTDIR/tools/build_options>. The method used for
+ making this guess is to first determine the combination of
+ operating system and hardware (eg. "linux_ia32") and then find a
+ working Fortran compiler within the user's path. When these
+ three items have been identified, genmake2 will try to find an
+ optfile that has a matching name.
+
+ Everyone is encouraged to submit their options files to the
+ MITgcm project for inclusion (please send to
+ MITgcm-support@mitgcm.org). We are particularly
+ grateful for options files tested on new or unique
+ platforms!
+
-pdepend=/PATH/FILENAME>
+
- This specifies the dependency file used for
- packages. If not specified, the default dependency file
- is $ROOTDIR/pkg/pkg_depend>. The syntax for
- this file is parsed on a line-by-line basis where each
- line containes either a comment ("#") or a simple
- "PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or
- "-" symbol specifies a "must be used with" or a "must
- not be used with" relationship, respectively. If no
- rule is specified, then it is assumed that the two
- packages are compatible and will function either with or
- without each other.
+ This specifies the dependency file used for packages. If
+ not specified, the default dependency file is
+ $ROOTDIR/pkg/pkg_depend>. The syntax for this file is
+ parsed on a line-by-line basis where each line containes either a
+ comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule
+ where the "+" or "-" symbol specifies a "must be used with" or a
+ "must not be used with" relationship, respectively. If no rule is
+ specified, then it is assumed that the two packages are compatible
+ and will function either with or without each other.
+
@@ -536,33 +562,55 @@
- >-mods=DIR
- >-mods='DIR1 [DIR2 ...]'
+ -mods=DIR>
+ -mods='DIR1 [DIR2 ...]'>
- This option specifies a list of directories
- containing "modifications". These are files that may
- (or may not) exist in the main MITgcm source tree but
- will be overridden by any identically-named sources
- within the "MODS" directories.
+ This option specifies a list of directories containing
+ "modifications". These directories contain files with names
+ that may (or may not) exist in the main MITgcm source tree but
+ will be overridden by any identically-named sources within the
+ "MODS" directories. The order of precedence for this
+ "name-hiding" is as follows:
+
+
+ "MODS" directories (in the order given)
+
+ Packages either explicitly specified or
+ provided by default (in the order given)
+ Packages included due to package dependencies
+ (in the order that that package dependencies are
+ parsed)
+ The "standard dirs" (which may have been
+ specified by the "-standarddirs" option)
+
+
+
+
+
+
+ -make=/path/to/gmake>
+
+ Due to the poor handling of soft-links and other bugs common
+ with the make> versions provided by commercial Unix
+ vendors, GNU make (sometimes called
+ gmake) should be preferred. This option
+ provides a means for specifying the make program to be
+ used.
- A successful run of genmake2> will produce
- both a Makefile> and a locally modified copy of
- the specified CPP_OPTIONS.h> file. The local copy
- of CPP_OPTIONS.h> will contain a list of
- genmake2>-created #DEFINE and #UNDEF statements
- that reflect the list of packages that will be compiled into
- the code (either directly through enable/disable/defaults
- options or indirectly through dependencies).
-
- In general, it is best to use genmake2> on a
- "clean" directory that is free of all source
- (*.[F,f],*.[F,f]90) and header (*.h,*.inc) files. Generally,
- this can be accomplished in an "un-clean" directory by running
- "make CLEAN" followed by "make makefile".
+ A successful run of genmake2> will produce a
+ Makefile>, a PACKAGES_CONFIG.h> file, and
+ various convenience files used for the automatic differentiation
+ process.
+
+ In general, it is best to use genmake2> on a "clean"
+ directory that is free of all source (*.[F,f],*.[F,f]90) and header
+ (*.h,*.inc) files. Generally, this can be accomplished in an
+ "un-clean" directory by running "make CLEAN" followed by "make
+ makefile".
@@ -570,7 +618,7 @@
Using Makefile>
Once a Makefile> has been created, one can
- build the executable using:
+ build an executable using:
$ make CLEAN
@@ -578,33 +626,31 @@
$ make
- The "make CLEAN" step will remove any local source
- files, include files, and links. It is strongly recommended
- for "un-clean" directories which may contain the (partial?)
- results of previous builds. Such "debris" can interfere with
- the next stage of the build.
-
- The "make depend" step will create a large number of
- symbolic links from the local directory to the source file
- locations. It also parses these files and creates an
- extensive list of dependencies within the
- Makefile> itself. The links that exist at this
- stage are mostly "large F" files (*.F and *.F90) that need to
- be processed by a C preprocessor ("CPP").
-
-
- The final "make" invokes the C preprocessor to produce
- the "little f" files (*.f and *.f90) and then compiles them to
- object code using the specified FORTRAN compiler and options.
- An intermediate script is often used during this stage to
- further process (usually, make simple substitutions) custom
- definitions such as variable types within the source files.
- This additional stage is necessary in order to overcome some
- of the inconsistencies in the sizes of objects (bytes) between
- different compilers.
+ The "make CLEAN" step will remove any stale source files, include
+ files, and links. It is strongly recommended for "un-clean"
+ directories which may contain the (perhaps partial) results of
+ previous builds. Such "debris" can interfere with the next stage of
+ the build.
+
+ The "make depend" step will create a large number of symbolic
+ links from the local directory to the source file locations. It also
+ parses these files and creates an extensive list of dependencies
+ within the Makefile> itself. The links that exist at this
+ stage are mostly "large F" files (*.F and *.F90) that need to be
+ processed by a C preprocessor ("CPP"). Since "make depend" edits the
+ Makefile>, it is important not to skip this step!
+
+ The final "make" invokes the C preprocessor to produce the "little
+ f" files (*.f and *.f90) and then compiles them to object code using
+ the specified FORTRAN compiler and options. An intermediate script is
+ often used during this stage to further process (usually, make simple
+ substitutions) custom definitions such as variable types within the
+ source files. This additional stage is necessary in order to overcome
+ some of the inconsistencies in the sizes of objects (bytes) between
+ different compilers.
Please report compilation failures or other problems to
- MITgcm-support@mitgcm.org.
+ MITgcm-support@mitgcm.org.
@@ -613,46 +659,69 @@
The Verification Suite
- The MITgcm CVS tree (within the
- $ROOTDIR/verification/> directory) includes more
- than a dozen examples intended for regression testing. Each one
- of these example directories contains "known-good" output files
- along with all the input (including both code and data files)
- required for their re-calculation. These example directories
- are further broken down into sets of subdirectories
- (eg. /input>, /code>) intended to
- expedite the testing process.
+ The MITgcm CVS tree (within the $ROOTDIR/verification/>
+ directory) includes more than a dozen examples intended for regression
+ testing. Each one of these example directories contains "known-good"
+ output files along with all the input (including both code and data
+ files) required for their re-calculation. These example directories are
+ further broken down into sets of subdirectories
+ (eg. /input>, /code>) intended to expedite the
+ testing process.
The testreport> Utility
- Also included in $ROOTDIR/verification/> are
- shell scripts for automated testing. The newest script (which
- was written to work with genmake2>) is called
- testreport>. Ths script can be used to build the
- different versions of the MITgcm code, run the various
- examples, compare the output, and (if specified) email the
- results of each one of these tests to a central
- repository.
+ Also included in $ROOTDIR/verification/> are shell
+ scripts for automated testing. The newest script (which was written
+ to work with genmake2>) is called testreport>.
+ This script can be used to build different versions of the MITgcm
+ code, run the various examples, compare the output, and (if specified)
+ email the results of each one of these tests to a central
+ repository.
+
+ On some systems, the testreport script can be run with a command
+ line as simple as:
+
+
+$ cd verification
+$ ./testreport -ieee
+
+
+ However, some systems (those lacking or wiht a broken "/bin/sh")
+ may require an explicit shell invocation such as:
+
+
+$ sh ./testreport -ieee -t 'exp0 exp4'
+$ /some/path/to/bash ./testreport -ieee -t 'ideal_2D_oce lab_sea natl_box'
+
The testreport> script accepts a number of
- command-line options which can be listed using the
- -help> option. The most important ones are:
+ command-line options which can be listed using the -help>
+ option. The most important ones are:
+ -ieee>
+
+ If allowed by the compiler (as defined in the "optfile"),
+ use IEEE arithmetic. This option, along with the GCC compiler,
+ is how the standard results were produced.
+
+
+
+
-tdir TESTDIR>
-tdir 'TDIR1 TDIR2 [...]'>
- This option specifies the test directory or list
- of test directories that should be used. Each of these
- entries should exactly (note: they're case sensitive!)
- match the names of directries in
- $ROOTDIR/verification/>. If this option is
- omitted, then all directories that are properly
- formatted (that is, containing an input>
- sub-directory and example output) will be used.
+ This option specifies the test directory or list of test
+ directories that should be used. Each of these entries should
+ exactly (note: they are case sensitive!) match the names of
+ directries in $ROOTDIR/verification/>. If this
+ option is omitted, then all directories that are properly
+ formatted (that is, containing an input>
+ sub-directory and a results/output.txt> file) will
+ be used.
@@ -660,12 +729,11 @@
-optfile=/PATH/FILENAME>
-optfile '/PATH/F1 [/PATH/F2 ...]'>
- This specifies a list of "options files" that will
- be passed to genmake2>. If multiple options
- files are used (say, to test different compilers or
- different sets of options for the same compiler), then
- each options file will be used with each of the test
- directories.
+ This specifies a list of "options files" that will be passed
+ to genmake2>. If multiple options files are used
+ (say, to test different compilers or different sets of options
+ for the same compiler), then each options file will be used with
+ each of the test directories.
@@ -674,40 +742,65 @@
-addr 'EMAIL1 EMAIL2 [...]'>
Send the results (namely, output.txt>,
- gm_local>, gm_state>, and
- Makefile>) to the specified email addresses.
- The results are gzipped, placed in a tar file, MIME
- encoded, and sent to an @mitgcm.org address. If no
- email addresses are specified, no mail is sent.
+ genmake_local>, genmake_state>, and
+ Makefile>) to the specified email addresses. The
+ results are gzipped, placed in a tar file, MIME encoded, and
+ sent to the specified address. If no email addresses are
+ specified, no mail is sent.
+
+
+
+
+ -mpi>
+
+ If the necessary files
+ (TESTDIR/code/CPP_EEOPTIONS.h_mpi> and
+ TESTDIR/code/SIZE.h_mpi>) exist, then use them for an
+ MPI--enabled run. Note that the use of MPI typically requires a
+ special command option (see "-command" below) to invoke the MPI
+ executable. Examples of PBS scripts using MPI with testreport can be
+ found in the
+ MITgcm-contrib area
+
+
+
+
+ -command='some command to run'>
+
+ For some tests, particularly MPI runs, the default "make
+ output.txt" is not sufficient. This option allows a more general
+ command (or shell script) to be invoked. Examples of PBS scripts
+ using MPI with testreport can be found in the
+ MITgcm-contrib area
- The testreport> script will write progress
- to the screen (stdout) as it runs. In addition, it will
- create a summary.txt> file that contains a brief
- comparison of the current output with the "known-good"
- output.
+ The testreport> script will write progress to the
+ screen (stdout) as it runs. In addition, it will create a
+ tr_out.txt> file that contains a brief comparison of the
+ current output with the "known-good" output.
+
Creating MITgcm Packages
- Optional parts of code have been separated from the MITgcmUV
- core driver code and organised into packages. The packaging
- structure provides a mechanism for maintaining suites of code,
- specific to particular classes of problems, in a way that is
- cleanly separated from the generic fluid dynamical
- engine.
-
- The MITgcmUV packaging structure is described below using
- generic package names ${pkg}. A concrete examples of a package
- is the code for implementing GM/Redi mixing. This code uses the
- package name
+ Optional parts of code have been separated from the MITgcmUV core
+ driver code and organised into packages. The packaging structure
+ provides a mechanism for maintaining suites of code, specific to
+ particular classes of problems, in a way that is cleanly separated from
+ the generic fluid dynamical engine.
+
+ The MITgcmUV packaging structure is described below using generic
+ package names ${pkg}. A concrete examples of a package is the code for
+ implementing GM/Redi mixing. This code uses the package name