--- MITgcm/doc/devel_HOWTO.sgml 2004/04/06 04:12:00 1.7 +++ MITgcm/doc/devel_HOWTO.sgml 2012/05/16 02:33:48 1.17 @@ -1,4 +1,8 @@ +
@@ -22,6 +26,30 @@ Initial version. + + 0.02 + 2010-01-21 + jmc + + Update links. + + + + 0.03 + 2010-04-25 + jmc + + Add subsection "Developer settings" (under CVS Repository). + + + + 0.04 + 2011-04-24 + jmc + + Update subsection "The verification suite". + + @@ -37,7 +65,7 @@ New Versions of This Document You can obtain the latest version of this document online in + url="http://mitgcm.org/public/docs.html">online in various formats. @@ -56,7 +84,7 @@ User Manual Before jumping into development, please familiarize yourself with - the MITgcm user manual + the MITgcm user manual . This document contains volumes of useful information and is included here by reference. @@ -94,6 +122,7 @@ CVS Repository + Layout @@ -104,46 +133,45 @@ others. The tree currently resembles: gcmpack/ - MITgcm-contrib contributed code - CS-regrid goes into utils - cvspolicy.html -save- - CVSROOT -save- - development experimental stuff - manual -save- - misc -?- + CVSROOT -hidden- MITgcm code - adjoint fold into genmake - bin stub for ecco build - compare01 old from 20th century - diags timeave f77 in pkgs now - doc tags -- connect to real docs? - eesupp cnh? - exe ecco user build - ,- jobs runtime shell scripts for - | various platforms - | lsopt line search - m| model main dynamics (core) - e| optimization_drivers ? - r| optim line search interface - g| pkg alternate and optional numerics, etc. - e|- tools - ?| tutorial_examples documented tests - | only populated on release1 branch - | and not validated during "testscript" - '- utils - verification std tests + bin empty + doc basic developpment documentation + eesupp execution environment support code (wrapper) + exe empty + jobs runtime shell scripts for + various platforms (not maintained) + lsopt line search + model main dynamics (core) + optim line search interface + pkg alternate and optional numerics, etc. + tools scripts to build (and test) + utils pre/post processing tools (matlab, ..) + verification standard regression tests + examples + + documented examples (tutorials) + tutorial_examples (only in release1 branch) + MITgcm_contrib contributed code - mitgcmdoc -> manual -remove- + acesgrid.org build acesgrid web site + development experimental stuff + gfd_lab -?- + manual source of MITgcm documentation mitgcm.org build web site - models -?- - packages -?- - preprocess -?- - tmp -?- + old_develop old and early development source + misc -?- + models -?- + packages -?- + preprocess -?- + pdfs some pdfs + planetinabottle.org unfinished web site + www.ecco-group.org build ecco web site ? + @@ -182,25 +210,124 @@ Branches As shown in the online - ViewCVS-generated tree, the MITgcm codebase is split into to two - branches or "lines" under which development proceeds. These two lines are - referred to as the "MAIN" and "ecco" versions of the code. While not - identical, the bulk of the MAIN and ecco lines are composed of files from - the same codebase. + url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph"> + ViewCVS-generated tree, the MITgcm codebase is split into + branches or "lines" under which development proceeds. The main line + of development is referred to as the "MAIN" version of the code. Periodically, a "Release" branch is formed from the "MAIN" development branch. This is done in order to create a relatively stable reference point for both users and developers. The intent is that once a - relese branch has been created, only bug-fixes will be added to it. + release branch has been created, only bug-fixes will be added to it. Meanwhile, development (which might "break" or otherwise render invalid the documentation, tutorials, and/or examples contained within a release - branch) is allowed to continue along the MAIN and ecco lines. + branch) is allowed to continue along the MAIN line. - Tagging + Developer settings + + CVS is a convenient tool to keep up-to-date a personal copy of the + MITgcm code (see: + using CVS ). The same tool is used by developers to + incorporate any change into the repository. However, this later + function requires specific settings, as detailed here after + + + You will need an account (login access) to the server + "mitgcm.org" (curently: forge.csail.mit.edu) + with the proper group setting (e.g., group "gcmctrb" to add/modify + code into MITgcm_contrib). + This kind of account is granted only upon well motivated request + (we recommend to ask your senior MITgcm-collaborator to send such + request to marshall-admin at techsquare.com with Cc to Chris Hill + for approval). + The access to the server mitgcm.org is + through ssh-key authorization which will need to be set properly on + both side (on your local machine and on your server account). + You need to be able to ssh to mitgcm.org + (or ssh MY_USER_NAME@mitgcm.org + in case of different user-name on both sides) to proceed further. + + + + You need to register to the + + mitgcm-cvs mailing list. + This ensures that other developers will receive email notification + when you make changes; you will also receive such email + when others make changes to the repository. + + + + + It is highly recommended that you register also to the + + mitgcm-devel mailing list (expect a short delay for + this request to be processed). + This list is intended for developer discussions. + + + + + The standard CVS-anonymous mode (using "cvsanon", + as mentionned + here ) does not provide check-in ("cvs commit") permission. + Instead, you will need to set our CVS environment as follow: + + $ export CVS_RSH=ssh + $ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack' + + The reason for such limitation is that when downloading a directory, + e.g.: myCopy, from the CVS repository (e.g., + MITgcm_contrib/thisPart) using the command: + + $ cvs co -P -d myCopy MITgcm_contrib/thisPart + + the type of CVS environment which has been used + is stored in the file myCopy/CVS/Root. + This prevent to re-use, for cvs-commit purpose, + a cvs local copy (myCopy) which was obtained + using the CVS anonymous mode. + + + + At this stage, you should be able to send your modified source + file (e.g., src_file) from your local copy directory + (myCopy) to the CVS repository + (MITgcm_contrib/thisPart) using the command + "cvs commit": + + $ cd myCopy + $ cvs -n update (optional; check if new changes have been made) + $ cvs diff src_file (optional; list your changes) + $ cvs commit src_file + + It is essential that you provide a short description of the + changes you made to src_file as you check-in + this file (the "cvs commit" command automatically opens your standard + editor for this purpose). + Note: + Please ignore the following warnings that the "cvs commit" command + produces if you are not part of the "gcmpack" group: + + cvs commit: failed to create lock directory for `/u/gcmpack/CVSROOT' + (/u/gcmpack/CVSROOT/#cvs.history.lock): Permission denied + cvs commit: failed to obtain history lock in repository `/u/gcmpack' + + These warnings are not affecting the changes you made to the CVS repository. + + + + + + + + + Main code development + (formerly named "Tagging" ; this section needs an update) + The intent of tagging is to create "known-good" checkpoints that developers can use as references. Traditionally, MITgcm tagging has @@ -216,8 +343,8 @@ The developer then runs the - testscript shell script to see if any problems are introduced. + url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport"> + testreport shell script to see if any problems are introduced. While not intended to be exhaustive, the test cases within the verification directory do provide some indication whether gross errors have been introduced. @@ -232,7 +359,7 @@ adds a "checkpointXY_pre" comment (where X is a checkpoint number and Y is a letter) to the + url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index"> tag-index file and checks it into the CVS repository @@ -253,17 +380,17 @@ o make KPP work with PTRACERS - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is nearly a copy of kpp_transport_s - - there is no analogue to SurfaceTendencyS, so I have to use + - there is no analogue to SurfaceTendencyS, so I have to use gPtr(of the surface layer) instead o add a new platform SunFire+mpi (SunFire 15000) to genmake checkpoint50e_pre checkpoint50d_post -o change kpp output from multiple-record state files to single-record state +o change kpp output from multiple-record state files to single-record state files analogous to write_state.F -o reduce the output frequency of cg3d-related stuff to the monitor frequency, - analogous to the cg2d-related output. -o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512, +o reduce the output frequency of cg3d-related stuff to the monitor frequency, + analogous to the cg2d-related output. +o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512, so that writing to internal file fn (with length 512) fails. checkpoint50d_pre @@ -277,108 +404,6 @@ - - Editing the Documentation - - - Getting the Docs and Code - - The first step towards editing the documentation is to checkout a - copy of code, docs, and build scripts from the CVS server using: - - - $ export CVS_RSH=ssh - $ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack' - $ mkdir scratch - $ cvs co MITgcm manual mitgcm.org - - - These commands extract the necessary information from the CVS server - and create a temporary (called scratch) directory for - the storage of the HTML and other files that will be created. Please note - that you must either create scratch as shown or edit - the various Makefiles and scripts used to create the - documentation. - - - - Editing the Documentation - - The documentation is contained in the manual - directory in a raw LaTeX format. The main document is - manual.tex and it uses \input{}s - to include the chapters and subsections. - - Since the same LaTeX source is used to produce PostScript, PDF, and - HTML output, care should be taken to follow certain conventions. Two of - the most important are the usage of the \filelink{}{} - and \varlink{}{} commands. Both of these commands have - been defined to simplify the connection between the automatically - generated ("code browser") HTML and the HTML version of the manual - produced by LaTeX2HTML. They each take two arguments (corresponding to - the contents of the two sets of curly braces) which are the text that the - author wishes to be "wrapped" within the link, and a specially formatted - link thats relative to the MITgcm directory within - the CVS tree. - - The result is a command that resembles either - - - - a reference to a variable or subroutine name such as - \varlink{tRef}{tRef}, or - - - - a reference to a file such as - \varlink{tRef}{path-to-the-file_name.F} - where the absolute path to the file is of the form - /foo/MITgcm/path/to/the/file_name.F - (please note how the leading "/foo/MITgcm" - component of the path is dropped leaving the path - relative to the head of the code - directory and each directory separator "/" is turned - into a "-") - - - - - - - - - Building the Documentation - - Given the directory structure of , the entire documentation for the web - site can be built using: - - - $ cd mitgcm.org/devel/buildweb - $ make All - - - Which builds the PDF from the LaTeX source, creates the HTML output - from the LaTeX source, parses the FORTRAN code base to produce a - hyperlinked HTML version of the source, and then determines the - cross-linking between the various HTML components. - - If there are no errors, the result of the build process (which can - take 30+ minutes on a P4/2.5Ghz) will be contained within a single - directory called scratch/dev_docs. This is a freshly - built version of the entire on-line users manual. If you have the correct - permissions, it can be directly copied to the web server area: - - - $ mv scratch/dev_docs /u/u0/httpd/html - - - and the update is complete. - - - - - Coding for MITgcm @@ -392,20 +417,19 @@ For MITgcm, the process is similar. Typical commands are: - $ genmake -mods=../code + $ genmake2 -mods=../code $ make depend $ make The following sections describe the individual steps in the build process. - + The <filename>genmake2</> Utility - Please note that the older genmake is - deprecated and will eventually be replaced by genmake2. - This HOWTO only describes the newer tool. + (Note: the older genmake + has been replaced by genmake2) The first step in any MITgcm build is to create a Unix-style Makefile which will be parsed by @@ -461,6 +485,12 @@ an "options file" as specified by the command-line option -optfile='FILENAME' + + a packages.conf file (in the current + directory or in one of the "MODS" directories, see below) + which contains the specific list of packages to compile + + then checking certain dependency rules (the package dependencies), @@ -514,33 +544,6 @@ - -pdepend=/PATH/FILENAME - - - This specifies the dependency file used for packages. If - not specified, the default dependency file is - $ROOTDIR/pkg/pkg_depend. The syntax for this file is - parsed on a line-by-line basis where each line containes either a - comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule - where the "+" or "-" symbol specifies a "must be used with" or a - "must not be used with" relationship, respectively. If no rule is - specified, then it is assumed that the two packages are compatible - and will function either with or without each other. - - - - - -pdefault=PKG - -pdefault='PKG1 [PKG2 PKG3 ...]' - - This option specifies the default set of packages - to be used. If not set, the default package list will - be read from - $ROOTDIR/pkg/pkg_default. - - - - -adof=/path/to/file -adoptfile=/path/to/file @@ -568,19 +571,49 @@ will be overridden by any identically-named sources within the "MODS" directories. The order of precedence for this "name-hiding" is as follows: - "MODS" directories (in the order given) Packages either explicitly specified or provided by default (in the order given) Packages included due to package dependencies - (in the order that that package dependencies are + (in the order that that package dependencies are parsed) The "standard dirs" (which may have been specified by the "-standarddirs" option) + + + + + -pgroups=/PATH/FILENAME + + This option specifies the file where package groups are + defined. If not set, the package-groups definition will + be read from + $ROOTDIR/pkg/pkg_groups. + + It also contains the default list of packages (defined + as the group "default_pkg_list") which is used + when no specific package list (file: packages.conf) + is found in current directory or in any "MODS" directory. + + + + + + -pdepend=/PATH/FILENAME + + This specifies the dependency file used for packages. If + not specified, the default dependency file is + $ROOTDIR/pkg/pkg_depend. The syntax for this file is + parsed on a line-by-line basis where each line containes either a + comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule + where the "+" or "-" symbol specifies a "must be used with" or a + "must not be used with" relationship, respectively. If no rule is + specified, then it is assumed that the two packages are compatible + and will function either with or without each other. @@ -606,7 +639,7 @@ In general, it is best to use genmake2 on a "clean" directory that is free of all source (*.[F,f],*.[F,f]90) and header (*.h,*.inc) files. Generally, this can be accomplished in an - "un-clean" directory by running "make CLEAN" followed by "make + "un-clean" directory by running "make Clean" followed by "make makefile". @@ -619,16 +652,18 @@ simulator) executable using: - $ make CLEAN + $ make Clean $ make depend $ make - The "make CLEAN" step will remove any stale source files, include + The "make Clean" step will remove any stale source files, include files, and links. It is strongly recommended for "un-clean" directories which may contain the (perhaps partial) results of - previous builds. Such "debris" can interfere with the next stage of - the build. + previous builds. Such "debris" can interfere with the next stage of + the build. + A more agressive cleaning option, "make CLEAN", can be used to also + remove the executable and output files from a previous run. The "make depend" step will create a large number of symbolic links from the local directory to the source file locations. It also @@ -661,7 +696,7 @@ This target produces an mitgcmuv_ad executable using the taf or staf adjoint - compiler. See the genmake2 "-adof" option for + compiler. See the genmake2 "-adof" option for compiler selection. @@ -687,39 +722,253 @@ The Verification Suite The MITgcm CVS tree (within the $ROOTDIR/verification/ - directory) includes more than a dozen examples intended for regression - testing. Each one of these example directories contains "known-good" + directory) includes many (> 90) examples intended for regression + testing. Each one of these test-experiment directories contains "known-good" output files along with all the input (including both code and data - files) required for their re-calculation. These example directories are - further broken down into sets of subdirectories - (eg. /input, /code) intended to expedite the - testing process. + files) required for their re-calculation. + Also included in $ROOTDIR/verification/ is the shell + script testreport to perform regression tests. + + + Test-experiment Directory Content + + Each test-experiment directory (TESTDIR) contains + several standard subdirectories and files which testreport + recognizes and uses when running a regression test. + The directories/files that testreport uses are different + for a forward test and an adjoint test (testreport -adm) + and some test-experiments are set-up for only one type of regression + test whereas others allow both types of tests (forward and adjoint). + Also some test-experiment allows, using the same MITgcm executable, + to perform multiple tests using different parameters and input files, + with a primary input set-up + (input/ or input_ad/ ) + and corresponding results (results/output.txt or + results/output_adm.txt) and with one or several secondary + inputs (input.OTHER/ or input_ad.OTHER/ ) + and corresponding results (results/output.OTHER.txt + or results/output_adm.OTHER.txt). + + + + directory TESTDIR/results/ + + contains reference standard output used for test comparison. + results/output.txt and + results/output_adm.txt + correspond respectively to primary forward and adjoint test + run on the reference platform (currently + baudelaire.csail.mit.edu) + on one processor (no MPI, single thread) + using the reference compiler (curently the GNU fortran + compiler gfortran). + The presence of these files determines whether or not + testreport is testing or skipping + this test-experiment. + Reference standard output for secondary tests + (results/output.OTHER.txt + or results/output_adm.OTHER.txt) are + also expected here. + The test comparison involves few model variables output, which are, + by default and for a forward test, the 2-D solver initial residual + (cg2d_init_res) and 3-D state variables (T,S,U,V) + monitor output, and, by default and for an adjoint test, the + cost-function and gradient-check. However, some test-experiments + use some package-specific variable/monitor output according to + the file TESTDIR/input[_ad][.OTHER]/tr_checklist + specification. + + + + + directory TESTDIR/build/ + + initially empty directory where testreport + will build the MITgcm executable for forward and adjoint test. + It might contains an experiment specific + genmake_local file (see ). + + Note that the original code[_ad]/SIZE.h_mpi + is not directly used as "SIZE.h" to build an MPI-executable ; + instead, a local copy build/SIZE.h.mpi is derived + from code[_ad]/SIZE.h_mpi + by adjusting the number of processors (nPx,nPy) according to + NUMBER_OF_PROCS (see , + testreport -MPI) ; then it is linked to "SIZE.h" + ( ln -s SIZE.h.mpi SIZE.h ) before building + the MPI-executable. + + + + + directory TESTDIR/code/ + + contains the test-experiment specific source code + used to build the MITgcm executable (mitgcmuv) + for forward-test (using genmake2 -mods=../code). + + It can also contain specific source files with the suffix + "_mpi" to be used + + in place of the corresponding file (without suffix) + for an MPI test (see ). + The presence or absence of SIZE.h_mpi + determines whether or not an MPI test on this + test-experiment is performed or skipped. + + + + + directory TESTDIR/code_ad/ + + contains the test-experiment specific source code + used to build the MITgcm executable (mitgcmuv_ad) + for adjoint-test (using genmake2 -mods=../code_ad). + It can also contain specific source files with the suffix + "_mpi" (see above). + + + + + directory TESTDIR/input/ + + contains the input and parameter files + used to run the primary forward test of this test-experiment. + + It can also contain specific parameter files with the suffix + ".mpi" to be used + + in place of the corresponding file (without suffix) for MPI + test, or with suffix ".mth" to be used for + multi-threaded test (see ). + The presence or absence of eedata.mth + determines whether or not a multi-threaded test on this + test-experiment is performed or skipped. + To save disk space and reduce downloading time, multiple + copies of the same input file is avoided by using a shell + script prepare_run. + When such a script is found in TESTDIR/input/ , + testreport run this script in directory + TESTDIR/run/ after linking all the input file + from TESTDIR/input/ . + + + + + + directory TESTDIR/input_ad/ + + contains the input and parameter files + used to run the primary adjoint test of this test-experiment. + It can also contain specific parameter files with the suffix + ".mpi" and shell script prepare_run + as described above. + + + + + directory TESTDIR/input.OTHER/ + + contains the input and parameter files + used to run the secondary OTHER forward test of this test-experiment. + It can also contain specific parameter files with suffix + ".mpi" or ".mth" and shell script + prepare_run (see above). + The presence or absence the file eedata.mth + determines whether or not a secondary multi-threaded test on this + test-experiment is performed or skipped. + + + + + directory TESTDIR/input_ad.OTHER/ + + contains the input and parameter files + used to run the secondary OTHER adjoint test of this test-experiment. + It can also contain specific parameter files with the suffix + ".mpi" and shell script prepare_run + (see above). + + + + + directory TESTDIR/run/ + + initially empty directory where testreport + will run the MITgcm executable for primary forward and adjoint + test. + Symbolic links (using command "ln -s") are made + for input and parameter files (from ../input/ + or from ../input_ad/ ) and for MITgcm executable + (from ../build/ ) before the run proceeds. + The sequence of links (function linkdata within shell + script testreport) for a forward test is: + +* link+rename or remove links + to special files with suffix ".mpi" or ".mth" from ../input/ +* link files from ../input/ +* execute ../input/prepare_run (if it exists) + + The sequence for an ajoint test is similar, with + ../input_ad/ replacing ../input/ . + + + + + + directory TESTDIR/tr_run.OTHER/ + + directory created by testreport + to run the MITgcm executable for secondary "OTHER" forward + or adjoint test. + The sequence of links for a forward secondary test is: + +* link+rename or remove links + to special files with suffix ".mpi" or ".mth" from ../input.OTHER/ +* link files from ../input.OTHER/ +* execute ../input.OTHER/prepare_run (if it exists) +* link files from ../input/ +* execute ../input/prepare_run (if it exists) + + The sequence for an ajoint test is similar, with + ../input_ad.OTHER/ and ../input_ad/ + replacing ../input.OTHER/ and ../input/ . + + + + + + The <filename>testreport</> Utility - Also included in $ROOTDIR/verification/ are shell - scripts for automated testing. The newest script (which was written - to work with genmake2) is called testreport. - This script can be used to build different versions of the MITgcm - code, run the various examples, compare the output, and (if specified) - email the results of each one of these tests to a central - repository. + The shell script testreport (in + $ROOTDIR/verification/), which was written to work with + genmake2, can be used to build different versions + of the MITgcm code, run the various examples, compare the output, + and (if specified) email the results of each one of these tests to a + central repository. On some systems, the testreport script can be run with a command line as simple as: $ cd verification - $ ./testreport -ieee + $ ./testreport However, some systems (those lacking or wiht a broken "/bin/sh") may require an explicit shell invocation such as: - $ sh ./testreport -ieee -t 'exp0 exp4' - $ /some/path/to/bash ./testreport -ieee -t 'ideal_2D_oce lab_sea natl_box' + $ sh ./testreport -t 'exp2 exp4' + $ /some/path/to/bash ./testreport -t 'ideal_2D_oce lab_sea natl_box' The testreport script accepts a number of @@ -729,11 +978,24 @@ - -ieee + -ieee (default) / -noieee If allowed by the compiler (as defined in the "optfile"), - use IEEE arithmetic. This option, along with the GCC compiler, - is how the standard results were produced. + use IEEE arithmetic (genmake2 -ieee). + This option, along with the gfortran / gcc compiler, + is how the standard results are produced. + + + + + -optfile=/PATH/FILENAME + -optfile '/PATH/F1 [/PATH/F2 ...]' + + This specifies a list of "options files" that will be passed + to genmake2. If multiple options files are used + (say, to test different compilers or different sets of options + for the same compiler), then each options file will be used with + each of the test directories. @@ -744,7 +1006,7 @@ This option specifies the test directory or list of test directories that should be used. Each of these entries should exactly (note: they are case sensitive!) match the names of - directries in $ROOTDIR/verification/. If this + directories in $ROOTDIR/verification/. If this option is omitted, then all directories that are properly formatted (that is, containing an input sub-directory and a results/output.txt file) will @@ -753,18 +1015,6 @@ - -optfile=/PATH/FILENAME - -optfile '/PATH/F1 [/PATH/F2 ...]' - - This specifies a list of "options files" that will be passed - to genmake2. If multiple options files are used - (say, to test different compilers or different sets of options - for the same compiler), then each options file will be used with - each of the test directories. - - - - -addr EMAIL -addr 'EMAIL1 EMAIL2 [...]' @@ -778,39 +1028,93 @@ + -MPI NUMBER_OF_PROCS -mpi - If the necessary files - (TESTDIR/code/CPP_EEOPTIONS.h_mpi and - TESTDIR/code/SIZE.h_mpi) exist, then use them for an - MPI--enabled run. Note that the use of MPI typically requires a + If the necessary file (TESTDIR/code/SIZE.h_mpi) + exists, then use it (and all TESTDIR/code/*_mpi files) + for an MPI--enabled run. The new option (-MPI followed + by the maximum number of processors) enable to build and run each + test-experiment using variable number of MPI processors (multiple + of nPx*nPy from TESTDIR/code/SIZE.h_mpi + and not larger than NUMBER_OF_PROCS). + The short option ("-mpi") can only be used to build and run on 2 MPI + processors (equivalent to "-MPI 2"). + Note that the use of MPI typically requires a special command option (see "-command" below) to invoke the MPI - executable. Examples of PBS scripts using MPI with testreport can be + executable. Examples of PBS scripts using testreport with MPI can be found in the - MITgcm-contrib area + url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/"> + tools/example_scripts directory. -command='some command to run' - For some tests, particularly MPI runs, the default "make - output.txt" is not sufficient. This option allows a more general + For some tests, particularly MPI runs, a specific command + might be needed to run the executable. This option allows a more general command (or shell script) to be invoked. Examples of PBS scripts - using MPI with testreport can be found in the - MITgcm-contrib area + using testreport with MPI can be found in the + tools/example_scripts directory. + + For the case where the number of MPI processors varies according + to each test-experiment, some key-words within the command-to-run + argument will be replaced by their effective value: + + TR_NPROC will be replaced by the actual number + of MPI processors needed to run the current test-experiment. + + TR_MFILE will be replaced by the name of local-file + that testreport creates from the full list of machines which + "testreport -mf MACHINE_FILE" provides, but truncated + to the exact number of machines. + + + + + -mf MACHINE_FILE + + + To use with -MPI NUMBER_OF_PROCS option, to specify + the file containing the full list of NUMBER_OF_PROCS + machines to use for the MPI runs. + + + + + -mth + + compile (with genmake2 -omp) and run with + multiple threads (using eedata.mth). - The testreport script will write progress to the - screen (stdout) as it runs. In addition, it will create a - tr_out.txt file that contains a brief comparison of the - current output with the "known-good" output. + The testreport script will create an output directory + tr_NAME_DATE_N/ , with hostname as default + NAME, DATE the current date followed by + a suffix number "N" to distinguish from previous + testreport output directories. + testreport writes progress to the screen (stdout) + and reports into the ouput directory as it runs. + In particular, one can find, in the ouput directory, + the summary.txt file that contains a brief comparison + of the current output with the "known-good" output. + At the end of the testing process, the tr_out.txt file + is generated in $ROOTDIR/verification/ + as a compact version of summry.txt file. + + + + The <filename>do_tst_2+2</> Utility + The shell script do_tst_2+2 (in + $ROOTDIR/tools/ ) + can be used to check the accuracy of the restart procedure. + @@ -826,7 +1130,7 @@ the generic fluid dynamical engine. The MITgcmUV packaging structure is described below using generic - package names ${pkg}. A concrete examples of a package is the code for + package names ${pkg}. A concrete examples of a package is the code for implementing GM/Redi mixing. This code uses the package name @@ -851,7 +1155,7 @@ below using generic package names ${pkg}. A concrete examples of a package is the code for implementing GM/Redi mixing. This code uses -the package name +the package name * ${PKG} = GMREDI * ${pkg} = gmredi * ${Pkg} = gmRedi @@ -886,6 +1190,9 @@ Package runtime config. options are imported into a common block held in a header file called "${PKG}.h". + Note: In some packages, the header file "${PKG}.h" is splitted + into "${PKG}_PARAMS.h" that contains the package parameters and + ${PKG}_VARS.h" for the field arrays. o The core driver part of the model can check for runtime enabling or disabling of individual packages @@ -908,12 +1215,14 @@ 1. Within the core driver code flags of the form ALLOW_${PKG} are used to include or exclude whole packages. The ALLOW_${PKG} flags are included - from a PKG_CPP_OPTIONS block which is currently + from a PACKAGES_CONFIG.h file that is automatically + generated by genmake2 (see genmake2 section). held in-line in the CPP_OPTIONS.h header file. e.g. Core model code ..... + #include "PACKAGES_CONFIG.h" #include "CPP_OPTIONS.h" : : @@ -925,8 +1234,8 @@ 2. Within an individual package a header file, "${PKG}_OPTIONS.h", is used to set CPP flags - specific to that package. It is not recommended - to include this file in "CPP_OPTIONS.h". + specific to that package. It also includes + "PACKAGES_CONFIG.h" and "CPP_OPTIONS.h". Package Boot Sequence @@ -939,7 +1248,7 @@ 1. S/R PACKAGES_BOOT() : CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... ) - + 2. S/R PACKAGES_READPARMS() : @@ -978,7 +1287,7 @@ #ifdef ALLOW_${PKG} if ( use${Pkg} ) - & CALL ${PKG}_DIAGS( ) + & CALL ${PKG}_OUTPUT( ) #endif 7. S/R PACKAGES_WRITE_PICKUP() @@ -991,20 +1300,20 @@ Description =========== - - ${PKG}_READPARMS() + - ${PKG}_READPARMS() is responsible for reading in the package parameters file data.${pkg}, and storing - the package parameters in "${PKG}.h". + the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h"). -> called from INITIALISE_FIXED in PACKAGES_READPARMS - - ${PKG}_INIT_FIXED() - is responsible for completing the internal setup of a package. + - ${PKG}_INIT_FIXED() + is responsible for completing the internal setup of a package. -> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED note: 1) some pkg use instead: CALL ${PKG}_INITIALISE ( or the old form CALL ${PKG}_INIT ) 2) for simple pkg setup, this part is done inside ${PKG}_READPARMS - - ${PKG}_CHECK() + - ${PKG}_CHECK() is responsible for validating basic package setup and inter-package dependencies. ${PKG}_CHECK can import other package parameters it may @@ -1013,29 +1322,32 @@ will not be reset during ${PKG}_CHECK(). -> called from INITIALISE_FIXED in PACKAGES_CHECK - - ${PKG}_INIT_VARIA() + - ${PKG}_INIT_VARIA() is responsible for fill-in all package variables with an initial value. Contains eventually a call to ${PKG}_READ_PICKUP that will read from a pickup file the package variables required to restart the model. - This routine is called after the core model state has been completely + This routine is called after the core model state has been completely initialised but before the core model timestepping starts. -> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES - note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg - use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old + note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg + use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old form ${PKG}_INIT - - ${PKG}_DIAGS() - is responsible for writing time-average diagnostics to output - files (but the cumulating step is done within the package main S/R). - Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR) - and write snap-shot fields that are hold in common blocks. Other + - ${PKG}_OUTPUT( ) + is responsible for writing time-average fields to output files + (but the cumulating step is done within the package main S/R). + Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR) + and write snap-shot fields that are hold in common blocks. Other temporary fields are directly dump to file where they are available. - NOTE: this part does not yet have a standard form and should be called - from a package dedicated S/R such as PACKAGE_WRITE_DIAGS + NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages + but is beeing replaced by ${PKG}_OUTPUT + to avoid confusion with pkg/diagnostics functionality. + 2) the output part is not yet in a standard form and might still + evolve a lot. -> called within DO_THE_MODEL_IO - - ${PKG}_WRITE_PICKUP() - is responsible for writing a package pickup file when necessary for + - ${PKG}_WRITE_PICKUP() + is responsible for writing a package pickup file when necessary for a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT ) -> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP @@ -1058,6 +1370,8 @@ ----------------------- * ${PKG}_OPTIONS.h has further package-specific CPP options * ${PKG}.h package-specific common block variables, fields + or ${PKG}_PARAMS.h package-specific common block parameters + and ${PKG}_VARS.h package-specific common block fields - FORTRAN source files ----------------------- @@ -1066,9 +1380,13 @@ * ${pkg}_check.F checks package dependencies and consistencies * ${pkg}_init_varia.F initialises package-related fields * ${pkg}_... .F package source code - * ${pkg}_diags.F write diagnostics to file. + * ${pkg}_output.F write output to file. * ${pkg}_write_pickup.F write a package pickup file to restart the model + New: Subroutine in one package (pkgA) that only contains code which + is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F) + will be named: pkgA_pkgB_something.F + - parameter file ----------------------- * data.${pkg} parameter file @@ -1077,6 +1395,108 @@ + + Editing the Documentation + + + Getting the Docs and Code + + The first step towards editing the documentation is to checkout a + copy of code, docs, and build scripts from the CVS server using: + + + $ export CVS_RSH=ssh + $ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack' + $ mkdir scratch + $ cvs co -P MITgcm manual mitgcm.org + + + These commands extract the necessary information from the CVS server + and create a temporary (called scratch) directory for + the storage of the HTML and other files that will be created. Please note + that you must either create scratch as shown or edit + the various Makefiles and scripts used to create the + documentation. + + + + Editing the Documentation + + The documentation is contained in the manual + directory in a raw LaTeX format. The main document is + manual.tex and it uses \input{}s + to include the chapters and subsections. + + Since the same LaTeX source is used to produce PostScript, PDF, and + HTML output, care should be taken to follow certain conventions. Two of + the most important are the usage of the \filelink{}{} + and \varlink{}{} commands. Both of these commands have + been defined to simplify the connection between the automatically + generated ("code browser") HTML and the HTML version of the manual + produced by LaTeX2HTML. They each take two arguments (corresponding to + the contents of the two sets of curly braces) which are the text that the + author wishes to be "wrapped" within the link, and a specially formatted + link thats relative to the MITgcm directory within + the CVS tree. + + The result is a command that resembles either + + + + a reference to a variable or subroutine name such as + \varlink{tRef}{tRef}, or + + + + a reference to a file such as + \varlink{tRef}{path-to-the-file_name.F} + where the absolute path to the file is of the form + /foo/MITgcm/path/to/the/file_name.F + (please note how the leading "/foo/MITgcm" + component of the path is dropped leaving the path + relative to the head of the code + directory and each directory separator "/" is turned + into a "-") + + + + + + + + + Building the Documentation + + Given the directory structure of , the entire documentation for the web + site can be built using: + + + $ cd mitgcm.org/devel/buildweb + $ make All + + + Which builds the PDF from the LaTeX source, creates the HTML output + from the LaTeX source, parses the FORTRAN code base to produce a + hyperlinked HTML version of the source, and then determines the + cross-linking between the various HTML components. + + If there are no errors, the result of the build process (which can + take 30+ minutes on a P4/2.5Ghz) will be contained within a single + directory called scratch/dev_docs. This is a freshly + built version of the entire on-line users manual. If you have the correct + permissions, it can be directly copied to the web server area: + + + $ mv scratch/dev_docs /u/u0/httpd/html + + + and the update is complete. + + + + +