1 |
<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN"> |
<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN"> |
2 |
|
<!-- |
3 |
|
$Header$ |
4 |
|
$Name$ |
5 |
|
--> |
6 |
|
|
7 |
<article id="MITgcm-Development-HOWTO"> |
<article id="MITgcm-Development-HOWTO"> |
8 |
|
|
26 |
Initial version. |
Initial version. |
27 |
</revremark> |
</revremark> |
28 |
</revision> |
</revision> |
29 |
|
<revision> |
30 |
|
<revnumber>0.02</revnumber> |
31 |
|
<date>2010-01-21</date> |
32 |
|
<authorinitials>jmc</authorinitials> |
33 |
|
<revremark> |
34 |
|
update links. |
35 |
|
</revremark> |
36 |
|
</revision> |
37 |
|
<revision> |
38 |
|
<revnumber>0.03</revnumber> |
39 |
|
<date>2010-04-25</date> |
40 |
|
<authorinitials>jmc</authorinitials> |
41 |
|
<revremark> |
42 |
|
Add subsection "Developer settings" (under CVS Repository). |
43 |
|
</revremark> |
44 |
|
</revision> |
45 |
</revhistory> |
</revhistory> |
46 |
|
|
47 |
<abstract> |
<abstract> |
57 |
<sect2> |
<sect2> |
58 |
<title>New Versions of This Document</title> <para>You can |
<title>New Versions of This Document</title> <para>You can |
59 |
obtain the latest version of this document <ulink |
obtain the latest version of this document <ulink |
60 |
url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in |
url="http://mitgcm.org/public/docs.html">online</ulink> in |
61 |
various formats.</para> |
various formats.</para> |
62 |
</sect2> |
</sect2> |
63 |
<sect2> |
<sect2> |
76 |
<title>User Manual</title> |
<title>User Manual</title> |
77 |
|
|
78 |
<para>Before jumping into development, please familiarize yourself with |
<para>Before jumping into development, please familiarize yourself with |
79 |
the <ulink url="http://mitgcm.org/docs.html"> MITgcm user manual |
the <ulink url="http://mitgcm.org/public/docs.html"> MITgcm user manual |
80 |
</ulink>. This document contains volumes of useful information and is |
</ulink>. This document contains volumes of useful information and is |
81 |
included here by reference.</para> |
included here by reference.</para> |
82 |
|
|
114 |
|
|
115 |
<sect1 id="cvs"> |
<sect1 id="cvs"> |
116 |
<title>CVS Repository</title> |
<title>CVS Repository</title> |
117 |
|
|
118 |
<sect2> |
<sect2> |
119 |
<title>Layout</title> |
<title>Layout</title> |
120 |
|
|
125 |
others. The tree currently resembles:</para> |
others. The tree currently resembles:</para> |
126 |
|
|
127 |
<programlisting>gcmpack/ |
<programlisting>gcmpack/ |
128 |
MITgcm-contrib contributed code |
CVSROOT -hidden- |
|
CS-regrid goes into utils |
|
|
cvspolicy.html -save- |
|
|
CVSROOT -save- |
|
|
development experimental stuff |
|
|
manual -save- |
|
|
misc -?- |
|
129 |
|
|
130 |
MITgcm code |
MITgcm code |
131 |
adjoint fold into genmake |
bin empty |
132 |
bin stub for ecco build |
doc basic developpment documentation |
133 |
compare01 old from 20th century |
eesupp execution environment support code (wrapper) |
134 |
diags timeave f77 in pkgs now |
exe empty |
135 |
doc tags -- connect to real docs? |
jobs runtime shell scripts for |
136 |
eesupp cnh? |
various platforms (not maintained) |
137 |
exe ecco user build |
lsopt line search |
138 |
,- jobs runtime shell scripts for |
model main dynamics (core) |
139 |
| various platforms |
optim line search interface |
140 |
| lsopt line search |
pkg alternate and optional numerics, etc. |
141 |
m| model main dynamics (core) |
tools scripts to build (and test) |
142 |
e| optimization_drivers ? |
utils pre/post processing tools (matlab, ..) |
143 |
r| optim line search interface |
verification standard regression tests + examples |
144 |
g| pkg alternate and optional numerics, etc. |
+ documented examples (tutorials) |
145 |
e|- tools |
tutorial_examples (only in release1 branch) |
|
?| tutorial_examples documented tests |
|
|
| only populated on release1 branch |
|
|
| and not validated during "testscript" |
|
|
'- utils |
|
|
verification std tests |
|
146 |
|
|
147 |
|
MITgcm_contrib contributed code |
148 |
|
|
149 |
mitgcmdoc -> manual -remove- |
acesgrid.org build acesgrid web site |
150 |
|
development experimental stuff |
151 |
|
gfd_lab -?- |
152 |
|
manual source of MITgcm documentation |
153 |
mitgcm.org build web site |
mitgcm.org build web site |
154 |
models -?- |
old_develop old and early development source |
155 |
packages -?- |
misc -?- |
156 |
preprocess -?- |
models -?- |
157 |
tmp -?- |
packages -?- |
158 |
|
preprocess -?- |
159 |
|
pdfs some pdfs |
160 |
|
planetinabottle.org unfinished web site |
161 |
|
www.ecco-group.org build ecco web site ? |
162 |
</programlisting> |
</programlisting> |
163 |
|
|
164 |
|
<!-- |
165 |
<para>Efforts are underway to reduce the complexity.</para> |
<para>Efforts are underway to reduce the complexity.</para> |
166 |
|
--> |
167 |
|
|
168 |
</sect2> |
</sect2> |
169 |
|
|
202 |
<title>Branches</title> |
<title>Branches</title> |
203 |
|
|
204 |
<para>As shown in the online <ulink |
<para>As shown in the online <ulink |
205 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174"> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph"> |
206 |
ViewCVS-generated tree</ulink>, the MITgcm codebase is split into to two |
ViewCVS-generated tree</ulink>, the MITgcm codebase is split into |
207 |
branches or "lines" under which development proceeds. These two lines are |
branches or "lines" under which development proceeds. The main line |
208 |
referred to as the "MAIN" and "ecco" versions of the code. While not |
of development is referred to as the "MAIN" version of the code. |
|
identical, the bulk of the MAIN and ecco lines are composed of files from |
|
|
the same codebase. |
|
209 |
</para> |
</para> |
210 |
|
|
211 |
<para>Periodically, a "Release" branch is formed from the "MAIN" |
<para>Periodically, a "Release" branch is formed from the "MAIN" |
212 |
development branch. This is done in order to create a relatively stable |
development branch. This is done in order to create a relatively stable |
213 |
reference point for both users and developers. The intent is that once a |
reference point for both users and developers. The intent is that once a |
214 |
relese branch has been created, only bug-fixes will be added to it. |
release branch has been created, only bug-fixes will be added to it. |
215 |
Meanwhile, development (which might "break" or otherwise render invalid |
Meanwhile, development (which might "break" or otherwise render invalid |
216 |
the documentation, tutorials, and/or examples contained within a release |
the documentation, tutorials, and/or examples contained within a release |
217 |
branch) is allowed to continue along the MAIN and ecco lines.</para> |
branch) is allowed to continue along the MAIN line.</para> |
218 |
|
</sect2> |
219 |
|
|
220 |
|
<sect2> |
221 |
|
<title> Developer settings </title> |
222 |
|
|
223 |
|
<para>CVS is a convenient tool to keep up-to-date a personal copy of the |
224 |
|
MITgcm code (see: <ulink url="http://mitgcm.org/public/using_cvs.html"> |
225 |
|
using CVS </ulink>). The same tool is used by developers to |
226 |
|
incorporate any change into the repository. However, this later |
227 |
|
function requires specific settings, as detailed here after:</para> |
228 |
|
<orderedlist> |
229 |
|
<listitem> |
230 |
|
<para> You will need an account (loggin access) to the server |
231 |
|
"mitgcm.org" with the proper group setting (e.g., |
232 |
|
group "gcmctrb" to add/modify code into MITgcm_contrib). |
233 |
|
This kind of account is granted only upon well motivated request. |
234 |
|
The access to the server mitgcm.org is through ssh-key authorization |
235 |
|
which will need to be set properly on both side (on your local machine |
236 |
|
and on your server account). You need to be able to |
237 |
|
to ssh to mitgcm.org (or <filename>ssh MY_USER_NAME@mitgcm.org</filename> |
238 |
|
in case of different user-name on both sides) to proceed further.</para> |
239 |
|
</listitem> |
240 |
|
|
241 |
|
<listitem> |
242 |
|
<para> You need to register to the |
243 |
|
<ulink url="http://mitgcm.org/mailman/listinfo/mitgcm-cvs"> |
244 |
|
mitgcm-cvs </ulink> mailing list. |
245 |
|
This ensures that other developers will receive email notification |
246 |
|
when you make changes; you will also receive as well such email |
247 |
|
when others make changes to the repository. |
248 |
|
</para> |
249 |
|
</listitem> |
250 |
|
|
251 |
|
<listitem> |
252 |
|
<para> It is highly recommended that you register also to the |
253 |
|
<ulink url="http://mitgcm.org/mailman/listinfo/mitgcm-devel"> |
254 |
|
mitgcm-devel </ulink> mailing list (expect a short delay for |
255 |
|
this request to be processed). |
256 |
|
This list is intended for developer discussions. |
257 |
|
</para> |
258 |
|
</listitem> |
259 |
|
|
260 |
|
<listitem> |
261 |
|
<para> The standard anonymous mode (using "cvsanon", as mentionned |
262 |
|
<ulink url="http://mitgcm.org/public/source_code.html"> |
263 |
|
here </ulink>) does not allow check-in ("cvs commit") permission. |
264 |
|
Instead, you will need to set our CVS environment as follow:</para> |
265 |
|
<screen> |
266 |
|
$ export CVS_RSH=ssh |
267 |
|
$ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack' |
268 |
|
</screen> |
269 |
|
<para> After downloading a directory, e.g.: <filename>myCopy</filename>, |
270 |
|
from the CVS repository (e.g., |
271 |
|
<filename>MITgcm_contrib/thisPart</filename>) using the command:</para> |
272 |
|
<screen> |
273 |
|
$ cvs co -P -d myCopy MITgcm_contrib/thisPart |
274 |
|
</screen> |
275 |
|
<para> the type of CVS environment which has been used |
276 |
|
is stored in the file <filename>myCopy/CVS/Root</filename> |
277 |
|
and makes it difficult to re-use, for cvs-commit purpose, |
278 |
|
a cvs local copy (<filename>myCopy</filename>) which was obtained |
279 |
|
using the CVS anonymous mode.</para> |
280 |
|
</listitem> |
281 |
|
|
282 |
|
<listitem> |
283 |
|
<para> At this stage, you should be able to send your modified source |
284 |
|
file (e.g., <filename>src_file</filename>) from your local copy directory |
285 |
|
(<filename>myCopy</filename>) to the CVS repository |
286 |
|
(<filename>MITgcm_contrib/thisPart</filename>) using the command |
287 |
|
"cvs commit":</para> |
288 |
|
<screen> |
289 |
|
$ cd myCopy |
290 |
|
$ cvs -n update (optional; check if new changes have been made) |
291 |
|
$ cvs diff src_file (optional; list your changes) |
292 |
|
$ cvs commit src_file |
293 |
|
</screen> |
294 |
|
<para> It is essential that you provide a short description of the |
295 |
|
changes you made to <filename>src_file</filename> as you check-in |
296 |
|
this file (the "cvs commit" command automatically opens your standard |
297 |
|
editor for this purpose).</para> |
298 |
|
</listitem> |
299 |
|
|
300 |
|
</orderedlist> |
301 |
|
|
302 |
</sect2> |
</sect2> |
303 |
|
|
304 |
<sect2> |
<sect2> |
305 |
<title>Tagging</title> |
<title>Main code development</title> |
306 |
|
<para><emphasis>(formerly named "Tagging" ; this section needs an update) |
307 |
|
</emphasis></para> |
308 |
|
|
309 |
<para>The intent of tagging is to create "known-good" checkpoints that |
<para>The intent of tagging is to create "known-good" checkpoints that |
310 |
developers can use as references. Traditionally, MITgcm tagging has |
developers can use as references. Traditionally, MITgcm tagging has |
320 |
|
|
321 |
<listitem> |
<listitem> |
322 |
<para>The developer then runs the <ulink |
<para>The developer then runs the <ulink |
323 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript"> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport"> |
324 |
testscript</ulink> shell script to see if any problems are introduced. |
testreport</ulink> shell script to see if any problems are introduced. |
325 |
While not intended to be exhaustive, the test cases within the |
While not intended to be exhaustive, the test cases within the |
326 |
verification directory do provide some indication whether gross errors |
verification directory do provide some indication whether gross errors |
327 |
have been introduced. |
have been introduced. |
336 |
<listitem> |
<listitem> |
337 |
<para>adds a "checkpointXY_pre" comment (where X is a checkpoint |
<para>adds a "checkpointXY_pre" comment (where X is a checkpoint |
338 |
number and Y is a letter) to the <ulink |
number and Y is a letter) to the <ulink |
339 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index"> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index"> |
340 |
tag-index</ulink> file and checks it into the CVS |
tag-index</ulink> file and checks it into the CVS |
341 |
repository</para> |
repository</para> |
342 |
</listitem> |
</listitem> |
381 |
</sect1> |
</sect1> |
382 |
|
|
383 |
|
|
|
<sect1 id="documentation"> |
|
|
<title>Editing the Documentation</title> |
|
|
|
|
|
<sect2 id="documentation_getting"> |
|
|
<title>Getting the Docs and Code</title> |
|
|
|
|
|
<para>The first step towards editing the documentation is to checkout a |
|
|
copy of code, docs, and build scripts from the CVS server using:</para> |
|
|
|
|
|
<screen> |
|
|
$ export CVS_RSH=ssh |
|
|
$ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack' |
|
|
$ mkdir scratch |
|
|
$ cvs co MITgcm manual mitgcm.org |
|
|
</screen> |
|
|
|
|
|
<para>These commands extract the necessary information from the CVS server |
|
|
and create a temporary (called <filename>scratch</filename>) directory for |
|
|
the storage of the HTML and other files that will be created. Please note |
|
|
that you must either create <filename>scratch</filename> as shown or edit |
|
|
the various <filename>Makefile</filename>s and scripts used to create the |
|
|
documentation.</para> |
|
|
</sect2> |
|
|
|
|
|
<sect2> |
|
|
<title>Editing the Documentation</title> |
|
|
|
|
|
<para>The documentation is contained in the <filename>manual</filename> |
|
|
directory in a raw LaTeX format. The main document is |
|
|
<filename>manual.tex</filename> and it uses <command>\input{}</command>s |
|
|
to include the chapters and subsections.</para> |
|
|
|
|
|
<para>Since the same LaTeX source is used to produce PostScript, PDF, and |
|
|
HTML output, care should be taken to follow certain conventions. Two of |
|
|
the most important are the usage of the <command>\filelink{}{}</command> |
|
|
and <command>\varlink{}{}</command> commands. Both of these commands have |
|
|
been defined to simplify the connection between the automatically |
|
|
generated ("code browser") HTML and the HTML version of the manual |
|
|
produced by LaTeX2HTML. They each take two arguments (corresponding to |
|
|
the contents of the two sets of curly braces) which are the text that the |
|
|
author wishes to be "wrapped" within the link, and a specially formatted |
|
|
link thats relative to the <filename>MITgcm</filename> directory within |
|
|
the CVS tree.</para> |
|
|
|
|
|
<para>The result is a command that resembles either</para> |
|
|
|
|
|
<orderedlist> |
|
|
<listitem> |
|
|
<para>a reference to a variable or subroutine name such as |
|
|
<command>\varlink{tRef}{tRef}</command>, or </para> |
|
|
</listitem> |
|
|
|
|
|
<listitem> |
|
|
<para>a reference to a file such as |
|
|
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
|
|
where the absolute path to the file is of the form |
|
|
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
|
|
<para>(please note how the leading "/foo/MITgcm" |
|
|
component of the path is dropped leaving the path |
|
|
<emphasis>relative</emphasis> to the head of the code |
|
|
directory and each directory separator "/" is turned |
|
|
into a "-")</para> |
|
|
</listitem> |
|
|
</orderedlist> |
|
|
|
|
|
|
|
|
|
|
|
</sect2> |
|
|
|
|
|
<sect2> |
|
|
<title>Building the Documentation</title> |
|
|
|
|
|
<para>Given the directory structure of <xref |
|
|
linkend="documentation_getting">, the entire documentation for the web |
|
|
site can be built using:</para> |
|
|
|
|
|
<screen> |
|
|
$ cd mitgcm.org/devel/buildweb |
|
|
$ make All |
|
|
</screen> |
|
|
|
|
|
<para>Which builds the PDF from the LaTeX source, creates the HTML output |
|
|
from the LaTeX source, parses the FORTRAN code base to produce a |
|
|
hyperlinked HTML version of the source, and then determines the |
|
|
cross-linking between the various HTML components.</para> |
|
|
|
|
|
<para>If there are no errors, the result of the build process (which can |
|
|
take 30+ minutes on a P4/2.5Ghz) will be contained within a single |
|
|
directory called <filename>scratch/dev_docs</filename>. This is a freshly |
|
|
built version of the entire on-line users manual. If you have the correct |
|
|
permissions, it can be directly copied to the web server area:</para> |
|
|
|
|
|
<screen> |
|
|
$ mv scratch/dev_docs /u/u0/httpd/html |
|
|
</screen> |
|
|
|
|
|
<para>and the update is complete.</para> |
|
|
|
|
|
</sect2> |
|
|
|
|
|
</sect1> |
|
|
|
|
384 |
<sect1 id="coding"> |
<sect1 id="coding"> |
385 |
<title>Coding for MITgcm</title> |
<title>Coding for MITgcm</title> |
386 |
|
|
394 |
For MITgcm, the process is similar. Typical commands are:</para> |
For MITgcm, the process is similar. Typical commands are:</para> |
395 |
|
|
396 |
<screen> |
<screen> |
397 |
$ genmake -mods=../code |
$ genmake2 -mods=../code |
398 |
$ make depend |
$ make depend |
399 |
$ make |
$ make |
400 |
</screen> |
</screen> |
405 |
<sect3 id="genmake"> |
<sect3 id="genmake"> |
406 |
<title>The <filename>genmake2</> Utility</title> |
<title>The <filename>genmake2</> Utility</title> |
407 |
|
|
408 |
<para><emphasis>Please note that the older <filename>genmake</> is |
<para><emphasis>(Note: the older <filename>genmake</> |
409 |
deprecated and will eventually be replaced by <filename>genmake2</>. |
has been replaced by <filename>genmake2</>)</emphasis></para> |
|
This HOWTO only describes the newer tool.</emphasis></para> |
|
410 |
|
|
411 |
<para>The first step in any MITgcm build is to create a Unix-style |
<para>The first step in any MITgcm build is to create a Unix-style |
412 |
<filename>Makefile</filename> which will be parsed by |
<filename>Makefile</filename> which will be parsed by |
462 |
<para>an "options file" as specified by the command-line option |
<para>an "options file" as specified by the command-line option |
463 |
<filename>-optfile='FILENAME'</filename></para> |
<filename>-optfile='FILENAME'</filename></para> |
464 |
</listitem> |
</listitem> |
465 |
|
<listitem> |
466 |
|
<para> a <filename>packages.conf</filename> file (in the current |
467 |
|
directory or in one of the "MODS" directories, see below) |
468 |
|
which contains the specific list of packages to compile |
469 |
|
</para> |
470 |
|
</listitem> |
471 |
</orderedlist> |
</orderedlist> |
472 |
|
|
473 |
<para>then checking certain dependency rules (the package dependencies), |
<para>then checking certain dependency rules (the package dependencies), |
521 |
</varlistentry> |
</varlistentry> |
522 |
|
|
523 |
<varlistentry> |
<varlistentry> |
|
<term><filename>-pdepend=/PATH/FILENAME</></term> |
|
|
|
|
|
<listitem> |
|
|
<para>This specifies the dependency file used for packages. If |
|
|
not specified, the default dependency file is |
|
|
<filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for this file is |
|
|
parsed on a line-by-line basis where each line containes either a |
|
|
comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule |
|
|
where the "+" or "-" symbol specifies a "must be used with" or a |
|
|
"must not be used with" relationship, respectively. If no rule is |
|
|
specified, then it is assumed that the two packages are compatible |
|
|
and will function either with or without each other.</para> |
|
|
</listitem> |
|
|
</varlistentry> |
|
|
|
|
|
<varlistentry> |
|
|
<term><filename>-pdefault=PKG</></term> |
|
|
<term><filename>-pdefault='PKG1 [PKG2 PKG3 ...]'</></term> |
|
|
<listitem> |
|
|
<para>This option specifies the default set of packages |
|
|
to be used. If not set, the default package list will |
|
|
be read from |
|
|
<filename>$ROOTDIR/pkg/pkg_default</>.</para> |
|
|
</listitem> |
|
|
</varlistentry> |
|
|
|
|
|
<varlistentry> |
|
524 |
<term><filename>-adof=/path/to/file</></term> |
<term><filename>-adof=/path/to/file</></term> |
525 |
<term><filename>-adoptfile=/path/to/file</></term> |
<term><filename>-adoptfile=/path/to/file</></term> |
526 |
<listitem> |
<listitem> |
548 |
will be overridden by any identically-named sources within the |
will be overridden by any identically-named sources within the |
549 |
"MODS" directories. The order of precedence for this |
"MODS" directories. The order of precedence for this |
550 |
"name-hiding" is as follows:</para> |
"name-hiding" is as follows:</para> |
|
|
|
551 |
<itemizedlist> |
<itemizedlist> |
552 |
<listitem><para>"MODS" directories (in the order given) |
<listitem><para>"MODS" directories (in the order given) |
553 |
</para></listitem> |
</para></listitem> |
559 |
<listitem><para>The "standard dirs" (which may have been |
<listitem><para>The "standard dirs" (which may have been |
560 |
specified by the "-standarddirs" option)</para></listitem> |
specified by the "-standarddirs" option)</para></listitem> |
561 |
</itemizedlist> |
</itemizedlist> |
562 |
|
</listitem> |
563 |
|
</varlistentry> |
564 |
|
|
565 |
|
<varlistentry> |
566 |
|
<term><filename>-pgroups=/PATH/FILENAME</></term> |
567 |
|
<listitem> |
568 |
|
<para>This option specifies the file where package groups are |
569 |
|
defined. If not set, the package-groups definition will |
570 |
|
be read from |
571 |
|
<filename>$ROOTDIR/pkg/pkg_groups</>.</para> |
572 |
|
<para> |
573 |
|
It also contains the default list of packages (defined |
574 |
|
as the group <filename>"default_pkg_list"</>) which is used |
575 |
|
when no specific package list (file: <filename>packages.conf</>) |
576 |
|
is found in current directory or in any "MODS" directory. |
577 |
|
</para> |
578 |
|
</listitem> |
579 |
|
</varlistentry> |
580 |
|
|
581 |
|
<varlistentry> |
582 |
|
<term><filename>-pdepend=/PATH/FILENAME</></term> |
583 |
|
|
584 |
|
<listitem> |
585 |
|
<para>This specifies the dependency file used for packages. If |
586 |
|
not specified, the default dependency file is |
587 |
|
<filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for this file is |
588 |
|
parsed on a line-by-line basis where each line containes either a |
589 |
|
comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule |
590 |
|
where the "+" or "-" symbol specifies a "must be used with" or a |
591 |
|
"must not be used with" relationship, respectively. If no rule is |
592 |
|
specified, then it is assumed that the two packages are compatible |
593 |
|
and will function either with or without each other.</para> |
594 |
</listitem> |
</listitem> |
595 |
</varlistentry> |
</varlistentry> |
596 |
|
|
616 |
<para>In general, it is best to use <filename>genmake2</> on a "clean" |
<para>In general, it is best to use <filename>genmake2</> on a "clean" |
617 |
directory that is free of all source (*.[F,f],*.[F,f]90) and header |
directory that is free of all source (*.[F,f],*.[F,f]90) and header |
618 |
(*.h,*.inc) files. Generally, this can be accomplished in an |
(*.h,*.inc) files. Generally, this can be accomplished in an |
619 |
"un-clean" directory by running "make CLEAN" followed by "make |
"un-clean" directory by running "make Clean" followed by "make |
620 |
makefile".</para> |
makefile".</para> |
621 |
|
|
622 |
</sect3> |
</sect3> |
629 |
simulator) executable using:</para> |
simulator) executable using:</para> |
630 |
|
|
631 |
<screen> |
<screen> |
632 |
$ make CLEAN |
$ make Clean |
633 |
$ make depend |
$ make depend |
634 |
$ make |
$ make |
635 |
</screen> |
</screen> |
636 |
|
|
637 |
<para>The "make CLEAN" step will remove any stale source files, include |
<para>The "make Clean" step will remove any stale source files, include |
638 |
files, and links. It is strongly recommended for "un-clean" |
files, and links. It is strongly recommended for "un-clean" |
639 |
directories which may contain the (perhaps partial) results of |
directories which may contain the (perhaps partial) results of |
640 |
previous builds. Such "debris" can interfere with the next stage of |
previous builds. Such "debris" can interfere with the next stage of |
641 |
the build.</para> |
the build. |
642 |
|
A more agressive cleaning option, "make CLEAN", can be used to also |
643 |
|
remove the executable and output files from a previous run.</para> |
644 |
|
|
645 |
<para>The "make depend" step will create a large number of symbolic |
<para>The "make depend" step will create a large number of symbolic |
646 |
links from the local directory to the source file locations. It also |
links from the local directory to the source file locations. It also |
699 |
<title>The Verification Suite</title> |
<title>The Verification Suite</title> |
700 |
|
|
701 |
<para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</> |
<para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</> |
702 |
directory) includes more than a dozen examples intended for regression |
directory) includes many (> 90) examples intended for regression |
703 |
testing. Each one of these example directories contains "known-good" |
testing. Each one of these example directories contains "known-good" |
704 |
output files along with all the input (including both code and data |
output files along with all the input (including both code and data |
705 |
files) required for their re-calculation. These example directories are |
files) required for their re-calculation. These example directories are |
711 |
<title>The <filename>testreport</> Utility</title> |
<title>The <filename>testreport</> Utility</title> |
712 |
|
|
713 |
<para>Also included in <filename>$ROOTDIR/verification/</> are shell |
<para>Also included in <filename>$ROOTDIR/verification/</> are shell |
714 |
scripts for automated testing. The newest script (which was written |
scripts for automated testing. The script (which was written |
715 |
to work with <filename>genmake2</>) is called <filename>testreport</>. |
to work with <filename>genmake2</>) is called <filename>testreport</>. |
716 |
This script can be used to build different versions of the MITgcm |
This script can be used to build different versions of the MITgcm |
717 |
code, run the various examples, compare the output, and (if specified) |
code, run the various examples, compare the output, and (if specified) |
723 |
|
|
724 |
<screen> |
<screen> |
725 |
$ cd verification |
$ cd verification |
726 |
$ ./testreport -ieee |
$ ./testreport |
727 |
</screen> |
</screen> |
728 |
|
|
729 |
<para>However, some systems (those lacking or wiht a broken "/bin/sh") |
<para>However, some systems (those lacking or wiht a broken "/bin/sh") |
730 |
may require an explicit shell invocation such as:</para> |
may require an explicit shell invocation such as:</para> |
731 |
|
|
732 |
<screen> |
<screen> |
733 |
$ sh ./testreport -ieee -t 'exp0 exp4' |
$ sh ./testreport -t 'exp2 exp4' |
734 |
$ /some/path/to/bash ./testreport -ieee -t 'ideal_2D_oce lab_sea natl_box' |
$ /some/path/to/bash ./testreport -t 'ideal_2D_oce lab_sea natl_box' |
735 |
</screen> |
</screen> |
736 |
|
|
737 |
<para>The <filename>testreport</> script accepts a number of |
<para>The <filename>testreport</> script accepts a number of |
741 |
<variablelist> |
<variablelist> |
742 |
|
|
743 |
<varlistentry> |
<varlistentry> |
744 |
<term><filename>-ieee</></term> |
<term><filename>-ieee</> (default) / <filename>-noieee</></term> |
745 |
<listitem> |
<listitem> |
746 |
<para>If allowed by the compiler (as defined in the "optfile"), |
<para>If allowed by the compiler (as defined in the "optfile"), |
747 |
use IEEE arithmetic. This option, along with the GCC compiler, |
use IEEE arithmetic (<filename>genmake2 -ieee</>). |
748 |
is how the standard results were produced.</para> |
This option, along with the gfortran / gcc compiler, |
749 |
|
is how the standard results are produced.</para> |
750 |
|
</listitem> |
751 |
|
</varlistentry> |
752 |
|
|
753 |
|
<varlistentry> |
754 |
|
<term><filename>-optfile=/PATH/FILENAME</></term> |
755 |
|
<term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term> |
756 |
|
<listitem> |
757 |
|
<para>This specifies a list of "options files" that will be passed |
758 |
|
to <filename>genmake2</>. If multiple options files are used |
759 |
|
(say, to test different compilers or different sets of options |
760 |
|
for the same compiler), then each options file will be used with |
761 |
|
each of the test directories.</para> |
762 |
</listitem> |
</listitem> |
763 |
</varlistentry> |
</varlistentry> |
764 |
|
|
769 |
<para>This option specifies the test directory or list of test |
<para>This option specifies the test directory or list of test |
770 |
directories that should be used. Each of these entries should |
directories that should be used. Each of these entries should |
771 |
exactly (note: they are case sensitive!) match the names of |
exactly (note: they are case sensitive!) match the names of |
772 |
directries in <filename>$ROOTDIR/verification/</>. If this |
directories in <filename>$ROOTDIR/verification/</>. If this |
773 |
option is omitted, then all directories that are properly |
option is omitted, then all directories that are properly |
774 |
formatted (that is, containing an <filename>input</> |
formatted (that is, containing an <filename>input</> |
775 |
sub-directory and a <filename>results/output.txt</> file) will |
sub-directory and a <filename>results/output.txt</> file) will |
778 |
</varlistentry> |
</varlistentry> |
779 |
|
|
780 |
<varlistentry> |
<varlistentry> |
|
<term><filename>-optfile=/PATH/FILENAME</></term> |
|
|
<term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term> |
|
|
<listitem> |
|
|
<para>This specifies a list of "options files" that will be passed |
|
|
to <filename>genmake2</>. If multiple options files are used |
|
|
(say, to test different compilers or different sets of options |
|
|
for the same compiler), then each options file will be used with |
|
|
each of the test directories.</para> |
|
|
</listitem> |
|
|
</varlistentry> |
|
|
|
|
|
<varlistentry> |
|
781 |
<term><filename>-addr EMAIL</></term> |
<term><filename>-addr EMAIL</></term> |
782 |
<term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term> |
<term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term> |
783 |
<listitem> |
<listitem> |
791 |
</varlistentry> |
</varlistentry> |
792 |
|
|
793 |
<varlistentry> |
<varlistentry> |
794 |
|
<term><filename>-MPI NUMBER_OF_PROCS</></term> |
795 |
<term><filename>-mpi</></term> |
<term><filename>-mpi</></term> |
796 |
<listitem> |
<listitem> |
797 |
<para>If the necessary files |
<para>If the necessary file (<filename>TESTDIR/code/SIZE.h_mpi</>) |
798 |
(<filename>TESTDIR/code/CPP_EEOPTIONS.h_mpi</> and |
exists, then use it (and all <filename>TESTDIR/code/*_mpi</> files) |
799 |
<filename>TESTDIR/code/SIZE.h_mpi</>) exist, then use them for an |
for an MPI--enabled run. The new option (<filename>-MPI</> followed |
800 |
MPI--enabled run. Note that the use of MPI typically requires a |
by the maximum number of processors) enable to build and run each |
801 |
|
test-experiment using variable number of MPI processors (multiple |
802 |
|
of <filename>nPx*nPy</> from <filename>TESTDIR/code/SIZE.h_mpi</> |
803 |
|
and not larger than <filename>NUMBER_OF_PROCS</>). |
804 |
|
The short option ("-mpi") can only be used to build and run on 2 MPI |
805 |
|
processors (equivalent to "<filename>-MPI 2</>").</para> |
806 |
|
<para>Note that the use of MPI typically requires a |
807 |
special command option (see "-command" below) to invoke the MPI |
special command option (see "-command" below) to invoke the MPI |
808 |
executable. Examples of PBS scripts using MPI with testreport can be |
executable. Examples of PBS scripts using testreport with MPI can be |
809 |
found in the <ulink |
found in the <ulink |
810 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/"> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/"> |
811 |
MITgcm-contrib area</ulink></para> |
tools/example_scripts directory</ulink>.</para> |
812 |
</listitem> |
</listitem> |
813 |
</varlistentry> |
</varlistentry> |
814 |
|
|
815 |
<varlistentry> |
<varlistentry> |
816 |
<term><filename>-command='some command to run'</></term> |
<term><filename>-command='some command to run'</></term> |
817 |
<listitem> |
<listitem> |
818 |
<para>For some tests, particularly MPI runs, the default "make |
<para>For some tests, particularly MPI runs, a specific command |
819 |
output.txt" is not sufficient. This option allows a more general |
might be needed to run the executable. This option allows a more general |
820 |
command (or shell script) to be invoked. Examples of PBS scripts |
command (or shell script) to be invoked. Examples of PBS scripts |
821 |
using MPI with testreport can be found in the <ulink |
using testreport with MPI can be found in the <ulink |
822 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/"> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/"> |
823 |
MITgcm-contrib area</ulink></para> |
tools/example_scripts directory</ulink>.</para> |
824 |
</listitem> |
</listitem> |
825 |
</varlistentry> |
</varlistentry> |
826 |
|
|
846 |
the generic fluid dynamical engine.</para> |
the generic fluid dynamical engine.</para> |
847 |
|
|
848 |
<para>The MITgcmUV packaging structure is described below using generic |
<para>The MITgcmUV packaging structure is described below using generic |
849 |
package names ${pkg}. A concrete examples of a package is the code for |
package names ${pkg}. A concrete examples of a package is the code for |
850 |
implementing GM/Redi mixing. This code uses the package name</para> |
implementing GM/Redi mixing. This code uses the package name</para> |
851 |
|
|
852 |
</sect2> |
</sect2> |
906 |
Package runtime config. options are imported |
Package runtime config. options are imported |
907 |
into a common block held in a header file |
into a common block held in a header file |
908 |
called "${PKG}.h". |
called "${PKG}.h". |
909 |
|
Note: In some packages, the header file "${PKG}.h" is splitted |
910 |
|
into "${PKG}_PARAMS.h" that contains the package parameters and |
911 |
|
${PKG}_VARS.h" for the field arrays. |
912 |
|
|
913 |
o The core driver part of the model can check |
o The core driver part of the model can check |
914 |
for runtime enabling or disabling of individual packages |
for runtime enabling or disabling of individual packages |
931 |
1. Within the core driver code flags of the form |
1. Within the core driver code flags of the form |
932 |
ALLOW_${PKG} are used to include or exclude |
ALLOW_${PKG} are used to include or exclude |
933 |
whole packages. The ALLOW_${PKG} flags are included |
whole packages. The ALLOW_${PKG} flags are included |
934 |
from a PKG_CPP_OPTIONS block which is currently |
from a PACKAGES_CONFIG.h file that is automatically |
935 |
|
generated by genmake2 (see genmake2 section). |
936 |
held in-line in the CPP_OPTIONS.h header file. |
held in-line in the CPP_OPTIONS.h header file. |
937 |
e.g. |
e.g. |
938 |
|
|
939 |
Core model code ..... |
Core model code ..... |
940 |
|
|
941 |
|
#include "PACKAGES_CONFIG.h" |
942 |
#include "CPP_OPTIONS.h" |
#include "CPP_OPTIONS.h" |
943 |
: |
: |
944 |
: |
: |
950 |
|
|
951 |
2. Within an individual package a header file, |
2. Within an individual package a header file, |
952 |
"${PKG}_OPTIONS.h", is used to set CPP flags |
"${PKG}_OPTIONS.h", is used to set CPP flags |
953 |
specific to that package. It is not recommended |
specific to that package. It also includes |
954 |
to include this file in "CPP_OPTIONS.h". |
"PACKAGES_CONFIG.h" and "CPP_OPTIONS.h". |
955 |
|
|
956 |
|
|
957 |
Package Boot Sequence |
Package Boot Sequence |
1003 |
|
|
1004 |
#ifdef ALLOW_${PKG} |
#ifdef ALLOW_${PKG} |
1005 |
if ( use${Pkg} ) |
if ( use${Pkg} ) |
1006 |
& CALL ${PKG}_DIAGS( ) |
& CALL ${PKG}_OUTPUT( ) |
1007 |
#endif |
#endif |
1008 |
|
|
1009 |
7. S/R PACKAGES_WRITE_PICKUP() |
7. S/R PACKAGES_WRITE_PICKUP() |
1019 |
- ${PKG}_READPARMS() |
- ${PKG}_READPARMS() |
1020 |
is responsible for reading |
is responsible for reading |
1021 |
in the package parameters file data.${pkg}, and storing |
in the package parameters file data.${pkg}, and storing |
1022 |
the package parameters in "${PKG}.h". |
the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h"). |
1023 |
-> called from INITIALISE_FIXED in PACKAGES_READPARMS |
-> called from INITIALISE_FIXED in PACKAGES_READPARMS |
1024 |
|
|
1025 |
- ${PKG}_INIT_FIXED() |
- ${PKG}_INIT_FIXED() |
1049 |
use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old |
use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old |
1050 |
form ${PKG}_INIT |
form ${PKG}_INIT |
1051 |
|
|
1052 |
- ${PKG}_DIAGS() |
- ${PKG}_OUTPUT( ) |
1053 |
is responsible for writing time-average diagnostics to output |
is responsible for writing time-average fields to output files |
1054 |
files (but the cumulating step is done within the package main S/R). |
(but the cumulating step is done within the package main S/R). |
1055 |
Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR) |
Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR) |
1056 |
and write snap-shot fields that are hold in common blocks. Other |
and write snap-shot fields that are hold in common blocks. Other |
1057 |
temporary fields are directly dump to file where they are available. |
temporary fields are directly dump to file where they are available. |
1058 |
NOTE: this part does not yet have a standard form and should be called |
NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages |
1059 |
from a package dedicated S/R such as PACKAGE_WRITE_DIAGS |
but is beeing replaced by ${PKG}_OUTPUT |
1060 |
|
to avoid confusion with pkg/diagnostics functionality. |
1061 |
|
2) the output part is not yet in a standard form and might still |
1062 |
|
evolve a lot. |
1063 |
-> called within DO_THE_MODEL_IO |
-> called within DO_THE_MODEL_IO |
1064 |
|
|
1065 |
- ${PKG}_WRITE_PICKUP() |
- ${PKG}_WRITE_PICKUP() |
1086 |
----------------------- |
----------------------- |
1087 |
* ${PKG}_OPTIONS.h has further package-specific CPP options |
* ${PKG}_OPTIONS.h has further package-specific CPP options |
1088 |
* ${PKG}.h package-specific common block variables, fields |
* ${PKG}.h package-specific common block variables, fields |
1089 |
|
or ${PKG}_PARAMS.h package-specific common block parameters |
1090 |
|
and ${PKG}_VARS.h package-specific common block fields |
1091 |
|
|
1092 |
- FORTRAN source files |
- FORTRAN source files |
1093 |
----------------------- |
----------------------- |
1096 |
* ${pkg}_check.F checks package dependencies and consistencies |
* ${pkg}_check.F checks package dependencies and consistencies |
1097 |
* ${pkg}_init_varia.F initialises package-related fields |
* ${pkg}_init_varia.F initialises package-related fields |
1098 |
* ${pkg}_... .F package source code |
* ${pkg}_... .F package source code |
1099 |
* ${pkg}_diags.F write diagnostics to file. |
* ${pkg}_output.F write output to file. |
1100 |
* ${pkg}_write_pickup.F write a package pickup file to restart the model |
* ${pkg}_write_pickup.F write a package pickup file to restart the model |
1101 |
|
|
1102 |
|
New: Subroutine in one package (pkgA) that only contains code which |
1103 |
|
is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F) |
1104 |
|
will be named: pkgA_pkgB_something.F |
1105 |
|
|
1106 |
- parameter file |
- parameter file |
1107 |
----------------------- |
----------------------- |
1108 |
* data.${pkg} parameter file |
* data.${pkg} parameter file |
1111 |
</sect1> |
</sect1> |
1112 |
|
|
1113 |
|
|
1114 |
|
<sect1 id="documentation"> |
1115 |
|
<title>Editing the Documentation</title> |
1116 |
|
|
1117 |
|
<sect2 id="documentation_getting"> |
1118 |
|
<title>Getting the Docs and Code</title> |
1119 |
|
|
1120 |
|
<para>The first step towards editing the documentation is to checkout a |
1121 |
|
copy of code, docs, and build scripts from the CVS server using:</para> |
1122 |
|
|
1123 |
|
<screen> |
1124 |
|
$ export CVS_RSH=ssh |
1125 |
|
$ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack' |
1126 |
|
$ mkdir scratch |
1127 |
|
$ cvs co -P MITgcm manual mitgcm.org |
1128 |
|
</screen> |
1129 |
|
|
1130 |
|
<para>These commands extract the necessary information from the CVS server |
1131 |
|
and create a temporary (called <filename>scratch</filename>) directory for |
1132 |
|
the storage of the HTML and other files that will be created. Please note |
1133 |
|
that you must either create <filename>scratch</filename> as shown or edit |
1134 |
|
the various <filename>Makefile</filename>s and scripts used to create the |
1135 |
|
documentation.</para> |
1136 |
|
</sect2> |
1137 |
|
|
1138 |
|
<sect2> |
1139 |
|
<title>Editing the Documentation</title> |
1140 |
|
|
1141 |
|
<para>The documentation is contained in the <filename>manual</filename> |
1142 |
|
directory in a raw LaTeX format. The main document is |
1143 |
|
<filename>manual.tex</filename> and it uses <command>\input{}</command>s |
1144 |
|
to include the chapters and subsections.</para> |
1145 |
|
|
1146 |
|
<para>Since the same LaTeX source is used to produce PostScript, PDF, and |
1147 |
|
HTML output, care should be taken to follow certain conventions. Two of |
1148 |
|
the most important are the usage of the <command>\filelink{}{}</command> |
1149 |
|
and <command>\varlink{}{}</command> commands. Both of these commands have |
1150 |
|
been defined to simplify the connection between the automatically |
1151 |
|
generated ("code browser") HTML and the HTML version of the manual |
1152 |
|
produced by LaTeX2HTML. They each take two arguments (corresponding to |
1153 |
|
the contents of the two sets of curly braces) which are the text that the |
1154 |
|
author wishes to be "wrapped" within the link, and a specially formatted |
1155 |
|
link thats relative to the <filename>MITgcm</filename> directory within |
1156 |
|
the CVS tree.</para> |
1157 |
|
|
1158 |
|
<para>The result is a command that resembles either</para> |
1159 |
|
|
1160 |
|
<orderedlist> |
1161 |
|
<listitem> |
1162 |
|
<para>a reference to a variable or subroutine name such as |
1163 |
|
<command>\varlink{tRef}{tRef}</command>, or </para> |
1164 |
|
</listitem> |
1165 |
|
|
1166 |
|
<listitem> |
1167 |
|
<para>a reference to a file such as |
1168 |
|
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
1169 |
|
where the absolute path to the file is of the form |
1170 |
|
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
1171 |
|
<para>(please note how the leading "/foo/MITgcm" |
1172 |
|
component of the path is dropped leaving the path |
1173 |
|
<emphasis>relative</emphasis> to the head of the code |
1174 |
|
directory and each directory separator "/" is turned |
1175 |
|
into a "-")</para> |
1176 |
|
</listitem> |
1177 |
|
</orderedlist> |
1178 |
|
|
1179 |
|
|
1180 |
|
|
1181 |
|
</sect2> |
1182 |
|
|
1183 |
|
<sect2> |
1184 |
|
<title>Building the Documentation</title> |
1185 |
|
|
1186 |
|
<para>Given the directory structure of <xref |
1187 |
|
linkend="documentation_getting">, the entire documentation for the web |
1188 |
|
site can be built using:</para> |
1189 |
|
|
1190 |
|
<screen> |
1191 |
|
$ cd mitgcm.org/devel/buildweb |
1192 |
|
$ make All |
1193 |
|
</screen> |
1194 |
|
|
1195 |
|
<para>Which builds the PDF from the LaTeX source, creates the HTML output |
1196 |
|
from the LaTeX source, parses the FORTRAN code base to produce a |
1197 |
|
hyperlinked HTML version of the source, and then determines the |
1198 |
|
cross-linking between the various HTML components.</para> |
1199 |
|
|
1200 |
|
<para>If there are no errors, the result of the build process (which can |
1201 |
|
take 30+ minutes on a P4/2.5Ghz) will be contained within a single |
1202 |
|
directory called <filename>scratch/dev_docs</filename>. This is a freshly |
1203 |
|
built version of the entire on-line users manual. If you have the correct |
1204 |
|
permissions, it can be directly copied to the web server area:</para> |
1205 |
|
|
1206 |
|
<screen> |
1207 |
|
$ mv scratch/dev_docs /u/u0/httpd/html |
1208 |
|
</screen> |
1209 |
|
|
1210 |
|
<para>and the update is complete.</para> |
1211 |
|
|
1212 |
|
</sect2> |
1213 |
|
|
1214 |
|
</sect1> |
1215 |
|
|
1216 |
</article> |
</article> |
1217 |
|
|
1218 |
|
|