| 1 |
<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN"> |
<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN"> |
|
|
|
|
<article id="MITgcm-Development-HOWTO"> |
|
|
|
|
| 2 |
<!-- |
<!-- |
| 3 |
Build commands: |
$Header$ |
| 4 |
db2ps -d ldp.dsl devel_HOWTO.sgml |
$Name$ |
|
db2pdf -d ldp.dsl devel_HOWTO.sgml |
|
|
db2html -d ./ldp.dsl devel_HOWTO.sgml |
|
|
db2html -u -d ./ldp.dsl devel_HOWTO.sgml |
|
| 5 |
--> |
--> |
| 6 |
|
|
| 7 |
|
<article id="MITgcm-Development-HOWTO"> |
| 8 |
|
|
| 9 |
<articleinfo> |
<articleinfo> |
| 10 |
<title>MITgcm Development HOWTO</title> |
<title>MITgcm Development HOWTO</title> |
| 11 |
|
|
| 26 |
Initial version. |
Initial version. |
| 27 |
</revremark> |
</revremark> |
| 28 |
</revision> |
</revision> |
| 29 |
|
<revision> |
| 30 |
|
<revnumber>0.02</revnumber> |
| 31 |
|
<date>2010-01-21</date> |
| 32 |
|
<authorinitials>jmc</authorinitials> |
| 33 |
|
<revremark> |
| 34 |
|
Update links. |
| 35 |
|
</revremark> |
| 36 |
|
</revision> |
| 37 |
|
<revision> |
| 38 |
|
<revnumber>0.03</revnumber> |
| 39 |
|
<date>2010-04-25</date> |
| 40 |
|
<authorinitials>jmc</authorinitials> |
| 41 |
|
<revremark> |
| 42 |
|
Add subsection "Developer settings" (under CVS Repository). |
| 43 |
|
</revremark> |
| 44 |
|
</revision> |
| 45 |
|
<revision> |
| 46 |
|
<revnumber>0.04</revnumber> |
| 47 |
|
<date>2011-04-24</date> |
| 48 |
|
<authorinitials>jmc</authorinitials> |
| 49 |
|
<revremark> |
| 50 |
|
Update subsection "The verification suite". |
| 51 |
|
</revremark> |
| 52 |
|
</revision> |
| 53 |
</revhistory> |
</revhistory> |
| 54 |
|
|
| 55 |
<abstract> |
<abstract> |
| 65 |
<sect2> |
<sect2> |
| 66 |
<title>New Versions of This Document</title> <para>You can |
<title>New Versions of This Document</title> <para>You can |
| 67 |
obtain the latest version of this document <ulink |
obtain the latest version of this document <ulink |
| 68 |
url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in |
url="http://mitgcm.org/public/docs.html">online</ulink> in |
| 69 |
various formats.</para> |
various formats.</para> |
| 70 |
</sect2> |
</sect2> |
| 71 |
<sect2> |
<sect2> |
| 83 |
<sect2> |
<sect2> |
| 84 |
<title>User Manual</title> |
<title>User Manual</title> |
| 85 |
|
|
| 86 |
<para>Before jumping into |
<para>Before jumping into development, please familiarize yourself with |
| 87 |
development, please familiarize yourself with the MITgcm user |
the <ulink url="http://mitgcm.org/public/docs.html"> MITgcm user manual |
| 88 |
manual which is available <ulink |
</ulink>. This document contains volumes of useful information and is |
| 89 |
url="http://mitgcm.org/">on the main web page</ulink>. This |
included here by reference.</para> |
|
document contains volumes of useful information and is included |
|
|
here by reference.</para> |
|
| 90 |
|
|
| 91 |
<para>Also, a "snapshot" or<ulink |
<!-- |
| 92 |
|
<para>Also, a "snapshot" or <ulink |
| 93 |
url="http://mitgcm.org/dev_docs/">development version</ulink> of |
url="http://mitgcm.org/dev_docs/">development version</ulink> of |
| 94 |
the user manual may be available, though this is only put on the |
the user manual may be available, though this is only put on the |
| 95 |
web for testing purposes.</para> |
web for testing purposes.</para> |
| 96 |
|
--> |
| 97 |
</sect2> |
</sect2> |
| 98 |
|
|
| 99 |
<sect2> |
<sect2> |
| 122 |
|
|
| 123 |
<sect1 id="cvs"> |
<sect1 id="cvs"> |
| 124 |
<title>CVS Repository</title> |
<title>CVS Repository</title> |
| 125 |
|
|
| 126 |
<sect2> |
<sect2> |
| 127 |
<title>Layout</title> |
<title>Layout</title> |
| 128 |
|
|
| 133 |
others. The tree currently resembles:</para> |
others. The tree currently resembles:</para> |
| 134 |
|
|
| 135 |
<programlisting>gcmpack/ |
<programlisting>gcmpack/ |
| 136 |
MITgcm-contrib contributed code |
CVSROOT -hidden- |
|
CS-regrid goes into utils |
|
|
cvspolicy.html -save- |
|
|
CVSROOT -save- |
|
|
development experimental stuff |
|
|
manual -save- |
|
|
misc -?- |
|
| 137 |
|
|
| 138 |
MITgcm code |
MITgcm code |
| 139 |
adjoint fold into genmake |
bin empty |
| 140 |
bin stub for ecco build |
doc basic developpment documentation |
| 141 |
compare01 old from 20th century |
eesupp execution environment support code (wrapper) |
| 142 |
diags timeave f77 in pkgs now |
exe empty |
| 143 |
doc tags -- connect to real docs? |
jobs runtime shell scripts for |
| 144 |
eesupp cnh? |
various platforms (not maintained) |
| 145 |
exe ecco user build |
lsopt line search |
| 146 |
*- jobs runtime shell scripts for |
model main dynamics (core) |
| 147 |
| various platforms |
optim line search interface |
| 148 |
| lsopt line search |
pkg alternate and optional numerics, etc. |
| 149 |
m| model main dynamics (core) |
tools scripts to build (and test) |
| 150 |
e| optimization_drivers ? |
utils pre/post processing tools (matlab, ..) |
| 151 |
r| optim line search interface |
verification standard regression tests + examples |
| 152 |
g| pkg alternate and optional numerics, etc. |
+ documented examples (tutorials) |
| 153 |
e*- tools |
tutorial_examples (only in release1 branch) |
|
?| tutorial_examples documented tests |
|
|
| only populated on release1 branch |
|
|
| and not validated during "testscript" |
|
|
*- utils |
|
|
verification std tests |
|
| 154 |
|
|
| 155 |
|
MITgcm_contrib contributed code |
| 156 |
|
|
| 157 |
mitgcmdoc -> manual -remove- |
acesgrid.org build acesgrid web site |
| 158 |
|
development experimental stuff |
| 159 |
|
gfd_lab -?- |
| 160 |
|
manual source of MITgcm documentation |
| 161 |
mitgcm.org build web site |
mitgcm.org build web site |
| 162 |
models -?- |
old_develop old and early development source |
| 163 |
packages -?- |
misc -?- |
| 164 |
preprocess -?- |
models -?- |
| 165 |
tmp -?- |
packages -?- |
| 166 |
|
preprocess -?- |
| 167 |
|
pdfs some pdfs |
| 168 |
|
planetinabottle.org unfinished web site |
| 169 |
|
www.ecco-group.org build ecco web site ? |
| 170 |
</programlisting> |
</programlisting> |
| 171 |
|
|
| 172 |
|
<!-- |
| 173 |
<para>Efforts are underway to reduce the complexity.</para> |
<para>Efforts are underway to reduce the complexity.</para> |
| 174 |
|
--> |
| 175 |
|
|
| 176 |
</sect2> |
</sect2> |
| 177 |
|
|
| 210 |
<title>Branches</title> |
<title>Branches</title> |
| 211 |
|
|
| 212 |
<para>As shown in the online <ulink |
<para>As shown in the online <ulink |
| 213 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph"> |
| 214 |
tree</ulink>, the MITgcm codebase is split into to two branches |
ViewCVS-generated tree</ulink>, the MITgcm codebase is split into |
| 215 |
or "lines" under which development proceeds. These two lines |
branches or "lines" under which development proceeds. The main line |
| 216 |
are referred to as the "MAIN" and "ecco" versions of the code. |
of development is referred to as the "MAIN" version of the code. |
|
While not identical, the bulk of the MAIN and ecco lines are |
|
|
composed of files from the same codebase. |
|
| 217 |
</para> |
</para> |
| 218 |
|
|
| 219 |
<para>Periodically, a "Release" branch is formed from the "MAIN" |
<para>Periodically, a "Release" branch is formed from the "MAIN" |
| 220 |
development branch. This is done in order to create a |
development branch. This is done in order to create a relatively stable |
| 221 |
relatively stable reference point for both users and developers. |
reference point for both users and developers. The intent is that once a |
| 222 |
The intent is that once a relese branch has been created, only |
release branch has been created, only bug-fixes will be added to it. |
| 223 |
bug-fixes will be added to it. Meanwhile, development (which |
Meanwhile, development (which might "break" or otherwise render invalid |
| 224 |
might "break" or otherwise render invalid the documentation, |
the documentation, tutorials, and/or examples contained within a release |
| 225 |
tutorials, and/or examples contained within a release branch) is |
branch) is allowed to continue along the MAIN line.</para> |
|
allowed to continue along the MAIN and ecco lines.</para> |
|
| 226 |
</sect2> |
</sect2> |
| 227 |
|
|
| 228 |
<sect2> |
<sect2> |
| 229 |
<title>Tagging</title> |
<title> Developer settings </title> |
|
|
|
|
<para>The intent of tagging is to create "known-good" |
|
|
checkpoints that developers can use as references. |
|
|
Traditionally, MITgcm tagging has maintained the following |
|
|
conventions:</para> |
|
| 230 |
|
|
| 231 |
|
<para>CVS is a convenient tool to keep up-to-date a personal copy of the |
| 232 |
|
MITgcm code (see: <ulink url="http://mitgcm.org/public/using_cvs.html"> |
| 233 |
|
using CVS </ulink>). The same tool is used by developers to |
| 234 |
|
incorporate any change into the repository. However, this later |
| 235 |
|
function requires specific settings, as detailed here after</para> |
| 236 |
<orderedlist> |
<orderedlist> |
| 237 |
<listitem> |
<listitem> |
| 238 |
<para>Developer checks out code into a local CVS-managed |
<para> You will need an account (login access) to the server |
| 239 |
directory, makes various changes/additions, tests these |
"mitgcm.org" (curently: <filename>forge.csail.mit.edu</filename>) |
| 240 |
edits, and eventually reaches a point where (s)he is |
with the proper group setting (e.g., group "gcmctrb" to add/modify |
| 241 |
satisfied that the changes form a new "useful" point in the |
code into MITgcm_contrib). |
| 242 |
evolution of the code.</para> |
This kind of account is granted only upon well motivated request |
| 243 |
|
(we recommend to ask your senior MITgcm-collaborator to send such |
| 244 |
|
request to marshall-admin at techsquare.com with Cc to Chris Hill |
| 245 |
|
for approval).</para> |
| 246 |
|
<para> The access to the server <filename>mitgcm.org</filename> is |
| 247 |
|
through ssh-key authorization which will need to be set properly on |
| 248 |
|
both side (on your local machine and on your server account). |
| 249 |
|
You need to be able to ssh to <filename>mitgcm.org</filename> |
| 250 |
|
(or <filename>ssh MY_USER_NAME@mitgcm.org</filename> |
| 251 |
|
in case of different user-name on both sides) to proceed further.</para> |
| 252 |
|
</listitem> |
| 253 |
|
|
| 254 |
|
<listitem> |
| 255 |
|
<para> You need to register to the |
| 256 |
|
<ulink url="http://mitgcm.org/mailman/listinfo/mitgcm-cvs"> |
| 257 |
|
mitgcm-cvs </ulink> mailing list. |
| 258 |
|
This ensures that other developers will receive email notification |
| 259 |
|
when you make changes; you will also receive such email |
| 260 |
|
when others make changes to the repository. |
| 261 |
|
</para> |
| 262 |
|
</listitem> |
| 263 |
|
|
| 264 |
|
<listitem> |
| 265 |
|
<para> It is highly recommended that you register also to the |
| 266 |
|
<ulink url="http://mitgcm.org/mailman/listinfo/mitgcm-devel"> |
| 267 |
|
mitgcm-devel </ulink> mailing list (expect a short delay for |
| 268 |
|
this request to be processed). |
| 269 |
|
This list is intended for developer discussions. |
| 270 |
|
</para> |
| 271 |
|
</listitem> |
| 272 |
|
|
| 273 |
|
<listitem> |
| 274 |
|
<para> The standard CVS-anonymous mode (using "cvsanon", |
| 275 |
|
as mentionned <ulink url="http://mitgcm.org/public/source_code.html"> |
| 276 |
|
here </ulink>) does not provide check-in ("cvs commit") permission. |
| 277 |
|
Instead, you will need to set our CVS environment as follow:</para> |
| 278 |
|
<screen> |
| 279 |
|
$ export CVS_RSH=ssh |
| 280 |
|
$ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack' |
| 281 |
|
</screen> |
| 282 |
|
<para> The reason for such limitation is that when downloading a directory, |
| 283 |
|
e.g.: <filename>myCopy</filename>, from the CVS repository (e.g., |
| 284 |
|
<filename>MITgcm_contrib/thisPart</filename>) using the command:</para> |
| 285 |
|
<screen> |
| 286 |
|
$ cvs co -P -d myCopy MITgcm_contrib/thisPart |
| 287 |
|
</screen> |
| 288 |
|
<para> the type of CVS environment which has been used |
| 289 |
|
is stored in the file <filename>myCopy/CVS/Root</filename>. |
| 290 |
|
This prevent to re-use, for cvs-commit purpose, |
| 291 |
|
a cvs local copy (<filename>myCopy</filename>) which was obtained |
| 292 |
|
using the CVS anonymous mode.</para> |
| 293 |
|
</listitem> |
| 294 |
|
|
| 295 |
|
<listitem> |
| 296 |
|
<para> At this stage, you should be able to send your modified source |
| 297 |
|
file (e.g., <filename>src_file</filename>) from your local copy directory |
| 298 |
|
(<filename>myCopy</filename>) to the CVS repository |
| 299 |
|
(<filename>MITgcm_contrib/thisPart</filename>) using the command |
| 300 |
|
"cvs commit":</para> |
| 301 |
|
<screen> |
| 302 |
|
$ cd myCopy |
| 303 |
|
$ cvs -n update (optional; check if new changes have been made) |
| 304 |
|
$ cvs diff src_file (optional; list your changes) |
| 305 |
|
$ cvs commit src_file |
| 306 |
|
</screen> |
| 307 |
|
<para> It is essential that you provide a short description of the |
| 308 |
|
changes you made to <filename>src_file</filename> as you check-in |
| 309 |
|
this file (the "cvs commit" command automatically opens your standard |
| 310 |
|
editor for this purpose).</para> |
| 311 |
|
<para> Note: |
| 312 |
|
Please ignore the following warnings that the "cvs commit" command |
| 313 |
|
produces if you are not part of the "gcmpack" group: |
| 314 |
|
<screen> |
| 315 |
|
cvs commit: failed to create lock directory for `/u/gcmpack/CVSROOT' |
| 316 |
|
(/u/gcmpack/CVSROOT/#cvs.history.lock): Permission denied |
| 317 |
|
cvs commit: failed to obtain history lock in repository `/u/gcmpack' |
| 318 |
|
</screen> |
| 319 |
|
These warnings are not affecting the changes you made to the CVS repository. |
| 320 |
|
</para> |
| 321 |
|
</listitem> |
| 322 |
|
|
| 323 |
|
</orderedlist> |
| 324 |
|
|
| 325 |
|
</sect2> |
| 326 |
|
|
| 327 |
|
<sect2> |
| 328 |
|
<title>Main code development</title> |
| 329 |
|
<para><emphasis>(formerly named "Tagging" ; this section needs an update) |
| 330 |
|
</emphasis></para> |
| 331 |
|
|
| 332 |
|
<para>The intent of tagging is to create "known-good" checkpoints that |
| 333 |
|
developers can use as references. Traditionally, MITgcm tagging has |
| 334 |
|
maintained the following conventions:</para> |
| 335 |
|
|
| 336 |
|
<orderedlist> |
| 337 |
|
<listitem> |
| 338 |
|
<para>Developer checks out code into a local CVS-managed directory, |
| 339 |
|
makes various changes/additions, tests these edits, and eventually |
| 340 |
|
reaches a point where (s)he is satisfied that the changes form a new |
| 341 |
|
"useful" point in the evolution of the code.</para> |
| 342 |
</listitem> |
</listitem> |
| 343 |
|
|
| 344 |
<listitem> |
<listitem> |
| 345 |
<para>The developer then runs the <ulink |
<para>The developer then runs the <ulink |
| 346 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport"> |
| 347 |
shell script to see if any problems are introduced. While |
testreport</ulink> shell script to see if any problems are introduced. |
| 348 |
not intended to be exhaustive, the test cases within the |
While not intended to be exhaustive, the test cases within the |
| 349 |
verification directory do provide some indication whether |
verification directory do provide some indication whether gross errors |
| 350 |
gross errors have been introduced. |
have been introduced. |
| 351 |
</para> |
</para> |
| 352 |
</listitem> |
</listitem> |
| 353 |
|
|
| 357 |
then:</para> |
then:</para> |
| 358 |
<orderedlist> |
<orderedlist> |
| 359 |
<listitem> |
<listitem> |
| 360 |
<para>adds a "checkpointXY_pre" comment (where X is a |
<para>adds a "checkpointXY_pre" comment (where X is a checkpoint |
| 361 |
checkpoint number and Y is a letter) to the <ulink |
number and Y is a letter) to the <ulink |
| 362 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index"> |
| 363 |
file and checks it into the CVS repository</para> |
tag-index</ulink> file and checks it into the CVS |
| 364 |
|
repository</para> |
| 365 |
</listitem> |
</listitem> |
| 366 |
<listitem> |
<listitem> |
| 367 |
<para>submits the set of changes to the CVS repository |
<para>submits the set of changes to the CVS repository and adds |
| 368 |
and adds comments to <filename>tag-index</filename> |
comments to <filename>tag-index</filename> describing what the |
| 369 |
describing what the changes are along with a matching |
changes are along with a matching "checkpointXY_post" entry</para> |
|
"checkpointXY_post" entry</para> |
|
| 370 |
</listitem> |
</listitem> |
| 371 |
</orderedlist> |
</orderedlist> |
| 372 |
</listitem> |
</listitem> |
| 373 |
</orderedlist> |
</orderedlist> |
| 374 |
|
|
| 375 |
<para>The result of this tagging procedure is a sequence of |
<para>The result of this tagging procedure is a sequence of development |
| 376 |
development checkpoints with comments which resembles:</para> |
checkpoints with comments which resembles:</para> |
| 377 |
|
|
| 378 |
<programlisting> |
<programlisting> |
| 379 |
checkpoint50e_post |
checkpoint50e_post |
| 380 |
o make KPP work with PTRACERS |
o make KPP work with PTRACERS |
| 381 |
- fix gad_calc_rhs to call new routine kpp_transport_ptr, which is |
- fix gad_calc_rhs to call new routine kpp_transport_ptr, which is |
| 382 |
nearly a copy of kpp_transport_s |
nearly a copy of kpp_transport_s |
| 383 |
- there is no analogue to SurfaceTendencyS, so I have to use |
- there is no analogue to SurfaceTendencyS, so I have to use |
| 384 |
gPtr(of the surface layer) instead |
gPtr(of the surface layer) instead |
| 385 |
o add a new platform SunFire+mpi (SunFire 15000) to genmake |
o add a new platform SunFire+mpi (SunFire 15000) to genmake |
| 386 |
checkpoint50e_pre |
checkpoint50e_pre |
| 387 |
|
|
| 388 |
checkpoint50d_post |
checkpoint50d_post |
| 389 |
o change kpp output from multiple-record state files to single-record state |
o change kpp output from multiple-record state files to single-record state |
| 390 |
files analogous to write_state.F |
files analogous to write_state.F |
| 391 |
o reduce the output frequency of cg3d-related stuff to the monitor frequency, |
o reduce the output frequency of cg3d-related stuff to the monitor frequency, |
| 392 |
analogous to the cg2d-related output. |
analogous to the cg2d-related output. |
| 393 |
o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512, |
o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512, |
| 394 |
so that writing to internal file fn (with length 512) fails. |
so that writing to internal file fn (with length 512) fails. |
| 395 |
checkpoint50d_pre |
checkpoint50d_pre |
| 396 |
</programlisting> |
</programlisting> |
| 397 |
|
|
| 398 |
<para>This information can be used to refer to various stages of |
<para>This information can be used to refer to various stages of the code |
| 399 |
the code development. For example, bugs can be traced to |
development. For example, bugs can be traced to individual sets of CVS |
| 400 |
individual sets of CVS checkins based upon their first |
checkins based upon their first appearance when comparing the results from |
| 401 |
appearance when comparing the results from different |
different checkpoints.</para> |
|
checkpoints.</para> |
|
| 402 |
|
|
| 403 |
</sect2> |
</sect2> |
| 404 |
</sect1> |
</sect1> |
| 405 |
|
|
| 406 |
|
|
|
<sect1 id="documentation"> |
|
|
<title>Editing the Documentation</title> |
|
|
|
|
|
<sect2 id="documentation_getting"> |
|
|
<title>Getting the Docs and Code</title> |
|
|
|
|
|
<para>The first step towards editing the documentation is to |
|
|
checkout a copy of code, docs, and build scripts from the CVS |
|
|
server using:</para> |
|
|
|
|
|
<screen> |
|
|
$ export CVS_RSH=ssh |
|
|
$ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack' |
|
|
$ mkdir scratch |
|
|
$ cvs co MITgcm manual mitgcm.org |
|
|
</screen> |
|
|
|
|
|
<para>These commands extract the necessary information from the |
|
|
CVS server and create a temporary (called |
|
|
<filename>scratch</filename>) directory for the storage of the |
|
|
HTML and other files that will be created. Please note that you |
|
|
must either create <filename>scratch</filename> as shown or edit |
|
|
the various <filename>Makefile</filename>s and scripts used to |
|
|
create the documentation.</para> |
|
|
</sect2> |
|
|
|
|
|
<sect2> |
|
|
<title>Editing</title> |
|
|
|
|
|
<para>The documentation is contained in the |
|
|
<filename>manual</filename> directory in a raw LaTeX format. |
|
|
The main document is <filename>manual.tex</filename> and it uses |
|
|
<command>\input{}</command>s to include the chapters and |
|
|
subsections.</para> |
|
|
|
|
|
<para>Since the same LaTeX source is used to produce PostScript, |
|
|
PDF, and HTML output, care should be taken to follow certain |
|
|
conventions. Two of the most important are the usage of the |
|
|
<command>\filelink{}{}</command> and |
|
|
<command>\varlink{}{}</command> commands. Both of these |
|
|
commands have been defined to simplify the connection between |
|
|
the automatically generated ("code browser") HTML and the HTML |
|
|
version of the manual produced by LaTeX2HTML. They each take |
|
|
two arguments (corresponding to the contents of the two sets of |
|
|
curly braces) which are the text that the author wishes to be |
|
|
"wrapped" within the link, and a specially formatted link thats |
|
|
relative to the <filename>MITgcm</filename> directory within the |
|
|
CVS tree.</para> |
|
|
|
|
|
<para>The result is a command that resembles either</para> |
|
|
|
|
|
<orderedlist> |
|
|
<listitem> |
|
|
<para>a reference to a variable or subroutine name such as |
|
|
<command>\varlink{tRef}{tRef}</command>, or </para> |
|
|
</listitem> |
|
|
|
|
|
<listitem> |
|
|
<para>a reference to a file such as |
|
|
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
|
|
where the absolute path to the file is of the form |
|
|
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
|
|
<para>(please note how the leading "/foo/MITgcm" |
|
|
component of the path is dropped leaving the path |
|
|
<emphasis>relative</emphasis> to the head of the code |
|
|
directory and each directory separator "/" is turned |
|
|
into a "-")</para> |
|
|
</listitem> |
|
|
</orderedlist> |
|
|
|
|
|
|
|
|
|
|
|
</sect2> |
|
|
|
|
|
<sect2> |
|
|
<title>Building</title> <para>Given the directory structure of |
|
|
<xref linkend="documentation_getting">, the entire documentation |
|
|
for the web site can be built using:</para> |
|
|
|
|
|
<screen> |
|
|
$ cd mitgcm.org/devel/buildweb |
|
|
$ make All |
|
|
</screen> |
|
|
|
|
|
<para>Which builds the PDF from the LaTeX source, creates the |
|
|
HTML output from the LaTeX source, parses the FORTRAN code base |
|
|
to produce a hyperlinked HTML version of the source, and then |
|
|
determines the cross-linking between the various HTML |
|
|
components.</para> |
|
|
|
|
|
<para>If there are no errors, the result of the build process |
|
|
(which can take 30+ minutes on a P4/2.5Ghz) will be contained |
|
|
within a single directory called |
|
|
<filename>scratch/dev_docs</filename>. This is a freshly built |
|
|
version of the entire on-line users manual. If you have the |
|
|
correct permissions, it can be directly copied to the web server |
|
|
area:</para> |
|
|
|
|
|
<screen> |
|
|
$ mv scratch/dev_docs /u/u0/httpd/html |
|
|
</screen> |
|
|
|
|
|
<para>and the update is complete.</para> |
|
|
|
|
|
</sect2> |
|
|
|
|
|
</sect1> |
|
|
|
|
| 407 |
<sect1 id="coding"> |
<sect1 id="coding"> |
| 408 |
<title>Coding for MITgcm</title> |
<title>Coding for MITgcm</title> |
| 409 |
|
|
| 410 |
<sect2 id="build_tools"> |
<sect2 id="build_tools"> |
| 411 |
<title>Build Tools</title> |
<title>Build Tools</title> |
| 412 |
|
|
| 413 |
<para>Many Open Source projects use the "GNU Autotools" to help |
<para>Many Open Source projects use the "GNU Autotools" to help streamline |
| 414 |
streamline the build process for various Unix and Unix-like |
the build process for various Unix and Unix-like architectures. For a |
| 415 |
architectures. For a user, the result is the common "configure" |
user, the result is the common "configure" (that is, |
| 416 |
(that is, "<filename>./configure && make && make |
"<filename>./configure && make && make install</filename>") commands. |
| 417 |
install</filename>") commands. For MITgcm, the process is |
For MITgcm, the process is similar. Typical commands are:</para> |
|
similar. Typical commands are:</para> |
|
| 418 |
|
|
| 419 |
<screen> |
<screen> |
| 420 |
$ genmake -mods=../code |
$ genmake2 -mods=../code |
| 421 |
$ make depend |
$ make depend |
| 422 |
$ make |
$ make |
| 423 |
</screen> |
</screen> |
| 424 |
|
|
| 425 |
<para>The following sections describe the individual steps in |
<para>The following sections describe the individual steps in the build |
| 426 |
the build process.</para> |
process.</para> |
| 427 |
|
|
| 428 |
<sect3 id="genmake"> |
<sect3 id="genmake"> |
| 429 |
<title>The <filename>genmake2</> Utility</title> |
<title>The <filename>genmake2</> Utility</title> |
| 430 |
|
|
| 431 |
<para><emphasis>Please note that the older |
<para><emphasis>(Note: the older <filename>genmake</> |
| 432 |
<filename>genmake</> is deprecated and will eventually |
has been replaced by <filename>genmake2</>)</emphasis></para> |
| 433 |
be replaced by <filename>genmake2</>. This HOWTO only |
|
| 434 |
describes the newer tool.</emphasis></para> |
<para>The first step in any MITgcm build is to create a Unix-style |
| 435 |
|
<filename>Makefile</filename> which will be parsed by |
| 436 |
<para>The first step in any MITgcm build is to create a |
<filename>make</filename> to specify how to compile the MITgcm source |
| 437 |
Unix-style <filename>Makefile</filename> which will be parsed |
files. For more detailed descriptions of what the make tools are and |
| 438 |
by <filename>make</filename> to specify how to compile the |
how they are used, please see:</para> |
|
MITgcm source files. For more detailed descriptions of what |
|
|
the make tools are and how they are used, please see:</para> |
|
| 439 |
|
|
| 440 |
<itemizedlist> |
<itemizedlist> |
| 441 |
<listitem> |
<listitem> |
| 442 |
<para><ulink url="http://www.gnu.org/software/make/make.html">http://www.gnu.org/software/make/make.html</></para> |
<para><ulink url="http://www.gnu.org/software/make/make.html"> |
| 443 |
|
http://www.gnu.org/software/make/make.html</></para> |
| 444 |
</listitem> |
</listitem> |
| 445 |
<listitem> |
<listitem> |
| 446 |
<para><ulink url="http://www.oreilly.com/catalog/make2/">http://www.oreilly.com/catalog/make2/</></para> |
<para><ulink url="http://www.oreilly.com/catalog/make2/"> |
| 447 |
|
http://www.oreilly.com/catalog/make2/</></para> |
| 448 |
</listitem> |
</listitem> |
| 449 |
</itemizedlist> |
</itemizedlist> |
| 450 |
|
|
| 451 |
<para>Due to the poor handling of soft-links and other bugs |
<para>Genmake can often be invoked successfully with a command line as |
| 452 |
common with the <filename>make</filename> versions provided by |
simple as:</para> |
| 453 |
commercial Unix vendors, GNU <filename>make</filename> |
|
| 454 |
(sometimes called <filename>gmake</filename>) should be |
<screen> |
| 455 |
preferred.</para> |
$ genmake2 -mods=../code |
| 456 |
|
</screen> |
| 457 |
<para>As the name implies, <filename>genmake2</filename> |
|
| 458 |
generates a <filename>Makefile</filename>. It does so by |
<para>However, some systems (particularly commercial Unixes that lack a |
| 459 |
first parsing the information supplied from the following |
more modern "/bin/sh" implementation or that have shells installed in |
| 460 |
sources</para> |
odd locations) may require an explicit shell invocation such as one of |
| 461 |
|
the following: </para> |
| 462 |
|
|
| 463 |
|
<screen> |
| 464 |
|
$ /usr/bin/sh genmake2 -make=gmake -mods=../code |
| 465 |
|
$ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code |
| 466 |
|
</screen> |
| 467 |
|
|
| 468 |
|
<para>The genmake2 code has been written in a Bourne and BASH (v1) |
| 469 |
|
compatible syntax so it should work with most "sh" and all recent "bash" |
| 470 |
|
implementations.</para> |
| 471 |
|
|
| 472 |
|
<para>As the name implies, <filename>genmake2</filename> generates a |
| 473 |
|
<filename>Makefile</filename>. It does so by first parsing the |
| 474 |
|
information supplied from the following sources</para> |
| 475 |
|
|
| 476 |
<orderedlist> |
<orderedlist> |
| 477 |
<listitem> |
<listitem> |
| 478 |
<para>a <filename>gm_local</filename> file in the current |
<para>a <filename>gemake_local</filename> file in the current |
| 479 |
directory</para> |
directory</para> |
| 480 |
</listitem> |
</listitem> |
| 481 |
<listitem> |
<listitem> |
| 482 |
<para>directly from command-line options</para> |
<para>directly from command-line options</para> |
| 483 |
</listitem> |
</listitem> |
| 484 |
<listitem> |
<listitem> |
| 485 |
<para>an "options file" as specified by the command-line |
<para>an "options file" as specified by the command-line option |
| 486 |
option <filename>-optfile='FILENAME'</filename></para> |
<filename>-optfile='FILENAME'</filename></para> |
| 487 |
|
</listitem> |
| 488 |
|
<listitem> |
| 489 |
|
<para> a <filename>packages.conf</filename> file (in the current |
| 490 |
|
directory or in one of the "MODS" directories, see below) |
| 491 |
|
which contains the specific list of packages to compile |
| 492 |
|
</para> |
| 493 |
</listitem> |
</listitem> |
| 494 |
</orderedlist> |
</orderedlist> |
| 495 |
|
|
| 496 |
<para>then checking certain dependency rules (the package |
<para>then checking certain dependency rules (the package dependencies), |
| 497 |
dependencies), and then writing a |
and finally writing a <filename>Makefile</filename> based upon the |
| 498 |
<filename>Makefile</filename> based upon the source code that |
source code that it finds. For convenience within various Unix |
| 499 |
it finds. For convenience with the various Unix shells, |
shells, <filename>genmake2</> supports both "long"- and "short"-style |
| 500 |
<filename>genmake2</> supports both "long"- and "shor"-style |
options. A complete list of the available options can be obtained |
| 501 |
options. A complete list of the available options can be |
from:</para> |
|
obtained from:</para> |
|
| 502 |
|
|
| 503 |
<screen> |
<screen> |
| 504 |
$ genmake2 -help |
$ genmake2 -help |
| 505 |
</screen> |
</screen> |
| 506 |
|
|
| 507 |
<para>The most important options for <filename>genmake2</> |
<para>The most important options for <filename>genmake2</> are:</para> |
|
are:</para> |
|
| 508 |
|
|
| 509 |
<variablelist> |
<variablelist> |
| 510 |
|
|
| 511 |
<varlistentry> |
<varlistentry> |
| 512 |
<term><filename>--optfile=/PATH/FILENAME</></term> |
<term><filename>--optfile=/PATH/FILENAME</></term> |
| 513 |
|
|
| 514 |
<listitem> |
<listitem> |
| 515 |
<para>This specifies the "options file" that should be |
<para>This specifies the "options file" that should be used for a |
| 516 |
used for a particular build. The options file is a |
particular build. The options file is a convenient and |
| 517 |
convenient and machine-indepenent way of specifying |
machine-indepenent way of specifying parameters such as the |
| 518 |
parameters such as the FORTRAN compiler |
FORTRAN compiler (<filename>FC=</>), FORTRAN compiler |
| 519 |
(<filename>FC=</>), FORTRAN compiler optimization flags |
optimization flags (<filename>FFLAGS=</>), and the locations of |
| 520 |
(<filename>FFLAGS=</>), and the locations of various |
various platform- and/or machine-specific tools |
| 521 |
platform- and/or machine-specific tools |
(eg. <filename>MAKEDEPEND=</>). As with <filename>genmake2</>, |
| 522 |
(eg. <filename>MAKEDEPEND=</>). As with |
all options files should be written to be compatible with |
| 523 |
<filename>genmake2</>, all options files should be |
Bourne--shell ("sh" or "BASH v1") syntax. Examples of various |
| 524 |
written a BASH v1-compatible syntax. Examples of |
options files can be found in |
| 525 |
various options files can be found in |
<filename>$ROOTDIR/tools/build_options</>.</para> |
| 526 |
<filename>$ROOTDIR/tools/build_options</>. Everyone is |
|
| 527 |
encouraged to submit their options files to the MITgcm |
<para>If no "optfile" is specified (either through the command lin |
| 528 |
project for inclusion (please send to |
or the environment variable), genmake2 will try to make a |
| 529 |
<email>MITgcm-support@mitgcm.org</email>). We are |
reasonable guess from the list provided in |
| 530 |
particularly grateful for options files tested on new or |
<filename>$ROOTDIR/tools/build_options</>. The method used for |
| 531 |
unique platforms!</para> |
making this guess is to first determine the combination of |
| 532 |
|
operating system and hardware (eg. "linux_ia32") and then find a |
| 533 |
|
working Fortran compiler within the user's path. When these |
| 534 |
|
three items have been identified, genmake2 will try to find an |
| 535 |
|
optfile that has a matching name. </para> |
| 536 |
|
|
| 537 |
|
<para>Everyone is encouraged to submit their options files to the |
| 538 |
|
MITgcm project for inclusion (please send to |
| 539 |
|
<email>MITgcm-support@mitgcm.org</email>). We are particularly |
| 540 |
|
grateful for options files tested on new or unique |
| 541 |
|
platforms!</para> |
| 542 |
</listitem> |
</listitem> |
| 543 |
|
|
| 544 |
</varlistentry> |
</varlistentry> |
| 545 |
|
|
| 546 |
<varlistentry> |
<varlistentry> |
| 547 |
<term><filename>-pdepend=/PATH/FILENAME</></term> |
<term><filename>-adof=/path/to/file</></term> |
| 548 |
|
<term><filename>-adoptfile=/path/to/file</></term> |
| 549 |
<listitem> |
<listitem> |
| 550 |
<para>This specifies the dependency file used for |
<para>This option specifies the "adjoint" or automatic |
| 551 |
packages. If not specified, the default dependency file |
differentiation options file to be used. The file is analogous |
| 552 |
is <filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for |
to the "optfile" defined above but it specifies information for |
| 553 |
this file is parsed on a line-by-line basis where each |
the AD build process. The default file is located in <filename> |
| 554 |
line containes either a comment ("#") or a simple |
$ROOTDIR/tools/adjoint_options/adjoint_default </> and it |
| 555 |
"PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or |
defines the "TAF" and "TAMC" compilers. An alternate version is |
| 556 |
"-" symbol specifies a "must be used with" or a "must |
also available at <filename> |
| 557 |
not be used with" relationship, respectively. If no |
$ROOTDIR/tools/adjoint_options/adjoint_staf </> that selects the |
| 558 |
rule is specified, then it is assumed that the two |
newer "STAF" compiler. As with any compilers, it is helpful to |
| 559 |
packages are compatible and will function either with or |
have their directories listed in your $PATH environment |
| 560 |
without each other.</para> |
variable.</para> |
| 561 |
</listitem> |
</listitem> |
| 562 |
</varlistentry> |
</varlistentry> |
| 563 |
|
|
| 564 |
<varlistentry> |
<varlistentry> |
| 565 |
<term><filename>-pdefault=PKG</></term> |
<term><filename>-mods=DIR</></term> |
| 566 |
<term><filename>-pdefault='PKG1 [PKG2 PKG3 ...]'</></term> |
<term><filename>-mods='DIR1 [DIR2 ...]'</></term> |
| 567 |
<listitem> |
<listitem> |
| 568 |
<para>This option specifies the default set of packages |
<para>This option specifies a list of directories containing |
| 569 |
to be used. If not set, the default package list will |
"modifications". These directories contain files with names |
| 570 |
|
that may (or may not) exist in the main MITgcm source tree but |
| 571 |
|
will be overridden by any identically-named sources within the |
| 572 |
|
"MODS" directories. The order of precedence for this |
| 573 |
|
"name-hiding" is as follows:</para> |
| 574 |
|
<itemizedlist> |
| 575 |
|
<listitem><para>"MODS" directories (in the order given) |
| 576 |
|
</para></listitem> |
| 577 |
|
<listitem><para>Packages either explicitly specified or |
| 578 |
|
provided by default (in the order given)</para></listitem> |
| 579 |
|
<listitem><para>Packages included due to package dependencies |
| 580 |
|
(in the order that that package dependencies are |
| 581 |
|
parsed)</para></listitem> |
| 582 |
|
<listitem><para>The "standard dirs" (which may have been |
| 583 |
|
specified by the "-standarddirs" option)</para></listitem> |
| 584 |
|
</itemizedlist> |
| 585 |
|
</listitem> |
| 586 |
|
</varlistentry> |
| 587 |
|
|
| 588 |
|
<varlistentry> |
| 589 |
|
<term><filename>-pgroups=/PATH/FILENAME</></term> |
| 590 |
|
<listitem> |
| 591 |
|
<para>This option specifies the file where package groups are |
| 592 |
|
defined. If not set, the package-groups definition will |
| 593 |
be read from |
be read from |
| 594 |
<filename>$ROOTDIR/pkg/pkg_default</>.</para> |
<filename>$ROOTDIR/pkg/pkg_groups</>.</para> |
| 595 |
|
<para> |
| 596 |
|
It also contains the default list of packages (defined |
| 597 |
|
as the group <filename>"default_pkg_list"</>) which is used |
| 598 |
|
when no specific package list (file: <filename>packages.conf</>) |
| 599 |
|
is found in current directory or in any "MODS" directory. |
| 600 |
|
</para> |
| 601 |
|
</listitem> |
| 602 |
|
</varlistentry> |
| 603 |
|
|
| 604 |
|
<varlistentry> |
| 605 |
|
<term><filename>-pdepend=/PATH/FILENAME</></term> |
| 606 |
|
|
| 607 |
|
<listitem> |
| 608 |
|
<para>This specifies the dependency file used for packages. If |
| 609 |
|
not specified, the default dependency file is |
| 610 |
|
<filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for this file is |
| 611 |
|
parsed on a line-by-line basis where each line containes either a |
| 612 |
|
comment ("#") or a simple "PKGNAME1 (+|-)PKGNAME2" pairwise rule |
| 613 |
|
where the "+" or "-" symbol specifies a "must be used with" or a |
| 614 |
|
"must not be used with" relationship, respectively. If no rule is |
| 615 |
|
specified, then it is assumed that the two packages are compatible |
| 616 |
|
and will function either with or without each other.</para> |
| 617 |
</listitem> |
</listitem> |
| 618 |
</varlistentry> |
</varlistentry> |
| 619 |
|
|
| 620 |
<varlistentry> |
<varlistentry> |
| 621 |
<term><filename></>-mods=DIR</term> |
<term><filename>-make=/path/to/gmake</></term> |
|
<term><filename></>-mods='DIR1 [DIR2 ...]'</term> |
|
| 622 |
<listitem> |
<listitem> |
| 623 |
<para>This option specifies a list of directories |
<para>Due to the poor handling of soft-links and other bugs common |
| 624 |
containing "modifications". These are files that may |
with the <filename>make</> versions provided by commercial Unix |
| 625 |
(or may not) exist in the main MITgcm source tree but |
vendors, GNU <filename>make</filename> (sometimes called |
| 626 |
will be overridden by any identically-named sources |
<filename>gmake</filename>) should be preferred. This option |
| 627 |
within the "MODS" directories.</para> |
provides a means for specifying the make program to be |
| 628 |
|
used.</para> |
| 629 |
</listitem> |
</listitem> |
| 630 |
</varlistentry> |
</varlistentry> |
| 631 |
|
|
| 632 |
</variablelist> |
</variablelist> |
| 633 |
|
|
| 634 |
<para>A successful run of <filename>genmake2</> will produce |
<para>A successful run of <filename>genmake2</> will produce a |
| 635 |
both a <filename>Makefile</> and a locally modified copy of |
<filename>Makefile</>, a <filename>PACKAGES_CONFIG.h</> file, and |
| 636 |
the specified <filename>CPP_OPTIONS.h</> file. The local copy |
various convenience files used for the automatic differentiation |
| 637 |
of <filename>CPP_OPTIONS.h</> will contain a list of |
process.</para> |
| 638 |
<filename>genmake2</>-created #DEFINE and #UNDEF statements |
|
| 639 |
that reflect the list of packages that will be compiled into |
<para>In general, it is best to use <filename>genmake2</> on a "clean" |
| 640 |
the code (either directly through enable/disable/defaults |
directory that is free of all source (*.[F,f],*.[F,f]90) and header |
| 641 |
options or indirectly through dependencies).</para> |
(*.h,*.inc) files. Generally, this can be accomplished in an |
| 642 |
|
"un-clean" directory by running "make Clean" followed by "make |
| 643 |
<para>In general, it is best to use <filename>genmake2</> on a |
makefile".</para> |
|
"clean" directory that is free of all source |
|
|
(*.[F,f],*.[F,f]90) and header (*.h,*.inc) files. Generally, |
|
|
this can be accomplished in an "un-clean" directory by running |
|
|
"make CLEAN" followed by "make makefile".</para> |
|
| 644 |
|
|
| 645 |
</sect3> |
</sect3> |
| 646 |
|
|
| 647 |
<sect3 id="makefile_use"> |
<sect3 id="makefile_use"> |
| 648 |
<title>Using <filename>Makefile</></title> |
<title>Using the <filename>Makefile</></title> |
| 649 |
|
|
| 650 |
<para>Once a <filename>Makefile</> has been created, one can |
<para>Once a <filename>Makefile</> has been created using |
| 651 |
build the executable using:</para> |
<filename>genmake2</>, one can build a "standard" (forward |
| 652 |
|
simulator) executable using:</para> |
| 653 |
|
|
| 654 |
<screen> |
<screen> |
| 655 |
$ make CLEAN |
$ make Clean |
| 656 |
$ make depend |
$ make depend |
| 657 |
$ make |
$ make |
| 658 |
</screen> |
</screen> |
| 659 |
|
|
| 660 |
<para>The "make CLEAN" step will remove any local source |
<para>The "make Clean" step will remove any stale source files, include |
| 661 |
files, include files, and links. It is strongly recommended |
files, and links. It is strongly recommended for "un-clean" |
| 662 |
for "un-clean" directories which may contain the (partial?) |
directories which may contain the (perhaps partial) results of |
| 663 |
results of previous builds. Such "debris" can interfere with |
previous builds. Such "debris" can interfere with the next stage of |
| 664 |
the next stage of the build.</para> |
the build. |
| 665 |
|
A more agressive cleaning option, "make CLEAN", can be used to also |
| 666 |
<para>The "make depend" step will create a large number of |
remove the executable and output files from a previous run.</para> |
| 667 |
symbolic links from the local directory to the source file |
|
| 668 |
locations. It also parses these files and creates an |
<para>The "make depend" step will create a large number of symbolic |
| 669 |
extensive list of dependencies within the |
links from the local directory to the source file locations. It also |
| 670 |
<filename>Makefile</> itself. The links that exist at this |
parses these files and creates an extensive list of dependencies |
| 671 |
stage are mostly "large F" files (*.F and *.F90) that need to |
within the <filename>Makefile</> itself. The links that exist at this |
| 672 |
be processed by a C preprocessor ("CPP"). |
stage are mostly "large F" files (*.F and *.F90) that need to be |
| 673 |
</para> |
processed by a C preprocessor ("CPP"). Since "make depend" edits the |
| 674 |
|
<filename>Makefile</>, it is important not to skip this step!</para> |
| 675 |
<para>The final "make" invokes the C preprocessor to produce |
|
| 676 |
the "little f" files (*.f and *.f90) and then compiles them to |
<para>The final "make" invokes the C preprocessor to produce the "little |
| 677 |
object code using the specified FORTRAN compiler and options. |
f" files (*.f and *.f90) and then compiles them to object code using |
| 678 |
An intermediate script is often used during this stage to |
the specified FORTRAN compiler and options. An intermediate script is |
| 679 |
further process (usually, make simple substitutions) custom |
often used during this stage to further process (usually, make simple |
| 680 |
definitions such as variable types within the source files. |
substitutions) custom definitions such as variable types within the |
| 681 |
This additional stage is necessary in order to overcome some |
source files. This additional stage is necessary in order to overcome |
| 682 |
of the inconsistencies in the sizes of objects (bytes) between |
some of the inconsistencies in the sizes of objects (bytes) between |
| 683 |
different compilers.</para> |
different compilers. The result of the build process is an executable |
| 684 |
|
with the name <filename>mitgcmuv</>.</para> |
| 685 |
|
|
| 686 |
|
<para>In addition to the forward simulator described above, the |
| 687 |
|
<filename>Makefile</> also has a number of targets that can be used to |
| 688 |
|
produce various adjoint and tangent-linear builds for optimization and |
| 689 |
|
other parameter-sensitivity problems. The additional targets within |
| 690 |
|
the <filename>Makefile</> are:</para> |
| 691 |
|
|
| 692 |
|
<variablelist> |
| 693 |
|
|
| 694 |
<para>Please report compilation failures or other problems to |
<varlistentry> |
| 695 |
<email>MITgcm-support@mitgcm.org</email>.</para> |
<term><filename>make adall</></term> |
| 696 |
|
<listitem> |
| 697 |
|
<para>This target produces an <filename>mitgcmuv_ad</> executable |
| 698 |
|
using the <filename>taf</> or <filename>staf</> adjoint |
| 699 |
|
compiler. See the <filename>genmake2</> "-adof" option for |
| 700 |
|
compiler selection.</para> |
| 701 |
|
</listitem> |
| 702 |
|
</varlistentry> |
| 703 |
|
|
| 704 |
|
<varlistentry> |
| 705 |
|
<term><filename>make ftlall</></term> |
| 706 |
|
<listitem> |
| 707 |
|
<para>Similar to <filename>make adall</> above, this |
| 708 |
|
produces...</para> |
| 709 |
|
</listitem> |
| 710 |
|
</varlistentry> |
| 711 |
|
|
| 712 |
|
</variablelist> |
| 713 |
|
|
| 714 |
|
<para>Please report any compilation failures or other build problems to |
| 715 |
|
the <email>MITgcm-support@mitgcm.org</email> list.</para> |
| 716 |
|
|
| 717 |
</sect3> |
</sect3> |
| 718 |
|
|
| 721 |
<sect2 id="verification"> |
<sect2 id="verification"> |
| 722 |
<title>The Verification Suite</title> |
<title>The Verification Suite</title> |
| 723 |
|
|
| 724 |
<para>The MITgcm CVS tree (within the |
<para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</> |
| 725 |
<filename>$ROOTDIR/verification/</> directory) includes more |
directory) includes many (> 90) examples intended for regression |
| 726 |
than a dozen examples intended for regression testing. Each one |
testing. Each one of these test-experiment directories contains "known-good" |
| 727 |
of these example directories contains "known-good" output files |
output files along with all the input (including both code and data |
| 728 |
along with all the input (including both code and data files) |
files) required for their re-calculation. |
| 729 |
required for their re-calculation. These example directories |
Also included in <filename>$ROOTDIR/verification/</> is the shell |
| 730 |
are further broken down into sets of subdirectories |
script <filename>testreport</> to perform regression tests.</para> |
| 731 |
(eg. <filename>/input</>, <filename>/code</>) intended to |
|
| 732 |
expedite the testing process.</para> |
<sect3 id="test-experiment"> |
| 733 |
|
<title>Test-experiment Directory Content</title> |
| 734 |
|
|
| 735 |
|
<para> Each test-experiment directory (<filename>TESTDIR</>) contains |
| 736 |
|
several standard subdirectories and files which <filename>testreport</> |
| 737 |
|
recognizes and uses when running a regression test. |
| 738 |
|
The directories/files that <filename>testreport</> uses are different |
| 739 |
|
for a forward test and an adjoint test (<filename>testreport -adm</>) |
| 740 |
|
and some test-experiments are set-up for only one type of regression |
| 741 |
|
test whereas others allow both types of tests (forward and adjoint).</para> |
| 742 |
|
<para>Also some test-experiment allows, using the same MITgcm executable, |
| 743 |
|
to perform multiple tests using different parameters and input files, |
| 744 |
|
with a primary input set-up |
| 745 |
|
(<filename>input/ </> or <filename>input_ad/ </>) |
| 746 |
|
and corresponding results (<filename>results/output.txt</> or |
| 747 |
|
<filename>results/output_adm.txt</>) and with one or several secondary |
| 748 |
|
inputs (<filename>input.OTHER/ </> or <filename>input_ad.OTHER/ </>) |
| 749 |
|
and corresponding results (<filename>results/output.OTHER.txt </> |
| 750 |
|
or <filename>results/output_adm.OTHER.txt</>).</para> |
| 751 |
|
<variablelist> |
| 752 |
|
|
| 753 |
|
<varlistentry> |
| 754 |
|
<term>directory <filename>TESTDIR/results/</></term> |
| 755 |
|
<listitem> |
| 756 |
|
<para>contains reference standard output used for test comparison. |
| 757 |
|
<filename>results/output.txt</> and |
| 758 |
|
<filename>results/output_adm.txt</> |
| 759 |
|
correspond respectively to primary forward and adjoint test |
| 760 |
|
run on the reference platform (currently |
| 761 |
|
<filename>baudelaire.csail.mit.edu</>) |
| 762 |
|
on one processor (no MPI, single thread) |
| 763 |
|
using the reference compiler (curently the GNU fortran |
| 764 |
|
compiler <filename>gfortran</>). |
| 765 |
|
The presence of these files determines whether or not |
| 766 |
|
<filename>testreport</> is testing or skipping |
| 767 |
|
this test-experiment. |
| 768 |
|
Reference standard output for secondary tests |
| 769 |
|
(<filename>results/output.OTHER.txt</> |
| 770 |
|
or <filename>results/output_adm.OTHER.txt</>) are |
| 771 |
|
also expected here.</para> |
| 772 |
|
<para>The test comparison involves few model variables output, which are, |
| 773 |
|
by default and for a forward test, the 2-D solver initial residual |
| 774 |
|
(<filename>cg2d_init_res</>) and 3-D state variables (T,S,U,V) |
| 775 |
|
monitor output, and, by default and for an adjoint test, the |
| 776 |
|
cost-function and gradient-check. However, some test-experiments |
| 777 |
|
use some package-specific variable/monitor output according to |
| 778 |
|
the file <filename>TESTDIR/input[_ad][.OTHER]/tr_checklist</> |
| 779 |
|
specification.</para> |
| 780 |
|
</listitem> |
| 781 |
|
</varlistentry> |
| 782 |
|
|
| 783 |
|
<varlistentry> |
| 784 |
|
<term>directory <filename>TESTDIR/build/</></term> |
| 785 |
|
<listitem> |
| 786 |
|
<para> initially empty directory where <filename>testreport</> |
| 787 |
|
will build the MITgcm executable for forward and adjoint test. |
| 788 |
|
It might contains an experiment specific |
| 789 |
|
<filename>genmake_local</> file (see <xref linkend="genmake">). |
| 790 |
|
</para> |
| 791 |
|
<para> Note that the original <filename>code[_ad]/SIZE.h_mpi</> |
| 792 |
|
is not directly used as "SIZE.h" to build an MPI-executable ; |
| 793 |
|
instead, a local copy <filename>build/SIZE.h.mpi</> is derived |
| 794 |
|
from <filename>code[_ad]/SIZE.h_mpi</> |
| 795 |
|
by adjusting the number of processors (nPx,nPy) according to |
| 796 |
|
<filename>NUMBER_OF_PROCS</> (see <xref linkend="testreport">, |
| 797 |
|
<filename>testreport -MPI</>) ; then it is linked to "SIZE.h" |
| 798 |
|
(<filename> ln -s SIZE.h.mpi SIZE.h </>) before building |
| 799 |
|
the MPI-executable.</para> |
| 800 |
|
</listitem> |
| 801 |
|
</varlistentry> |
| 802 |
|
|
| 803 |
|
<varlistentry> |
| 804 |
|
<term>directory <filename>TESTDIR/code/</></term> |
| 805 |
|
<listitem> |
| 806 |
|
<para>contains the test-experiment specific source code |
| 807 |
|
used to build the MITgcm executable (<filename>mitgcmuv</>) |
| 808 |
|
for forward-test (using <filename>genmake2 -mods=../code</>). |
| 809 |
|
</para> |
| 810 |
|
<para> It can also contain specific source files with the suffix |
| 811 |
|
"<filename>_mpi</>" to be used |
| 812 |
|
<!-- |
| 813 |
|
in the <filename>TESTDIR/build/</> directory |
| 814 |
|
--> |
| 815 |
|
in place of the corresponding file (without suffix) |
| 816 |
|
for an MPI test (see <xref linkend="testreport">). |
| 817 |
|
The presence or absence of <filename>SIZE.h_mpi</> |
| 818 |
|
determines whether or not an MPI test on this |
| 819 |
|
test-experiment is performed or skipped.</para> |
| 820 |
|
</listitem> |
| 821 |
|
</varlistentry> |
| 822 |
|
|
| 823 |
|
<varlistentry> |
| 824 |
|
<term>directory <filename>TESTDIR/code_ad/</></term> |
| 825 |
|
<listitem> |
| 826 |
|
<para>contains the test-experiment specific source code |
| 827 |
|
used to build the MITgcm executable (<filename>mitgcmuv_ad</>) |
| 828 |
|
for adjoint-test (using <filename>genmake2 -mods=../code_ad</>). |
| 829 |
|
It can also contain specific source files with the suffix |
| 830 |
|
"<filename>_mpi</>" (see above).</para> |
| 831 |
|
</listitem> |
| 832 |
|
</varlistentry> |
| 833 |
|
|
| 834 |
|
<varlistentry> |
| 835 |
|
<term>directory <filename>TESTDIR/input/</></term> |
| 836 |
|
<listitem> |
| 837 |
|
<para>contains the input and parameter files |
| 838 |
|
used to run the primary forward test of this test-experiment. |
| 839 |
|
</para> |
| 840 |
|
<para>It can also contain specific parameter files with the suffix |
| 841 |
|
"<filename>.mpi</>" to be used |
| 842 |
|
<!-- |
| 843 |
|
in the <filename>TESTDIR/run/</> directory |
| 844 |
|
--> |
| 845 |
|
in place of the corresponding file (without suffix) for MPI |
| 846 |
|
test, or with suffix "<filename>.mth</>" to be used for |
| 847 |
|
multi-threaded test (see <xref linkend="testreport">). |
| 848 |
|
The presence or absence of <filename>eedata.mth</> |
| 849 |
|
determines whether or not a multi-threaded test on this |
| 850 |
|
test-experiment is performed or skipped.</para> |
| 851 |
|
<para>To save disk space and reduce downloading time, multiple |
| 852 |
|
copies of the same input file is avoided by using a shell |
| 853 |
|
script <filename>prepare_run</>. |
| 854 |
|
When such a script is found in <filename>TESTDIR/input/ </>, |
| 855 |
|
<filename>testreport</> run this script in directory |
| 856 |
|
<filename>TESTDIR/run/ </> after linking all the input file |
| 857 |
|
from <filename>TESTDIR/input/ </>. |
| 858 |
|
</para> |
| 859 |
|
</listitem> |
| 860 |
|
</varlistentry> |
| 861 |
|
|
| 862 |
|
<varlistentry> |
| 863 |
|
<term>directory <filename>TESTDIR/input_ad/</></term> |
| 864 |
|
<listitem> |
| 865 |
|
<para>contains the input and parameter files |
| 866 |
|
used to run the primary adjoint test of this test-experiment. |
| 867 |
|
It can also contain specific parameter files with the suffix |
| 868 |
|
"<filename>.mpi</>" and shell script <filename>prepare_run</> |
| 869 |
|
as described above.</para> |
| 870 |
|
</listitem> |
| 871 |
|
</varlistentry> |
| 872 |
|
|
| 873 |
|
<varlistentry> |
| 874 |
|
<term>directory <filename>TESTDIR/input.OTHER/</></term> |
| 875 |
|
<listitem> |
| 876 |
|
<para>contains the input and parameter files |
| 877 |
|
used to run the secondary OTHER forward test of this test-experiment. |
| 878 |
|
It can also contain specific parameter files with suffix |
| 879 |
|
"<filename>.mpi</>" or "<filename>.mth</>" and shell script |
| 880 |
|
<filename>prepare_run</> (see above).</para> |
| 881 |
|
<para> The presence or absence the file <filename>eedata.mth</> |
| 882 |
|
determines whether or not a secondary multi-threaded test on this |
| 883 |
|
test-experiment is performed or skipped.</para> |
| 884 |
|
</listitem> |
| 885 |
|
</varlistentry> |
| 886 |
|
|
| 887 |
|
<varlistentry> |
| 888 |
|
<term>directory <filename>TESTDIR/input_ad.OTHER/</></term> |
| 889 |
|
<listitem> |
| 890 |
|
<para>contains the input and parameter files |
| 891 |
|
used to run the secondary OTHER adjoint test of this test-experiment. |
| 892 |
|
It can also contain specific parameter files with the suffix |
| 893 |
|
"<filename>.mpi</>" and shell script <filename>prepare_run</> |
| 894 |
|
(see above).</para> |
| 895 |
|
</listitem> |
| 896 |
|
</varlistentry> |
| 897 |
|
<!-- |
| 898 |
|
--> |
| 899 |
|
<varlistentry> |
| 900 |
|
<term>directory <filename>TESTDIR/run/</></term> |
| 901 |
|
<listitem> |
| 902 |
|
<para> initially empty directory where <filename>testreport</> |
| 903 |
|
will run the MITgcm executable for primary forward and adjoint |
| 904 |
|
test.</para> |
| 905 |
|
<para>Symbolic links (using command "<filename>ln -s</>") are made |
| 906 |
|
for input and parameter files (from <filename>../input/ </> |
| 907 |
|
or from <filename>../input_ad/ </>) and for MITgcm executable |
| 908 |
|
(from <filename>../build/ </>) before the run proceeds. |
| 909 |
|
The sequence of links (function <filename>linkdata</> within shell |
| 910 |
|
script <filename>testreport</>) for a forward test is:</para> |
| 911 |
|
<screen> |
| 912 |
|
* link+rename or remove links |
| 913 |
|
to special files with suffix ".mpi" or ".mth" from ../input/ |
| 914 |
|
* link files from ../input/ |
| 915 |
|
* execute ../input/prepare_run (if it exists) |
| 916 |
|
</screen> |
| 917 |
|
<para>The sequence for an ajoint test is similar, with |
| 918 |
|
<filename>../input_ad/ </> replacing <filename>../input/ </>. |
| 919 |
|
</para> |
| 920 |
|
</listitem> |
| 921 |
|
</varlistentry> |
| 922 |
|
|
| 923 |
|
<varlistentry> |
| 924 |
|
<term>directory <filename>TESTDIR/tr_run.OTHER/</></term> |
| 925 |
|
<listitem> |
| 926 |
|
<para> directory created by <filename>testreport</> |
| 927 |
|
to run the MITgcm executable for secondary "OTHER" forward |
| 928 |
|
or adjoint test.</para> |
| 929 |
|
<para> The sequence of links for a forward secondary test is:</para> |
| 930 |
|
<screen> |
| 931 |
|
* link+rename or remove links |
| 932 |
|
to special files with suffix ".mpi" or ".mth" from ../input.OTHER/ |
| 933 |
|
* link files from ../input.OTHER/ |
| 934 |
|
* execute ../input.OTHER/prepare_run (if it exists) |
| 935 |
|
* link files from ../input/ |
| 936 |
|
* execute ../input/prepare_run (if it exists) |
| 937 |
|
</screen> |
| 938 |
|
<para>The sequence for an ajoint test is similar, with |
| 939 |
|
<filename>../input_ad.OTHER/ </> and <filename>../input_ad/ </> |
| 940 |
|
replacing <filename>../input.OTHER/ </> and <filename>../input/ </>. |
| 941 |
|
</para> |
| 942 |
|
</listitem> |
| 943 |
|
</varlistentry> |
| 944 |
|
|
| 945 |
|
</variablelist> |
| 946 |
|
</sect3> |
| 947 |
|
|
| 948 |
<sect3 id="testreport"> |
<sect3 id="testreport"> |
| 949 |
<title>The <filename>testreport</> Utility</title> |
<title>The <filename>testreport</> Utility</title> |
| 950 |
|
|
| 951 |
<para>Also included in <filename>$ROOTDIR/verification/</> are |
<para> The shell script <filename>testreport</> (in |
| 952 |
shell scripts for automated testing. The newest script (which |
<filename>$ROOTDIR/verification/</>), which was written to work with |
| 953 |
was written to work with <filename>genmake2</>) is called |
<filename>genmake2</>, can be used to build different versions |
| 954 |
<filename>testreport</>. Ths script can be used to build the |
of the MITgcm code, run the various examples, compare the output, |
| 955 |
different versions of the MITgcm code, run the various |
and (if specified) email the results of each one of these tests to a |
| 956 |
examples, compare the output, and (if specified) email the |
central repository.</para> |
| 957 |
results of each one of these tests to a central |
|
| 958 |
repository.</para> |
<para>On some systems, the testreport script can be run with a command |
| 959 |
|
line as simple as:</para> |
| 960 |
|
|
| 961 |
|
<screen> |
| 962 |
|
$ cd verification |
| 963 |
|
$ ./testreport |
| 964 |
|
</screen> |
| 965 |
|
|
| 966 |
|
<para>However, some systems (those lacking or wiht a broken "/bin/sh") |
| 967 |
|
may require an explicit shell invocation such as:</para> |
| 968 |
|
|
| 969 |
|
<screen> |
| 970 |
|
$ sh ./testreport -t 'exp2 exp4' |
| 971 |
|
$ /some/path/to/bash ./testreport -t 'ideal_2D_oce lab_sea natl_box' |
| 972 |
|
</screen> |
| 973 |
|
|
| 974 |
<para>The <filename>testreport</> script accepts a number of |
<para>The <filename>testreport</> script accepts a number of |
| 975 |
command-line options which can be listed using the |
command-line options which can be listed using the <filename>-help</> |
| 976 |
<filename>-help</> option. The most important ones are:</para> |
option. The most important ones are:</para> |
| 977 |
|
|
| 978 |
<variablelist> |
<variablelist> |
| 979 |
|
|
| 980 |
<varlistentry> |
<varlistentry> |
| 981 |
<term><filename>-tdir TESTDIR</></term> |
<term><filename>-ieee</> (default) / <filename>-noieee</></term> |
|
<term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term> |
|
| 982 |
<listitem> |
<listitem> |
| 983 |
<para>This option specifies the test directory or list |
<para>If allowed by the compiler (as defined in the "optfile"), |
| 984 |
of test directories that should be used. Each of these |
use IEEE arithmetic (<filename>genmake2 -ieee</>). |
| 985 |
entries should exactly (note: they're case sensitive!) |
This option, along with the gfortran / gcc compiler, |
| 986 |
match the names of directries in |
is how the standard results are produced.</para> |
|
<filename>$ROOTDIR/verification/</>. If this option is |
|
|
omitted, then all directories that are properly |
|
|
formatted (that is, containing an <filename>input</> |
|
|
sub-directory and example output) will be used.</para> |
|
| 987 |
</listitem> |
</listitem> |
| 988 |
</varlistentry> |
</varlistentry> |
| 989 |
|
|
| 991 |
<term><filename>-optfile=/PATH/FILENAME</></term> |
<term><filename>-optfile=/PATH/FILENAME</></term> |
| 992 |
<term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term> |
<term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term> |
| 993 |
<listitem> |
<listitem> |
| 994 |
<para>This specifies a list of "options files" that will |
<para>This specifies a list of "options files" that will be passed |
| 995 |
be passed to <filename>genmake2</>. If multiple options |
to <filename>genmake2</>. If multiple options files are used |
| 996 |
files are used (say, to test different compilers or |
(say, to test different compilers or different sets of options |
| 997 |
different sets of options for the same compiler), then |
for the same compiler), then each options file will be used with |
| 998 |
each options file will be used with each of the test |
each of the test directories.</para> |
| 999 |
directories.</para> |
</listitem> |
| 1000 |
|
</varlistentry> |
| 1001 |
|
|
| 1002 |
|
<varlistentry> |
| 1003 |
|
<term><filename>-tdir TESTDIR</></term> |
| 1004 |
|
<term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term> |
| 1005 |
|
<listitem> |
| 1006 |
|
<para>This option specifies the test directory or list of test |
| 1007 |
|
directories that should be used. Each of these entries should |
| 1008 |
|
exactly (note: they are case sensitive!) match the names of |
| 1009 |
|
directories in <filename>$ROOTDIR/verification/</>. If this |
| 1010 |
|
option is omitted, then all directories that are properly |
| 1011 |
|
formatted (that is, containing an <filename>input</> |
| 1012 |
|
sub-directory and a <filename>results/output.txt</> file) will |
| 1013 |
|
be used.</para> |
| 1014 |
</listitem> |
</listitem> |
| 1015 |
</varlistentry> |
</varlistentry> |
| 1016 |
|
|
| 1019 |
<term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term> |
<term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term> |
| 1020 |
<listitem> |
<listitem> |
| 1021 |
<para>Send the results (namely, <filename>output.txt</>, |
<para>Send the results (namely, <filename>output.txt</>, |
| 1022 |
<filename>gm_local</>, <filename>gm_state</>, and |
<filename>genmake_local</>, <filename>genmake_state</>, and |
| 1023 |
<filename>Makefile</>) to the specified email addresses. |
<filename>Makefile</>) to the specified email addresses. The |
| 1024 |
The results are gzipped, placed in a tar file, MIME |
results are gzipped, placed in a tar file, MIME encoded, and |
| 1025 |
encoded, and sent to an @mitgcm.org address. If no |
sent to the specified address. If no email addresses are |
| 1026 |
email addresses are specified, no mail is sent.</para> |
specified, no mail is sent.</para> |
| 1027 |
|
</listitem> |
| 1028 |
|
</varlistentry> |
| 1029 |
|
|
| 1030 |
|
<varlistentry> |
| 1031 |
|
<term><filename>-MPI NUMBER_OF_PROCS</></term> |
| 1032 |
|
<term><filename>-mpi</></term> |
| 1033 |
|
<listitem> |
| 1034 |
|
<para>If the necessary file (<filename>TESTDIR/code/SIZE.h_mpi</>) |
| 1035 |
|
exists, then use it (and all <filename>TESTDIR/code/*_mpi</> files) |
| 1036 |
|
for an MPI--enabled run. The new option (<filename>-MPI</> followed |
| 1037 |
|
by the maximum number of processors) enable to build and run each |
| 1038 |
|
test-experiment using variable number of MPI processors (multiple |
| 1039 |
|
of <filename>nPx*nPy</> from <filename>TESTDIR/code/SIZE.h_mpi</> |
| 1040 |
|
and not larger than <filename>NUMBER_OF_PROCS</>). |
| 1041 |
|
The short option ("-mpi") can only be used to build and run on 2 MPI |
| 1042 |
|
processors (equivalent to "<filename>-MPI 2</>").</para> |
| 1043 |
|
<para>Note that the use of MPI typically requires a |
| 1044 |
|
special command option (see "-command" below) to invoke the MPI |
| 1045 |
|
executable. Examples of PBS scripts using testreport with MPI can be |
| 1046 |
|
found in the <ulink |
| 1047 |
|
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/"> |
| 1048 |
|
tools/example_scripts directory</ulink>.</para> |
| 1049 |
|
</listitem> |
| 1050 |
|
</varlistentry> |
| 1051 |
|
|
| 1052 |
|
<varlistentry> |
| 1053 |
|
<term><filename>-command='some command to run'</></term> |
| 1054 |
|
<listitem> |
| 1055 |
|
<para>For some tests, particularly MPI runs, a specific command |
| 1056 |
|
might be needed to run the executable. This option allows a more general |
| 1057 |
|
command (or shell script) to be invoked. Examples of PBS scripts |
| 1058 |
|
using testreport with MPI can be found in the <ulink |
| 1059 |
|
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/"> |
| 1060 |
|
tools/example_scripts directory</ulink>.</para> |
| 1061 |
|
<para> |
| 1062 |
|
For the case where the number of MPI processors varies according |
| 1063 |
|
to each test-experiment, some key-words within the command-to-run |
| 1064 |
|
argument will be replaced by their effective value:</para> |
| 1065 |
|
<para> |
| 1066 |
|
<filename>TR_NPROC </> will be replaced by the actual number |
| 1067 |
|
of MPI processors needed to run the current test-experiment.</para> |
| 1068 |
|
<para> |
| 1069 |
|
<filename>TR_MFILE </> will be replaced by the name of local-file |
| 1070 |
|
that testreport creates from the full list of machines which |
| 1071 |
|
"<filename>testreport -mf MACHINE_FILE</>" provides, but truncated |
| 1072 |
|
to the exact number of machines.</para> |
| 1073 |
|
</listitem> |
| 1074 |
|
</varlistentry> |
| 1075 |
|
|
| 1076 |
|
<varlistentry> |
| 1077 |
|
<term><filename>-mf MACHINE_FILE</></term> |
| 1078 |
|
<listitem> |
| 1079 |
|
<para> |
| 1080 |
|
To use with <filename>-MPI NUMBER_OF_PROCS</> option, to specify |
| 1081 |
|
the file containing the full list of <filename>NUMBER_OF_PROCS</> |
| 1082 |
|
machines to use for the MPI runs.</para> |
| 1083 |
|
</listitem> |
| 1084 |
|
</varlistentry> |
| 1085 |
|
|
| 1086 |
|
<varlistentry> |
| 1087 |
|
<term><filename>-mth</></term> |
| 1088 |
|
<listitem> |
| 1089 |
|
<para>compile (with <filename>genmake2 -omp</>) and run with |
| 1090 |
|
multiple threads (using eedata.mth).</para> |
| 1091 |
</listitem> |
</listitem> |
| 1092 |
</varlistentry> |
</varlistentry> |
| 1093 |
|
|
| 1094 |
</variablelist> |
</variablelist> |
| 1095 |
|
|
| 1096 |
<para>The <filename>testreport</> script will write progress |
<para>The <filename>testreport</> script will create an output directory |
| 1097 |
to the screen (stdout) as it runs. In addition, it will |
<filename>tr_NAME_DATE_N/ </>, with <filename>hostname</> as default |
| 1098 |
create a <filename>summary.txt</> file that contains a brief |
<filename>NAME</>, <filename>DATE</> the current date followed by |
| 1099 |
comparison of the current output with the "known-good" |
a suffix number "N" to distinguish from previous |
| 1100 |
output.</para> |
<filename>testreport</> output directories. |
| 1101 |
|
<filename>testreport</> writes progress to the screen (stdout) |
| 1102 |
|
and reports into the ouput directory as it runs. |
| 1103 |
|
In particular, one can find, in the ouput directory, |
| 1104 |
|
the <filename>summary.txt</> file that contains a brief comparison |
| 1105 |
|
of the current output with the "known-good" output. |
| 1106 |
|
At the end of the testing process, the <filename>tr_out.txt</> file |
| 1107 |
|
is generated in <filename>$ROOTDIR/verification/ </> |
| 1108 |
|
as a compact version of <filename>summry.txt</> file.</para> |
| 1109 |
|
|
| 1110 |
|
</sect3> |
| 1111 |
|
|
| 1112 |
|
<sect3 id="tst_2plus2"> |
| 1113 |
|
<title>The <filename>do_tst_2+2</> Utility</title> |
| 1114 |
|
<para> The shell script <filename>do_tst_2+2</> (in |
| 1115 |
|
<filename>$ROOTDIR/tools/ </>) |
| 1116 |
|
can be used to check the accuracy of the restart procedure. |
| 1117 |
|
</para> |
| 1118 |
</sect3> |
</sect3> |
| 1119 |
|
|
| 1120 |
</sect2> |
</sect2> |
| 1121 |
|
|
| 1122 |
|
|
| 1123 |
<sect2 id="packages"> |
<sect2 id="packages"> |
| 1124 |
<title>Creating MITgcm Packages</title> |
<title>Creating MITgcm Packages</title> |
| 1125 |
|
|
| 1126 |
<para>Optional parts of code have been separated from the MITgcmUV |
<para>Optional parts of code have been separated from the MITgcmUV core |
| 1127 |
core driver code and organised into packages. The packaging |
driver code and organised into packages. The packaging structure |
| 1128 |
structure provides a mechanism for maintaining suites of code, |
provides a mechanism for maintaining suites of code, specific to |
| 1129 |
specific to particular classes of problems, in a way that is |
particular classes of problems, in a way that is cleanly separated from |
| 1130 |
cleanly separated from the generic fluid dynamical |
the generic fluid dynamical engine.</para> |
| 1131 |
engine.</para> |
|
| 1132 |
|
<para>The MITgcmUV packaging structure is described below using generic |
| 1133 |
<para>The MITgcmUV packaging structure is described below using |
package names ${pkg}. A concrete examples of a package is the code for |
| 1134 |
generic package names ${pkg}. A concrete examples of a package |
implementing GM/Redi mixing. This code uses the package name</para> |
|
is the code for implementing GM/Redi mixing. This code uses the |
|
|
package name</para> |
|
| 1135 |
|
|
| 1136 |
</sect2> |
</sect2> |
| 1137 |
|
|
| 1155 |
below using generic package names ${pkg}. |
below using generic package names ${pkg}. |
| 1156 |
A concrete examples of a package is the code |
A concrete examples of a package is the code |
| 1157 |
for implementing GM/Redi mixing. This code uses |
for implementing GM/Redi mixing. This code uses |
| 1158 |
the package name |
the package name |
| 1159 |
* ${PKG} = GMREDI |
* ${PKG} = GMREDI |
| 1160 |
* ${pkg} = gmredi |
* ${pkg} = gmredi |
| 1161 |
* ${Pkg} = gmRedi |
* ${Pkg} = gmRedi |
| 1190 |
Package runtime config. options are imported |
Package runtime config. options are imported |
| 1191 |
into a common block held in a header file |
into a common block held in a header file |
| 1192 |
called "${PKG}.h". |
called "${PKG}.h". |
| 1193 |
|
Note: In some packages, the header file "${PKG}.h" is splitted |
| 1194 |
|
into "${PKG}_PARAMS.h" that contains the package parameters and |
| 1195 |
|
${PKG}_VARS.h" for the field arrays. |
| 1196 |
|
|
| 1197 |
o The core driver part of the model can check |
o The core driver part of the model can check |
| 1198 |
for runtime enabling or disabling of individual packages |
for runtime enabling or disabling of individual packages |
| 1215 |
1. Within the core driver code flags of the form |
1. Within the core driver code flags of the form |
| 1216 |
ALLOW_${PKG} are used to include or exclude |
ALLOW_${PKG} are used to include or exclude |
| 1217 |
whole packages. The ALLOW_${PKG} flags are included |
whole packages. The ALLOW_${PKG} flags are included |
| 1218 |
from a PKG_CPP_OPTIONS block which is currently |
from a PACKAGES_CONFIG.h file that is automatically |
| 1219 |
|
generated by genmake2 (see genmake2 section). |
| 1220 |
held in-line in the CPP_OPTIONS.h header file. |
held in-line in the CPP_OPTIONS.h header file. |
| 1221 |
e.g. |
e.g. |
| 1222 |
|
|
| 1223 |
Core model code ..... |
Core model code ..... |
| 1224 |
|
|
| 1225 |
|
#include "PACKAGES_CONFIG.h" |
| 1226 |
#include "CPP_OPTIONS.h" |
#include "CPP_OPTIONS.h" |
| 1227 |
: |
: |
| 1228 |
: |
: |
| 1234 |
|
|
| 1235 |
2. Within an individual package a header file, |
2. Within an individual package a header file, |
| 1236 |
"${PKG}_OPTIONS.h", is used to set CPP flags |
"${PKG}_OPTIONS.h", is used to set CPP flags |
| 1237 |
specific to that package. It is not recommended |
specific to that package. It also includes |
| 1238 |
to include this file in "CPP_OPTIONS.h". |
"PACKAGES_CONFIG.h" and "CPP_OPTIONS.h". |
| 1239 |
|
|
| 1240 |
|
|
| 1241 |
Package Boot Sequence |
Package Boot Sequence |
| 1248 |
1. S/R PACKAGES_BOOT() |
1. S/R PACKAGES_BOOT() |
| 1249 |
: |
: |
| 1250 |
CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... ) |
CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... ) |
| 1251 |
|
|
| 1252 |
|
|
| 1253 |
2. S/R PACKAGES_READPARMS() |
2. S/R PACKAGES_READPARMS() |
| 1254 |
: |
: |
| 1257 |
& CALL ${PKG}_READPARMS( retCode ) |
& CALL ${PKG}_READPARMS( retCode ) |
| 1258 |
#endif |
#endif |
| 1259 |
|
|
| 1260 |
2. S/R PACKAGES_CHECK() |
3. S/R PACKAGES_INIT_FIXED() |
| 1261 |
|
: |
| 1262 |
|
#ifdef ALLOW_${PKG} |
| 1263 |
|
if ( use${Pkg} ) |
| 1264 |
|
& CALL ${PKG}_INIT_FIXED( retCode ) |
| 1265 |
|
#endif |
| 1266 |
|
|
| 1267 |
|
4. S/R PACKAGES_CHECK() |
| 1268 |
: |
: |
| 1269 |
#ifdef ALLOW_${PKG} |
#ifdef ALLOW_${PKG} |
| 1270 |
if ( use${Pkg} ) |
if ( use${Pkg} ) |
| 1274 |
& CALL PACKAGES_CHECK_ERROR('${PKG}') |
& CALL PACKAGES_CHECK_ERROR('${PKG}') |
| 1275 |
#endif |
#endif |
| 1276 |
|
|
| 1277 |
3. S/R PACKAGES_INIT() |
5. S/R PACKAGES_INIT_VARIABLES() |
| 1278 |
: |
: |
| 1279 |
#ifdef ALLOW_${PKG} |
#ifdef ALLOW_${PKG} |
| 1280 |
if ( use${Pkg} ) |
if ( use${Pkg} ) |
| 1281 |
& CALL ${PKG}_INIT( retCode ) |
& CALL ${PKG}_INIT_VARIA( ) |
| 1282 |
|
#endif |
| 1283 |
|
|
| 1284 |
|
Package Output |
| 1285 |
|
============== |
| 1286 |
|
6. S/R DO_THE_MODEL_IO |
| 1287 |
|
|
| 1288 |
|
#ifdef ALLOW_${PKG} |
| 1289 |
|
if ( use${Pkg} ) |
| 1290 |
|
& CALL ${PKG}_OUTPUT( ) |
| 1291 |
#endif |
#endif |
| 1292 |
|
|
| 1293 |
|
7. S/R PACKAGES_WRITE_PICKUP() |
| 1294 |
|
|
| 1295 |
|
#ifdef ALLOW_${PKG} |
| 1296 |
|
if ( use${Pkg} ) |
| 1297 |
|
& CALL ${PKG}_WRITE_PICKUP( ) |
| 1298 |
|
#endif |
| 1299 |
|
|
| 1300 |
Description |
Description |
| 1301 |
=========== |
=========== |
| 1302 |
|
|
| 1303 |
- ${PKG}_READPARMS() |
- ${PKG}_READPARMS() |
| 1304 |
is responsible for reading |
is responsible for reading |
| 1305 |
in the package parameters file data.${pkg}, and storing |
in the package parameters file data.${pkg}, and storing |
| 1306 |
the package parameters in "${PKG}.h". |
the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h"). |
| 1307 |
-> called in INITIALISE_FIXED |
-> called from INITIALISE_FIXED in PACKAGES_READPARMS |
| 1308 |
|
|
| 1309 |
|
- ${PKG}_INIT_FIXED() |
| 1310 |
|
is responsible for completing the internal setup of a package. |
| 1311 |
|
-> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED |
| 1312 |
|
note: 1) some pkg use instead: |
| 1313 |
|
CALL ${PKG}_INITIALISE ( or the old form CALL ${PKG}_INIT ) |
| 1314 |
|
2) for simple pkg setup, this part is done inside ${PKG}_READPARMS |
| 1315 |
|
|
| 1316 |
- ${PKG}_CHECK() |
- ${PKG}_CHECK() |
| 1317 |
is responsible for validating |
is responsible for validating |
| 1318 |
basic package setup and inter-package dependencies. |
basic package setup and inter-package dependencies. |
| 1319 |
${PKG}_CHECK can import other package parameters it may |
${PKG}_CHECK can import other package parameters it may |
| 1320 |
need to check. This is done through header files "${PKG}.h". |
need to check. This is done through header files "${PKG}.h". |
| 1321 |
It is assumed that parameters owned by other packages |
It is assumed that parameters owned by other packages |
| 1322 |
will not be reset during ${PKG}_CHECK(). |
will not be reset during ${PKG}_CHECK(). |
| 1323 |
-> called in INITIALISE_FIXED |
-> called from INITIALISE_FIXED in PACKAGES_CHECK |
| 1324 |
|
|
| 1325 |
- ${PKG}_INIT() |
- ${PKG}_INIT_VARIA() |
| 1326 |
is responsible for completing the |
is responsible for fill-in all package variables with an initial value. |
| 1327 |
internal setup of a package. This routine is called after |
Contains eventually a call to ${PKG}_READ_PICKUP that will read |
| 1328 |
the core model state has been completely initialised |
from a pickup file the package variables required to restart the model. |
| 1329 |
but before the core model timestepping starts. |
This routine is called after the core model state has been completely |
| 1330 |
-> called in INITIALISE_VARIA |
initialised but before the core model timestepping starts. |
| 1331 |
|
-> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES |
| 1332 |
|
note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg |
| 1333 |
|
use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old |
| 1334 |
|
form ${PKG}_INIT |
| 1335 |
|
|
| 1336 |
|
- ${PKG}_OUTPUT( ) |
| 1337 |
|
is responsible for writing time-average fields to output files |
| 1338 |
|
(but the cumulating step is done within the package main S/R). |
| 1339 |
|
Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR) |
| 1340 |
|
and write snap-shot fields that are hold in common blocks. Other |
| 1341 |
|
temporary fields are directly dump to file where they are available. |
| 1342 |
|
NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages |
| 1343 |
|
but is beeing replaced by ${PKG}_OUTPUT |
| 1344 |
|
to avoid confusion with pkg/diagnostics functionality. |
| 1345 |
|
2) the output part is not yet in a standard form and might still |
| 1346 |
|
evolve a lot. |
| 1347 |
|
-> called within DO_THE_MODEL_IO |
| 1348 |
|
|
| 1349 |
|
- ${PKG}_WRITE_PICKUP() |
| 1350 |
|
is responsible for writing a package pickup file when necessary for |
| 1351 |
|
a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT ) |
| 1352 |
|
-> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP |
| 1353 |
|
|
| 1354 |
Summary |
Summary |
| 1355 |
======= |
======= |
| 1370 |
----------------------- |
----------------------- |
| 1371 |
* ${PKG}_OPTIONS.h has further package-specific CPP options |
* ${PKG}_OPTIONS.h has further package-specific CPP options |
| 1372 |
* ${PKG}.h package-specific common block variables, fields |
* ${PKG}.h package-specific common block variables, fields |
| 1373 |
|
or ${PKG}_PARAMS.h package-specific common block parameters |
| 1374 |
|
and ${PKG}_VARS.h package-specific common block fields |
| 1375 |
|
|
| 1376 |
- FORTRAN source files |
- FORTRAN source files |
| 1377 |
----------------------- |
----------------------- |
| 1378 |
* ${pkg}_readparms.F reads parameters from file data.${pkg} |
* ${pkg}_readparms.F reads parameters from file data.${pkg} |
| 1379 |
* ${pkg}_check.F checks package dependencies and consistencies |
* ${pkg}_init_fixed.F complete the package setup |
| 1380 |
* ${pkg}_init.F initialises package-related fields |
* ${pkg}_check.F checks package dependencies and consistencies |
| 1381 |
* ${pkg}_... .F package source code |
* ${pkg}_init_varia.F initialises package-related fields |
| 1382 |
|
* ${pkg}_... .F package source code |
| 1383 |
|
* ${pkg}_output.F write output to file. |
| 1384 |
|
* ${pkg}_write_pickup.F write a package pickup file to restart the model |
| 1385 |
|
|
| 1386 |
|
New: Subroutine in one package (pkgA) that only contains code which |
| 1387 |
|
is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F) |
| 1388 |
|
will be named: pkgA_pkgB_something.F |
| 1389 |
|
|
| 1390 |
- parameter file |
- parameter file |
| 1391 |
----------------------- |
----------------------- |
| 1395 |
</sect1> |
</sect1> |
| 1396 |
|
|
| 1397 |
|
|
| 1398 |
|
<sect1 id="documentation"> |
| 1399 |
|
<title>Editing the Documentation</title> |
| 1400 |
|
|
| 1401 |
|
<sect2 id="documentation_getting"> |
| 1402 |
|
<title>Getting the Docs and Code</title> |
| 1403 |
|
|
| 1404 |
|
<para>The first step towards editing the documentation is to checkout a |
| 1405 |
|
copy of code, docs, and build scripts from the CVS server using:</para> |
| 1406 |
|
|
| 1407 |
|
<screen> |
| 1408 |
|
$ export CVS_RSH=ssh |
| 1409 |
|
$ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack' |
| 1410 |
|
$ mkdir scratch |
| 1411 |
|
$ cvs co -P MITgcm manual mitgcm.org |
| 1412 |
|
</screen> |
| 1413 |
|
|
| 1414 |
|
<para>These commands extract the necessary information from the CVS server |
| 1415 |
|
and create a temporary (called <filename>scratch</filename>) directory for |
| 1416 |
|
the storage of the HTML and other files that will be created. Please note |
| 1417 |
|
that you must either create <filename>scratch</filename> as shown or edit |
| 1418 |
|
the various <filename>Makefile</filename>s and scripts used to create the |
| 1419 |
|
documentation.</para> |
| 1420 |
|
</sect2> |
| 1421 |
|
|
| 1422 |
|
<sect2> |
| 1423 |
|
<title>Editing the Documentation</title> |
| 1424 |
|
|
| 1425 |
|
<para>The documentation is contained in the <filename>manual</filename> |
| 1426 |
|
directory in a raw LaTeX format. The main document is |
| 1427 |
|
<filename>manual.tex</filename> and it uses <command>\input{}</command>s |
| 1428 |
|
to include the chapters and subsections.</para> |
| 1429 |
|
|
| 1430 |
|
<para>Since the same LaTeX source is used to produce PostScript, PDF, and |
| 1431 |
|
HTML output, care should be taken to follow certain conventions. Two of |
| 1432 |
|
the most important are the usage of the <command>\filelink{}{}</command> |
| 1433 |
|
and <command>\varlink{}{}</command> commands. Both of these commands have |
| 1434 |
|
been defined to simplify the connection between the automatically |
| 1435 |
|
generated ("code browser") HTML and the HTML version of the manual |
| 1436 |
|
produced by LaTeX2HTML. They each take two arguments (corresponding to |
| 1437 |
|
the contents of the two sets of curly braces) which are the text that the |
| 1438 |
|
author wishes to be "wrapped" within the link, and a specially formatted |
| 1439 |
|
link thats relative to the <filename>MITgcm</filename> directory within |
| 1440 |
|
the CVS tree.</para> |
| 1441 |
|
|
| 1442 |
|
<para>The result is a command that resembles either</para> |
| 1443 |
|
|
| 1444 |
|
<orderedlist> |
| 1445 |
|
<listitem> |
| 1446 |
|
<para>a reference to a variable or subroutine name such as |
| 1447 |
|
<command>\varlink{tRef}{tRef}</command>, or </para> |
| 1448 |
|
</listitem> |
| 1449 |
|
|
| 1450 |
|
<listitem> |
| 1451 |
|
<para>a reference to a file such as |
| 1452 |
|
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
| 1453 |
|
where the absolute path to the file is of the form |
| 1454 |
|
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
| 1455 |
|
<para>(please note how the leading "/foo/MITgcm" |
| 1456 |
|
component of the path is dropped leaving the path |
| 1457 |
|
<emphasis>relative</emphasis> to the head of the code |
| 1458 |
|
directory and each directory separator "/" is turned |
| 1459 |
|
into a "-")</para> |
| 1460 |
|
</listitem> |
| 1461 |
|
</orderedlist> |
| 1462 |
|
|
| 1463 |
|
|
| 1464 |
|
|
| 1465 |
|
</sect2> |
| 1466 |
|
|
| 1467 |
|
<sect2> |
| 1468 |
|
<title>Building the Documentation</title> |
| 1469 |
|
|
| 1470 |
|
<para>Given the directory structure of <xref |
| 1471 |
|
linkend="documentation_getting">, the entire documentation for the web |
| 1472 |
|
site can be built using:</para> |
| 1473 |
|
|
| 1474 |
|
<screen> |
| 1475 |
|
$ cd mitgcm.org/devel/buildweb |
| 1476 |
|
$ make All |
| 1477 |
|
</screen> |
| 1478 |
|
|
| 1479 |
|
<para>Which builds the PDF from the LaTeX source, creates the HTML output |
| 1480 |
|
from the LaTeX source, parses the FORTRAN code base to produce a |
| 1481 |
|
hyperlinked HTML version of the source, and then determines the |
| 1482 |
|
cross-linking between the various HTML components.</para> |
| 1483 |
|
|
| 1484 |
|
<para>If there are no errors, the result of the build process (which can |
| 1485 |
|
take 30+ minutes on a P4/2.5Ghz) will be contained within a single |
| 1486 |
|
directory called <filename>scratch/dev_docs</filename>. This is a freshly |
| 1487 |
|
built version of the entire on-line users manual. If you have the correct |
| 1488 |
|
permissions, it can be directly copied to the web server area:</para> |
| 1489 |
|
|
| 1490 |
|
<screen> |
| 1491 |
|
$ mv scratch/dev_docs /u/u0/httpd/html |
| 1492 |
|
</screen> |
| 1493 |
|
|
| 1494 |
|
<para>and the update is complete.</para> |
| 1495 |
|
|
| 1496 |
|
</sect2> |
| 1497 |
|
|
| 1498 |
|
</sect1> |
| 1499 |
|
|
| 1500 |
</article> |
</article> |
| 1501 |
|
|
| 1502 |
|
|
Parent Directory
| 
Revision Log
| 
Revision Graph
| 
Patch