1 |
<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN"> |
2 |
|
3 |
<article id="MITgcm-Development-HOWTO"> |
4 |
|
5 |
<!-- |
6 |
Build commands: |
7 |
db2ps -d ldp.dsl devel_HOWTO.sgml |
8 |
db2pdf -d ldp.dsl devel_HOWTO.sgml |
9 |
db2html -d ./ldp.dsl devel_HOWTO.sgml |
10 |
db2html -u -d ./ldp.dsl devel_HOWTO.sgml |
11 |
--> |
12 |
|
13 |
<articleinfo> |
14 |
<title>MITgcm Development HOWTO</title> |
15 |
|
16 |
<author> |
17 |
<firstname>Ed</firstname> |
18 |
<surname>Hill III</surname> |
19 |
<affiliation> |
20 |
<address><email>eh3@mit.edu</email></address> |
21 |
</affiliation> |
22 |
</author> |
23 |
|
24 |
<revhistory> |
25 |
<revision> |
26 |
<revnumber>0.01</revnumber> |
27 |
<date>2003-08-07</date> |
28 |
<authorinitials>eh3</authorinitials> |
29 |
<revremark> |
30 |
Initial version. |
31 |
</revremark> |
32 |
</revision> |
33 |
</revhistory> |
34 |
|
35 |
<abstract> |
36 |
<para>This document describes how to develop software for the |
37 |
MITgcm project.</para> |
38 |
</abstract> |
39 |
</articleinfo> |
40 |
|
41 |
<sect1 id="intro"> |
42 |
<title>Introduction</title> <para>The purpose of this document is |
43 |
to help new developers get "up to speed" with MITgcm |
44 |
development.</para> |
45 |
<sect2> |
46 |
<title>New Versions of This Document</title> <para>You can |
47 |
obtain the latest version of this document <ulink |
48 |
url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in |
49 |
various formats.</para> |
50 |
</sect2> |
51 |
<sect2> |
52 |
<title>Feedback and corrections</title> <para>If you have |
53 |
questions or comments about this document, please feel free to |
54 |
<ulink url="mailto:MITgcm-support@mitgcm.org">contact the |
55 |
authors</ulink>. |
56 |
</para> |
57 |
</sect2> |
58 |
</sect1> |
59 |
|
60 |
<sect1 id="background"> |
61 |
<title>Background</title> |
62 |
|
63 |
<sect2> |
64 |
<title>User Manual</title> |
65 |
|
66 |
<para>Before jumping into |
67 |
development, please familiarize yourself with the MITgcm user |
68 |
manual which is available <ulink |
69 |
url="http://mitgcm.org/">on the main web page</ulink>. This |
70 |
document contains volumes of useful information and is included |
71 |
here by reference.</para> |
72 |
|
73 |
<para>Also, a "snapshot" or<ulink |
74 |
url="http://mitgcm.org/dev_docs/">development version</ulink> of |
75 |
the user manual may be available, though this is only put on the |
76 |
web for testing purposes.</para> |
77 |
</sect2> |
78 |
|
79 |
<sect2> |
80 |
<title>Prerequisites</title> <para>To develop for MITgcm project |
81 |
you will need a UNIX or UNIX-like set of build tools including |
82 |
the following:</para> |
83 |
<blockquote> |
84 |
<simplelist type="inline"> |
85 |
<member>CVS client</member> |
86 |
<member>make or (preferably) GNU make</member> |
87 |
<member>FORTRAN compiler</member> |
88 |
<member>C compiler</member> |
89 |
<member>[ba]sh and [t]csh shells</member> |
90 |
<member>PERL</member> |
91 |
<member>LaTeX and LaTeX2HTML</member> |
92 |
</simplelist> |
93 |
</blockquote> |
94 |
<para>Essentially all of the work described here has been tested |
95 |
on recent versions of Red Hat Linux (eg. 7.3 through 9). Except |
96 |
where noted, all shell commands will be provided using bash |
97 |
syntax. |
98 |
</para> |
99 |
</sect2> |
100 |
|
101 |
</sect1> |
102 |
|
103 |
<sect1 id="cvs"> |
104 |
<title>CVS Repository</title> |
105 |
<sect2> |
106 |
<title>Layout</title> |
107 |
|
108 |
<para>Unlike many open source projects, the MITgcm CVS tree does |
109 |
not follow a simple "src", "docs", "share", and "test" directory |
110 |
layout. Instead, there are multiple higher-level directories |
111 |
that each, to some extent, depend upon the presence of the |
112 |
others. The tree currently resembles:</para> |
113 |
|
114 |
<programlisting>gcmpack/ |
115 |
MITgcm-contrib contributed code |
116 |
CS-regrid goes into utils |
117 |
cvspolicy.html -save- |
118 |
CVSROOT -save- |
119 |
development experimental stuff |
120 |
manual -save- |
121 |
misc -?- |
122 |
|
123 |
MITgcm code |
124 |
adjoint fold into genmake |
125 |
bin stub for ecco build |
126 |
compare01 old from 20th century |
127 |
diags timeave f77 in pkgs now |
128 |
doc tags -- connect to real docs? |
129 |
eesupp cnh? |
130 |
exe ecco user build |
131 |
*- jobs runtime shell scripts for |
132 |
| various platforms |
133 |
| lsopt line search |
134 |
m| model main dynamics (core) |
135 |
e| optimization_drivers ? |
136 |
r| optim line search interface |
137 |
g| pkg alternate and optional numerics, etc. |
138 |
e*- tools |
139 |
?| tutorial_examples documented tests |
140 |
| only populated on release1 branch |
141 |
| and not validated during "testscript" |
142 |
*- utils |
143 |
verification std tests |
144 |
|
145 |
|
146 |
mitgcmdoc -> manual -remove- |
147 |
mitgcm.org build web site |
148 |
models -?- |
149 |
packages -?- |
150 |
preprocess -?- |
151 |
tmp -?- |
152 |
</programlisting> |
153 |
|
154 |
<para>Efforts are underway to reduce the complexity.</para> |
155 |
|
156 |
</sect2> |
157 |
|
158 |
<!-- |
159 |
<sect2> |
160 |
<title>Releases</title> <para>Currently, there are two main |
161 |
branches:</para> |
162 |
<itemizedlist mark="bullet"> |
163 |
<listitem> |
164 |
<para>Development</para> |
165 |
<itemizedlist mark="bullet"> |
166 |
<listitem> |
167 |
<para>MAIN</para> |
168 |
</listitem> |
169 |
<listitem> |
170 |
<para>ecco-branch</para> |
171 |
</listitem> |
172 |
</itemizedlist> |
173 |
</listitem> |
174 |
<listitem> |
175 |
<para>Production</para> |
176 |
<itemizedlist mark="bullet"> |
177 |
<listitem> |
178 |
<para>Release1</para> |
179 |
</listitem> |
180 |
<listitem> |
181 |
<para>Release2</para> |
182 |
</listitem> |
183 |
</itemizedlist> |
184 |
</listitem> |
185 |
</itemizedlist> |
186 |
</sect2> |
187 |
--> |
188 |
|
189 |
<sect2> |
190 |
<title>Branches</title> |
191 |
|
192 |
<para>As shown in the online <ulink |
193 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated |
194 |
tree</ulink>, the MITgcm codebase is split into to two branches |
195 |
or "lines" under which development proceeds. These two lines |
196 |
are referred to as the "MAIN" and "ecco" versions of the code. |
197 |
While not identical, the bulk of the MAIN and ecco lines are |
198 |
composed of files from the same codebase. |
199 |
</para> |
200 |
|
201 |
<para>Periodically, a "Release" branch is formed from the "MAIN" |
202 |
development branch. This is done in order to create a |
203 |
relatively stable reference point for both users and developers. |
204 |
The intent is that once a relese branch has been created, only |
205 |
bug-fixes will be added to it. Meanwhile, development (which |
206 |
might "break" or otherwise render invalid the documentation, |
207 |
tutorials, and/or examples contained within a release branch) is |
208 |
allowed to continue along the MAIN and ecco lines.</para> |
209 |
</sect2> |
210 |
|
211 |
<sect2> |
212 |
<title>Tagging</title> |
213 |
|
214 |
<para>The intent of tagging is to create "known-good" |
215 |
checkpoints that developers can use as references. |
216 |
Traditionally, MITgcm tagging has maintained the following |
217 |
conventions:</para> |
218 |
|
219 |
<orderedlist> |
220 |
<listitem> |
221 |
<para>Developer checks out code into a local CVS-managed |
222 |
directory, makes various changes/additions, tests these |
223 |
edits, and eventually reaches a point where (s)he is |
224 |
satisfied that the changes form a new "useful" point in the |
225 |
evolution of the code.</para> |
226 |
</listitem> |
227 |
|
228 |
<listitem> |
229 |
<para>The developer then runs the <ulink |
230 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink> |
231 |
shell script to see if any problems are introduced. While |
232 |
not intended to be exhaustive, the test cases within the |
233 |
verification directory do provide some indication whether |
234 |
gross errors have been introduced. |
235 |
</para> |
236 |
</listitem> |
237 |
|
238 |
<listitem> |
239 |
<para>Having satisfied him- or herself that the changes are |
240 |
ready to be committed to the CVS repository, the developer |
241 |
then:</para> |
242 |
<orderedlist> |
243 |
<listitem> |
244 |
<para>adds a "checkpointXY_pre" comment (where X is a |
245 |
checkpoint number and Y is a letter) to the <ulink |
246 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink> |
247 |
file and checks it into the CVS repository</para> |
248 |
</listitem> |
249 |
<listitem> |
250 |
<para>submits the set of changes to the CVS repository |
251 |
and adds comments to <filename>tag-index</filename> |
252 |
describing what the changes are along with a matching |
253 |
"checkpointXY_post" entry</para> |
254 |
</listitem> |
255 |
</orderedlist> |
256 |
</listitem> |
257 |
</orderedlist> |
258 |
|
259 |
<para>The result of this tagging procedure is a sequence of |
260 |
development checkpoints with comments which resembles:</para> |
261 |
|
262 |
<programlisting> |
263 |
checkpoint50e_post |
264 |
o make KPP work with PTRACERS |
265 |
- fix gad_calc_rhs to call new routine kpp_transport_ptr, which is |
266 |
nearly a copy of kpp_transport_s |
267 |
- there is no analogue to SurfaceTendencyS, so I have to use |
268 |
gPtr(of the surface layer) instead |
269 |
o add a new platform SunFire+mpi (SunFire 15000) to genmake |
270 |
checkpoint50e_pre |
271 |
|
272 |
checkpoint50d_post |
273 |
o change kpp output from multiple-record state files to single-record state |
274 |
files analogous to write_state.F |
275 |
o reduce the output frequency of cg3d-related stuff to the monitor frequency, |
276 |
analogous to the cg2d-related output. |
277 |
o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512, |
278 |
so that writing to internal file fn (with length 512) fails. |
279 |
checkpoint50d_pre |
280 |
</programlisting> |
281 |
|
282 |
<para>This information can be used to refer to various stages of |
283 |
the code development. For example, bugs can be traced to |
284 |
individual sets of CVS checkins based upon their first |
285 |
appearance when comparing the results from different |
286 |
checkpoints.</para> |
287 |
|
288 |
</sect2> |
289 |
</sect1> |
290 |
|
291 |
|
292 |
<sect1 id="documentation"> |
293 |
<title>Editing the Documentation</title> |
294 |
|
295 |
<sect2 id="documentation_getting"> |
296 |
<title>Getting the Docs and Code</title> |
297 |
|
298 |
<para>The first step towards editing the documentation is to |
299 |
checkout a copy of code, docs, and build scripts from the CVS |
300 |
server using:</para> |
301 |
|
302 |
<screen> |
303 |
$ export CVS_RSH=ssh |
304 |
$ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack' |
305 |
$ mkdir scratch |
306 |
$ cvs co MITgcm manual mitgcm.org |
307 |
</screen> |
308 |
|
309 |
<para>These commands extract the necessary information from the |
310 |
CVS server and create a temporary (called |
311 |
<filename>scratch</filename>) directory for the storage of the |
312 |
HTML and other files that will be created. Please note that you |
313 |
must either create <filename>scratch</filename> as shown or edit |
314 |
the various <filename>Makefile</filename>s and scripts used to |
315 |
create the documentation.</para> |
316 |
</sect2> |
317 |
|
318 |
<sect2> |
319 |
<title>Editing</title> |
320 |
|
321 |
<para>The documentation is contained in the |
322 |
<filename>manual</filename> directory in a raw LaTeX format. |
323 |
The main document is <filename>manual.tex</filename> and it uses |
324 |
<command>\input{}</command>s to include the chapters and |
325 |
subsections.</para> |
326 |
|
327 |
<para>Since the same LaTeX source is used to produce PostScript, |
328 |
PDF, and HTML output, care should be taken to follow certain |
329 |
conventions. Two of the most important are the usage of the |
330 |
<command>\filelink{}{}</command> and |
331 |
<command>\varlink{}{}</command> commands. Both of these |
332 |
commands have been defined to simplify the connection between |
333 |
the automatically generated ("code browser") HTML and the HTML |
334 |
version of the manual produced by LaTeX2HTML. They each take |
335 |
two arguments (corresponding to the contents of the two sets of |
336 |
curly braces) which are the text that the author wishes to be |
337 |
"wrapped" within the link, and a specially formatted link thats |
338 |
relative to the <filename>MITgcm</filename> directory within the |
339 |
CVS tree.</para> |
340 |
|
341 |
<para>The result is a command that resembles either</para> |
342 |
|
343 |
<orderedlist> |
344 |
<listitem> |
345 |
<para>a reference to a variable or subroutine name such as |
346 |
<command>\varlink{tRef}{tRef}</command>, or </para> |
347 |
</listitem> |
348 |
|
349 |
<listitem> |
350 |
<para>a reference to a file such as |
351 |
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
352 |
where the absolute path to the file is of the form |
353 |
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
354 |
<para>(please note how the leading "/foo/MITgcm" |
355 |
component of the path is dropped leaving the path |
356 |
<emphasis>relative</emphasis> to the head of the code |
357 |
directory and each directory separator "/" is turned |
358 |
into a "-")</para> |
359 |
</listitem> |
360 |
</orderedlist> |
361 |
|
362 |
|
363 |
|
364 |
</sect2> |
365 |
|
366 |
<sect2> |
367 |
<title>Building</title> <para>Given the directory structure of |
368 |
<xref linkend="documentation_getting">, the entire documentation |
369 |
for the web site can be built using:</para> |
370 |
|
371 |
<screen> |
372 |
$ cd mitgcm.org/devel/buildweb |
373 |
$ make All |
374 |
</screen> |
375 |
|
376 |
<para>Which builds the PDF from the LaTeX source, creates the |
377 |
HTML output from the LaTeX source, parses the FORTRAN code base |
378 |
to produce a hyperlinked HTML version of the source, and then |
379 |
determines the cross-linking between the various HTML |
380 |
components.</para> |
381 |
|
382 |
<para>If there are no errors, the result of the build process |
383 |
(which can take 30+ minutes on a P4/2.5Ghz) will be contained |
384 |
within a single directory called |
385 |
<filename>scratch/dev_docs</filename>. This is a freshly built |
386 |
version of the entire on-line users manual. If you have the |
387 |
correct permissions, it can be directly copied to the web server |
388 |
area:</para> |
389 |
|
390 |
<screen> |
391 |
$ mv scratch/dev_docs /u/u0/httpd/html |
392 |
</screen> |
393 |
|
394 |
<para>and the update is complete.</para> |
395 |
|
396 |
</sect2> |
397 |
|
398 |
</sect1> |
399 |
|
400 |
<sect1 id="coding"> |
401 |
<title>Coding for MITgcm</title> |
402 |
|
403 |
<sect2 id="build_tools"> |
404 |
<title>Build Tools</title> |
405 |
|
406 |
<para>Many Open Source projects use the "GNU Autotools" to help |
407 |
streamline the build process for various Unix and Unix-like |
408 |
architectures. For a user, the result is the common "configure" |
409 |
(that is, "<filename>./configure && make && make |
410 |
install</filename>") commands. For MITgcm, the process is |
411 |
similar. Typical commands are:</para> |
412 |
|
413 |
<screen> |
414 |
$ genmake -mods=../code |
415 |
$ make depend |
416 |
$ make |
417 |
</screen> |
418 |
|
419 |
<para>The following sections describe the individual steps in |
420 |
the build process.</para> |
421 |
|
422 |
<sect3 id="genmake"> |
423 |
<title>The <filename>genmake2</> Utility</title> |
424 |
|
425 |
<para><emphasis>Please note that the older |
426 |
<filename>genmake</> is deprecated and will eventually |
427 |
be replaced by <filename>genmake2</>. This HOWTO only |
428 |
describes the newer tool.</emphasis></para> |
429 |
|
430 |
<para>The first step in any MITgcm build is to create a |
431 |
Unix-style <filename>Makefile</filename> which will be parsed |
432 |
by <filename>make</filename> to specify how to compile the |
433 |
MITgcm source files. For more detailed descriptions of what |
434 |
the make tools are and how they are used, please see:</para> |
435 |
|
436 |
<itemizedlist> |
437 |
<listitem> |
438 |
<para><ulink url="http://www.gnu.org/software/make/make.html">http://www.gnu.org/software/make/make.html</></para> |
439 |
</listitem> |
440 |
<listitem> |
441 |
<para><ulink url="http://www.oreilly.com/catalog/make2/">http://www.oreilly.com/catalog/make2/</></para> |
442 |
</listitem> |
443 |
</itemizedlist> |
444 |
|
445 |
<para>Due to the poor handling of soft-links and other bugs |
446 |
common with the <filename>make</filename> versions provided by |
447 |
commercial Unix vendors, GNU <filename>make</filename> |
448 |
(sometimes called <filename>gmake</filename>) should be |
449 |
preferred.</para> |
450 |
|
451 |
<para>As the name implies, <filename>genmake2</filename> |
452 |
generates a <filename>Makefile</filename>. It does so by |
453 |
first parsing the information supplied from the following |
454 |
sources</para> |
455 |
|
456 |
<orderedlist> |
457 |
<listitem> |
458 |
<para>a <filename>gm_local</filename> file in the current |
459 |
directory</para> |
460 |
</listitem> |
461 |
<listitem> |
462 |
<para>directly from command-line options</para> |
463 |
</listitem> |
464 |
<listitem> |
465 |
<para>an "options file" as specified by the command-line |
466 |
option <filename>-optfile='FILENAME'</filename></para> |
467 |
</listitem> |
468 |
</orderedlist> |
469 |
|
470 |
<para>then checking certain dependency rules (the package |
471 |
dependencies), and then writing a |
472 |
<filename>Makefile</filename> based upon the source code that |
473 |
it finds. For convenience with the various Unix shells, |
474 |
<filename>genmake2</> supports both "long"- and "shor"-style |
475 |
options. A complete list of the available options can be |
476 |
obtained from:</para> |
477 |
|
478 |
<screen> |
479 |
$ genmake2 -help |
480 |
</screen> |
481 |
|
482 |
<para>The most important options for <filename>genmake2</> |
483 |
are:</para> |
484 |
|
485 |
<variablelist> |
486 |
|
487 |
<varlistentry> |
488 |
<term><filename>--optfile=/PATH/FILENAME</></term> |
489 |
<listitem> |
490 |
<para>This specifies the "options file" that should be |
491 |
used for a particular build. The options file is a |
492 |
convenient and machine-indepenent way of specifying |
493 |
parameters such as the FORTRAN compiler |
494 |
(<filename>FC=</>), FORTRAN compiler optimization flags |
495 |
(<filename>FFLAGS=</>), and the locations of various |
496 |
platform- and/or machine-specific tools |
497 |
(eg. <filename>MAKEDEPEND=</>). As with |
498 |
<filename>genmake2</>, all options files should be |
499 |
written a BASH v1-compatible syntax. Examples of |
500 |
various options files can be found in |
501 |
<filename>$ROOTDIR/tools/build_options</>. Everyone is |
502 |
encouraged to submit their options files to the MITgcm |
503 |
project for inclusion (please send to |
504 |
<email>MITgcm-support@mitgcm.org</email>). We are |
505 |
particularly grateful for options files tested on new or |
506 |
unique platforms!</para> |
507 |
</listitem> |
508 |
</varlistentry> |
509 |
|
510 |
<varlistentry> |
511 |
<term><filename>-pdepend=/PATH/FILENAME</></term> |
512 |
<listitem> |
513 |
<para>This specifies the dependency file used for |
514 |
packages. If not specified, the default dependency file |
515 |
is <filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for |
516 |
this file is parsed on a line-by-line basis where each |
517 |
line containes either a comment ("#") or a simple |
518 |
"PKGNAME1 (+|-)PKGNAME2" pairwise rule where the "+" or |
519 |
"-" symbol specifies a "must be used with" or a "must |
520 |
not be used with" relationship, respectively. If no |
521 |
rule is specified, then it is assumed that the two |
522 |
packages are compatible and will function either with or |
523 |
without each other.</para> |
524 |
</listitem> |
525 |
</varlistentry> |
526 |
|
527 |
<varlistentry> |
528 |
<term><filename>-pdefault=PKG</></term> |
529 |
<term><filename>-pdefault='PKG1 [PKG2 PKG3 ...]'</></term> |
530 |
<listitem> |
531 |
<para>This option specifies the default set of packages |
532 |
to be used. If not set, the default package list will |
533 |
be read from |
534 |
<filename>$ROOTDIR/pkg/pkg_default</>.</para> |
535 |
</listitem> |
536 |
</varlistentry> |
537 |
|
538 |
<varlistentry> |
539 |
<term><filename></>-mods=DIR</term> |
540 |
<term><filename></>-mods='DIR1 [DIR2 ...]'</term> |
541 |
<listitem> |
542 |
<para>This option specifies a list of directories |
543 |
containing "modifications". These are files that may |
544 |
(or may not) exist in the main MITgcm source tree but |
545 |
will be overridden by any identically-named sources |
546 |
within the "MODS" directories.</para> |
547 |
</listitem> |
548 |
</varlistentry> |
549 |
|
550 |
</variablelist> |
551 |
|
552 |
<para>A successful run of <filename>genmake2</> will produce |
553 |
both a <filename>Makefile</> and a locally modified copy of |
554 |
the specified <filename>CPP_OPTIONS.h</> file. The local copy |
555 |
of <filename>CPP_OPTIONS.h</> will contain a list of |
556 |
<filename>genmake2</>-created #DEFINE and #UNDEF statements |
557 |
that reflect the list of packages that will be compiled into |
558 |
the code (either directly through enable/disable/defaults |
559 |
options or indirectly through dependencies).</para> |
560 |
|
561 |
<para>In general, it is best to use <filename>genmake2</> on a |
562 |
"clean" directory that is free of all source |
563 |
(*.[F,f],*.[F,f]90) and header (*.h,*.inc) files. Generally, |
564 |
this can be accomplished in an "un-clean" directory by running |
565 |
"make CLEAN" followed by "make makefile".</para> |
566 |
|
567 |
</sect3> |
568 |
|
569 |
<sect3 id="makefile_use"> |
570 |
<title>Using <filename>Makefile</></title> |
571 |
|
572 |
<para>Once a <filename>Makefile</> has been created, one can |
573 |
build the executable using:</para> |
574 |
|
575 |
<screen> |
576 |
$ make CLEAN |
577 |
$ make depend |
578 |
$ make |
579 |
</screen> |
580 |
|
581 |
<para>The "make CLEAN" step will remove any local source |
582 |
files, include files, and links. It is strongly recommended |
583 |
for "un-clean" directories which may contain the (partial?) |
584 |
results of previous builds. Such "debris" can interfere with |
585 |
the next stage of the build.</para> |
586 |
|
587 |
<para>The "make depend" step will create a large number of |
588 |
symbolic links from the local directory to the source file |
589 |
locations. It also parses these files and creates an |
590 |
extensive list of dependencies within the |
591 |
<filename>Makefile</> itself. The links that exist at this |
592 |
stage are mostly "large F" files (*.F and *.F90) that need to |
593 |
be processed by a C preprocessor ("CPP"). |
594 |
</para> |
595 |
|
596 |
<para>The final "make" invokes the C preprocessor to produce |
597 |
the "little f" files (*.f and *.f90) and then compiles them to |
598 |
object code using the specified FORTRAN compiler and options. |
599 |
An intermediate script is often used during this stage to |
600 |
further process (usually, make simple substitutions) custom |
601 |
definitions such as variable types within the source files. |
602 |
This additional stage is necessary in order to overcome some |
603 |
of the inconsistencies in the sizes of objects (bytes) between |
604 |
different compilers.</para> |
605 |
|
606 |
<para>Please report compilation failures or other problems to |
607 |
<email>MITgcm-support@mitgcm.org</email>.</para> |
608 |
|
609 |
</sect3> |
610 |
|
611 |
</sect2> |
612 |
|
613 |
<sect2 id="verification"> |
614 |
<title>The Verification Suite</title> |
615 |
|
616 |
<para>The MITgcm CVS tree (within the |
617 |
<filename>$ROOTDIR/verification/</> directory) includes more |
618 |
than a dozen examples intended for regression testing. Each one |
619 |
of these example directories contains "known-good" output files |
620 |
along with all the input (including both code and data files) |
621 |
required for their re-calculation. These example directories |
622 |
are further broken down into sets of subdirectories |
623 |
(eg. <filename>/input</>, <filename>/code</>) intended to |
624 |
expedite the testing process.</para> |
625 |
|
626 |
<sect3 id="testreport"> |
627 |
<title>The <filename>testreport</> Utility</title> |
628 |
|
629 |
<para>Also included in <filename>$ROOTDIR/verification/</> are |
630 |
shell scripts for automated testing. The newest script (which |
631 |
was written to work with <filename>genmake2</>) is called |
632 |
<filename>testreport</>. Ths script can be used to build the |
633 |
different versions of the MITgcm code, run the various |
634 |
examples, compare the output, and (if specified) email the |
635 |
results of each one of these tests to a central |
636 |
repository.</para> |
637 |
|
638 |
<para>The <filename>testreport</> script accepts a number of |
639 |
command-line options which can be listed using the |
640 |
<filename>-help</> option. The most important ones are:</para> |
641 |
|
642 |
<variablelist> |
643 |
|
644 |
<varlistentry> |
645 |
<term><filename>-tdir TESTDIR</></term> |
646 |
<term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term> |
647 |
<listitem> |
648 |
<para>This option specifies the test directory or list |
649 |
of test directories that should be used. Each of these |
650 |
entries should exactly (note: they're case sensitive!) |
651 |
match the names of directries in |
652 |
<filename>$ROOTDIR/verification/</>. If this option is |
653 |
omitted, then all directories that are properly |
654 |
formatted (that is, containing an <filename>input</> |
655 |
sub-directory and example output) will be used.</para> |
656 |
</listitem> |
657 |
</varlistentry> |
658 |
|
659 |
<varlistentry> |
660 |
<term><filename>-optfile=/PATH/FILENAME</></term> |
661 |
<term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term> |
662 |
<listitem> |
663 |
<para>This specifies a list of "options files" that will |
664 |
be passed to <filename>genmake2</>. If multiple options |
665 |
files are used (say, to test different compilers or |
666 |
different sets of options for the same compiler), then |
667 |
each options file will be used with each of the test |
668 |
directories.</para> |
669 |
</listitem> |
670 |
</varlistentry> |
671 |
|
672 |
<varlistentry> |
673 |
<term><filename>-addr EMAIL</></term> |
674 |
<term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term> |
675 |
<listitem> |
676 |
<para>Send the results (namely, <filename>output.txt</>, |
677 |
<filename>gm_local</>, <filename>gm_state</>, and |
678 |
<filename>Makefile</>) to the specified email addresses. |
679 |
The results are gzipped, placed in a tar file, MIME |
680 |
encoded, and sent to an @mitgcm.org address. If no |
681 |
email addresses are specified, no mail is sent.</para> |
682 |
</listitem> |
683 |
</varlistentry> |
684 |
|
685 |
</variablelist> |
686 |
|
687 |
<para>The <filename>testreport</> script will write progress |
688 |
to the screen (stdout) as it runs. In addition, it will |
689 |
create a <filename>summary.txt</> file that contains a brief |
690 |
comparison of the current output with the "known-good" |
691 |
output.</para> |
692 |
|
693 |
</sect3> |
694 |
|
695 |
</sect2> |
696 |
|
697 |
<sect2 id="packages"> |
698 |
<title>Creating MITgcm Packages</title> |
699 |
|
700 |
<para>Optional parts of code have been separated from the MITgcmUV |
701 |
core driver code and organised into packages. The packaging |
702 |
structure provides a mechanism for maintaining suites of code, |
703 |
specific to particular classes of problems, in a way that is |
704 |
cleanly separated from the generic fluid dynamical |
705 |
engine.</para> |
706 |
|
707 |
<para>The MITgcmUV packaging structure is described below using |
708 |
generic package names ${pkg}. A concrete examples of a package |
709 |
is the code for implementing GM/Redi mixing. This code uses the |
710 |
package name</para> |
711 |
|
712 |
</sect2> |
713 |
|
714 |
</sect1> |
715 |
|
716 |
<sect1> |
717 |
<title>Chris's Notes...</title> |
718 |
|
719 |
<programlisting> |
720 |
MITgcmUV Packages |
721 |
================= |
722 |
|
723 |
Optional parts of code are separated from |
724 |
the MITgcmUV core driver code and organised into |
725 |
packages. The packaging structure provides a mechanism for |
726 |
maintaining suites of code, specific to particular |
727 |
classes of problem, in a way that is cleanly |
728 |
separated from the generic fluid dynamical engine. |
729 |
|
730 |
The MITgcmUV packaging structure is describe |
731 |
below using generic package names ${pkg}. |
732 |
A concrete examples of a package is the code |
733 |
for implementing GM/Redi mixing. This code uses |
734 |
the package name |
735 |
* ${PKG} = GMREDI |
736 |
* ${pkg} = gmredi |
737 |
* ${Pkg} = gmRedi |
738 |
|
739 |
Package states |
740 |
============== |
741 |
|
742 |
Packages can be any one of four states, included, |
743 |
excluded, enabled, disabled as follows: |
744 |
|
745 |
included(excluded) compile time state which |
746 |
includes(excludes) package |
747 |
code and routine calls from |
748 |
compilation/linking etc... |
749 |
|
750 |
enabled(disabled) run-time state which |
751 |
enables(disables) package code |
752 |
execution. |
753 |
|
754 |
Every call to a ${pkg}_... routine from outside the package |
755 |
should be placed within both a |
756 |
#ifdef ALLOW_${PKG} ... block and a |
757 |
if ( use${Pkg} ) ... then block. |
758 |
Package states are generally not expected to change during |
759 |
a model run. |
760 |
|
761 |
Package structure |
762 |
================= |
763 |
|
764 |
o Each package gets its runtime configuration |
765 |
parameters from a file named "data.${pkg}" |
766 |
Package runtime config. options are imported |
767 |
into a common block held in a header file |
768 |
called "${PKG}.h". |
769 |
|
770 |
o The core driver part of the model can check |
771 |
for runtime enabling or disabling of individual packages |
772 |
through logical flags use${Pkg}. |
773 |
The information is loaded from a |
774 |
global package setup file called "data.pkg". |
775 |
The use${Pkg} flags are not used within |
776 |
individual packages. |
777 |
|
778 |
o Included in "${PKG}.h" is a logical flag |
779 |
called ${Pkg}IsOn. The "${PKG}.h" header file can be imported |
780 |
by other packages to check dependencies and requirements |
781 |
from other packages ( see "Package Boot Sequence" section). |
782 |
NOTE: This procedure is not presently implemented, |
783 |
----- neither for kpp nor for gmRedi. |
784 |
|
785 |
CPP Flags |
786 |
========= |
787 |
|
788 |
1. Within the core driver code flags of the form |
789 |
ALLOW_${PKG} are used to include or exclude |
790 |
whole packages. The ALLOW_${PKG} flags are included |
791 |
from a PKG_CPP_OPTIONS block which is currently |
792 |
held in-line in the CPP_OPTIONS.h header file. |
793 |
e.g. |
794 |
|
795 |
Core model code ..... |
796 |
|
797 |
#include "CPP_OPTIONS.h" |
798 |
: |
799 |
: |
800 |
: |
801 |
|
802 |
#ifdef ALLOW_${PKG} |
803 |
if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...) |
804 |
#endif |
805 |
|
806 |
2. Within an individual package a header file, |
807 |
"${PKG}_OPTIONS.h", is used to set CPP flags |
808 |
specific to that package. It is not recommended |
809 |
to include this file in "CPP_OPTIONS.h". |
810 |
|
811 |
|
812 |
Package Boot Sequence |
813 |
===================== |
814 |
|
815 |
Calls to package routines within the core code timestepping |
816 |
loop can vary. However, all packages follow a required |
817 |
"boot" sequence outlined here: |
818 |
|
819 |
1. S/R PACKAGES_BOOT() |
820 |
: |
821 |
CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... ) |
822 |
|
823 |
|
824 |
2. S/R PACKAGES_READPARMS() |
825 |
: |
826 |
#ifdef ALLOW_${PKG} |
827 |
if ( use${Pkg} ) |
828 |
& CALL ${PKG}_READPARMS( retCode ) |
829 |
#endif |
830 |
|
831 |
2. S/R PACKAGES_CHECK() |
832 |
: |
833 |
#ifdef ALLOW_${PKG} |
834 |
if ( use${Pkg} ) |
835 |
& CALL ${PKG}_CHECK( retCode ) |
836 |
#else |
837 |
if ( use${Pkg} ) |
838 |
& CALL PACKAGES_CHECK_ERROR('${PKG}') |
839 |
#endif |
840 |
|
841 |
3. S/R PACKAGES_INIT() |
842 |
: |
843 |
#ifdef ALLOW_${PKG} |
844 |
if ( use${Pkg} ) |
845 |
& CALL ${PKG}_INIT( retCode ) |
846 |
#endif |
847 |
|
848 |
|
849 |
Description |
850 |
=========== |
851 |
|
852 |
- ${PKG}_READPARMS() |
853 |
is responsible for reading |
854 |
in the package parameters file data.${pkg}, and storing |
855 |
the package parameters in "${PKG}.h". |
856 |
-> called in INITIALISE_FIXED |
857 |
|
858 |
- ${PKG}_CHECK() |
859 |
is responsible for validating |
860 |
basic package setup and inter-package dependencies. |
861 |
${PKG}_CHECK can import other package parameters it may |
862 |
need to check. This is done through header files "${PKG}.h". |
863 |
It is assumed that parameters owned by other packages |
864 |
will not be reset during ${PKG}_CHECK(). |
865 |
-> called in INITIALISE_FIXED |
866 |
|
867 |
- ${PKG}_INIT() |
868 |
is responsible for completing the |
869 |
internal setup of a package. This routine is called after |
870 |
the core model state has been completely initialised |
871 |
but before the core model timestepping starts. |
872 |
-> called in INITIALISE_VARIA |
873 |
|
874 |
Summary |
875 |
======= |
876 |
|
877 |
- CPP options: |
878 |
----------------------- |
879 |
* ALLOW_${PKG} include/exclude package for compilation |
880 |
|
881 |
- FORTRAN logical: |
882 |
----------------------- |
883 |
* use${Pkg} enable package for execution at runtime |
884 |
-> declared in PARAMS.h |
885 |
* ${Pkg}IsOn for package cross-dependency check |
886 |
-> declared in ${PKG}.h |
887 |
N.B.: Not presently used! |
888 |
|
889 |
- header files |
890 |
----------------------- |
891 |
* ${PKG}_OPTIONS.h has further package-specific CPP options |
892 |
* ${PKG}.h package-specific common block variables, fields |
893 |
|
894 |
- FORTRAN source files |
895 |
----------------------- |
896 |
* ${pkg}_readparms.F reads parameters from file data.${pkg} |
897 |
* ${pkg}_check.F checks package dependencies and consistencies |
898 |
* ${pkg}_init.F initialises package-related fields |
899 |
* ${pkg}_... .F package source code |
900 |
|
901 |
- parameter file |
902 |
----------------------- |
903 |
* data.${pkg} parameter file |
904 |
</programlisting> |
905 |
|
906 |
</sect1> |
907 |
|
908 |
|
909 |
</article> |