2 |
|
|
3 |
<article id="MITgcm-Development-HOWTO"> |
<article id="MITgcm-Development-HOWTO"> |
4 |
|
|
|
<!-- |
|
|
Build commands: |
|
|
db2ps -d ldp.dsl devel_HOWTO.sgml |
|
|
db2pdf -d ldp.dsl devel_HOWTO.sgml |
|
|
db2html -d ./ldp.dsl devel_HOWTO.sgml |
|
|
db2html -u -d ./ldp.dsl devel_HOWTO.sgml |
|
|
--> |
|
|
|
|
5 |
<articleinfo> |
<articleinfo> |
6 |
<title>MITgcm Development HOWTO</title> |
<title>MITgcm Development HOWTO</title> |
7 |
|
|
22 |
Initial version. |
Initial version. |
23 |
</revremark> |
</revremark> |
24 |
</revision> |
</revision> |
25 |
|
<revision> |
26 |
|
<revnumber>0.02</revnumber> |
27 |
|
<date>2010-01-21</date> |
28 |
|
<authorinitials>jmc</authorinitials> |
29 |
|
<revremark> |
30 |
|
update links. |
31 |
|
</revremark> |
32 |
|
</revision> |
33 |
|
<revision> |
34 |
|
<revnumber>0.03</revnumber> |
35 |
|
<date>2010-04-25</date> |
36 |
|
<authorinitials>jmc</authorinitials> |
37 |
|
<revremark> |
38 |
|
Add subsection "Developer settings" (under CVS Repository). |
39 |
|
</revremark> |
40 |
|
</revision> |
41 |
</revhistory> |
</revhistory> |
42 |
|
|
43 |
<abstract> |
<abstract> |
53 |
<sect2> |
<sect2> |
54 |
<title>New Versions of This Document</title> <para>You can |
<title>New Versions of This Document</title> <para>You can |
55 |
obtain the latest version of this document <ulink |
obtain the latest version of this document <ulink |
56 |
url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in |
url="http://mitgcm.org/public/docs.html">online</ulink> in |
57 |
various formats.</para> |
various formats.</para> |
58 |
</sect2> |
</sect2> |
59 |
<sect2> |
<sect2> |
71 |
<sect2> |
<sect2> |
72 |
<title>User Manual</title> |
<title>User Manual</title> |
73 |
|
|
74 |
<para>Before jumping into |
<para>Before jumping into development, please familiarize yourself with |
75 |
development, please familiarize yourself with the MITgcm user |
the <ulink url="http://mitgcm.org/public/docs.html"> MITgcm user manual |
76 |
manual which is available <ulink |
</ulink>. This document contains volumes of useful information and is |
77 |
url="http://mitgcm.org/">on the main web page</ulink>. This |
included here by reference.</para> |
|
document contains volumes of useful information and is included |
|
|
here by reference.</para> |
|
78 |
|
|
79 |
<para>Also, a "snapshot" or<ulink |
<!-- |
80 |
|
<para>Also, a "snapshot" or <ulink |
81 |
url="http://mitgcm.org/dev_docs/">development version</ulink> of |
url="http://mitgcm.org/dev_docs/">development version</ulink> of |
82 |
the user manual may be available, though this is only put on the |
the user manual may be available, though this is only put on the |
83 |
web for testing purposes.</para> |
web for testing purposes.</para> |
84 |
|
--> |
85 |
</sect2> |
</sect2> |
86 |
|
|
87 |
<sect2> |
<sect2> |
110 |
|
|
111 |
<sect1 id="cvs"> |
<sect1 id="cvs"> |
112 |
<title>CVS Repository</title> |
<title>CVS Repository</title> |
113 |
|
|
114 |
<sect2> |
<sect2> |
115 |
<title>Layout</title> |
<title>Layout</title> |
116 |
|
|
121 |
others. The tree currently resembles:</para> |
others. The tree currently resembles:</para> |
122 |
|
|
123 |
<programlisting>gcmpack/ |
<programlisting>gcmpack/ |
|
MITgcm-contrib contributed code |
|
124 |
CS-regrid goes into utils |
CS-regrid goes into utils |
125 |
cvspolicy.html -save- |
CVSROOT -hidden- |
|
CVSROOT -save- |
|
|
development experimental stuff |
|
|
manual -save- |
|
|
misc -?- |
|
126 |
|
|
127 |
MITgcm code |
MITgcm code |
128 |
adjoint fold into genmake |
bin empty |
129 |
bin stub for ecco build |
doc basic developpment documentation |
130 |
compare01 old from 20th century |
eesupp execution environment support code (wrapper) |
131 |
diags timeave f77 in pkgs now |
exe empty |
132 |
doc tags -- connect to real docs? |
jobs runtime shell scripts for |
133 |
eesupp cnh? |
various platforms (not maintained) |
134 |
exe ecco user build |
lsopt line search |
135 |
,- jobs runtime shell scripts for |
model main dynamics (core) |
136 |
| various platforms |
optim line search interface |
137 |
| lsopt line search |
pkg alternate and optional numerics, etc. |
138 |
m| model main dynamics (core) |
tools scripts to build (and test) |
139 |
e| optimization_drivers ? |
utils pre/post processing tools (matlab, ..) |
140 |
r| optim line search interface |
verification standard regression tests + examples |
141 |
g| pkg alternate and optional numerics, etc. |
+ documented examples (tutorials) |
142 |
e|- tools |
tutorial_examples (only in release1 branch) |
|
?| tutorial_examples documented tests |
|
|
| only populated on release1 branch |
|
|
| and not validated during "testscript" |
|
|
'- utils |
|
|
verification std tests |
|
143 |
|
|
144 |
|
MITgcm_contrib contributed code |
145 |
|
|
146 |
mitgcmdoc -> manual -remove- |
acesgrid.org build acesgrid web site |
147 |
|
development experimental stuff |
148 |
|
gcmpack an old back-up copy ? |
149 |
|
gfd_lab -?- |
150 |
|
manual -save- |
151 |
|
misc -?- |
152 |
mitgcm.org build web site |
mitgcm.org build web site |
153 |
|
mitgcmdoc -> manual -remove- |
154 |
models -?- |
models -?- |
155 |
packages -?- |
packages -?- |
156 |
|
pdfs some pdfs |
157 |
|
planetinabottle.org unfinished web site |
158 |
preprocess -?- |
preprocess -?- |
159 |
tmp -?- |
tmp -?- |
160 |
|
www.ecco-group.org build ecco web site ? |
161 |
</programlisting> |
</programlisting> |
162 |
|
|
163 |
|
<!-- |
164 |
<para>Efforts are underway to reduce the complexity.</para> |
<para>Efforts are underway to reduce the complexity.</para> |
165 |
|
--> |
166 |
|
|
167 |
</sect2> |
</sect2> |
168 |
|
|
201 |
<title>Branches</title> |
<title>Branches</title> |
202 |
|
|
203 |
<para>As shown in the online <ulink |
<para>As shown in the online <ulink |
204 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph"> |
205 |
tree</ulink>, the MITgcm codebase is split into to two branches |
ViewCVS-generated tree</ulink>, the MITgcm codebase is split into |
206 |
or "lines" under which development proceeds. These two lines |
branches or "lines" under which development proceeds. The main line |
207 |
are referred to as the "MAIN" and "ecco" versions of the code. |
of development is referred to as the "MAIN" version of the code. |
|
While not identical, the bulk of the MAIN and ecco lines are |
|
|
composed of files from the same codebase. |
|
208 |
</para> |
</para> |
209 |
|
|
210 |
<para>Periodically, a "Release" branch is formed from the "MAIN" |
<para>Periodically, a "Release" branch is formed from the "MAIN" |
211 |
development branch. This is done in order to create a |
development branch. This is done in order to create a relatively stable |
212 |
relatively stable reference point for both users and developers. |
reference point for both users and developers. The intent is that once a |
213 |
The intent is that once a relese branch has been created, only |
release branch has been created, only bug-fixes will be added to it. |
214 |
bug-fixes will be added to it. Meanwhile, development (which |
Meanwhile, development (which might "break" or otherwise render invalid |
215 |
might "break" or otherwise render invalid the documentation, |
the documentation, tutorials, and/or examples contained within a release |
216 |
tutorials, and/or examples contained within a release branch) is |
branch) is allowed to continue along the MAIN line.</para> |
217 |
allowed to continue along the MAIN and ecco lines.</para> |
</sect2> |
218 |
|
|
219 |
|
<sect2> |
220 |
|
<title> Developer settings </title> |
221 |
|
|
222 |
|
<para>CVS is a convenient tool to keep up-to-date a personal copy of the |
223 |
|
MITgcm code (see: <ulink url="http://mitgcm.org/public/using_cvs.html"> |
224 |
|
using CVS </ulink>). The same tool is used by developers to |
225 |
|
incorporate any change into the repository. However, this later |
226 |
|
function requires specific settings, as detailed here after:</para> |
227 |
|
<orderedlist> |
228 |
|
<listitem> |
229 |
|
<para> You will need an account (loggin access) to the server |
230 |
|
"mitgcm.org" with the proper group setting (e.g., |
231 |
|
group "gcmctrb" to add/modify code into MITgcm_contrib). |
232 |
|
This kind of account is granted only upon well motivated request. |
233 |
|
The access to the server mitgcm.org is through ssh-key authorization |
234 |
|
which will need to be set properly on both side (on your local machine |
235 |
|
and on your server account). You need to be able to |
236 |
|
to ssh to mitgcm.org (or <filename>ssh MY_USER_NAME@mitgcm.org</filename> |
237 |
|
in case of different user-name on both sides) to proceed further.</para> |
238 |
|
</listitem> |
239 |
|
|
240 |
|
<listitem> |
241 |
|
<para> You need to register to the |
242 |
|
<ulink url="http://mitgcm.org/mailman/listinfo/mitgcm-cvs"> |
243 |
|
mitgcm-cvs </ulink> mailing list. |
244 |
|
This ensures that other developers will receive email notification |
245 |
|
when you make changes; you will also receive as well such email |
246 |
|
when others make changes to the repository. |
247 |
|
</para> |
248 |
|
</listitem> |
249 |
|
|
250 |
|
<listitem> |
251 |
|
<para> It is highly recommended that you register also to the |
252 |
|
<ulink url="http://mitgcm.org/mailman/listinfo/mitgcm-devel"> |
253 |
|
mitgcm-devel </ulink> mailing list (expect a short delay for |
254 |
|
this request to be processed). |
255 |
|
This list is intended for developer discussions. |
256 |
|
</para> |
257 |
|
</listitem> |
258 |
|
|
259 |
|
<listitem> |
260 |
|
<para> The standard anonymous mode (using "cvsanon", as mentionned |
261 |
|
<ulink url="http://mitgcm.org/public/source_code.html"> |
262 |
|
here </ulink>) does not allow check-in ("cvs commit") permission. |
263 |
|
Instead, you will need to set our CVS environment as follow:</para> |
264 |
|
<screen> |
265 |
|
$ export CVS_RSH=ssh |
266 |
|
$ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack' |
267 |
|
</screen> |
268 |
|
<para> After downloading a directory, e.g.: <filename>myCopy</filename>, |
269 |
|
from the CVS repository (e.g., |
270 |
|
<filename>MITgcm_contrib/thisPart</filename>) using the command:</para> |
271 |
|
<screen> |
272 |
|
$ cvs co -P -d myCopy MITgcm_contrib/thisPart |
273 |
|
</screen> |
274 |
|
<para> the type of CVS environment which has been used |
275 |
|
is stored in the file <filename>myCopy/CVS/Root</filename> |
276 |
|
and makes it difficult to re-use, for cvs-commit purpose, |
277 |
|
a cvs local copy (<filename>myCopy</filename>) which was obtained |
278 |
|
using the CVS anonymous mode.</para> |
279 |
|
</listitem> |
280 |
|
|
281 |
|
<listitem> |
282 |
|
<para> At this stage, you should be able to send your modified source |
283 |
|
file (e.g., <filename>src_file</filename>) from your local copy directory |
284 |
|
(<filename>myCopy</filename>) to the CVS repository |
285 |
|
(<filename>MITgcm_contrib/thisPart</filename>) using the command |
286 |
|
"cvs commit":</para> |
287 |
|
<screen> |
288 |
|
$ cd myCopy |
289 |
|
$ cvs -n update (optional; check if new changes have been made) |
290 |
|
$ cvs diff src_file (optional; list your changes) |
291 |
|
$ cvs commit src_file |
292 |
|
</screen> |
293 |
|
<para> It is essential that you provide a short description of the |
294 |
|
changes you made to <filename>src_file</filename> as you check-in |
295 |
|
this file (the "cvs commit" command automatically opens your standard |
296 |
|
editor for this purpose).</para> |
297 |
|
</listitem> |
298 |
|
|
299 |
|
</orderedlist> |
300 |
|
|
301 |
</sect2> |
</sect2> |
302 |
|
|
303 |
<sect2> |
<sect2> |
304 |
<title>Tagging</title> |
<title>Main code development</title> |
305 |
|
<para>(formerly named "Tagging" ; this section needs an update)</para> |
306 |
<para>The intent of tagging is to create "known-good" |
|
307 |
checkpoints that developers can use as references. |
<para>The intent of tagging is to create "known-good" checkpoints that |
308 |
Traditionally, MITgcm tagging has maintained the following |
developers can use as references. Traditionally, MITgcm tagging has |
309 |
conventions:</para> |
maintained the following conventions:</para> |
310 |
|
|
311 |
<orderedlist> |
<orderedlist> |
312 |
<listitem> |
<listitem> |
313 |
<para>Developer checks out code into a local CVS-managed |
<para>Developer checks out code into a local CVS-managed directory, |
314 |
directory, makes various changes/additions, tests these |
makes various changes/additions, tests these edits, and eventually |
315 |
edits, and eventually reaches a point where (s)he is |
reaches a point where (s)he is satisfied that the changes form a new |
316 |
satisfied that the changes form a new "useful" point in the |
"useful" point in the evolution of the code.</para> |
|
evolution of the code.</para> |
|
317 |
</listitem> |
</listitem> |
318 |
|
|
319 |
<listitem> |
<listitem> |
320 |
<para>The developer then runs the <ulink |
<para>The developer then runs the <ulink |
321 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport"> |
322 |
shell script to see if any problems are introduced. While |
testreport</ulink> shell script to see if any problems are introduced. |
323 |
not intended to be exhaustive, the test cases within the |
While not intended to be exhaustive, the test cases within the |
324 |
verification directory do provide some indication whether |
verification directory do provide some indication whether gross errors |
325 |
gross errors have been introduced. |
have been introduced. |
326 |
</para> |
</para> |
327 |
</listitem> |
</listitem> |
328 |
|
|
332 |
then:</para> |
then:</para> |
333 |
<orderedlist> |
<orderedlist> |
334 |
<listitem> |
<listitem> |
335 |
<para>adds a "checkpointXY_pre" comment (where X is a |
<para>adds a "checkpointXY_pre" comment (where X is a checkpoint |
336 |
checkpoint number and Y is a letter) to the <ulink |
number and Y is a letter) to the <ulink |
337 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index"> |
338 |
file and checks it into the CVS repository</para> |
tag-index</ulink> file and checks it into the CVS |
339 |
|
repository</para> |
340 |
</listitem> |
</listitem> |
341 |
<listitem> |
<listitem> |
342 |
<para>submits the set of changes to the CVS repository |
<para>submits the set of changes to the CVS repository and adds |
343 |
and adds comments to <filename>tag-index</filename> |
comments to <filename>tag-index</filename> describing what the |
344 |
describing what the changes are along with a matching |
changes are along with a matching "checkpointXY_post" entry</para> |
|
"checkpointXY_post" entry</para> |
|
345 |
</listitem> |
</listitem> |
346 |
</orderedlist> |
</orderedlist> |
347 |
</listitem> |
</listitem> |
348 |
</orderedlist> |
</orderedlist> |
349 |
|
|
350 |
<para>The result of this tagging procedure is a sequence of |
<para>The result of this tagging procedure is a sequence of development |
351 |
development checkpoints with comments which resembles:</para> |
checkpoints with comments which resembles:</para> |
352 |
|
|
353 |
<programlisting> |
<programlisting> |
354 |
checkpoint50e_post |
checkpoint50e_post |
370 |
checkpoint50d_pre |
checkpoint50d_pre |
371 |
</programlisting> |
</programlisting> |
372 |
|
|
373 |
<para>This information can be used to refer to various stages of |
<para>This information can be used to refer to various stages of the code |
374 |
the code development. For example, bugs can be traced to |
development. For example, bugs can be traced to individual sets of CVS |
375 |
individual sets of CVS checkins based upon their first |
checkins based upon their first appearance when comparing the results from |
376 |
appearance when comparing the results from different |
different checkpoints.</para> |
|
checkpoints.</para> |
|
377 |
|
|
378 |
</sect2> |
</sect2> |
379 |
</sect1> |
</sect1> |
380 |
|
|
381 |
|
|
|
<sect1 id="documentation"> |
|
|
<title>Editing the Documentation</title> |
|
|
|
|
|
<sect2 id="documentation_getting"> |
|
|
<title>Getting the Docs and Code</title> |
|
|
|
|
|
<para>The first step towards editing the documentation is to |
|
|
checkout a copy of code, docs, and build scripts from the CVS |
|
|
server using:</para> |
|
|
|
|
|
<screen> |
|
|
$ export CVS_RSH=ssh |
|
|
$ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack' |
|
|
$ mkdir scratch |
|
|
$ cvs co MITgcm manual mitgcm.org |
|
|
</screen> |
|
|
|
|
|
<para>These commands extract the necessary information from the |
|
|
CVS server and create a temporary (called |
|
|
<filename>scratch</filename>) directory for the storage of the |
|
|
HTML and other files that will be created. Please note that you |
|
|
must either create <filename>scratch</filename> as shown or edit |
|
|
the various <filename>Makefile</filename>s and scripts used to |
|
|
create the documentation.</para> |
|
|
</sect2> |
|
|
|
|
|
<sect2> |
|
|
<title>Editing the Documentation</title> |
|
|
|
|
|
<para>The documentation is contained in the |
|
|
<filename>manual</filename> directory in a raw LaTeX format. |
|
|
The main document is <filename>manual.tex</filename> and it uses |
|
|
<command>\input{}</command>s to include the chapters and |
|
|
subsections.</para> |
|
|
|
|
|
<para>Since the same LaTeX source is used to produce PostScript, |
|
|
PDF, and HTML output, care should be taken to follow certain |
|
|
conventions. Two of the most important are the usage of the |
|
|
<command>\filelink{}{}</command> and |
|
|
<command>\varlink{}{}</command> commands. Both of these |
|
|
commands have been defined to simplify the connection between |
|
|
the automatically generated ("code browser") HTML and the HTML |
|
|
version of the manual produced by LaTeX2HTML. They each take |
|
|
two arguments (corresponding to the contents of the two sets of |
|
|
curly braces) which are the text that the author wishes to be |
|
|
"wrapped" within the link, and a specially formatted link thats |
|
|
relative to the <filename>MITgcm</filename> directory within the |
|
|
CVS tree.</para> |
|
|
|
|
|
<para>The result is a command that resembles either</para> |
|
|
|
|
|
<orderedlist> |
|
|
<listitem> |
|
|
<para>a reference to a variable or subroutine name such as |
|
|
<command>\varlink{tRef}{tRef}</command>, or </para> |
|
|
</listitem> |
|
|
|
|
|
<listitem> |
|
|
<para>a reference to a file such as |
|
|
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
|
|
where the absolute path to the file is of the form |
|
|
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
|
|
<para>(please note how the leading "/foo/MITgcm" |
|
|
component of the path is dropped leaving the path |
|
|
<emphasis>relative</emphasis> to the head of the code |
|
|
directory and each directory separator "/" is turned |
|
|
into a "-")</para> |
|
|
</listitem> |
|
|
</orderedlist> |
|
|
|
|
|
|
|
|
|
|
|
</sect2> |
|
|
|
|
|
<sect2> |
|
|
<title>Building the Documentation</title> |
|
|
|
|
|
<para>Given the directory structure of <xref |
|
|
linkend="documentation_getting">, the entire documentation for the web |
|
|
site can be built using:</para> |
|
|
|
|
|
<screen> |
|
|
$ cd mitgcm.org/devel/buildweb |
|
|
$ make All |
|
|
</screen> |
|
|
|
|
|
<para>Which builds the PDF from the LaTeX source, creates the |
|
|
HTML output from the LaTeX source, parses the FORTRAN code base |
|
|
to produce a hyperlinked HTML version of the source, and then |
|
|
determines the cross-linking between the various HTML |
|
|
components.</para> |
|
|
|
|
|
<para>If there are no errors, the result of the build process |
|
|
(which can take 30+ minutes on a P4/2.5Ghz) will be contained |
|
|
within a single directory called |
|
|
<filename>scratch/dev_docs</filename>. This is a freshly built |
|
|
version of the entire on-line users manual. If you have the |
|
|
correct permissions, it can be directly copied to the web server |
|
|
area:</para> |
|
|
|
|
|
<screen> |
|
|
$ mv scratch/dev_docs /u/u0/httpd/html |
|
|
</screen> |
|
|
|
|
|
<para>and the update is complete.</para> |
|
|
|
|
|
</sect2> |
|
|
|
|
|
</sect1> |
|
|
|
|
382 |
<sect1 id="coding"> |
<sect1 id="coding"> |
383 |
<title>Coding for MITgcm</title> |
<title>Coding for MITgcm</title> |
384 |
|
|
392 |
For MITgcm, the process is similar. Typical commands are:</para> |
For MITgcm, the process is similar. Typical commands are:</para> |
393 |
|
|
394 |
<screen> |
<screen> |
395 |
$ genmake -mods=../code |
$ genmake -mods=../code |
396 |
$ make depend |
$ make depend |
397 |
$ make |
$ make |
398 |
</screen> |
</screen> |
399 |
|
|
400 |
<para>The following sections describe the individual steps in the build |
<para>The following sections describe the individual steps in the build |
428 |
simple as:</para> |
simple as:</para> |
429 |
|
|
430 |
<screen> |
<screen> |
431 |
$ genmake2 -mods=../code |
$ genmake2 -mods=../code |
432 |
</screen> |
</screen> |
433 |
|
|
434 |
<para>However, some systems (particularly commercial Unixes that lack a |
<para>However, some systems (particularly commercial Unixes that lack a |
437 |
the following: </para> |
the following: </para> |
438 |
|
|
439 |
<screen> |
<screen> |
440 |
$ /usr/bin/sh genmake2 -make=gmake -mods=../code |
$ /usr/bin/sh genmake2 -make=gmake -mods=../code |
441 |
$ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code |
$ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code |
442 |
</screen> |
</screen> |
443 |
|
|
444 |
<para>The genmake2 code has been written in a Bourne and BASH (v1) |
<para>The genmake2 code has been written in a Bourne and BASH (v1) |
451 |
|
|
452 |
<orderedlist> |
<orderedlist> |
453 |
<listitem> |
<listitem> |
454 |
<para>a <filename>gm_local</filename> file in the current |
<para>a <filename>gemake_local</filename> file in the current |
455 |
directory</para> |
directory</para> |
456 |
</listitem> |
</listitem> |
457 |
<listitem> |
<listitem> |
471 |
from:</para> |
from:</para> |
472 |
|
|
473 |
<screen> |
<screen> |
474 |
$ genmake2 -help |
$ genmake2 -help |
475 |
</screen> |
</screen> |
476 |
|
|
477 |
<para>The most important options for <filename>genmake2</> are:</para> |
<para>The most important options for <filename>genmake2</> are:</para> |
527 |
specified, then it is assumed that the two packages are compatible |
specified, then it is assumed that the two packages are compatible |
528 |
and will function either with or without each other.</para> |
and will function either with or without each other.</para> |
529 |
</listitem> |
</listitem> |
|
|
|
530 |
</varlistentry> |
</varlistentry> |
531 |
|
|
532 |
<varlistentry> |
<varlistentry> |
541 |
</varlistentry> |
</varlistentry> |
542 |
|
|
543 |
<varlistentry> |
<varlistentry> |
544 |
|
<term><filename>-adof=/path/to/file</></term> |
545 |
|
<term><filename>-adoptfile=/path/to/file</></term> |
546 |
|
<listitem> |
547 |
|
<para>This option specifies the "adjoint" or automatic |
548 |
|
differentiation options file to be used. The file is analogous |
549 |
|
to the "optfile" defined above but it specifies information for |
550 |
|
the AD build process. The default file is located in <filename> |
551 |
|
$ROOTDIR/tools/adjoint_options/adjoint_default </> and it |
552 |
|
defines the "TAF" and "TAMC" compilers. An alternate version is |
553 |
|
also available at <filename> |
554 |
|
$ROOTDIR/tools/adjoint_options/adjoint_staf </> that selects the |
555 |
|
newer "STAF" compiler. As with any compilers, it is helpful to |
556 |
|
have their directories listed in your $PATH environment |
557 |
|
variable.</para> |
558 |
|
</listitem> |
559 |
|
</varlistentry> |
560 |
|
|
561 |
|
<varlistentry> |
562 |
<term><filename>-mods=DIR</></term> |
<term><filename>-mods=DIR</></term> |
563 |
<term><filename>-mods='DIR1 [DIR2 ...]'</></term> |
<term><filename>-mods='DIR1 [DIR2 ...]'</></term> |
564 |
<listitem> |
<listitem> |
612 |
</sect3> |
</sect3> |
613 |
|
|
614 |
<sect3 id="makefile_use"> |
<sect3 id="makefile_use"> |
615 |
<title>Using <filename>Makefile</></title> |
<title>Using the <filename>Makefile</></title> |
616 |
|
|
617 |
<para>Once a <filename>Makefile</> has been created, one can |
<para>Once a <filename>Makefile</> has been created using |
618 |
build an executable using:</para> |
<filename>genmake2</>, one can build a "standard" (forward |
619 |
|
simulator) executable using:</para> |
620 |
|
|
621 |
<screen> |
<screen> |
622 |
$ make CLEAN |
$ make CLEAN |
623 |
$ make depend |
$ make depend |
624 |
$ make |
$ make |
625 |
</screen> |
</screen> |
626 |
|
|
627 |
<para>The "make CLEAN" step will remove any stale source files, include |
<para>The "make CLEAN" step will remove any stale source files, include |
645 |
substitutions) custom definitions such as variable types within the |
substitutions) custom definitions such as variable types within the |
646 |
source files. This additional stage is necessary in order to overcome |
source files. This additional stage is necessary in order to overcome |
647 |
some of the inconsistencies in the sizes of objects (bytes) between |
some of the inconsistencies in the sizes of objects (bytes) between |
648 |
different compilers.</para> |
different compilers. The result of the build process is an executable |
649 |
|
with the name <filename>mitgcmuv</>.</para> |
650 |
|
|
651 |
|
<para>In addition to the forward simulator described above, the |
652 |
|
<filename>Makefile</> also has a number of targets that can be used to |
653 |
|
produce various adjoint and tangent-linear builds for optimization and |
654 |
|
other parameter-sensitivity problems. The additional targets within |
655 |
|
the <filename>Makefile</> are:</para> |
656 |
|
|
657 |
|
<variablelist> |
658 |
|
|
659 |
<para>Please report compilation failures or other problems to |
<varlistentry> |
660 |
<email>MITgcm-support@mitgcm.org</email>.</para> |
<term><filename>make adall</></term> |
661 |
|
<listitem> |
662 |
|
<para>This target produces an <filename>mitgcmuv_ad</> executable |
663 |
|
using the <filename>taf</> or <filename>staf</> adjoint |
664 |
|
compiler. See the <filename>genmake2</> "-adof" option for |
665 |
|
compiler selection.</para> |
666 |
|
</listitem> |
667 |
|
</varlistentry> |
668 |
|
|
669 |
|
<varlistentry> |
670 |
|
<term><filename>make ftlall</></term> |
671 |
|
<listitem> |
672 |
|
<para>Similar to <filename>make adall</> above, this |
673 |
|
produces...</para> |
674 |
|
</listitem> |
675 |
|
</varlistentry> |
676 |
|
|
677 |
|
</variablelist> |
678 |
|
|
679 |
|
<para>Please report any compilation failures or other build problems to |
680 |
|
the <email>MITgcm-support@mitgcm.org</email> list.</para> |
681 |
|
|
682 |
</sect3> |
</sect3> |
683 |
|
|
710 |
line as simple as:</para> |
line as simple as:</para> |
711 |
|
|
712 |
<screen> |
<screen> |
713 |
$ cd verification |
$ cd verification |
714 |
$ ./testreport -ieee |
$ ./testreport -ieee |
715 |
</screen> |
</screen> |
716 |
|
|
717 |
<para>However, some systems (those lacking or wiht a broken "/bin/sh") |
<para>However, some systems (those lacking or wiht a broken "/bin/sh") |
718 |
may require an explicit shell invocation such as:</para> |
may require an explicit shell invocation such as:</para> |
719 |
|
|
720 |
<screen> |
<screen> |
721 |
$ sh ./testreport -ieee -t 'exp0 exp4' |
$ sh ./testreport -ieee -t 'exp0 exp4' |
722 |
$ /some/path/to/bash ./testreport -ieee -t 'ideal_2D_oce lab_sea natl_box' |
$ /some/path/to/bash ./testreport -ieee -t 'ideal_2D_oce lab_sea natl_box' |
723 |
</screen> |
</screen> |
724 |
|
|
725 |
<para>The <filename>testreport</> script accepts a number of |
<para>The <filename>testreport</> script accepts a number of |
787 |
special command option (see "-command" below) to invoke the MPI |
special command option (see "-command" below) to invoke the MPI |
788 |
executable. Examples of PBS scripts using MPI with testreport can be |
executable. Examples of PBS scripts using MPI with testreport can be |
789 |
found in the <ulink |
found in the <ulink |
790 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/"> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm_contrib/test_scripts/"> |
791 |
MITgcm-contrib area</ulink></para> |
MITgcm-contrib area</ulink></para> |
792 |
</listitem> |
</listitem> |
793 |
</varlistentry> |
</varlistentry> |
799 |
output.txt" is not sufficient. This option allows a more general |
output.txt" is not sufficient. This option allows a more general |
800 |
command (or shell script) to be invoked. Examples of PBS scripts |
command (or shell script) to be invoked. Examples of PBS scripts |
801 |
using MPI with testreport can be found in the <ulink |
using MPI with testreport can be found in the <ulink |
802 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm_contrib/test_scripts/"> |
url="http://mitgcm.org/viewvc/MITgcm/MITgcm_contrib/test_scripts/"> |
803 |
MITgcm-contrib area</ulink></para> |
MITgcm-contrib area</ulink></para> |
804 |
</listitem> |
</listitem> |
805 |
</varlistentry> |
</varlistentry> |
826 |
the generic fluid dynamical engine.</para> |
the generic fluid dynamical engine.</para> |
827 |
|
|
828 |
<para>The MITgcmUV packaging structure is described below using generic |
<para>The MITgcmUV packaging structure is described below using generic |
829 |
package names ${pkg}. A concrete examples of a package is the code for |
package names ${pkg}. A concrete examples of a package is the code for |
830 |
implementing GM/Redi mixing. This code uses the package name</para> |
implementing GM/Redi mixing. This code uses the package name</para> |
831 |
|
|
832 |
</sect2> |
</sect2> |
886 |
Package runtime config. options are imported |
Package runtime config. options are imported |
887 |
into a common block held in a header file |
into a common block held in a header file |
888 |
called "${PKG}.h". |
called "${PKG}.h". |
889 |
|
Note: In some packages, the header file "${PKG}.h" is splitted |
890 |
|
into "${PKG}_PARAMS.h" that contains the package parameters and |
891 |
|
${PKG}_VARS.h" for the field arrays. |
892 |
|
|
893 |
o The core driver part of the model can check |
o The core driver part of the model can check |
894 |
for runtime enabling or disabling of individual packages |
for runtime enabling or disabling of individual packages |
911 |
1. Within the core driver code flags of the form |
1. Within the core driver code flags of the form |
912 |
ALLOW_${PKG} are used to include or exclude |
ALLOW_${PKG} are used to include or exclude |
913 |
whole packages. The ALLOW_${PKG} flags are included |
whole packages. The ALLOW_${PKG} flags are included |
914 |
from a PKG_CPP_OPTIONS block which is currently |
from a PACKAGES_CONFIG.h file that is automatically |
915 |
|
generated by genmake2 (see genmake2 section). |
916 |
held in-line in the CPP_OPTIONS.h header file. |
held in-line in the CPP_OPTIONS.h header file. |
917 |
e.g. |
e.g. |
918 |
|
|
919 |
Core model code ..... |
Core model code ..... |
920 |
|
|
921 |
|
#include "PACKAGES_CONFIG.h" |
922 |
#include "CPP_OPTIONS.h" |
#include "CPP_OPTIONS.h" |
923 |
: |
: |
924 |
: |
: |
930 |
|
|
931 |
2. Within an individual package a header file, |
2. Within an individual package a header file, |
932 |
"${PKG}_OPTIONS.h", is used to set CPP flags |
"${PKG}_OPTIONS.h", is used to set CPP flags |
933 |
specific to that package. It is not recommended |
specific to that package. It also includes |
934 |
to include this file in "CPP_OPTIONS.h". |
"PACKAGES_CONFIG.h" and "CPP_OPTIONS.h". |
935 |
|
|
936 |
|
|
937 |
Package Boot Sequence |
Package Boot Sequence |
953 |
& CALL ${PKG}_READPARMS( retCode ) |
& CALL ${PKG}_READPARMS( retCode ) |
954 |
#endif |
#endif |
955 |
|
|
956 |
2. S/R PACKAGES_CHECK() |
3. S/R PACKAGES_INIT_FIXED() |
957 |
|
: |
958 |
|
#ifdef ALLOW_${PKG} |
959 |
|
if ( use${Pkg} ) |
960 |
|
& CALL ${PKG}_INIT_FIXED( retCode ) |
961 |
|
#endif |
962 |
|
|
963 |
|
4. S/R PACKAGES_CHECK() |
964 |
: |
: |
965 |
#ifdef ALLOW_${PKG} |
#ifdef ALLOW_${PKG} |
966 |
if ( use${Pkg} ) |
if ( use${Pkg} ) |
970 |
& CALL PACKAGES_CHECK_ERROR('${PKG}') |
& CALL PACKAGES_CHECK_ERROR('${PKG}') |
971 |
#endif |
#endif |
972 |
|
|
973 |
3. S/R PACKAGES_INIT() |
5. S/R PACKAGES_INIT_VARIABLES() |
974 |
: |
: |
975 |
#ifdef ALLOW_${PKG} |
#ifdef ALLOW_${PKG} |
976 |
if ( use${Pkg} ) |
if ( use${Pkg} ) |
977 |
& CALL ${PKG}_INIT( retCode ) |
& CALL ${PKG}_INIT_VARIA( ) |
978 |
#endif |
#endif |
979 |
|
|
980 |
|
Package Output |
981 |
|
============== |
982 |
|
6. S/R DO_THE_MODEL_IO |
983 |
|
|
984 |
|
#ifdef ALLOW_${PKG} |
985 |
|
if ( use${Pkg} ) |
986 |
|
& CALL ${PKG}_OUTPUT( ) |
987 |
|
#endif |
988 |
|
|
989 |
|
7. S/R PACKAGES_WRITE_PICKUP() |
990 |
|
|
991 |
|
#ifdef ALLOW_${PKG} |
992 |
|
if ( use${Pkg} ) |
993 |
|
& CALL ${PKG}_WRITE_PICKUP( ) |
994 |
|
#endif |
995 |
|
|
996 |
Description |
Description |
997 |
=========== |
=========== |
999 |
- ${PKG}_READPARMS() |
- ${PKG}_READPARMS() |
1000 |
is responsible for reading |
is responsible for reading |
1001 |
in the package parameters file data.${pkg}, and storing |
in the package parameters file data.${pkg}, and storing |
1002 |
the package parameters in "${PKG}.h". |
the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h"). |
1003 |
-> called in INITIALISE_FIXED |
-> called from INITIALISE_FIXED in PACKAGES_READPARMS |
1004 |
|
|
1005 |
|
- ${PKG}_INIT_FIXED() |
1006 |
|
is responsible for completing the internal setup of a package. |
1007 |
|
-> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED |
1008 |
|
note: 1) some pkg use instead: |
1009 |
|
CALL ${PKG}_INITIALISE ( or the old form CALL ${PKG}_INIT ) |
1010 |
|
2) for simple pkg setup, this part is done inside ${PKG}_READPARMS |
1011 |
|
|
1012 |
- ${PKG}_CHECK() |
- ${PKG}_CHECK() |
1013 |
is responsible for validating |
is responsible for validating |
1016 |
need to check. This is done through header files "${PKG}.h". |
need to check. This is done through header files "${PKG}.h". |
1017 |
It is assumed that parameters owned by other packages |
It is assumed that parameters owned by other packages |
1018 |
will not be reset during ${PKG}_CHECK(). |
will not be reset during ${PKG}_CHECK(). |
1019 |
-> called in INITIALISE_FIXED |
-> called from INITIALISE_FIXED in PACKAGES_CHECK |
1020 |
|
|
1021 |
- ${PKG}_INIT() |
- ${PKG}_INIT_VARIA() |
1022 |
is responsible for completing the |
is responsible for fill-in all package variables with an initial value. |
1023 |
internal setup of a package. This routine is called after |
Contains eventually a call to ${PKG}_READ_PICKUP that will read |
1024 |
the core model state has been completely initialised |
from a pickup file the package variables required to restart the model. |
1025 |
but before the core model timestepping starts. |
This routine is called after the core model state has been completely |
1026 |
-> called in INITIALISE_VARIA |
initialised but before the core model timestepping starts. |
1027 |
|
-> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES |
1028 |
|
note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg |
1029 |
|
use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old |
1030 |
|
form ${PKG}_INIT |
1031 |
|
|
1032 |
|
- ${PKG}_OUTPUT( ) |
1033 |
|
is responsible for writing time-average fields to output files |
1034 |
|
(but the cumulating step is done within the package main S/R). |
1035 |
|
Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR) |
1036 |
|
and write snap-shot fields that are hold in common blocks. Other |
1037 |
|
temporary fields are directly dump to file where they are available. |
1038 |
|
NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages |
1039 |
|
but is beeing replaced by ${PKG}_OUTPUT |
1040 |
|
to avoid confusion with pkg/diagnostics functionality. |
1041 |
|
2) the output part is not yet in a standard form and might still |
1042 |
|
evolve a lot. |
1043 |
|
-> called within DO_THE_MODEL_IO |
1044 |
|
|
1045 |
|
- ${PKG}_WRITE_PICKUP() |
1046 |
|
is responsible for writing a package pickup file when necessary for |
1047 |
|
a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT ) |
1048 |
|
-> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP |
1049 |
|
|
1050 |
Summary |
Summary |
1051 |
======= |
======= |
1066 |
----------------------- |
----------------------- |
1067 |
* ${PKG}_OPTIONS.h has further package-specific CPP options |
* ${PKG}_OPTIONS.h has further package-specific CPP options |
1068 |
* ${PKG}.h package-specific common block variables, fields |
* ${PKG}.h package-specific common block variables, fields |
1069 |
|
or ${PKG}_PARAMS.h package-specific common block parameters |
1070 |
|
and ${PKG}_VARS.h package-specific common block fields |
1071 |
|
|
1072 |
- FORTRAN source files |
- FORTRAN source files |
1073 |
----------------------- |
----------------------- |
1074 |
* ${pkg}_readparms.F reads parameters from file data.${pkg} |
* ${pkg}_readparms.F reads parameters from file data.${pkg} |
1075 |
* ${pkg}_check.F checks package dependencies and consistencies |
* ${pkg}_init_fixed.F complete the package setup |
1076 |
* ${pkg}_init.F initialises package-related fields |
* ${pkg}_check.F checks package dependencies and consistencies |
1077 |
* ${pkg}_... .F package source code |
* ${pkg}_init_varia.F initialises package-related fields |
1078 |
|
* ${pkg}_... .F package source code |
1079 |
|
* ${pkg}_output.F write output to file. |
1080 |
|
* ${pkg}_write_pickup.F write a package pickup file to restart the model |
1081 |
|
|
1082 |
|
New: Subroutine in one package (pkgA) that only contains code which |
1083 |
|
is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F) |
1084 |
|
will be named: pkgA_pkgB_something.F |
1085 |
|
|
1086 |
- parameter file |
- parameter file |
1087 |
----------------------- |
----------------------- |
1091 |
</sect1> |
</sect1> |
1092 |
|
|
1093 |
|
|
1094 |
|
<sect1 id="documentation"> |
1095 |
|
<title>Editing the Documentation</title> |
1096 |
|
|
1097 |
|
<sect2 id="documentation_getting"> |
1098 |
|
<title>Getting the Docs and Code</title> |
1099 |
|
|
1100 |
|
<para>The first step towards editing the documentation is to checkout a |
1101 |
|
copy of code, docs, and build scripts from the CVS server using:</para> |
1102 |
|
|
1103 |
|
<screen> |
1104 |
|
$ export CVS_RSH=ssh |
1105 |
|
$ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack' |
1106 |
|
$ mkdir scratch |
1107 |
|
$ cvs co -P MITgcm manual mitgcm.org |
1108 |
|
</screen> |
1109 |
|
|
1110 |
|
<para>These commands extract the necessary information from the CVS server |
1111 |
|
and create a temporary (called <filename>scratch</filename>) directory for |
1112 |
|
the storage of the HTML and other files that will be created. Please note |
1113 |
|
that you must either create <filename>scratch</filename> as shown or edit |
1114 |
|
the various <filename>Makefile</filename>s and scripts used to create the |
1115 |
|
documentation.</para> |
1116 |
|
</sect2> |
1117 |
|
|
1118 |
|
<sect2> |
1119 |
|
<title>Editing the Documentation</title> |
1120 |
|
|
1121 |
|
<para>The documentation is contained in the <filename>manual</filename> |
1122 |
|
directory in a raw LaTeX format. The main document is |
1123 |
|
<filename>manual.tex</filename> and it uses <command>\input{}</command>s |
1124 |
|
to include the chapters and subsections.</para> |
1125 |
|
|
1126 |
|
<para>Since the same LaTeX source is used to produce PostScript, PDF, and |
1127 |
|
HTML output, care should be taken to follow certain conventions. Two of |
1128 |
|
the most important are the usage of the <command>\filelink{}{}</command> |
1129 |
|
and <command>\varlink{}{}</command> commands. Both of these commands have |
1130 |
|
been defined to simplify the connection between the automatically |
1131 |
|
generated ("code browser") HTML and the HTML version of the manual |
1132 |
|
produced by LaTeX2HTML. They each take two arguments (corresponding to |
1133 |
|
the contents of the two sets of curly braces) which are the text that the |
1134 |
|
author wishes to be "wrapped" within the link, and a specially formatted |
1135 |
|
link thats relative to the <filename>MITgcm</filename> directory within |
1136 |
|
the CVS tree.</para> |
1137 |
|
|
1138 |
|
<para>The result is a command that resembles either</para> |
1139 |
|
|
1140 |
|
<orderedlist> |
1141 |
|
<listitem> |
1142 |
|
<para>a reference to a variable or subroutine name such as |
1143 |
|
<command>\varlink{tRef}{tRef}</command>, or </para> |
1144 |
|
</listitem> |
1145 |
|
|
1146 |
|
<listitem> |
1147 |
|
<para>a reference to a file such as |
1148 |
|
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
1149 |
|
where the absolute path to the file is of the form |
1150 |
|
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
1151 |
|
<para>(please note how the leading "/foo/MITgcm" |
1152 |
|
component of the path is dropped leaving the path |
1153 |
|
<emphasis>relative</emphasis> to the head of the code |
1154 |
|
directory and each directory separator "/" is turned |
1155 |
|
into a "-")</para> |
1156 |
|
</listitem> |
1157 |
|
</orderedlist> |
1158 |
|
|
1159 |
|
|
1160 |
|
|
1161 |
|
</sect2> |
1162 |
|
|
1163 |
|
<sect2> |
1164 |
|
<title>Building the Documentation</title> |
1165 |
|
|
1166 |
|
<para>Given the directory structure of <xref |
1167 |
|
linkend="documentation_getting">, the entire documentation for the web |
1168 |
|
site can be built using:</para> |
1169 |
|
|
1170 |
|
<screen> |
1171 |
|
$ cd mitgcm.org/devel/buildweb |
1172 |
|
$ make All |
1173 |
|
</screen> |
1174 |
|
|
1175 |
|
<para>Which builds the PDF from the LaTeX source, creates the HTML output |
1176 |
|
from the LaTeX source, parses the FORTRAN code base to produce a |
1177 |
|
hyperlinked HTML version of the source, and then determines the |
1178 |
|
cross-linking between the various HTML components.</para> |
1179 |
|
|
1180 |
|
<para>If there are no errors, the result of the build process (which can |
1181 |
|
take 30+ minutes on a P4/2.5Ghz) will be contained within a single |
1182 |
|
directory called <filename>scratch/dev_docs</filename>. This is a freshly |
1183 |
|
built version of the entire on-line users manual. If you have the correct |
1184 |
|
permissions, it can be directly copied to the web server area:</para> |
1185 |
|
|
1186 |
|
<screen> |
1187 |
|
$ mv scratch/dev_docs /u/u0/httpd/html |
1188 |
|
</screen> |
1189 |
|
|
1190 |
|
<para>and the update is complete.</para> |
1191 |
|
|
1192 |
|
</sect2> |
1193 |
|
|
1194 |
|
</sect1> |
1195 |
|
|
1196 |
</article> |
</article> |
1197 |
|
|
1198 |
|
|