1 |
edhill |
1.1 |
<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN"> |
2 |
|
|
|
3 |
|
|
<article id="MITgcm-Development-HOWTO"> |
4 |
|
|
|
5 |
|
|
<!-- |
6 |
|
|
Build commands: |
7 |
|
|
db2pdf -d ldp.dsl devel_HOWTO.sgml |
8 |
|
|
db2html -d ./ldp.dsl devel_HOWTO.sgml |
9 |
|
|
--> |
10 |
|
|
|
11 |
|
|
<articleinfo> |
12 |
|
|
<title>MITgcm Development HOWTO</title> |
13 |
|
|
|
14 |
|
|
<author> |
15 |
|
|
<firstname>Ed</firstname> |
16 |
|
|
<surname>Hill III</surname> |
17 |
|
|
<affiliation> |
18 |
|
|
<address><email>eh3@mit.edu</email></address> |
19 |
|
|
</affiliation> |
20 |
|
|
</author> |
21 |
|
|
|
22 |
|
|
<revhistory> |
23 |
|
|
<revision> |
24 |
|
|
<revnumber>0.01</revnumber> |
25 |
|
|
<date>2003-0-07</date> |
26 |
|
|
<authorinitials>eh3</authorinitials> |
27 |
|
|
<revremark> |
28 |
|
|
Initial version. |
29 |
|
|
</revremark> |
30 |
|
|
</revision> |
31 |
|
|
</revhistory> |
32 |
|
|
|
33 |
|
|
<abstract> |
34 |
|
|
<para>This document describes how to develop software for the |
35 |
|
|
MITgcm project.</para> |
36 |
|
|
</abstract> |
37 |
|
|
</articleinfo> |
38 |
|
|
|
39 |
|
|
<sect1 id="intro"> |
40 |
|
|
<title>Introduction</title> <para>The purpose of this document is |
41 |
|
|
to help new developers get "up to speed" with MITgcm |
42 |
|
|
development.</para> |
43 |
|
|
<sect2> |
44 |
|
|
<title>New Versions of This Document</title> <para>You can |
45 |
|
|
obtain the latest version of this document <ulink |
46 |
|
|
url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in |
47 |
|
|
various formats.</para> |
48 |
|
|
</sect2> |
49 |
|
|
<sect2> |
50 |
|
|
<title>Feedback and corrections</title> <para>If you have |
51 |
|
|
questions or comments about this document, please feel free to |
52 |
|
|
<ulink url="mailto:MITgcm-support@mitgcm.org">contact the |
53 |
|
|
authors</ulink>. |
54 |
|
|
</para> |
55 |
|
|
</sect2> |
56 |
|
|
</sect1> |
57 |
|
|
|
58 |
|
|
<sect1 id="background"> |
59 |
|
|
<title>Background</title> |
60 |
|
|
|
61 |
|
|
<sect2> |
62 |
|
|
<title>User Manual</title> |
63 |
|
|
|
64 |
|
|
<para>Before jumping into |
65 |
|
|
development, please familiarize yourself with the MITgcm user |
66 |
|
|
manual which is available <ulink |
67 |
|
|
url="http://mitgcm.org/">on the main web page</ulink>. This |
68 |
|
|
document contains volumes of useful information and is included |
69 |
|
|
here by reference.</para> |
70 |
|
|
|
71 |
|
|
<para>Also, a "snapshot" or<ulink |
72 |
|
|
url="http://mitgcm.org/dev_docs/">development version</ulink> of |
73 |
|
|
the user manual may be available, though this is only put on the |
74 |
|
|
web for testing purposes.</para> |
75 |
|
|
</sect2> |
76 |
|
|
|
77 |
|
|
<sect2> |
78 |
|
|
<title>Prerequisites</title> <para>To develop for MITgcm project |
79 |
|
|
you will need a UNIX or UNIX-like set of build tools including |
80 |
|
|
the following:</para> |
81 |
|
|
<blockquote> |
82 |
|
|
<simplelist type="inline"> |
83 |
|
|
<member>CVS client</member> |
84 |
|
|
<member>make or (preferably) GNU make</member> |
85 |
|
|
<member>FORTRAN compiler</member> |
86 |
|
|
<member>C compiler</member> |
87 |
|
|
<member>[ba]sh and [t]csh shells</member> |
88 |
|
|
<member>PERL</member> |
89 |
|
|
<member>LaTeX and LaTeX2HTML</member> |
90 |
|
|
</simplelist> |
91 |
|
|
</blockquote> |
92 |
|
|
<para>Essentially all of the work described here has been tested |
93 |
|
|
on recent versions of Red Hat Linux (eg. 7.3 through 9). Except |
94 |
|
|
where noted, all shell commands will be provided using bash |
95 |
|
|
syntax. |
96 |
|
|
</para> |
97 |
|
|
</sect2> |
98 |
|
|
|
99 |
|
|
</sect1> |
100 |
|
|
|
101 |
|
|
<sect1 id="cvs"> |
102 |
|
|
<title>CVS Repository</title> |
103 |
|
|
<sect2> |
104 |
|
|
<title>Layout</title> |
105 |
|
|
|
106 |
|
|
<para>Unlike many open source projects, the MITgcm CVS tree does |
107 |
|
|
not follow a simple "src", "docs", "share", and "test" directory |
108 |
|
|
layout. Instead, there are multiple higher-level directories |
109 |
|
|
that each, to some extent, depend upon the presence of the |
110 |
|
|
others. The tree currently resembles:</para> |
111 |
|
|
|
112 |
|
|
<programlisting>gcmpack/ |
113 |
|
|
MITgcm-contrib contributed code |
114 |
|
|
CS-regrid goes into utils |
115 |
|
|
cvspolicy.html -save- |
116 |
|
|
CVSROOT -save- |
117 |
|
|
development experimental stuff |
118 |
|
|
manual -save- |
119 |
|
|
misc -?- |
120 |
|
|
|
121 |
|
|
MITgcm code |
122 |
|
|
adjoint fold into genmake |
123 |
|
|
bin stub for ecco build |
124 |
|
|
compare01 old from 20th century |
125 |
|
|
diags timeave f77 in pkgs now |
126 |
|
|
doc tags -- connect to real docs? |
127 |
|
|
eesupp cnh? |
128 |
|
|
exe ecco user build |
129 |
|
|
*- jobs runtime shell scripts for |
130 |
|
|
| various platforms |
131 |
|
|
| lsopt line search |
132 |
|
|
m| model main dynamics (core) |
133 |
|
|
e| optimization_drivers ? |
134 |
|
|
r| optim line search interface |
135 |
|
|
g| pkg alternate and optional numerics, etc. |
136 |
|
|
e*- tools |
137 |
|
|
?| tutorial_examples documented tests |
138 |
|
|
| only populated on release1 branch |
139 |
|
|
| and not validated during "testscript" |
140 |
|
|
*- utils |
141 |
|
|
verification std tests |
142 |
|
|
|
143 |
|
|
|
144 |
|
|
mitgcmdoc -> manual -remove- |
145 |
|
|
mitgcm.org build web site |
146 |
|
|
models -?- |
147 |
|
|
packages -?- |
148 |
|
|
preprocess -?- |
149 |
|
|
tmp -?- |
150 |
|
|
</programlisting> |
151 |
|
|
|
152 |
|
|
<para>Efforts are underway to reduce the complexity.</para> |
153 |
|
|
|
154 |
|
|
</sect2> |
155 |
|
|
|
156 |
|
|
<!-- |
157 |
|
|
<sect2> |
158 |
|
|
<title>Releases</title> <para>Currently, there are two main |
159 |
|
|
branches:</para> |
160 |
|
|
<itemizedlist mark="bullet"> |
161 |
|
|
<listitem> |
162 |
|
|
<para>Development</para> |
163 |
|
|
<itemizedlist mark="bullet"> |
164 |
|
|
<listitem> |
165 |
|
|
<para>MAIN</para> |
166 |
|
|
</listitem> |
167 |
|
|
<listitem> |
168 |
|
|
<para>ecco-branch</para> |
169 |
|
|
</listitem> |
170 |
|
|
</itemizedlist> |
171 |
|
|
</listitem> |
172 |
|
|
<listitem> |
173 |
|
|
<para>Production</para> |
174 |
|
|
<itemizedlist mark="bullet"> |
175 |
|
|
<listitem> |
176 |
|
|
<para>Release1</para> |
177 |
|
|
</listitem> |
178 |
|
|
<listitem> |
179 |
|
|
<para>Release2</para> |
180 |
|
|
</listitem> |
181 |
|
|
</itemizedlist> |
182 |
|
|
</listitem> |
183 |
|
|
</itemizedlist> |
184 |
|
|
</sect2> |
185 |
|
|
--> |
186 |
|
|
|
187 |
|
|
<sect2> |
188 |
|
|
<title>Branches</title> |
189 |
|
|
|
190 |
|
|
<para>As shown in the online <ulink |
191 |
|
|
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated |
192 |
|
|
tree</ulink>, the MITgcm codebase is split into to two branches |
193 |
|
|
or "lines" under which development proceeds. These two lines |
194 |
|
|
are referred to as the "MAIN" and "ecco" versions of the code. |
195 |
|
|
While not identical, the bulk of the MAIN and ecco lines are |
196 |
|
|
composed of files from the same codebase. |
197 |
|
|
</para> |
198 |
|
|
|
199 |
|
|
<para>Periodically, a "Release" branch is formed from the "MAIN" |
200 |
|
|
development branch. This is done in order to create a |
201 |
|
|
relatively stable reference point for both users and developers. |
202 |
|
|
The intent is that once a relese branch has been created, only |
203 |
|
|
bug-fixes will be added to it. Meanwhile, development (which |
204 |
|
|
might "break" or otherwise render invalid the documentation, |
205 |
|
|
tutorials, and/or examples contained within a release branch) is |
206 |
|
|
allowed to continue along the MAIN and ecco lines.</para> |
207 |
|
|
</sect2> |
208 |
|
|
|
209 |
|
|
<sect2> |
210 |
|
|
<title>Tagging</title> |
211 |
|
|
|
212 |
|
|
<para>The intent of tagging is to create "known-good" |
213 |
|
|
checkpoints that developers can use as references. |
214 |
|
|
Traditionally, MITgcm tagging has maintained the following |
215 |
|
|
conventions:</para> |
216 |
|
|
|
217 |
|
|
<orderedlist> |
218 |
|
|
<listitem> |
219 |
|
|
<para>Developer checks out code into a local CVS-managed |
220 |
|
|
directory, makes various changes/additions, tests these |
221 |
|
|
edits, and eventually reaches a point where (s)he is |
222 |
|
|
satisfied that the changes form a new "useful" point in the |
223 |
|
|
evolution of the code.</para> |
224 |
|
|
</listitem> |
225 |
|
|
|
226 |
|
|
<listitem> |
227 |
|
|
<para>The developer then runs the <ulink |
228 |
|
|
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink> |
229 |
|
|
shell script to see if any problems are introduced. While |
230 |
|
|
not intended to be exhaustive, the test cases within the |
231 |
|
|
verification directory do provide some indication whether |
232 |
|
|
gross errors have been introduced. |
233 |
|
|
</para> |
234 |
|
|
</listitem> |
235 |
|
|
|
236 |
|
|
<listitem> |
237 |
|
|
<para>Having satisfied him- or herself that the changes are |
238 |
|
|
ready to be committed to the CVS repository, the developer |
239 |
|
|
then:</para> |
240 |
|
|
<orderedlist> |
241 |
|
|
<listitem> |
242 |
|
|
<para>adds a "checkpointXY_pre" comment (where X is a |
243 |
|
|
checkpoint number and Y is a letter) to the <ulink |
244 |
|
|
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink> |
245 |
|
|
file and checks it into the CVS repository</para> |
246 |
|
|
</listitem> |
247 |
|
|
<listitem> |
248 |
|
|
<para>submits the set of changes to the CVS repository |
249 |
|
|
and adds comments to <filename>tag-index</filename> |
250 |
|
|
describing what the changes are along with a matching |
251 |
|
|
"checkpointXY_post" entry</para> |
252 |
|
|
</listitem> |
253 |
|
|
</orderedlist> |
254 |
|
|
</listitem> |
255 |
|
|
</orderedlist> |
256 |
|
|
|
257 |
|
|
<para>The result of this tagging procedure is a sequence of |
258 |
|
|
development checkpoints with comments which resembles:</para> |
259 |
|
|
|
260 |
|
|
<programlisting> |
261 |
|
|
checkpoint50e_post |
262 |
|
|
o make KPP work with PTRACERS |
263 |
|
|
- fix gad_calc_rhs to call new routine kpp_transport_ptr, which is |
264 |
|
|
nearly a copy of kpp_transport_s |
265 |
|
|
- there is no analogue to SurfaceTendencyS, so I have to use |
266 |
|
|
gPtr(of the surface layer) instead |
267 |
|
|
o add a new platform SunFire+mpi (SunFire 15000) to genmake |
268 |
|
|
checkpoint50e_pre |
269 |
|
|
|
270 |
|
|
checkpoint50d_post |
271 |
|
|
o change kpp output from multiple-record state files to single-record state |
272 |
|
|
files analogous to write_state.F |
273 |
|
|
o reduce the output frequency of cg3d-related stuff to the monitor frequency, |
274 |
|
|
analogous to the cg2d-related output. |
275 |
|
|
o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512, |
276 |
|
|
so that writing to internal file fn (with length 512) fails. |
277 |
|
|
checkpoint50d_pre |
278 |
|
|
</programlisting> |
279 |
|
|
|
280 |
|
|
<para>This information can be used to refer to various stages of |
281 |
|
|
the code development. For example, bugs can be traced to |
282 |
|
|
individual sets of CVS checkins based upon their first |
283 |
|
|
appearance when comparing the results from different |
284 |
|
|
checkpoints.</para> |
285 |
|
|
|
286 |
|
|
</sect2> |
287 |
|
|
</sect1> |
288 |
|
|
|
289 |
|
|
|
290 |
|
|
<sect1 id="documentation"> |
291 |
|
|
<title>Editing the Documentation</title> |
292 |
|
|
|
293 |
|
|
<sect2 id="documentation_getting"> |
294 |
|
|
<title>Getting the Docs and Code</title> |
295 |
|
|
|
296 |
|
|
<para>The first step towards editing the documentation is to |
297 |
|
|
checkout a copy of code, docs, and build scripts from the CVS |
298 |
|
|
server using:</para> |
299 |
|
|
|
300 |
|
|
<screen> |
301 |
|
|
$ export CVS_RSH=ssh |
302 |
|
|
$ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack' |
303 |
|
|
$ mkdir scratch |
304 |
|
|
$ cvs co MITgcm manual mitgcm.org |
305 |
|
|
</screen> |
306 |
|
|
|
307 |
|
|
<para>These commands extract the necessary information from the |
308 |
|
|
CVS server and create a temporary (called |
309 |
|
|
<filename>scratch</filename>) directory for the storage of the |
310 |
|
|
HTML and other files that will be created. Please note that you |
311 |
|
|
must either create <filename>scratch</filename> as shown or edit |
312 |
|
|
the various <filename>Makefile</filename>s and scripts used to |
313 |
|
|
create the documentation.</para> |
314 |
|
|
</sect2> |
315 |
|
|
|
316 |
|
|
<sect2> |
317 |
|
|
<title>Editing</title> |
318 |
|
|
|
319 |
|
|
<para>The documentation is contained in the |
320 |
|
|
<filename>manual</filename> directory in a raw LaTeX format. |
321 |
|
|
The main document is <filename>manual.tex</filename> and it uses |
322 |
|
|
<command>\input{}</command>s to include the chapters and |
323 |
|
|
subsections.</para> |
324 |
|
|
|
325 |
|
|
<para>Since the same LaTeX source is used to produce PostScript, |
326 |
|
|
PDF, and HTML output, care should be taken to follow certain |
327 |
|
|
conventions. Two of the most important are the usage of the |
328 |
|
|
<command>\filelink{}{}</command> and |
329 |
|
|
<command>\varlink{}{}</command> commands. Both of these |
330 |
|
|
commands have been defined to simplify the connection between |
331 |
|
|
the automatically generated ("code browser") HTML and the HTML |
332 |
|
|
version of the manual produced by LaTeX2HTML. They each take |
333 |
|
|
two arguments (corresponding to the contents of the two sets of |
334 |
|
|
curly braces) which are the text that the author wishes to be |
335 |
|
|
"wrapped" within the link, and a specially formatted link thats |
336 |
|
|
relative to the <filename>MITgcm</filename> directory within the |
337 |
|
|
CVS tree.</para> |
338 |
|
|
|
339 |
|
|
<para>The result is a command that resembles either</para> |
340 |
|
|
|
341 |
|
|
<orderedlist> |
342 |
|
|
<listitem> |
343 |
|
|
<para>a reference to a variable or subroutine name such as |
344 |
|
|
<command>\varlink{tRef}{tRef}</command>, or </para> |
345 |
|
|
</listitem> |
346 |
|
|
|
347 |
|
|
<listitem> |
348 |
|
|
<para>a reference to a file such as |
349 |
|
|
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
350 |
|
|
where the absolute path to the file is of the form |
351 |
|
|
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
352 |
|
|
<para>(please note how the leading "/foo/MITgcm" |
353 |
|
|
component of the path is dropped leaving the path |
354 |
|
|
<emphasis>relative</emphasis> to the head of the code |
355 |
|
|
directory and each directory separator "/" is turned |
356 |
|
|
into a "-")</para> |
357 |
|
|
</listitem> |
358 |
|
|
</orderedlist> |
359 |
|
|
|
360 |
|
|
|
361 |
|
|
|
362 |
|
|
</sect2> |
363 |
|
|
|
364 |
|
|
<sect2> |
365 |
|
|
<title>Building</title> <para>Given the directory structure of |
366 |
|
|
<xref linkend="documentation_getting">, the entire documentation |
367 |
|
|
for the web site can be built using:</para> |
368 |
|
|
|
369 |
|
|
<screen> |
370 |
|
|
$ cd mitgcm.org/devel/buildweb |
371 |
|
|
$ make All |
372 |
|
|
</screen> |
373 |
|
|
|
374 |
|
|
<para>Which builds the PDF from the LaTeX source, creates the |
375 |
|
|
HTML output from the LaTeX source, parses the FORTRAN code base |
376 |
|
|
to produce a hyperlinked HTML version of the source, and then |
377 |
|
|
determines the cross-linking between the various HTML |
378 |
|
|
components.</para> |
379 |
|
|
|
380 |
|
|
<para>If there are no errors, the result of the build process |
381 |
|
|
(which can take 30+ minutes on a P4/2.5Ghz) will be contained |
382 |
|
|
within a single directory called |
383 |
|
|
<filename>scratch/dev_docs</filename>. This is a freshly built |
384 |
|
|
version of the entire on-line users manual. If you have the |
385 |
|
|
correct permissions, it can be directly copied to the web server |
386 |
|
|
area:</para> |
387 |
|
|
|
388 |
|
|
<screen> |
389 |
|
|
$ mv scratch/dev_docs /u/u0/httpd/html |
390 |
|
|
</screen> |
391 |
|
|
|
392 |
|
|
<para>and the update is complete.</para> |
393 |
|
|
|
394 |
|
|
</sect2> |
395 |
|
|
|
396 |
|
|
</sect1> |
397 |
|
|
|
398 |
|
|
|
399 |
|
|
|
400 |
|
|
</article> |
401 |
|
|
|
402 |
|
|
|