1 |
<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN"> |
2 |
|
3 |
<article id="MITgcm-Development-HOWTO"> |
4 |
|
5 |
<!-- |
6 |
Build commands: |
7 |
db2pdf -d ldp.dsl devel_HOWTO.sgml |
8 |
db2html -d ./ldp.dsl devel_HOWTO.sgml |
9 |
--> |
10 |
|
11 |
<articleinfo> |
12 |
<title>MITgcm Development HOWTO</title> |
13 |
|
14 |
<author> |
15 |
<firstname>Ed</firstname> |
16 |
<surname>Hill III</surname> |
17 |
<affiliation> |
18 |
<address><email>eh3@mit.edu</email></address> |
19 |
</affiliation> |
20 |
</author> |
21 |
|
22 |
<revhistory> |
23 |
<revision> |
24 |
<revnumber>0.01</revnumber> |
25 |
<date>2003-0-07</date> |
26 |
<authorinitials>eh3</authorinitials> |
27 |
<revremark> |
28 |
Initial version. |
29 |
</revremark> |
30 |
</revision> |
31 |
</revhistory> |
32 |
|
33 |
<abstract> |
34 |
<para>This document describes how to develop software for the |
35 |
MITgcm project.</para> |
36 |
</abstract> |
37 |
</articleinfo> |
38 |
|
39 |
<sect1 id="intro"> |
40 |
<title>Introduction</title> <para>The purpose of this document is |
41 |
to help new developers get "up to speed" with MITgcm |
42 |
development.</para> |
43 |
<sect2> |
44 |
<title>New Versions of This Document</title> <para>You can |
45 |
obtain the latest version of this document <ulink |
46 |
url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in |
47 |
various formats.</para> |
48 |
</sect2> |
49 |
<sect2> |
50 |
<title>Feedback and corrections</title> <para>If you have |
51 |
questions or comments about this document, please feel free to |
52 |
<ulink url="mailto:MITgcm-support@mitgcm.org">contact the |
53 |
authors</ulink>. |
54 |
</para> |
55 |
</sect2> |
56 |
</sect1> |
57 |
|
58 |
<sect1 id="background"> |
59 |
<title>Background</title> |
60 |
|
61 |
<sect2> |
62 |
<title>User Manual</title> |
63 |
|
64 |
<para>Before jumping into |
65 |
development, please familiarize yourself with the MITgcm user |
66 |
manual which is available <ulink |
67 |
url="http://mitgcm.org/">on the main web page</ulink>. This |
68 |
document contains volumes of useful information and is included |
69 |
here by reference.</para> |
70 |
|
71 |
<para>Also, a "snapshot" or<ulink |
72 |
url="http://mitgcm.org/dev_docs/">development version</ulink> of |
73 |
the user manual may be available, though this is only put on the |
74 |
web for testing purposes.</para> |
75 |
</sect2> |
76 |
|
77 |
<sect2> |
78 |
<title>Prerequisites</title> <para>To develop for MITgcm project |
79 |
you will need a UNIX or UNIX-like set of build tools including |
80 |
the following:</para> |
81 |
<blockquote> |
82 |
<simplelist type="inline"> |
83 |
<member>CVS client</member> |
84 |
<member>make or (preferably) GNU make</member> |
85 |
<member>FORTRAN compiler</member> |
86 |
<member>C compiler</member> |
87 |
<member>[ba]sh and [t]csh shells</member> |
88 |
<member>PERL</member> |
89 |
<member>LaTeX and LaTeX2HTML</member> |
90 |
</simplelist> |
91 |
</blockquote> |
92 |
<para>Essentially all of the work described here has been tested |
93 |
on recent versions of Red Hat Linux (eg. 7.3 through 9). Except |
94 |
where noted, all shell commands will be provided using bash |
95 |
syntax. |
96 |
</para> |
97 |
</sect2> |
98 |
|
99 |
</sect1> |
100 |
|
101 |
<sect1 id="cvs"> |
102 |
<title>CVS Repository</title> |
103 |
<sect2> |
104 |
<title>Layout</title> |
105 |
|
106 |
<para>Unlike many open source projects, the MITgcm CVS tree does |
107 |
not follow a simple "src", "docs", "share", and "test" directory |
108 |
layout. Instead, there are multiple higher-level directories |
109 |
that each, to some extent, depend upon the presence of the |
110 |
others. The tree currently resembles:</para> |
111 |
|
112 |
<programlisting>gcmpack/ |
113 |
MITgcm-contrib contributed code |
114 |
CS-regrid goes into utils |
115 |
cvspolicy.html -save- |
116 |
CVSROOT -save- |
117 |
development experimental stuff |
118 |
manual -save- |
119 |
misc -?- |
120 |
|
121 |
MITgcm code |
122 |
adjoint fold into genmake |
123 |
bin stub for ecco build |
124 |
compare01 old from 20th century |
125 |
diags timeave f77 in pkgs now |
126 |
doc tags -- connect to real docs? |
127 |
eesupp cnh? |
128 |
exe ecco user build |
129 |
*- jobs runtime shell scripts for |
130 |
| various platforms |
131 |
| lsopt line search |
132 |
m| model main dynamics (core) |
133 |
e| optimization_drivers ? |
134 |
r| optim line search interface |
135 |
g| pkg alternate and optional numerics, etc. |
136 |
e*- tools |
137 |
?| tutorial_examples documented tests |
138 |
| only populated on release1 branch |
139 |
| and not validated during "testscript" |
140 |
*- utils |
141 |
verification std tests |
142 |
|
143 |
|
144 |
mitgcmdoc -> manual -remove- |
145 |
mitgcm.org build web site |
146 |
models -?- |
147 |
packages -?- |
148 |
preprocess -?- |
149 |
tmp -?- |
150 |
</programlisting> |
151 |
|
152 |
<para>Efforts are underway to reduce the complexity.</para> |
153 |
|
154 |
</sect2> |
155 |
|
156 |
<!-- |
157 |
<sect2> |
158 |
<title>Releases</title> <para>Currently, there are two main |
159 |
branches:</para> |
160 |
<itemizedlist mark="bullet"> |
161 |
<listitem> |
162 |
<para>Development</para> |
163 |
<itemizedlist mark="bullet"> |
164 |
<listitem> |
165 |
<para>MAIN</para> |
166 |
</listitem> |
167 |
<listitem> |
168 |
<para>ecco-branch</para> |
169 |
</listitem> |
170 |
</itemizedlist> |
171 |
</listitem> |
172 |
<listitem> |
173 |
<para>Production</para> |
174 |
<itemizedlist mark="bullet"> |
175 |
<listitem> |
176 |
<para>Release1</para> |
177 |
</listitem> |
178 |
<listitem> |
179 |
<para>Release2</para> |
180 |
</listitem> |
181 |
</itemizedlist> |
182 |
</listitem> |
183 |
</itemizedlist> |
184 |
</sect2> |
185 |
--> |
186 |
|
187 |
<sect2> |
188 |
<title>Branches</title> |
189 |
|
190 |
<para>As shown in the online <ulink |
191 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated |
192 |
tree</ulink>, the MITgcm codebase is split into to two branches |
193 |
or "lines" under which development proceeds. These two lines |
194 |
are referred to as the "MAIN" and "ecco" versions of the code. |
195 |
While not identical, the bulk of the MAIN and ecco lines are |
196 |
composed of files from the same codebase. |
197 |
</para> |
198 |
|
199 |
<para>Periodically, a "Release" branch is formed from the "MAIN" |
200 |
development branch. This is done in order to create a |
201 |
relatively stable reference point for both users and developers. |
202 |
The intent is that once a relese branch has been created, only |
203 |
bug-fixes will be added to it. Meanwhile, development (which |
204 |
might "break" or otherwise render invalid the documentation, |
205 |
tutorials, and/or examples contained within a release branch) is |
206 |
allowed to continue along the MAIN and ecco lines.</para> |
207 |
</sect2> |
208 |
|
209 |
<sect2> |
210 |
<title>Tagging</title> |
211 |
|
212 |
<para>The intent of tagging is to create "known-good" |
213 |
checkpoints that developers can use as references. |
214 |
Traditionally, MITgcm tagging has maintained the following |
215 |
conventions:</para> |
216 |
|
217 |
<orderedlist> |
218 |
<listitem> |
219 |
<para>Developer checks out code into a local CVS-managed |
220 |
directory, makes various changes/additions, tests these |
221 |
edits, and eventually reaches a point where (s)he is |
222 |
satisfied that the changes form a new "useful" point in the |
223 |
evolution of the code.</para> |
224 |
</listitem> |
225 |
|
226 |
<listitem> |
227 |
<para>The developer then runs the <ulink |
228 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink> |
229 |
shell script to see if any problems are introduced. While |
230 |
not intended to be exhaustive, the test cases within the |
231 |
verification directory do provide some indication whether |
232 |
gross errors have been introduced. |
233 |
</para> |
234 |
</listitem> |
235 |
|
236 |
<listitem> |
237 |
<para>Having satisfied him- or herself that the changes are |
238 |
ready to be committed to the CVS repository, the developer |
239 |
then:</para> |
240 |
<orderedlist> |
241 |
<listitem> |
242 |
<para>adds a "checkpointXY_pre" comment (where X is a |
243 |
checkpoint number and Y is a letter) to the <ulink |
244 |
url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink> |
245 |
file and checks it into the CVS repository</para> |
246 |
</listitem> |
247 |
<listitem> |
248 |
<para>submits the set of changes to the CVS repository |
249 |
and adds comments to <filename>tag-index</filename> |
250 |
describing what the changes are along with a matching |
251 |
"checkpointXY_post" entry</para> |
252 |
</listitem> |
253 |
</orderedlist> |
254 |
</listitem> |
255 |
</orderedlist> |
256 |
|
257 |
<para>The result of this tagging procedure is a sequence of |
258 |
development checkpoints with comments which resembles:</para> |
259 |
|
260 |
<programlisting> |
261 |
checkpoint50e_post |
262 |
o make KPP work with PTRACERS |
263 |
- fix gad_calc_rhs to call new routine kpp_transport_ptr, which is |
264 |
nearly a copy of kpp_transport_s |
265 |
- there is no analogue to SurfaceTendencyS, so I have to use |
266 |
gPtr(of the surface layer) instead |
267 |
o add a new platform SunFire+mpi (SunFire 15000) to genmake |
268 |
checkpoint50e_pre |
269 |
|
270 |
checkpoint50d_post |
271 |
o change kpp output from multiple-record state files to single-record state |
272 |
files analogous to write_state.F |
273 |
o reduce the output frequency of cg3d-related stuff to the monitor frequency, |
274 |
analogous to the cg2d-related output. |
275 |
o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512, |
276 |
so that writing to internal file fn (with length 512) fails. |
277 |
checkpoint50d_pre |
278 |
</programlisting> |
279 |
|
280 |
<para>This information can be used to refer to various stages of |
281 |
the code development. For example, bugs can be traced to |
282 |
individual sets of CVS checkins based upon their first |
283 |
appearance when comparing the results from different |
284 |
checkpoints.</para> |
285 |
|
286 |
</sect2> |
287 |
</sect1> |
288 |
|
289 |
|
290 |
<sect1 id="documentation"> |
291 |
<title>Editing the Documentation</title> |
292 |
|
293 |
<sect2 id="documentation_getting"> |
294 |
<title>Getting the Docs and Code</title> |
295 |
|
296 |
<para>The first step towards editing the documentation is to |
297 |
checkout a copy of code, docs, and build scripts from the CVS |
298 |
server using:</para> |
299 |
|
300 |
<screen> |
301 |
$ export CVS_RSH=ssh |
302 |
$ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack' |
303 |
$ mkdir scratch |
304 |
$ cvs co MITgcm manual mitgcm.org |
305 |
</screen> |
306 |
|
307 |
<para>These commands extract the necessary information from the |
308 |
CVS server and create a temporary (called |
309 |
<filename>scratch</filename>) directory for the storage of the |
310 |
HTML and other files that will be created. Please note that you |
311 |
must either create <filename>scratch</filename> as shown or edit |
312 |
the various <filename>Makefile</filename>s and scripts used to |
313 |
create the documentation.</para> |
314 |
</sect2> |
315 |
|
316 |
<sect2> |
317 |
<title>Editing</title> |
318 |
|
319 |
<para>The documentation is contained in the |
320 |
<filename>manual</filename> directory in a raw LaTeX format. |
321 |
The main document is <filename>manual.tex</filename> and it uses |
322 |
<command>\input{}</command>s to include the chapters and |
323 |
subsections.</para> |
324 |
|
325 |
<para>Since the same LaTeX source is used to produce PostScript, |
326 |
PDF, and HTML output, care should be taken to follow certain |
327 |
conventions. Two of the most important are the usage of the |
328 |
<command>\filelink{}{}</command> and |
329 |
<command>\varlink{}{}</command> commands. Both of these |
330 |
commands have been defined to simplify the connection between |
331 |
the automatically generated ("code browser") HTML and the HTML |
332 |
version of the manual produced by LaTeX2HTML. They each take |
333 |
two arguments (corresponding to the contents of the two sets of |
334 |
curly braces) which are the text that the author wishes to be |
335 |
"wrapped" within the link, and a specially formatted link thats |
336 |
relative to the <filename>MITgcm</filename> directory within the |
337 |
CVS tree.</para> |
338 |
|
339 |
<para>The result is a command that resembles either</para> |
340 |
|
341 |
<orderedlist> |
342 |
<listitem> |
343 |
<para>a reference to a variable or subroutine name such as |
344 |
<command>\varlink{tRef}{tRef}</command>, or </para> |
345 |
</listitem> |
346 |
|
347 |
<listitem> |
348 |
<para>a reference to a file such as |
349 |
<command>\varlink{tRef}{path-to-the-file_name.F}</command> |
350 |
where the absolute path to the file is of the form |
351 |
<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para> |
352 |
<para>(please note how the leading "/foo/MITgcm" |
353 |
component of the path is dropped leaving the path |
354 |
<emphasis>relative</emphasis> to the head of the code |
355 |
directory and each directory separator "/" is turned |
356 |
into a "-")</para> |
357 |
</listitem> |
358 |
</orderedlist> |
359 |
|
360 |
|
361 |
|
362 |
</sect2> |
363 |
|
364 |
<sect2> |
365 |
<title>Building</title> <para>Given the directory structure of |
366 |
<xref linkend="documentation_getting">, the entire documentation |
367 |
for the web site can be built using:</para> |
368 |
|
369 |
<screen> |
370 |
$ cd mitgcm.org/devel/buildweb |
371 |
$ make All |
372 |
</screen> |
373 |
|
374 |
<para>Which builds the PDF from the LaTeX source, creates the |
375 |
HTML output from the LaTeX source, parses the FORTRAN code base |
376 |
to produce a hyperlinked HTML version of the source, and then |
377 |
determines the cross-linking between the various HTML |
378 |
components.</para> |
379 |
|
380 |
<para>If there are no errors, the result of the build process |
381 |
(which can take 30+ minutes on a P4/2.5Ghz) will be contained |
382 |
within a single directory called |
383 |
<filename>scratch/dev_docs</filename>. This is a freshly built |
384 |
version of the entire on-line users manual. If you have the |
385 |
correct permissions, it can be directly copied to the web server |
386 |
area:</para> |
387 |
|
388 |
<screen> |
389 |
$ mv scratch/dev_docs /u/u0/httpd/html |
390 |
</screen> |
391 |
|
392 |
<para>and the update is complete.</para> |
393 |
|
394 |
</sect2> |
395 |
|
396 |
</sect1> |
397 |
|
398 |
|
399 |
|
400 |
</article> |