/[MITgcm]/MITgcm/doc/devel_HOWTO.sgml
ViewVC logotype

Contents of /MITgcm/doc/devel_HOWTO.sgml

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.2 - (show annotations) (download) (as text)
Wed Aug 20 19:47:14 2003 UTC (20 years, 7 months ago) by edhill
Branch: MAIN
Changes since 1.1: +216 -1 lines
File MIME type: text/x-sgml
Two confirmed "optfiles" for genmake2 and minor updates to the devel_HOWTO.

1 <!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
2
3 <article id="MITgcm-Development-HOWTO">
4
5 <!--
6 Build commands:
7 db2ps -d ldp.dsl devel_HOWTO.sgml
8 db2pdf -d ldp.dsl devel_HOWTO.sgml
9 db2html -d ./ldp.dsl devel_HOWTO.sgml
10 db2html -u -d ./ldp.dsl devel_HOWTO.sgml
11 -->
12
13 <articleinfo>
14 <title>MITgcm Development HOWTO</title>
15
16 <author>
17 <firstname>Ed</firstname>
18 <surname>Hill III</surname>
19 <affiliation>
20 <address><email>eh3@mit.edu</email></address>
21 </affiliation>
22 </author>
23
24 <revhistory>
25 <revision>
26 <revnumber>0.01</revnumber>
27 <date>2003-08-07</date>
28 <authorinitials>eh3</authorinitials>
29 <revremark>
30 Initial version.
31 </revremark>
32 </revision>
33 </revhistory>
34
35 <abstract>
36 <para>This document describes how to develop software for the
37 MITgcm project.</para>
38 </abstract>
39 </articleinfo>
40
41 <sect1 id="intro">
42 <title>Introduction</title> <para>The purpose of this document is
43 to help new developers get "up to speed" with MITgcm
44 development.</para>
45 <sect2>
46 <title>New Versions of This Document</title> <para>You can
47 obtain the latest version of this document <ulink
48 url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in
49 various formats.</para>
50 </sect2>
51 <sect2>
52 <title>Feedback and corrections</title> <para>If you have
53 questions or comments about this document, please feel free to
54 <ulink url="mailto:MITgcm-support@mitgcm.org">contact the
55 authors</ulink>.
56 </para>
57 </sect2>
58 </sect1>
59
60 <sect1 id="background">
61 <title>Background</title>
62
63 <sect2>
64 <title>User Manual</title>
65
66 <para>Before jumping into
67 development, please familiarize yourself with the MITgcm user
68 manual which is available <ulink
69 url="http://mitgcm.org/">on the main web page</ulink>. This
70 document contains volumes of useful information and is included
71 here by reference.</para>
72
73 <para>Also, a "snapshot" or<ulink
74 url="http://mitgcm.org/dev_docs/">development version</ulink> of
75 the user manual may be available, though this is only put on the
76 web for testing purposes.</para>
77 </sect2>
78
79 <sect2>
80 <title>Prerequisites</title> <para>To develop for MITgcm project
81 you will need a UNIX or UNIX-like set of build tools including
82 the following:</para>
83 <blockquote>
84 <simplelist type="inline">
85 <member>CVS client</member>
86 <member>make or (preferably) GNU make</member>
87 <member>FORTRAN compiler</member>
88 <member>C compiler</member>
89 <member>[ba]sh and [t]csh shells</member>
90 <member>PERL</member>
91 <member>LaTeX and LaTeX2HTML</member>
92 </simplelist>
93 </blockquote>
94 <para>Essentially all of the work described here has been tested
95 on recent versions of Red Hat Linux (eg. 7.3 through 9). Except
96 where noted, all shell commands will be provided using bash
97 syntax.
98 </para>
99 </sect2>
100
101 </sect1>
102
103 <sect1 id="cvs">
104 <title>CVS Repository</title>
105 <sect2>
106 <title>Layout</title>
107
108 <para>Unlike many open source projects, the MITgcm CVS tree does
109 not follow a simple "src", "docs", "share", and "test" directory
110 layout. Instead, there are multiple higher-level directories
111 that each, to some extent, depend upon the presence of the
112 others. The tree currently resembles:</para>
113
114 <programlisting>gcmpack/
115 MITgcm-contrib contributed code
116 CS-regrid goes into utils
117 cvspolicy.html -save-
118 CVSROOT -save-
119 development experimental stuff
120 manual -save-
121 misc -?-
122
123 MITgcm code
124 adjoint fold into genmake
125 bin stub for ecco build
126 compare01 old from 20th century
127 diags timeave f77 in pkgs now
128 doc tags -- connect to real docs?
129 eesupp cnh?
130 exe ecco user build
131 *- jobs runtime shell scripts for
132 | various platforms
133 | lsopt line search
134 m| model main dynamics (core)
135 e| optimization_drivers ?
136 r| optim line search interface
137 g| pkg alternate and optional numerics, etc.
138 e*- tools
139 ?| tutorial_examples documented tests
140 | only populated on release1 branch
141 | and not validated during "testscript"
142 *- utils
143 verification std tests
144
145
146 mitgcmdoc -> manual -remove-
147 mitgcm.org build web site
148 models -?-
149 packages -?-
150 preprocess -?-
151 tmp -?-
152 </programlisting>
153
154 <para>Efforts are underway to reduce the complexity.</para>
155
156 </sect2>
157
158 <!--
159 <sect2>
160 <title>Releases</title> <para>Currently, there are two main
161 branches:</para>
162 <itemizedlist mark="bullet">
163 <listitem>
164 <para>Development</para>
165 <itemizedlist mark="bullet">
166 <listitem>
167 <para>MAIN</para>
168 </listitem>
169 <listitem>
170 <para>ecco-branch</para>
171 </listitem>
172 </itemizedlist>
173 </listitem>
174 <listitem>
175 <para>Production</para>
176 <itemizedlist mark="bullet">
177 <listitem>
178 <para>Release1</para>
179 </listitem>
180 <listitem>
181 <para>Release2</para>
182 </listitem>
183 </itemizedlist>
184 </listitem>
185 </itemizedlist>
186 </sect2>
187 -->
188
189 <sect2>
190 <title>Branches</title>
191
192 <para>As shown in the online <ulink
193 url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated
194 tree</ulink>, the MITgcm codebase is split into to two branches
195 or "lines" under which development proceeds. These two lines
196 are referred to as the "MAIN" and "ecco" versions of the code.
197 While not identical, the bulk of the MAIN and ecco lines are
198 composed of files from the same codebase.
199 </para>
200
201 <para>Periodically, a "Release" branch is formed from the "MAIN"
202 development branch. This is done in order to create a
203 relatively stable reference point for both users and developers.
204 The intent is that once a relese branch has been created, only
205 bug-fixes will be added to it. Meanwhile, development (which
206 might "break" or otherwise render invalid the documentation,
207 tutorials, and/or examples contained within a release branch) is
208 allowed to continue along the MAIN and ecco lines.</para>
209 </sect2>
210
211 <sect2>
212 <title>Tagging</title>
213
214 <para>The intent of tagging is to create "known-good"
215 checkpoints that developers can use as references.
216 Traditionally, MITgcm tagging has maintained the following
217 conventions:</para>
218
219 <orderedlist>
220 <listitem>
221 <para>Developer checks out code into a local CVS-managed
222 directory, makes various changes/additions, tests these
223 edits, and eventually reaches a point where (s)he is
224 satisfied that the changes form a new "useful" point in the
225 evolution of the code.</para>
226 </listitem>
227
228 <listitem>
229 <para>The developer then runs the <ulink
230 url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink>
231 shell script to see if any problems are introduced. While
232 not intended to be exhaustive, the test cases within the
233 verification directory do provide some indication whether
234 gross errors have been introduced.
235 </para>
236 </listitem>
237
238 <listitem>
239 <para>Having satisfied him- or herself that the changes are
240 ready to be committed to the CVS repository, the developer
241 then:</para>
242 <orderedlist>
243 <listitem>
244 <para>adds a "checkpointXY_pre" comment (where X is a
245 checkpoint number and Y is a letter) to the <ulink
246 url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink>
247 file and checks it into the CVS repository</para>
248 </listitem>
249 <listitem>
250 <para>submits the set of changes to the CVS repository
251 and adds comments to <filename>tag-index</filename>
252 describing what the changes are along with a matching
253 "checkpointXY_post" entry</para>
254 </listitem>
255 </orderedlist>
256 </listitem>
257 </orderedlist>
258
259 <para>The result of this tagging procedure is a sequence of
260 development checkpoints with comments which resembles:</para>
261
262 <programlisting>
263 checkpoint50e_post
264 o make KPP work with PTRACERS
265 - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is
266 nearly a copy of kpp_transport_s
267 - there is no analogue to SurfaceTendencyS, so I have to use
268 gPtr(of the surface layer) instead
269 o add a new platform SunFire+mpi (SunFire 15000) to genmake
270 checkpoint50e_pre
271
272 checkpoint50d_post
273 o change kpp output from multiple-record state files to single-record state
274 files analogous to write_state.F
275 o reduce the output frequency of cg3d-related stuff to the monitor frequency,
276 analogous to the cg2d-related output.
277 o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,
278 so that writing to internal file fn (with length 512) fails.
279 checkpoint50d_pre
280 </programlisting>
281
282 <para>This information can be used to refer to various stages of
283 the code development. For example, bugs can be traced to
284 individual sets of CVS checkins based upon their first
285 appearance when comparing the results from different
286 checkpoints.</para>
287
288 </sect2>
289 </sect1>
290
291
292 <sect1 id="documentation">
293 <title>Editing the Documentation</title>
294
295 <sect2 id="documentation_getting">
296 <title>Getting the Docs and Code</title>
297
298 <para>The first step towards editing the documentation is to
299 checkout a copy of code, docs, and build scripts from the CVS
300 server using:</para>
301
302 <screen>
303 $ export CVS_RSH=ssh
304 $ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack'
305 $ mkdir scratch
306 $ cvs co MITgcm manual mitgcm.org
307 </screen>
308
309 <para>These commands extract the necessary information from the
310 CVS server and create a temporary (called
311 <filename>scratch</filename>) directory for the storage of the
312 HTML and other files that will be created. Please note that you
313 must either create <filename>scratch</filename> as shown or edit
314 the various <filename>Makefile</filename>s and scripts used to
315 create the documentation.</para>
316 </sect2>
317
318 <sect2>
319 <title>Editing</title>
320
321 <para>The documentation is contained in the
322 <filename>manual</filename> directory in a raw LaTeX format.
323 The main document is <filename>manual.tex</filename> and it uses
324 <command>\input{}</command>s to include the chapters and
325 subsections.</para>
326
327 <para>Since the same LaTeX source is used to produce PostScript,
328 PDF, and HTML output, care should be taken to follow certain
329 conventions. Two of the most important are the usage of the
330 <command>\filelink{}{}</command> and
331 <command>\varlink{}{}</command> commands. Both of these
332 commands have been defined to simplify the connection between
333 the automatically generated ("code browser") HTML and the HTML
334 version of the manual produced by LaTeX2HTML. They each take
335 two arguments (corresponding to the contents of the two sets of
336 curly braces) which are the text that the author wishes to be
337 "wrapped" within the link, and a specially formatted link thats
338 relative to the <filename>MITgcm</filename> directory within the
339 CVS tree.</para>
340
341 <para>The result is a command that resembles either</para>
342
343 <orderedlist>
344 <listitem>
345 <para>a reference to a variable or subroutine name such as
346 <command>\varlink{tRef}{tRef}</command>, or </para>
347 </listitem>
348
349 <listitem>
350 <para>a reference to a file such as
351 <command>\varlink{tRef}{path-to-the-file_name.F}</command>
352 where the absolute path to the file is of the form
353 <filename>/foo/MITgcm/path/to/the/file_name.F</filename></para>
354 <para>(please note how the leading "/foo/MITgcm"
355 component of the path is dropped leaving the path
356 <emphasis>relative</emphasis> to the head of the code
357 directory and each directory separator "/" is turned
358 into a "-")</para>
359 </listitem>
360 </orderedlist>
361
362
363
364 </sect2>
365
366 <sect2>
367 <title>Building</title> <para>Given the directory structure of
368 <xref linkend="documentation_getting">, the entire documentation
369 for the web site can be built using:</para>
370
371 <screen>
372 $ cd mitgcm.org/devel/buildweb
373 $ make All
374 </screen>
375
376 <para>Which builds the PDF from the LaTeX source, creates the
377 HTML output from the LaTeX source, parses the FORTRAN code base
378 to produce a hyperlinked HTML version of the source, and then
379 determines the cross-linking between the various HTML
380 components.</para>
381
382 <para>If there are no errors, the result of the build process
383 (which can take 30+ minutes on a P4/2.5Ghz) will be contained
384 within a single directory called
385 <filename>scratch/dev_docs</filename>. This is a freshly built
386 version of the entire on-line users manual. If you have the
387 correct permissions, it can be directly copied to the web server
388 area:</para>
389
390 <screen>
391 $ mv scratch/dev_docs /u/u0/httpd/html
392 </screen>
393
394 <para>and the update is complete.</para>
395
396 </sect2>
397
398 </sect1>
399
400 <sect1 id="coding">
401 <title>Coding</title>
402
403 <sect2 id="packages">
404 <title>Coding Packages</title>
405
406 <para>Optional parts of code have been separated from the MITgcmUV
407 core driver code and organised into packages. The packaging
408 structure provides a mechanism for maintaining suites of code,
409 specific to particular classes of problems, in a way that is
410 cleanly separated from the generic fluid dynamical
411 engine.</para>
412
413 <para>The MITgcmUV packaging structure is described below using
414 generic package names ${pkg}. A concrete examples of a package
415 is the code for implementing GM/Redi mixing. This code uses the
416 package name</para>
417
418 </sect2>
419
420 </sect1>
421
422 <sect1>
423 <title>Chris's Notes...</title>
424
425 <programlisting>
426 MITgcmUV Packages
427 =================
428
429 Optional parts of code are separated from
430 the MITgcmUV core driver code and organised into
431 packages. The packaging structure provides a mechanism for
432 maintaining suites of code, specific to particular
433 classes of problem, in a way that is cleanly
434 separated from the generic fluid dynamical engine.
435
436 The MITgcmUV packaging structure is describe
437 below using generic package names ${pkg}.
438 A concrete examples of a package is the code
439 for implementing GM/Redi mixing. This code uses
440 the package name
441 * ${PKG} = GMREDI
442 * ${pkg} = gmredi
443 * ${Pkg} = gmRedi
444
445 Package states
446 ==============
447
448 Packages can be any one of four states, included,
449 excluded, enabled, disabled as follows:
450
451 included(excluded) compile time state which
452 includes(excludes) package
453 code and routine calls from
454 compilation/linking etc...
455
456 enabled(disabled) run-time state which
457 enables(disables) package code
458 execution.
459
460 Every call to a ${pkg}_... routine from outside the package
461 should be placed within both a
462 #ifdef ALLOW_${PKG} ... block and a
463 if ( use${Pkg} ) ... then block.
464 Package states are generally not expected to change during
465 a model run.
466
467 Package structure
468 =================
469
470 o Each package gets its runtime configuration
471 parameters from a file named "data.${pkg}"
472 Package runtime config. options are imported
473 into a common block held in a header file
474 called "${PKG}.h".
475
476 o The core driver part of the model can check
477 for runtime enabling or disabling of individual packages
478 through logical flags use${Pkg}.
479 The information is loaded from a
480 global package setup file called "data.pkg".
481 The use${Pkg} flags are not used within
482 individual packages.
483
484 o Included in "${PKG}.h" is a logical flag
485 called ${Pkg}IsOn. The "${PKG}.h" header file can be imported
486 by other packages to check dependencies and requirements
487 from other packages ( see "Package Boot Sequence" section).
488 NOTE: This procedure is not presently implemented,
489 ----- neither for kpp nor for gmRedi.
490
491 CPP Flags
492 =========
493
494 1. Within the core driver code flags of the form
495 ALLOW_${PKG} are used to include or exclude
496 whole packages. The ALLOW_${PKG} flags are included
497 from a PKG_CPP_OPTIONS block which is currently
498 held in-line in the CPP_OPTIONS.h header file.
499 e.g.
500
501 Core model code .....
502
503 #include "CPP_OPTIONS.h"
504 :
505 :
506 :
507
508 #ifdef ALLOW_${PKG}
509 if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...)
510 #endif
511
512 2. Within an individual package a header file,
513 "${PKG}_OPTIONS.h", is used to set CPP flags
514 specific to that package. It is not recommended
515 to include this file in "CPP_OPTIONS.h".
516
517
518 Package Boot Sequence
519 =====================
520
521 Calls to package routines within the core code timestepping
522 loop can vary. However, all packages follow a required
523 "boot" sequence outlined here:
524
525 1. S/R PACKAGES_BOOT()
526 :
527 CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
528
529
530 2. S/R PACKAGES_READPARMS()
531 :
532 #ifdef ALLOW_${PKG}
533 if ( use${Pkg} )
534 & CALL ${PKG}_READPARMS( retCode )
535 #endif
536
537 2. S/R PACKAGES_CHECK()
538 :
539 #ifdef ALLOW_${PKG}
540 if ( use${Pkg} )
541 & CALL ${PKG}_CHECK( retCode )
542 #else
543 if ( use${Pkg} )
544 & CALL PACKAGES_CHECK_ERROR('${PKG}')
545 #endif
546
547 3. S/R PACKAGES_INIT()
548 :
549 #ifdef ALLOW_${PKG}
550 if ( use${Pkg} )
551 & CALL ${PKG}_INIT( retCode )
552 #endif
553
554
555 Description
556 ===========
557
558 - ${PKG}_READPARMS()
559 is responsible for reading
560 in the package parameters file data.${pkg}, and storing
561 the package parameters in "${PKG}.h".
562 -> called in INITIALISE_FIXED
563
564 - ${PKG}_CHECK()
565 is responsible for validating
566 basic package setup and inter-package dependencies.
567 ${PKG}_CHECK can import other package parameters it may
568 need to check. This is done through header files "${PKG}.h".
569 It is assumed that parameters owned by other packages
570 will not be reset during ${PKG}_CHECK().
571 -> called in INITIALISE_FIXED
572
573 - ${PKG}_INIT()
574 is responsible for completing the
575 internal setup of a package. This routine is called after
576 the core model state has been completely initialised
577 but before the core model timestepping starts.
578 -> called in INITIALISE_VARIA
579
580 Summary
581 =======
582
583 - CPP options:
584 -----------------------
585 * ALLOW_${PKG} include/exclude package for compilation
586
587 - FORTRAN logical:
588 -----------------------
589 * use${Pkg} enable package for execution at runtime
590 -> declared in PARAMS.h
591 * ${Pkg}IsOn for package cross-dependency check
592 -> declared in ${PKG}.h
593 N.B.: Not presently used!
594
595 - header files
596 -----------------------
597 * ${PKG}_OPTIONS.h has further package-specific CPP options
598 * ${PKG}.h package-specific common block variables, fields
599
600 - FORTRAN source files
601 -----------------------
602 * ${pkg}_readparms.F reads parameters from file data.${pkg}
603 * ${pkg}_check.F checks package dependencies and consistencies
604 * ${pkg}_init.F initialises package-related fields
605 * ${pkg}_... .F package source code
606
607 - parameter file
608 -----------------------
609 * data.${pkg} parameter file
610 </programlisting>
611
612 </sect1>
613
614
615 </article>

  ViewVC Help
Powered by ViewVC 1.1.22