/[MITgcm]/MITgcm/doc/devel_HOWTO.sgml
ViewVC logotype

Annotation of /MITgcm/doc/devel_HOWTO.sgml

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.2 - (hide annotations) (download) (as text)
Wed Aug 20 19:47:14 2003 UTC (20 years, 8 months ago) by edhill
Branch: MAIN
Changes since 1.1: +216 -1 lines
File MIME type: text/x-sgml
Two confirmed "optfiles" for genmake2 and minor updates to the devel_HOWTO.

1 edhill 1.1 <!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
2    
3     <article id="MITgcm-Development-HOWTO">
4    
5     <!--
6     Build commands:
7 edhill 1.2 db2ps -d ldp.dsl devel_HOWTO.sgml
8 edhill 1.1 db2pdf -d ldp.dsl devel_HOWTO.sgml
9     db2html -d ./ldp.dsl devel_HOWTO.sgml
10 edhill 1.2 db2html -u -d ./ldp.dsl devel_HOWTO.sgml
11 edhill 1.1 -->
12    
13     <articleinfo>
14     <title>MITgcm Development HOWTO</title>
15    
16     <author>
17     <firstname>Ed</firstname>
18     <surname>Hill III</surname>
19     <affiliation>
20     <address><email>eh3@mit.edu</email></address>
21     </affiliation>
22     </author>
23    
24     <revhistory>
25     <revision>
26     <revnumber>0.01</revnumber>
27 edhill 1.2 <date>2003-08-07</date>
28 edhill 1.1 <authorinitials>eh3</authorinitials>
29     <revremark>
30     Initial version.
31     </revremark>
32     </revision>
33     </revhistory>
34    
35     <abstract>
36     <para>This document describes how to develop software for the
37     MITgcm project.</para>
38     </abstract>
39     </articleinfo>
40    
41     <sect1 id="intro">
42     <title>Introduction</title> <para>The purpose of this document is
43     to help new developers get "up to speed" with MITgcm
44     development.</para>
45     <sect2>
46     <title>New Versions of This Document</title> <para>You can
47     obtain the latest version of this document <ulink
48     url="http://mitgcm.org/dev_docs/devel_HOWTO/">online</ulink> in
49     various formats.</para>
50     </sect2>
51     <sect2>
52     <title>Feedback and corrections</title> <para>If you have
53     questions or comments about this document, please feel free to
54     <ulink url="mailto:MITgcm-support@mitgcm.org">contact the
55     authors</ulink>.
56     </para>
57     </sect2>
58     </sect1>
59    
60     <sect1 id="background">
61     <title>Background</title>
62    
63     <sect2>
64     <title>User Manual</title>
65    
66     <para>Before jumping into
67     development, please familiarize yourself with the MITgcm user
68     manual which is available <ulink
69     url="http://mitgcm.org/">on the main web page</ulink>. This
70     document contains volumes of useful information and is included
71     here by reference.</para>
72    
73     <para>Also, a "snapshot" or<ulink
74     url="http://mitgcm.org/dev_docs/">development version</ulink> of
75     the user manual may be available, though this is only put on the
76     web for testing purposes.</para>
77     </sect2>
78    
79     <sect2>
80     <title>Prerequisites</title> <para>To develop for MITgcm project
81     you will need a UNIX or UNIX-like set of build tools including
82     the following:</para>
83     <blockquote>
84     <simplelist type="inline">
85     <member>CVS client</member>
86     <member>make or (preferably) GNU make</member>
87     <member>FORTRAN compiler</member>
88     <member>C compiler</member>
89     <member>[ba]sh and [t]csh shells</member>
90     <member>PERL</member>
91     <member>LaTeX and LaTeX2HTML</member>
92     </simplelist>
93     </blockquote>
94     <para>Essentially all of the work described here has been tested
95     on recent versions of Red Hat Linux (eg. 7.3 through 9). Except
96     where noted, all shell commands will be provided using bash
97     syntax.
98     </para>
99     </sect2>
100    
101     </sect1>
102    
103     <sect1 id="cvs">
104     <title>CVS Repository</title>
105     <sect2>
106     <title>Layout</title>
107    
108     <para>Unlike many open source projects, the MITgcm CVS tree does
109     not follow a simple "src", "docs", "share", and "test" directory
110     layout. Instead, there are multiple higher-level directories
111     that each, to some extent, depend upon the presence of the
112     others. The tree currently resembles:</para>
113    
114     <programlisting>gcmpack/
115     MITgcm-contrib contributed code
116     CS-regrid goes into utils
117     cvspolicy.html -save-
118     CVSROOT -save-
119     development experimental stuff
120     manual -save-
121     misc -?-
122    
123     MITgcm code
124     adjoint fold into genmake
125     bin stub for ecco build
126     compare01 old from 20th century
127     diags timeave f77 in pkgs now
128     doc tags -- connect to real docs?
129     eesupp cnh?
130     exe ecco user build
131     *- jobs runtime shell scripts for
132     | various platforms
133     | lsopt line search
134     m| model main dynamics (core)
135     e| optimization_drivers ?
136     r| optim line search interface
137     g| pkg alternate and optional numerics, etc.
138     e*- tools
139     ?| tutorial_examples documented tests
140     | only populated on release1 branch
141     | and not validated during "testscript"
142     *- utils
143     verification std tests
144    
145    
146     mitgcmdoc -> manual -remove-
147     mitgcm.org build web site
148     models -?-
149     packages -?-
150     preprocess -?-
151     tmp -?-
152     </programlisting>
153    
154     <para>Efforts are underway to reduce the complexity.</para>
155    
156     </sect2>
157    
158     <!--
159     <sect2>
160     <title>Releases</title> <para>Currently, there are two main
161     branches:</para>
162     <itemizedlist mark="bullet">
163     <listitem>
164     <para>Development</para>
165     <itemizedlist mark="bullet">
166     <listitem>
167     <para>MAIN</para>
168     </listitem>
169     <listitem>
170     <para>ecco-branch</para>
171     </listitem>
172     </itemizedlist>
173     </listitem>
174     <listitem>
175     <para>Production</para>
176     <itemizedlist mark="bullet">
177     <listitem>
178     <para>Release1</para>
179     </listitem>
180     <listitem>
181     <para>Release2</para>
182     </listitem>
183     </itemizedlist>
184     </listitem>
185     </itemizedlist>
186     </sect2>
187     -->
188    
189     <sect2>
190     <title>Branches</title>
191    
192     <para>As shown in the online <ulink
193     url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index?graph=1.174">ViewCVS-generated
194     tree</ulink>, the MITgcm codebase is split into to two branches
195     or "lines" under which development proceeds. These two lines
196     are referred to as the "MAIN" and "ecco" versions of the code.
197     While not identical, the bulk of the MAIN and ecco lines are
198     composed of files from the same codebase.
199     </para>
200    
201     <para>Periodically, a "Release" branch is formed from the "MAIN"
202     development branch. This is done in order to create a
203     relatively stable reference point for both users and developers.
204     The intent is that once a relese branch has been created, only
205     bug-fixes will be added to it. Meanwhile, development (which
206     might "break" or otherwise render invalid the documentation,
207     tutorials, and/or examples contained within a release branch) is
208     allowed to continue along the MAIN and ecco lines.</para>
209     </sect2>
210    
211     <sect2>
212     <title>Tagging</title>
213    
214     <para>The intent of tagging is to create "known-good"
215     checkpoints that developers can use as references.
216     Traditionally, MITgcm tagging has maintained the following
217     conventions:</para>
218    
219     <orderedlist>
220     <listitem>
221     <para>Developer checks out code into a local CVS-managed
222     directory, makes various changes/additions, tests these
223     edits, and eventually reaches a point where (s)he is
224     satisfied that the changes form a new "useful" point in the
225     evolution of the code.</para>
226     </listitem>
227    
228     <listitem>
229     <para>The developer then runs the <ulink
230     url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/verification/testscript">testscript</ulink>
231     shell script to see if any problems are introduced. While
232     not intended to be exhaustive, the test cases within the
233     verification directory do provide some indication whether
234     gross errors have been introduced.
235     </para>
236     </listitem>
237    
238     <listitem>
239     <para>Having satisfied him- or herself that the changes are
240     ready to be committed to the CVS repository, the developer
241     then:</para>
242     <orderedlist>
243     <listitem>
244     <para>adds a "checkpointXY_pre" comment (where X is a
245     checkpoint number and Y is a letter) to the <ulink
246     url="http://dev.mitgcm.org/cgi-bin/viewcvs.cgi/MITgcm/doc/tag-index">tag-index</ulink>
247     file and checks it into the CVS repository</para>
248     </listitem>
249     <listitem>
250     <para>submits the set of changes to the CVS repository
251     and adds comments to <filename>tag-index</filename>
252     describing what the changes are along with a matching
253     "checkpointXY_post" entry</para>
254     </listitem>
255     </orderedlist>
256     </listitem>
257     </orderedlist>
258    
259     <para>The result of this tagging procedure is a sequence of
260     development checkpoints with comments which resembles:</para>
261    
262     <programlisting>
263     checkpoint50e_post
264     o make KPP work with PTRACERS
265     - fix gad_calc_rhs to call new routine kpp_transport_ptr, which is
266     nearly a copy of kpp_transport_s
267     - there is no analogue to SurfaceTendencyS, so I have to use
268     gPtr(of the surface layer) instead
269     o add a new platform SunFire+mpi (SunFire 15000) to genmake
270     checkpoint50e_pre
271    
272     checkpoint50d_post
273     o change kpp output from multiple-record state files to single-record state
274     files analogous to write_state.F
275     o reduce the output frequency of cg3d-related stuff to the monitor frequency,
276     analogous to the cg2d-related output.
277     o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,
278     so that writing to internal file fn (with length 512) fails.
279     checkpoint50d_pre
280     </programlisting>
281    
282     <para>This information can be used to refer to various stages of
283     the code development. For example, bugs can be traced to
284     individual sets of CVS checkins based upon their first
285     appearance when comparing the results from different
286     checkpoints.</para>
287    
288     </sect2>
289     </sect1>
290    
291    
292     <sect1 id="documentation">
293     <title>Editing the Documentation</title>
294    
295     <sect2 id="documentation_getting">
296     <title>Getting the Docs and Code</title>
297    
298     <para>The first step towards editing the documentation is to
299     checkout a copy of code, docs, and build scripts from the CVS
300     server using:</para>
301    
302     <screen>
303     $ export CVS_RSH=ssh
304     $ export CVSROOT=':ext:auden.lcs.mit.edu:/u/u3/gcmpack'
305     $ mkdir scratch
306     $ cvs co MITgcm manual mitgcm.org
307     </screen>
308    
309     <para>These commands extract the necessary information from the
310     CVS server and create a temporary (called
311     <filename>scratch</filename>) directory for the storage of the
312     HTML and other files that will be created. Please note that you
313     must either create <filename>scratch</filename> as shown or edit
314     the various <filename>Makefile</filename>s and scripts used to
315     create the documentation.</para>
316     </sect2>
317    
318     <sect2>
319     <title>Editing</title>
320    
321     <para>The documentation is contained in the
322     <filename>manual</filename> directory in a raw LaTeX format.
323     The main document is <filename>manual.tex</filename> and it uses
324     <command>\input{}</command>s to include the chapters and
325     subsections.</para>
326    
327     <para>Since the same LaTeX source is used to produce PostScript,
328     PDF, and HTML output, care should be taken to follow certain
329     conventions. Two of the most important are the usage of the
330     <command>\filelink{}{}</command> and
331     <command>\varlink{}{}</command> commands. Both of these
332     commands have been defined to simplify the connection between
333     the automatically generated ("code browser") HTML and the HTML
334     version of the manual produced by LaTeX2HTML. They each take
335     two arguments (corresponding to the contents of the two sets of
336     curly braces) which are the text that the author wishes to be
337     "wrapped" within the link, and a specially formatted link thats
338     relative to the <filename>MITgcm</filename> directory within the
339     CVS tree.</para>
340    
341     <para>The result is a command that resembles either</para>
342    
343     <orderedlist>
344     <listitem>
345     <para>a reference to a variable or subroutine name such as
346     <command>\varlink{tRef}{tRef}</command>, or </para>
347     </listitem>
348    
349     <listitem>
350     <para>a reference to a file such as
351     <command>\varlink{tRef}{path-to-the-file_name.F}</command>
352     where the absolute path to the file is of the form
353     <filename>/foo/MITgcm/path/to/the/file_name.F</filename></para>
354     <para>(please note how the leading "/foo/MITgcm"
355     component of the path is dropped leaving the path
356     <emphasis>relative</emphasis> to the head of the code
357     directory and each directory separator "/" is turned
358     into a "-")</para>
359     </listitem>
360     </orderedlist>
361    
362    
363    
364     </sect2>
365    
366     <sect2>
367     <title>Building</title> <para>Given the directory structure of
368     <xref linkend="documentation_getting">, the entire documentation
369     for the web site can be built using:</para>
370    
371     <screen>
372     $ cd mitgcm.org/devel/buildweb
373     $ make All
374     </screen>
375    
376     <para>Which builds the PDF from the LaTeX source, creates the
377     HTML output from the LaTeX source, parses the FORTRAN code base
378     to produce a hyperlinked HTML version of the source, and then
379     determines the cross-linking between the various HTML
380     components.</para>
381    
382     <para>If there are no errors, the result of the build process
383     (which can take 30+ minutes on a P4/2.5Ghz) will be contained
384     within a single directory called
385     <filename>scratch/dev_docs</filename>. This is a freshly built
386     version of the entire on-line users manual. If you have the
387     correct permissions, it can be directly copied to the web server
388     area:</para>
389    
390     <screen>
391     $ mv scratch/dev_docs /u/u0/httpd/html
392     </screen>
393    
394     <para>and the update is complete.</para>
395    
396     </sect2>
397    
398     </sect1>
399    
400 edhill 1.2 <sect1 id="coding">
401     <title>Coding</title>
402    
403     <sect2 id="packages">
404     <title>Coding Packages</title>
405    
406     <para>Optional parts of code have been separated from the MITgcmUV
407     core driver code and organised into packages. The packaging
408     structure provides a mechanism for maintaining suites of code,
409     specific to particular classes of problems, in a way that is
410     cleanly separated from the generic fluid dynamical
411     engine.</para>
412    
413     <para>The MITgcmUV packaging structure is described below using
414     generic package names ${pkg}. A concrete examples of a package
415     is the code for implementing GM/Redi mixing. This code uses the
416     package name</para>
417    
418     </sect2>
419    
420     </sect1>
421    
422     <sect1>
423     <title>Chris's Notes...</title>
424    
425     <programlisting>
426     MITgcmUV Packages
427     =================
428    
429     Optional parts of code are separated from
430     the MITgcmUV core driver code and organised into
431     packages. The packaging structure provides a mechanism for
432     maintaining suites of code, specific to particular
433     classes of problem, in a way that is cleanly
434     separated from the generic fluid dynamical engine.
435    
436     The MITgcmUV packaging structure is describe
437     below using generic package names ${pkg}.
438     A concrete examples of a package is the code
439     for implementing GM/Redi mixing. This code uses
440     the package name
441     * ${PKG} = GMREDI
442     * ${pkg} = gmredi
443     * ${Pkg} = gmRedi
444    
445     Package states
446     ==============
447    
448     Packages can be any one of four states, included,
449     excluded, enabled, disabled as follows:
450    
451     included(excluded) compile time state which
452     includes(excludes) package
453     code and routine calls from
454     compilation/linking etc...
455    
456     enabled(disabled) run-time state which
457     enables(disables) package code
458     execution.
459    
460     Every call to a ${pkg}_... routine from outside the package
461     should be placed within both a
462     #ifdef ALLOW_${PKG} ... block and a
463     if ( use${Pkg} ) ... then block.
464     Package states are generally not expected to change during
465     a model run.
466    
467     Package structure
468     =================
469    
470     o Each package gets its runtime configuration
471     parameters from a file named "data.${pkg}"
472     Package runtime config. options are imported
473     into a common block held in a header file
474     called "${PKG}.h".
475    
476     o The core driver part of the model can check
477     for runtime enabling or disabling of individual packages
478     through logical flags use${Pkg}.
479     The information is loaded from a
480     global package setup file called "data.pkg".
481     The use${Pkg} flags are not used within
482     individual packages.
483    
484     o Included in "${PKG}.h" is a logical flag
485     called ${Pkg}IsOn. The "${PKG}.h" header file can be imported
486     by other packages to check dependencies and requirements
487     from other packages ( see "Package Boot Sequence" section).
488     NOTE: This procedure is not presently implemented,
489     ----- neither for kpp nor for gmRedi.
490    
491     CPP Flags
492     =========
493    
494     1. Within the core driver code flags of the form
495     ALLOW_${PKG} are used to include or exclude
496     whole packages. The ALLOW_${PKG} flags are included
497     from a PKG_CPP_OPTIONS block which is currently
498     held in-line in the CPP_OPTIONS.h header file.
499     e.g.
500    
501     Core model code .....
502    
503     #include "CPP_OPTIONS.h"
504     :
505     :
506     :
507    
508     #ifdef ALLOW_${PKG}
509     if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...)
510     #endif
511    
512     2. Within an individual package a header file,
513     "${PKG}_OPTIONS.h", is used to set CPP flags
514     specific to that package. It is not recommended
515     to include this file in "CPP_OPTIONS.h".
516    
517    
518     Package Boot Sequence
519     =====================
520    
521     Calls to package routines within the core code timestepping
522     loop can vary. However, all packages follow a required
523     "boot" sequence outlined here:
524    
525     1. S/R PACKAGES_BOOT()
526     :
527     CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
528    
529    
530     2. S/R PACKAGES_READPARMS()
531     :
532     #ifdef ALLOW_${PKG}
533     if ( use${Pkg} )
534     & CALL ${PKG}_READPARMS( retCode )
535     #endif
536    
537     2. S/R PACKAGES_CHECK()
538     :
539     #ifdef ALLOW_${PKG}
540     if ( use${Pkg} )
541     & CALL ${PKG}_CHECK( retCode )
542     #else
543     if ( use${Pkg} )
544     & CALL PACKAGES_CHECK_ERROR('${PKG}')
545     #endif
546    
547     3. S/R PACKAGES_INIT()
548     :
549     #ifdef ALLOW_${PKG}
550     if ( use${Pkg} )
551     & CALL ${PKG}_INIT( retCode )
552     #endif
553    
554    
555     Description
556     ===========
557    
558     - ${PKG}_READPARMS()
559     is responsible for reading
560     in the package parameters file data.${pkg}, and storing
561     the package parameters in "${PKG}.h".
562     -> called in INITIALISE_FIXED
563    
564     - ${PKG}_CHECK()
565     is responsible for validating
566     basic package setup and inter-package dependencies.
567     ${PKG}_CHECK can import other package parameters it may
568     need to check. This is done through header files "${PKG}.h".
569     It is assumed that parameters owned by other packages
570     will not be reset during ${PKG}_CHECK().
571     -> called in INITIALISE_FIXED
572    
573     - ${PKG}_INIT()
574     is responsible for completing the
575     internal setup of a package. This routine is called after
576     the core model state has been completely initialised
577     but before the core model timestepping starts.
578     -> called in INITIALISE_VARIA
579    
580     Summary
581     =======
582    
583     - CPP options:
584     -----------------------
585     * ALLOW_${PKG} include/exclude package for compilation
586    
587     - FORTRAN logical:
588     -----------------------
589     * use${Pkg} enable package for execution at runtime
590     -> declared in PARAMS.h
591     * ${Pkg}IsOn for package cross-dependency check
592     -> declared in ${PKG}.h
593     N.B.: Not presently used!
594    
595     - header files
596     -----------------------
597     * ${PKG}_OPTIONS.h has further package-specific CPP options
598     * ${PKG}.h package-specific common block variables, fields
599    
600     - FORTRAN source files
601     -----------------------
602     * ${pkg}_readparms.F reads parameters from file data.${pkg}
603     * ${pkg}_check.F checks package dependencies and consistencies
604     * ${pkg}_init.F initialises package-related fields
605     * ${pkg}_... .F package source code
606    
607     - parameter file
608     -----------------------
609     * data.${pkg} parameter file
610     </programlisting>
611    
612     </sect1>
613 edhill 1.1
614    
615     </article>
616    
617    

  ViewVC Help
Powered by ViewVC 1.1.22