[MITgcm] Annotation of /MITgcm/doc/devel

1	edhill	1.1	<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
2	jmc	1.13	<!--
3	jmc	1.19	$Header: /u/gcmpack/MITgcm/doc/devel_HOWTO.sgml,v 1.18 2017/12/19 18:38:40 jmc Exp $
4	jmc	1.13	$Name: $
5			-->
6	edhill	1.1
7			<article id="MITgcm-Development-HOWTO">
8
9			<articleinfo>
10			<title>MITgcm Development HOWTO</title>
11
12			<author>
13			<firstname>Ed</firstname>
14			<surname>Hill III</surname>
15			<affiliation>
16			<address><email>eh3@mit.edu</email></address>
17			</affiliation>
18			</author>
19
20			<revhistory>
21			<revision>
22			<revnumber>0.01</revnumber>
23	edhill	1.2	<date>2003-08-07</date>
24	edhill	1.1	<authorinitials>eh3</authorinitials>
25			<revremark>
26			Initial version.
27			</revremark>
28			</revision>
29	jmc	1.10	<revision>
30			<revnumber>0.02</revnumber>
31			<date>2010-01-21</date>
32			<authorinitials>jmc</authorinitials>
33			<revremark>
34	jmc	1.16	Update links.
35	jmc	1.10	</revremark>
36			</revision>
37	jmc	1.11	<revision>
38			<revnumber>0.03</revnumber>
39			<date>2010-04-25</date>
40			<authorinitials>jmc</authorinitials>
41			<revremark>
42			Add subsection "Developer settings" (under CVS Repository).
43			</revremark>
44			</revision>
45	jmc	1.16	<revision>
46			<revnumber>0.04</revnumber>
47			<date>2011-04-24</date>
48			<authorinitials>jmc</authorinitials>
49			<revremark>
50			Update subsection "The verification suite".
51			</revremark>
52			</revision>
53	edhill	1.1	</revhistory>
54
55			<abstract>
56			<para>This document describes how to develop software for the
57			MITgcm project.</para>
58			</abstract>
59			</articleinfo>
60
61			<sect1 id="intro">
62			<title>Introduction</title> <para>The purpose of this document is
63			to help new developers get "up to speed" with MITgcm
64			development.</para>
65			<sect2>
66			<title>New Versions of This Document</title> <para>You can
67			obtain the latest version of this document <ulink
68	jmc	1.10	url="http://mitgcm.org/public/docs.html">online</ulink> in
69	edhill	1.1	various formats.</para>
70			</sect2>
71			<sect2>
72			<title>Feedback and corrections</title> <para>If you have
73			questions or comments about this document, please feel free to
74			<ulink url="mailto:MITgcm-support@mitgcm.org">contact the
75			authors</ulink>.
76			</para>
77			</sect2>
78			</sect1>
79
80			<sect1 id="background">
81			<title>Background</title>
82
83			<sect2>
84			<title>User Manual</title>
85
86	edhill	1.5	<para>Before jumping into development, please familiarize yourself with
87	jmc	1.10	the <ulink url="http://mitgcm.org/public/docs.html"> MITgcm user manual
88	edhill	1.5	</ulink>. This document contains volumes of useful information and is
89			included here by reference.</para>
90	edhill	1.1
91	edhill	1.5	<!--
92			<para>Also, a "snapshot" or <ulink
93	edhill	1.1	url="http://mitgcm.org/dev_docs/">development version</ulink> of
94			the user manual may be available, though this is only put on the
95			web for testing purposes.</para>
96	edhill	1.5	-->
97	edhill	1.1	</sect2>
98
99			<sect2>
100			<title>Prerequisites</title> <para>To develop for MITgcm project
101			you will need a UNIX or UNIX-like set of build tools including
102			the following:</para>
103			<blockquote>
104			<simplelist type="inline">
105			<member>CVS client</member>
106			<member>make or (preferably) GNU make</member>
107			<member>FORTRAN compiler</member>
108			<member>C compiler</member>
109			<member>[ba]sh and [t]csh shells</member>
110			<member>PERL</member>
111			<member>LaTeX and LaTeX2HTML</member>
112			</simplelist>
113			</blockquote>
114			<para>Essentially all of the work described here has been tested
115			on recent versions of Red Hat Linux (eg. 7.3 through 9). Except
116			where noted, all shell commands will be provided using bash
117			syntax.
118			</para>
119			</sect2>
120
121			</sect1>
122
123			<sect1 id="cvs">
124			<title>CVS Repository</title>
125	jmc	1.11
126	edhill	1.1	<sect2>
127			<title>Layout</title>
128
129			<para>Unlike many open source projects, the MITgcm CVS tree does
130			not follow a simple "src", "docs", "share", and "test" directory
131			layout. Instead, there are multiple higher-level directories
132			that each, to some extent, depend upon the presence of the
133			others. The tree currently resembles:</para>
134
135			<programlisting>gcmpack/
136	jmc	1.11	CVSROOT -hidden-
137	edhill	1.1
138			MITgcm code
139	jmc	1.11	bin empty
140			doc basic developpment documentation
141			eesupp execution environment support code (wrapper)
142			exe empty
143	jmc	1.16	jobs runtime shell scripts for
144	jmc	1.11	various platforms (not maintained)
145			lsopt line search
146			model main dynamics (core)
147			optim line search interface
148			pkg alternate and optional numerics, etc.
149			tools scripts to build (and test)
150			utils pre/post processing tools (matlab, ..)
151			verification standard regression tests + examples
152			+ documented examples (tutorials)
153			tutorial_examples (only in release1 branch)
154	edhill	1.1
155	jmc	1.11	MITgcm_contrib contributed code
156	edhill	1.1
157	jmc	1.11	acesgrid.org build acesgrid web site
158			development experimental stuff
159			gfd_lab -?-
160	jmc	1.13	manual source of MITgcm documentation
161	edhill	1.1	mitgcm.org build web site
162	jmc	1.16	old_develop old and early development source
163	jmc	1.13	misc -?-
164			models -?-
165			packages -?-
166			preprocess -?-
167	jmc	1.11	pdfs some pdfs
168			planetinabottle.org unfinished web site
169			www.ecco-group.org build ecco web site ?
170	edhill	1.1	</programlisting>
171
172	jmc	1.11	<!--
173	edhill	1.1	<para>Efforts are underway to reduce the complexity.</para>
174	jmc	1.11	-->
175	edhill	1.1
176			</sect2>
177
178			<!--
179			<sect2>
180			<title>Releases</title> <para>Currently, there are two main
181			branches:</para>
182			<itemizedlist mark="bullet">
183			<listitem>
184			<para>Development</para>
185			<itemizedlist mark="bullet">
186			<listitem>
187			<para>MAIN</para>
188			</listitem>
189			<listitem>
190			<para>ecco-branch</para>
191			</listitem>
192			</itemizedlist>
193			</listitem>
194			<listitem>
195			<para>Production</para>
196			<itemizedlist mark="bullet">
197			<listitem>
198			<para>Release1</para>
199			</listitem>
200			<listitem>
201			<para>Release2</para>
202			</listitem>
203			</itemizedlist>
204			</listitem>
205			</itemizedlist>
206			</sect2>
207			-->
208
209			<sect2>
210			<title>Branches</title>
211
212			<para>As shown in the online <ulink
213	jmc	1.11	url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph">
214	dimitri	1.12	ViewCVS-generated tree</ulink>, the MITgcm codebase is split into
215			branches or "lines" under which development proceeds. The main line
216			of development is referred to as the "MAIN" version of the code.
217	edhill	1.1	</para>
218
219			<para>Periodically, a "Release" branch is formed from the "MAIN"
220	edhill	1.7	development branch. This is done in order to create a relatively stable
221			reference point for both users and developers. The intent is that once a
222	dimitri	1.12	release branch has been created, only bug-fixes will be added to it.
223	edhill	1.7	Meanwhile, development (which might "break" or otherwise render invalid
224			the documentation, tutorials, and/or examples contained within a release
225	dimitri	1.12	branch) is allowed to continue along the MAIN line.</para>
226	edhill	1.1	</sect2>
227
228			<sect2>
229	jmc	1.11	<title> Developer settings </title>
230
231	jmc	1.16	<para>CVS is a convenient tool to keep up-to-date a personal copy of the
232			MITgcm code (see: <ulink url="http://mitgcm.org/public/using_cvs.html">
233			using CVS </ulink>). The same tool is used by developers to
234	jmc	1.11	incorporate any change into the repository. However, this later
235	jmc	1.17	function requires specific settings, as detailed here after</para>
236	jmc	1.11	<orderedlist>
237			<listitem>
238	jmc	1.17	<para> You will need an account (login access) to the server
239	jmc	1.19	"mitgcm.org" (curently: <filename>mitgcmcvs.mit.edu</filename>)
240			with the proper group setting (e.g., group "gcmctrb" to add/modify
241	jmc	1.17	code into MITgcm_contrib).
242	jmc	1.19	This kind of account is granted only upon well motivated request
243			(we recommend to ask your senior MITgcm-collaborator to send such
244			request to marshall-admin at techsquare.com with Cc to Chris Hill
245	jmc	1.17	for approval).</para>
246	jmc	1.19	<para> The access to the server <filename>mitgcm.org</filename> is
247			through ssh-key authorization which will need to be set properly on
248			both side (on your local machine and on your server account).
249			You need to be able to ssh to <filename>mitgcm.org</filename>
250	jmc	1.17	(or <filename>ssh MY_USER_NAME@mitgcm.org</filename>
251			in case of different user-name on both sides) to proceed further.</para>
252	jmc	1.11	</listitem>
253
254			<listitem>
255	jmc	1.16	<para> You need to register to the
256	jmc	1.18	<ulink url="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-cvs">
257	jmc	1.17	mitgcm-cvs </ulink> mailing list.
258	jmc	1.16	This ensures that other developers will receive email notification
259	jmc	1.17	when you make changes; you will also receive such email
260	jmc	1.11	when others make changes to the repository.
261			</para>
262			</listitem>
263
264			<listitem>
265	jmc	1.16	<para> It is highly recommended that you register also to the
266	jmc	1.18	<ulink url="http://mailman.mitgcm.org/mailman/listinfo/mitgcm-devel">
267	jmc	1.17	mitgcm-devel </ulink> mailing list (expect a short delay for
268	jmc	1.11	this request to be processed).
269			This list is intended for developer discussions.
270			</para>
271			</listitem>
272
273			<listitem>
274	jmc	1.17	<para> The standard CVS-anonymous mode (using "cvsanon",
275			as mentionned <ulink url="http://mitgcm.org/public/source_code.html">
276			here </ulink>) does not provide check-in ("cvs commit") permission.
277	jmc	1.11	Instead, you will need to set our CVS environment as follow:</para>
278			<screen>
279			$ export CVS_RSH=ssh
280			$ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack'
281			</screen>
282	jmc	1.17	<para> The reason for such limitation is that when downloading a directory,
283			e.g.: <filename>myCopy</filename>, from the CVS repository (e.g.,
284	jmc	1.11	<filename>MITgcm_contrib/thisPart</filename>) using the command:</para>
285			<screen>
286			$ cvs co -P -d myCopy MITgcm_contrib/thisPart
287			</screen>
288	jmc	1.16	<para> the type of CVS environment which has been used
289	jmc	1.17	is stored in the file <filename>myCopy/CVS/Root</filename>.
290			This prevent to re-use, for cvs-commit purpose,
291	jmc	1.16	a cvs local copy (<filename>myCopy</filename>) which was obtained
292	jmc	1.11	using the CVS anonymous mode.</para>
293			</listitem>
294
295			<listitem>
296	jmc	1.16	<para> At this stage, you should be able to send your modified source
297			file (e.g., <filename>src_file</filename>) from your local copy directory
298	jmc	1.11	(<filename>myCopy</filename>) to the CVS repository
299			(<filename>MITgcm_contrib/thisPart</filename>) using the command
300			"cvs commit":</para>
301			<screen>
302			$ cd myCopy
303			$ cvs -n update (optional; check if new changes have been made)
304			$ cvs diff src_file (optional; list your changes)
305			$ cvs commit src_file
306			</screen>
307	jmc	1.16	<para> It is essential that you provide a short description of the
308			changes you made to <filename>src_file</filename> as you check-in
309			this file (the "cvs commit" command automatically opens your standard
310	jmc	1.11	editor for this purpose).</para>
311	jmc	1.17	<para> Note:
312	jmc	1.19	Please ignore the following warnings that the "cvs commit" command
313	jmc	1.17	produces if you are not part of the "gcmpack" group:
314			<screen>
315			cvs commit: failed to create lock directory for `/u/gcmpack/CVSROOT'
316			(/u/gcmpack/CVSROOT/#cvs.history.lock): Permission denied
317			cvs commit: failed to obtain history lock in repository `/u/gcmpack'
318			</screen>
319			These warnings are not affecting the changes you made to the CVS repository.
320			</para>
321	jmc	1.11	</listitem>
322
323			</orderedlist>
324
325			</sect2>
326
327			<sect2>
328			<title>Main code development</title>
329	jmc	1.14	<para><emphasis>(formerly named "Tagging" ; this section needs an update)
330			</emphasis></para>
331	edhill	1.1
332	edhill	1.7	<para>The intent of tagging is to create "known-good" checkpoints that
333			developers can use as references. Traditionally, MITgcm tagging has
334			maintained the following conventions:</para>
335	edhill	1.1
336			<orderedlist>
337			<listitem>
338	edhill	1.7	<para>Developer checks out code into a local CVS-managed directory,
339			makes various changes/additions, tests these edits, and eventually
340			reaches a point where (s)he is satisfied that the changes form a new
341			"useful" point in the evolution of the code.</para>
342	edhill	1.1	</listitem>
343
344			<listitem>
345			<para>The developer then runs the <ulink
346	jmc	1.10	url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport">
347			testreport</ulink> shell script to see if any problems are introduced.
348	edhill	1.7	While not intended to be exhaustive, the test cases within the
349			verification directory do provide some indication whether gross errors
350			have been introduced.
351	edhill	1.1	</para>
352			</listitem>
353
354			<listitem>
355			<para>Having satisfied him- or herself that the changes are
356			ready to be committed to the CVS repository, the developer
357			then:</para>
358			<orderedlist>
359			<listitem>
360	edhill	1.7	<para>adds a "checkpointXY_pre" comment (where X is a checkpoint
361			number and Y is a letter) to the <ulink
362	jmc	1.10	url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index">
363	edhill	1.7	tag-index</ulink> file and checks it into the CVS
364			repository</para>
365	edhill	1.1	</listitem>
366			<listitem>
367	edhill	1.7	<para>submits the set of changes to the CVS repository and adds
368			comments to <filename>tag-index</filename> describing what the
369			changes are along with a matching "checkpointXY_post" entry</para>
370	edhill	1.1	</listitem>
371			</orderedlist>
372			</listitem>
373			</orderedlist>
374
375	edhill	1.7	<para>The result of this tagging procedure is a sequence of development
376			checkpoints with comments which resembles:</para>
377	edhill	1.1
378			<programlisting>
379			checkpoint50e_post
380			o make KPP work with PTRACERS
381			- fix gad_calc_rhs to call new routine kpp_transport_ptr, which is
382			nearly a copy of kpp_transport_s
383	jmc	1.16	- there is no analogue to SurfaceTendencyS, so I have to use
384	edhill	1.1	gPtr(of the surface layer) instead
385			o add a new platform SunFire+mpi (SunFire 15000) to genmake
386			checkpoint50e_pre
387
388			checkpoint50d_post
389	jmc	1.16	o change kpp output from multiple-record state files to single-record state
390	edhill	1.1	files analogous to write_state.F
391	jmc	1.16	o reduce the output frequency of cg3d-related stuff to the monitor frequency,
392			analogous to the cg2d-related output.
393			o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,
394	edhill	1.1	so that writing to internal file fn (with length 512) fails.
395			checkpoint50d_pre
396			</programlisting>
397
398	edhill	1.7	<para>This information can be used to refer to various stages of the code
399			development. For example, bugs can be traced to individual sets of CVS
400			checkins based upon their first appearance when comparing the results from
401			different checkpoints.</para>
402	edhill	1.1
403			</sect2>
404			</sect1>
405
406	edhill	1.2	<sect1 id="coding">
407	edhill	1.3	<title>Coding for MITgcm</title>
408
409			<sect2 id="build_tools">
410			<title>Build Tools</title>
411
412	edhill	1.4	<para>Many Open Source projects use the "GNU Autotools" to help streamline
413			the build process for various Unix and Unix-like architectures. For a
414			user, the result is the common "configure" (that is,
415			"<filename>./configure && make && make install</filename>") commands.
416			For MITgcm, the process is similar. Typical commands are:</para>
417	edhill	1.3
418			<screen>
419	jmc	1.14	$ genmake2 -mods=../code
420	edhill	1.5	$ make depend
421			$ make
422	edhill	1.3	</screen>
423
424	edhill	1.4	<para>The following sections describe the individual steps in the build
425			process.</para>
426	jmc	1.16
427	edhill	1.3	<sect3 id="genmake">
428			<title>The <filename>genmake2</> Utility</title>
429
430	jmc	1.14	<para><emphasis>(Note: the older <filename>genmake</>
431			has been replaced by <filename>genmake2</>)</emphasis></para>
432	edhill	1.4
433			<para>The first step in any MITgcm build is to create a Unix-style
434			<filename>Makefile</filename> which will be parsed by
435			<filename>make</filename> to specify how to compile the MITgcm source
436			files. For more detailed descriptions of what the make tools are and
437			how they are used, please see:</para>
438	edhill	1.3
439			<itemizedlist>
440			<listitem>
441	edhill	1.4	<para><ulink url="http://www.gnu.org/software/make/make.html">
442			http://www.gnu.org/software/make/make.html</></para>
443	edhill	1.3	</listitem>
444			<listitem>
445	edhill	1.4	<para><ulink url="http://www.oreilly.com/catalog/make2/">
446			http://www.oreilly.com/catalog/make2/</></para>
447	edhill	1.3	</listitem>
448			</itemizedlist>
449
450	edhill	1.4	<para>Genmake can often be invoked successfully with a command line as
451			simple as:</para>
452
453			<screen>
454	edhill	1.5	$ genmake2 -mods=../code
455	edhill	1.4	</screen>
456
457			<para>However, some systems (particularly commercial Unixes that lack a
458			more modern "/bin/sh" implementation or that have shells installed in
459			odd locations) may require an explicit shell invocation such as one of
460			the following: </para>
461
462			<screen>
463	edhill	1.5	$ /usr/bin/sh genmake2 -make=gmake -mods=../code
464			$ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code
465	edhill	1.4	</screen>
466
467			<para>The genmake2 code has been written in a Bourne and BASH (v1)
468			compatible syntax so it should work with most "sh" and all recent "bash"
469			implementations.</para>
470
471			<para>As the name implies, <filename>genmake2</filename> generates a
472			<filename>Makefile</filename>. It does so by first parsing the
473			information supplied from the following sources</para>
474	edhill	1.3
475			<orderedlist>
476			<listitem>
477	jmc	1.6	<para>a <filename>gemake_local</filename> file in the current
478	edhill	1.4	directory</para>
479	edhill	1.3	</listitem>
480			<listitem>
481			<para>directly from command-line options</para>
482			</listitem>
483			<listitem>
484	edhill	1.4	<para>an "options file" as specified by the command-line option
485			<filename>-optfile='FILENAME'</filename></para>
486	edhill	1.3	</listitem>
487	jmc	1.14	<listitem>
488			<para> a <filename>packages.conf</filename> file (in the current
489			directory or in one of the "MODS" directories, see below)
490			which contains the specific list of packages to compile
491			</para>
492			</listitem>
493	edhill	1.3	</orderedlist>
494
495	edhill	1.4	<para>then checking certain dependency rules (the package dependencies),
496			and finally writing a <filename>Makefile</filename> based upon the
497			source code that it finds. For convenience within various Unix
498			shells, <filename>genmake2</> supports both "long"- and "short"-style
499			options. A complete list of the available options can be obtained
500			from:</para>
501	edhill	1.3
502			<screen>
503	edhill	1.5	$ genmake2 -help
504	edhill	1.3	</screen>
505
506	edhill	1.4	<para>The most important options for <filename>genmake2</> are:</para>
507	edhill	1.3
508			<variablelist>
509
510			<varlistentry>
511			<term><filename>--optfile=/PATH/FILENAME</></term>
512	edhill	1.4
513	edhill	1.3	<listitem>
514	edhill	1.4	<para>This specifies the "options file" that should be used for a
515			particular build. The options file is a convenient and
516			machine-indepenent way of specifying parameters such as the
517			FORTRAN compiler (<filename>FC=</>), FORTRAN compiler
518			optimization flags (<filename>FFLAGS=</>), and the locations of
519			various platform- and/or machine-specific tools
520			(eg. <filename>MAKEDEPEND=</>). As with <filename>genmake2</>,
521			all options files should be written to be compatible with
522			Bourne--shell ("sh" or "BASH v1") syntax. Examples of various
523			options files can be found in
524			<filename>$ROOTDIR/tools/build_options</>.</para>
525
526			<para>If no "optfile" is specified (either through the command lin
527			or the environment variable), genmake2 will try to make a
528			reasonable guess from the list provided in
529			<filename>$ROOTDIR/tools/build_options</>. The method used for
530			making this guess is to first determine the combination of
531			operating system and hardware (eg. "linux_ia32") and then find a
532			working Fortran compiler within the user's path. When these
533			three items have been identified, genmake2 will try to find an
534			optfile that has a matching name. </para>
535
536			<para>Everyone is encouraged to submit their options files to the
537			MITgcm project for inclusion (please send to
538			<email>MITgcm-support@mitgcm.org</email>). We are particularly
539			grateful for options files tested on new or unique
540			platforms!</para>
541	edhill	1.3	</listitem>
542	edhill	1.4
543	edhill	1.3	</varlistentry>
544
545			<varlistentry>
546	edhill	1.5	<term><filename>-adof=/path/to/file</></term>
547			<term><filename>-adoptfile=/path/to/file</></term>
548			<listitem>
549			<para>This option specifies the "adjoint" or automatic
550			differentiation options file to be used. The file is analogous
551			to the "optfile" defined above but it specifies information for
552			the AD build process. The default file is located in <filename>
553			$ROOTDIR/tools/adjoint_options/adjoint_default </> and it
554			defines the "TAF" and "TAMC" compilers. An alternate version is
555			also available at <filename>
556			$ROOTDIR/tools/adjoint_options/adjoint_staf </> that selects the
557			newer "STAF" compiler. As with any compilers, it is helpful to
558			have their directories listed in your $PATH environment
559			variable.</para>
560			</listitem>
561			</varlistentry>
562
563			<varlistentry>
564	edhill	1.4	<term><filename>-mods=DIR</></term>
565			<term><filename>-mods='DIR1 [DIR2 ...]'</></term>
566	edhill	1.3	<listitem>
567	edhill	1.4	<para>This option specifies a list of directories containing
568			"modifications". These directories contain files with names
569			that may (or may not) exist in the main MITgcm source tree but
570			will be overridden by any identically-named sources within the
571			"MODS" directories. The order of precedence for this
572			"name-hiding" is as follows:</para>
573			<itemizedlist>
574			<listitem><para>"MODS" directories (in the order given)
575			</para></listitem>
576			<listitem><para>Packages either explicitly specified or
577			provided by default (in the order given)</para></listitem>
578			<listitem><para>Packages included due to package dependencies
579	jmc	1.16	(in the order that that package dependencies are
580	edhill	1.4	parsed)</para></listitem>
581			<listitem><para>The "standard dirs" (which may have been
582			specified by the "-standarddirs" option)</para></listitem>
583			</itemizedlist>
584	jmc	1.14	</listitem>
585			</varlistentry>
586
587			<varlistentry>
588			<term><filename>-pgroups=/PATH/FILENAME</></term>
589			<listitem>
590			<para>This option specifies the file where package groups are
591			defined. If not set, the package-groups definition will
592			be read from
593			<filename>$ROOTDIR/pkg/pkg_groups</>.</para>
594			<para>
595			It also contains the default list of packages (defined
596			as the group <filename>"default_pkg_list"</>) which is used
597			when no specific package list (file: <filename>packages.conf</>)
598			is found in current directory or in any "MODS" directory.
599			</para>
600			</listitem>
601			</varlistentry>
602	edhill	1.4
603	jmc	1.14	<varlistentry>
604			<term><filename>-pdepend=/PATH/FILENAME</></term>
605
606			<listitem>
607			<para>This specifies the dependency file used for packages. If
608			not specified, the default dependency file is
609			<filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for this file is
610			parsed on a line-by-line basis where each line containes either a
611			comment ("#") or a simple "PKGNAME1 (+\|-)PKGNAME2" pairwise rule
612			where the "+" or "-" symbol specifies a "must be used with" or a
613			"must not be used with" relationship, respectively. If no rule is
614			specified, then it is assumed that the two packages are compatible
615			and will function either with or without each other.</para>
616	edhill	1.4	</listitem>
617			</varlistentry>
618
619			<varlistentry>
620			<term><filename>-make=/path/to/gmake</></term>
621			<listitem>
622			<para>Due to the poor handling of soft-links and other bugs common
623			with the <filename>make</> versions provided by commercial Unix
624			vendors, GNU <filename>make</filename> (sometimes called
625			<filename>gmake</filename>) should be preferred. This option
626			provides a means for specifying the make program to be
627			used.</para>
628	edhill	1.3	</listitem>
629			</varlistentry>
630
631			</variablelist>
632
633	edhill	1.4	<para>A successful run of <filename>genmake2</> will produce a
634			<filename>Makefile</>, a <filename>PACKAGES_CONFIG.h</> file, and
635			various convenience files used for the automatic differentiation
636			process.</para>
637
638			<para>In general, it is best to use <filename>genmake2</> on a "clean"
639			directory that is free of all source (.[F,f],.[F,f]90) and header
640			(.h,.inc) files. Generally, this can be accomplished in an
641	jmc	1.14	"un-clean" directory by running "make Clean" followed by "make
642	edhill	1.4	makefile".</para>
643	edhill	1.3
644			</sect3>
645
646			<sect3 id="makefile_use">
647	edhill	1.5	<title>Using the <filename>Makefile</></title>
648	edhill	1.3
649	edhill	1.5	<para>Once a <filename>Makefile</> has been created using
650			<filename>genmake2</>, one can build a "standard" (forward
651			simulator) executable using:</para>
652	edhill	1.3
653			<screen>
654	jmc	1.14	$ make Clean
655	edhill	1.5	$ make depend
656			$ make
657	edhill	1.3	</screen>
658
659	jmc	1.14	<para>The "make Clean" step will remove any stale source files, include
660	edhill	1.4	files, and links. It is strongly recommended for "un-clean"
661			directories which may contain the (perhaps partial) results of
662	jmc	1.14	previous builds. Such "debris" can interfere with the next stage of
663	jmc	1.16	the build.
664			A more agressive cleaning option, "make CLEAN", can be used to also
665	jmc	1.14	remove the executable and output files from a previous run.</para>
666	edhill	1.4
667			<para>The "make depend" step will create a large number of symbolic
668			links from the local directory to the source file locations. It also
669			parses these files and creates an extensive list of dependencies
670			within the <filename>Makefile</> itself. The links that exist at this
671			stage are mostly "large F" files (.F and .F90) that need to be
672			processed by a C preprocessor ("CPP"). Since "make depend" edits the
673			<filename>Makefile</>, it is important not to skip this step!</para>
674
675			<para>The final "make" invokes the C preprocessor to produce the "little
676			f" files (.f and .f90) and then compiles them to object code using
677			the specified FORTRAN compiler and options. An intermediate script is
678			often used during this stage to further process (usually, make simple
679			substitutions) custom definitions such as variable types within the
680			source files. This additional stage is necessary in order to overcome
681			some of the inconsistencies in the sizes of objects (bytes) between
682	edhill	1.5	different compilers. The result of the build process is an executable
683			with the name <filename>mitgcmuv</>.</para>
684
685			<para>In addition to the forward simulator described above, the
686			<filename>Makefile</> also has a number of targets that can be used to
687			produce various adjoint and tangent-linear builds for optimization and
688			other parameter-sensitivity problems. The additional targets within
689			the <filename>Makefile</> are:</para>
690
691			<variablelist>
692
693			<varlistentry>
694			<term><filename>make adall</></term>
695			<listitem>
696			<para>This target produces an <filename>mitgcmuv_ad</> executable
697			using the <filename>taf</> or <filename>staf</> adjoint
698	jmc	1.16	compiler. See the <filename>genmake2</> "-adof" option for
699	edhill	1.5	compiler selection.</para>
700			</listitem>
701			</varlistentry>
702
703			<varlistentry>
704			<term><filename>make ftlall</></term>
705			<listitem>
706			<para>Similar to <filename>make adall</> above, this
707			produces...</para>
708			</listitem>
709			</varlistentry>
710
711			</variablelist>
712	edhill	1.3
713	edhill	1.5	<para>Please report any compilation failures or other build problems to
714			the <email>MITgcm-support@mitgcm.org</email> list.</para>
715	edhill	1.3
716			</sect3>
717
718			</sect2>
719
720			<sect2 id="verification">
721			<title>The Verification Suite</title>
722
723	edhill	1.4	<para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</>
724	jmc	1.15	directory) includes many (> 90) examples intended for regression
725	jmc	1.16	testing. Each one of these test-experiment directories contains "known-good"
726	edhill	1.4	output files along with all the input (including both code and data
727	jmc	1.16	files) required for their re-calculation.
728			Also included in <filename>$ROOTDIR/verification/</> is the shell
729			script <filename>testreport</> to perform regression tests.</para>
730
731			<sect3 id="test-experiment">
732			<title>Test-experiment Directory Content</title>
733
734			<para> Each test-experiment directory (<filename>TESTDIR</>) contains
735			several standard subdirectories and files which <filename>testreport</>
736			recognizes and uses when running a regression test.
737			The directories/files that <filename>testreport</> uses are different
738			for a forward test and an adjoint test (<filename>testreport -adm</>)
739			and some test-experiments are set-up for only one type of regression
740			test whereas others allow both types of tests (forward and adjoint).</para>
741			<para>Also some test-experiment allows, using the same MITgcm executable,
742			to perform multiple tests using different parameters and input files,
743			with a primary input set-up
744			(<filename>input/ </> or <filename>input_ad/ </>)
745			and corresponding results (<filename>results/output.txt</> or
746			<filename>results/output_adm.txt</>) and with one or several secondary
747			inputs (<filename>input.OTHER/ </> or <filename>input_ad.OTHER/ </>)
748			and corresponding results (<filename>results/output.OTHER.txt </>
749			or <filename>results/output_adm.OTHER.txt</>).</para>
750			<variablelist>
751
752			<varlistentry>
753			<term>directory <filename>TESTDIR/results/</></term>
754			<listitem>
755			<para>contains reference standard output used for test comparison.
756			<filename>results/output.txt</> and
757			<filename>results/output_adm.txt</>
758			correspond respectively to primary forward and adjoint test
759			run on the reference platform (currently
760			<filename>baudelaire.csail.mit.edu</>)
761			on one processor (no MPI, single thread)
762			using the reference compiler (curently the GNU fortran
763			compiler <filename>gfortran</>).
764			The presence of these files determines whether or not
765			<filename>testreport</> is testing or skipping
766			this test-experiment.
767			Reference standard output for secondary tests
768			(<filename>results/output.OTHER.txt</>
769			or <filename>results/output_adm.OTHER.txt</>) are
770			also expected here.</para>
771			<para>The test comparison involves few model variables output, which are,
772			by default and for a forward test, the 2-D solver initial residual
773			(<filename>cg2d_init_res</>) and 3-D state variables (T,S,U,V)
774			monitor output, and, by default and for an adjoint test, the
775			cost-function and gradient-check. However, some test-experiments
776			use some package-specific variable/monitor output according to
777			the file <filename>TESTDIR/input[_ad][.OTHER]/tr_checklist</>
778			specification.</para>
779			</listitem>
780			</varlistentry>
781
782			<varlistentry>
783			<term>directory <filename>TESTDIR/build/</></term>
784			<listitem>
785			<para> initially empty directory where <filename>testreport</>
786			will build the MITgcm executable for forward and adjoint test.
787			It might contains an experiment specific
788			<filename>genmake_local</> file (see <xref linkend="genmake">).
789			</para>
790			<para> Note that the original <filename>code[_ad]/SIZE.h_mpi</>
791			is not directly used as "SIZE.h" to build an MPI-executable ;
792			instead, a local copy <filename>build/SIZE.h.mpi</> is derived
793			from <filename>code[_ad]/SIZE.h_mpi</>
794			by adjusting the number of processors (nPx,nPy) according to
795			<filename>NUMBER_OF_PROCS</> (see <xref linkend="testreport">,
796			<filename>testreport -MPI</>) ; then it is linked to "SIZE.h"
797			(<filename> ln -s SIZE.h.mpi SIZE.h </>) before building
798			the MPI-executable.</para>
799			</listitem>
800			</varlistentry>
801
802			<varlistentry>
803			<term>directory <filename>TESTDIR/code/</></term>
804			<listitem>
805			<para>contains the test-experiment specific source code
806			used to build the MITgcm executable (<filename>mitgcmuv</>)
807			for forward-test (using <filename>genmake2 -mods=../code</>).
808			</para>
809			<para> It can also contain specific source files with the suffix
810			"<filename>_mpi</>" to be used
811			<!--
812			in the <filename>TESTDIR/build/</> directory
813			-->
814			in place of the corresponding file (without suffix)
815			for an MPI test (see <xref linkend="testreport">).
816			The presence or absence of <filename>SIZE.h_mpi</>
817			determines whether or not an MPI test on this
818			test-experiment is performed or skipped.</para>
819			</listitem>
820			</varlistentry>
821
822			<varlistentry>
823			<term>directory <filename>TESTDIR/code_ad/</></term>
824			<listitem>
825			<para>contains the test-experiment specific source code
826			used to build the MITgcm executable (<filename>mitgcmuv_ad</>)
827			for adjoint-test (using <filename>genmake2 -mods=../code_ad</>).
828			It can also contain specific source files with the suffix
829			"<filename>_mpi</>" (see above).</para>
830			</listitem>
831			</varlistentry>
832
833			<varlistentry>
834			<term>directory <filename>TESTDIR/input/</></term>
835			<listitem>
836			<para>contains the input and parameter files
837			used to run the primary forward test of this test-experiment.
838			</para>
839			<para>It can also contain specific parameter files with the suffix
840			"<filename>.mpi</>" to be used
841			<!--
842			in the <filename>TESTDIR/run/</> directory
843			-->
844			in place of the corresponding file (without suffix) for MPI
845			test, or with suffix "<filename>.mth</>" to be used for
846			multi-threaded test (see <xref linkend="testreport">).
847			The presence or absence of <filename>eedata.mth</>
848			determines whether or not a multi-threaded test on this
849			test-experiment is performed or skipped.</para>
850			<para>To save disk space and reduce downloading time, multiple
851			copies of the same input file is avoided by using a shell
852			script <filename>prepare_run</>.
853			When such a script is found in <filename>TESTDIR/input/ </>,
854			<filename>testreport</> run this script in directory
855			<filename>TESTDIR/run/ </> after linking all the input file
856			from <filename>TESTDIR/input/ </>.
857			</para>
858			</listitem>
859			</varlistentry>
860
861			<varlistentry>
862			<term>directory <filename>TESTDIR/input_ad/</></term>
863			<listitem>
864			<para>contains the input and parameter files
865			used to run the primary adjoint test of this test-experiment.
866			It can also contain specific parameter files with the suffix
867			"<filename>.mpi</>" and shell script <filename>prepare_run</>
868			as described above.</para>
869			</listitem>
870			</varlistentry>
871
872			<varlistentry>
873			<term>directory <filename>TESTDIR/input.OTHER/</></term>
874			<listitem>
875			<para>contains the input and parameter files
876			used to run the secondary OTHER forward test of this test-experiment.
877			It can also contain specific parameter files with suffix
878			"<filename>.mpi</>" or "<filename>.mth</>" and shell script
879			<filename>prepare_run</> (see above).</para>
880			<para> The presence or absence the file <filename>eedata.mth</>
881			determines whether or not a secondary multi-threaded test on this
882			test-experiment is performed or skipped.</para>
883			</listitem>
884			</varlistentry>
885
886			<varlistentry>
887			<term>directory <filename>TESTDIR/input_ad.OTHER/</></term>
888			<listitem>
889			<para>contains the input and parameter files
890			used to run the secondary OTHER adjoint test of this test-experiment.
891			It can also contain specific parameter files with the suffix
892			"<filename>.mpi</>" and shell script <filename>prepare_run</>
893			(see above).</para>
894			</listitem>
895			</varlistentry>
896			<!--
897			-->
898			<varlistentry>
899			<term>directory <filename>TESTDIR/run/</></term>
900			<listitem>
901			<para> initially empty directory where <filename>testreport</>
902			will run the MITgcm executable for primary forward and adjoint
903			test.</para>
904			<para>Symbolic links (using command "<filename>ln -s</>") are made
905			for input and parameter files (from <filename>../input/ </>
906			or from <filename>../input_ad/ </>) and for MITgcm executable
907			(from <filename>../build/ </>) before the run proceeds.
908			The sequence of links (function <filename>linkdata</> within shell
909			script <filename>testreport</>) for a forward test is:</para>
910			<screen>
911			* link+rename or remove links
912			to special files with suffix ".mpi" or ".mth" from ../input/
913			* link files from ../input/
914			* execute ../input/prepare_run (if it exists)
915			</screen>
916			<para>The sequence for an ajoint test is similar, with
917			<filename>../input_ad/ </> replacing <filename>../input/ </>.
918			</para>
919			</listitem>
920			</varlistentry>
921
922			<varlistentry>
923			<term>directory <filename>TESTDIR/tr_run.OTHER/</></term>
924			<listitem>
925			<para> directory created by <filename>testreport</>
926			to run the MITgcm executable for secondary "OTHER" forward
927			or adjoint test.</para>
928			<para> The sequence of links for a forward secondary test is:</para>
929			<screen>
930			* link+rename or remove links
931			to special files with suffix ".mpi" or ".mth" from ../input.OTHER/
932			* link files from ../input.OTHER/
933			* execute ../input.OTHER/prepare_run (if it exists)
934			* link files from ../input/
935			* execute ../input/prepare_run (if it exists)
936			</screen>
937			<para>The sequence for an ajoint test is similar, with
938			<filename>../input_ad.OTHER/ </> and <filename>../input_ad/ </>
939			replacing <filename>../input.OTHER/ </> and <filename>../input/ </>.
940			</para>
941			</listitem>
942			</varlistentry>
943
944			</variablelist>
945			</sect3>
946	edhill	1.3
947			<sect3 id="testreport">
948			<title>The <filename>testreport</> Utility</title>
949
950	jmc	1.16	<para> The shell script <filename>testreport</> (in
951			<filename>$ROOTDIR/verification/</>), which was written to work with
952			<filename>genmake2</>, can be used to build different versions
953			of the MITgcm code, run the various examples, compare the output,
954			and (if specified) email the results of each one of these tests to a
955			central repository.</para>
956	edhill	1.4
957			<para>On some systems, the testreport script can be run with a command
958			line as simple as:</para>
959
960			<screen>
961	edhill	1.5	$ cd verification
962	jmc	1.15	$ ./testreport
963	edhill	1.4	</screen>
964
965			<para>However, some systems (those lacking or wiht a broken "/bin/sh")
966			may require an explicit shell invocation such as:</para>
967
968			<screen>
969	jmc	1.15	$ sh ./testreport -t 'exp2 exp4'
970			$ /some/path/to/bash ./testreport -t 'ideal_2D_oce lab_sea natl_box'
971	edhill	1.4	</screen>
972	edhill	1.3
973			<para>The <filename>testreport</> script accepts a number of
974	edhill	1.4	command-line options which can be listed using the <filename>-help</>
975			option. The most important ones are:</para>
976	edhill	1.3
977			<variablelist>
978
979			<varlistentry>
980	jmc	1.15	<term><filename>-ieee</> (default) / <filename>-noieee</></term>
981	edhill	1.4	<listitem>
982			<para>If allowed by the compiler (as defined in the "optfile"),
983	jmc	1.16	use IEEE arithmetic (<filename>genmake2 -ieee</>).
984	jmc	1.15	This option, along with the gfortran / gcc compiler,
985			is how the standard results are produced.</para>
986			</listitem>
987			</varlistentry>
988
989			<varlistentry>
990			<term><filename>-optfile=/PATH/FILENAME</></term>
991			<term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>
992			<listitem>
993			<para>This specifies a list of "options files" that will be passed
994			to <filename>genmake2</>. If multiple options files are used
995			(say, to test different compilers or different sets of options
996			for the same compiler), then each options file will be used with
997			each of the test directories.</para>
998	edhill	1.4	</listitem>
999			</varlistentry>
1000
1001			<varlistentry>
1002	edhill	1.3	<term><filename>-tdir TESTDIR</></term>
1003			<term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term>
1004			<listitem>
1005	edhill	1.4	<para>This option specifies the test directory or list of test
1006			directories that should be used. Each of these entries should
1007			exactly (note: they are case sensitive!) match the names of
1008	jmc	1.15	directories in <filename>$ROOTDIR/verification/</>. If this
1009	edhill	1.4	option is omitted, then all directories that are properly
1010			formatted (that is, containing an <filename>input</>
1011			sub-directory and a <filename>results/output.txt</> file) will
1012			be used.</para>
1013	edhill	1.3	</listitem>
1014			</varlistentry>
1015
1016			<varlistentry>
1017			<term><filename>-addr EMAIL</></term>
1018			<term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>
1019			<listitem>
1020			<para>Send the results (namely, <filename>output.txt</>,
1021	edhill	1.4	<filename>genmake_local</>, <filename>genmake_state</>, and
1022			<filename>Makefile</>) to the specified email addresses. The
1023			results are gzipped, placed in a tar file, MIME encoded, and
1024			sent to the specified address. If no email addresses are
1025			specified, no mail is sent.</para>
1026			</listitem>
1027			</varlistentry>
1028
1029			<varlistentry>
1030	jmc	1.15	<term><filename>-MPI NUMBER_OF_PROCS</></term>
1031	edhill	1.4	<term><filename>-mpi</></term>
1032			<listitem>
1033	jmc	1.16	<para>If the necessary file (<filename>TESTDIR/code/SIZE.h_mpi</>)
1034	jmc	1.15	exists, then use it (and all <filename>TESTDIR/code/*_mpi</> files)
1035	jmc	1.16	for an MPI--enabled run. The new option (<filename>-MPI</> followed
1036			by the maximum number of processors) enable to build and run each
1037	jmc	1.15	test-experiment using variable number of MPI processors (multiple
1038	jmc	1.16	of <filename>nPx*nPy</> from <filename>TESTDIR/code/SIZE.h_mpi</>
1039			and not larger than <filename>NUMBER_OF_PROCS</>).
1040			The short option ("-mpi") can only be used to build and run on 2 MPI
1041			processors (equivalent to "<filename>-MPI 2</>").</para>
1042	jmc	1.15	<para>Note that the use of MPI typically requires a
1043	edhill	1.4	special command option (see "-command" below) to invoke the MPI
1044	jmc	1.15	executable. Examples of PBS scripts using testreport with MPI can be
1045	edhill	1.4	found in the <ulink
1046	jmc	1.15	url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
1047			tools/example_scripts directory</ulink>.</para>
1048	edhill	1.4	</listitem>
1049			</varlistentry>
1050
1051			<varlistentry>
1052			<term><filename>-command='some command to run'</></term>
1053			<listitem>
1054	jmc	1.15	<para>For some tests, particularly MPI runs, a specific command
1055			might be needed to run the executable. This option allows a more general
1056	edhill	1.4	command (or shell script) to be invoked. Examples of PBS scripts
1057	jmc	1.15	using testreport with MPI can be found in the <ulink
1058			url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
1059			tools/example_scripts directory</ulink>.</para>
1060	jmc	1.16	<para>
1061			For the case where the number of MPI processors varies according
1062			to each test-experiment, some key-words within the command-to-run
1063			argument will be replaced by their effective value:</para>
1064			<para>
1065			<filename>TR_NPROC </> will be replaced by the actual number
1066			of MPI processors needed to run the current test-experiment.</para>
1067			<para>
1068			<filename>TR_MFILE </> will be replaced by the name of local-file
1069			that testreport creates from the full list of machines which
1070			"<filename>testreport -mf MACHINE_FILE</>" provides, but truncated
1071			to the exact number of machines.</para>
1072			</listitem>
1073			</varlistentry>
1074
1075			<varlistentry>
1076			<term><filename>-mf MACHINE_FILE</></term>
1077			<listitem>
1078			<para>
1079			To use with <filename>-MPI NUMBER_OF_PROCS</> option, to specify
1080			the file containing the full list of <filename>NUMBER_OF_PROCS</>
1081			machines to use for the MPI runs.</para>
1082			</listitem>
1083			</varlistentry>
1084
1085			<varlistentry>
1086			<term><filename>-mth</></term>
1087			<listitem>
1088			<para>compile (with <filename>genmake2 -omp</>) and run with
1089			multiple threads (using eedata.mth).</para>
1090	edhill	1.3	</listitem>
1091			</varlistentry>
1092
1093			</variablelist>
1094
1095	jmc	1.16	<para>The <filename>testreport</> script will create an output directory
1096			<filename>tr_NAME_DATE_N/ </>, with <filename>hostname</> as default
1097			<filename>NAME</>, <filename>DATE</> the current date followed by
1098			a suffix number "N" to distinguish from previous
1099			<filename>testreport</> output directories.
1100			<filename>testreport</> writes progress to the screen (stdout)
1101			and reports into the ouput directory as it runs.
1102			In particular, one can find, in the ouput directory,
1103			the <filename>summary.txt</> file that contains a brief comparison
1104			of the current output with the "known-good" output.
1105			At the end of the testing process, the <filename>tr_out.txt</> file
1106			is generated in <filename>$ROOTDIR/verification/ </>
1107			as a compact version of <filename>summry.txt</> file.</para>
1108
1109			</sect3>
1110	edhill	1.3
1111	jmc	1.16	<sect3 id="tst_2plus2">
1112			<title>The <filename>do_tst_2+2</> Utility</title>
1113			<para> The shell script <filename>do_tst_2+2</> (in
1114			<filename>$ROOTDIR/tools/ </>)
1115			can be used to check the accuracy of the restart procedure.
1116			</para>
1117	edhill	1.3	</sect3>
1118
1119			</sect2>
1120	edhill	1.2
1121			<sect2 id="packages">
1122	edhill	1.3	<title>Creating MITgcm Packages</title>
1123	edhill	1.2
1124	edhill	1.4	<para>Optional parts of code have been separated from the MITgcmUV core
1125			driver code and organised into packages. The packaging structure
1126			provides a mechanism for maintaining suites of code, specific to
1127			particular classes of problems, in a way that is cleanly separated from
1128			the generic fluid dynamical engine.</para>
1129
1130			<para>The MITgcmUV packaging structure is described below using generic
1131	jmc	1.11	package names ${pkg}. A concrete examples of a package is the code for
1132	edhill	1.4	implementing GM/Redi mixing. This code uses the package name</para>
1133	edhill	1.2
1134			</sect2>
1135
1136			</sect1>
1137
1138			<sect1>
1139			<title>Chris's Notes...</title>
1140
1141			<programlisting>
1142			MITgcmUV Packages
1143			=================
1144
1145			Optional parts of code are separated from
1146			the MITgcmUV core driver code and organised into
1147			packages. The packaging structure provides a mechanism for
1148			maintaining suites of code, specific to particular
1149			classes of problem, in a way that is cleanly
1150			separated from the generic fluid dynamical engine.
1151
1152			The MITgcmUV packaging structure is describe
1153			below using generic package names ${pkg}.
1154			A concrete examples of a package is the code
1155			for implementing GM/Redi mixing. This code uses
1156	jmc	1.16	the package name
1157	edhill	1.2	* ${PKG} = GMREDI
1158			* ${pkg} = gmredi
1159			* ${Pkg} = gmRedi
1160
1161			Package states
1162			==============
1163
1164			Packages can be any one of four states, included,
1165			excluded, enabled, disabled as follows:
1166
1167			included(excluded) compile time state which
1168			includes(excludes) package
1169			code and routine calls from
1170			compilation/linking etc...
1171
1172			enabled(disabled) run-time state which
1173			enables(disables) package code
1174			execution.
1175
1176			Every call to a ${pkg}_... routine from outside the package
1177			should be placed within both a
1178			#ifdef ALLOW_${PKG} ... block and a
1179			if ( use${Pkg} ) ... then block.
1180			Package states are generally not expected to change during
1181			a model run.
1182
1183			Package structure
1184			=================
1185
1186			o Each package gets its runtime configuration
1187			parameters from a file named "data.${pkg}"
1188			Package runtime config. options are imported
1189			into a common block held in a header file
1190			called "${PKG}.h".
1191	jmc	1.16	Note: In some packages, the header file "${PKG}.h" is splitted
1192			into "${PKG}_PARAMS.h" that contains the package parameters and
1193	jmc	1.8	${PKG}_VARS.h" for the field arrays.
1194	edhill	1.2
1195			o The core driver part of the model can check
1196			for runtime enabling or disabling of individual packages
1197			through logical flags use${Pkg}.
1198			The information is loaded from a
1199			global package setup file called "data.pkg".
1200			The use${Pkg} flags are not used within
1201			individual packages.
1202
1203			o Included in "${PKG}.h" is a logical flag
1204			called ${Pkg}IsOn. The "${PKG}.h" header file can be imported
1205			by other packages to check dependencies and requirements
1206			from other packages ( see "Package Boot Sequence" section).
1207			NOTE: This procedure is not presently implemented,
1208			----- neither for kpp nor for gmRedi.
1209
1210			CPP Flags
1211			=========
1212
1213			1. Within the core driver code flags of the form
1214			ALLOW_${PKG} are used to include or exclude
1215			whole packages. The ALLOW_${PKG} flags are included
1216	jmc	1.16	from a PACKAGES_CONFIG.h file that is automatically
1217	jmc	1.8	generated by genmake2 (see genmake2 section).
1218	edhill	1.2	held in-line in the CPP_OPTIONS.h header file.
1219			e.g.
1220
1221			Core model code .....
1222
1223	jmc	1.8	#include "PACKAGES_CONFIG.h"
1224	edhill	1.2	#include "CPP_OPTIONS.h"
1225			:
1226			:
1227			:
1228
1229			#ifdef ALLOW_${PKG}
1230			if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...)
1231			#endif
1232
1233			2. Within an individual package a header file,
1234			"${PKG}_OPTIONS.h", is used to set CPP flags
1235	jmc	1.16	specific to that package. It also includes
1236	jmc	1.8	"PACKAGES_CONFIG.h" and "CPP_OPTIONS.h".
1237	edhill	1.2
1238			Package Boot Sequence
1239			=====================
1240
1241			Calls to package routines within the core code timestepping
1242			loop can vary. However, all packages follow a required
1243			"boot" sequence outlined here:
1244
1245			1. S/R PACKAGES_BOOT()
1246			:
1247			CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
1248	jmc	1.16
1249	edhill	1.2	2. S/R PACKAGES_READPARMS()
1250			:
1251			#ifdef ALLOW_${PKG}
1252			if ( use${Pkg} )
1253			& CALL ${PKG}_READPARMS( retCode )
1254			#endif
1255
1256	jmc	1.6	3. S/R PACKAGES_INIT_FIXED()
1257			:
1258			#ifdef ALLOW_${PKG}
1259			if ( use${Pkg} )
1260			& CALL ${PKG}_INIT_FIXED( retCode )
1261			#endif
1262
1263			4. S/R PACKAGES_CHECK()
1264	edhill	1.2	:
1265			#ifdef ALLOW_${PKG}
1266			if ( use${Pkg} )
1267			& CALL ${PKG}_CHECK( retCode )
1268			#else
1269			if ( use${Pkg} )
1270			& CALL PACKAGES_CHECK_ERROR('${PKG}')
1271			#endif
1272
1273	jmc	1.6	5. S/R PACKAGES_INIT_VARIABLES()
1274	edhill	1.2	:
1275			#ifdef ALLOW_${PKG}
1276			if ( use${Pkg} )
1277	jmc	1.6	& CALL ${PKG}_INIT_VARIA( )
1278			#endif
1279
1280			Package Output
1281			==============
1282			6. S/R DO_THE_MODEL_IO
1283
1284			#ifdef ALLOW_${PKG}
1285			if ( use${Pkg} )
1286	jmc	1.9	& CALL ${PKG}_OUTPUT( )
1287	edhill	1.2	#endif
1288
1289	jmc	1.6	7. S/R PACKAGES_WRITE_PICKUP()
1290
1291			#ifdef ALLOW_${PKG}
1292			if ( use${Pkg} )
1293			& CALL ${PKG}_WRITE_PICKUP( )
1294			#endif
1295	edhill	1.2
1296			Description
1297			===========
1298
1299	jmc	1.16	- ${PKG}_READPARMS()
1300	edhill	1.2	is responsible for reading
1301			in the package parameters file data.${pkg}, and storing
1302	jmc	1.8	the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h").
1303	jmc	1.6	-> called from INITIALISE_FIXED in PACKAGES_READPARMS
1304
1305	jmc	1.16	- ${PKG}_INIT_FIXED()
1306			is responsible for completing the internal setup of a package.
1307	jmc	1.6	-> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED
1308			note: 1) some pkg use instead:
1309			CALL ${PKG}_INITIALISE ( or the old form CALL ${PKG}_INIT )
1310			2) for simple pkg setup, this part is done inside ${PKG}_READPARMS
1311	edhill	1.2
1312	jmc	1.16	- ${PKG}_CHECK()
1313	edhill	1.2	is responsible for validating
1314			basic package setup and inter-package dependencies.
1315			${PKG}_CHECK can import other package parameters it may
1316			need to check. This is done through header files "${PKG}.h".
1317			It is assumed that parameters owned by other packages
1318			will not be reset during ${PKG}_CHECK().
1319	jmc	1.6	-> called from INITIALISE_FIXED in PACKAGES_CHECK
1320	edhill	1.2
1321	jmc	1.16	- ${PKG}_INIT_VARIA()
1322	jmc	1.6	is responsible for fill-in all package variables with an initial value.
1323			Contains eventually a call to ${PKG}_READ_PICKUP that will read
1324			from a pickup file the package variables required to restart the model.
1325	jmc	1.16	This routine is called after the core model state has been completely
1326	jmc	1.6	initialised but before the core model timestepping starts.
1327			-> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES
1328	jmc	1.16	note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg
1329			use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old
1330	jmc	1.6	form ${PKG}_INIT
1331
1332	jmc	1.9	- ${PKG}_OUTPUT( )
1333	jmc	1.8	is responsible for writing time-average fields to output files
1334			(but the cumulating step is done within the package main S/R).
1335	jmc	1.16	Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR)
1336			and write snap-shot fields that are hold in common blocks. Other
1337	jmc	1.6	temporary fields are directly dump to file where they are available.
1338	jmc	1.16	NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages
1339	jmc	1.9	but is beeing replaced by ${PKG}_OUTPUT
1340	jmc	1.8	to avoid confusion with pkg/diagnostics functionality.
1341			2) the output part is not yet in a standard form and might still
1342			evolve a lot.
1343	jmc	1.6	-> called within DO_THE_MODEL_IO
1344
1345	jmc	1.16	- ${PKG}_WRITE_PICKUP()
1346			is responsible for writing a package pickup file when necessary for
1347	jmc	1.6	a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT )
1348			-> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP
1349	edhill	1.2
1350			Summary
1351			=======
1352
1353			- CPP options:
1354			-----------------------
1355			* ALLOW_${PKG} include/exclude package for compilation
1356
1357			- FORTRAN logical:
1358			-----------------------
1359			* use${Pkg} enable package for execution at runtime
1360			-> declared in PARAMS.h
1361			* ${Pkg}IsOn for package cross-dependency check
1362			-> declared in ${PKG}.h
1363			N.B.: Not presently used!
1364
1365			- header files
1366			-----------------------
1367			* ${PKG}_OPTIONS.h has further package-specific CPP options
1368			* ${PKG}.h package-specific common block variables, fields
1369	jmc	1.8	or ${PKG}_PARAMS.h package-specific common block parameters
1370			and ${PKG}_VARS.h package-specific common block fields
1371	edhill	1.2
1372			- FORTRAN source files
1373			-----------------------
1374	jmc	1.6	* ${pkg}_readparms.F reads parameters from file data.${pkg}
1375			* ${pkg}_init_fixed.F complete the package setup
1376			* ${pkg}_check.F checks package dependencies and consistencies
1377			* ${pkg}_init_varia.F initialises package-related fields
1378			* ${pkg}_... .F package source code
1379	jmc	1.9	* ${pkg}_output.F write output to file.
1380	jmc	1.6	* ${pkg}_write_pickup.F write a package pickup file to restart the model
1381	edhill	1.2
1382	jmc	1.16	New: Subroutine in one package (pkgA) that only contains code which
1383	jmc	1.8	is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F)
1384			will be named: pkgA_pkgB_something.F
1385
1386	edhill	1.2	- parameter file
1387			-----------------------
1388			* data.${pkg} parameter file
1389			</programlisting>
1390
1391			</sect1>
1392	edhill	1.1
1393	jmc	1.11	<sect1 id="documentation">
1394			<title>Editing the Documentation</title>
1395
1396			<sect2 id="documentation_getting">
1397			<title>Getting the Docs and Code</title>
1398
1399			<para>The first step towards editing the documentation is to checkout a
1400			copy of code, docs, and build scripts from the CVS server using:</para>
1401
1402			<screen>
1403			$ export CVS_RSH=ssh
1404			$ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack'
1405			$ mkdir scratch
1406			$ cvs co -P MITgcm manual mitgcm.org
1407			</screen>
1408
1409			<para>These commands extract the necessary information from the CVS server
1410			and create a temporary (called <filename>scratch</filename>) directory for
1411			the storage of the HTML and other files that will be created. Please note
1412			that you must either create <filename>scratch</filename> as shown or edit
1413			the various <filename>Makefile</filename>s and scripts used to create the
1414			documentation.</para>
1415			</sect2>
1416
1417			<sect2>
1418			<title>Editing the Documentation</title>
1419
1420			<para>The documentation is contained in the <filename>manual</filename>
1421			directory in a raw LaTeX format. The main document is
1422			<filename>manual.tex</filename> and it uses <command>\input{}</command>s
1423			to include the chapters and subsections.</para>
1424
1425			<para>Since the same LaTeX source is used to produce PostScript, PDF, and
1426			HTML output, care should be taken to follow certain conventions. Two of
1427			the most important are the usage of the <command>\filelink{}{}</command>
1428			and <command>\varlink{}{}</command> commands. Both of these commands have
1429			been defined to simplify the connection between the automatically
1430			generated ("code browser") HTML and the HTML version of the manual
1431			produced by LaTeX2HTML. They each take two arguments (corresponding to
1432			the contents of the two sets of curly braces) which are the text that the
1433			author wishes to be "wrapped" within the link, and a specially formatted
1434			link thats relative to the <filename>MITgcm</filename> directory within
1435			the CVS tree.</para>
1436
1437			<para>The result is a command that resembles either</para>
1438	jmc	1.16
1439	jmc	1.11	<orderedlist>
1440			<listitem>
1441			<para>a reference to a variable or subroutine name such as
1442			<command>\varlink{tRef}{tRef}</command>, or </para>
1443			</listitem>
1444
1445			<listitem>
1446			<para>a reference to a file such as
1447			<command>\varlink{tRef}{path-to-the-file_name.F}</command>
1448			where the absolute path to the file is of the form
1449			<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para>
1450			<para>(please note how the leading "/foo/MITgcm"
1451			component of the path is dropped leaving the path
1452			<emphasis>relative</emphasis> to the head of the code
1453			directory and each directory separator "/" is turned
1454			into a "-")</para>
1455			</listitem>
1456			</orderedlist>
1457	jmc	1.16
1458	jmc	1.11
1459			</sect2>
1460
1461			<sect2>
1462			<title>Building the Documentation</title>
1463	jmc	1.16
1464	jmc	1.11	<para>Given the directory structure of <xref
1465			linkend="documentation_getting">, the entire documentation for the web
1466			site can be built using:</para>
1467
1468			<screen>
1469			$ cd mitgcm.org/devel/buildweb
1470			$ make All
1471			</screen>
1472
1473			<para>Which builds the PDF from the LaTeX source, creates the HTML output
1474			from the LaTeX source, parses the FORTRAN code base to produce a
1475			hyperlinked HTML version of the source, and then determines the
1476			cross-linking between the various HTML components.</para>
1477
1478			<para>If there are no errors, the result of the build process (which can
1479			take 30+ minutes on a P4/2.5Ghz) will be contained within a single
1480			directory called <filename>scratch/dev_docs</filename>. This is a freshly
1481			built version of the entire on-line users manual. If you have the correct
1482			permissions, it can be directly copied to the web server area:</para>
1483
1484			<screen>
1485			$ mv scratch/dev_docs /u/u0/httpd/html
1486			</screen>
1487
1488			<para>and the update is complete.</para>
1489
1490			</sect2>
1491
1492			</sect1>
1493
1494	edhill	1.1	</article>
1495

	ViewVC Help
Powered by ViewVC 1.1.22

Annotation of /MITgcm/doc/devel_HOWTO.sgml