[MITgcm] Annotation of /MITgcm/doc/devel

1	edhill	1.1	<!DOCTYPE ARTICLE PUBLIC "-//OASIS//DTD DocBook V4.1//EN">
2	jmc	1.13	<!--
3	jmc	1.15	$Header: /u/gcmpack/MITgcm/doc/devel_HOWTO.sgml,v 1.14 2010/05/28 01:47:32 jmc Exp $
4	jmc	1.13	$Name: $
5			-->
6	edhill	1.1
7			<article id="MITgcm-Development-HOWTO">
8
9			<articleinfo>
10			<title>MITgcm Development HOWTO</title>
11
12			<author>
13			<firstname>Ed</firstname>
14			<surname>Hill III</surname>
15			<affiliation>
16			<address><email>eh3@mit.edu</email></address>
17			</affiliation>
18			</author>
19
20			<revhistory>
21			<revision>
22			<revnumber>0.01</revnumber>
23	edhill	1.2	<date>2003-08-07</date>
24	edhill	1.1	<authorinitials>eh3</authorinitials>
25			<revremark>
26			Initial version.
27			</revremark>
28			</revision>
29	jmc	1.10	<revision>
30			<revnumber>0.02</revnumber>
31			<date>2010-01-21</date>
32			<authorinitials>jmc</authorinitials>
33			<revremark>
34			update links.
35			</revremark>
36			</revision>
37	jmc	1.11	<revision>
38			<revnumber>0.03</revnumber>
39			<date>2010-04-25</date>
40			<authorinitials>jmc</authorinitials>
41			<revremark>
42			Add subsection "Developer settings" (under CVS Repository).
43			</revremark>
44			</revision>
45	edhill	1.1	</revhistory>
46
47			<abstract>
48			<para>This document describes how to develop software for the
49			MITgcm project.</para>
50			</abstract>
51			</articleinfo>
52
53			<sect1 id="intro">
54			<title>Introduction</title> <para>The purpose of this document is
55			to help new developers get "up to speed" with MITgcm
56			development.</para>
57			<sect2>
58			<title>New Versions of This Document</title> <para>You can
59			obtain the latest version of this document <ulink
60	jmc	1.10	url="http://mitgcm.org/public/docs.html">online</ulink> in
61	edhill	1.1	various formats.</para>
62			</sect2>
63			<sect2>
64			<title>Feedback and corrections</title> <para>If you have
65			questions or comments about this document, please feel free to
66			<ulink url="mailto:MITgcm-support@mitgcm.org">contact the
67			authors</ulink>.
68			</para>
69			</sect2>
70			</sect1>
71
72			<sect1 id="background">
73			<title>Background</title>
74
75			<sect2>
76			<title>User Manual</title>
77
78	edhill	1.5	<para>Before jumping into development, please familiarize yourself with
79	jmc	1.10	the <ulink url="http://mitgcm.org/public/docs.html"> MITgcm user manual
80	edhill	1.5	</ulink>. This document contains volumes of useful information and is
81			included here by reference.</para>
82	edhill	1.1
83	edhill	1.5	<!--
84			<para>Also, a "snapshot" or <ulink
85	edhill	1.1	url="http://mitgcm.org/dev_docs/">development version</ulink> of
86			the user manual may be available, though this is only put on the
87			web for testing purposes.</para>
88	edhill	1.5	-->
89	edhill	1.1	</sect2>
90
91			<sect2>
92			<title>Prerequisites</title> <para>To develop for MITgcm project
93			you will need a UNIX or UNIX-like set of build tools including
94			the following:</para>
95			<blockquote>
96			<simplelist type="inline">
97			<member>CVS client</member>
98			<member>make or (preferably) GNU make</member>
99			<member>FORTRAN compiler</member>
100			<member>C compiler</member>
101			<member>[ba]sh and [t]csh shells</member>
102			<member>PERL</member>
103			<member>LaTeX and LaTeX2HTML</member>
104			</simplelist>
105			</blockquote>
106			<para>Essentially all of the work described here has been tested
107			on recent versions of Red Hat Linux (eg. 7.3 through 9). Except
108			where noted, all shell commands will be provided using bash
109			syntax.
110			</para>
111			</sect2>
112
113			</sect1>
114
115			<sect1 id="cvs">
116			<title>CVS Repository</title>
117	jmc	1.11
118	edhill	1.1	<sect2>
119			<title>Layout</title>
120
121			<para>Unlike many open source projects, the MITgcm CVS tree does
122			not follow a simple "src", "docs", "share", and "test" directory
123			layout. Instead, there are multiple higher-level directories
124			that each, to some extent, depend upon the presence of the
125			others. The tree currently resembles:</para>
126
127			<programlisting>gcmpack/
128	jmc	1.11	CVSROOT -hidden-
129	edhill	1.1
130			MITgcm code
131	jmc	1.11	bin empty
132			doc basic developpment documentation
133			eesupp execution environment support code (wrapper)
134			exe empty
135			jobs runtime shell scripts for
136			various platforms (not maintained)
137			lsopt line search
138			model main dynamics (core)
139			optim line search interface
140			pkg alternate and optional numerics, etc.
141			tools scripts to build (and test)
142			utils pre/post processing tools (matlab, ..)
143			verification standard regression tests + examples
144			+ documented examples (tutorials)
145			tutorial_examples (only in release1 branch)
146	edhill	1.1
147	jmc	1.11	MITgcm_contrib contributed code
148	edhill	1.1
149	jmc	1.11	acesgrid.org build acesgrid web site
150			development experimental stuff
151			gfd_lab -?-
152	jmc	1.13	manual source of MITgcm documentation
153	edhill	1.1	mitgcm.org build web site
154	jmc	1.13	old_develop old and early development source
155			misc -?-
156			models -?-
157			packages -?-
158			preprocess -?-
159	jmc	1.11	pdfs some pdfs
160			planetinabottle.org unfinished web site
161			www.ecco-group.org build ecco web site ?
162	edhill	1.1	</programlisting>
163
164	jmc	1.11	<!--
165	edhill	1.1	<para>Efforts are underway to reduce the complexity.</para>
166	jmc	1.11	-->
167	edhill	1.1
168			</sect2>
169
170			<!--
171			<sect2>
172			<title>Releases</title> <para>Currently, there are two main
173			branches:</para>
174			<itemizedlist mark="bullet">
175			<listitem>
176			<para>Development</para>
177			<itemizedlist mark="bullet">
178			<listitem>
179			<para>MAIN</para>
180			</listitem>
181			<listitem>
182			<para>ecco-branch</para>
183			</listitem>
184			</itemizedlist>
185			</listitem>
186			<listitem>
187			<para>Production</para>
188			<itemizedlist mark="bullet">
189			<listitem>
190			<para>Release1</para>
191			</listitem>
192			<listitem>
193			<para>Release2</para>
194			</listitem>
195			</itemizedlist>
196			</listitem>
197			</itemizedlist>
198			</sect2>
199			-->
200
201			<sect2>
202			<title>Branches</title>
203
204			<para>As shown in the online <ulink
205	jmc	1.11	url="http://mitgcm.org/viewvc/MITgcm/MITgcm/model/src/forward_step.F?view=graph">
206	dimitri	1.12	ViewCVS-generated tree</ulink>, the MITgcm codebase is split into
207			branches or "lines" under which development proceeds. The main line
208			of development is referred to as the "MAIN" version of the code.
209	edhill	1.1	</para>
210
211			<para>Periodically, a "Release" branch is formed from the "MAIN"
212	edhill	1.7	development branch. This is done in order to create a relatively stable
213			reference point for both users and developers. The intent is that once a
214	dimitri	1.12	release branch has been created, only bug-fixes will be added to it.
215	edhill	1.7	Meanwhile, development (which might "break" or otherwise render invalid
216			the documentation, tutorials, and/or examples contained within a release
217	dimitri	1.12	branch) is allowed to continue along the MAIN line.</para>
218	edhill	1.1	</sect2>
219
220			<sect2>
221	jmc	1.11	<title> Developer settings </title>
222
223			<para>CVS is a convenient tool to keep up-to-date a personal copy of the
224			MITgcm code (see: <ulink url="http://mitgcm.org/public/using_cvs.html">
225			using CVS </ulink>). The same tool is used by developers to
226			incorporate any change into the repository. However, this later
227			function requires specific settings, as detailed here after:</para>
228			<orderedlist>
229			<listitem>
230			<para> You will need an account (loggin access) to the server
231			"mitgcm.org" with the proper group setting (e.g.,
232			group "gcmctrb" to add/modify code into MITgcm_contrib).
233			This kind of account is granted only upon well motivated request.
234			The access to the server mitgcm.org is through ssh-key authorization
235			which will need to be set properly on both side (on your local machine
236			and on your server account). You need to be able to
237			to ssh to mitgcm.org (or <filename>ssh MY_USER_NAME@mitgcm.org</filename>
238			in case of different user-name on both sides) to proceed further.</para>
239			</listitem>
240
241			<listitem>
242			<para> You need to register to the
243			<ulink url="http://mitgcm.org/mailman/listinfo/mitgcm-cvs">
244			mitgcm-cvs </ulink> mailing list.
245			This ensures that other developers will receive email notification
246			when you make changes; you will also receive as well such email
247			when others make changes to the repository.
248			</para>
249			</listitem>
250
251			<listitem>
252			<para> It is highly recommended that you register also to the
253			<ulink url="http://mitgcm.org/mailman/listinfo/mitgcm-devel">
254			mitgcm-devel </ulink> mailing list (expect a short delay for
255			this request to be processed).
256			This list is intended for developer discussions.
257			</para>
258			</listitem>
259
260			<listitem>
261			<para> The standard anonymous mode (using "cvsanon", as mentionned
262			<ulink url="http://mitgcm.org/public/source_code.html">
263			here </ulink>) does not allow check-in ("cvs commit") permission.
264			Instead, you will need to set our CVS environment as follow:</para>
265			<screen>
266			$ export CVS_RSH=ssh
267			$ export CVSROOT=':ext:MY_USER_NAME@mitgcm.org:/u/gcmpack'
268			</screen>
269			<para> After downloading a directory, e.g.: <filename>myCopy</filename>,
270			from the CVS repository (e.g.,
271			<filename>MITgcm_contrib/thisPart</filename>) using the command:</para>
272			<screen>
273			$ cvs co -P -d myCopy MITgcm_contrib/thisPart
274			</screen>
275			<para> the type of CVS environment which has been used
276			is stored in the file <filename>myCopy/CVS/Root</filename>
277			and makes it difficult to re-use, for cvs-commit purpose,
278			a cvs local copy (<filename>myCopy</filename>) which was obtained
279			using the CVS anonymous mode.</para>
280			</listitem>
281
282			<listitem>
283			<para> At this stage, you should be able to send your modified source
284			file (e.g., <filename>src_file</filename>) from your local copy directory
285			(<filename>myCopy</filename>) to the CVS repository
286			(<filename>MITgcm_contrib/thisPart</filename>) using the command
287			"cvs commit":</para>
288			<screen>
289			$ cd myCopy
290			$ cvs -n update (optional; check if new changes have been made)
291			$ cvs diff src_file (optional; list your changes)
292			$ cvs commit src_file
293			</screen>
294			<para> It is essential that you provide a short description of the
295			changes you made to <filename>src_file</filename> as you check-in
296			this file (the "cvs commit" command automatically opens your standard
297			editor for this purpose).</para>
298			</listitem>
299
300			</orderedlist>
301
302			</sect2>
303
304			<sect2>
305			<title>Main code development</title>
306	jmc	1.14	<para><emphasis>(formerly named "Tagging" ; this section needs an update)
307			</emphasis></para>
308	edhill	1.1
309	edhill	1.7	<para>The intent of tagging is to create "known-good" checkpoints that
310			developers can use as references. Traditionally, MITgcm tagging has
311			maintained the following conventions:</para>
312	edhill	1.1
313			<orderedlist>
314			<listitem>
315	edhill	1.7	<para>Developer checks out code into a local CVS-managed directory,
316			makes various changes/additions, tests these edits, and eventually
317			reaches a point where (s)he is satisfied that the changes form a new
318			"useful" point in the evolution of the code.</para>
319	edhill	1.1	</listitem>
320
321			<listitem>
322			<para>The developer then runs the <ulink
323	jmc	1.10	url="http://mitgcm.org/viewvc/MITgcm/MITgcm/verification/testreport">
324			testreport</ulink> shell script to see if any problems are introduced.
325	edhill	1.7	While not intended to be exhaustive, the test cases within the
326			verification directory do provide some indication whether gross errors
327			have been introduced.
328	edhill	1.1	</para>
329			</listitem>
330
331			<listitem>
332			<para>Having satisfied him- or herself that the changes are
333			ready to be committed to the CVS repository, the developer
334			then:</para>
335			<orderedlist>
336			<listitem>
337	edhill	1.7	<para>adds a "checkpointXY_pre" comment (where X is a checkpoint
338			number and Y is a letter) to the <ulink
339	jmc	1.10	url="http://mitgcm.org/viewvc/MITgcm/MITgcm/doc/tag-index">
340	edhill	1.7	tag-index</ulink> file and checks it into the CVS
341			repository</para>
342	edhill	1.1	</listitem>
343			<listitem>
344	edhill	1.7	<para>submits the set of changes to the CVS repository and adds
345			comments to <filename>tag-index</filename> describing what the
346			changes are along with a matching "checkpointXY_post" entry</para>
347	edhill	1.1	</listitem>
348			</orderedlist>
349			</listitem>
350			</orderedlist>
351
352	edhill	1.7	<para>The result of this tagging procedure is a sequence of development
353			checkpoints with comments which resembles:</para>
354	edhill	1.1
355			<programlisting>
356			checkpoint50e_post
357			o make KPP work with PTRACERS
358			- fix gad_calc_rhs to call new routine kpp_transport_ptr, which is
359			nearly a copy of kpp_transport_s
360			- there is no analogue to SurfaceTendencyS, so I have to use
361			gPtr(of the surface layer) instead
362			o add a new platform SunFire+mpi (SunFire 15000) to genmake
363			checkpoint50e_pre
364
365			checkpoint50d_post
366			o change kpp output from multiple-record state files to single-record state
367			files analogous to write_state.F
368			o reduce the output frequency of cg3d-related stuff to the monitor frequency,
369			analogous to the cg2d-related output.
370			o fix small problem with in ptracers_write_checkpoint.F: len(suff)=512,
371			so that writing to internal file fn (with length 512) fails.
372			checkpoint50d_pre
373			</programlisting>
374
375	edhill	1.7	<para>This information can be used to refer to various stages of the code
376			development. For example, bugs can be traced to individual sets of CVS
377			checkins based upon their first appearance when comparing the results from
378			different checkpoints.</para>
379	edhill	1.1
380			</sect2>
381			</sect1>
382
383
384	edhill	1.2	<sect1 id="coding">
385	edhill	1.3	<title>Coding for MITgcm</title>
386
387			<sect2 id="build_tools">
388			<title>Build Tools</title>
389
390	edhill	1.4	<para>Many Open Source projects use the "GNU Autotools" to help streamline
391			the build process for various Unix and Unix-like architectures. For a
392			user, the result is the common "configure" (that is,
393			"<filename>./configure && make && make install</filename>") commands.
394			For MITgcm, the process is similar. Typical commands are:</para>
395	edhill	1.3
396			<screen>
397	jmc	1.14	$ genmake2 -mods=../code
398	edhill	1.5	$ make depend
399			$ make
400	edhill	1.3	</screen>
401
402	edhill	1.4	<para>The following sections describe the individual steps in the build
403			process.</para>
404
405	edhill	1.3	<sect3 id="genmake">
406			<title>The <filename>genmake2</> Utility</title>
407
408	jmc	1.14	<para><emphasis>(Note: the older <filename>genmake</>
409			has been replaced by <filename>genmake2</>)</emphasis></para>
410	edhill	1.4
411			<para>The first step in any MITgcm build is to create a Unix-style
412			<filename>Makefile</filename> which will be parsed by
413			<filename>make</filename> to specify how to compile the MITgcm source
414			files. For more detailed descriptions of what the make tools are and
415			how they are used, please see:</para>
416	edhill	1.3
417			<itemizedlist>
418			<listitem>
419	edhill	1.4	<para><ulink url="http://www.gnu.org/software/make/make.html">
420			http://www.gnu.org/software/make/make.html</></para>
421	edhill	1.3	</listitem>
422			<listitem>
423	edhill	1.4	<para><ulink url="http://www.oreilly.com/catalog/make2/">
424			http://www.oreilly.com/catalog/make2/</></para>
425	edhill	1.3	</listitem>
426			</itemizedlist>
427
428	edhill	1.4	<para>Genmake can often be invoked successfully with a command line as
429			simple as:</para>
430
431			<screen>
432	edhill	1.5	$ genmake2 -mods=../code
433	edhill	1.4	</screen>
434
435			<para>However, some systems (particularly commercial Unixes that lack a
436			more modern "/bin/sh" implementation or that have shells installed in
437			odd locations) may require an explicit shell invocation such as one of
438			the following: </para>
439
440			<screen>
441	edhill	1.5	$ /usr/bin/sh genmake2 -make=gmake -mods=../code
442			$ /opt/gnu/bin/bash genmake2 -ieee -make=/usr/local/bin/gmake -mods=../code
443	edhill	1.4	</screen>
444
445			<para>The genmake2 code has been written in a Bourne and BASH (v1)
446			compatible syntax so it should work with most "sh" and all recent "bash"
447			implementations.</para>
448
449			<para>As the name implies, <filename>genmake2</filename> generates a
450			<filename>Makefile</filename>. It does so by first parsing the
451			information supplied from the following sources</para>
452	edhill	1.3
453			<orderedlist>
454			<listitem>
455	jmc	1.6	<para>a <filename>gemake_local</filename> file in the current
456	edhill	1.4	directory</para>
457	edhill	1.3	</listitem>
458			<listitem>
459			<para>directly from command-line options</para>
460			</listitem>
461			<listitem>
462	edhill	1.4	<para>an "options file" as specified by the command-line option
463			<filename>-optfile='FILENAME'</filename></para>
464	edhill	1.3	</listitem>
465	jmc	1.14	<listitem>
466			<para> a <filename>packages.conf</filename> file (in the current
467			directory or in one of the "MODS" directories, see below)
468			which contains the specific list of packages to compile
469			</para>
470			</listitem>
471	edhill	1.3	</orderedlist>
472
473	edhill	1.4	<para>then checking certain dependency rules (the package dependencies),
474			and finally writing a <filename>Makefile</filename> based upon the
475			source code that it finds. For convenience within various Unix
476			shells, <filename>genmake2</> supports both "long"- and "short"-style
477			options. A complete list of the available options can be obtained
478			from:</para>
479	edhill	1.3
480			<screen>
481	edhill	1.5	$ genmake2 -help
482	edhill	1.3	</screen>
483
484	edhill	1.4	<para>The most important options for <filename>genmake2</> are:</para>
485	edhill	1.3
486			<variablelist>
487
488			<varlistentry>
489			<term><filename>--optfile=/PATH/FILENAME</></term>
490	edhill	1.4
491	edhill	1.3	<listitem>
492	edhill	1.4	<para>This specifies the "options file" that should be used for a
493			particular build. The options file is a convenient and
494			machine-indepenent way of specifying parameters such as the
495			FORTRAN compiler (<filename>FC=</>), FORTRAN compiler
496			optimization flags (<filename>FFLAGS=</>), and the locations of
497			various platform- and/or machine-specific tools
498			(eg. <filename>MAKEDEPEND=</>). As with <filename>genmake2</>,
499			all options files should be written to be compatible with
500			Bourne--shell ("sh" or "BASH v1") syntax. Examples of various
501			options files can be found in
502			<filename>$ROOTDIR/tools/build_options</>.</para>
503
504			<para>If no "optfile" is specified (either through the command lin
505			or the environment variable), genmake2 will try to make a
506			reasonable guess from the list provided in
507			<filename>$ROOTDIR/tools/build_options</>. The method used for
508			making this guess is to first determine the combination of
509			operating system and hardware (eg. "linux_ia32") and then find a
510			working Fortran compiler within the user's path. When these
511			three items have been identified, genmake2 will try to find an
512			optfile that has a matching name. </para>
513
514			<para>Everyone is encouraged to submit their options files to the
515			MITgcm project for inclusion (please send to
516			<email>MITgcm-support@mitgcm.org</email>). We are particularly
517			grateful for options files tested on new or unique
518			platforms!</para>
519	edhill	1.3	</listitem>
520	edhill	1.4
521	edhill	1.3	</varlistentry>
522
523			<varlistentry>
524	edhill	1.5	<term><filename>-adof=/path/to/file</></term>
525			<term><filename>-adoptfile=/path/to/file</></term>
526			<listitem>
527			<para>This option specifies the "adjoint" or automatic
528			differentiation options file to be used. The file is analogous
529			to the "optfile" defined above but it specifies information for
530			the AD build process. The default file is located in <filename>
531			$ROOTDIR/tools/adjoint_options/adjoint_default </> and it
532			defines the "TAF" and "TAMC" compilers. An alternate version is
533			also available at <filename>
534			$ROOTDIR/tools/adjoint_options/adjoint_staf </> that selects the
535			newer "STAF" compiler. As with any compilers, it is helpful to
536			have their directories listed in your $PATH environment
537			variable.</para>
538			</listitem>
539			</varlistentry>
540
541			<varlistentry>
542	edhill	1.4	<term><filename>-mods=DIR</></term>
543			<term><filename>-mods='DIR1 [DIR2 ...]'</></term>
544	edhill	1.3	<listitem>
545	edhill	1.4	<para>This option specifies a list of directories containing
546			"modifications". These directories contain files with names
547			that may (or may not) exist in the main MITgcm source tree but
548			will be overridden by any identically-named sources within the
549			"MODS" directories. The order of precedence for this
550			"name-hiding" is as follows:</para>
551			<itemizedlist>
552			<listitem><para>"MODS" directories (in the order given)
553			</para></listitem>
554			<listitem><para>Packages either explicitly specified or
555			provided by default (in the order given)</para></listitem>
556			<listitem><para>Packages included due to package dependencies
557			(in the order that that package dependencies are
558			parsed)</para></listitem>
559			<listitem><para>The "standard dirs" (which may have been
560			specified by the "-standarddirs" option)</para></listitem>
561			</itemizedlist>
562	jmc	1.14	</listitem>
563			</varlistentry>
564
565			<varlistentry>
566			<term><filename>-pgroups=/PATH/FILENAME</></term>
567			<listitem>
568			<para>This option specifies the file where package groups are
569			defined. If not set, the package-groups definition will
570			be read from
571			<filename>$ROOTDIR/pkg/pkg_groups</>.</para>
572			<para>
573			It also contains the default list of packages (defined
574			as the group <filename>"default_pkg_list"</>) which is used
575			when no specific package list (file: <filename>packages.conf</>)
576			is found in current directory or in any "MODS" directory.
577			</para>
578			</listitem>
579			</varlistentry>
580	edhill	1.4
581	jmc	1.14	<varlistentry>
582			<term><filename>-pdepend=/PATH/FILENAME</></term>
583
584			<listitem>
585			<para>This specifies the dependency file used for packages. If
586			not specified, the default dependency file is
587			<filename>$ROOTDIR/pkg/pkg_depend</>. The syntax for this file is
588			parsed on a line-by-line basis where each line containes either a
589			comment ("#") or a simple "PKGNAME1 (+\|-)PKGNAME2" pairwise rule
590			where the "+" or "-" symbol specifies a "must be used with" or a
591			"must not be used with" relationship, respectively. If no rule is
592			specified, then it is assumed that the two packages are compatible
593			and will function either with or without each other.</para>
594	edhill	1.4	</listitem>
595			</varlistentry>
596
597			<varlistentry>
598			<term><filename>-make=/path/to/gmake</></term>
599			<listitem>
600			<para>Due to the poor handling of soft-links and other bugs common
601			with the <filename>make</> versions provided by commercial Unix
602			vendors, GNU <filename>make</filename> (sometimes called
603			<filename>gmake</filename>) should be preferred. This option
604			provides a means for specifying the make program to be
605			used.</para>
606	edhill	1.3	</listitem>
607			</varlistentry>
608
609			</variablelist>
610
611	edhill	1.4	<para>A successful run of <filename>genmake2</> will produce a
612			<filename>Makefile</>, a <filename>PACKAGES_CONFIG.h</> file, and
613			various convenience files used for the automatic differentiation
614			process.</para>
615
616			<para>In general, it is best to use <filename>genmake2</> on a "clean"
617			directory that is free of all source (.[F,f],.[F,f]90) and header
618			(.h,.inc) files. Generally, this can be accomplished in an
619	jmc	1.14	"un-clean" directory by running "make Clean" followed by "make
620	edhill	1.4	makefile".</para>
621	edhill	1.3
622			</sect3>
623
624			<sect3 id="makefile_use">
625	edhill	1.5	<title>Using the <filename>Makefile</></title>
626	edhill	1.3
627	edhill	1.5	<para>Once a <filename>Makefile</> has been created using
628			<filename>genmake2</>, one can build a "standard" (forward
629			simulator) executable using:</para>
630	edhill	1.3
631			<screen>
632	jmc	1.14	$ make Clean
633	edhill	1.5	$ make depend
634			$ make
635	edhill	1.3	</screen>
636
637	jmc	1.14	<para>The "make Clean" step will remove any stale source files, include
638	edhill	1.4	files, and links. It is strongly recommended for "un-clean"
639			directories which may contain the (perhaps partial) results of
640	jmc	1.14	previous builds. Such "debris" can interfere with the next stage of
641			the build.
642			A more agressive cleaning option, "make CLEAN", can be used to also
643			remove the executable and output files from a previous run.</para>
644	edhill	1.4
645			<para>The "make depend" step will create a large number of symbolic
646			links from the local directory to the source file locations. It also
647			parses these files and creates an extensive list of dependencies
648			within the <filename>Makefile</> itself. The links that exist at this
649			stage are mostly "large F" files (.F and .F90) that need to be
650			processed by a C preprocessor ("CPP"). Since "make depend" edits the
651			<filename>Makefile</>, it is important not to skip this step!</para>
652
653			<para>The final "make" invokes the C preprocessor to produce the "little
654			f" files (.f and .f90) and then compiles them to object code using
655			the specified FORTRAN compiler and options. An intermediate script is
656			often used during this stage to further process (usually, make simple
657			substitutions) custom definitions such as variable types within the
658			source files. This additional stage is necessary in order to overcome
659			some of the inconsistencies in the sizes of objects (bytes) between
660	edhill	1.5	different compilers. The result of the build process is an executable
661			with the name <filename>mitgcmuv</>.</para>
662
663			<para>In addition to the forward simulator described above, the
664			<filename>Makefile</> also has a number of targets that can be used to
665			produce various adjoint and tangent-linear builds for optimization and
666			other parameter-sensitivity problems. The additional targets within
667			the <filename>Makefile</> are:</para>
668
669			<variablelist>
670
671			<varlistentry>
672			<term><filename>make adall</></term>
673			<listitem>
674			<para>This target produces an <filename>mitgcmuv_ad</> executable
675			using the <filename>taf</> or <filename>staf</> adjoint
676			compiler. See the <filename>genmake2</> "-adof" option for
677			compiler selection.</para>
678			</listitem>
679			</varlistentry>
680
681			<varlistentry>
682			<term><filename>make ftlall</></term>
683			<listitem>
684			<para>Similar to <filename>make adall</> above, this
685			produces...</para>
686			</listitem>
687			</varlistentry>
688
689			</variablelist>
690	edhill	1.3
691	edhill	1.5	<para>Please report any compilation failures or other build problems to
692			the <email>MITgcm-support@mitgcm.org</email> list.</para>
693	edhill	1.3
694			</sect3>
695
696			</sect2>
697
698			<sect2 id="verification">
699			<title>The Verification Suite</title>
700
701	edhill	1.4	<para>The MITgcm CVS tree (within the <filename>$ROOTDIR/verification/</>
702	jmc	1.15	directory) includes many (> 90) examples intended for regression
703	edhill	1.4	testing. Each one of these example directories contains "known-good"
704			output files along with all the input (including both code and data
705			files) required for their re-calculation. These example directories are
706			further broken down into sets of subdirectories
707			(eg. <filename>/input</>, <filename>/code</>) intended to expedite the
708			testing process.</para>
709	edhill	1.3
710			<sect3 id="testreport">
711			<title>The <filename>testreport</> Utility</title>
712
713	edhill	1.4	<para>Also included in <filename>$ROOTDIR/verification/</> are shell
714	jmc	1.15	scripts for automated testing. The script (which was written
715	edhill	1.4	to work with <filename>genmake2</>) is called <filename>testreport</>.
716			This script can be used to build different versions of the MITgcm
717			code, run the various examples, compare the output, and (if specified)
718			email the results of each one of these tests to a central
719			repository.</para>
720
721			<para>On some systems, the testreport script can be run with a command
722			line as simple as:</para>
723
724			<screen>
725	edhill	1.5	$ cd verification
726	jmc	1.15	$ ./testreport
727	edhill	1.4	</screen>
728
729			<para>However, some systems (those lacking or wiht a broken "/bin/sh")
730			may require an explicit shell invocation such as:</para>
731
732			<screen>
733	jmc	1.15	$ sh ./testreport -t 'exp2 exp4'
734			$ /some/path/to/bash ./testreport -t 'ideal_2D_oce lab_sea natl_box'
735	edhill	1.4	</screen>
736	edhill	1.3
737			<para>The <filename>testreport</> script accepts a number of
738	edhill	1.4	command-line options which can be listed using the <filename>-help</>
739			option. The most important ones are:</para>
740	edhill	1.3
741			<variablelist>
742
743			<varlistentry>
744	jmc	1.15	<term><filename>-ieee</> (default) / <filename>-noieee</></term>
745	edhill	1.4	<listitem>
746			<para>If allowed by the compiler (as defined in the "optfile"),
747	jmc	1.15	use IEEE arithmetic (<filename>genmake2 -ieee</>).
748			This option, along with the gfortran / gcc compiler,
749			is how the standard results are produced.</para>
750			</listitem>
751			</varlistentry>
752
753			<varlistentry>
754			<term><filename>-optfile=/PATH/FILENAME</></term>
755			<term><filename>-optfile '/PATH/F1 [/PATH/F2 ...]'</></term>
756			<listitem>
757			<para>This specifies a list of "options files" that will be passed
758			to <filename>genmake2</>. If multiple options files are used
759			(say, to test different compilers or different sets of options
760			for the same compiler), then each options file will be used with
761			each of the test directories.</para>
762	edhill	1.4	</listitem>
763			</varlistentry>
764
765			<varlistentry>
766	edhill	1.3	<term><filename>-tdir TESTDIR</></term>
767			<term><filename>-tdir 'TDIR1 TDIR2 [...]'</></term>
768			<listitem>
769	edhill	1.4	<para>This option specifies the test directory or list of test
770			directories that should be used. Each of these entries should
771			exactly (note: they are case sensitive!) match the names of
772	jmc	1.15	directories in <filename>$ROOTDIR/verification/</>. If this
773	edhill	1.4	option is omitted, then all directories that are properly
774			formatted (that is, containing an <filename>input</>
775			sub-directory and a <filename>results/output.txt</> file) will
776			be used.</para>
777	edhill	1.3	</listitem>
778			</varlistentry>
779
780			<varlistentry>
781			<term><filename>-addr EMAIL</></term>
782			<term><filename>-addr 'EMAIL1 EMAIL2 [...]'</></term>
783			<listitem>
784			<para>Send the results (namely, <filename>output.txt</>,
785	edhill	1.4	<filename>genmake_local</>, <filename>genmake_state</>, and
786			<filename>Makefile</>) to the specified email addresses. The
787			results are gzipped, placed in a tar file, MIME encoded, and
788			sent to the specified address. If no email addresses are
789			specified, no mail is sent.</para>
790			</listitem>
791			</varlistentry>
792
793			<varlistentry>
794	jmc	1.15	<term><filename>-MPI NUMBER_OF_PROCS</></term>
795	edhill	1.4	<term><filename>-mpi</></term>
796			<listitem>
797	jmc	1.15	<para>If the necessary file (<filename>TESTDIR/code/SIZE.h_mpi</>)
798			exists, then use it (and all <filename>TESTDIR/code/*_mpi</> files)
799			for an MPI--enabled run. The new option (<filename>-MPI</> followed
800			by the maximum number of processors) enable to build and run each
801			test-experiment using variable number of MPI processors (multiple
802			of <filename>nPx*nPy</> from <filename>TESTDIR/code/SIZE.h_mpi</>
803			and not larger than <filename>NUMBER_OF_PROCS</>).
804			The short option ("-mpi") can only be used to build and run on 2 MPI
805			processors (equivalent to "<filename>-MPI 2</>").</para>
806			<para>Note that the use of MPI typically requires a
807	edhill	1.4	special command option (see "-command" below) to invoke the MPI
808	jmc	1.15	executable. Examples of PBS scripts using testreport with MPI can be
809	edhill	1.4	found in the <ulink
810	jmc	1.15	url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
811			tools/example_scripts directory</ulink>.</para>
812	edhill	1.4	</listitem>
813			</varlistentry>
814
815			<varlistentry>
816			<term><filename>-command='some command to run'</></term>
817			<listitem>
818	jmc	1.15	<para>For some tests, particularly MPI runs, a specific command
819			might be needed to run the executable. This option allows a more general
820	edhill	1.4	command (or shell script) to be invoked. Examples of PBS scripts
821	jmc	1.15	using testreport with MPI can be found in the <ulink
822			url="http://mitgcm.org/viewvc/MITgcm/MITgcm/tools/example_scripts/">
823			tools/example_scripts directory</ulink>.</para>
824	edhill	1.3	</listitem>
825			</varlistentry>
826
827			</variablelist>
828
829	edhill	1.4	<para>The <filename>testreport</> script will write progress to the
830			screen (stdout) as it runs. In addition, it will create a
831			<filename>tr_out.txt</> file that contains a brief comparison of the
832			current output with the "known-good" output.</para>
833	edhill	1.3
834			</sect3>
835
836			</sect2>
837	edhill	1.2
838	edhill	1.4
839	edhill	1.2	<sect2 id="packages">
840	edhill	1.3	<title>Creating MITgcm Packages</title>
841	edhill	1.2
842	edhill	1.4	<para>Optional parts of code have been separated from the MITgcmUV core
843			driver code and organised into packages. The packaging structure
844			provides a mechanism for maintaining suites of code, specific to
845			particular classes of problems, in a way that is cleanly separated from
846			the generic fluid dynamical engine.</para>
847
848			<para>The MITgcmUV packaging structure is described below using generic
849	jmc	1.11	package names ${pkg}. A concrete examples of a package is the code for
850	edhill	1.4	implementing GM/Redi mixing. This code uses the package name</para>
851	edhill	1.2
852			</sect2>
853
854			</sect1>
855
856			<sect1>
857			<title>Chris's Notes...</title>
858
859			<programlisting>
860			MITgcmUV Packages
861			=================
862
863			Optional parts of code are separated from
864			the MITgcmUV core driver code and organised into
865			packages. The packaging structure provides a mechanism for
866			maintaining suites of code, specific to particular
867			classes of problem, in a way that is cleanly
868			separated from the generic fluid dynamical engine.
869
870			The MITgcmUV packaging structure is describe
871			below using generic package names ${pkg}.
872			A concrete examples of a package is the code
873			for implementing GM/Redi mixing. This code uses
874			the package name
875			* ${PKG} = GMREDI
876			* ${pkg} = gmredi
877			* ${Pkg} = gmRedi
878
879			Package states
880			==============
881
882			Packages can be any one of four states, included,
883			excluded, enabled, disabled as follows:
884
885			included(excluded) compile time state which
886			includes(excludes) package
887			code and routine calls from
888			compilation/linking etc...
889
890			enabled(disabled) run-time state which
891			enables(disables) package code
892			execution.
893
894			Every call to a ${pkg}_... routine from outside the package
895			should be placed within both a
896			#ifdef ALLOW_${PKG} ... block and a
897			if ( use${Pkg} ) ... then block.
898			Package states are generally not expected to change during
899			a model run.
900
901			Package structure
902			=================
903
904			o Each package gets its runtime configuration
905			parameters from a file named "data.${pkg}"
906			Package runtime config. options are imported
907			into a common block held in a header file
908			called "${PKG}.h".
909	jmc	1.8	Note: In some packages, the header file "${PKG}.h" is splitted
910			into "${PKG}_PARAMS.h" that contains the package parameters and
911			${PKG}_VARS.h" for the field arrays.
912	edhill	1.2
913			o The core driver part of the model can check
914			for runtime enabling or disabling of individual packages
915			through logical flags use${Pkg}.
916			The information is loaded from a
917			global package setup file called "data.pkg".
918			The use${Pkg} flags are not used within
919			individual packages.
920
921			o Included in "${PKG}.h" is a logical flag
922			called ${Pkg}IsOn. The "${PKG}.h" header file can be imported
923			by other packages to check dependencies and requirements
924			from other packages ( see "Package Boot Sequence" section).
925			NOTE: This procedure is not presently implemented,
926			----- neither for kpp nor for gmRedi.
927
928			CPP Flags
929			=========
930
931			1. Within the core driver code flags of the form
932			ALLOW_${PKG} are used to include or exclude
933			whole packages. The ALLOW_${PKG} flags are included
934	jmc	1.8	from a PACKAGES_CONFIG.h file that is automatically
935			generated by genmake2 (see genmake2 section).
936	edhill	1.2	held in-line in the CPP_OPTIONS.h header file.
937			e.g.
938
939			Core model code .....
940
941	jmc	1.8	#include "PACKAGES_CONFIG.h"
942	edhill	1.2	#include "CPP_OPTIONS.h"
943			:
944			:
945			:
946
947			#ifdef ALLOW_${PKG}
948			if ( use${Pkg} ) CALL ${PKG}_DO_SOMETHING(...)
949			#endif
950
951			2. Within an individual package a header file,
952			"${PKG}_OPTIONS.h", is used to set CPP flags
953	jmc	1.8	specific to that package. It also includes
954			"PACKAGES_CONFIG.h" and "CPP_OPTIONS.h".
955	edhill	1.2
956
957			Package Boot Sequence
958			=====================
959
960			Calls to package routines within the core code timestepping
961			loop can vary. However, all packages follow a required
962			"boot" sequence outlined here:
963
964			1. S/R PACKAGES_BOOT()
965			:
966			CALL OPEN_COPY_DATA_FILE( 'data.pkg', 'PACKAGES_BOOT', ... )
967
968
969			2. S/R PACKAGES_READPARMS()
970			:
971			#ifdef ALLOW_${PKG}
972			if ( use${Pkg} )
973			& CALL ${PKG}_READPARMS( retCode )
974			#endif
975
976	jmc	1.6	3. S/R PACKAGES_INIT_FIXED()
977			:
978			#ifdef ALLOW_${PKG}
979			if ( use${Pkg} )
980			& CALL ${PKG}_INIT_FIXED( retCode )
981			#endif
982
983			4. S/R PACKAGES_CHECK()
984	edhill	1.2	:
985			#ifdef ALLOW_${PKG}
986			if ( use${Pkg} )
987			& CALL ${PKG}_CHECK( retCode )
988			#else
989			if ( use${Pkg} )
990			& CALL PACKAGES_CHECK_ERROR('${PKG}')
991			#endif
992
993	jmc	1.6	5. S/R PACKAGES_INIT_VARIABLES()
994	edhill	1.2	:
995			#ifdef ALLOW_${PKG}
996			if ( use${Pkg} )
997	jmc	1.6	& CALL ${PKG}_INIT_VARIA( )
998			#endif
999
1000			Package Output
1001			==============
1002			6. S/R DO_THE_MODEL_IO
1003
1004			#ifdef ALLOW_${PKG}
1005			if ( use${Pkg} )
1006	jmc	1.9	& CALL ${PKG}_OUTPUT( )
1007	edhill	1.2	#endif
1008
1009	jmc	1.6	7. S/R PACKAGES_WRITE_PICKUP()
1010
1011			#ifdef ALLOW_${PKG}
1012			if ( use${Pkg} )
1013			& CALL ${PKG}_WRITE_PICKUP( )
1014			#endif
1015	edhill	1.2
1016			Description
1017			===========
1018
1019			- ${PKG}_READPARMS()
1020			is responsible for reading
1021			in the package parameters file data.${pkg}, and storing
1022	jmc	1.8	the package parameters in "${PKG}.h" (or in "${PKG}_PARAMS.h").
1023	jmc	1.6	-> called from INITIALISE_FIXED in PACKAGES_READPARMS
1024
1025			- ${PKG}_INIT_FIXED()
1026			is responsible for completing the internal setup of a package.
1027			-> called from INITIALISE_FIXED in PACKAGES_INIT_FIXED
1028			note: 1) some pkg use instead:
1029			CALL ${PKG}_INITIALISE ( or the old form CALL ${PKG}_INIT )
1030			2) for simple pkg setup, this part is done inside ${PKG}_READPARMS
1031	edhill	1.2
1032			- ${PKG}_CHECK()
1033			is responsible for validating
1034			basic package setup and inter-package dependencies.
1035			${PKG}_CHECK can import other package parameters it may
1036			need to check. This is done through header files "${PKG}.h".
1037			It is assumed that parameters owned by other packages
1038			will not be reset during ${PKG}_CHECK().
1039	jmc	1.6	-> called from INITIALISE_FIXED in PACKAGES_CHECK
1040	edhill	1.2
1041	jmc	1.6	- ${PKG}_INIT_VARIA()
1042			is responsible for fill-in all package variables with an initial value.
1043			Contains eventually a call to ${PKG}_READ_PICKUP that will read
1044			from a pickup file the package variables required to restart the model.
1045			This routine is called after the core model state has been completely
1046			initialised but before the core model timestepping starts.
1047			-> called from INITIALISE_VARIA in PACKAGES_INIT_VARIABLES
1048			note: the name ${PKG}_INIT_VARIA is not yet standard and some pkg
1049			use for e.g. ${PKG}_INI_VARS, ${PKG}_INIT_VARIABLES, or the old
1050			form ${PKG}_INIT
1051
1052	jmc	1.9	- ${PKG}_OUTPUT( )
1053	jmc	1.8	is responsible for writing time-average fields to output files
1054			(but the cumulating step is done within the package main S/R).
1055	jmc	1.6	Can also contain other diagnostics (.e.g. CALL ${PKG}_MONITOR)
1056			and write snap-shot fields that are hold in common blocks. Other
1057			temporary fields are directly dump to file where they are available.
1058	jmc	1.9	NOTE: 1) the S/R old name ${PKG}_DIAGS is used in some packages
1059			but is beeing replaced by ${PKG}_OUTPUT
1060	jmc	1.8	to avoid confusion with pkg/diagnostics functionality.
1061			2) the output part is not yet in a standard form and might still
1062			evolve a lot.
1063	jmc	1.6	-> called within DO_THE_MODEL_IO
1064
1065			- ${PKG}_WRITE_PICKUP()
1066			is responsible for writing a package pickup file when necessary for
1067			a restart. (found also the old name: ${PKG}_WRITE_CHECKPOINT )
1068			-> called from FORWARD_STEP and THE_MODEL_MAIN in PACKAGES_WRITE_PICKUP
1069	edhill	1.2
1070			Summary
1071			=======
1072
1073			- CPP options:
1074			-----------------------
1075			* ALLOW_${PKG} include/exclude package for compilation
1076
1077			- FORTRAN logical:
1078			-----------------------
1079			* use${Pkg} enable package for execution at runtime
1080			-> declared in PARAMS.h
1081			* ${Pkg}IsOn for package cross-dependency check
1082			-> declared in ${PKG}.h
1083			N.B.: Not presently used!
1084
1085			- header files
1086			-----------------------
1087			* ${PKG}_OPTIONS.h has further package-specific CPP options
1088			* ${PKG}.h package-specific common block variables, fields
1089	jmc	1.8	or ${PKG}_PARAMS.h package-specific common block parameters
1090			and ${PKG}_VARS.h package-specific common block fields
1091	edhill	1.2
1092			- FORTRAN source files
1093			-----------------------
1094	jmc	1.6	* ${pkg}_readparms.F reads parameters from file data.${pkg}
1095			* ${pkg}_init_fixed.F complete the package setup
1096			* ${pkg}_check.F checks package dependencies and consistencies
1097			* ${pkg}_init_varia.F initialises package-related fields
1098			* ${pkg}_... .F package source code
1099	jmc	1.9	* ${pkg}_output.F write output to file.
1100	jmc	1.6	* ${pkg}_write_pickup.F write a package pickup file to restart the model
1101	edhill	1.2
1102	jmc	1.8	New: Subroutine in one package (pkgA) that only contains code which
1103			is connected to a 2nd package (pkgB) (e.g.: gmredi_diagnostics_init.F)
1104			will be named: pkgA_pkgB_something.F
1105
1106	edhill	1.2	- parameter file
1107			-----------------------
1108			* data.${pkg} parameter file
1109			</programlisting>
1110
1111			</sect1>
1112	edhill	1.1
1113
1114	jmc	1.11	<sect1 id="documentation">
1115			<title>Editing the Documentation</title>
1116
1117			<sect2 id="documentation_getting">
1118			<title>Getting the Docs and Code</title>
1119
1120			<para>The first step towards editing the documentation is to checkout a
1121			copy of code, docs, and build scripts from the CVS server using:</para>
1122
1123			<screen>
1124			$ export CVS_RSH=ssh
1125			$ export CVSROOT=':ext:NAME@mitgcm.org:/u/gcmpack'
1126			$ mkdir scratch
1127			$ cvs co -P MITgcm manual mitgcm.org
1128			</screen>
1129
1130			<para>These commands extract the necessary information from the CVS server
1131			and create a temporary (called <filename>scratch</filename>) directory for
1132			the storage of the HTML and other files that will be created. Please note
1133			that you must either create <filename>scratch</filename> as shown or edit
1134			the various <filename>Makefile</filename>s and scripts used to create the
1135			documentation.</para>
1136			</sect2>
1137
1138			<sect2>
1139			<title>Editing the Documentation</title>
1140
1141			<para>The documentation is contained in the <filename>manual</filename>
1142			directory in a raw LaTeX format. The main document is
1143			<filename>manual.tex</filename> and it uses <command>\input{}</command>s
1144			to include the chapters and subsections.</para>
1145
1146			<para>Since the same LaTeX source is used to produce PostScript, PDF, and
1147			HTML output, care should be taken to follow certain conventions. Two of
1148			the most important are the usage of the <command>\filelink{}{}</command>
1149			and <command>\varlink{}{}</command> commands. Both of these commands have
1150			been defined to simplify the connection between the automatically
1151			generated ("code browser") HTML and the HTML version of the manual
1152			produced by LaTeX2HTML. They each take two arguments (corresponding to
1153			the contents of the two sets of curly braces) which are the text that the
1154			author wishes to be "wrapped" within the link, and a specially formatted
1155			link thats relative to the <filename>MITgcm</filename> directory within
1156			the CVS tree.</para>
1157
1158			<para>The result is a command that resembles either</para>
1159
1160			<orderedlist>
1161			<listitem>
1162			<para>a reference to a variable or subroutine name such as
1163			<command>\varlink{tRef}{tRef}</command>, or </para>
1164			</listitem>
1165
1166			<listitem>
1167			<para>a reference to a file such as
1168			<command>\varlink{tRef}{path-to-the-file_name.F}</command>
1169			where the absolute path to the file is of the form
1170			<filename>/foo/MITgcm/path/to/the/file_name.F</filename></para>
1171			<para>(please note how the leading "/foo/MITgcm"
1172			component of the path is dropped leaving the path
1173			<emphasis>relative</emphasis> to the head of the code
1174			directory and each directory separator "/" is turned
1175			into a "-")</para>
1176			</listitem>
1177			</orderedlist>
1178
1179
1180
1181			</sect2>
1182
1183			<sect2>
1184			<title>Building the Documentation</title>
1185
1186			<para>Given the directory structure of <xref
1187			linkend="documentation_getting">, the entire documentation for the web
1188			site can be built using:</para>
1189
1190			<screen>
1191			$ cd mitgcm.org/devel/buildweb
1192			$ make All
1193			</screen>
1194
1195			<para>Which builds the PDF from the LaTeX source, creates the HTML output
1196			from the LaTeX source, parses the FORTRAN code base to produce a
1197			hyperlinked HTML version of the source, and then determines the
1198			cross-linking between the various HTML components.</para>
1199
1200			<para>If there are no errors, the result of the build process (which can
1201			take 30+ minutes on a P4/2.5Ghz) will be contained within a single
1202			directory called <filename>scratch/dev_docs</filename>. This is a freshly
1203			built version of the entire on-line users manual. If you have the correct
1204			permissions, it can be directly copied to the web server area:</para>
1205
1206			<screen>
1207			$ mv scratch/dev_docs /u/u0/httpd/html
1208			</screen>
1209
1210			<para>and the update is complete.</para>
1211
1212			</sect2>
1213
1214			</sect1>
1215
1216	edhill	1.1	</article>
1217
1218

	ViewVC Help
Powered by ViewVC 1.1.22

Annotation of /MITgcm/doc/devel_HOWTO.sgml