1 |
% $Header: /u/gcmpack/mitgcmdoc/part5/doc_ad_examples.tex,v 1.1 2002/02/28 19:32:20 cnh Exp $ |
2 |
% $Name: $ |
3 |
|
4 |
%********************************************************************** |
5 |
\section{Sensitivity of Air-Sea Exchange to Tracer Injection Site } |
6 |
\label{sect:eg-simple-tracer-adjoint} |
7 |
\label{sec_ad_setup_ex} |
8 |
%********************************************************************** |
9 |
|
10 |
The MITGCM has been adapted to enable AD using TAMC or TAF. |
11 |
The present description, therefore, is specific to the |
12 |
use of TAMC or TAF as AD tool. |
13 |
The following sections describe the steps which are necessary to |
14 |
generate a tangent linear or adjoint model of the MITGCM. |
15 |
We take as an example the sensitivity of carbon sequestration |
16 |
in the ocean. |
17 |
The AD-relevant hooks in the code are sketched in |
18 |
\ref{fig:adthemodel}, \ref{fig:adthemain}. |
19 |
|
20 |
\subsection{Overview of the experiment} |
21 |
|
22 |
We describe an adjoint sensitivity analysis of out-gassing from |
23 |
the ocean into the atmosphere of a carbon-like tracer injected |
24 |
into the ocean interior (see \cite{hil-eta:01}). |
25 |
|
26 |
\subsubsection{Passive tracer equation} |
27 |
|
28 |
For this work the MITGCM was augmented with a thermodynamically |
29 |
inactive tracer, $C$. Tracer residing in the ocean |
30 |
model surface layer is out-gassed according to a relaxation time scale, |
31 |
$\mu$. Within the ocean interior, the tracer is passively advected |
32 |
by the ocean model currents. The full equation for the time evolution |
33 |
% |
34 |
\begin{equation} |
35 |
\label{carbon_ddt} |
36 |
\frac{\partial C}{\partial t} \, = \, |
37 |
-U\cdot \nabla C \, - \, \mu C \, + \, \Gamma(C) \,+ \, S |
38 |
\end{equation} |
39 |
% |
40 |
also includes a source term $S$. This term |
41 |
represents interior sources of $C$ such as would arise due to |
42 |
direct injection. |
43 |
The velocity term, $U$, is the sum of the |
44 |
model Eulerian circulation and an eddy-induced velocity, the latter |
45 |
parameterized according to Gent/McWilliams |
46 |
(\cite{gen-mcw:90, gen-eta:95}). |
47 |
The convection function, $\Gamma$, mixes $C$ vertically wherever the |
48 |
fluid is locally statically unstable. |
49 |
|
50 |
The out-gassing time scale, $\mu$, in eqn. (\ref{carbon_ddt}) |
51 |
is set so that \( 1/\mu \sim 1 \ \mathrm{year} \) for the surface |
52 |
ocean and $\mu=0$ elsewhere. With this value, eqn. (\ref{carbon_ddt}) |
53 |
is valid as a prognostic equation for small perturbations in oceanic |
54 |
carbon concentrations. This configuration provides a |
55 |
powerful tool for examining the impact of large-scale ocean circulation |
56 |
on $ CO_2 $ out-gassing due to interior injections. |
57 |
As source we choose a constant in time injection of |
58 |
$ S = 1 \,\, {\rm mol / s}$. |
59 |
|
60 |
\subsubsection{Model configuration} |
61 |
|
62 |
The model configuration employed has a constant |
63 |
$4^\circ \times 4^\circ$ resolution horizontal grid and realistic |
64 |
geography and bathymetry. Twenty vertical layers are used with |
65 |
vertical spacing ranging |
66 |
from 50 m near the surface to 815 m at depth. |
67 |
Driven to steady-state by climatological wind-stress, heat and |
68 |
fresh-water forcing the model reproduces well known large-scale |
69 |
features of the ocean general circulation. |
70 |
|
71 |
\subsubsection{Out-gassing cost function} |
72 |
|
73 |
To quantify and understand out-gassing due to injections of $C$ |
74 |
in eqn. (\ref{carbon_ddt}), |
75 |
we define a cost function $ {\cal J} $ that measures the total amount of |
76 |
tracer out-gassed at each timestep: |
77 |
% |
78 |
\begin{equation} |
79 |
\label{cost_tracer} |
80 |
{\cal J}(t=T)=\int_{t=0}^{t=T}\int_{A} \mu C \, dA \, dt |
81 |
\end{equation} |
82 |
% |
83 |
Equation(\ref{cost_tracer}) integrates the out-gassing term, $\mu C$, |
84 |
from (\ref{carbon_ddt}) |
85 |
over the entire ocean surface area, $A$, and accumulates it |
86 |
up to time $T$. |
87 |
Physically, ${\cal J}$ can be thought of as representing the amount of |
88 |
$CO_2$ that our model predicts would be out-gassed following an |
89 |
injection at rate $S$. |
90 |
The sensitivity of ${\cal J}$ to the spatial location of $S$, |
91 |
$\frac{\partial {\cal J}}{\partial S}$, |
92 |
can be used to identify regions from which circulation |
93 |
would cause $CO_2$ to rapidly out-gas following injection |
94 |
and regions in which $CO_2$ injections would remain effectively |
95 |
sequestered within the ocean. |
96 |
|
97 |
\subsection{Code configuration} |
98 |
|
99 |
The model configuration for this experiment resides under the |
100 |
directory {\it verification/carbon/}. |
101 |
The code customization routines are in {\it verification/carbon/code/}: |
102 |
% |
103 |
\begin{itemize} |
104 |
% |
105 |
\item {\it .genmakerc} |
106 |
% |
107 |
\item {\it COST\_CPPOPTIONS.h} |
108 |
% |
109 |
\item {\it CPP\_EEOPTIONS.h} |
110 |
% |
111 |
\item {\it CPP\_OPTIONS.h} |
112 |
% |
113 |
\item {\it CTRL\_OPTIONS.h} |
114 |
% |
115 |
\item {\it ECCO\_OPTIONS.h} |
116 |
% |
117 |
\item {\it SIZE.h} |
118 |
% |
119 |
\item {\it adcommon.h} |
120 |
% |
121 |
\item {\it tamc.h} |
122 |
% |
123 |
\end{itemize} |
124 |
% |
125 |
The runtime flag and parameters settings are contained in |
126 |
{\it verification/carbon/input/}, |
127 |
together with the forcing fields and and restart files: |
128 |
% |
129 |
\begin{itemize} |
130 |
% |
131 |
\item {\it data} |
132 |
% |
133 |
\item {\it data.cost} |
134 |
% |
135 |
\item {\it data.ctrl} |
136 |
% |
137 |
\item {\it data.gmredi} |
138 |
% |
139 |
\item {\it data.grdchk} |
140 |
% |
141 |
\item {\it data.optim} |
142 |
% |
143 |
\item {\it data.pkg} |
144 |
% |
145 |
\item {\it eedata} |
146 |
% |
147 |
\item {\it topog.bin} |
148 |
% |
149 |
\item {\it windx.bin, windy.bin} |
150 |
% |
151 |
\item {\it salt.bin, theta.bin} |
152 |
% |
153 |
\item {\it SSS.bin, SST.bin} |
154 |
% |
155 |
\item {\it pickup*} |
156 |
% |
157 |
\end{itemize} |
158 |
% |
159 |
Finally, the file to generate the adjoint code resides in |
160 |
$ adjoint/ $: |
161 |
% |
162 |
\begin{itemize} |
163 |
% |
164 |
\item {\it makefile} |
165 |
% |
166 |
\end{itemize} |
167 |
% |
168 |
|
169 |
Below we describe the customizations of this files which are |
170 |
specific to this experiment. |
171 |
|
172 |
\subsubsection{File {\it .genmakerc}} |
173 |
This file overwrites default settings of {\it genmake}. |
174 |
In the present example it is used to switch on the following |
175 |
packages which are related to automatic differentiation |
176 |
and are disabled by default: \\ |
177 |
\hspace*{4ex} {\tt set ENABLE=( autodiff cost ctrl ecco gmredi grdchk kpp )} \\ |
178 |
Other packages which are not needed are switched off: \\ |
179 |
\hspace*{4ex} {\tt set DISABLE=( aim obcs zonal\_filt shap\_filt cal exf )} |
180 |
|
181 |
\subsubsection{File {\it COST\_CPPOPTIONS.h, CTRL\_OPTIONS.h}} |
182 |
|
183 |
These files used to contain package-specific CPP-options |
184 |
(see Section \ref{???}). |
185 |
For technical reasons those options have been grouped together |
186 |
in the file {\it ECCO\_OPTIONS.h}. |
187 |
To retain the modularity, the files have been kept and contain |
188 |
the standard include of the {\it CPP\_OPTIONS.h} file. |
189 |
|
190 |
\subsubsection{File {\it CPP\_EEOPTIONS.h}} |
191 |
|
192 |
This file contains 'wrapper'-specific CPP options. |
193 |
It only needs to be changed if the code is to be run |
194 |
in a parallel environment (see Section \ref{???}). |
195 |
|
196 |
\subsubsection{File {\it CPP\_OPTIONS.h}} |
197 |
|
198 |
This file contains model-specific CPP options |
199 |
(see Section \ref{???}). |
200 |
Most options are related to the forward model setup. |
201 |
They are identical to the global steady circulation setup of |
202 |
{\it verification/global\_ocean.90x40x15/}. |
203 |
The three options specific to this experiment are \\ |
204 |
\hspace*{4ex} {\tt \#define ALLOW\_PASSIVE\_TRACER} \\ |
205 |
This flag enables the code to carry through the |
206 |
advection/diffusion of a passive tracer along the |
207 |
model integration. \\ |
208 |
\hspace*{4ex} {\tt \#define ALLOW\_MIT\_ADJOINT\_RUN} \\ |
209 |
This flag enables the inclusion of some AD-related fields |
210 |
concerning initialization, link between control variables |
211 |
and forward model variables, and the call to the top-level |
212 |
forward/adjoint subroutine {\it adthe\_main\_loop} |
213 |
instead of {\it the\_main\_loop}. \\ |
214 |
\hspace*{4ex} {\tt \#define ALLOW\_GRADIENT\_CHECK} \\ |
215 |
This flag enables the gradient check package. |
216 |
After computing the unperturbed cost function and its gradient, |
217 |
a series of computations are performed for which \\ |
218 |
$\bullet$ an element of the control vector is perturbed \\ |
219 |
$\bullet$ the cost function w.r.t. the perturbed element is |
220 |
computed \\ |
221 |
$\bullet$ the difference between the perturbed and unperturbed |
222 |
cost function is computed to compute the finite difference gradient \\ |
223 |
$\bullet$ the finite difference gradient is compared with the |
224 |
adjoint-generated gradient. |
225 |
The gradient check package is further described in Section ???. |
226 |
|
227 |
\subsubsection{File {\it ECCO\_OPTIONS.h}} |
228 |
|
229 |
The CPP options of several AD-related packages are grouped |
230 |
in this file: |
231 |
% |
232 |
\begin{itemize} |
233 |
% |
234 |
\item |
235 |
Overall ECCO-related execution modus: \\ |
236 |
These determine whether a pure forward run, |
237 |
a sensitivity run or an iteration of optimization is |
238 |
performed. These options are not needed in the present context. |
239 |
% |
240 |
\item |
241 |
Adjoint support package: {\it pkg/autodiff/} \\ |
242 |
This package contains hand-written adjoint code such as |
243 |
active file handling, flow directives for files which must not |
244 |
be differentiated, and TAMC-specific header files. \\ |
245 |
% |
246 |
\hspace*{4ex} {\tt \#define ALLOW\_AUTODIFF\_TAMC} \\ |
247 |
defines TAMC-related features in the code. \\ |
248 |
% |
249 |
\hspace*{4ex} {\tt \#define ALLOW\_TAMC\_CHECKPOINTING} \\ |
250 |
enables the checkpointing feature of TAMC |
251 |
(see Section \ref{???}). |
252 |
In the present example a 3-level checkpointing is implemented. |
253 |
The code contains the relevant store directives, common block |
254 |
and tape initializations, storing key computation, |
255 |
and loop index handling. |
256 |
The checkpointing length at each level is defined in |
257 |
file {\it tamc.h}, cf. below. |
258 |
The out and intermediate loop directivs are contained |
259 |
in the files {\it checkpoint\_lev3\_directives.h}, |
260 |
{\it checkpoint\_lev2\_directives.h} (package {\it pkg/autodiff}). \\ |
261 |
% |
262 |
\hspace*{4ex} {\tt \#define ALLOW\_AUTODIFF\_MONITOOR} \\ |
263 |
enables the monitoring of intermediate adjoint variables |
264 |
(see Section \ref{???}). \\ |
265 |
% |
266 |
\hspace*{4ex} {\tt \#define ALLOW\_DIVIDED\_ADJOINT} \\ |
267 |
enables adjoint dump and restart |
268 |
(see Section \ref{???}). |
269 |
% |
270 |
\item Cost function package: {\it pkg/cost/} \\ |
271 |
This package contains all relevant routines for |
272 |
initializing, accumulating and finalizing the cost function |
273 |
(see Section \ref{???}). \\ |
274 |
\hspace*{4ex} {\tt \#define ALLOW\_COST} \\ |
275 |
enables all general aspects of the cost function handling, |
276 |
in particular the hooks in the forward code for |
277 |
initializing, accumulating and finalizing the cost function. \\ |
278 |
\hspace*{4ex} {\tt \#define ALLOW\_COST\_TRACER} \\ |
279 |
includes the call to the cost function for this |
280 |
particular experiment, eqn. (\ref{cost_tracer}). |
281 |
% |
282 |
\item Control variable package: {\it pkg/ctrl/} \\ |
283 |
This package contains all relevant routines for |
284 |
the handling of the control vector. |
285 |
Each control variable can be enabled/disabled with its own flag: \\ |
286 |
\begin{tabular}{ll} |
287 |
\hspace*{2ex} {\tt \#define ALLOW\_THETA0\_CONTROL} & |
288 |
initial temperature \\ |
289 |
\hspace*{2ex} {\tt \#define ALLOW\_SALT0\_CONTROL} & |
290 |
initial salinity \\ |
291 |
\hspace*{2ex} {\tt \#define ALLOW\_TR0\_CONTROL} & |
292 |
initial passive tracer concentration \\ |
293 |
\hspace*{2ex} {\tt \#define ALLOW\_TAUU0\_CONTROL} & |
294 |
zonal wind stress \\ |
295 |
\hspace*{2ex} {\tt \#define ALLOW\_TAUV0\_CONTROL} & |
296 |
meridional wind stress \\ |
297 |
\hspace*{2ex} {\tt \#define ALLOW\_SFLUX0\_CONTROL} & |
298 |
freshwater flux \\ |
299 |
\hspace*{2ex} {\tt \#define ALLOW\_HFLUX0\_CONTROL} & |
300 |
heat flux \\ |
301 |
\hspace*{2ex} {\tt \#define ALLOW\_DIFFKR\_CONTROL} & |
302 |
diapycnal diffusivity \\ |
303 |
\hspace*{2ex} {\tt \#undef ALLOW\_KAPPAGM\_CONTROL} & |
304 |
isopycnal diffusivity \\ |
305 |
\end{tabular} |
306 |
% |
307 |
\end{itemize} |
308 |
|
309 |
\subsubsection{File {\it SIZE.h}} |
310 |
|
311 |
The file contains the grid point dimensions of the forward |
312 |
model. It is identical to the {\it verification/exp2/}: \\ |
313 |
\hspace*{4ex} {\tt sNx = 90} \\ |
314 |
\hspace*{4ex} {\tt sNy = 40} \\ |
315 |
\hspace*{4ex} {\tt Nr = 20} \\ |
316 |
It corresponds to a single-tile/single-processor setup: |
317 |
{\tt nSx = nSy = 1, nPx = nPy = 1}, |
318 |
with standard overlap dimensioning |
319 |
{\tt OLx = OLy = 3}. |
320 |
|
321 |
\subsubsection{File {\it adcommon.h}} |
322 |
|
323 |
This file contains common blocks of some adjoint variables |
324 |
that are generated by TAMC. |
325 |
The common blocks are used by the adjoint support routine |
326 |
{\it addummy\_in\_stepping} which needs to access those variables: |
327 |
|
328 |
\begin{tabular}{ll} |
329 |
\hspace*{4ex} {\tt common /addynvars\_r/} & |
330 |
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
331 |
\hspace*{4ex} {\tt common /addynvars\_cd/} & |
332 |
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
333 |
\hspace*{4ex} {\tt common /addynvars\_diffkr/} & |
334 |
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
335 |
\hspace*{4ex} {\tt common /addynvars\_kapgm/} & |
336 |
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
337 |
\hspace*{4ex} {\tt common /adtr1\_r/} & |
338 |
\hspace*{4ex} is related to {\it TR1.h} \\ |
339 |
\hspace*{4ex} {\tt common /adffields/} & |
340 |
\hspace*{4ex} is related to {\it FFIELDS.h}\\ |
341 |
\end{tabular} |
342 |
|
343 |
Note that if the structure of the common block changes in the |
344 |
above header files of the forward code, the structure |
345 |
of the adjoint common blocks will change accordingly. |
346 |
Thus, it has to be made sure that the structure of the |
347 |
adjoint common block in the hand-written file {\it adcommon.h} |
348 |
complies with the automatically generated adjoint common blocks |
349 |
in {\it adjoint\_model.F}. |
350 |
The header file is enabled via the CPP-option |
351 |
{\bf ALLOW\_AUTODIFF\_MONITOR}. |
352 |
|
353 |
\subsubsection{File {\it tamc.h}} |
354 |
|
355 |
This routine contains the dimensions for TAMC checkpointing |
356 |
and some indices relevant for storing ky computations. |
357 |
% |
358 |
\begin{itemize} |
359 |
% |
360 |
\item {\tt \#ifdef ALLOW\_TAMC\_CHECKPOINTING} \\ |
361 |
3-level checkpointing is enabled, i.e. the timestepping |
362 |
is divided into three different levels (see Section \ref{???}). |
363 |
The model state of the outermost ({\tt nchklev\_3}) and the |
364 |
intermediate ({\tt nchklev\_2}) timestepping loop are stored to file |
365 |
(handled in {\it the\_main\_loop}). |
366 |
The innermost loop ({\tt nchklev\_1}) |
367 |
avoids I/O by storing all required variables |
368 |
to common blocks. This storing may also be necessary if |
369 |
no checkpointing is chosen |
370 |
(nonlinear functions, if-statements, iterative loops, ...). |
371 |
In the present example the dimensions are chosen as follows: \\ |
372 |
\hspace*{4ex} {\tt nchklev\_1 = 36 } \\ |
373 |
\hspace*{4ex} {\tt nchklev\_2 = 30 } \\ |
374 |
\hspace*{4ex} {\tt nchklev\_3 = 60 } \\ |
375 |
To guarantee that the checkpointing intervals span the entire |
376 |
integration period the following relation must be satisfied: \\ |
377 |
\hspace*{4ex} {\tt nchklev\_1*nchklev\_2*nchklev\_3 $ \ge $ nTimeSteps} \\ |
378 |
where {\tt nTimeSteps} is either specified in {\it data} |
379 |
or computed via \\ |
380 |
\hspace*{4ex} {\tt nTimeSteps = (endTime-startTime)/deltaTClock }. |
381 |
% |
382 |
\item {\tt \#undef ALLOW\_TAMC\_CHECKPOINTING} \\ |
383 |
No checkpointing is enabled. |
384 |
In this case the relevant counter is {\tt nchklev\_0}. |
385 |
Similar to above, the following relation has to be satisfied \\ |
386 |
\hspace*{4ex} {\tt nchklev\_0 $ \ge $ nTimeSteps}. |
387 |
% |
388 |
\end{itemize} |
389 |
|
390 |
The following parameters may be worth describing: \\ |
391 |
% |
392 |
\hspace*{4ex} {\tt isbyte} \\ |
393 |
\hspace*{4ex} {\tt maxpass} \\ |
394 |
~ |
395 |
|
396 |
\subsubsection{File {\it makefile}} |
397 |
|
398 |
This file contains all relevant parameter flags and |
399 |
lists to run TAMC or TAF. |
400 |
It is assumed that TAMC is available to you, either locally, |
401 |
being installed on your network, or remotely through the 'TAMC Utility'. |
402 |
TAMC is called with the command {\tt tamc} followed by a |
403 |
number of options. They are described in detail in the |
404 |
TAMC manual \cite{gie:99}. |
405 |
Here we briefly discuss the main flags used in the {\it makefile} |
406 |
% |
407 |
\begin{itemize} |
408 |
\item [{\tt tamc}] {\tt |
409 |
-input <variable names> |
410 |
-output <variable name> -i4 -r4 ... \\ |
411 |
-toplevel <S/R name> -reverse <file names> |
412 |
} |
413 |
\item [{\tt taf}] {\tt |
414 |
-input <variable names> |
415 |
-output <variable name> -i4 -r4 ... \\ |
416 |
-toplevel <S/R name> -reverse <file names> \\ |
417 |
-nonew_arg -flow taf_flow.log |
418 |
} |
419 |
\end{itemize} |
420 |
% |
421 |
\begin{itemize} |
422 |
% |
423 |
\item {\tt -toplevel <S/R name>} \\ |
424 |
Name of the toplevel routine, with respect to which the |
425 |
control flow analysis is performed. |
426 |
% |
427 |
\item {\tt -input <variable names>} \\ |
428 |
List of independent variables $ u $ with respect to which the |
429 |
dependent variable $ J $ is differentiated. |
430 |
% |
431 |
\item {\tt -output <variable name>} \\ |
432 |
Dependent variable $ J $ which is to be differentiated. |
433 |
% |
434 |
\item {\tt -reverse <file names>} \\ |
435 |
Adjoint code is generated to compute the sensitivity of an |
436 |
independent variable w.r.t. many dependent variables. |
437 |
In the discussion of Section ??? |
438 |
the generated adjoint top-level routine computes the product |
439 |
of the transposed Jacobian matrix $ M^T $ times |
440 |
the gradient vector $ \nabla_v J $. |
441 |
\\ |
442 |
{\tt <file names>} refers to the list of files {\it .f} which are to be |
443 |
analyzed by TAMC. This list is generally smaller than the full list |
444 |
of code to be compiled. The files not contained are either |
445 |
above the top-level routine (some initializations), or are |
446 |
deliberately hidden from TAMC, either because hand-written |
447 |
adjoint routines exist, or the routines must not (or don't have to) |
448 |
be differentiated. For each routine which is part of the flow tree |
449 |
of the top-level routine, but deliberately hidden from TAMC |
450 |
(or for each package which contains such routines), |
451 |
a corresponding file {\it .flow} exists containing flow directives |
452 |
for TAMC. |
453 |
% |
454 |
\item {\tt -i4 -r4} \\ |
455 |
~ |
456 |
% |
457 |
\item {\tt -flow taf_flow.log} \\ |
458 |
Will cause TAF to produce a flow listing file in which |
459 |
the set of active and passive variables are identified |
460 |
for each subroutine. |
461 |
% |
462 |
\item {\tt -nonew_arg} \\ |
463 |
The default in the order of the parameter list of |
464 |
adjoint routines has changed. |
465 |
Before TAF 1.3 the default was compatible with the |
466 |
TAMC-generated list. As of TAF 1.3 the order of adjoint |
467 |
routine parameter lists is no longer copatible with TAMC. |
468 |
To restore compatibility when using TAF 1.3 and higher, |
469 |
this argument is needed. |
470 |
It is currently crucial to use since all hand-written |
471 |
adjoint routines refer to the TAMC default. |
472 |
% |
473 |
\end{itemize} |
474 |
|
475 |
|
476 |
\subsubsection{The input parameter files} |
477 |
|
478 |
\paragraph{File {\it data}} |
479 |
|
480 |
\paragraph{File {\it data.cost}} |
481 |
|
482 |
\paragraph{File {\it data.ctrl}} |
483 |
|
484 |
\paragraph{File {\it data.gmredi}} |
485 |
|
486 |
\paragraph{File {\it data.grdchk}} |
487 |
|
488 |
\paragraph{File {\it data.optim}} |
489 |
|
490 |
\paragraph{File {\it data.pkg}} |
491 |
|
492 |
\paragraph{File {\it eedata}} |
493 |
|
494 |
\paragraph{File {\it topog.bin}} |
495 |
|
496 |
\paragraph{File {\it windx.bin, windy.bin}} |
497 |
|
498 |
\paragraph{File {\it salt.bin, theta.bin}} |
499 |
|
500 |
\paragraph{File {\it SSS.bin, SST.bin}} |
501 |
|
502 |
\paragraph{File {\it pickup*}} |
503 |
|
504 |
\subsection{Compiling the model and its adjoint} |
505 |
|
506 |
The built process of the adjoint model is slightly more |
507 |
complex than that of compiling the forward code. |
508 |
The main reason is that the adjoint code generation requires |
509 |
a specific list of routines that are to be differentiated |
510 |
(as opposed to the automatic generation of a list of |
511 |
files to be compiled by genmake). |
512 |
This list excludes routines that don't have to be or must not be |
513 |
differentiated. For some of the latter routines flow directives |
514 |
may be necessary, a list of which has to be given as well. |
515 |
For this reason, a separate {\it makefile} is currently |
516 |
maintained in the directory {\tt adjoint/}. This |
517 |
makefile is responsible for the adjoint code generation. |
518 |
|
519 |
In the following we describe the build process step by step, |
520 |
assuming you are in the directory {\tt bin/}. |
521 |
A summary of steps to follow is given at the end. |
522 |
|
523 |
\paragraph{Adjoint code generation and compilation -- step by step} |
524 |
|
525 |
\begin{enumerate} |
526 |
% |
527 |
\item |
528 |
{\tt ln -s ../verification/???/code/.genmakerc .} \\ |
529 |
{\tt ln -s ../verification/???/code/*.[Fh] .} \\ |
530 |
Link your customized genmake options, header files, |
531 |
and modified code to the compile directory. |
532 |
% |
533 |
\item |
534 |
{\tt ../tools/genmake -makefile} \\ |
535 |
Generate your Makefile (cf. Section ???). |
536 |
% |
537 |
\item |
538 |
{\tt make depend} \\ |
539 |
Dependency analysis for the CPP pre-compiler (cf. Section ???). |
540 |
% |
541 |
\item |
542 |
{\tt make small\_f} \\ |
543 |
This is the first difference between forward code compilation |
544 |
and adjoint code generation and compilation. |
545 |
Instead of going through the entire compilation process |
546 |
(CPP precompiling -- {\tt .f}, object code generation -- {\tt .o}, |
547 |
linking of object files and libraries to generate executable), |
548 |
only the CPP compiler is invoked at this stage to generate |
549 |
the {\tt .f} files. |
550 |
% |
551 |
\item |
552 |
{\tt cd ../adjoint} \\ |
553 |
{\tt make adtaf} or {\tt make adtamc} \\ |
554 |
Depending on whether you have TAF or TAMC at your disposal, |
555 |
you'll choose {\tt adtaf} or {\tt adtamc} as your |
556 |
make target for the {\it makefile} in the directory {\tt adjoint/}. |
557 |
Several things happen at this stage. |
558 |
% |
559 |
\begin{enumerate} |
560 |
% |
561 |
\item |
562 |
The initial template file {\it adjoint\_model.F} which is part |
563 |
of the compiling list created by {\it genmake} is restored. |
564 |
% |
565 |
\item |
566 |
All Fortran routines {\tt *.f} in {\tt bin/} are |
567 |
concatenated into a single file (it's current name is |
568 |
{\it tamc\_code.f}). |
569 |
% |
570 |
\item |
571 |
Adjoint code is generated by TAMC or TAF. |
572 |
The adjoint code is written to the file {\it tamc\_code\_ad.f}. |
573 |
It contains all adjoint routines of the forward routines |
574 |
concatenated in {\it tamc\_code.f}. |
575 |
For a given forward routines {\tt subroutine routinename} |
576 |
the adjoint routine is named {\tt adsubroutine routinename} |
577 |
by default (that default can be changed via the flag |
578 |
{\tt -admark <markname>}). |
579 |
Furthermore, it may contain modified code which |
580 |
incorporates the translation of adjoint store directives |
581 |
into specific Fortran code. |
582 |
For a given forward routines {\tt subroutine routinename} |
583 |
the modified routine is named {\tt mdsubroutine routinename}. |
584 |
TAMC or TAF info is written to file |
585 |
{\it tamc\_code.prot} or {\it taf.log}, respectively. |
586 |
% |
587 |
\end{enumerate} |
588 |
% |
589 |
\item |
590 |
{\tt make adchange} \\ |
591 |
The multi-threading capability of the MITGCM requires a slight |
592 |
change in the parameter list of some routines that are related to |
593 |
to active file handling. |
594 |
This post-processing invokes the sed script {\it adjoint\_ecco\_sed.com} |
595 |
to insert the threading counter {\bf myThId} into the parameter list |
596 |
of those subroutines. |
597 |
The resulting code is written to file {\it tamc\_code\_sed\_ad.f} |
598 |
and appended to the file {\it adjoint\_model.F}. |
599 |
This concludes the adjoint code generation. |
600 |
% |
601 |
\item |
602 |
{\tt cd ../bin} \\ |
603 |
{\tt make} \\ |
604 |
The file {\it adjoint\_model.F} now contains the full adjoint code. |
605 |
All routines are now compiled. |
606 |
% |
607 |
\end{enumerate} |
608 |
|
609 |
\paragraph{Adjoint code generation and compilation -- summary} |
610 |
~ \\ |
611 |
|
612 |
\[ |
613 |
\boxed{ |
614 |
\begin{split} |
615 |
~ & \mbox{\tt cd bin} \\ |
616 |
~ & \mbox{\tt ln -s ../verification/my\_experiment/code/.genmakerc .} \\ |
617 |
~ & \mbox{\tt ln -s ../verification/my\_experiment/code/*.[Fh] .} \\ |
618 |
~ & \mbox{\tt ../tools/genmake -makefile} \\ |
619 |
~ & \mbox{\tt make depend} \\ |
620 |
~ & \mbox{\tt make small\_f} \\ |
621 |
~ & \mbox{\tt cd ../adjoint} \\ |
622 |
~ & \mbox{\tt make adtaf <OR: make adtamc>} \\ |
623 |
~ & \mbox{\tt make adchange} \\ |
624 |
~ & \mbox{\tt cd ../bin} \\ |
625 |
~ & \mbox{\tt make} \\ |
626 |
\end{split} |
627 |
} |
628 |
\] |
629 |
|