1 |
jmc |
1.3 |
% $Header: /u/gcmpack/manual/s_examples/tracer_adjsens/doc_ad_examples.tex,v 1.2 2008/01/15 20:16:44 jmc Exp $ |
2 |
molod |
1.1 |
% $Name: $ |
3 |
|
|
|
4 |
|
|
%********************************************************************** |
5 |
|
|
\section{Sensitivity of Air-Sea Exchange to Tracer Injection Site } |
6 |
jmc |
1.3 |
%\label{www:tutorials} |
7 |
|
|
\label{sec:eg-simple-tracer-adjoint} |
8 |
molod |
1.1 |
\label{sec_ad_setup_ex} |
9 |
jmc |
1.3 |
\label{sec:tutorialIII} |
10 |
molod |
1.1 |
\begin{rawhtml} |
11 |
|
|
<!-- CMIREDIR:sec_ad_setup_ex: --> |
12 |
|
|
\end{rawhtml} |
13 |
jmc |
1.2 |
\begin{center} |
14 |
|
|
(in directory: {\it verification/tutorial\_tracer\_adjsens/}) |
15 |
|
|
\end{center} |
16 |
molod |
1.1 |
%********************************************************************** |
17 |
jmc |
1.3 |
\label{ask_the_author:tracer_adjsens:doc_ad_examples} |
18 |
molod |
1.1 |
|
19 |
|
|
MITgcm has been adapted to enable AD using TAMC or TAF. |
20 |
|
|
The present description, therefore, is specific to the |
21 |
|
|
use of TAMC or TAF as AD tool. |
22 |
|
|
The following sections describe the steps which are necessary to |
23 |
|
|
generate a tangent linear or adjoint model of MITgcm. |
24 |
|
|
We take as an example the sensitivity of carbon sequestration |
25 |
|
|
in the ocean. |
26 |
|
|
The AD-relevant hooks in the code are sketched in |
27 |
|
|
\ref{fig:adthemodel}, \ref{fig:adthemain}. |
28 |
|
|
|
29 |
|
|
\subsection{Overview of the experiment} |
30 |
jmc |
1.3 |
%\label{www:tutorials} |
31 |
molod |
1.1 |
|
32 |
|
|
We describe an adjoint sensitivity analysis of out-gassing from |
33 |
|
|
the ocean into the atmosphere of a carbon-like tracer injected |
34 |
|
|
into the ocean interior (see \cite{hil-eta:01}). |
35 |
|
|
|
36 |
|
|
\subsubsection{Passive tracer equation} |
37 |
jmc |
1.3 |
%\label{www:tutorials} |
38 |
molod |
1.1 |
|
39 |
|
|
For this work MITgcm was augmented with a thermodynamically |
40 |
|
|
inactive tracer, $C$. Tracer residing in the ocean |
41 |
|
|
model surface layer is out-gassed according to a relaxation time scale, |
42 |
|
|
$\mu$. Within the ocean interior, the tracer is passively advected |
43 |
|
|
by the ocean model currents. The full equation for the time evolution |
44 |
|
|
% |
45 |
|
|
\begin{equation} |
46 |
|
|
\label{carbon_ddt} |
47 |
|
|
\frac{\partial C}{\partial t} \, = \, |
48 |
|
|
-U\cdot \nabla C \, - \, \mu C \, + \, \Gamma(C) \,+ \, S |
49 |
|
|
\end{equation} |
50 |
|
|
% |
51 |
|
|
also includes a source term $S$. This term |
52 |
|
|
represents interior sources of $C$ such as would arise due to |
53 |
|
|
direct injection. |
54 |
|
|
The velocity term, $U$, is the sum of the |
55 |
|
|
model Eulerian circulation and an eddy-induced velocity, the latter |
56 |
|
|
parameterized according to Gent/McWilliams |
57 |
|
|
(\cite{gen-mcw:90, gen-eta:95}). |
58 |
|
|
The convection function, $\Gamma$, mixes $C$ vertically wherever the |
59 |
|
|
fluid is locally statically unstable. |
60 |
|
|
|
61 |
|
|
The out-gassing time scale, $\mu$, in eqn. (\ref{carbon_ddt}) |
62 |
|
|
is set so that \( 1/\mu \sim 1 \ \mathrm{year} \) for the surface |
63 |
|
|
ocean and $\mu=0$ elsewhere. With this value, eqn. (\ref{carbon_ddt}) |
64 |
|
|
is valid as a prognostic equation for small perturbations in oceanic |
65 |
|
|
carbon concentrations. This configuration provides a |
66 |
|
|
powerful tool for examining the impact of large-scale ocean circulation |
67 |
|
|
on $ CO_2 $ out-gassing due to interior injections. |
68 |
|
|
As source we choose a constant in time injection of |
69 |
|
|
$ S = 1 \,\, {\rm mol / s}$. |
70 |
|
|
|
71 |
|
|
\subsubsection{Model configuration} |
72 |
jmc |
1.3 |
%\label{www:tutorials} |
73 |
molod |
1.1 |
|
74 |
|
|
The model configuration employed has a constant |
75 |
|
|
$4^\circ \times 4^\circ$ resolution horizontal grid and realistic |
76 |
|
|
geography and bathymetry. Twenty vertical layers are used with |
77 |
|
|
vertical spacing ranging |
78 |
|
|
from 50 m near the surface to 815 m at depth. |
79 |
|
|
Driven to steady-state by climatological wind-stress, heat and |
80 |
|
|
fresh-water forcing the model reproduces well known large-scale |
81 |
|
|
features of the ocean general circulation. |
82 |
|
|
|
83 |
|
|
\subsubsection{Out-gassing cost function} |
84 |
jmc |
1.3 |
%\label{www:tutorials} |
85 |
molod |
1.1 |
|
86 |
|
|
To quantify and understand out-gassing due to injections of $C$ |
87 |
|
|
in eqn. (\ref{carbon_ddt}), |
88 |
|
|
we define a cost function $ {\cal J} $ that measures the total amount of |
89 |
|
|
tracer out-gassed at each timestep: |
90 |
|
|
% |
91 |
|
|
\begin{equation} |
92 |
|
|
\label{cost_tracer} |
93 |
|
|
{\cal J}(t=T)=\int_{t=0}^{t=T}\int_{A} \mu C \, dA \, dt |
94 |
|
|
\end{equation} |
95 |
|
|
% |
96 |
|
|
Equation(\ref{cost_tracer}) integrates the out-gassing term, $\mu C$, |
97 |
|
|
from (\ref{carbon_ddt}) |
98 |
|
|
over the entire ocean surface area, $A$, and accumulates it |
99 |
|
|
up to time $T$. |
100 |
|
|
Physically, ${\cal J}$ can be thought of as representing the amount of |
101 |
|
|
$CO_2$ that our model predicts would be out-gassed following an |
102 |
|
|
injection at rate $S$. |
103 |
|
|
The sensitivity of ${\cal J}$ to the spatial location of $S$, |
104 |
|
|
$\frac{\partial {\cal J}}{\partial S}$, |
105 |
|
|
can be used to identify regions from which circulation |
106 |
|
|
would cause $CO_2$ to rapidly out-gas following injection |
107 |
|
|
and regions in which $CO_2$ injections would remain effectively |
108 |
|
|
sequestered within the ocean. |
109 |
|
|
|
110 |
|
|
\subsection{Code configuration} |
111 |
jmc |
1.3 |
%\label{www:tutorials} |
112 |
molod |
1.1 |
|
113 |
|
|
The model configuration for this experiment resides under the |
114 |
|
|
directory {\it verification/carbon/}. |
115 |
|
|
The code customization routines are in {\it verification/carbon/code/}: |
116 |
|
|
% |
117 |
|
|
\begin{itemize} |
118 |
|
|
% |
119 |
|
|
\item {\it .genmakerc} |
120 |
|
|
% |
121 |
|
|
\item {\it COST\_CPPOPTIONS.h} |
122 |
|
|
% |
123 |
|
|
\item {\it CPP\_EEOPTIONS.h} |
124 |
|
|
% |
125 |
|
|
\item {\it CPP\_OPTIONS.h} |
126 |
|
|
% |
127 |
|
|
\item {\it CTRL\_OPTIONS.h} |
128 |
|
|
% |
129 |
|
|
\item {\it ECCO\_OPTIONS.h} |
130 |
|
|
% |
131 |
|
|
\item {\it SIZE.h} |
132 |
|
|
% |
133 |
|
|
\item {\it adcommon.h} |
134 |
|
|
% |
135 |
|
|
\item {\it tamc.h} |
136 |
|
|
% |
137 |
|
|
\end{itemize} |
138 |
|
|
% |
139 |
|
|
The runtime flag and parameters settings are contained in |
140 |
|
|
{\it verification/carbon/input/}, |
141 |
|
|
together with the forcing fields and and restart files: |
142 |
|
|
% |
143 |
|
|
\begin{itemize} |
144 |
|
|
% |
145 |
|
|
\item {\it data} |
146 |
|
|
% |
147 |
|
|
\item {\it data.cost} |
148 |
|
|
% |
149 |
|
|
\item {\it data.ctrl} |
150 |
|
|
% |
151 |
|
|
\item {\it data.gmredi} |
152 |
|
|
% |
153 |
|
|
\item {\it data.grdchk} |
154 |
|
|
% |
155 |
|
|
\item {\it data.optim} |
156 |
|
|
% |
157 |
|
|
\item {\it data.pkg} |
158 |
|
|
% |
159 |
|
|
\item {\it eedata} |
160 |
|
|
% |
161 |
|
|
\item {\it topog.bin} |
162 |
|
|
% |
163 |
|
|
\item {\it windx.bin, windy.bin} |
164 |
|
|
% |
165 |
|
|
\item {\it salt.bin, theta.bin} |
166 |
|
|
% |
167 |
|
|
\item {\it SSS.bin, SST.bin} |
168 |
|
|
% |
169 |
|
|
\item {\it pickup*} |
170 |
|
|
% |
171 |
|
|
\end{itemize} |
172 |
|
|
% |
173 |
|
|
Finally, the file to generate the adjoint code resides in |
174 |
|
|
$ adjoint/ $: |
175 |
|
|
% |
176 |
|
|
\begin{itemize} |
177 |
|
|
% |
178 |
|
|
\item {\it makefile} |
179 |
|
|
% |
180 |
|
|
\end{itemize} |
181 |
|
|
% |
182 |
|
|
|
183 |
|
|
Below we describe the customizations of this files which are |
184 |
|
|
specific to this experiment. |
185 |
|
|
|
186 |
|
|
\subsubsection{File {\it .genmakerc}} |
187 |
jmc |
1.3 |
%\label{www:tutorials} |
188 |
molod |
1.1 |
This file overwrites default settings of {\it genmake}. |
189 |
|
|
In the present example it is used to switch on the following |
190 |
|
|
packages which are related to automatic differentiation |
191 |
|
|
and are disabled by default: \\ |
192 |
|
|
\hspace*{4ex} {\tt set ENABLE=( autodiff cost ctrl ecco gmredi grdchk kpp )} \\ |
193 |
|
|
Other packages which are not needed are switched off: \\ |
194 |
|
|
\hspace*{4ex} {\tt set DISABLE=( aim obcs zonal\_filt shap\_filt cal exf )} |
195 |
|
|
|
196 |
|
|
\subsubsection{File {\it COST\_CPPOPTIONS.h, CTRL\_OPTIONS.h}} |
197 |
jmc |
1.3 |
%\label{www:tutorials} |
198 |
molod |
1.1 |
|
199 |
|
|
These files used to contain package-specific CPP-options |
200 |
jmc |
1.3 |
(see Section \ref{ask_the_author:tracer_adjsens:doc_ad_examples}). |
201 |
molod |
1.1 |
For technical reasons those options have been grouped together |
202 |
|
|
in the file {\it ECCO\_OPTIONS.h}. |
203 |
|
|
To retain the modularity, the files have been kept and contain |
204 |
|
|
the standard include of the {\it CPP\_OPTIONS.h} file. |
205 |
|
|
|
206 |
|
|
\subsubsection{File {\it CPP\_EEOPTIONS.h}} |
207 |
jmc |
1.3 |
%\label{www:tutorials} |
208 |
molod |
1.1 |
|
209 |
|
|
This file contains 'wrapper'-specific CPP options. |
210 |
|
|
It only needs to be changed if the code is to be run |
211 |
jmc |
1.3 |
in a parallel environment (see Section \ref{ask_the_author:tracer_adjsens:doc_ad_examples}). |
212 |
molod |
1.1 |
|
213 |
|
|
\subsubsection{File {\it CPP\_OPTIONS.h}} |
214 |
jmc |
1.3 |
%\label{www:tutorials} |
215 |
molod |
1.1 |
|
216 |
|
|
This file contains model-specific CPP options |
217 |
jmc |
1.3 |
(see Section \ref{ask_the_author:tracer_adjsens:doc_ad_examples}). |
218 |
molod |
1.1 |
Most options are related to the forward model setup. |
219 |
|
|
They are identical to the global steady circulation setup of |
220 |
|
|
{\it verification/global\_ocean.90x40x15/}. |
221 |
|
|
The three options specific to this experiment are \\ |
222 |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_PASSIVE\_TRACER} \\ |
223 |
|
|
This flag enables the code to carry through the |
224 |
|
|
advection/diffusion of a passive tracer along the |
225 |
|
|
model integration. \\ |
226 |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_MIT\_ADJOINT\_RUN} \\ |
227 |
|
|
This flag enables the inclusion of some AD-related fields |
228 |
|
|
concerning initialization, link between control variables |
229 |
|
|
and forward model variables, and the call to the top-level |
230 |
|
|
forward/adjoint subroutine {\it adthe\_main\_loop} |
231 |
|
|
instead of {\it the\_main\_loop}. \\ |
232 |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_GRADIENT\_CHECK} \\ |
233 |
|
|
This flag enables the gradient check package. |
234 |
|
|
After computing the unperturbed cost function and its gradient, |
235 |
|
|
a series of computations are performed for which \\ |
236 |
|
|
$\bullet$ an element of the control vector is perturbed \\ |
237 |
|
|
$\bullet$ the cost function w.r.t. the perturbed element is |
238 |
|
|
computed \\ |
239 |
|
|
$\bullet$ the difference between the perturbed and unperturbed |
240 |
|
|
cost function is computed to compute the finite difference gradient \\ |
241 |
|
|
$\bullet$ the finite difference gradient is compared with the |
242 |
|
|
adjoint-generated gradient. |
243 |
|
|
The gradient check package is further described in Section ???. |
244 |
|
|
|
245 |
|
|
\subsubsection{File {\it ECCO\_OPTIONS.h}} |
246 |
jmc |
1.3 |
%\label{www:tutorials} |
247 |
molod |
1.1 |
|
248 |
|
|
The CPP options of several AD-related packages are grouped |
249 |
|
|
in this file: |
250 |
|
|
% |
251 |
|
|
\begin{itemize} |
252 |
|
|
% |
253 |
|
|
\item |
254 |
|
|
Overall ECCO-related execution modus: \\ |
255 |
|
|
These determine whether a pure forward run, |
256 |
|
|
a sensitivity run or an iteration of optimization is |
257 |
|
|
performed. These options are not needed in the present context. |
258 |
|
|
% |
259 |
|
|
\item |
260 |
|
|
Adjoint support package: {\it pkg/autodiff/} \\ |
261 |
|
|
This package contains hand-written adjoint code such as |
262 |
|
|
active file handling, flow directives for files which must not |
263 |
|
|
be differentiated, and TAMC-specific header files. \\ |
264 |
|
|
% |
265 |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_AUTODIFF\_TAMC} \\ |
266 |
|
|
defines TAMC-related features in the code. \\ |
267 |
|
|
% |
268 |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_TAMC\_CHECKPOINTING} \\ |
269 |
|
|
enables the checkpointing feature of TAMC |
270 |
jmc |
1.3 |
(see Section \ref{ask_the_author:tracer_adjsens:doc_ad_examples}). |
271 |
molod |
1.1 |
In the present example a 3-level checkpointing is implemented. |
272 |
|
|
The code contains the relevant store directives, common block |
273 |
|
|
and tape initializations, storing key computation, |
274 |
|
|
and loop index handling. |
275 |
|
|
The checkpointing length at each level is defined in |
276 |
|
|
file {\it tamc.h}, cf. below. |
277 |
|
|
The out and intermediate loop directivs are contained |
278 |
|
|
in the files {\it checkpoint\_lev3\_directives.h}, |
279 |
|
|
{\it checkpoint\_lev2\_directives.h} (package {\it pkg/autodiff}). \\ |
280 |
|
|
% |
281 |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_AUTODIFF\_MONITOOR} \\ |
282 |
|
|
enables the monitoring of intermediate adjoint variables |
283 |
jmc |
1.3 |
(see Section \ref{ask_the_author:tracer_adjsens:doc_ad_examples}). \\ |
284 |
molod |
1.1 |
% |
285 |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_DIVIDED\_ADJOINT} \\ |
286 |
|
|
enables adjoint dump and restart |
287 |
jmc |
1.3 |
(see Section \ref{ask_the_author:tracer_adjsens:doc_ad_examples}). |
288 |
molod |
1.1 |
% |
289 |
|
|
\item Cost function package: {\it pkg/cost/} \\ |
290 |
|
|
This package contains all relevant routines for |
291 |
|
|
initializing, accumulating and finalizing the cost function |
292 |
jmc |
1.3 |
(see Section \ref{ask_the_author:tracer_adjsens:doc_ad_examples}). \\ |
293 |
molod |
1.1 |
\hspace*{4ex} {\tt \#define ALLOW\_COST} \\ |
294 |
|
|
enables all general aspects of the cost function handling, |
295 |
|
|
in particular the hooks in the forward code for |
296 |
|
|
initializing, accumulating and finalizing the cost function. \\ |
297 |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_COST\_TRACER} \\ |
298 |
|
|
includes the call to the cost function for this |
299 |
|
|
particular experiment, eqn. (\ref{cost_tracer}). |
300 |
|
|
% |
301 |
|
|
\item Control variable package: {\it pkg/ctrl/} \\ |
302 |
|
|
This package contains all relevant routines for |
303 |
|
|
the handling of the control vector. |
304 |
|
|
Each control variable can be enabled/disabled with its own flag: \\ |
305 |
|
|
\begin{tabular}{ll} |
306 |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_THETA0\_CONTROL} & |
307 |
|
|
initial temperature \\ |
308 |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_SALT0\_CONTROL} & |
309 |
|
|
initial salinity \\ |
310 |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_TR0\_CONTROL} & |
311 |
|
|
initial passive tracer concentration \\ |
312 |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_TAUU0\_CONTROL} & |
313 |
|
|
zonal wind stress \\ |
314 |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_TAUV0\_CONTROL} & |
315 |
|
|
meridional wind stress \\ |
316 |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_SFLUX0\_CONTROL} & |
317 |
|
|
freshwater flux \\ |
318 |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_HFLUX0\_CONTROL} & |
319 |
|
|
heat flux \\ |
320 |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_DIFFKR\_CONTROL} & |
321 |
|
|
diapycnal diffusivity \\ |
322 |
|
|
\hspace*{2ex} {\tt \#undef ALLOW\_KAPPAGM\_CONTROL} & |
323 |
|
|
isopycnal diffusivity \\ |
324 |
|
|
\end{tabular} |
325 |
|
|
% |
326 |
|
|
\end{itemize} |
327 |
|
|
|
328 |
|
|
\subsubsection{File {\it SIZE.h}} |
329 |
jmc |
1.3 |
%\label{www:tutorials} |
330 |
molod |
1.1 |
|
331 |
|
|
The file contains the grid point dimensions of the forward |
332 |
|
|
model. It is identical to the {\it verification/exp2/}: \\ |
333 |
|
|
\hspace*{4ex} {\tt sNx = 90} \\ |
334 |
|
|
\hspace*{4ex} {\tt sNy = 40} \\ |
335 |
|
|
\hspace*{4ex} {\tt Nr = 20} \\ |
336 |
|
|
It corresponds to a single-tile/single-processor setup: |
337 |
|
|
{\tt nSx = nSy = 1, nPx = nPy = 1}, |
338 |
|
|
with standard overlap dimensioning |
339 |
|
|
{\tt OLx = OLy = 3}. |
340 |
|
|
|
341 |
|
|
\subsubsection{File {\it adcommon.h}} |
342 |
jmc |
1.3 |
%\label{www:tutorials} |
343 |
molod |
1.1 |
|
344 |
|
|
This file contains common blocks of some adjoint variables |
345 |
|
|
that are generated by TAMC. |
346 |
|
|
The common blocks are used by the adjoint support routine |
347 |
|
|
{\it addummy\_in\_stepping} which needs to access those variables: |
348 |
|
|
|
349 |
|
|
\begin{tabular}{ll} |
350 |
|
|
\hspace*{4ex} {\tt common /addynvars\_r/} & |
351 |
|
|
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
352 |
|
|
\hspace*{4ex} {\tt common /addynvars\_cd/} & |
353 |
|
|
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
354 |
|
|
\hspace*{4ex} {\tt common /addynvars\_diffkr/} & |
355 |
|
|
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
356 |
|
|
\hspace*{4ex} {\tt common /addynvars\_kapgm/} & |
357 |
|
|
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
358 |
|
|
\hspace*{4ex} {\tt common /adtr1\_r/} & |
359 |
|
|
\hspace*{4ex} is related to {\it TR1.h} \\ |
360 |
|
|
\hspace*{4ex} {\tt common /adffields/} & |
361 |
|
|
\hspace*{4ex} is related to {\it FFIELDS.h}\\ |
362 |
|
|
\end{tabular} |
363 |
|
|
|
364 |
|
|
Note that if the structure of the common block changes in the |
365 |
|
|
above header files of the forward code, the structure |
366 |
|
|
of the adjoint common blocks will change accordingly. |
367 |
|
|
Thus, it has to be made sure that the structure of the |
368 |
|
|
adjoint common block in the hand-written file {\it adcommon.h} |
369 |
|
|
complies with the automatically generated adjoint common blocks |
370 |
|
|
in {\it adjoint\_model.F}. |
371 |
|
|
The header file is enabled via the CPP-option |
372 |
|
|
{\bf ALLOW\_AUTODIFF\_MONITOR}. |
373 |
|
|
|
374 |
|
|
\subsubsection{File {\it tamc.h}} |
375 |
jmc |
1.3 |
%\label{www:tutorials} |
376 |
molod |
1.1 |
|
377 |
|
|
This routine contains the dimensions for TAMC checkpointing |
378 |
|
|
and some indices relevant for storing ky computations. |
379 |
|
|
% |
380 |
|
|
\begin{itemize} |
381 |
|
|
% |
382 |
|
|
\item {\tt \#ifdef ALLOW\_TAMC\_CHECKPOINTING} \\ |
383 |
|
|
3-level checkpointing is enabled, i.e. the timestepping |
384 |
jmc |
1.3 |
is divided into three different levels (see Section \ref{ask_the_author:tracer_adjsens:doc_ad_examples}). |
385 |
molod |
1.1 |
The model state of the outermost ({\tt nchklev\_3}) and the |
386 |
|
|
intermediate ({\tt nchklev\_2}) timestepping loop are stored to file |
387 |
|
|
(handled in {\it the\_main\_loop}). |
388 |
|
|
The innermost loop ({\tt nchklev\_1}) |
389 |
|
|
avoids I/O by storing all required variables |
390 |
|
|
to common blocks. This storing may also be necessary if |
391 |
|
|
no checkpointing is chosen |
392 |
|
|
(nonlinear functions, if-statements, iterative loops, ...). |
393 |
|
|
In the present example the dimensions are chosen as follows: \\ |
394 |
|
|
\hspace*{4ex} {\tt nchklev\_1 = 36 } \\ |
395 |
|
|
\hspace*{4ex} {\tt nchklev\_2 = 30 } \\ |
396 |
|
|
\hspace*{4ex} {\tt nchklev\_3 = 60 } \\ |
397 |
|
|
To guarantee that the checkpointing intervals span the entire |
398 |
|
|
integration period the following relation must be satisfied: \\ |
399 |
|
|
\hspace*{4ex} {\tt nchklev\_1*nchklev\_2*nchklev\_3 $ \ge $ nTimeSteps} \\ |
400 |
|
|
where {\tt nTimeSteps} is either specified in {\it data} |
401 |
|
|
or computed via \\ |
402 |
|
|
\hspace*{4ex} {\tt nTimeSteps = (endTime-startTime)/deltaTClock }. |
403 |
|
|
% |
404 |
|
|
\item {\tt \#undef ALLOW\_TAMC\_CHECKPOINTING} \\ |
405 |
|
|
No checkpointing is enabled. |
406 |
|
|
In this case the relevant counter is {\tt nchklev\_0}. |
407 |
|
|
Similar to above, the following relation has to be satisfied \\ |
408 |
|
|
\hspace*{4ex} {\tt nchklev\_0 $ \ge $ nTimeSteps}. |
409 |
|
|
% |
410 |
|
|
\end{itemize} |
411 |
|
|
|
412 |
|
|
The following parameters may be worth describing: \\ |
413 |
|
|
% |
414 |
|
|
\hspace*{4ex} {\tt isbyte} \\ |
415 |
|
|
\hspace*{4ex} {\tt maxpass} \\ |
416 |
|
|
~ |
417 |
|
|
|
418 |
|
|
\subsubsection{File {\it makefile}} |
419 |
jmc |
1.3 |
%\label{www:tutorials} |
420 |
molod |
1.1 |
|
421 |
|
|
This file contains all relevant parameter flags and |
422 |
|
|
lists to run TAMC or TAF. |
423 |
|
|
It is assumed that TAMC is available to you, either locally, |
424 |
|
|
being installed on your network, or remotely through the 'TAMC Utility'. |
425 |
|
|
TAMC is called with the command {\tt tamc} followed by a |
426 |
|
|
number of options. They are described in detail in the |
427 |
|
|
TAMC manual \cite{gie:99}. |
428 |
|
|
Here we briefly discuss the main flags used in the {\it makefile}. |
429 |
|
|
The standard output for TAF is written to file |
430 |
|
|
{\it taf.log}. |
431 |
|
|
% |
432 |
|
|
\begin{itemize} |
433 |
|
|
\item [{\tt tamc}] {\tt |
434 |
|
|
-input <variable names> |
435 |
|
|
-output <variable name> -i4 -r4 ... \\ |
436 |
|
|
-toplevel <S/R name> -reverse <file names> |
437 |
|
|
} |
438 |
|
|
\item [{\tt taf}] {\tt |
439 |
|
|
-input <variable names> |
440 |
|
|
-output <variable name> -i4 -r4 ... \\ |
441 |
|
|
-toplevel <S/R name> -reverse <file names> \\ |
442 |
|
|
-flow taf\_flow.log -nonew\_arg |
443 |
|
|
} |
444 |
|
|
\end{itemize} |
445 |
|
|
% |
446 |
|
|
\begin{itemize} |
447 |
|
|
% |
448 |
|
|
\item {\tt -toplevel <S/R name>} \\ |
449 |
|
|
Name of the toplevel routine, with respect to which the |
450 |
|
|
control flow analysis is performed. |
451 |
|
|
% |
452 |
|
|
\item {\tt -input <variable names>} \\ |
453 |
|
|
List of independent variables $ u $ with respect to which the |
454 |
|
|
dependent variable $ J $ is differentiated. |
455 |
|
|
% |
456 |
|
|
\item {\tt -output <variable name>} \\ |
457 |
|
|
Dependent variable $ J $ which is to be differentiated. |
458 |
|
|
% |
459 |
|
|
\item {\tt -reverse <file names>} \\ |
460 |
|
|
Adjoint code is generated to compute the sensitivity of an |
461 |
|
|
independent variable w.r.t. many dependent variables. |
462 |
|
|
In the discussion of Section ??? |
463 |
|
|
the generated adjoint top-level routine computes the product |
464 |
|
|
of the transposed Jacobian matrix $ M^T $ times |
465 |
|
|
the gradient vector $ \nabla_v J $. |
466 |
|
|
\\ |
467 |
|
|
{\tt <file names>} refers to the list of files {\it .f} which are to be |
468 |
|
|
analyzed by TAMC. This list is generally smaller than the full list |
469 |
|
|
of code to be compiled. The files not contained are either |
470 |
|
|
above the top-level routine (some initializations), or are |
471 |
|
|
deliberately hidden from TAMC, either because hand-written |
472 |
|
|
adjoint routines exist, or the routines must not (or don't have to) |
473 |
|
|
be differentiated. For each routine which is part of the flow tree |
474 |
|
|
of the top-level routine, but deliberately hidden from TAMC |
475 |
|
|
(or for each package which contains such routines), |
476 |
|
|
a corresponding file {\it .flow} exists containing flow directives |
477 |
|
|
for TAMC. |
478 |
|
|
% |
479 |
|
|
\item {\tt -i4 -r4} \\ |
480 |
|
|
~ |
481 |
|
|
% |
482 |
|
|
\item {\tt -flow taf\_flow.log} \\ |
483 |
|
|
Will cause TAF to produce a flow listing file |
484 |
|
|
named {\it taf\_flow.log} in which |
485 |
|
|
the set of active and passive variables are identified |
486 |
|
|
for each subroutine. |
487 |
|
|
% |
488 |
|
|
\item {\tt -nonew\_arg} \\ |
489 |
|
|
The default in the order of the parameter list of |
490 |
|
|
adjoint routines has changed. |
491 |
|
|
Before TAF 1.3 the default was compatible with the |
492 |
|
|
TAMC-generated list. As of TAF 1.3 the order of adjoint |
493 |
|
|
routine parameter lists is no longer copatible with TAMC. |
494 |
|
|
To restore compatibility when using TAF 1.3 and higher, |
495 |
|
|
this argument is needed. |
496 |
|
|
It is currently crucial to use since all hand-written |
497 |
|
|
adjoint routines refer to the TAMC default. |
498 |
|
|
% |
499 |
|
|
\end{itemize} |
500 |
|
|
|
501 |
|
|
|
502 |
|
|
\subsubsection{The input parameter files} |
503 |
jmc |
1.3 |
%\label{www:tutorials} |
504 |
molod |
1.1 |
|
505 |
|
|
\paragraph{File {\it data}} |
506 |
|
|
|
507 |
|
|
\paragraph{File {\it data.cost}} |
508 |
|
|
|
509 |
|
|
\paragraph{File {\it data.ctrl}} |
510 |
|
|
|
511 |
|
|
\paragraph{File {\it data.gmredi}} |
512 |
|
|
|
513 |
|
|
\paragraph{File {\it data.grdchk}} |
514 |
|
|
|
515 |
|
|
\paragraph{File {\it data.optim}} |
516 |
|
|
|
517 |
|
|
\paragraph{File {\it data.pkg}} |
518 |
|
|
|
519 |
|
|
\paragraph{File {\it eedata}} |
520 |
|
|
|
521 |
|
|
\paragraph{File {\it topog.bin}} ~ \\ |
522 |
|
|
% |
523 |
|
|
Contains two-dimendional bathymetry information |
524 |
|
|
|
525 |
|
|
\paragraph{File {\it windx.bin, windy.bin, salt.bin, theta.bin, |
526 |
|
|
SSS.bin, SST.bin}} ~ \\ |
527 |
|
|
% |
528 |
|
|
These contain the initial values |
529 |
|
|
(salinity, temperature, {\it salt.bin, theta.bin}), |
530 |
|
|
surface boundary values (surface wind stresses, |
531 |
|
|
({\it windx.bin, windy.bin}), and surface restoring fields |
532 |
|
|
({\it SSS.bin, SST.bin}). |
533 |
|
|
|
534 |
|
|
\paragraph{File {\it pickup*}} ~ \\ |
535 |
|
|
% |
536 |
|
|
Contains model state after model spinup. |
537 |
|
|
|
538 |
|
|
\subsection{Compiling the model and its adjoint} |
539 |
jmc |
1.3 |
%\label{www:tutorials} |
540 |
molod |
1.1 |
|
541 |
|
|
The built process of the adjoint model is slightly more |
542 |
|
|
complex than that of compiling the forward code. |
543 |
|
|
The main reason is that the adjoint code generation requires |
544 |
|
|
a specific list of routines that are to be differentiated |
545 |
|
|
(as opposed to the automatic generation of a list of |
546 |
|
|
files to be compiled by genmake). |
547 |
|
|
This list excludes routines that don't have to be or must not be |
548 |
|
|
differentiated. For some of the latter routines flow directives |
549 |
|
|
may be necessary, a list of which has to be given as well. |
550 |
|
|
For this reason, a separate {\it makefile} is currently |
551 |
|
|
maintained in the directory {\tt adjoint/}. This |
552 |
|
|
makefile is responsible for the adjoint code generation. |
553 |
|
|
|
554 |
|
|
In the following we describe the build process step by step, |
555 |
|
|
assuming you are in the directory {\tt bin/}. |
556 |
|
|
A summary of steps to follow is given at the end. |
557 |
|
|
|
558 |
|
|
\paragraph{Adjoint code generation and compilation -- step by step} |
559 |
|
|
|
560 |
|
|
\begin{enumerate} |
561 |
|
|
% |
562 |
|
|
\item |
563 |
|
|
{\tt ln -s ../verification/???/code/.genmakerc .} \\ |
564 |
|
|
{\tt ln -s ../verification/???/code/*.[Fh] .} \\ |
565 |
|
|
Link your customized genmake options, header files, |
566 |
|
|
and modified code to the compile directory. |
567 |
|
|
% |
568 |
|
|
\item |
569 |
|
|
{\tt ../tools/genmake -makefile} \\ |
570 |
|
|
Generate your Makefile (cf. Section ???). |
571 |
|
|
% |
572 |
|
|
\item |
573 |
|
|
{\tt make depend} \\ |
574 |
|
|
Dependency analysis for the CPP pre-compiler (cf. Section ???). |
575 |
|
|
% |
576 |
|
|
\item |
577 |
|
|
{\tt cd ../adjoint} \\ |
578 |
|
|
{\tt make adtaf} or {\tt make adtamc} \\ |
579 |
|
|
Depending on whether you have TAF or TAMC at your disposal, |
580 |
|
|
you'll choose {\tt adtaf} or {\tt adtamc} as your |
581 |
|
|
make target for the {\it makefile} in the directory {\tt adjoint/}. |
582 |
|
|
Several things happen at this stage. |
583 |
|
|
% |
584 |
|
|
\begin{enumerate} |
585 |
|
|
% |
586 |
|
|
\item |
587 |
|
|
{\tt make adrestore, make ftlrestore} \\ |
588 |
|
|
The initial template files {\it adjoint\_model.F} |
589 |
|
|
and {\it tangentlinear\_model.F} in {\it pkg/autodiff} |
590 |
|
|
which are part |
591 |
|
|
of the compiling list created by {\it genmake} are restored. |
592 |
|
|
% |
593 |
|
|
\item {\tt make depend, make small\_f} \\ |
594 |
|
|
The {\tt bin/} directory is brought up to date, |
595 |
|
|
i.e. for recent changes in header or source code |
596 |
|
|
{\it .[Fh]}, corresponding {\it .f} routines are generated |
597 |
|
|
or re-generated. |
598 |
|
|
Note that here, only CPP precompiling is performed; |
599 |
|
|
no object code {\it .o} is generated as yet. |
600 |
|
|
Precompiling is necessary for TAMC to see the full code. |
601 |
|
|
% |
602 |
|
|
\item |
603 |
|
|
{\tt make allcode} \\ |
604 |
|
|
All Fortran routines {\tt *.f} in {\tt bin/} are |
605 |
|
|
concatenated into a single file called |
606 |
|
|
{\it tamc\_code.f}. |
607 |
|
|
% |
608 |
|
|
\item |
609 |
|
|
{\tt make admodeltaf/admodeltamc} \\ |
610 |
|
|
Adjoint code is generated by TAMC or TAF. |
611 |
|
|
The adjoint code is written to the file {\it tamc\_code\_ad.f}. |
612 |
|
|
It contains all adjoint routines of the forward routines |
613 |
|
|
concatenated in {\it tamc\_code.f}. |
614 |
|
|
For a given forward routines {\tt subroutine routinename} |
615 |
|
|
the adjoint routine is named {\tt adsubroutine routinename} |
616 |
|
|
by default (that default can be changed via the flag |
617 |
|
|
{\tt -admark <markname>}). |
618 |
|
|
Furthermore, it may contain modified code which |
619 |
|
|
incorporates the translation of adjoint store directives |
620 |
|
|
into specific Fortran code. |
621 |
|
|
For a given forward routines {\tt subroutine routinename} |
622 |
|
|
the modified routine is named {\tt mdsubroutine routinename}. |
623 |
|
|
TAMC or TAF info is written to file |
624 |
|
|
{\it tamc\_code.prot} or {\it taf.log}, respectively. |
625 |
|
|
% |
626 |
|
|
\item |
627 |
|
|
{\tt make adchange} \\ |
628 |
|
|
The multi-threading capability of MITgcm requires a slight |
629 |
|
|
change in the parameter list of some routines that are related to |
630 |
|
|
to active file handling. |
631 |
|
|
This post-processing invokes the sed script {\it adjoint\_ecco\_sed.com} |
632 |
|
|
to insert the threading counter {\bf myThId} into the parameter list |
633 |
|
|
of those subroutines. |
634 |
|
|
The resulting code is written to file {\it tamc\_code\_sed\_ad.f} |
635 |
|
|
and appended to the file {\it adjoint\_model.F}. |
636 |
|
|
This concludes the adjoint code generation. |
637 |
|
|
% |
638 |
|
|
\end{enumerate} |
639 |
|
|
% |
640 |
|
|
\item |
641 |
|
|
{\tt cd ../bin} \\ |
642 |
|
|
{\tt make} \\ |
643 |
|
|
The file {\it adjoint\_model.F} now contains the full adjoint code. |
644 |
|
|
All routines are now compiled. |
645 |
|
|
% |
646 |
|
|
\end{enumerate} |
647 |
|
|
|
648 |
|
|
N.B.: The targets {\tt make adtaf/adtamc} now comprise a |
649 |
|
|
series of targets that in previous versions had to be |
650 |
|
|
invoked separarely. This was probably preferable at a more |
651 |
|
|
experimental stage, but has now been dropped in favour of |
652 |
|
|
a more straightforward build process. |
653 |
|
|
|
654 |
|
|
|
655 |
|
|
\paragraph{Adjoint code generation and compilation -- summary} |
656 |
|
|
~ \\ |
657 |
|
|
|
658 |
|
|
{\small |
659 |
|
|
\[ |
660 |
|
|
\boxed{ |
661 |
jmc |
1.3 |
\begin{aligned} |
662 |
molod |
1.1 |
~ & \mbox{\tt cd bin} \\ |
663 |
|
|
~ & \mbox{\tt ln -s ../verification/my\_experiment/code/.genmakerc .} \\ |
664 |
|
|
~ & \mbox{\tt ln -s ../verification/my\_experiment/code/*.[Fh] .} \\ |
665 |
|
|
~ & \mbox{\tt ../tools/genmake -makefile} \\ |
666 |
|
|
~ & \mbox{\tt make depend} \\ |
667 |
|
|
~ & \mbox{\tt cd ../adjoint} \\ |
668 |
|
|
~ & \mbox{\tt make adtaf <OR: make adtamc>} \\ |
669 |
|
|
~ & \hspace*{6ex} \mbox{\tt contains the targets:} \\ |
670 |
|
|
~ & \hspace*{6ex} \mbox{\tt adrestore small\_f allcode admodeltaf/admodeltamc adchange} \\ |
671 |
|
|
~ & \mbox{\tt cd ../bin} \\ |
672 |
|
|
~ & \mbox{\tt make} \\ |
673 |
jmc |
1.3 |
\end{aligned} |
674 |
molod |
1.1 |
} |
675 |
|
|
\] |
676 |
|
|
} |