| 21 |
Tangent linear and Adjoint Model Compiler (TAMC) and its successor TAF |
Tangent linear and Adjoint Model Compiler (TAMC) and its successor TAF |
| 22 |
(Transformation of Algorithms in Fortran), developed |
(Transformation of Algorithms in Fortran), developed |
| 23 |
by Ralf Giering (\cite{gie-kam:98}, \cite{gie:99,gie:00}). |
by Ralf Giering (\cite{gie-kam:98}, \cite{gie:99,gie:00}). |
| 24 |
The first application of the adjoint of the MITGCM for senistivity |
The first application of the adjoint of the MITGCM for sensitivity |
| 25 |
studies has been published by \cite{maro-eta:99}. |
studies has been published by \cite{maro-eta:99}. |
| 26 |
\cite{sta-eta:97,sta-eta:01} use the MITGCM and its adjoint |
\cite{sta-eta:97,sta-eta:01} use the MITGCM and its adjoint |
| 27 |
for ocean state estimation studies. |
for ocean state estimation studies. |
| 52 |
such as forcing functions) to the $n$-dimensional space |
such as forcing functions) to the $n$-dimensional space |
| 53 |
$V \subset I\!\!R^n$ of |
$V \subset I\!\!R^n$ of |
| 54 |
model output variable $\vec{v}=(v_1,\ldots,v_n)$ |
model output variable $\vec{v}=(v_1,\ldots,v_n)$ |
| 55 |
(model state, model diagnostcs, objective function, ...) |
(model state, model diagnostics, objective function, ...) |
| 56 |
under consideration, |
under consideration, |
| 57 |
% |
% |
| 58 |
\begin{equation} |
\begin{equation} |
| 220 |
starting at step 0 and moving up to step $\Lambda$, with intermediate |
starting at step 0 and moving up to step $\Lambda$, with intermediate |
| 221 |
${\cal M}_{\lambda} (\vec{u}) = \vec{v}^{(\lambda+1)}$ and final |
${\cal M}_{\lambda} (\vec{u}) = \vec{v}^{(\lambda+1)}$ and final |
| 222 |
${\cal M}_{\Lambda} (\vec{u}) = \vec{v}^{(\Lambda+1)} = \vec{v}$. |
${\cal M}_{\Lambda} (\vec{u}) = \vec{v}^{(\Lambda+1)} = \vec{v}$. |
| 223 |
Let ${\cal J}$ be a cost funciton which explicitly depends on the |
Let ${\cal J}$ be a cost function which explicitly depends on the |
| 224 |
final state $\vec{v}$ only |
final state $\vec{v}$ only |
| 225 |
(this restriction is for clarity reasons only). |
(this restriction is for clarity reasons only). |
| 226 |
% |
% |
| 301 |
are the Lagrange multipliers of the model equations which determine |
are the Lagrange multipliers of the model equations which determine |
| 302 |
$ \vec{v}^{(\lambda)}$. |
$ \vec{v}^{(\lambda)}$. |
| 303 |
|
|
| 304 |
In coponents, eq. (\ref{adjoint}) reads as follows. |
In components, eq. (\ref{adjoint}) reads as follows. |
| 305 |
Let |
Let |
| 306 |
\[ |
\[ |
| 307 |
\begin{array}{rclcrcl} |
\begin{array}{rclcrcl} |
| 322 |
\end{array} |
\end{array} |
| 323 |
\] |
\] |
| 324 |
denote the perturbations in $\vec{u}$ and $\vec{v}$, respectively, |
denote the perturbations in $\vec{u}$ and $\vec{v}$, respectively, |
| 325 |
and their adjoint varaiables; |
and their adjoint variables; |
| 326 |
further |
further |
| 327 |
\[ |
\[ |
| 328 |
M \, = \, \left( |
M \, = \, \left( |
| 468 |
{\it all} intermediate states $ \vec{v}^{(\lambda)} $) are sought. |
{\it all} intermediate states $ \vec{v}^{(\lambda)} $) are sought. |
| 469 |
In order to be able to solve for each component of the gradient |
In order to be able to solve for each component of the gradient |
| 470 |
$ \partial {\cal J} / \partial u_{i} $ in (\ref{forward}) |
$ \partial {\cal J} / \partial u_{i} $ in (\ref{forward}) |
| 471 |
a forward calulation has to be performed for each component seperately, |
a forward calculation has to be performed for each component separately, |
| 472 |
i.e. $ \delta \vec{u} = \delta u_{i} {\vec{e}_{i}} $ |
i.e. $ \delta \vec{u} = \delta u_{i} {\vec{e}_{i}} $ |
| 473 |
for the $i$-th forward calculation. |
for the $i$-th forward calculation. |
| 474 |
Then, (\ref{forward}) represents the |
Then, (\ref{forward}) represents the |
| 487 |
\nabla_u {\cal J}^T \cdot \delta \vec{J} |
\nabla_u {\cal J}^T \cdot \delta \vec{J} |
| 488 |
\] |
\] |
| 489 |
where now $ \delta \vec{J} \in I\!\!R^l $ is a vector of |
where now $ \delta \vec{J} \in I\!\!R^l $ is a vector of |
| 490 |
dimenison $ l $. |
dimension $ l $. |
| 491 |
In this case $ l $ reverse simulations have to be performed |
In this case $ l $ reverse simulations have to be performed |
| 492 |
for each $ \delta J_{k}, \,\, k = 1, \ldots, l $. |
for each $ \delta J_{k}, \,\, k = 1, \ldots, l $. |
| 493 |
Then, the reverse mode is more efficient as long as |
Then, the reverse mode is more efficient as long as |
| 494 |
$ l < n $, otherwise the forward mode is preferable. |
$ l < n $, otherwise the forward mode is preferable. |
| 495 |
Stricly, the reverse mode is called adjoint mode only for |
Strictly, the reverse mode is called adjoint mode only for |
| 496 |
$ l = 1 $. |
$ l = 1 $. |
| 497 |
|
|
| 498 |
A detailed analysis of the underlying numerical operations |
A detailed analysis of the underlying numerical operations |
| 557 |
(a derivative is defined w.r.t. a point along the trajectory), |
(a derivative is defined w.r.t. a point along the trajectory), |
| 558 |
the intermediate results of the model trajectory |
the intermediate results of the model trajectory |
| 559 |
$\vec{v}^{(\lambda+1)}={\cal M}_{\lambda}(v^{(\lambda)})$ |
$\vec{v}^{(\lambda+1)}={\cal M}_{\lambda}(v^{(\lambda)})$ |
| 560 |
are needed to evaluate the intermediate Jacobian |
may be required to evaluate the intermediate Jacobian |
| 561 |
$M_{\lambda}|_{\vec{v}^{(\lambda)}} \, \delta \vec{v}^{(\lambda)} $. |
$M_{\lambda}|_{\vec{v}^{(\lambda)}} \, \delta \vec{v}^{(\lambda)} $. |
| 562 |
|
This is the case e.g. for nonlinear expressions |
| 563 |
|
(momentum advection, nonlinear equation of state), state-dependent |
| 564 |
|
conditional statements (parameterization schemes). |
| 565 |
In the forward mode, the intermediate results are required |
In the forward mode, the intermediate results are required |
| 566 |
in the same order as computed by the full forward model ${\cal M}$, |
in the same order as computed by the full forward model ${\cal M}$, |
| 567 |
but in the reverse mode they are required in the reverse order. |
but in the reverse mode they are required in the reverse order. |
| 572 |
|
|
| 573 |
A method to balance the amount of recomputations vs. |
A method to balance the amount of recomputations vs. |
| 574 |
storage requirements is called {\sf checkpointing} |
storage requirements is called {\sf checkpointing} |
| 575 |
(e.g. \cite{res-eta:98}). |
(e.g. \cite{gri:92}, \cite{res-eta:98}). |
| 576 |
It is depicted in \ref{fig:3levelcheck} for a 3-level checkpointing |
It is depicted in \ref{fig:3levelcheck} for a 3-level checkpointing |
| 577 |
[as an example, we give explicit numbers for a 3-day |
[as an example, we give explicit numbers for a 3-day |
| 578 |
integration with a 1-hourly timestep in square brackets]. |
integration with a 1-hourly timestep in square brackets]. |
| 583 |
$ {n}^{lev3} $ subsections [$ {n}^{lev3} $=3 1-day intervals], |
$ {n}^{lev3} $ subsections [$ {n}^{lev3} $=3 1-day intervals], |
| 584 |
with the label $lev3$ for this outermost loop. |
with the label $lev3$ for this outermost loop. |
| 585 |
The model is then integrated along the full trajectory, |
The model is then integrated along the full trajectory, |
| 586 |
and the model state stored only at every $ k_{i}^{lev3} $-th timestep |
and the model state stored to disk only at every $ k_{i}^{lev3} $-th timestep |
| 587 |
[i.e. 3 times, at |
[i.e. 3 times, at |
| 588 |
$ i = 0,1,2 $ corresponding to $ k_{i}^{lev3} = 0, 24, 48 $]. |
$ i = 0,1,2 $ corresponding to $ k_{i}^{lev3} = 0, 24, 48 $]. |
| 589 |
|
In addition, the cost function is computed, if needed. |
| 590 |
% |
% |
| 591 |
\item [$lev2$] |
\item [$lev2$] |
| 592 |
In a second step each subsection itself is divided into |
In a second step each subsection itself is divided into |
| 593 |
$ {n}^{lev2} $ sub-subsections |
$ {n}^{lev2} $ subsections |
| 594 |
[$ {n}^{lev2} $=4 6-hour intervals per subsection]. |
[$ {n}^{lev2} $=4 6-hour intervals per subsection]. |
| 595 |
The model picks up at the last outermost dumped state |
The model picks up at the last outermost dumped state |
| 596 |
$ v_{k_{n}^{lev3}} $ and is integrated forward in time along |
$ v_{k_{n}^{lev3}} $ and is integrated forward in time along |
| 597 |
the last subsection, with the label $lev2$ for this |
the last subsection, with the label $lev2$ for this |
| 598 |
intermediate loop. |
intermediate loop. |
| 599 |
The model state is now stored at every $ k_{i}^{lev2} $-th |
The model state is now stored to disk at every $ k_{i}^{lev2} $-th |
| 600 |
timestep |
timestep |
| 601 |
[i.e. 4 times, at |
[i.e. 4 times, at |
| 602 |
$ i = 0,1,2,3 $ corresponding to $ k_{i}^{lev2} = 48, 54, 60, 66 $]. |
$ i = 0,1,2,3 $ corresponding to $ k_{i}^{lev2} = 48, 54, 60, 66 $]. |
| 604 |
\item [$lev1$] |
\item [$lev1$] |
| 605 |
Finally, the model picks up at the last intermediate dump state |
Finally, the model picks up at the last intermediate dump state |
| 606 |
$ v_{k_{n}^{lev2}} $ and is integrated forward in time along |
$ v_{k_{n}^{lev2}} $ and is integrated forward in time along |
| 607 |
the last sub-subsection, with the label $lev1$ for this |
the last subsection, with the label $lev1$ for this |
| 608 |
intermediate loop. |
intermediate loop. |
| 609 |
Within this sub-subsection only, the model state is stored |
Within this sub-subsection only, parts of the model state is stored |
| 610 |
at every timestep |
to memory at every timestep |
| 611 |
[i.e. every hour $ i=0,...,5$ corresponding to |
[i.e. every hour $ i=0,...,5$ corresponding to |
| 612 |
$ k_{i}^{lev1} = 66, 67, \ldots, 71 $]. |
$ k_{i}^{lev1} = 66, 67, \ldots, 71 $]. |
| 613 |
Thus, the final state $ v_n = v_{k_{n}^{lev1}} $ is reached |
The final state $ v_n = v_{k_{n}^{lev1}} $ is reached |
| 614 |
and the model state of all peceeding timesteps along the last |
and the model state of all preceding timesteps along the last |
| 615 |
sub-subsections are available, enabling integration backwards |
innermost subsection are available, enabling integration backwards |
| 616 |
in time along the last sub-subsection. |
in time along the last subsection. |
| 617 |
Thus, the adjoint can be computed along this last |
The adjoint can thus be computed along this last |
| 618 |
sub-subsection $k_{n}^{lev2}$. |
subsection $k_{n}^{lev2}$. |
| 619 |
% |
% |
| 620 |
\end{itemize} |
\end{itemize} |
| 621 |
% |
% |
| 622 |
This procedure is repeated consecutively for each previous |
This procedure is repeated consecutively for each previous |
| 623 |
sub-subsection $k_{n-1}^{lev2}, \ldots, k_{1}^{lev2} $ |
subsection $k_{n-1}^{lev2}, \ldots, k_{1}^{lev2} $ |
| 624 |
carrying the adjoint computation to the initial time |
carrying the adjoint computation to the initial time |
| 625 |
of the subsection $k_{n}^{lev3}$. |
of the subsection $k_{n}^{lev3}$. |
| 626 |
Then, the procedure is repeated for the previous subsection |
Then, the procedure is repeated for the previous subsection |
| 631 |
For the full model trajectory of |
For the full model trajectory of |
| 632 |
$ n^{lev3} \cdot n^{lev2} \cdot n^{lev1} $ timesteps |
$ n^{lev3} \cdot n^{lev2} \cdot n^{lev1} $ timesteps |
| 633 |
the required storing of the model state was significantly reduced to |
the required storing of the model state was significantly reduced to |
| 634 |
$ n^{lev1} + n^{lev2} + n^{lev3} $ |
$ n^{lev2} + n^{lev3} $ to disk and roughly $ n^{lev1} $ to memory |
| 635 |
[i.e. for the 3-day integration with a total oof 72 timesteps |
[i.e. for the 3-day integration with a total oof 72 timesteps |
| 636 |
the model state was stored 13 times]. |
the model state was stored 7 times to disk and roughly 6 times |
| 637 |
|
to memory]. |
| 638 |
This saving in memory comes at a cost of a required |
This saving in memory comes at a cost of a required |
| 639 |
3 full forward integrations of the model (one for each |
3 full forward integrations of the model (one for each |
| 640 |
checkpointing level). |
checkpointing level). |
| 641 |
The balance of storage vs. recomputation certainly depends |
The optimal balance of storage vs. recomputation certainly depends |
| 642 |
on the computing resources available. |
on the computing resources available and may be adjusted by |
| 643 |
|
adjusting the partitioning among the |
| 644 |
|
$ n^{lev3}, \,\, n^{lev2}, \,\, n^{lev1} $. |
| 645 |
|
|
| 646 |
\begin{figure}[t!] |
\begin{figure}[t!] |
| 647 |
\begin{center} |
\begin{center} |
| 671 |
% \subsection{Error covariance estimate and Hessian matrix} |
% \subsection{Error covariance estimate and Hessian matrix} |
| 672 |
% \label{sec_hessian} |
% \label{sec_hessian} |
| 673 |
|
|
|
\newpage |
|
|
|
|
|
%********************************************************************** |
|
|
\section{AD-specific setup by example: sensitivity of carbon sequestration} |
|
|
\label{sec_ad_setup_ex} |
|
|
%********************************************************************** |
|
|
|
|
|
The MITGCM has been adapted to enable AD using TAMC or TAF. |
|
|
The present description, therefore, is specific to the |
|
|
use of TAMC or TAF as AD tool. |
|
|
The following sections describe the steps which are necessary to |
|
|
generate a tangent linear or adjoint model of the MITGCM. |
|
|
We take as an example the sensitivity of carbon sequestration |
|
|
in the ocean. |
|
|
The AD-relevant hooks in the code are sketched in |
|
|
\ref{fig:adthemodel}, \ref{fig:adthemain}. |
|
|
|
|
|
\subsection{Overview of the experiment} |
|
|
|
|
|
We describe an adjoint sensitivity analysis of outgassing from |
|
|
the ocean into the atmosphere of a carbon-like tracer injected |
|
|
into the ocean interior (see \cite{hil-eta:01}). |
|
|
|
|
|
\subsubsection{Passive tracer equation} |
|
|
|
|
|
For this work the MITGCM was augmented with a thermodynamically |
|
|
inactive tracer, $C$. Tracer residing in the ocean |
|
|
model surface layer is outgassed according to a relaxation time scale, |
|
|
$\mu$. Within the ocean interior, the tracer is passively advected |
|
|
by the ocean model currents. The full equation for the time evolution |
|
|
% |
|
|
\begin{equation} |
|
|
\label{carbon_ddt} |
|
|
\frac{\partial C}{\partial t} \, = \, |
|
|
-U\cdot \nabla C \, - \, \mu C \, + \, \Gamma(C) \,+ \, S |
|
|
\end{equation} |
|
|
% |
|
|
also includes a source term $S$. This term |
|
|
represents interior sources of $C$ such as would arise due to |
|
|
direct injection. |
|
|
The velocity term, $U$, is the sum of the |
|
|
model Eulerian circulation and an eddy-induced velocity, the latter |
|
|
parameterized according to Gent/McWilliams |
|
|
(\cite{gen-mcw:90, gen-eta:95}). |
|
|
The convection function, $\Gamma$, mixes $C$ vertically wherever the |
|
|
fluid is locally statically unstable. |
|
|
|
|
|
The outgassing time scale, $\mu$, in eqn. (\ref{carbon_ddt}) |
|
|
is set so that \( 1/\mu \sim 1 \ \mathrm{year} \) for the surface |
|
|
ocean and $\mu=0$ elsewhere. With this value, eqn. (\ref{carbon_ddt}) |
|
|
is valid as a prognostic equation for small perturbations in oceanic |
|
|
carbon concentrations. This configuration provides a |
|
|
powerful tool for examining the impact of large-scale ocean circulation |
|
|
on $ CO_2 $ outgassing due to interior injections. |
|
|
As source we choose a constant in time injection of |
|
|
$ S = 1 \,\, {\rm mol / s}$. |
|
|
|
|
|
\subsubsection{Model configuration} |
|
|
|
|
|
The model configuration employed has a constant |
|
|
$4^\circ \times 4^\circ$ resolution horizontal grid and realistic |
|
|
geography and bathymetry. Twenty vertical layers are used with |
|
|
vertical spacing ranging |
|
|
from 50 m near the surface to 815 m at depth. |
|
|
Driven to steady-state by climatalogical wind-stress, heat and |
|
|
fresh-water forcing the model reproduces well known large-scale |
|
|
features of the ocean general circulation. |
|
|
|
|
|
\subsubsection{Outgassing cost function} |
|
|
|
|
|
To quantify and understand outgassing due to injections of $C$ |
|
|
in eqn. (\ref{carbon_ddt}), |
|
|
we define a cost function $ {\cal J} $ that measures the total amount of |
|
|
tracer outgassed at each timestep: |
|
|
% |
|
|
\begin{equation} |
|
|
\label{cost_tracer} |
|
|
{\cal J}(t=T)=\int_{t=0}^{t=T}\int_{A} \mu C \, dA \, dt |
|
|
\end{equation} |
|
|
% |
|
|
Equation(\ref{cost_tracer}) integrates the outgassing term, $\mu C$, |
|
|
from (\ref{carbon_ddt}) |
|
|
over the entire ocean surface area, $A$, and accumulates it |
|
|
up to time $T$. |
|
|
Physically, ${\cal J}$ can be thought of as representing the amount of |
|
|
$CO_2$ that our model predicts would be outgassed following an |
|
|
injection at rate $S$. |
|
|
The sensitivity of ${\cal J}$ to the spatial location of $S$, |
|
|
$\frac{\partial {\cal J}}{\partial S}$, |
|
|
can be used to identify regions from which circulation |
|
|
would cause $CO_2$ to rapidly outgas following injection |
|
|
and regions in which $CO_2$ injections would remain effectively |
|
|
sequesterd within the ocean. |
|
|
|
|
|
\subsection{Code configuration} |
|
|
|
|
|
The model configuration for this experiment resides under the |
|
|
directory {\it verification/carbon/}. |
|
|
The code customisation routines are in {\it verification/carbon/code/}: |
|
|
% |
|
|
\begin{itemize} |
|
|
% |
|
|
\item {\it .genmakerc} |
|
|
% |
|
|
\item {\it COST\_CPPOPTIONS.h} |
|
|
% |
|
|
\item {\it CPP\_EEOPTIONS.h} |
|
|
% |
|
|
\item {\it CPP\_OPTIONS.h} |
|
|
% |
|
|
\item {\it CTRL\_OPTIONS.h} |
|
|
% |
|
|
\item {\it ECCO\_OPTIONS.h} |
|
|
% |
|
|
\item {\it SIZE.h} |
|
|
% |
|
|
\item {\it adcommon.h} |
|
|
% |
|
|
\item {\it tamc.h} |
|
|
% |
|
|
\end{itemize} |
|
|
% |
|
|
The runtime flag and parameters settings are contained in |
|
|
{\it verification/carbon/input/}, |
|
|
together with the forcing fields and and restart files: |
|
|
% |
|
|
\begin{itemize} |
|
|
% |
|
|
\item {\it data} |
|
|
% |
|
|
\item {\it data.cost} |
|
|
% |
|
|
\item {\it data.ctrl} |
|
|
% |
|
|
\item {\it data.gmredi} |
|
|
% |
|
|
\item {\it data.grdchk} |
|
|
% |
|
|
\item {\it data.optim} |
|
|
% |
|
|
\item {\it data.pkg} |
|
|
% |
|
|
\item {\it eedata} |
|
|
% |
|
|
\item {\it topog.bin} |
|
|
% |
|
|
\item {\it windx.bin, windy.bin} |
|
|
% |
|
|
\item {\it salt.bin, theta.bin} |
|
|
% |
|
|
\item {\it SSS.bin, SST.bin} |
|
|
% |
|
|
\item {\it pickup*} |
|
|
% |
|
|
\end{itemize} |
|
|
% |
|
|
Finally, the file to generate the adjoint code resides in |
|
|
$ adjoint/ $: |
|
|
% |
|
|
\begin{itemize} |
|
|
% |
|
|
\item {\it makefile} |
|
|
% |
|
|
\end{itemize} |
|
|
% |
|
|
|
|
|
Below we describe the customisations of this files which are |
|
|
specific to this experiment. |
|
|
|
|
|
\subsubsection{File {\it .genmakerc}} |
|
|
This file overwrites default settings of {\it genmake}. |
|
|
In the present example it is used to switch on the following |
|
|
packages which are related to automatic differentiation |
|
|
and are disabled by default: \\ |
|
|
\hspace*{4ex} {\tt set ENABLE=( autodiff cost ctrl ecco gmredi grdchk kpp )} \\ |
|
|
Other packages which are not needed are switched off: \\ |
|
|
\hspace*{4ex} {\tt set DISABLE=( aim obcs zonal\_filt shap\_filt cal exf )} |
|
|
|
|
|
\subsubsection{File {\it COST\_CPPOPTIONS.h, CTRL\_OPTIONS.h}} |
|
|
|
|
|
These files used to contain package-specific CPP-options |
|
|
(see Section \ref{???}). |
|
|
For technical reasons those options have been grouped together |
|
|
in the file {\it ECCO\_OPTIONS.h}. |
|
|
To retain the modularity, the files have been kept and contain |
|
|
the standard include of the {\it CPP\_OPTIONS.h} file. |
|
|
|
|
|
\subsubsection{File {\it CPP\_EEOPTIONS.h}} |
|
|
|
|
|
This file contains 'wrapper'-specific CPP options. |
|
|
It only needs to be changed if the code is to be run |
|
|
in a parallel environment (see Section \ref{???}). |
|
|
|
|
|
\subsubsection{File {\it CPP\_OPTIONS.h}} |
|
|
|
|
|
This file contains model-specific CPP options |
|
|
(see Section \ref{???}). |
|
|
Most options are related to the forward model setup. |
|
|
They are identical to the global steady circulation setup of |
|
|
{\it verification/exp2/}. |
|
|
The three options specific to this experiment are \\ |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_PASSIVE\_TRACER} \\ |
|
|
This flag enables the code to carry through the |
|
|
advection/diffusion of a passive tracer along the |
|
|
model integration. \\ |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_MIT\_ADJOINT\_RUN} \\ |
|
|
This flag enables the inclusion of some AD-related fields |
|
|
concerning initialisation, link between control variables |
|
|
and forward model variables, and the call to the top-level |
|
|
forward/adjoint subroutine {\it adthe\_main\_loop} |
|
|
instead of {\it the\_main\_loop}. \\ |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_GRADIENT\_CHECK} \\ |
|
|
This flag enables the gradient check package. |
|
|
After computing the unperturbed cost function and its gradient, |
|
|
a series of computations are performed for which \\ |
|
|
$\bullet$ an element of the control vector is perturbed \\ |
|
|
$\bullet$ the cost function w.r.t. the perturbed element is |
|
|
computed \\ |
|
|
$\bullet$ the difference between the perturbed and unperturbed |
|
|
cost function is computed to compute the finite difference gradient \\ |
|
|
$\bullet$ the finite difference gradient is compared with the |
|
|
adjoint-generated gradient. |
|
|
The gradient check package is further described in Section ???. |
|
|
|
|
|
\subsubsection{File {\it ECCO\_OPTIONS.h}} |
|
|
|
|
|
The CPP options of several AD-related packages are grouped |
|
|
in this file: |
|
|
% |
|
|
\begin{itemize} |
|
|
% |
|
|
\item |
|
|
Adjoint support package: {\it pkg/autodiff/} \\ |
|
|
This package contains hand-written adjoint code such as |
|
|
active file handling, flow directives for files which must not |
|
|
be differentiated, and TAMC-specific header files. \\ |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_AUTODIFF\_TAMC} \\ |
|
|
defines TAMC-related features in the code. \\ |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_TAMC\_CHECKPOINTING} \\ |
|
|
enables the checkpointing feature of TAMC |
|
|
(see Section \ref{???}). |
|
|
In the present example a 3-level checkpointing is implemented. |
|
|
The code contains the relevant store directives, common block |
|
|
and tape initialisations, storing key computation, |
|
|
and loop index handling. |
|
|
The checkpointing length at each level is defined in |
|
|
file {\it tamc.h}, cf. below. |
|
|
% |
|
|
\item Cost function package: {\it pkg/cost/} \\ |
|
|
This package contains all relevant routines for |
|
|
initialising, accumulating and finalizing the cost function |
|
|
(see Section \ref{???}). \\ |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_COST} \\ |
|
|
enables all general aspects of the cost function handling, |
|
|
in particular the hooks in the foorward code for |
|
|
initialising, accumulating and finalizing the cost function. \\ |
|
|
\hspace*{4ex} {\tt \#define ALLOW\_COST\_TRACER} \\ |
|
|
includes the call to the cost function for this |
|
|
particular experiment, eqn. (\ref{cost_tracer}). |
|
|
% |
|
|
\item Control variable package: {\it pkg/ctrl/} \\ |
|
|
This package contains all relevant routines for |
|
|
the handling of the control vector. |
|
|
Each control variable can be enabled/disabled with its own flag: \\ |
|
|
\begin{tabular}{ll} |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_THETA0\_CONTROL} & |
|
|
initial temperature \\ |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_SALT0\_CONTROL} & |
|
|
initial salinity \\ |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_TR0\_CONTROL} & |
|
|
initial passive tracer concentration \\ |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_TAUU0\_CONTROL} & |
|
|
zonal wind stress \\ |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_TAUV0\_CONTROL} & |
|
|
meridional wind stress \\ |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_SFLUX0\_CONTROL} & |
|
|
freshwater flux \\ |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_HFLUX0\_CONTROL} & |
|
|
heat flux \\ |
|
|
\hspace*{2ex} {\tt \#define ALLOW\_DIFFKR\_CONTROL} & |
|
|
diapycnal diffusivity \\ |
|
|
\hspace*{2ex} {\tt \#undef ALLOW\_KAPPAGM\_CONTROL} & |
|
|
isopycnal diffusivity \\ |
|
|
\end{tabular} |
|
|
% |
|
|
\end{itemize} |
|
|
|
|
|
\subsubsection{File {\it SIZE.h}} |
|
|
|
|
|
The file contains the grid point dimensions of the forward |
|
|
model. It is identical to the {\it verification/exp2/}: \\ |
|
|
\hspace*{4ex} {\tt sNx = 90} \\ |
|
|
\hspace*{4ex} {\tt sNy = 40} \\ |
|
|
\hspace*{4ex} {\tt Nr = 20} \\ |
|
|
It correpsponds to a single-tile/single-processor setup: |
|
|
{\tt nSx = nSy = 1, nPx = nPy = 1}, |
|
|
with standard overlap dimensioning |
|
|
{\tt OLx = OLy = 3}. |
|
|
|
|
|
\subsubsection{File {\it adcommon.h}} |
|
|
|
|
|
This file contains common blocks of some adjoint variables |
|
|
that are generated by TAMC. |
|
|
The common blocks are used by the adjoint support routine |
|
|
{\it addummy\_in\_stepping} which needs to access those variables: |
|
|
|
|
|
\begin{tabular}{ll} |
|
|
\hspace*{4ex} {\tt common /addynvars\_r/} & |
|
|
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
|
|
\hspace*{4ex} {\tt common /addynvars\_cd/} & |
|
|
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
|
|
\hspace*{4ex} {\tt common /addynvars\_diffkr/} & |
|
|
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
|
|
\hspace*{4ex} {\tt common /addynvars\_kapgm/} & |
|
|
\hspace*{4ex} is related to {\it DYNVARS.h} \\ |
|
|
\hspace*{4ex} {\tt common /adtr1\_r/} & |
|
|
\hspace*{4ex} is related to {\it TR1.h} \\ |
|
|
\hspace*{4ex} {\tt common /adffields/} & |
|
|
\hspace*{4ex} is related to {\it FFIELDS.h}\\ |
|
|
\end{tabular} |
|
|
|
|
|
Note that if the structure of the common block changes in the |
|
|
above header files of the forward code, the structure |
|
|
of the adjoint common blocks will change accordingly. |
|
|
Thus, it has to be made sure that the structure of the |
|
|
adjoint common block in the hand-written file {\it adcommon.h} |
|
|
complies with the automatically generated adjoint common blocks |
|
|
in {\it adjoint\_model.F}. |
|
|
|
|
|
\subsubsection{File {\it tamc.h}} |
|
|
|
|
|
This routine contains the dimensions for TAMC checkpointing. |
|
|
% |
|
|
\begin{itemize} |
|
|
% |
|
|
\item {\tt \#ifdef ALLOW\_TAMC\_CHECKPOINTING} \\ |
|
|
3-level checkpointing is enabled, i.e. the timestepping |
|
|
is divided into three different levels (see Section \ref{???}). |
|
|
The model state of the outermost ({\tt nchklev\_3}) and the |
|
|
intermediate ({\tt nchklev\_2}) timestepping loop are stored to file |
|
|
(handled in {\it the\_main\_loop}). |
|
|
The innermost loop ({\tt nchklev\_1}) |
|
|
avoids I/O by storing all required variables |
|
|
to common blocks. This storing may also be necessary if |
|
|
no checkpointing is chosen |
|
|
(nonlinear functions, if-statements, iterative loops, ...). |
|
|
In the present example the dimensions are chosen as follows: \\ |
|
|
\hspace*{4ex} {\tt nchklev\_1 = 36 } \\ |
|
|
\hspace*{4ex} {\tt nchklev\_2 = 30 } \\ |
|
|
\hspace*{4ex} {\tt nchklev\_3 = 60 } \\ |
|
|
To guarantee that the checkpointing intervals span the entire |
|
|
integration period the following relation must be satisfied: \\ |
|
|
\hspace*{4ex} {\tt nchklev\_1*nchklev\_2*nchklev\_3 $ \ge $ nTimeSteps} \\ |
|
|
where {\tt nTimeSteps} is either specified in {\it data} |
|
|
or computed via \\ |
|
|
\hspace*{4ex} {\tt nTimeSteps = (endTime-startTime)/deltaTClock }. |
|
|
% |
|
|
\item {\tt \#undef ALLOW\_TAMC\_CHECKPOINTING} \\ |
|
|
No checkpointing is enabled. |
|
|
In this case the relevant counter is {\tt nchklev\_0}. |
|
|
Similar to above, the following relation has to be satisfied \\ |
|
|
\hspace*{4ex} {\tt nchklev\_0 $ \ge $ nTimeSteps}. |
|
|
% |
|
|
\end{itemize} |
|
|
|
|
|
The following parameters may be worth describing: \\ |
|
|
% |
|
|
\hspace*{4ex} {\tt isbyte} \\ |
|
|
\hspace*{4ex} {\tt maxpass} \\ |
|
|
~ |
|
|
|
|
|
\subsubsection{File {\it makefile}} |
|
|
|
|
|
This file contains all relevant paramter flags and |
|
|
lists to run TAMC or TAF. |
|
|
It is assumed that TAMC is available to you, either locally, |
|
|
being installed on your network, or remotely through the 'TAMC Utility'. |
|
|
TAMC is called with the command {\tt tamc} followed by a |
|
|
number of options. They are described in detail in the |
|
|
TAMC manual \cite{gie:99}. |
|
|
Here we briefly discuss the main flags used in the {\it makefile} |
|
|
% |
|
|
\begin{itemize} |
|
|
\item [{\tt tamc}] {\tt |
|
|
-input <variable names> |
|
|
-output <variable name> -r4 ... \\ |
|
|
-toplevel <S/R name> -reverse <file names> |
|
|
} |
|
|
\end{itemize} |
|
|
% |
|
|
\begin{itemize} |
|
|
% |
|
|
\item {\tt -toplevel <S/R name>} \\ |
|
|
Name of the toplevel routine, with respect to which the |
|
|
control flow analysis is performed. |
|
|
% |
|
|
\item {\tt -input <variable names>} \\ |
|
|
List of independent variables $ u $ with respect to which the |
|
|
dependent variable $ J $ is differentiated. |
|
|
% |
|
|
\item {\tt -output <variable name>} \\ |
|
|
Dependent variable $ J $ which is to be differentiated. |
|
|
% |
|
|
\item {\tt -reverse <file names>} \\ |
|
|
Adjoint code is generated to compute the sensitivity of an |
|
|
independent variable w.r.t. many dependent variables. |
|
|
In the discussion of Section ??? |
|
|
the generated adjoint top-level routine computes the product |
|
|
of the transposed Jacobian matrix $ M^T $ times |
|
|
the gradient vector $ \nabla_v J $. |
|
|
\\ |
|
|
{\tt <file names>} refers to the list of files {\it .f} which are to be |
|
|
analyzed by TAMC. This list is generally smaller than the full list |
|
|
of code to be compiled. The files not contained are either |
|
|
above the top-level routine (some initialisations), or are |
|
|
deliberately hidden from TAMC, either because hand-written |
|
|
adjoint routines exist, or the routines must not (or don't have to) |
|
|
be differentiated. For each routine which is part of the flow tree |
|
|
of the top-level routine, but deliberately hidden from TAMC |
|
|
(or for each package which contains such routines), |
|
|
a corresponding file {\it .flow} exists containing flow directives |
|
|
for TAMC. |
|
|
% |
|
|
\item {\tt -r4} \\ |
|
|
~ |
|
|
% |
|
|
\end{itemize} |
|
|
|
|
|
|
|
|
\subsubsection{The input parameter files} |
|
|
|
|
|
\paragraph{File {\it data}} |
|
|
|
|
|
\paragraph{File {\it data.cost}} |
|
|
|
|
|
\paragraph{File {\it data.ctrl}} |
|
|
|
|
|
\paragraph{File {\it data.gmredi}} |
|
|
|
|
|
\paragraph{File {\it data.grdchk}} |
|
|
|
|
|
\paragraph{File {\it data.optim}} |
|
|
|
|
|
\paragraph{File {\it data.pkg}} |
|
|
|
|
|
\paragraph{File {\it eedata}} |
|
|
|
|
|
\paragraph{File {\it topog.bin}} |
|
|
|
|
|
\paragraph{File {\it windx.bin, windy.bin}} |
|
|
|
|
|
\paragraph{File {\it salt.bin, theta.bin}} |
|
|
|
|
|
\paragraph{File {\it SSS.bin, SST.bin}} |
|
|
|
|
|
\paragraph{File {\it pickup*}} |
|
|
|
|
|
\subsection{Compiling the model and its adjoint} |
|
|
|
|
|
The built process of the adjoint model is slightly more |
|
|
complex than that of compiling the forward code. |
|
|
The main reason is that the adjoint code generation requires |
|
|
a specific list of routines that are to be differentiated |
|
|
(as opposed to the automatic generation of a list of |
|
|
files to be compiled by genmake). |
|
|
This list excludes routines that don't have to be or must not be |
|
|
differentiated. For some of the latter routines flow directives |
|
|
may be necessary, a list of which has to be given as well. |
|
|
For this reason, a separate {\it makefile} is currently |
|
|
maintained in the directory {\tt adjoint/}. This |
|
|
makefile is responsible for the adjoint code generation. |
|
|
|
|
|
In the following we describe the build process step by step, |
|
|
assuming you are in the directory {\tt bin/}. |
|
|
A summary of steps to follow is given at the end. |
|
|
|
|
|
\paragraph{Adjoint code generation and compilation -- step by step} |
|
|
|
|
|
\begin{enumerate} |
|
|
% |
|
|
\item |
|
|
{\tt ln -s ../verification/???/code/.genmakerc .} \\ |
|
|
{\tt ln -s ../verification/???/code/*.[Fh] .} \\ |
|
|
Link your customized genmake options, header files, |
|
|
and modified code to the compile directory. |
|
|
% |
|
|
\item |
|
|
{\tt ../tools/genmake -makefile} \\ |
|
|
Generate your Makefile (cf. Section ???). |
|
|
% |
|
|
\item |
|
|
{\tt make depend} \\ |
|
|
Dependency analysis for the CPP pre-compiler (cf. Section ???). |
|
|
% |
|
|
\item |
|
|
{\tt make small\_f} \\ |
|
|
This is the first difference between forward code compilation |
|
|
and adjoint code generation and compilation. |
|
|
Instead of going through the entire compilation process |
|
|
(CPP precompiling -- {\tt .f}, object code generation -- {\tt .o}, |
|
|
linking of object files and libraries to generate executable), |
|
|
only the CPP compiler is invoked at this stage to generate |
|
|
the {\tt .f} files. |
|
|
% |
|
|
\item |
|
|
{\tt cd ../adjoint} \\ |
|
|
{\tt make adtaf} or {\tt make adtamc} \\ |
|
|
Depending on whether you have TAF or TAMC at your disposal, |
|
|
you'll choose {\tt adtaf} or {\tt adtamc} as your |
|
|
make target for the {\it makefile} in the directory {\tt adjoint/}. |
|
|
Several things happen at this stage. |
|
|
% |
|
|
\begin{enumerate} |
|
|
% |
|
|
\item |
|
|
The initial template file {\it adjoint\_model.F} which is part |
|
|
of the compiling list created by {\it genmake} is restored. |
|
|
% |
|
|
\item |
|
|
All Fortran routines {\tt *.f} in {\tt bin/} are |
|
|
concatenated into a single file (it's current name is |
|
|
{\it tamc\_code.f}). |
|
|
% |
|
|
\item |
|
|
Adjoint code is generated by TAMC or TAF. |
|
|
The adjoint code is written to the file {\it tamc\_code\_ad.f}. |
|
|
It contains all adjoint routines of the forward routines |
|
|
concatenated in {\it tamc\_code.f}. |
|
|
For a given forward routines {\tt subroutine routinename} |
|
|
the adjoint routine is named {\tt adsubroutine routinename} |
|
|
by default (that default can be changed via the flag |
|
|
{\tt -admark <markname>}). |
|
|
Furthermore, it may contain modified code which |
|
|
incorporates the translation of adjoint store directives |
|
|
into specific Fortran code. |
|
|
For a given forward routines {\tt subroutine routinename} |
|
|
the modified routine is named {\tt mdsubroutine routinename}. |
|
|
TAMC or TAF info is written to file |
|
|
{\it tamc\_code.prot} or {\it taf.log}, respectively. |
|
|
% |
|
|
\end{enumerate} |
|
|
% |
|
|
\item |
|
|
{\tt make adchange} \\ |
|
|
The multi-threading capability of the MITGCM requires a slight |
|
|
change in the parameter list of some routines that are related to |
|
|
to active file handling. |
|
|
This postprocessing invokes the sed script {\it adjoint\_ecco\_sed.com} |
|
|
to insert the threading counter {\bf myThId} into the parameter list |
|
|
of those subroutines. |
|
|
The resulting code is written to file {\it tamc\_code\_sed\_ad.f} |
|
|
and appended to the file {\it adjoint\_model.F}. |
|
|
This concludes the adjoint codel generation. |
|
|
% |
|
|
\item |
|
|
{\tt cd ../bin} \\ |
|
|
{\tt make} \\ |
|
|
The file {\it adjoint\_model.F} now contains the full adjoint code. |
|
|
All routines are now compiled. |
|
|
% |
|
|
\end{enumerate} |
|
|
|
|
|
\paragraph{Adjoint code generation and compilation -- summary} |
|
|
~ \\ |
|
|
|
|
|
\[ |
|
|
\boxed{ |
|
|
\begin{split} |
|
|
~ & \mbox{\tt cd bin} \\ |
|
|
~ & \mbox{\tt ln -s ../verification/my\_experiment/code/.genmakerc .} \\ |
|
|
~ & \mbox{\tt ln -s ../verification/my\_experiment/code/*.[Fh] .} \\ |
|
|
~ & \mbox{\tt ../tools/genmake -makefile} \\ |
|
|
~ & \mbox{\tt make depend} \\ |
|
|
~ & \mbox{\tt make small\_f} \\ |
|
|
~ & \mbox{\tt cd ../adjoint} \\ |
|
|
~ & \mbox{\tt make adtaf <OR: make adtamc>} \\ |
|
|
~ & \mbox{\tt make adchange} \\ |
|
|
~ & \mbox{\tt cd ../bin} \\ |
|
|
~ & \mbox{\tt make} \\ |
|
|
\end{split} |
|
|
} |
|
|
\] |
|
|
|
|
| 674 |
\newpage |
\newpage |
| 675 |
|
|
| 676 |
%********************************************************************** |
%********************************************************************** |
| 682 |
the parts of the code that are relevant for automatic |
the parts of the code that are relevant for automatic |
| 683 |
differentiation using the software tool TAMC. |
differentiation using the software tool TAMC. |
| 684 |
|
|
|
\begin{figure}[b!] |
|
| 685 |
\input{part5/doc_ad_the_model} |
\input{part5/doc_ad_the_model} |
|
\caption{~} |
|
|
\label{fig:adthemodel} |
|
|
\end{figure} |
|
| 686 |
|
|
| 687 |
The basic flow is depicted in \ref{fig:adthemodel}. |
The basic flow is depicted in \ref{fig:adthemodel}. |
| 688 |
If the option {\tt ALLOW\_AUTODIFF\_TAMC} is defined, the driver routine |
If the option {\tt ALLOW\_AUTODIFF\_TAMC} is defined, the driver routine |
| 689 |
{\it the\_model\_main}, instead of calling {\it the\_main\_loop}, |
{\it the\_model\_main}, instead of calling {\it the\_main\_loop}, |
| 690 |
invokes the adjoint of this routine, {\it adthe\_main\_loop}, |
invokes the adjoint of this routine, {\it adthe\_main\_loop}, |
| 691 |
which is the toplevel routine in terms of reverse mode computation. |
which is the toplevel routine in terms of reverse mode computation. |
| 692 |
The routine {\it adthe\_main\_loop} has been generated using TAMC. |
The routine {\it adthe\_main\_loop} has been generated by TAMC. |
| 693 |
It contains both the forward integration of the full model, |
It contains both the forward integration of the full model, |
| 694 |
any additional storing that is required for efficient checkpointing, |
any additional storing that is required for efficient checkpointing, |
| 695 |
and the reverse integration of the adjoint model. |
and the reverse integration of the adjoint model. |
| 697 |
simplified for clarification; in particular, no checkpointing |
simplified for clarification; in particular, no checkpointing |
| 698 |
procedures are shown here. |
procedures are shown here. |
| 699 |
Prior to the call of {\it adthe\_main\_loop}, the routine |
Prior to the call of {\it adthe\_main\_loop}, the routine |
| 700 |
{\it ctrl\_unpack} is invoked to unpack the control vector, |
{\it ctrl\_unpack} is invoked to unpack the control vector |
| 701 |
and following that call, the routine {\it ctrl\_pack} |
or initialise the control variables. |
| 702 |
|
Following the call of {\it adthe\_main\_loop}, |
| 703 |
|
the routine {\it ctrl\_pack} |
| 704 |
is invoked to pack the control vector |
is invoked to pack the control vector |
| 705 |
(cf. Section \ref{section_ctrl}). |
(cf. Section \ref{section_ctrl}). |
| 706 |
If gradient checks are to be performed, the option |
If gradient checks are to be performed, the option |
| 715 |
The cost function $ {\cal J} $ is referred to as the {\sf dependent variable}. |
The cost function $ {\cal J} $ is referred to as the {\sf dependent variable}. |
| 716 |
It is a function of the input variables $ \vec{u} $ via the composition |
It is a function of the input variables $ \vec{u} $ via the composition |
| 717 |
$ {\cal J}(\vec{u}) \, = \, {\cal J}(M(\vec{u})) $. |
$ {\cal J}(\vec{u}) \, = \, {\cal J}(M(\vec{u})) $. |
| 718 |
The input is referred to as the |
The input are referred to as the |
| 719 |
{\sf independent variables} or {\sf control variables}. |
{\sf independent variables} or {\sf control variables}. |
| 720 |
All aspects relevant to the treatment of the cost function $ {\cal J} $ |
All aspects relevant to the treatment of the cost function $ {\cal J} $ |
| 721 |
(parameter setting, initialisation, accumulation, |
(parameter setting, initialization, accumulation, |
| 722 |
final evaluation), are controlled by the package {\it pkg/cost}. |
final evaluation), are controlled by the package {\it pkg/cost}. |
| 723 |
|
The aspects relevant to the treatment of the independent variables |
| 724 |
|
are controlled by the package {\it pkg/ctrl} and will be treated |
| 725 |
|
in the next section. |
| 726 |
|
|
|
\begin{figure}[h!] |
|
| 727 |
\input{part5/doc_cost_flow} |
\input{part5/doc_cost_flow} |
|
\caption{~} |
|
|
\label{fig:costflow} |
|
|
\end{figure} |
|
| 728 |
|
|
| 729 |
\subsubsection{genmake and CPP options} |
\subsubsection{genmake and CPP options} |
| 730 |
% |
% |
| 757 |
{\tt genmake -enable=cost}. |
{\tt genmake -enable=cost}. |
| 758 |
% |
% |
| 759 |
\end{enumerate} |
\end{enumerate} |
| 760 |
|
N.B.: In general the following packages ought to be enabled |
| 761 |
|
simultaneously: {\it autodiff, cost, ctrl}. |
| 762 |
The basic CPP option to enable the cost function is {\bf ALLOW\_COST}. |
The basic CPP option to enable the cost function is {\bf ALLOW\_COST}. |
| 763 |
Each specific cost function contribution has its own option. |
Each specific cost function contribution has its own option. |
| 764 |
For the present example the option is {\bf ALLOW\_COST\_TRACER}. |
For the present example the option is {\bf ALLOW\_COST\_TRACER}. |
| 765 |
All cost-specific options are set in {\it ECCO\_CPPOPTIONS.h} |
All cost-specific options are set in {\it ECCO\_CPPOPTIONS.h} |
| 766 |
Since the cost function is usually used in conjunction with |
Since the cost function is usually used in conjunction with |
| 767 |
automatic differentiation, the CPP option |
automatic differentiation, the CPP option |
| 768 |
{\bf ALLOW\_ADJOINT\_RUN} should be defined |
{\bf ALLOW\_ADJOINT\_RUN} (file {\it CPP\_OPTIONS.h}) and |
| 769 |
(file {\it CPP\_OPTIONS.h}). |
{\bf ALLOW\_AUTODIFF\_TAMC} (file {\it ECCO\_CPPOPTIONS.h}) |
| 770 |
|
should be defined. |
| 771 |
|
|
| 772 |
\subsubsection{Initialisation} |
\subsubsection{Initialization} |
| 773 |
% |
% |
| 774 |
The initialisation of the {\it cost} package is readily enabled |
The initialization of the {\it cost} package is readily enabled |
| 775 |
as soon as the CPP option {\bf ALLOW\_ADJOINT\_RUN} is defined. |
as soon as the CPP option {\bf ALLOW\_COST} is defined. |
| 776 |
% |
% |
| 777 |
\begin{itemize} |
\begin{itemize} |
| 778 |
% |
% |
| 802 |
} |
} |
| 803 |
\\ |
\\ |
| 804 |
This S/R |
This S/R |
| 805 |
initialises the different cost function contributions. |
initializes the different cost function contributions. |
| 806 |
The contribtion for the present example is {\bf objf\_tracer} |
The contribution for the present example is {\bf objf\_tracer} |
| 807 |
which is defined on each tile (bi,bj). |
which is defined on each tile (bi,bj). |
| 808 |
% |
% |
| 809 |
\end{itemize} |
\end{itemize} |
| 846 |
\begin{equation} |
\begin{equation} |
| 847 |
{\cal J} \, = \, |
{\cal J} \, = \, |
| 848 |
{\rm fc} \, = \, |
{\rm fc} \, = \, |
| 849 |
{\rm mult\_tracer} \sum_{bi,\,bj}^{nSx,\,nSy} |
{\rm mult\_tracer} \sum_{\text{global sum}} \sum_{bi,\,bj}^{nSx,\,nSy} |
| 850 |
{\rm objf\_tracer}(bi,bj) \, + \, ... |
{\rm objf\_tracer}(bi,bj) \, + \, ... |
| 851 |
\end{equation} |
\end{equation} |
| 852 |
% |
% |
| 858 |
|
|
| 859 |
%%%% \end{document} |
%%%% \end{document} |
| 860 |
|
|
|
\begin{figure} |
|
| 861 |
\input{part5/doc_ad_the_main} |
\input{part5/doc_ad_the_main} |
|
\caption{~} |
|
|
\label{fig:adthemain} |
|
|
\end{figure} |
|
| 862 |
|
|
| 863 |
\subsection{The control variables (independent variables) |
\subsection{The control variables (independent variables) |
| 864 |
\label{section_ctrl}} |
\label{section_ctrl}} |
| 875 |
active variables are written and from which active variables |
active variables are written and from which active variables |
| 876 |
are read are called {\sf active files}. |
are read are called {\sf active files}. |
| 877 |
All aspects relevant to the treatment of the control variables |
All aspects relevant to the treatment of the control variables |
| 878 |
(parameter setting, initialisation, perturbation) |
(parameter setting, initialization, perturbation) |
| 879 |
are controled by the package {\it pkg/ctrl}. |
are controlled by the package {\it pkg/ctrl}. |
| 880 |
|
|
|
\begin{figure}[h!] |
|
| 881 |
\input{part5/doc_ctrl_flow} |
\input{part5/doc_ctrl_flow} |
|
\caption{~} |
|
|
\label{fig:ctrlflow} |
|
|
\end{figure} |
|
| 882 |
|
|
| 883 |
\subsubsection{genmake and CPP options} |
\subsubsection{genmake and CPP options} |
| 884 |
% |
% |
| 894 |
% |
% |
| 895 |
To enable the directory to be included to the compile list, |
To enable the directory to be included to the compile list, |
| 896 |
{\bf ctrl} has to be added to the {\bf enable} list in |
{\bf ctrl} has to be added to the {\bf enable} list in |
| 897 |
{\it .genmakerc} (or {\it genmake} itself). |
{\it .genmakerc} or in {\it genmake} itself (analogous to {\it cost} |
| 898 |
|
package, cf. previous section). |
| 899 |
Each control variable is enabled via its own CPP option |
Each control variable is enabled via its own CPP option |
| 900 |
in {\it ECCO\_CPPOPTIONS.h}. |
in {\it ECCO\_CPPOPTIONS.h}. |
| 901 |
|
|
| 902 |
\subsubsection{Initialisation} |
\subsubsection{Initialization} |
| 903 |
% |
% |
| 904 |
\begin{itemize} |
\begin{itemize} |
| 905 |
% |
% |
| 939 |
variables in the MITGCM need to be addressed. |
variables in the MITGCM need to be addressed. |
| 940 |
First, in order to save memory, the control variable arrays |
First, in order to save memory, the control variable arrays |
| 941 |
are not kept in memory, but rather read from file and added |
are not kept in memory, but rather read from file and added |
| 942 |
to the initial fields during the model initialisation phase. |
to the initial fields during the model initialization phase. |
| 943 |
Similarly, the corresponding adjoint fields which represent |
Similarly, the corresponding adjoint fields which represent |
| 944 |
the gradient of the cost function w.r.t. the control variables |
the gradient of the cost function w.r.t. the control variables |
| 945 |
are written to file at the end of the adjoint integration. |
are written to file at the end of the adjoint integration. |
| 1019 |
and an 'active read' routine of the adjoint support |
and an 'active read' routine of the adjoint support |
| 1020 |
package {\it pkg/autodiff} is invoked. |
package {\it pkg/autodiff} is invoked. |
| 1021 |
The read-procedure is tagged with the variable |
The read-procedure is tagged with the variable |
| 1022 |
{\bf xx\_tr1\_dummy} enabbling TAMC to recognize the |
{\bf xx\_tr1\_dummy} enabling TAMC to recognize the |
| 1023 |
initialisation of the perturbation. |
initialization of the perturbation. |
| 1024 |
The modified call of TAMC thus reads |
The modified call of TAMC thus reads |
| 1025 |
% |
% |
| 1026 |
\begin{verbatim} |
\begin{verbatim} |
| 1039 |
% |
% |
| 1040 |
Note, that reading an active variable corresponds |
Note, that reading an active variable corresponds |
| 1041 |
to a variable assignment. Its derivative corresponds |
to a variable assignment. Its derivative corresponds |
| 1042 |
to a write statement of the adjoint variable. |
to a write statement of the adjoint variable, followed by |
| 1043 |
|
a reset. |
| 1044 |
The 'active file' routines have been designed |
The 'active file' routines have been designed |
| 1045 |
to support active read and corresponding adjoint active write |
to support active read and corresponding adjoint active write |
| 1046 |
operations (and vice versa). |
operations (and vice versa). |
| 1132 |
\begin{itemize} |
\begin{itemize} |
| 1133 |
% |
% |
| 1134 |
\item {\bf vector\_ctrl}: the control vector \\ |
\item {\bf vector\_ctrl}: the control vector \\ |
| 1135 |
At the very beginning of the model initialisation, |
At the very beginning of the model initialization, |
| 1136 |
the updated compressed control vector is read (or initialised) |
the updated compressed control vector is read (or initialised) |
| 1137 |
and distributed to 2-dim. and 3-dim. control variable fields. |
and distributed to 2-dim. and 3-dim. control variable fields. |
| 1138 |
% |
% |
| 1157 |
{\it addummy\_in\_stepping}. |
{\it addummy\_in\_stepping}. |
| 1158 |
This routine is part of the adjoint support package |
This routine is part of the adjoint support package |
| 1159 |
{\it pkg/autodiff} (cf.f. below). |
{\it pkg/autodiff} (cf.f. below). |
| 1160 |
|
The procedure is enabled using via the CPP-option |
| 1161 |
|
{\bf ALLOW\_AUTODIFF\_MONITOR} (file {\it ECCO\_CPPOPTIONS.h}). |
| 1162 |
To be part of the adjoint code, the corresponding S/R |
To be part of the adjoint code, the corresponding S/R |
| 1163 |
{\it dummy\_in\_stepping} has to be called in the forward |
{\it dummy\_in\_stepping} has to be called in the forward |
| 1164 |
model (S/R {\it the\_main\_loop}) at the appropriate place. |
model (S/R {\it the\_main\_loop}) at the appropriate place. |
| 1165 |
|
The adjoint common blocks are extracted from the adjoint code |
| 1166 |
|
via the header file {\it adcommon.h}. |
| 1167 |
|
|
| 1168 |
{\it dummy\_in\_stepping} is essentially empty, |
{\it dummy\_in\_stepping} is essentially empty, |
| 1169 |
the corresponding adjoint routine is hand-written rather |
the corresponding adjoint routine is hand-written rather |
| 1190 |
{\bf /adtr1\_r/}, {\bf /adffields/}, |
{\bf /adtr1\_r/}, {\bf /adffields/}, |
| 1191 |
which have been extracted from the adjoint code to enable |
which have been extracted from the adjoint code to enable |
| 1192 |
access to the adjoint variables. |
access to the adjoint variables. |
| 1193 |
|
|
| 1194 |
|
{\bf WARNING:} If the structure of the common blocks |
| 1195 |
|
{\bf /dynvars\_r/}, {\bf /dynvars\_cd/}, etc., changes |
| 1196 |
|
similar changes will occur in the adjoint common blocks. |
| 1197 |
|
Therefore, consistency between the TAMC-generated common blocks |
| 1198 |
|
and those in {\it adcommon.h} have to be checked. |
| 1199 |
% |
% |
| 1200 |
\end{itemize} |
\end{itemize} |
| 1201 |
|
|
| 1210 |
with the value of the cost function itself $ {\cal J}(u_{[k]}) $ |
with the value of the cost function itself $ {\cal J}(u_{[k]}) $ |
| 1211 |
at iteration step $ k $ serve |
at iteration step $ k $ serve |
| 1212 |
as input to a minimization routine (e.g. quasi-Newton method, |
as input to a minimization routine (e.g. quasi-Newton method, |
| 1213 |
conjugate gradient, ... \cite{gil_lem:89}) |
conjugate gradient, ... \cite{gil-lem:89}) |
| 1214 |
to compute an update in the |
to compute an update in the |
| 1215 |
control variable for iteration step $k+1$ |
control variable for iteration step $k+1$ |
| 1216 |
\[ |
\[ |
| 1341 |
Finally, {\it ctrl\_pack} collects all adjoint files |
Finally, {\it ctrl\_pack} collects all adjoint files |
| 1342 |
and writes them to the compressed vector file |
and writes them to the compressed vector file |
| 1343 |
{\bf vector\_grad\_$<$k$>$}. |
{\bf vector\_grad\_$<$k$>$}. |
|
|
|
|
\subsection{TLM and ADM generation via TAMC} |
|
|
|
|
|
|
|
|
|
|
|
\subsection{Flow directives and adjoint support routines \label{section_flowdir}} |
|
|
|
|
|
\subsection{Store directives and checkpointing \label{section_checkpointing}} |
|
|
|
|
|
\subsection{Gradient checks \label{section_grdchk}} |
|
|
|
|
|
\subsection{Second derivative generation via TAMC} |
|
|
|
|
|
\section{Example of adjoint code} |
|
Parent Directory
| 
Revision Log
| 
Revision Graph
| 
Patch