/[MITgcm]/manual/s_autodiff/text/doc_ad_2.tex

Diff of /manual/s_autodiff/text/doc_ad_2.tex

Parent Directory | Revision Log | View Revision Graph Revision Graph | View Patch Patch

-revision 1.14 by cnh,
Thu Feb 28 19:32:20 2002 UTC
+revision 1.23 by jmc,
Mon Aug 30 23:09:19 2010 UTC
 Line 1
  % $Header$
  % $Name$
+ Author: Patrick Heimbach
+ \label{ask_the_author:doc_ad_2}
  {\sf Automatic differentiation} (AD), also referred to as algorithmic
  (or, more loosely, computational) differentiation, involves
- automatically deriving code to calculate
+ automatically deriving code to calculate partial derivatives from an
- partial derivatives from an existing fully non-linear prognostic code.
+ existing fully non-linear prognostic code.  (see \cite{gri:00}).  A
- (see \cite{gri:00}).
+ software tool is used that parses and transforms source files
- A software tool is used that parses and transforms source files
+ according to a set of linguistic and mathematical rules.  AD tools are
- according to a set of linguistic and mathematical rules.
+ like source-to-source translators in that they parse a program code as
- AD tools are like source-to-source translators in that
+ input and produce a new program code as output
- they parse a program code as input and produce a new program code
+ (we restrict our discussion to source-to-source tools, ignoring
- as output.
+ operator-overloading tools).  However, unlike a
- However, unlike a pure source-to-source translation, the output program
+ pure source-to-source translation, the output program represents a new
- represents a new algorithm, such as the evaluation of the
+ algorithm, such as the evaluation of the Jacobian, the Hessian, or
- Jacobian, the Hessian, or higher derivative operators.
+ higher derivative operators.  In principle, a variety of derived
- In principle, a variety of derived algorithms
+ algorithms can be generated automatically in this way.
- can be generated automatically in this way.
+ MITgcm has been adapted for use with the Tangent linear and Adjoint
- The MITGCM has been adapted for use with the
+ Model Compiler (TAMC) and its successor TAF (Transformation of
- Tangent linear and Adjoint Model Compiler (TAMC) and its successor TAF
+ Algorithms in Fortran), developed by Ralf Giering (\cite{gie-kam:98},
- (Transformation of Algorithms in Fortran), developed
+ \cite{gie:99,gie:00}).  The first application of the adjoint of MITgcm
- by Ralf Giering (\cite{gie-kam:98}, \cite{gie:99,gie:00}).
+ for sensitivity studies has been published by \cite{maro-eta:99}.
- The first application of the adjoint of the MITGCM for sensitivity
+ \cite{stam-etal:97,stam-etal:02} use MITgcm and its adjoint for ocean
- studies has been published by \cite{maro-eta:99}.
+ state estimation studies.  In the following we shall refer to TAMC and
- \cite{sta-eta:97,sta-eta:01} use the MITGCM and its adjoint
+ TAF synonymously, except were explicitly stated otherwise.
- for ocean state estimation studies.
- In the following we shall refer to TAMC and TAF synonymously,
+ As of mid-2007 we are also able to generate fairly efficient
- except were explicitly stated otherwise.
+ adjoint code of the MITgcm using a new, open-source AD tool,
+ called OpenAD (see \cite{naum-etal:06,utke-etal:08}.
- TAMC exploits the chain rule for computing the first
+ This enables us for the first time to compare adjoint models
- derivative of a function with
+ generated from different AD tools, providing an additional
- respect to a set of input variables.
+ accuracy check, complementary to finite-difference gradient checks.
- Treating a given forward code as a composition of operations --
+ OpenAD and its application to  MITgcm is described in detail
- each line representing a compositional element, the chain rule is
+ in section \ref{sec_ad_openad}.
- rigorously applied to the code, line by line. The resulting
- tangent linear or adjoint code,
+ The AD tool exploits the chain rule for computing the first derivative of a
- then, may be thought of as the composition in
+ function with respect to a set of input variables.  Treating a given
- forward or reverse order, respectively, of the
+ forward code as a composition of operations -- each line representing
- Jacobian matrices of the forward code's compositional elements.
+ a compositional element, the chain rule is rigorously applied to the
+ code, line by line. The resulting tangent linear or adjoint code,
+ then, may be thought of as the composition in forward or reverse
+ order, respectively, of the Jacobian matrices of the forward code's
+ compositional elements.
  %**********************************************************************
  \section{Some basic algebra}
  \label{sec_ad_algebra}
+ \begin{rawhtml}
+ <!-- CMIREDIR:sec_ad_algebra: -->
+ \end{rawhtml}
  %**********************************************************************
  Let $ \cal{M} $ be a general nonlinear, model, i.e. a
-Line 56 
 model output variable $\vec{v}=(v_1,\ldo
+Line 66 
 model output variable $\vec{v}=(v_1,\ldo
  under consideration,
  %
  \begin{equation}
- \begin{split}
+ \begin{aligned}
  {\cal M} \, : & \, U \,\, \longrightarrow \, V \\
  ~      & \, \vec{u} \,\, \longmapsto \, \vec{v} \, = \,
  {\cal M}(\vec{u})
  \label{fulloperator}
- \end{split}
+ \end{aligned}
  \end{equation}
  %
  The vectors $ \vec{u} \in U $ and $ v \in V $ may be represented w.r.t.
-Line 141 
 w.r.t. their corresponding inner product
+Line 151 
 w.r.t. their corresponding inner product
  $\left\langle \,\, , \,\, \right\rangle $
  %
  \begin{equation}
- \begin{split}
+ \begin{aligned}
  {\cal J} & = \,
  {\cal J} |_{\vec{u}^{(0)}} \, + \,
  \left\langle \, \nabla _{u}{\cal J}^T |_{\vec{u}^{(0)}} \, , \, \delta \vec{u} \, \right\rangle
-Line 150 
 $\left\langle \,\, , \,\, \right\rangle
+Line 160 
 $\left\langle \,\, , \,\, \right\rangle
  {\cal J} |_{\vec{v}^{(0)}} \, + \,
  \left\langle \, \nabla _{v}{\cal J}^T |_{\vec{v}^{(0)}} \, , \, \delta \vec{v} \, \right\rangle
  \, + \, O(\delta \vec{v}^2)
- \end{split}
+ \end{aligned}
  \label{deljidentity}
  \end{equation}
  %
-Line 191 
 the gradient $ \nabla _{u}{\cal J} $ can
+Line 201 
 the gradient $ \nabla _{u}{\cal J} $ can
  invoking the adjoint $ M^{\ast } $ of the tangent linear model $ M $
  %
  \begin{equation}
- \begin{split}
+ \begin{aligned}
  \nabla _{u}{\cal J}^T |_{\vec{u}} &
  = \, M^T |_{\vec{u}} \cdot \nabla _{v}{\cal J}^T |_{\vec{v}}  \\
  ~ & = \, M^T |_{\vec{u}} \cdot \delta \vec{v}^{\ast} \\
  ~ & = \, \delta \vec{u}^{\ast}
- \end{split}
+ \end{aligned}
  \label{adjoint}
  \end{equation}
  %
-Line 244 
 $ \langle \, \nabla _{v}{\cal J}^T \, ,
+Line 254 
 $ \langle \, \nabla _{v}{\cal J}^T \, ,
  = \nabla_v {\cal J} \cdot \delta \vec{v} $ )
  %
  \begin{equation}
- \begin{split}
+ \begin{aligned}
  \nabla_v {\cal J} (M(\delta \vec{u})) & = \,
  \nabla_v {\cal J} \cdot M_{\Lambda}
  \cdot ...... \cdot M_{\lambda} \cdot ...... \cdot
  M_{1} \cdot M_{0} \cdot \delta \vec{u} \\
  ~ & = \, \nabla_v {\cal J} \cdot \delta \vec{v} \\
- \end{split}
+ \end{aligned}
  \label{forward}
  \end{equation}
  %
-Line 258 
 whereas in reverse mode we have
+Line 268 
 whereas in reverse mode we have
  %
  \begin{equation}
  \boxed{
- \begin{split}
+ \begin{aligned}
  M^T ( \nabla_v {\cal J}^T) & = \,
  M_{0}^T \cdot M_{1}^T
  \cdot ...... \cdot M_{\lambda}^T \cdot ...... \cdot
-Line 267 
 M_{\Lambda}^T \cdot \nabla_v {\cal J}^T
+Line 277 
 M_{\Lambda}^T \cdot \nabla_v {\cal J}^T
  \cdot ...... \cdot
  \nabla_{v^{(\lambda)}} {\cal J}^T \\
  ~ & = \, \nabla_u {\cal J}^T
- \end{split}
+ \end{aligned}
  }
  \label{reverse}
  \end{equation}
-Line 286 
 $ \vec{v}^{(\lambda)} $ at each intermed
+Line 296 
 $ \vec{v}^{(\lambda)} $ at each intermed
  %
  \begin{equation}
  \boxed{
- \begin{split}
+ \begin{aligned}
  \nabla_{v^{(\lambda)}} {\cal J}^T |_{\vec{v}^{(\lambda)}}
  & = \,
  M_{\lambda}^T |_{\vec{v}^{(\lambda)}} \cdot ...... \cdot
  M_{\Lambda}^T |_{\vec{v}^{(\lambda)}} \cdot \delta \vec{v}^{\ast} \\
  ~ & = \, \delta \vec{v}^{(\lambda) \, \ast}
- \end{split}
+ \end{aligned}
  }
  \end{equation}
  %
-Line 409 
 and the shorthand notation for the adjoi
+Line 419 
 and the shorthand notation for the adjoi
  $ \delta v^{(\lambda) \, \ast}_{j} = \frac{\partial}{\partial v^{(\lambda)}_{j}}
  {\cal J}^T $, $ j = 1, \ldots , n_{\lambda} $,
  for intermediate components, yielding
+ {\small
  \begin{equation}
- \small
+ \begin{aligned}
- \begin{split}
  \left(
  \begin{array}{c}
  \delta v^{(\lambda) \, \ast}_1 \\
-Line 456 
 for intermediate components, yielding
+Line 466 
 for intermediate components, yielding
  \delta v^{\ast}_{n} \\
  \end{array}
  \right)
- \end{split}
+ \end{aligned}
  \end{equation}
+ }
  Eq. (\ref{forward}) and (\ref{reverse}) are perhaps clearest in
  showing the advantage of the reverse over the forward mode
-Line 528 
 operator which maps the model state spac
+Line 539 
 operator which maps the model state spac
  Then, $ \nabla_v {\cal J} $ takes the form
  %
  \begin{equation*}
- \begin{split}
+ \begin{aligned}
  \nabla_v {\cal J}^T & = \, 2 \, \, H \cdot
  \left( \, {\cal H}(\vec{v}) - \vec{d} \, \right) \\
  ~          & = \, 2 \sum_{j} \left\{ \sum_k
  \frac{\partial {\cal H}_k}{\partial v_{j}}
  \left( {\cal H}_k (\vec{v}) - d_k \right)
  \right\} \, {\vec{f}_{j}} \\
- \end{split}
+ \end{aligned}
  \end{equation*}
  %
  where $H_{kj} = \partial {\cal H}_k / \partial v_{j} $ is the
-Line 557 
 Because of the local character of the de
+Line 568 
 Because of the local character of the de
  (a derivative is defined w.r.t. a point along the trajectory),
  the intermediate results of the model trajectory
  $\vec{v}^{(\lambda+1)}={\cal M}_{\lambda}(v^{(\lambda)})$
- are needed to evaluate the intermediate Jacobian
+ may be required to evaluate the intermediate Jacobian
  $M_{\lambda}|_{\vec{v}^{(\lambda)}} \, \delta \vec{v}^{(\lambda)} $.
+ This is the case e.g. for nonlinear expressions
+ (momentum advection, nonlinear equation of state), state-dependent
+ conditional statements (parameterization schemes).
  In the forward mode, the intermediate results are required
  in the same order as computed by the full forward model ${\cal M}$,
  but in the reverse mode they are required in the reverse order.
-Line 569 
 point of evaluation has to be recomputed
+Line 583 
 point of evaluation has to be recomputed
  A method to balance the amount of recomputations vs.
  storage requirements is called {\sf checkpointing}
- (e.g. \cite{res-eta:98}).
+ (e.g. \cite{gri:92}, \cite{res-eta:98}).
  It is depicted in \ref{fig:3levelcheck} for a 3-level checkpointing
  [as an example, we give explicit numbers for a 3-day
  integration with a 1-hourly timestep in square brackets].
-Line 580 
 In a first step, the model trajectory is
+Line 594 
 In a first step, the model trajectory is
  $ {n}^{lev3} $ subsections [$ {n}^{lev3} $=3 1-day intervals],
  with the label $lev3$ for this outermost loop.
  The model is then integrated along the full trajectory,
- and the model state stored only at every $ k_{i}^{lev3} $-th timestep
+ and the model state stored to disk only at every $ k_{i}^{lev3} $-th timestep
  [i.e. 3 times, at
  $ i = 0,1,2 $ corresponding to $ k_{i}^{lev3} = 0, 24, 48 $].
+ In addition, the cost function is computed, if needed.
  %
  \item [$lev2$]
  In a second step each subsection itself is divided into
- $ {n}^{lev2} $ sub-subsections
+ $ {n}^{lev2} $ subsections
  [$ {n}^{lev2} $=4 6-hour intervals per subsection].
  The model picks up at the last outermost dumped state
  $ v_{k_{n}^{lev3}} $ and is integrated forward in time along
  the last subsection, with the label $lev2$ for this
  intermediate loop.
- The model state is now stored at every $ k_{i}^{lev2} $-th
+ The model state is now stored to disk at every $ k_{i}^{lev2} $-th
  timestep
  [i.e. 4 times, at
  $ i = 0,1,2,3 $ corresponding to $ k_{i}^{lev2} = 48, 54, 60, 66 $].
-Line 600 
 $ i = 0,1,2,3 $ corresponding to $ k_{i}
+Line 615 
 $ i = 0,1,2,3 $ corresponding to $ k_{i}
  \item [$lev1$]
  Finally, the model picks up at the last intermediate dump state
  $ v_{k_{n}^{lev2}} $ and is integrated forward in time along
- the last sub-subsection, with the label $lev1$ for this
+ the last subsection, with the label $lev1$ for this
  intermediate loop.
- Within this sub-subsection only, the model state is stored
+ Within this sub-subsection only, parts of the model state is stored
- at every timestep
+ to memory at every timestep
  [i.e. every hour $ i=0,...,5$ corresponding to
  $ k_{i}^{lev1} = 66, 67, \ldots, 71 $].
- Thus, the  final state $ v_n = v_{k_{n}^{lev1}} $ is reached
+ The  final state $ v_n = v_{k_{n}^{lev1}} $ is reached
- and the model state of all proceeding timesteps along the last
+ and the model state of all preceding timesteps along the last
- sub-subsections are available, enabling integration backwards
+ innermost subsection are available, enabling integration backwards
- in time along the last sub-subsection.
+ in time along the last subsection.
- Thus, the adjoint can be computed along this last
+ The adjoint can thus be computed along this last
- sub-subsection $k_{n}^{lev2}$.
+ subsection $k_{n}^{lev2}$.
  %
  \end{itemize}
  %
  This procedure is repeated consecutively for each previous
- sub-subsection $k_{n-1}^{lev2}, \ldots, k_{1}^{lev2} $
+ subsection $k_{n-1}^{lev2}, \ldots, k_{1}^{lev2} $
  carrying the adjoint computation to the initial time
  of the subsection $k_{n}^{lev3}$.
  Then, the procedure is repeated for the previous subsection
-Line 627 
 $k_{1}^{lev3}$.
+Line 642 
 $k_{1}^{lev3}$.
  For the full model trajectory of
  $ n^{lev3} \cdot n^{lev2} \cdot n^{lev1} $ timesteps
  the required storing of the model state was significantly reduced to
- $ n^{lev1} + n^{lev2} + n^{lev3} $
+ $ n^{lev2} + n^{lev3} $ to disk and roughly $ n^{lev1} $ to memory
  [i.e. for the 3-day integration with a total oof 72 timesteps
- the model state was stored 13 times].
+ the model state was stored 7 times to disk and roughly 6 times
+ to memory].
  This saving in memory comes at a cost of a required
 full forward integrations of the model (one for each
  checkpointing level).
- The balance of storage vs. recomputation certainly depends
+ The optimal balance of storage vs. recomputation certainly depends
- on the computing resources available.
+ on the computing resources available and may be adjusted by
+ adjusting the partitioning among the
+ $ n^{lev3}, \,\, n^{lev2}, \,\, n^{lev1} $.
  \begin{figure}[t!]
  \begin{center}
-Line 647 
 on the computing resources available.
+Line 665 
 on the computing resources available.
  %\psfrag{v_kn^lev2}{\mathinfigure{v_{k_{n}^{lev2}}}}
  %\psfrag{v_k1^lev1}{\mathinfigure{v_{k_{1}^{lev1}}}}
  %\psfrag{v_kn^lev1}{\mathinfigure{v_{k_{n}^{lev1}}}}
- %\mbox{\epsfig{file=part5/checkpointing.eps, width=0.8\textwidth}}
+ %\mbox{\epsfig{file=s_autodiff/figs/checkpointing.eps, width=0.8\textwidth}}
- \resizebox{5.5in}{!}{\includegraphics{part5/checkpointing.eps}}
+ \resizebox{5.5in}{!}{\includegraphics{s_autodiff/figs/checkpointing.eps}}
  %\psfull
  \end{center}
  \caption{
-Line 669 
 Schematic view of intermediate dump and
+Line 687 
 Schematic view of intermediate dump and
  %**********************************************************************
  \section{TLM and ADM generation in general}
  \label{sec_ad_setup_gen}
+ \begin{rawhtml}
+ <!-- CMIREDIR:sec_ad_setup_gen: -->
+ \end{rawhtml}
  %**********************************************************************
  In this section we describe in a general fashion
  the parts of the code that are relevant for automatic
- differentiation using the software tool TAMC.
+ differentiation using the software tool TAF.
+ Modifications to use OpenAD are described in \ref{sec_ad_openad}.
- \input{part5/doc_ad_the_model}
+ \input{s_autodiff/text/doc_ad_the_model}
  The basic flow is depicted in \ref{fig:adthemodel}.
- If the option {\tt ALLOW\_AUTODIFF\_TAMC} is defined, the driver routine
+ If CPP option \texttt{ALLOW\_AUTODIFF\_TAMC} is defined,
+ the driver routine
  {\it the\_model\_main}, instead of calling {\it the\_main\_loop},
- invokes the adjoint of this routine, {\it adthe\_main\_loop},
+ invokes the adjoint of this routine, {\it adthe\_main\_loop}
- which is the toplevel routine in terms of reverse mode computation.
+ (case \texttt{\#define ALLOW\_ADJOINT\_RUN}), or
- The routine {\it adthe\_main\_loop} has been generated using TAMC.
+ the tangent linear of this routine {\it g\_the\_main\_loop}
- It contains both the forward integration of the full model,
+ (case \texttt{\#define ALLOW\_TANGENTLINEAR\_RUN}),
+ which are the toplevel routines in terms of automatic differentiation.
+ The routines {\it adthe\_main\_loop} or {\it g\_the\_main\_loop}
+ are generated by TAF.
+ It contains both the forward integration of the full model, the
+ cost function calculation,
  any additional storing that is required for efficient checkpointing,
  and the reverse integration of the adjoint model.
- The structure of {\it adthe\_main\_loop} has been strongly
- simplified for clarification; in particular, no checkpointing
+ [DESCRIBE IN A SEPARATE SECTION THE WORKING OF THE TLM]
+ In Fig. \ref{fig:adthemodel}
+ the structure of {\it adthe\_main\_loop} has been strongly
+ simplified to focus on the essentials; in particular, no checkpointing
  procedures are shown here.
  Prior to the call of {\it adthe\_main\_loop}, the routine
- {\it ctrl\_unpack} is invoked to unpack the control vector,
+ {\it ctrl\_unpack} is invoked to unpack the control vector
- and following that call, the routine {\it ctrl\_pack}
+ or initialise the control variables.
+ Following the call of {\it adthe\_main\_loop},
+ the routine {\it ctrl\_pack}
  is invoked to pack the control vector
  (cf. Section \ref{section_ctrl}).
  If gradient checks are to be performed, the option
  {\tt ALLOW\_GRADIENT\_CHECK} is defined. In this case
  the driver routine {\it grdchk\_main} is called after
  the gradient has been computed via the adjoint
- (cf. Section \ref{section_grdchk}).
+ (cf. Section \ref{sec:ad_gradient_check}).
+ %------------------------------------------------------------------
+ \subsection{General setup
+ \label{section_ad_setup}}
+ In order to configure AD-related setups the following packages need
+ to be enabled:
+ {\it
+ \begin{table}[!ht]
+ \begin{tabular}{l}
+ autodiff \\
+ ctrl \\
+ cost \\
+ grdchk \\
+ \end{tabular}
+ \end{table}
+ }
+ The packages are enabled by adding them to your experiment-specific
+ configuration file
+ {\it packages.conf} (see Section ???).
+ The following AD-specific CPP option files need to be customized:
+ %
+ \begin{itemize}
+ %
+ \item {\it ECCO\_CPPOPTIONS.h} \\
+ This header file collects CPP options for the packages
+ {\it autodiff, cost, ctrl} as well as AD-unrelated options for
+ the external forcing package {\it exf}.
+ \footnote{NOTE: These options are not set in their package-specific
+ headers such as {\it COST\_CPPOPTIONS.h}, but are instead collected
+ in the single header file {\it ECCO\_CPPOPTIONS.h}.
+ The package-specific header files serve as simple
+ placeholders at this point.}
+ %
+ \item {\it tamc.h} \\
+ This header configures the splitting of the time stepping loop
+ w.r.t. the 3-level checkpointing (see section ???).
+ %
+ \end{itemize}
+ %------------------------------------------------------------------
+ \subsection{Building the AD code using TAF
+ \label{section_ad_build}}
+ The build process of an AD code is very similar to building
+ the forward model. However, depending on which AD code one wishes
+ to generate, and on which AD tool is available (TAF or TAMC),
+ the following {\tt make} targets are available:
+ \begin{table}[!ht]
+ {\footnotesize
+ \begin{tabular}{|ccll|}
+ \hline
+ ~ & {\it AD-target} & {\it output} & {\it description} \\
+ \hline
+ \hline
+ (1) & {\tt <MODE><TOOL>only} & {\tt <MODE>\_<TOOL>\_output.f}  &
+ generates code for $<$MODE$>$ using $<$TOOL$>$ \\
+ ~ & ~ & ~ & no {\tt make} dependencies on {\tt .F .h} \\
+ ~ & ~ & ~ & useful for compiling on remote platforms \\
+ \hline
+ (2) & {\tt <MODE><TOOL>} & {\tt <MODE>\_<TOOL>\_output.f}  &
+ generates code for $<$MODE$>$ using $<$TOOL$>$ \\
+ ~ & ~ & ~ & includes {\tt make} dependencies on {\tt .F .h} \\
+ ~ & ~ & ~ & i.e. input for $<$TOOL$>$ may be re-generated \\
+ \hline
+ (3) & {\tt <MODE>all} & {\tt mitgcmuv\_<MODE>}  &
+ generates code for $<$MODE$>$ using $<$TOOL$>$ \\
+ ~ & ~ & ~ & and compiles all code \\
+ ~ & ~ & ~ & (use of TAF is set as default) \\
+ \hline
+ \end{tabular}
+ }
+ \end{table}
+ %
+ Here, the following placeholders are used
+ %
+ \begin{itemize}
+ %
+ \item $<$TOOL$>$
+ %
+ \begin{itemize}
+ %
+ \item {\tt TAF}
+ \item {\tt TAMC}
+ %
+ \end{itemize}
+ %
+ \item $<$MODE$>$
+ %
+ \begin{itemize}
+ %
+ \item {\tt ad} generates the adjoint model (ADM)
+ \item {\tt ftl} generates the tangent linear model (TLM)
+ \item {\tt svd} generates both ADM and TLM for \\
+ singular value decomposition (SVD) type calculations
+ %
+ \end{itemize}
+ %
+ \end{itemize}
+ For example, to generate the adjoint model using TAF after routines ({\tt .F})
+ or headers ({\tt .h}) have been modified, but without compilation,
+ type {\tt make adtaf};
+ or, to generate the tangent linear model using TAMC without
+ re-generating the input code, type {\tt make ftltamconly}.
+ A typical full build process to generate the ADM via TAF would
+ look like follows:
+ \begin{verbatim}
+ % mkdir build
+ % cd build
+ % ../../../tools/genmake2 -mods=../code_ad
+ % make depend
+ % make adall
+ \end{verbatim}
+ %------------------------------------------------------------------
+ \subsection{The AD build process in detail
+ \label{section_ad_build_detail}}
+ The {\tt make <MODE>all} target consists of the following procedures:
+ \begin{enumerate}
+ %
+ \item
+ A header file {\tt AD\_CONFIG.h} is generated which contains a CPP option
+ on which code ought to be generated. Depending on the {\tt make} target,
+ the contents is one of the following:
+ \begin{itemize}
+ \item
+ {\tt \#define ALLOW\_ADJOINT\_RUN}
+ \item
+ {\tt \#define ALLOW\_TANGENTLINEAR\_RUN}
+ \item
+ {\tt \#define ALLOW\_ECCO\_OPTIMIZATION}
+ \end{itemize}
+ %
+ \item
+ A single file {\tt <MODE>\_input\_code.f} is concatenated
+ consisting of all {\tt .f} files that are part of the list {\bf AD\_FILES}
+ and all {\tt .flow} files that are part of the list {\bf AD\_FLOW\_FILES}.
+ %
+ \item
+ The AD tool is invoked with the {\tt <MODE>\_<TOOL>\_FLAGS}.
+ The default AD tool flags in {\tt genmake2} can be overrwritten by
+ an {\tt adjoint\_options} file (similar to the platform-specific
+ {\tt build\_options}, see Section ???.
+ The AD tool writes the resulting AD code into the file
+ {\tt <MODE>\_input\_code\_ad.f}
+ %
+ \item
+ A short sed script {\tt adjoint\_sed} is applied to
+ {\tt <MODE>\_input\_code\_ad.f}
+ to reinstate {\bf myThid} into the CALL argument list of active file I/O.
+ The result is written to file {\tt <MODE>\_<TOOL>\_output.f}.
+ %
+ \item
+ All routines are compiled and an executable is generated
+ (see Table ???).
+ %
+ \end{enumerate}
+ \subsubsection{The list AD\_FILES and {\tt .list} files}
+ Not all routines are presented to the AD tool.
+ Routines typically hidden are diagnostics routines which
+ do not influence the cost function, but may create
+ artificial flow dependencies such as I/O of active variables.
+ {\tt genmake2} generates a list (or variable) {\bf AD\_FILES}
+ which contains all routines that are shown to the AD tool.
+ This list is put together from all files with suffix {\tt .list}
+ that {\tt genmake2} finds in its search directories.
+ The list file for the core MITgcm routines is in {\tt model/src/}
+ is called {\tt model\_ad\_diff.list}.
+ Note that no wrapper routine is shown to TAF. These are either
+ not visible at all to the AD code, or hand-written AD code
+ is available (see next section).
+ Each package directory contains its package-specific
+ list file {\tt <PKG>\_ad\_diff.list}. For example,
+ {\tt pkg/ptracers/} contains the file {\tt ptracers\_ad\_diff.list}.
+ Thus, enabling a package will automatically extend the
+ {\bf AD\_FILES} list of {\tt genmake2} to incorporate the
+ package-specific routines.
+ Note that you will need to regenerate the {\tt Makefile} if
+ you enable a package (e.g. by adding it to {\tt packages.conf})
+ and a {\tt Makefile} already exists.
+ \subsubsection{The list AD\_FLOW\_FILES and {\tt .flow} files}
+ TAMC and TAF can evaluate user-specified directives
+ that start with a specific syntax ({\tt CADJ}, {\tt C\$TAF}, {\tt !\$TAF}).
+ The main categories of directives are STORE directives and
+ FLOW directives. Here, we are concerned with flow directives,
+ store directives are treated elsewhere.
+ Flow directives enable the AD tool to evaluate how it should treat
+ routines that are 'hidden' by the user, i.e. routines which are
+ not contained in the {\bf AD\_FILES} list (see previous section),
+ but which are called in part of the code that the AD tool does see.
+ The flow directive tell the AD tool
+ %
+ \begin{itemize}
+ %
+ \item which subroutine arguments are input/output
+ \item which subroutine arguments are active
+ \item which subroutine arguments are required to compute the cost
+ \item which subroutine arguments are dependent
+ %
+ \end{itemize}
+ %
+ The syntax for the flow directives can be found in the
+ AD tool manuals.
+ {\tt genmake2} generates a list (or variable) {\bf AD\_FLOW\_FILES}
+ which contains all files with suffix{\tt .flow} that it finds
+ in its search directories.
+ The flow directives for the core MITgcm routines of
+ {\tt eesupp/src/} and {\tt model/src/}
+ reside in {\tt pkg/autodiff/}.
+ This directory also contains hand-written adjoint code
+ for the MITgcm WRAPPER (section \ref{chap:sarch}).
+ Flow directives for package-specific routines are contained in
+ the corresponding package directories in the file
+ {\tt <PKG>\_ad.flow}, e.g. ptracers-specific directives are in
+ {\tt ptracers\_ad.flow}.
+ \subsubsection{Store directives for 3-level checkpointing}
+ The storing that is required at each period of the
+-level checkpointing is controled by three
+ top-level headers.
+ \begin{verbatim}
+ do ilev_3 = 1, nchklev_3
+ #  include ``checkpoint_lev3.h''
+    do ilev_2 = 1, nchklev_2
+ #     include ``checkpoint_lev2.h''
+       do ilev_1 = 1, nchklev_1
+ #        include ``checkpoint_lev1.h''
+ ...
+       end do
+    end do
+ end do
+ \end{verbatim}
+ All files {\tt checkpoint\_lev?.h} are contained in directory
+ {\tt pkg/autodiff/}.
+ \subsubsection{Changing the default AD tool flags: ad\_options files}
+ \subsubsection{Hand-written adjoint code}
+ %------------------------------------------------------------------
  \subsection{The cost function (dependent variable)
  \label{section_cost}}
-Line 706 
 the gradient has been computed via the a
+Line 1007 
 the gradient has been computed via the a
  The cost function $ {\cal J} $ is referred to as the {\sf dependent variable}.
  It is a function of the input variables $ \vec{u} $ via the composition
  $ {\cal J}(\vec{u}) \, = \, {\cal J}(M(\vec{u})) $.
- The input is referred to as the
+ The input are referred to as the
  {\sf independent variables} or {\sf control variables}.
  All aspects relevant to the treatment of the cost function $ {\cal J} $
  (parameter setting, initialization, accumulation,
  final evaluation), are controlled by the package {\it pkg/cost}.
+ The aspects relevant to the treatment of the independent variables
+ are controlled by the package {\it pkg/ctrl} and will be treated
+ in the next section.
- \input{part5/doc_cost_flow}
+ \input{s_autodiff/text/doc_cost_flow}
+ \subsubsection{Enabling the package}
- \subsubsection{genmake and CPP options}
- %
- \begin{itemize}
- %
- \item
  \fbox{
  \begin{minipage}{12cm}
- {\it genmake}, {\it CPP\_OPTIONS.h}, {\it ECCO\_CPPOPTIONS.h}
+ {\it packages.conf}, {\it ECCO\_CPPOPTIONS.h}
  \end{minipage}
  }
- \end{itemize}
+ \begin{itemize}
- %
- The directory {\it pkg/cost} can be included to the
- compile list in 3 different ways (cf. Section \ref{???}):
  %
- \begin{enumerate}
+ \item
+ The package is enabled by adding {\it cost} to your file {\it packages.conf}
+ (see Section ???)
  %
- \item {\it genmake}: \\
+ \item
- Change the default settings in the file {\it genmake} by adding
- {\bf cost} to the {\bf enable} list (not recommended).
- %
+ \end{itemize}
- \item {\it .genmakerc}: \\
- Customize the settings of {\bf enable}, {\bf disable} which are
- appropriate for your experiment in the file {\it .genmakerc}
- and add the file to your compile directory.
- %
- \item genmake-options: \\
- Call {\it genmake} with the option
- {\tt genmake -enable=cost}.
  %
- \end{enumerate}
+ N.B.: In general the following packages ought to be enabled
+ simultaneously: {\it autodiff, cost, ctrl}.
  The basic CPP option to enable the cost function is {\bf ALLOW\_COST}.
  Each specific cost function contribution has its own option.
  For the present example the option is {\bf ALLOW\_COST\_TRACER}.
  All cost-specific options are set in {\it ECCO\_CPPOPTIONS.h}
  Since the cost function is usually used in conjunction with
  automatic differentiation, the CPP option
- {\bf ALLOW\_ADJOINT\_RUN} should be defined
+ {\bf ALLOW\_ADJOINT\_RUN} (file {\it CPP\_OPTIONS.h}) and
- (file {\it CPP\_OPTIONS.h}).
+ {\bf ALLOW\_AUTODIFF\_TAMC} (file {\it ECCO\_CPPOPTIONS.h})
+ should be defined.
  \subsubsection{Initialization}
  %
  The initialization of the {\it cost} package is readily enabled
- as soon as the CPP option {\bf ALLOW\_ADJOINT\_RUN} is defined.
+ as soon as the CPP option {\bf ALLOW\_COST} is defined.
  %
  \begin{itemize}
  %
-Line 811 
 Within this 'driver' routine, S/R are ca
+Line 1106 
 Within this 'driver' routine, S/R are ca
  the chosen cost function contributions.
  In the present example ({\bf ALLOW\_COST\_TRACER}),
  S/R {\it cost\_tracer} is called.
- It accumulates {\bf objf\_tracer} according to eqn. (\ref{???}).
+ It accumulates {\bf objf\_tracer} according to eqn. (\ref{ask_the_author:doc_ad_2}).
  %
  \subsubsection{Finalize all contributions}
  %
-Line 831 
 from each contribution and sums over all
+Line 1126 
 from each contribution and sums over all
  \begin{equation}
  {\cal J} \, = \,
  {\rm fc} \, = \,
- {\rm mult\_tracer} \sum_{bi,\,bj}^{nSx,\,nSy}
+ {\rm mult\_tracer} \sum_{\text{global sum}} \sum_{bi,\,bj}^{nSx,\,nSy}
  {\rm objf\_tracer}(bi,bj) \, + \, ...
  \end{equation}
  %
  The total cost function {\bf fc} will be the
- 'dependent' variable in the argument list for TAMC, i.e.
+ 'dependent' variable in the argument list for TAF, i.e.
  \begin{verbatim}
- tamc -output 'fc' ...
+ taf -output 'fc' ...
  \end{verbatim}
  %%%% \end{document}
- \input{part5/doc_ad_the_main}
+ \input{s_autodiff/text/doc_ad_the_main}
  \subsection{The control variables (independent variables)
  \label{section_ctrl}}
-Line 863 
 All aspects relevant to the treatment of
+Line 1158 
 All aspects relevant to the treatment of
  (parameter setting, initialization, perturbation)
  are controlled by the package {\it pkg/ctrl}.
- \input{part5/doc_ctrl_flow}
+ \input{s_autodiff/text/doc_ctrl_flow}
  \subsubsection{genmake and CPP options}
  %
-Line 879 
 are controlled by the package {\it pkg/c
+Line 1174 
 are controlled by the package {\it pkg/c
  %
  To enable the directory to be included to the compile list,
  {\bf ctrl} has to be added to the {\bf enable} list in
- {\it .genmakerc} (or {\it genmake} itself).
+ {\it .genmakerc} or in {\it genmake} itself (analogous to {\it cost}
+ package, cf. previous section).
  Each control variable is enabled via its own CPP option
  in {\it ECCO\_CPPOPTIONS.h}.
-Line 920 
 and their gradients: {\it ctrl\_unpack}
+Line 1216 
 and their gradients: {\it ctrl\_unpack}
  \\
  %
  Two important issues related to the handling of the control
- variables in the MITGCM need to be addressed.
+ variables in MITgcm need to be addressed.
  First, in order to save memory, the control variable arrays
  are not kept in memory, but rather read from file and added
  to the initial fields during the model initialization phase.
-Line 952 
 and gradient are generated and initialis
+Line 1248 
 and gradient are generated and initialis
  %
  The dependency flow for differentiation w.r.t. the controls
  starts with adding a perturbation onto the input variable,
- thus defining the independent or control variables for TAMC.
+ thus defining the independent or control variables for TAF.
  Three types of controls may be considered:
  %
  \begin{itemize}
-Line 973 
 temperature and salinity are initialised
+Line 1269 
 temperature and salinity are initialised
  a perturbation anomaly is added to the field in S/R
  {\it ctrl\_map\_ini}
  %
+ %\begin{eqnarray}
  \begin{equation}
- \begin{split}
+ \begin{aligned}
  u         & = \, u_{[0]} \, + \, \Delta u \\
  {\bf tr1}(...) & = \, {\bf tr1_{ini}}(...) \, + \, {\bf xx\_tr1}(...)
  \label{perturb}
- \end{split}
+ \end{aligned}
  \end{equation}
+ %\end{eqnarray}
  %
  {\bf xx\_tr1} is a 3-dim. global array
  holding the perturbation. In the case of a simple
  sensitivity study this array is identical to zero.
  However, it's specification is essential in the context
- of automatic differentiation since TAMC
+ of automatic differentiation since TAF
  treats the corresponding line in the code symbolically
  when determining the differentiation chain and its origin.
  Thus, the variable names are part of the argument list
- when calling TAMC:
+ when calling TAF:
  %
  \begin{verbatim}
- tamc -input 'xx_tr1 ...' ...
+ taf -input 'xx_tr1 ...' ...
  \end{verbatim}
  %
- Now, as mentioned above, the MITGCM avoids maintaining
+ Now, as mentioned above, MITgcm avoids maintaining
  an array for each control variable by reading the
  perturbation to a temporary array from file.
- To ensure the symbolic link to be recognized by TAMC, a scalar
+ To ensure the symbolic link to be recognized by TAF, a scalar
  dummy variable {\bf xx\_tr1\_dummy} is introduced
  and an 'active read' routine of the adjoint support
  package {\it pkg/autodiff} is invoked.
  The read-procedure is tagged with the variable
- {\bf xx\_tr1\_dummy} enabling TAMC to recognize the
+ {\bf xx\_tr1\_dummy} enabling TAF to recognize the
  initialization of the perturbation.
- The modified call of TAMC thus reads
+ The modified call of TAF thus reads
  %
  \begin{verbatim}
- tamc -input 'xx_tr1_dummy ...' ...
+ taf -input 'xx_tr1_dummy ...' ...
  \end{verbatim}
  %
  and the modified operation to (\ref{perturb})
-Line 1023 
 in the code takes on the form
+Line 1321 
 in the code takes on the form
  %
  Note, that reading an active variable corresponds
  to a variable assignment. Its derivative corresponds
- to a write statement of the adjoint variable.
+ to a write statement of the adjoint variable, followed by
+ a reset.
  The 'active file' routines have been designed
  to support active read and corresponding adjoint active write
  operations (and vice versa).
-Line 1140 
 at intermediate times can be written usi
+Line 1439 
 at intermediate times can be written usi
  {\it addummy\_in\_stepping}.
  This routine is part of the adjoint support package
  {\it pkg/autodiff} (cf.f. below).
+ The procedure is enabled using via the CPP-option
+ {\bf ALLOW\_AUTODIFF\_MONITOR} (file {\it ECCO\_CPPOPTIONS.h}).
  To be part of the adjoint code, the corresponding S/R
  {\it dummy\_in\_stepping} has to be called in the forward
  model (S/R {\it the\_main\_loop}) at the appropriate place.
+ The adjoint common blocks are extracted from the adjoint code
+ via the header file {\it adcommon.h}.
  {\it dummy\_in\_stepping} is essentially empty,
  the corresponding adjoint routine is hand-written rather
-Line 1169 
 the common blocks
+Line 1472 
 the common blocks
  {\bf /adtr1\_r/}, {\bf /adffields/},
  which have been extracted from the adjoint code to enable
  access to the adjoint variables.
+ {\bf WARNING:} If the structure of the common blocks
+ {\bf /dynvars\_r/}, {\bf /dynvars\_cd/}, etc., changes
+ similar changes will occur in the adjoint common blocks.
+ Therefore, consistency between the TAMC-generated common blocks
+ and those in {\it adcommon.h} have to be checked.
  %
  \end{itemize}
-Line 1194 
 u_{[k+1]} \, = \,  u_{[0]} \, + \, \Delt
+Line 1503 
 u_{[k+1]} \, = \,  u_{[0]} \, + \, \Delt
  $ u_{[k+1]} $ then serves as input for a forward/adjoint run
  to determine $ {\cal J} $ and $ \nabla _{u}{\cal J} $ at iteration step
  $ k+1 $.
- Tab. \ref{???} sketches the flow between forward/adjoint model
+ Tab. \ref{ask_the_author:doc_ad_2} sketches the flow between forward/adjoint model
  and the minimization routine.
+ {\scriptsize
  \begin{eqnarray*}
- \scriptsize
  \begin{array}{ccccc}
  u_{[0]} \,\, ,  \,\, \Delta u_{[k]}    & ~ & ~ & ~ & ~ \\
  {\Big\downarrow}
-Line 1249 
 ad \, v_{[k]} (\delta {\cal J}) =
+Line 1558 
 ad \, v_{[k]} (\delta {\cal J}) =
   ~ & ~ & ~ & ~ & \Delta u_{[k+1]} \\
  \end{array}
  \end{eqnarray*}
+ }
  The routines {\it ctrl\_unpack} and {\it ctrl\_pack} provide
  the link between the model and the minimization routine.
- As described in Section \ref{???}
+ As described in Section \ref{ask_the_author:doc_ad_2}
  the {\it unpack} and {\it pack} routines read and write
  control and gradient {\it vectors} which are compressed
  to contain only wet points, in addition to the full

 Legend:



Removed from v.1.14
 


changed lines


 
Added in v.1.23
 Legend:



Removed from v.1.14
 


changed lines


 
Added in v.1.23
-Removed from v.1.14
+Added in v.1.23

	ViewVC Help
Powered by ViewVC 1.1.22