/[MITgcm]/manual/s_software/text/sarch.tex

Diff of /manual/s_software/text/sarch.tex

Parent Directory | Revision Log | View Revision Graph Revision Graph | View Patch Patch

-revision 1.21 by edhill,
Tue Apr  4 15:54:55 2006 UTC
+revision 1.26 by jmc,
Mon Aug 30 23:09:22 2010 UTC
 Line 1
  % $Header$
- This chapter focuses on describing the {\bf WRAPPER} environment within which
+ This chapter focuses on describing the {\bf WRAPPER} environment
- both the core numerics and the pluggable packages operate. The description
+ within which both the core numerics and the pluggable packages
- presented here is intended to be a detailed exposition and contains significant
+ operate. The description presented here is intended to be a detailed
- background material, as well as advanced details on working with the WRAPPER.
+ exposition and contains significant background material, as well as
- The tutorial sections of this manual (see sections
+ advanced details on working with the WRAPPER.  The tutorial sections
- \ref{sect:tutorials}  and \ref{sect:tutorialIII})
+ of this manual (see sections \ref{sec:modelExamples} and
- contain more succinct, step-by-step instructions on running basic numerical
+ \ref{sec:tutorialIII}) contain more succinct, step-by-step
- experiments, of varous types, both sequentially and in parallel. For many
+ instructions on running basic numerical experiments, of varous types,
- projects simply starting from an example code and adapting it to suit a
+ both sequentially and in parallel. For many projects simply starting
- particular situation
+ from an example code and adapting it to suit a particular situation
- will be all that is required.
+ will be all that is required.  The first part of this chapter
- The first part of this chapter discusses the MITgcm architecture at an
+ discusses the MITgcm architecture at an abstract level. In the second
- abstract level. In the second part of the chapter we described practical
+ part of the chapter we described practical details of the MITgcm
- details of the MITgcm implementation and of current tools and operating system
+ implementation and of current tools and operating system features that
- features that are employed.
+ are employed.
  \section{Overall architectural goals}
  \begin{rawhtml}
 Line 25 
 Broadly, the goals of the software archi
  three-fold
  \begin{itemize}
- \item We wish to be able to study a very broad range
+ \item We wish to be able to study a very broad range of interesting
- of interesting and challenging rotating fluids problems.
+   and challenging rotating fluids problems.
- \item We wish the model code to be readily targeted to
+ \item We wish the model code to be readily targeted to a wide range of
- a wide range of platforms
+   platforms
- \item On any given platform we would like to be
+ \item On any given platform we would like to be able to achieve
- able to achieve performance comparable to an implementation
+   performance comparable to an implementation developed and
- developed and specialized specifically for that platform.
+   specialized specifically for that platform.
  \end{itemize}
- These points are summarized in figure \ref{fig:mitgcm_architecture_goals}
+ These points are summarized in figure
- which conveys the goals of the MITgcm design. The goals lead to
+ \ref{fig:mitgcm_architecture_goals} which conveys the goals of the
- a software architecture which at the high-level can be viewed as consisting
+ MITgcm design. The goals lead to a software architecture which at the
- of
+ high-level can be viewed as consisting of
  \begin{enumerate}
  \item A core set of numerical and support code. This is discussed in
 Line 69 
 is required.
  \begin{figure}
  \begin{center}
- \resizebox{!}{2.5in}{\includegraphics{part4/mitgcm_goals.eps}}
+ \resizebox{!}{2.5in}{\includegraphics{s_software/figs/mitgcm_goals.eps}}
  \end{center}
- \caption{
+ \caption{ The MITgcm architecture is designed to allow simulation of a
- The MITgcm architecture is designed to allow simulation of a wide
+   wide range of physical problems on a wide range of hardware. The
- range of physical problems on a wide range of hardware. The computational
+   computational resource requirements of the applications targeted
- resource requirements of the applications targeted range from around
+   range from around $10^7$ bytes ($\approx 10$ megabytes) of memory to
- $10^7$ bytes ( $\approx 10$ megabytes ) of memory to $10^{11}$ bytes
+   $10^{11}$ bytes ($\approx 100$ gigabytes). Arithmetic operation
- ( $\approx 100$ gigabytes). Arithmetic operation counts for the applications of
+   counts for the applications of interest range from $10^{9}$ floating
- interest range from $10^{9}$ floating point operations to more than $10^{17}$
+   point operations to more than $10^{17}$ floating point operations.}
- floating point operations.}
  \label{fig:mitgcm_architecture_goals}
  \end{figure}
-Line 87 
 floating point operations.}
+Line 86 
 floating point operations.}
  <!-- CMIREDIR:wrapper: -->
  \end{rawhtml}
- A significant element of the software architecture utilized in
+ A significant element of the software architecture utilized in MITgcm
- MITgcm is a software superstructure and substructure collectively
+ is a software superstructure and substructure collectively called the
- called the WRAPPER (Wrappable Application Parallel Programming
+ WRAPPER (Wrappable Application Parallel Programming Environment
- Environment Resource). All numerical and support code in MITgcm is written
+ Resource). All numerical and support code in MITgcm is written to
- to ``fit'' within the WRAPPER infrastructure. Writing code to ``fit'' within
+ ``fit'' within the WRAPPER infrastructure. Writing code to ``fit''
- the WRAPPER means that coding has to follow certain, relatively
+ within the WRAPPER means that coding has to follow certain, relatively
  straightforward, rules and conventions (these are discussed further in
- section \ref{sect:specifying_a_decomposition}).
+ section \ref{sec:specifying_a_decomposition}).
  The approach taken by the WRAPPER is illustrated in figure
- \ref{fig:fit_in_wrapper} which shows how the WRAPPER serves to insulate code
+ \ref{fig:fit_in_wrapper} which shows how the WRAPPER serves to
- that fits within it from architectural differences between hardware platforms
+ insulate code that fits within it from architectural differences
- and operating systems. This allows numerical code to be easily retargetted.
+ between hardware platforms and operating systems. This allows
+ numerical code to be easily retargetted.
  \begin{figure}
  \begin{center}
- \resizebox{!}{4.5in}{\includegraphics{part4/fit_in_wrapper.eps}}
+ \resizebox{!}{4.5in}{\includegraphics{s_software/figs/fit_in_wrapper.eps}}
  \end{center}
  \caption{
  Numerical code is written to fit within a software support
 Line 118 
 optimized for that platform.}
  \end{figure}
  \subsection{Target hardware}
- \label{sect:target_hardware}
+ \label{sec:target_hardware}
  The WRAPPER is designed to target as broad as possible a range of
  computer systems.  The original development of the WRAPPER took place
 Line 136 
 routinely used on large scale MPP system
  IBM SP systems). In all cases numerical code, operating within the
  WRAPPER, performs and scales very competitively with equivalent
  numerical code that has been modified to contain native optimizations
- for a particular system \ref{ref hoe and hill, ecmwf}.
+ for a particular system \cite{hoe-hill:99}.
  \subsection{Supporting hardware neutrality}
- The different systems listed in section \ref{sect:target_hardware} can
+ The different systems listed in section \ref{sec:target_hardware} can
  be categorized in many different ways. For example, one common
  distinction is between shared-memory parallel systems (SMP and PVP)
  and distributed memory parallel systems (for example x86 clusters and
 Line 165 
 manner, any computer architecture curren
  scientific computing community.
  \subsection{Machine model parallelism}
+ \label{sec:domain_decomposition}
  \begin{rawhtml}
  <!-- CMIREDIR:domain_decomp: -->
  \end{rawhtml}
-Line 210 
 fashion.
+Line 211 
 fashion.
  \begin{figure}
  \begin{center}
   \resizebox{5in}{!}{
-   \includegraphics{part4/domain_decomp.eps}
+   \includegraphics{s_software/figs/domain_decomp.eps}
   }
  \end{center}
  \caption{ The WRAPPER provides support for one and two dimensional
-Line 239 
 an overlap region whenever it requires v
+Line 240 
 an overlap region whenever it requires v
  domain it owns.  Periodically processors will make calls to WRAPPER
  functions to communicate data between tiles, in order to keep the
  overlap regions up to date (see section
- \ref{sect:communication_primitives}).  The WRAPPER functions can use a
+ \ref{sec:communication_primitives}).  The WRAPPER functions can use a
  variety of different mechanisms to communicate data between tiles.
  \begin{figure}
  \begin{center}
   \resizebox{5in}{!}{
-   \includegraphics{part4/tiled-world.eps}
+   \includegraphics{s_software/figs/tiled-world.eps}
   }
  \end{center}
  \caption{ A global grid subdivided into tiles.
-Line 279 
 possible mechanisms.
+Line 280 
 possible mechanisms.
    call a function in the API of the communication library to
    communicate data from a tile that it owns to a tile that another CPU
    owns. By default the WRAPPER binds to the MPI communication library
-   \ref{MPI} for this style of communication.
+   \cite{MPI-std-20} for this style of communication.
  \end{itemize}
  The WRAPPER assumes that communication will use one of these two styles
-Line 328 
 value to be communicated between CPUs.
+Line 329 
 value to be communicated between CPUs.
  \end{figure}
  \subsection{Shared memory communication}
- \label{sect:shared_memory_communication}
+ \label{sec:shared_memory_communication}
  Under shared communication independent CPUs are operating on the
  exact same global address space at the application level.  This means
-Line 355 
 method of communication very efficient p
+Line 356 
 method of communication very efficient p
  appropriately.
  \subsubsection{Memory consistency}
- \label{sect:memory_consistency}
+ \label{sec:memory_consistency}
  When using shared memory communication between multiple processors the
  WRAPPER level shields user applications from certain counter-intuitive
-Line 381 
 invoke the appropriate mechanism to ensu
+Line 382 
 invoke the appropriate mechanism to ensu
  particular platform.
  \subsubsection{Cache effects and false sharing}
- \label{sect:cache_effects_and_false_sharing}
+ \label{sec:cache_effects_and_false_sharing}
  Shared-memory machines often have local to processor memory caches
  which contain mirrored copies of main memory.  Automatic cache-coherence
-Line 401 
 compute threads. Multiple threads operat
+Line 402 
 compute threads. Multiple threads operat
  the standard mechanism for supporting shared memory that the WRAPPER
  utilizes. Configuring and launching code to run in multi-threaded mode
  on specific platforms is discussed in section
- \ref{sect:multi-threaded-execution}.  However, on many systems,
+ \ref{sec:multi_threaded_execution}.  However, on many systems,
  potentially very efficient mechanisms for using shared memory
  communication between multiple processes (in contrast to multiple
  threads within a single process) also exist. In most cases this works
  by making a limited region of memory shared between processes. The
- MMAP \ref{magicgarden} and IPC \ref{magicgarden} facilities in UNIX
+ MMAP %\ref{magicgarden}
+ and IPC %\ref{magicgarden}
+ facilities in UNIX
  systems provide this capability as do vendor specific tools like LAPI
- \ref{IBMLAPI} and IMC \ref{Memorychannel}.  Extensions exist for the
+ %\ref{IBMLAPI}
+ and IMC. %\ref{Memorychannel}.
+ Extensions exist for the
  WRAPPER that allow these mechanisms to be used for shared memory
  communication. However, these mechanisms are not distributed with the
  default WRAPPER sources, because of their proprietary nature.
  \subsection{Distributed memory communication}
- \label{sect:distributed_memory_communication}
+ \label{sec:distributed_memory_communication}
  Many parallel systems are not constructed in a way where it is
- possible or practical for an application to use shared memory
+ possible or practical for an application to use shared memory for
- for communication. For example cluster systems consist of individual computers
+ communication. For example cluster systems consist of individual
- connected by a fast network. On such systems there is no notion of shared memory
+ computers connected by a fast network. On such systems there is no
- at the system level. For this sort of system the WRAPPER provides support
+ notion of shared memory at the system level. For this sort of system
- for communication based on a bespoke communication library
+ the WRAPPER provides support for communication based on a bespoke
- (see figure \ref{fig:comm_msg}).  The default communication library used is MPI
+ communication library (see figure \ref{fig:comm_msg}).  The default
- \ref{mpi}. However, it is relatively straightforward to implement bindings to
+ communication library used is MPI \cite{MPI-std-20}. However, it is
- optimized platform specific communication libraries. For example the work
+ relatively straightforward to implement bindings to optimized platform
- described in \ref{hoe-hill:99} substituted standard MPI communication for a
+ specific communication libraries. For example the work described in
- highly optimized library.
+ \cite{hoe-hill:99} substituted standard MPI communication for a highly
+ optimized library.
  \subsection{Communication primitives}
- \label{sect:communication_primitives}
+ \label{sec:communication_primitives}
  \begin{figure}
  \begin{center}
   \resizebox{5in}{!}{
-   \includegraphics{part4/comm-primm.eps}
+   \includegraphics{s_software/figs/comm-primm.eps}
   }
  \end{center}
  \caption{Three performance critical parallel primitives are provided
-Line 516 
 sub-domains.
+Line 522 
 sub-domains.
  \begin{figure}
  \begin{center}
   \resizebox{5in}{!}{
-   \includegraphics{part4/tiling_detail.eps}
+   \includegraphics{s_software/figs/tiling_detail.eps}
   }
  \end{center}
  \caption{The tiling strategy that the WRAPPER supports allows tiles
-Line 576 
 provided by the WRAPPER are
+Line 582 
 provided by the WRAPPER are
    computing CPUs.
  \end{enumerate}
  This section describes the details of each of these operations.
- Section \ref{sect:specifying_a_decomposition} explains how the way in
+ Section \ref{sec:specifying_a_decomposition} explains how the way in
  which a domain is decomposed (or composed) is expressed. Section
- \ref{sect:starting_a_code} describes practical details of running
+ \ref{sec:starting_the_code} describes practical details of running
  codes in various different parallel modes on contemporary computer
- systems.  Section \ref{sect:controlling_communication} explains the
+ systems.  Section \ref{sec:controlling_communication} explains the
  internal information that the WRAPPER uses to control how information
  is communicated between tiles.
  \subsection{Specifying a domain decomposition}
- \label{sect:specifying_a_decomposition}
+ \label{sec:specifying_a_decomposition}
  At its heart much of the WRAPPER works only in terms of a collection of tiles
  which are interconnected to each other. This is also true of application
-Line 622 
 not cause any other problems.
+Line 628 
 not cause any other problems.
  \begin{figure}
  \begin{center}
   \resizebox{5in}{!}{
-   \includegraphics{part4/size_h.eps}
+   \includegraphics{s_software/figs/size_h.eps}
   }
  \end{center}
  \caption{ The three level domain decomposition hierarchy employed by the
-Line 637 
 be created within a single process. Each
+Line 643 
 be created within a single process. Each
  dimensions of {\em sNx} and {\em sNy}. If, when the code is executed, these tiles are
  allocated to different threads of a process that are then bound to
  different physical processors ( see the multi-threaded
- execution discussion in section \ref{sect:starting_the_code} ) then
+ execution discussion in section \ref{sec:starting_the_code} ) then
  computation will be performed concurrently on each tile. However, it is also
  possible to run the same decomposition within a process running a single thread on
  a single processor. In this case the tiles will be computed over sequentially.
-Line 834 
 There are six tiles allocated to six sep
+Line 840 
 There are six tiles allocated to six sep
  This set of values can be used for a cube sphere calculation.
  Each tile of size $32 \times 32$ represents a face of the
  cube. Initializing the tile connectivity correctly ( see section
- \ref{sect:cube_sphere_communication}. allows the rotations associated with
+ \ref{sec:cube_sphere_communication}. allows the rotations associated with
  moving between the six cube faces to be embedded within the
  tile-tile communication code.
  \end{enumerate}
  \subsection{Starting the code}
- \label{sect:starting_the_code}
+ \label{sec:starting_the_code}
  When code is started under the WRAPPER, execution begins in a main routine {\em
  eesupp/src/main.F} that is owned by the WRAPPER. Control is transferred
  to the application through a routine called {\em THE\_MODEL\_MAIN()}
-Line 888 
 occurs through the procedure {\em THE\_M
+Line 894 
 occurs through the procedure {\em THE\_M
  \end{figure}
  \subsubsection{Multi-threaded execution}
- \label{sect:multi-threaded-execution}
+ \label{sec:multi_threaded_execution}
  Prior to transferring control to the procedure {\em THE\_MODEL\_MAIN()} the
  WRAPPER may cause several coarse grain threads to be initialized. The routine
  {\em THE\_MODEL\_MAIN()} is called once for each thread and is passed a single
  stack argument which is the thread number, stored in the
  variable {\em myThid}. In addition to specifying a decomposition with
- multiple tiles per process ( see section \ref{sect:specifying_a_decomposition})
+ multiple tiles per process ( see section \ref{sec:specifying_a_decomposition})
  configuring and starting a code to run using multiple threads requires the following
  steps.\\
-Line 976 
 Parameter:  {\em nTy}
+Line 982 
 Parameter:  {\em nTy}
  } \\
  \subsubsection{Multi-process execution}
- \label{sect:multi-process-execution}
+ \label{sec:multi_process_execution}
  Despite its appealing programming model, multi-threaded execution
  remains less common than multi-process execution. One major reason for
-Line 988 
 support for multi-threaded programming m
+Line 994 
 support for multi-threaded programming m
  Multi-process execution is more ubiquitous.  In order to run code in a
  multi-process configuration a decomposition specification (see section
- \ref{sect:specifying_a_decomposition}) is given (in which the at least
+ \ref{sec:specifying_a_decomposition}) is given (in which the at least
  one of the parameters {\em nPx} or {\em nPy} will be greater than one)
  and then, as for multi-threaded operation, appropriate compile time
  and run time steps must be taken.
-Line 1008 
 linking MITgcm.  (Previously this was do
+Line 1014 
 linking MITgcm.  (Previously this was do
    ALLOW\_USE\_MPI} and {\em ALWAYS\_USE\_MPI} flags in the {\em
    CPP\_EEOPTIONS.h} file.)  More detailed information about the use of
  {\em genmake2} for specifying
- local compiler flags is located in section \ref{sect:genmake}.\\
+ local compiler flags is located in section \ref{sec:genmake}.\\
  \fbox{
-Line 1112 
 Parameter: {\em pidN       }
+Line 1118 
 Parameter: {\em pidN       }
  \subsection{Controlling communication}
+ \label{sec:controlling_communication}
  The WRAPPER maintains internal information that is used for communication
  operations and that can be customized for different platforms. This section
  describes the information that is held and used.
-Line 1134 
 describes the information that is held a
+Line 1141 
 describes the information that is held a
    a particular face. A value of {\em COMM\_MSG} is used to indicate
    that some form of distributed memory communication is required to
    communicate between these tile faces (see section
-   \ref{sect:distributed_memory_communication}).  A value of {\em
+   \ref{sec:distributed_memory_communication}).  A value of {\em
      COMM\_PUT} or {\em COMM\_GET} is used to indicate forms of shared
    memory communication (see section
-   \ref{sect:shared_memory_communication}). The {\em COMM\_PUT} value
+   \ref{sec:shared_memory_communication}). The {\em COMM\_PUT} value
    indicates that a CPU should communicate by writing to data
    structures owned by another CPU. A {\em COMM\_GET} value indicates
    that a CPU should communicate by reading from data structures owned
-Line 1194 
 Parameter: {\em tileCommModeS} \\
+Line 1201 
 Parameter: {\em tileCommModeS} \\
    the file {\em eedata}. If the value of {\em nThreads} is
    inconsistent with the number of threads requested from the operating
    system (for example by using an environment variable as described in
-   section \ref{sect:multi_threaded_execution}) then usually an error
+   section \ref{sec:multi_threaded_execution}) then usually an error
    will be reported by the routine {\em CHECK\_THREADS}.
  \fbox{
-Line 1211 
 Parameter: {\em nTy} \\
+Line 1218 
 Parameter: {\em nTy} \\
  }
  \item {\bf memsync flags}
-   As discussed in section \ref{sect:memory_consistency}, a low-level
+   As discussed in section \ref{sec:memory_consistency}, a low-level
    system function may be need to force memory consistency on some
    shared memory systems.  The routine {\em MEMSYNC()} is used for this
    purpose. This routine should not need modifying and the information
-Line 1237 
 asm("lock; addl $0,0(%%esp)": : :"memory
+Line 1244 
 asm("lock; addl $0,0(%%esp)": : :"memory
  \end{verbatim}
  \item {\bf Cache line size}
-   As discussed in section \ref{sect:cache_effects_and_false_sharing},
+   As discussed in section \ref{sec:cache_effects_and_false_sharing},
    milti-threaded codes explicitly avoid penalties associated with
    excessive coherence traffic on an SMP system. To do this the shared
    memory data structures used by the {\em GLOBAL\_SUM}, {\em
-Line 1267 
 asm("lock; addl $0,0(%%esp)": : :"memory
+Line 1274 
 asm("lock; addl $0,0(%%esp)": : :"memory
      CPP\_EEMACROS.h}.  The \_GSUM macro is a performance critical
    operation, especially for large processor count, small tile size
    configurations.  The custom communication example discussed in
-   section \ref{sect:jam_example} shows how the macro is used to invoke
+   section \ref{sec:jam_example} shows how the macro is used to invoke
    a custom global sum routine for a specific set of hardware.
  \item {\bf \_EXCH}
-Line 1280 
 asm("lock; addl $0,0(%%esp)": : :"memory
+Line 1287 
 asm("lock; addl $0,0(%%esp)": : :"memory
    the header file {\em CPP\_EEMACROS.h}. As with \_GSUM, the \_EXCH
    operation plays a crucial role in scaling to small tile, large
    logical and physical processor count configurations.  The example in
-   section \ref{sect:jam_example} discusses defining an optimized and
+   section \ref{sec:jam_example} discusses defining an optimized and
    specialized form on the \_EXCH operation.
    The \_EXCH operation is also central to supporting grids such as the
-Line 1321 
 asm("lock; addl $0,0(%%esp)": : :"memory
+Line 1328 
 asm("lock; addl $0,0(%%esp)": : :"memory
    if this mechanism is unavailable then the work arrays can be extended
    with dimensions using the tile dimensioning scheme of {\em nSx} and
    {\em nSy} (as described in section
-   \ref{sect:specifying_a_decomposition}). However, if the
+   \ref{sec:specifying_a_decomposition}). However, if the
    configuration being specified involves many more tiles than OS
    threads then it can save memory resources to reduce the variable
    {\em MAX\_NO\_THREADS} to be equal to the actual number of threads
-Line 1379 
 Here we show how it can be used to impro
+Line 1386 
 Here we show how it can be used to impro
  how it can be used to adapt to new griding approaches.
  \subsubsection{JAM example}
- \label{sect:jam_example}
+ \label{sec:jam_example}
  On some platforms a big performance boost can be obtained by binding
  the communication routines {\em \_EXCH} and {\em \_GSUM} to
  specialized native libraries (for example, the shmem library on CRAY
-Line 1403 
 Developing specialized code for other li
+Line 1410 
 Developing specialized code for other li
  pattern.
  \subsubsection{Cube sphere communication}
- \label{sect:cube_sphere_communication}
+ \label{sec:cube_sphere_communication}
  Actual {\em \_EXCH} routine code is generated automatically from a
  series of template files, for example {\em exch\_rx.template}.  This
  is done to allow a large number of variations on the exchange process
-Line 1438 
 point locations.
+Line 1445 
 point locations.
  Fitting together the WRAPPER elements, package elements and
  MITgcm core equation elements of the source code produces calling
- sequence shown in section \ref{sect:calling_sequence}
+ sequence shown in section \ref{sec:calling_sequence}
  \subsection{Annotated call tree for MITgcm and WRAPPER}
- \label{sect:calling_sequence}
+ \label{sec:calling_sequence}
  WRAPPER layer.
-Line 1480 
 Core equations plus packages.
+Line 1487 
 Core equations plus packages.
  {\footnotesize
  \begin{verbatim}
  C
- C
  C Invocation from WRAPPER level...
  C  :
  C  :
-Line 1544 
 C    | | |-CTRL_INIT           :: Contro
+Line 1550 
 C    | | |-CTRL_INIT           :: Contro
  C    | | |-OPTIM_READPARMS     :: Optimisation support package. see pkg/ctrl
  C    | | |-GRDCHK_READPARMS    :: Gradient check package. see pkg/grdchk
  C    | | |-ECCO_READPARMS      :: ECCO Support Package. see pkg/ecco
+ C    | | |-PTRACERS_READPARMS  :: multiple tracer package, see pkg/ptracers
+ C    | | |-GCHEM_READPARMS     :: tracer interface package, see pkg/gchem
  C    | |
  C    | |-PACKAGES_CHECK
  C    | | |
  C    | | |-KPP_CHECK           :: KPP Package. pkg/kpp
- C    | | |-OBCS_CHECK          :: Open bndy Package. pkg/obcs
+ C    | | |-OBCS_CHECK          :: Open bndy Pacakge. pkg/obcs
  C    | | |-GMREDI_CHECK        :: GM Package. pkg/gmredi
  C    | |
  C    | |-PACKAGES_INIT_FIXED
  C    | | |-OBCS_INIT_FIXED     :: Open bndy Package. see pkg/obcs
  C    | | |-FLT_INIT            :: Floats Package. see pkg/flt
+ C    | | |-GCHEM_INIT_FIXED    :: tracer interface pachage, see pkg/gchem
  C    | |
  C    | |-ZONAL_FILT_INIT       :: FFT filter Package. see pkg/zonal_filt
  C    | |
- C    | |-INI_CG2D              :: 2d con. grad solver initialisation.
+ C    | |-INI_CG2D              :: 2d con. grad solver initialization.
  C    | |
- C    | |-INI_CG3D              :: 3d con. grad solver initialisation.
+ C    | |-INI_CG3D              :: 3d con. grad solver initialization.
  C    | |
  C    | |-CONFIG_SUMMARY        :: Provide synopsis of kernel setup.
  C    |                         :: Includes annotated table of kernel
-Line 1584 
 C    | | |
+Line 1593 
 C    | | |
  C    | | |-INI_CORI     :: Set coriolis term. zero, f-plane, beta-plane,
  C    | | |              :: sphere options are coded.
  C    | | |
- C    | | |-INI_CG2D     :: 2d con. grad solver initialisation.
+ C    | | |-INI_CG2D     :: 2d con. grad solver initialization.
- C    | | |-INI_CG3D     :: 3d con. grad solver initialisation.
+ C    | | |-INI_CG3D     :: 3d con. grad solver initialization.
- C    | | |-INI_MIXING   :: Initialise diapycnal diffusivity.
+ C    | | |-INI_MIXING   :: Initialize diapycnal diffusivity.
- C    | | |-INI_DYNVARS  :: Initialise to zero all DYNVARS.h arrays (dynamical
+ C    | | |-INI_DYNVARS  :: Initialize to zero all DYNVARS.h arrays (dynamical
  C    | | |              :: fields).
  C    | | |
  C    | | |-INI_FIELDS   :: Control initializing model fields to non-zero
-Line 1595 
 C    | | | |-INI_VEL    :: Initialize 3D
+Line 1604 
 C    | | | |-INI_VEL    :: Initialize 3D
  C    | | | |-INI_THETA  :: Set model initial temperature field.
  C    | | | |-INI_SALT   :: Set model initial salinity field.
  C    | | | |-INI_PSURF  :: Set model initial free-surface height/pressure.
- C    | | |
+ C    | | | |-INI_PRESSURE :: Compute model initial hydrostatic pressure
- C    | | |-INI_TR1      :: Set initial tracer 1 distribution.
+ C    | | | |-READ_CHECKPOINT :: Read the checkpoint
  C    | | |
  C    | | |-THE_CORRECTION_STEP :: Step forward to next time step.
  C    | | | |                   :: Here applied to move restart conditions
-Line 1623 
 C    | | | |-FIND_RHO  :: Find adjacent
+Line 1632 
 C    | | | |-FIND_RHO  :: Find adjacent
  C    | | | |-CONVECT   :: Mix static instability.
  C    | | | |-TIMEAVE_CUMULATE :: Update convection statistics.
  C    | | |
- C    | | |-PACKAGES_INIT_VARIABLES :: Does initialisation of time evolving
+ C    | | |-PACKAGES_INIT_VARIABLES :: Does initialization of time evolving
  C    | | | |                       :: package data.
  C    | | | |
  C    | | | |-GMREDI_INIT          :: GM package. ( see pkg/gmredi )
  C    | | | |-KPP_INIT             :: KPP package. ( see pkg/kpp )
  C    | | | |-KPP_OPEN_DIAGS
  C    | | | |-OBCS_INIT_VARIABLES  :: Open bndy. package. ( see pkg/obcs )
+ C    | | | |-PTRACERS_INIT        :: multi. tracer package,(see pkg/ptracers)
+ C    | | | |-GCHEM_INIT           :: tracer interface pkg (see pkh/gchem)
  C    | | | |-AIM_INIT             :: Interm. atmos package. ( see pkg/aim )
  C    | | | |-CTRL_MAP_INI         :: Control vector package.( see pkg/ctrl )
  C    | | | |-COST_INIT            :: Cost function package. ( see pkg/cost )
-Line 1672 
 C/\  | | | |                    :: Simpl
+Line 1683 
 C/\  | | | |                    :: Simpl
  C/\  | | | |                    :: for forcing datasets.
  C/\  | | | |
  C/\  | | | |-EXCH :: Sync forcing. in overlap regions.
+ C/\  | | |-SEAICE_MODEL   :: Compute sea-ice terms. ( pkg/seaice )
+ C/\  | | |-FREEZE         :: Limit surface temperature.
+ C/\  | | |-GCHEM_FIELD_LOAD :: load tracer forcing fields (pkg/gchem)
  C/\  | | |
  C/\  | | |-THERMODYNAMICS :: theta, salt + tracer equations driver.
  C/\  | | | |
  C/\  | | | |-INTEGRATE_FOR_W :: Integrate for vertical velocity.
  C/\  | | | |-OBCS_APPLY_W    :: Open bndy. package ( see pkg/obcs ).
- C/\  | | | |-FIND_RHO        :: Calculates [rho(S,T,z)-Rhonil] of a slice
+ C/\  | | | |-FIND_RHO        :: Calculates [rho(S,T,z)-RhoConst] of a slice
  C/\  | | | |-GRAD_SIGMA      :: Calculate isoneutral gradients
  C/\  | | | |-CALC_IVDC       :: Set Implicit Vertical Diffusivity for Convection
  C/\  | | | |
  C/\  | | | |-OBCS_CALC            :: Open bndy. package ( see pkg/obcs ).
  C/\  | | | |-EXTERNAL_FORCING_SURF:: Accumulates appropriately dimensioned
- C/\  | | | |                      :: forcing terms.
+ C/\  | | | | |                    :: forcing terms.
+ C/\  | | | | |-PTRACERS_FORCING_SURF :: Tracer package ( see pkg/ptracers ).
  C/\  | | | |
  C/\  | | | |-GMREDI_CALC_TENSOR   :: GM package ( see pkg/gmredi ).
  C/\  | | | |-GMREDI_CALC_TENSOR_DUMMY :: GM package ( see pkg/gmredi ).
-Line 1701 
 C/\  | | | |
+Line 1716 
 C/\  | | | |
  C/\  | | | |-CALC_GT              :: Calculate the temperature tendency terms
  C/\  | | | | |
  C/\  | | | | |-GAD_CALC_RHS       :: Generalised advection package
- C/\  | | | | |                    :: ( see pkg/gad )
+ C/\  | | | | | |                  :: ( see pkg/gad )
+ C/\  | | | | | |-KPP_TRANSPORT_T  :: KPP non-local transport ( see pkg/kpp ).
+ C/\  | | | | |
  C/\  | | | | |-EXTERNAL_FORCING_T :: Problem specific forcing for temperature.
  C/\  | | | | |-ADAMS_BASHFORTH2   :: Extrapolate tendencies forward in time.
  C/\  | | | | |-FREESURF_RESCALE_G :: Re-scale Gt for free-surface height.
-Line 1711 
 C/\  | | | |
+Line 1728 
 C/\  | | | |
  C/\  | | | |-CALC_GS              :: Calculate the salinity tendency terms
  C/\  | | | | |
  C/\  | | | | |-GAD_CALC_RHS       :: Generalised advection package
- C/\  | | | | |                    :: ( see pkg/gad )
+ C/\  | | | | | |                  :: ( see pkg/gad )
+ C/\  | | | | | |-KPP_TRANSPORT_S  :: KPP non-local transport ( see pkg/kpp ).
+ C/\  | | | | |
  C/\  | | | | |-EXTERNAL_FORCING_S :: Problem specific forcing for salt.
  C/\  | | | | |-ADAMS_BASHFORTH2   :: Extrapolate tendencies forward in time.
  C/\  | | | | |-FREESURF_RESCALE_G :: Re-scale Gs for free-surface height.
  C/\  | | | |
  C/\  | | | |-TIMESTEP_TRACER      :: Step tracer field forward in time
  C/\  | | | |
- C/\  | | | |-CALC_GTR1            :: Calculate other tracer(s) tendency terms
+ C/\  | | | |-TIMESTEP_TRACER      :: Step tracer field forward in time
+ C/\  | | | |
+ C/\  | | | |-PTRACERS_INTEGRATE   :: Integrate other tracer(s) (see pkg/ptracers).
  C/\  | | | | |
  C/\  | | | | |-GAD_CALC_RHS       :: Generalised advection package
- C/\  | | | | |                    :: ( see pkg/gad )
+ C/\  | | | | | |                  :: ( see pkg/gad )
- C/\  | | | | |-EXTERNAL_FORCING_TR:: Problem specific forcing for tracer.
+ C/\  | | | | | |-KPP_TRANSPORT_PTR:: KPP non-local transport ( see pkg/kpp ).
+ C/\  | | | | |
+ C/\  | | | | |-PTRACERS_FORCING   :: Problem specific forcing for tracer.
+ C/\  | | | | |-GCHEM_FORCING_INT  :: tracer forcing for gchem pkg (if all
+ C/\  | | | | |                       tendancy terms calcualted together)
  C/\  | | | | |-ADAMS_BASHFORTH2   :: Extrapolate tendencies forward in time.
  C/\  | | | | |-FREESURF_RESCALE_G :: Re-scale Gs for free-surface height.
+ C/\  | | | | |-TIMESTEP_TRACER    :: Step tracer field forward in time
  C/\  | | | |
- C/\  | | | |-TIMESTEP_TRACER      :: Step tracer field forward in time
  C/\  | | | |-OBCS_APPLY_TS        :: Open bndy. package (see pkg/obcs ).
- C/\  | | | |-FREEZE               :: Limit range of temperature.
  C/\  | | | |
  C/\  | | | |-IMPLDIFF             :: Solve vertical implicit diffusion equation.
  C/\  | | | |-OBCS_APPLY_TS        :: Open bndy. package (see pkg/obcs ).
-Line 1787 
 C/\  | | |
+Line 1811 
 C/\  | | |
  C/\  | | |-DO_FIELDS_BLOCKING_EXCHANGES :: Sync up overlap regions.
  C/\  | | | |-EXCH
  C/\  | | |
+ C/\  | | |-GCHEM_FORCING_SEP :: tracer forcing for gchem pkg (if
+ C/\  | | |                      tracer dependent tendencies calculated
+ C/\  | | |                      separatly)
+ C/\  | | |
  C/\  | | |-FLT_MAIN         :: Float package ( pkg/flt ).
  C/\  | | |
  C/\  | | |-MONITOR          :: Monitor package ( pkg/monitor ).
-Line 1797 
 C/\  | | | |-TIMEAVE_STATV_WRITE :: Time
+Line 1825 
 C/\  | | | |-TIMEAVE_STATV_WRITE :: Time
  C/\  | | | |-AIM_WRITE_DIAGS     :: Intermed. atmos diags. see pkg/aim
  C/\  | | | |-GMREDI_DIAGS        :: GM diags. see pkg/gmredi
  C/\  | | | |-KPP_DO_DIAGS        :: KPP diags. see pkg/kpp
+ C/\  | | | |-SBO_CALC            :: SBO diags. see pkg/sbo
+ C/\  | | | |-SBO_DIAGS           :: SBO diags. see pkg/sbo
+ C/\  | | | |-SEAICE_DO_DIAGS     :: SEAICE diags. see pkg/seaice
+ C/\  | | | |-GCHEM_DIAGS         :: gchem diags. see pkg/gchem
  C/\  | | |
  C/\  | | |-WRITE_CHECKPOINT :: Do I/O for restart files.
  C/\  | |
-Line 1814 
 C    |-TIMER_PRINTALL :: Computational t
+Line 1846 
 C    |-TIMER_PRINTALL :: Computational t
  C    |
  C    |-COMM_STATS     :: Summarise inter-proc and inter-thread communication
  C                     :: events.
  C
  \end{verbatim}
  }

 Legend:



Removed from v.1.21
 


changed lines


 
Added in v.1.26
 Legend:



Removed from v.1.21
 


changed lines


 
Added in v.1.26
-Removed from v.1.21
+Added in v.1.26

	ViewVC Help
Powered by ViewVC 1.1.22