manual/s_phys_pkgs/mnc.tex

% $Header: /u/gcmpack/manual/part6/mnc.tex,v 1.8 2004/04/03 23:05:05 edhill Exp $
% $Name:  $

\section{NetCDF I/O Integration: MNC}
\label{sec:pkg:mnc}

The \texttt{mnc} package is a set of convenience routines written to
expedite the process of creating, appending, and reading NetCDF files.
NetCDF is an increasingly popular self-describing file format
\cite{rew:97} intended primarily for scientific data sets.  An
extensive collection of NetCDF reference papers, user guides,
software, FAQs, and other information can be obtained from UCAR's web
site at:
\begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
\begin{verbatim}
http://www.unidata.ucar.edu/packages/netcdf/
\end{verbatim}
\begin{rawhtml} </A> \end{rawhtml}


\subsection{Introduction}

The \texttt{mnc} package is a two-level convenience library (or
``wrapper'') for most of the NetCDF Fortran API.  Its purpose is to
streamline the user interface to NetCDF by maintaining internal
relations (look-up tables) keyed with strings (or names) and entities
such as NetCDF files, variables, and attributes.

The two levels of the \texttt{mnc} package are:
\begin{description}

\item[Upper level] \ 
  
  The upper level contains information about two kinds of
  associations:
  \begin{description}
  \item[grid type] is lookup table indexed with a grid type name.
    Each grid type name is associated with a number of dimensions, the
    dimension sizes (one of which may be unlimited), and starting and
    ending index arrays.  The intent is to store all the necessary
    size and shape information for the Fortran arrays containing
    MITgcm--style ``tile'' variables (that is, a central region
    surrounded by a variably-sized ``halo'' or exchange region as
    shown in Figures \ref{fig:communication_primitives} and
    \ref{fig:tiling-strategy}).
  
  \item[variable type] is a lookup table indexed by a variable type
    name.  For each name, the table contains a reference to a grid
    type for the variable and the names and values of various
    attributes.
  \end{description}
  
  Within the upper level, these associations are not permanently tied
  to any particular NetCDF file.  This allows the information to be
  re-used over multiple file reads and writes.

\item[Lower level] \ 
  
  In the lower (or internal) level, associations are stored for NetCDF
  files and many of the entities that they contain including
  dimensions, variables, and global attributes.  All associations are
  on a per-file basis.  Thus, each entity is tied to a unique NetCDF
  file and will be created or destroyed when files are, respectively,
  opened or closed.

\end{description}


\subsection{Using MNC}

\subsubsection{Grid--Types and Variable--Types}

As a convenience for users, the MNC package includes numerous routines
to aid in the writing of data to NetCDF format.  Probably the biggest
convenience is the use of pre-defined ``grid types'' and ``variable
types''.  These ``types'' are simply look-up tables that store
dimensions, indicies, attributes, and other information that can all
be retrieved using a single character string.

The ``grid types'' are a way of mapping variables within MITgcm to
NetCDF arrays.  Within MITgcm, most spatial variables are defined
using two-- or three--dimensional arrays with ``overlap'' regions (see
Figures \ref{fig:communication_primitives}, a possible vertical index,
and \ref{fig:tiling-strategy}) and tile indicies such as the following
``U'' velocity:
\begin{verbatim}
      _RL  uVel (1-OLx:sNx+OLx,1-OLy:sNy+OLy,Nr,nSx,nSy)
\end{verbatim}
as defined in \filelink{model/inc/DYNVARS.h}{model-inc-DYNVARS.h}

The grid type is a character string that encodes the presence and
types associated with the four possible dimensions.  The character
string follows the format
\begin{center}
  \texttt{H0\_H1\_H2\_\_V\_\_T}
\end{center}
where the terms \textit{H0}, \textit{H1}, \textit{H2}, \textit{V},
\textit{T} can be almost any combination of the following:
\begin{center}
  \begin{tabular}[h]{|ccc|c|c|}\hline
    \multicolumn{3}{|c|}{Horizontal} & Vertical & Time \\
    \textbf{H0}: location & \textbf{H1}: dimensions & \textbf{H2}: halo 
          & \textbf{V}: location & \textbf{T}: level  \\\hline
    \texttt{-} & xy & Hn & \texttt{-} & \texttt{-} \\
    U  &  x  &  Hy  &  i  &  t  \\
    V  &  y  &      &  c  &     \\
    Cen  &   &      &     &     \\
    Cor  &   &      &     &     \\\hline
  \end{tabular}
\end{center}
A example list of all pre-defined combinations is contained in the
file
\begin{center}
  \texttt{pkg/mnc/pre-defined\_grids.txt}.
\end{center}

The variable type is an association between a variable type name and the
following items:
\begin{center}
  \begin{tabular}[h]{|l|l|}\hline
    \textbf{Item}  & \textbf{Purpose}  \\\hline
    grid type  &  defines the in-memory arrangement  \\
    \texttt{bi,bj} dimensions  &  tiling indices, if present  \\\hline
  \end{tabular}
\end{center}
and is used by the \texttt{mnc\_cw\_*\_[R|W]} subroutines for reading
and writing variables.


\subsubsection{An Example}

Writing variables to NetCDF files can be accomplished in as few as two
function calls.  The first function call defines a variable type,
associates it with a name (character string), and provides additional
information about the indicies for the tile (\texttt{bi},\texttt{bj})
dimensions.  The second function call will write the data at, if
necessary, the current time level within the model.

Examples of the initialization calls can be found in the file 
\filelink{model/src/ini\_mnc\_io.F}{model-src-ini_mnc_io.F}
where these function calls:
{\footnotesize
\begin{verbatim}
C     Create MNC definitions for DYNVARS.h variables
      CALL MNC_CW_ADD_VNAME('iter', '-_-_--__-__t', 0,0, myThid)
      CALL MNC_CW_ADD_VATTR_TEXT('iter',1,
     &     'long_name','iteration_count', myThid)

      CALL MNC_CW_ADD_VNAME('model_time', '-_-_--__-__t', 0,0, myThid)
      CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,
     &     'long_name','Model Time', myThid)
      CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,'units','s', myThid)

      CALL MNC_CW_ADD_VNAME('U', 'U_xy_Hn__C__t', 4,5, myThid)
      CALL MNC_CW_ADD_VATTR_TEXT('U',1,'units','m/s', myThid)
      CALL MNC_CW_ADD_VATTR_TEXT('U',1,
     &     'coordinates','XU YU RC iter', myThid)

      CALL MNC_CW_ADD_VNAME('T', 'Cen_xy_Hn__C__t', 4,5, myThid)
      CALL MNC_CW_ADD_VATTR_TEXT('T',1,'units','degC', myThid)
      CALL MNC_CW_ADD_VATTR_TEXT('T',1,'long_name',
     &     'potential_temperature', myThid)
      CALL MNC_CW_ADD_VATTR_TEXT('T',1,
     &     'coordinates','XC YC RC iter', myThid)
\end{verbatim}
}
{\noindent initialize four \texttt{VNAME}s and add one or more NetCDF
  attributes to each.}
    
The four variables defined above are subsequently written at specific
time steps within
\filelink{model/src/write\_state.F}{model-src-write_state.F}
using the function calls:
{\footnotesize
\begin{verbatim}
C       Write dynvars using the MNC package
        CALL MNC_CW_SET_UDIM('state', -1, myThid)
        CALL MNC_CW_RL_W('I','state',0,0,'iter', myIter, myThid)
        CALL MNC_CW_SET_UDIM('state', 0, myThid)
        CALL MNC_CW_RL_W('D','state',0,0,'model_time',myTime, myThid)
        CALL MNC_CW_RL_W('D','state',0,0,'U', uVel, myThid)
        CALL MNC_CW_RL_W('D','state',0,0,'T', theta, myThid)
\end{verbatim}
}


\subsubsection{Parameters}

All the MNC parameters are contained within a file named
\texttt{data.mnc}.  If this file does not exist, then the MNC package
will interpret that as an indication that it is not to be used.  If
the \texttt{data.mnc} does exist, then it may contain the following
parameters:

\begin{center}
  {\footnotesize
    \begin{tabular}[htb]{|l|l|l|l|}\hline
      &  &  &  \\
      \textbf{Name}  &  \textbf{Type}  &  
      \textbf{Default}  &  \textbf{Description}  \\\hline
      &  &  &  \\
      \texttt{useMNC}  &  Logical  & \texttt{.FALSE.}  &  
      \textbf{overall MNC ON/OFF switch}  \\
      \texttt{mnc\_echo\_gvtypes}  &  Logical  & \texttt{.FALSE.}  &  
      echo pre-defined ``types'' to STDOUT?   \\
      \texttt{mnc\_use\_outdir}  &  Logical  & \texttt{.FALSE.}  &  
      create a directory for output?  \\
      \texttt{mnc\_outdir\_str}  &  String  & \texttt{'mnc\_'}  &  
      output directory name \\
      \texttt{mnc\_outdir\_date}  &  Logical  & \texttt{.FALSE.}  &  
      embed date in output directory name?  \\
      \texttt{mnc\_pickup\_write}  &  Logical  & \texttt{.FALSE.}  &  
      use MNC to write (create) pickup files?  \\
      \texttt{mnc\_pickup\_read}  &  Logical  & \texttt{.FALSE.}  &  
      use MNC to read pickup files?  \\
      \texttt{mnc\_use\_indir}  &  Logical  & \texttt{.FALSE.}  &  
      use a directory (path) for input?  \\
      \texttt{mnc\_indir\_str}  &  String  & \texttt{''}  &  
      input directory (or path) name  \\
      \texttt{mnc\_use\_for\_mon}  &  Logical  & \texttt{.FALSE.}  &  
      write \texttt{monitor} output using MNC?  \\\hline
    \end{tabular}
  }
\end{center}

%\subsection{Package Reference}

1	edhill	1.9	% $Header: /u/gcmpack/manual/part6/mnc.tex,v 1.8 2004/04/03 23:05:05 edhill Exp $
2	edhill	1.1	% $Name: $
3
4	edhill	1.6	\section{NetCDF I/O Integration: MNC}
5	edhill	1.3	\label{sec:pkg:mnc}
6	edhill	1.1
7	edhill	1.3	The \texttt{mnc} package is a set of convenience routines written to
8			expedite the process of creating, appending, and reading NetCDF files.
9			NetCDF is an increasingly popular self-describing file format
10			\cite{rew:97} intended primarily for scientific data sets. An
11			extensive collection of NetCDF reference papers, user guides,
12			software, FAQs, and other information can be obtained from UCAR's web
13			site at:
14			\begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
15			\begin{verbatim}
16			http://www.unidata.ucar.edu/packages/netcdf/
17			\end{verbatim}
18			\begin{rawhtml} </A> \end{rawhtml}
19	edhill	1.1
20
21			\subsection{Introduction}
22
23	edhill	1.3	The \texttt{mnc} package is a two-level convenience library (or
24			``wrapper'') for most of the NetCDF Fortran API. Its purpose is to
25			streamline the user interface to NetCDF by maintaining internal
26	edhill	1.6	relations (look-up tables) keyed with strings (or names) and entities
27			such as NetCDF files, variables, and attributes.
28	edhill	1.3
29			The two levels of the \texttt{mnc} package are:
30			\begin{description}
31
32			\item[Upper level] \
33
34			The upper level contains information about two kinds of
35			associations:
36			\begin{description}
37	edhill	1.6	\item[grid type] is lookup table indexed with a grid type name.
38	edhill	1.3	Each grid type name is associated with a number of dimensions, the
39			dimension sizes (one of which may be unlimited), and starting and
40			ending index arrays. The intent is to store all the necessary
41			size and shape information for the Fortran arrays containing
42			MITgcm--style ``tile'' variables (that is, a central region
43			surrounded by a variably-sized ``halo'' or exchange region as
44			shown in Figures \ref{fig:communication_primitives} and
45			\ref{fig:tiling-strategy}).
46
47	edhill	1.6	\item[variable type] is a lookup table indexed by a variable type
48	edhill	1.3	name. For each name, the table contains a reference to a grid
49			type for the variable and the names and values of various
50			attributes.
51			\end{description}
52
53			Within the upper level, these associations are not permanently tied
54			to any particular NetCDF file. This allows the information to be
55			re-used over multiple file reads and writes.
56
57			\item[Lower level] \
58
59			In the lower (or internal) level, associations are stored for NetCDF
60			files and many of the entities that they contain including
61			dimensions, variables, and global attributes. All associations are
62			on a per-file basis. Thus, each entity is tied to a unique NetCDF
63			file and will be created or destroyed when files are, respectively,
64			opened or closed.
65
66			\end{description}
67	edhill	1.1
68
69	edhill	1.5	\subsection{Using MNC}
70
71	edhill	1.6	\subsubsection{Grid--Types and Variable--Types}
72	edhill	1.5
73			As a convenience for users, the MNC package includes numerous routines
74			to aid in the writing of data to NetCDF format. Probably the biggest
75			convenience is the use of pre-defined ``grid types'' and ``variable
76			types''. These ``types'' are simply look-up tables that store
77			dimensions, indicies, attributes, and other information that can all
78			be retrieved using a single character string.
79
80			The ``grid types'' are a way of mapping variables within MITgcm to
81			NetCDF arrays. Within MITgcm, most spatial variables are defined
82			using two-- or three--dimensional arrays with ``overlap'' regions (see
83			Figures \ref{fig:communication_primitives}, a possible vertical index,
84			and \ref{fig:tiling-strategy}) and tile indicies such as the following
85			``U'' velocity:
86			\begin{verbatim}
87			_RL uVel (1-OLx:sNx+OLx,1-OLy:sNy+OLy,Nr,nSx,nSy)
88			\end{verbatim}
89			as defined in \filelink{model/inc/DYNVARS.h}{model-inc-DYNVARS.h}
90
91			The grid type is a character string that encodes the presence and
92			types associated with the four possible dimensions. The character
93			string follows the format
94			\begin{center}
95			\texttt{H0\_H1\_H2\_\_V\_\_T}
96			\end{center}
97	edhill	1.6	where the terms \textit{H0}, \textit{H1}, \textit{H2}, \textit{V},
98			\textit{T} can be almost any combination of the following:
99	edhill	1.5	\begin{center}
100			\begin{tabular}[h]{\|ccc\|c\|c\|}\hline
101	edhill	1.6	\multicolumn{3}{\|c\|}{Horizontal} & Vertical & Time \\
102	edhill	1.7	\textbf{H0}: location & \textbf{H1}: dimensions & \textbf{H2}: halo
103			& \textbf{V}: location & \textbf{T}: level \\\hline
104	edhill	1.5	\texttt{-} & xy & Hn & \texttt{-} & \texttt{-} \\
105			U & x & Hy & i & t \\
106			V & y & & c & \\
107			Cen & & & & \\
108			Cor & & & & \\\hline
109			\end{tabular}
110			\end{center}
111			A example list of all pre-defined combinations is contained in the
112			file
113			\begin{center}
114			\texttt{pkg/mnc/pre-defined\_grids.txt}.
115			\end{center}
116	edhill	1.7
117			The variable type is an association between a variable type name and the
118			following items:
119			\begin{center}
120	edhill	1.9	\begin{tabular}[h]{\|l\|l\|}\hline
121	edhill	1.7	\textbf{Item} & \textbf{Purpose} \\\hline
122			grid type & defines the in-memory arrangement \\
123			\texttt{bi,bj} dimensions & tiling indices, if present \\\hline
124			\end{tabular}
125			\end{center}
126			and is used by the \texttt{mnc\_cw\_*\_[R\|W]} subroutines for reading
127			and writing variables.
128	edhill	1.5
129
130			\subsubsection{An Example}
131
132			Writing variables to NetCDF files can be accomplished in as few as two
133			function calls. The first function call defines a variable type,
134			associates it with a name (character string), and provides additional
135	edhill	1.6	information about the indicies for the tile (\texttt{bi},\texttt{bj})
136			dimensions. The second function call will write the data at, if
137			necessary, the current time level within the model.
138	edhill	1.5
139			Examples of the initialization calls can be found in the file
140	edhill	1.8	\filelink{model/src/ini\_mnc\_io.F}{model-src-ini_mnc_io.F}
141	edhill	1.9	where these function calls:
142	edhill	1.6	{\footnotesize
143	edhill	1.5	\begin{verbatim}
144	edhill	1.8	C Create MNC definitions for DYNVARS.h variables
145			CALL MNC_CW_ADD_VNAME('iter', '-_-_--__-__t', 0,0, myThid)
146			CALL MNC_CW_ADD_VATTR_TEXT('iter',1,
147			& 'long_name','iteration_count', myThid)
148
149			CALL MNC_CW_ADD_VNAME('model_time', '-_-_--__-__t', 0,0, myThid)
150			CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,
151			& 'long_name','Model Time', myThid)
152			CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,'units','s', myThid)
153
154			CALL MNC_CW_ADD_VNAME('U', 'U_xy_Hn__C__t', 4,5, myThid)
155			CALL MNC_CW_ADD_VATTR_TEXT('U',1,'units','m/s', myThid)
156			CALL MNC_CW_ADD_VATTR_TEXT('U',1,
157			& 'coordinates','XU YU RC iter', myThid)
158
159			CALL MNC_CW_ADD_VNAME('T', 'Cen_xy_Hn__C__t', 4,5, myThid)
160			CALL MNC_CW_ADD_VATTR_TEXT('T',1,'units','degC', myThid)
161			CALL MNC_CW_ADD_VATTR_TEXT('T',1,'long_name',
162			& 'potential_temperature', myThid)
163			CALL MNC_CW_ADD_VATTR_TEXT('T',1,
164			& 'coordinates','XC YC RC iter', myThid)
165	edhill	1.6	\end{verbatim}
166			}
167	edhill	1.9	{\noindent initialize four \texttt{VNAME}s and add one or more NetCDF
168			attributes to each.}
169	edhill	1.5
170	edhill	1.9	The four variables defined above are subsequently written at specific
171	edhill	1.6	time steps within
172			\filelink{model/src/write\_state.F}{model-src-write_state.F}
173			using the function calls:
174			{\footnotesize
175	edhill	1.5	\begin{verbatim}
176	edhill	1.8	C Write dynvars using the MNC package
177			CALL MNC_CW_SET_UDIM('state', -1, myThid)
178	edhill	1.9	CALL MNC_CW_RL_W('I','state',0,0,'iter', myIter, myThid)
179	edhill	1.8	CALL MNC_CW_SET_UDIM('state', 0, myThid)
180			CALL MNC_CW_RL_W('D','state',0,0,'model_time',myTime, myThid)
181			CALL MNC_CW_RL_W('D','state',0,0,'U', uVel, myThid)
182			CALL MNC_CW_RL_W('D','state',0,0,'T', theta, myThid)
183	edhill	1.6	\end{verbatim}
184			}
185	edhill	1.5
186	edhill	1.1
187	edhill	1.9	\subsubsection{Parameters}
188
189			All the MNC parameters are contained within a file named
190			\texttt{data.mnc}. If this file does not exist, then the MNC package
191			will interpret that as an indication that it is not to be used. If
192			the \texttt{data.mnc} does exist, then it may contain the following
193			parameters:
194
195			\begin{center}
196			{\footnotesize
197			\begin{tabular}[htb]{\|l\|l\|l\|l\|}\hline
198			& & & \\
199			\textbf{Name} & \textbf{Type} &
200			\textbf{Default} & \textbf{Description} \\\hline
201			& & & \\
202			\texttt{useMNC} & Logical & \texttt{.FALSE.} &
203			\textbf{overall MNC ON/OFF switch} \\
204			\texttt{mnc\_echo\_gvtypes} & Logical & \texttt{.FALSE.} &
205			echo pre-defined ``types'' to STDOUT? \\
206			\texttt{mnc\_use\_outdir} & Logical & \texttt{.FALSE.} &
207			create a directory for output? \\
208			\texttt{mnc\_outdir\_str} & String & \texttt{'mnc\_'} &
209			output directory name \\
210			\texttt{mnc\_outdir\_date} & Logical & \texttt{.FALSE.} &
211			embed date in output directory name? \\
212			\texttt{mnc\_pickup\_write} & Logical & \texttt{.FALSE.} &
213			use MNC to write (create) pickup files? \\
214			\texttt{mnc\_pickup\_read} & Logical & \texttt{.FALSE.} &
215			use MNC to read pickup files? \\
216			\texttt{mnc\_use\_indir} & Logical & \texttt{.FALSE.} &
217			use a directory (path) for input? \\
218			\texttt{mnc\_indir\_str} & String & \texttt{''} &
219			input directory (or path) name \\
220			\texttt{mnc\_use\_for\_mon} & Logical & \texttt{.FALSE.} &
221			write \texttt{monitor} output using MNC? \\\hline
222			\end{tabular}
223			}
224			\end{center}
225
226			%\subsection{Package Reference}
227	edhill	1.5