/[MITgcm]/manual/s_phys_pkgs/mnc.tex
ViewVC logotype

Diff of /manual/s_phys_pkgs/mnc.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.5 by edhill, Thu Feb 12 03:35:05 2004 UTC revision 1.13 by edhill, Tue Oct 12 21:44:59 2004 UTC
# Line 1  Line 1 
1  % $Header$  % $Header$
2  % $Name$  % $Name$
3    
4  \section{NetCDF I/O Integration}  \section{NetCDF I/O Integration: MNC}
5  \label{sec:pkg:mnc}  \label{sec:pkg:mnc}
6    \begin{rawhtml}
7    <!-- CMIREDIR:package_mnc: -->
8    \end{rawhtml}
9    
10  The \texttt{mnc} package is a set of convenience routines written to  The \texttt{mnc} package is a set of convenience routines written to
11  expedite the process of creating, appending, and reading NetCDF files.  expedite the process of creating, appending, and reading NetCDF files.
# Line 18  http://www.unidata.ucar.edu/packages/net Line 21  http://www.unidata.ucar.edu/packages/net
21  \begin{rawhtml} </A> \end{rawhtml}  \begin{rawhtml} </A> \end{rawhtml}
22    
23    
24  \subsection{Introduction}  \subsection{Using MNC}
25    
26    \subsubsection{MNC Configuration and Inputs}
27    
28    As with all MITgcm packages, MNC can be turned on/off at compile time
29    using the \texttt{packages.conf} file or the genmake2
30    \texttt{-enable=mnc} or \texttt{-disable=mnc} switches.
31    
32    For run-time configuration, most of the MNC--related model parameters
33    are contained within a Fortran namelist file called \texttt{data.mnc}.
34    If this file does not exist, then the MNC package will interpret that
35    as an indication that it is not to be used.  If the \texttt{data.mnc}
36    file does exist, then it may contain the following parameters:
37    
38    \begin{center}
39      {\footnotesize
40        \begin{tabular}[htb]{|l|c|l|l|}\hline
41          \textbf{Name}  &  \textbf{T}  &  
42          \textbf{Default}  &  \textbf{Description}  \\\hline
43          &  &  &  \\
44          \texttt{useMNC}  &  L  & \texttt{.FALSE.}  &  
45          \textbf{overall MNC ON/OFF switch}  \\
46          \texttt{mnc\_echo\_gvtypes}  &  L  & \texttt{.FALSE.}  &  
47          echo pre-defined ``types'' (debugging)   \\
48          \texttt{mnc\_use\_outdir}  &  L  & \texttt{.FALSE.}  &  
49          create a directory for output  \\
50          \texttt{mnc\_outdir\_str}  &  S  & \texttt{'mnc\_'}  &  
51          output directory name \\
52          \texttt{mnc\_outdir\_date}  &  L  & \texttt{.FALSE.}  &  
53          embed date in the output dir name  \\
54          \texttt{pickup\_write\_mnc}  &  L  & \texttt{.FALSE.}  &  
55          use MNC to write (create) pickup files  \\
56          \texttt{pickup\_read\_mnc}  &  L  & \texttt{.FALSE.}  &  
57          use MNC to read pickup files  \\
58          \texttt{mnc\_use\_indir}  &  L  & \texttt{.FALSE.}  &  
59          use a directory (path) for input  \\
60          \texttt{mnc\_indir\_str}  &  S  & \texttt{''}  &  
61          input directory (or path) name  \\
62          \texttt{snapshot\_mnc}  &  L  & \texttt{.FALSE.}  &  
63          write \texttt{snapshot} (instantaneous) w/MNC  \\
64          \texttt{monitor\_mnc}  &  L  & \texttt{.FALSE.}  &  
65          write \texttt{monitor} w/MNC  \\
66          \texttt{timeave\_mnc}  &  L  & \texttt{.FALSE.}  &  
67          write \texttt{timeave} w/MNC  \\
68          \texttt{autodiff\_mnc}  &  L  & \texttt{.FALSE.}  &  
69          write \texttt{autodiff} w/MNC  \\\hline
70        \end{tabular}
71      }
72    \end{center}
73    
74    Additional MNC--related parameters are contained within the main
75    \texttt{data} namelist file and in some of the namelist files for
76    individual packages.  These options are:
77    \begin{center}
78      {\footnotesize
79        \begin{tabular}[htb]{|l|c|l|l|}\hline
80          \textbf{Name}  &  \textbf{T}  &  
81          \textbf{Default}  &  \textbf{Description}  \\\hline
82          \multicolumn{4}{|c|}{\ }  \\
83          \multicolumn{4}{|c|}{Main namelist file:
84            ``\textbf{data}''}  \\\hline
85          \texttt{snapshot\_ioinc}  &  L  & \texttt{.FALSE.}  &  
86          write \texttt{snapshot} ``inclusively''  \\
87          \texttt{timeave\_ioinc}  &  L  & \texttt{.FALSE.}  &  
88          write \texttt{timeave} ``inclusively''  \\
89          \texttt{monitor\_ioinc}  &  L  & \texttt{.FALSE.}  &  
90          write \texttt{monitor} ``inclusively''  \\
91          \texttt{the\_run\_name}  &  C  & ``name...''  &  
92          name is included in all MNC output  \\\hline
93          \multicolumn{4}{|c|}{\ }  \\
94          \multicolumn{4}{|c|}{Diagnostics namelist file:
95            ``\textbf{data.diagnostics}''}  \\\hline
96          \texttt{diag\_mnc}  &  L  & \texttt{.FALSE.}  &  
97          write \texttt{diagnostics} w/MNC  \\
98          \texttt{diag\_ioinc}  &  L  & \texttt{.FALSE.}  &  
99          write \texttt{diagnostics} ``inclusively''  \\\hline
100        \end{tabular}
101      }
102    \end{center}
103    
104    By default, turning on MNC for a particular output type will result in
105    turning off all the corresponding (usually, default) MDSIO or STDOUT
106    output mechanisms.  In other words, output defaults to being an
107    exclusive selection.  To enable multiple kinds of simultaneous output,
108    flags of the form \texttt{NAME\_ioinc} have been created where
109    \texttt{NAME} corresponds to the various MNC output flags.  When a
110    \texttt{NAME\_ioinc} flag is set to \texttt{.TRUE.}, then multiple
111    simultaneous forms of output are allowed for the \texttt{NAME} output
112    mechanism.  The intent of this design is that typical users will only
113    want one kind of output while people debugging the code (particularly
114    the I/O routines) may want simultaneous types of output.
115    
116    This ``inclusive'' versus ``exclusive'' design is easily applied in
117    cases where three or more kinds of output may be generated.  Thus, it
118    can be readily extended to additional new output types (eg. HDF5).
119    
120    Input types are always exclusive.
121    
122    \subsubsection{MNC Output}
123    
124    While NetCDF files are supposed to be ``self-describing'', it is
125    helpful to note the following:
126    
127    \begin{itemize}
128    \item The constraints placed upon the ``unlimited'' (or ``record'')
129      dimension inherent with NetCDF v3.x make it very inefficient to put
130      variables written at potentially different intervals within the same
131      file.  For this reason, MNC output is split into a few file ``base
132      names'' which try to reflect the nature of their content.
133      
134    \item All MNC output is currently done in a ``tile-per-file'' fashion
135      since most NetCDF v3.x implementions cannot write safely within MPI
136      or multi-threaded environments.  This tiling is done in a global
137      fashion and the tile numbers are appended to the base names
138      described above.  Some scripts to ``assemble'' output are available
139      (\texttt{MITgcm/utils/matlab}).  More general manipulations can be
140      accomplished with the
141      \begin{rawhtml}
142        <A href="http://nco.sourceforge.net">
143      \end{rawhtml}
144    \begin{verbatim}
145    NetCDF Operators (or ``NCO'') at http://nco.sourceforge.net
146    \end{verbatim}
147      \begin{rawhtml} </A> \end{rawhtml}
148      which is a very powerful and convenient set of tools for working
149      with all NetCDF files.
150      
151    \item On many systems, NetCDF has practical file size limits on the
152      order of 2--4GB (the maximium memory addressable with 32bit
153      pointers) due to a lack of operating system, compiler, and/or
154      library support.  In cases where this limit is reached, it is
155      generally a good idea to reduce write frequencies or restart from
156      pickups.
157      
158    \item MNC does not (yet) provide a mechanism for reading information
159      from a single ``global'' file as can be done with the MDSIO
160      package.  This is in progress.
161    
162    \end{itemize}
163    
164    
165    \subsection{MNC Internals}
166    
167  The \texttt{mnc} package is a two-level convenience library (or  The \texttt{mnc} package is a two-level convenience library (or
168  ``wrapper'') for most of the NetCDF Fortran API.  Its purpose is to  ``wrapper'') for most of the NetCDF Fortran API.  Its purpose is to
169  streamline the user interface to NetCDF by maintaining internal  streamline the user interface to NetCDF by maintaining internal
170  relations (``look-up tables'') keyed with strings (or ``names'') and  relations (look-up tables) keyed with strings (or names) and entities
171  entities such as NetCDF files, variables, and attributes.  such as NetCDF files, variables, and attributes.
172    
173  The two levels of the \texttt{mnc} package are:  The two levels of the \texttt{mnc} package are:
174  \begin{description}  \begin{description}
# Line 34  The two levels of the \texttt{mnc} packa Line 178  The two levels of the \texttt{mnc} packa
178    The upper level contains information about two kinds of    The upper level contains information about two kinds of
179    associations:    associations:
180    \begin{description}    \begin{description}
181    \item[Grid type] is lookup table indexed with a grid type name.    \item[grid type] is lookup table indexed with a grid type name.
182      Each grid type name is associated with a number of dimensions, the      Each grid type name is associated with a number of dimensions, the
183      dimension sizes (one of which may be unlimited), and starting and      dimension sizes (one of which may be unlimited), and starting and
184      ending index arrays.  The intent is to store all the necessary      ending index arrays.  The intent is to store all the necessary
# Line 44  The two levels of the \texttt{mnc} packa Line 188  The two levels of the \texttt{mnc} packa
188      shown in Figures \ref{fig:communication_primitives} and      shown in Figures \ref{fig:communication_primitives} and
189      \ref{fig:tiling-strategy}).      \ref{fig:tiling-strategy}).
190        
191    \item[Variable type] is a lookup table indexed by a variable type    \item[variable type] is a lookup table indexed by a variable type
192      name.  For each name, the table contains a reference to a grid      name.  For each name, the table contains a reference to a grid
193      type for the variable and the names and values of various      type for the variable and the names and values of various
194      attributes.      attributes.
# Line 66  The two levels of the \texttt{mnc} packa Line 210  The two levels of the \texttt{mnc} packa
210  \end{description}  \end{description}
211    
212    
213  \subsection{Using MNC}  \subsubsection{MNC Grid--Types and Variable--Types}
   
 \subsubsection{``Grid--Types'' and ``Variable--Types''}  
214    
215  As a convenience for users, the MNC package includes numerous routines  As a convenience for users, the MNC package includes numerous routines
216  to aid in the writing of data to NetCDF format.  Probably the biggest  to aid in the writing of data to NetCDF format.  Probably the biggest
# Line 94  string follows the format Line 236  string follows the format
236  \begin{center}  \begin{center}
237    \texttt{H0\_H1\_H2\_\_V\_\_T}    \texttt{H0\_H1\_H2\_\_V\_\_T}
238  \end{center}  \end{center}
239  where the terms \textit{H0,H1,H2,V,T} can be almost any combination of  where the terms \textit{H0}, \textit{H1}, \textit{H2}, \textit{V},
240  the following:  \textit{T} can be almost any combination of the following:
241  \begin{center}  \begin{center}
242    \begin{tabular}[h]{|ccc|c|c|}\hline    \begin{tabular}[h]{|ccc|c|c|}\hline
243      \multicolumn{3}{|c|}{Horizontal} & Vertical &  \\      \multicolumn{3}{|c|}{Horizontal} & Vertical & Time \\
244      Location & Direction & Halo & Location & Time  \\\hline      \textbf{H0}: location & \textbf{H1}: dimensions & \textbf{H2}: halo
245      H0  &  H1  &  H2  &  V  &  T  \\\hline            & \textbf{V}: location & \textbf{T}: level  \\\hline
246      \texttt{-} & xy & Hn & \texttt{-} & \texttt{-} \\      \texttt{-} & xy & Hn & \texttt{-} & \texttt{-} \\
247      U  &  x  &  Hy  &  i  &  t  \\      U  &  x  &  Hy  &  i  &  t  \\
248      V  &  y  &      &  c  &     \\      V  &  y  &      &  c  &     \\
# Line 114  file Line 256  file
256    \texttt{pkg/mnc/pre-defined\_grids.txt}.    \texttt{pkg/mnc/pre-defined\_grids.txt}.
257  \end{center}  \end{center}
258    
259    The variable type is an association between a variable type name and the
260    following items:
261    \begin{center}
262      \begin{tabular}[h]{|l|l|}\hline
263        \textbf{Item}  & \textbf{Purpose}  \\\hline
264        grid type  &  defines the in-memory arrangement  \\
265        \texttt{bi,bj} dimensions  &  tiling indices, if present  \\\hline
266      \end{tabular}
267    \end{center}
268    and is used by the \texttt{mnc\_cw\_*\_[R|W]} subroutines for reading
269    and writing variables.
270    
271  \subsubsection{An Example}  
272    \subsubsection{Using MNC: Examples}
273    
274  Writing variables to NetCDF files can be accomplished in as few as two  Writing variables to NetCDF files can be accomplished in as few as two
275  function calls.  The first function call defines a variable type,  function calls.  The first function call defines a variable type,
276  associates it with a name (character string), and provides additional  associates it with a name (character string), and provides additional
277  information about the indicies for \texttt{bi,bj} dimensions.  The  information about the indicies for the tile (\texttt{bi},\texttt{bj})
278  second function call will write variable at, if necessary, the  dimensions.  The second function call will write the data at, if
279  current time level within the model.  necessary, the current time level within the model.
280    
281  Examples of the initialization calls can be found in the file  Examples of the initialization calls can be found in the file
282  \begin{center}  \filelink{model/src/ini\_mnc\_io.F}{model-src-ini_mnc_io.F}
283    \filelink{model/src/initialise\_fixed.F}{model-src-initialise_fixed.F}  where these function calls:
284  \end{center}  {\footnotesize
 where these four function calls: {\footnotesize  
285  \begin{verbatim}  \begin{verbatim}
286    C     Create MNC definitions for DYNVARS.h variables  C     Create MNC definitions for DYNVARS.h variables
287          CALL MNC_CW_ADD_VNAME(myThid, 'iter', '-_-_--__-__t', 0,0)        CALL MNC_CW_ADD_VNAME('iter', '-_-_--__-__t', 0,0, myThid)
288          CALL MNC_CW_ADD_VATTR_TEXT(myThid,'iter',1,        CALL MNC_CW_ADD_VATTR_TEXT('iter',1,
289         &     'long_name','iteration_count')       &     'long_name','iteration_count', myThid)
290          CALL MNC_CW_ADD_VNAME(myThid, 'U', 'U_xy_Hn__C__t', 4,5)  
291          CALL MNC_CW_ADD_VATTR_TEXT(myThid,'U',1,'units','m/s')        CALL MNC_CW_ADD_VNAME('model_time', '-_-_--__-__t', 0,0, myThid)
292  \end{verbatim} }        CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,
293  {\noindent initialize two \texttt{VNAME}s and add one attribute to each.}       &     'long_name','Model Time', myThid)
294          CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,'units','s', myThid)
295    
296          CALL MNC_CW_ADD_VNAME('U', 'U_xy_Hn__C__t', 4,5, myThid)
297          CALL MNC_CW_ADD_VATTR_TEXT('U',1,'units','m/s', myThid)
298          CALL MNC_CW_ADD_VATTR_TEXT('U',1,
299         &     'coordinates','XU YU RC iter', myThid)
300    
301          CALL MNC_CW_ADD_VNAME('T', 'Cen_xy_Hn__C__t', 4,5, myThid)
302          CALL MNC_CW_ADD_VATTR_TEXT('T',1,'units','degC', myThid)
303          CALL MNC_CW_ADD_VATTR_TEXT('T',1,'long_name',
304         &     'potential_temperature', myThid)
305          CALL MNC_CW_ADD_VATTR_TEXT('T',1,
306         &     'coordinates','XC YC RC iter', myThid)
307    \end{verbatim}
308    }
309    {\noindent initialize four \texttt{VNAME}s and add one or more NetCDF
310      attributes to each.}
311            
312  The two variables are subsequently written at specific time steps  The four variables defined above are subsequently written at specific
313  within  time steps within
314  \begin{center}  \filelink{model/src/write\_state.F}{model-src-write_state.F}
315    \filelink{model/src/write\_state.F}{model-src-write_state.F}  using the function calls:
316  \end{center}  {\footnotesize
 using the function calls: {\footnotesize  
317  \begin{verbatim}  \begin{verbatim}
318    C     Write the DYNVARS.h variables using the MNC package  C       Write dynvars using the MNC package
319          mnc_iter = myIter          CALL MNC_CW_SET_UDIM('state', -1, myThid)
320          CALL MNC_CW_RL_W_R(myThid,'state',0,0,'iter',-1,mnc_iter)          CALL MNC_CW_I_W('I','state',0,0,'iter', myIter, myThid)
321          CALL MNC_CW_RL_W_D(myThid,'state',0,0,'U', 0, uVel)          CALL MNC_CW_SET_UDIM('state', 0, myThid)
322  \end{verbatim} }          CALL MNC_CW_RL_W('D','state',0,0,'model_time',myTime, myThid)
323            CALL MNC_CW_RL_W('D','state',0,0,'U', uVel, myThid)
324            CALL MNC_CW_RL_W('D','state',0,0,'T', theta, myThid)
325  \subsection{Key subroutines, parameters and files}  \end{verbatim}
326    }
 All of the variables used to implement the lookup tables are described  
 in \filelink{model/src/write\_state.F}{model-src-write_state.F}  
   
327    
328    While it is easiest to write variables within typical 2D and 3D fields
329    where all data is known at a given time, it is also possible to write
330    fields where only a portion (\textit{eg.} a ``slab'' or ``slice'') is
331    known at a given instant.  An example is provided within
332    \filelink{pkg/mom\_vecinv/mom\_vecinv.F}{pkg-mom_vecinv-mom_vecinv.F}
333    where an offset vector is used: {\footnotesize
334    \begin{verbatim}
335           IF (useMNC .AND. snapshot_mnc) THEN
336             CALL MNC_CW_RL_W_OFFSET('D','mom_vi',bi,bj, 'fV', uCf,
337       &          offsets, myThid)
338             CALL MNC_CW_RL_W_OFFSET('D','mom_vi',bi,bj, 'fU', vCf,
339       &          offsets, myThid)
340           ENDIF
341    \end{verbatim}
342    }
343    to write a 3D field one depth slice at a time.
344    
345  \subsection{Package Reference}  Each element in the offset vector corresponds (in order) to the
346    dimensions of the ``full'' (or virtual) array and specifies which are
347    known at the time of the call.  A zero within the offset array means
348    that all values along that dimension are available while a positive
349    integer means that only values along that index of the dimension are
350    available.  In all cases, the matrix passed is assumed to start (that
351    is, have an in-memory structure) coinciding with the start of the
352    specified slice.  Thus, using this offset array mechanism, a slice
353    can be written along any single dimension or combinations of
354    dimensions.
355    
Legend:
Removed from v.1.5  
changed lines
  Added in v.1.13
  ViewVC Help
Powered by ViewVC 1.1.22