/[MITgcm]/manual/s_phys_pkgs/mnc.tex
ViewVC logotype

Annotation of /manual/s_phys_pkgs/mnc.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph


Revision 1.12 - (hide annotations) (download) (as text)
Tue Oct 12 18:16:03 2004 UTC (19 years, 7 months ago) by edhill
Branch: MAIN
Changes since 1.11: +167 -92 lines
File MIME type: application/x-tex
 o add HTML reference tags so that URLs are more permanent and legible
 o update MNC

1 edhill 1.12 % $Header: /u/gcmpack/manual/part6/mnc.tex,v 1.11 2004/09/22 15:11:32 edhill Exp $
2 edhill 1.1 % $Name: $
3    
4 edhill 1.6 \section{NetCDF I/O Integration: MNC}
5 edhill 1.3 \label{sec:pkg:mnc}
6 edhill 1.12 \begin{rawhtml}
7     <!-- CMIREDIR:package_mnc: -->
8     \end{rawhtml}
9 edhill 1.1
10 edhill 1.3 The \texttt{mnc} package is a set of convenience routines written to
11     expedite the process of creating, appending, and reading NetCDF files.
12     NetCDF is an increasingly popular self-describing file format
13     \cite{rew:97} intended primarily for scientific data sets. An
14     extensive collection of NetCDF reference papers, user guides,
15     software, FAQs, and other information can be obtained from UCAR's web
16     site at:
17     \begin{rawhtml} <A href="http://www.unidata.ucar.edu/packages/netcdf/"> \end{rawhtml}
18     \begin{verbatim}
19     http://www.unidata.ucar.edu/packages/netcdf/
20     \end{verbatim}
21     \begin{rawhtml} </A> \end{rawhtml}
22 edhill 1.1
23    
24 edhill 1.12 \subsection{Using MNC}
25    
26     \subsubsection{MNC Configuration and Inputs}
27    
28     As with all MITgcm packages, MNC can be turned on/off at compile time
29     using the \texttt{packages.conf} file or the genmake2
30     \texttt{-enable=mnc} or \texttt{-disable=mnc} switches.
31    
32     For run-time configuration, most of the MNC--related model parameters
33     are contained within a Fortran namelist file called \texttt{data.mnc}.
34     If this file does not exist, then the MNC package will interpret that
35     as an indication that it is not to be used. If the \texttt{data.mnc}
36     file does exist, then it may contain the following parameters:
37    
38     \begin{center}
39     {\footnotesize
40     \begin{tabular}[htb]{|l|c|l|l|}\hline
41     \textbf{Name} & \textbf{T} &
42     \textbf{Default} & \textbf{Description} \\\hline
43     & & & \\
44     \texttt{useMNC} & L & \texttt{.FALSE.} &
45     \textbf{overall MNC ON/OFF switch} \\
46     \texttt{mnc\_echo\_gvtypes} & L & \texttt{.FALSE.} &
47     echo pre-defined ``types'' (debugging) \\
48     \texttt{mnc\_use\_outdir} & L & \texttt{.FALSE.} &
49     create a directory for output \\
50     \texttt{mnc\_outdir\_str} & S & \texttt{'mnc\_'} &
51     output directory name \\
52     \texttt{mnc\_outdir\_date} & L & \texttt{.FALSE.} &
53     embed date in the output dir name \\
54     \texttt{pickup\_write\_mnc} & L & \texttt{.FALSE.} &
55     use MNC to write (create) pickup files \\
56     \texttt{pickup\_read\_mnc} & L & \texttt{.FALSE.} &
57     use MNC to read pickup files \\
58     \texttt{mnc\_use\_indir} & L & \texttt{.FALSE.} &
59     use a directory (path) for input \\
60     \texttt{mnc\_indir\_str} & S & \texttt{''} &
61     input directory (or path) name \\
62     \texttt{snapshot\_mnc} & L & \texttt{.FALSE.} &
63     write \texttt{snapshot} (instantaneous) w/MNC \\
64     \texttt{monitor\_mnc} & L & \texttt{.FALSE.} &
65     write \texttt{monitor} w/MNC \\
66     \texttt{timeave\_mnc} & L & \texttt{.FALSE.} &
67     write \texttt{timeave} w/MNC \\
68     \texttt{autodiff\_mnc} & L & \texttt{.FALSE.} &
69     write \texttt{autodiff} w/MNC \\\hline
70     \end{tabular}
71     }
72     \end{center}
73    
74     Additional MNC--related parameters are contained within the main
75     \texttt{data} namelist file and in some of the namelist files for
76     individual packages. These options are:
77     \begin{center}
78     {\footnotesize
79     \begin{tabular}[htb]{|l|c|l|l|}\hline
80     \textbf{Name} & \textbf{T} &
81     \textbf{Default} & \textbf{Description} \\\hline
82     \multicolumn{4}{|c|}{\ } \\
83     \multicolumn{4}{|c|}{Main namelist file:
84     ``\textbf{data}''} \\\hline
85     \texttt{snapshot\_ioinc} & L & \texttt{.FALSE.} &
86     write \texttt{snapshot} ``inclusively'' \\
87     \texttt{timeave\_ioinc} & L & \texttt{.FALSE.} &
88     write \texttt{timeave} ``inclusively'' \\
89     \texttt{monitor\_ioinc} & L & \texttt{.FALSE.} &
90     write \texttt{monitor} ``inclusively'' \\
91     \texttt{the\_run\_name} & C & ``name...'' &
92     name is included in all MNC output \\\hline
93     \multicolumn{4}{|c|}{\ } \\
94     \multicolumn{4}{|c|}{Diagnostics namelist file:
95     ``\textbf{data.diagnostics}''} \\\hline
96     \texttt{diag\_mnc} & L & \texttt{.FALSE.} &
97     write \texttt{diagnostics} w/MNC \\
98     \texttt{diag\_ioinc} & L & \texttt{.FALSE.} &
99     write \texttt{diagnostics} ``inclusively'' \\\hline
100     \end{tabular}
101     }
102     \end{center}
103    
104     By default, turning on MNC for a particular output type will result in
105     turning off all the corresponding (usually, default) MDSIO or STDOUT
106     output mechanisms. In other words, output defaults to being an
107     exclusive selection. To enable multiple kinds of simultaneous output,
108     flags of the form \texttt{NAME\_ioinc} have been created where
109     \texttt{NAME} corresponds to the various MNC output flags. When a
110     \texttt{NAME\_ioinc} flag is set to \texttt{.TRUE.}, then multiple
111     simultaneous forms of output are allowed for the \texttt{NAME} output
112     mechanism. The intent of this design is that typical users will only
113     want one kind of output while people debugging the code (particularly
114     the I/O routines) may want simultaneous types of output.
115    
116     This ``inclusive'' versus ``exclusive'' design is easily applied in
117     cases where three or more kinds of output may be generated. Thus, it
118     can be readily extended to additional new output types (eg. HDF5).
119    
120     Input types are always exclusive.
121    
122     \subsubsection{MNC Output}
123    
124     While NetCDF files are supposed to be ``self-describing'', it is
125     helpful to note the following:
126    
127     \begin{itemize}
128     \item The constraints placed upon the ``unlimited'' (or ``record'')
129     dimension inherent with NetCDF v3.x make it very inefficient to put
130     variables written at potentially different intervals within the same
131     file. For this reason, MNC output is split into a few file ``base
132     names'' which try to reflect the nature of their content.
133    
134     \item All MNC output is currently done in a ``tile-per-file'' fashion
135     since most NetCDF v3.x implementions cannot write safely within MPI
136     or multi-threaded environments. This tiling is done in a global
137     fashion and the tile numbers are appended to the base names
138     described above. Some scripts to ``assemble'' output are available
139     (\texttt{MITgcm/utils/matlab}). More general manipulations can be
140     accomplished with the
141     \begin{rawhtml}
142     <A href="http://nco.sourceforge.net">
143     \end{rawhtml}
144     \begin{verbatim}
145     NetCDF Operators (or ``NCO'') at http://nco.sourceforge.net
146     \end{verbatim}
147     \begin{rawhtml} </A> \end{rawhtml}
148     which is a very powerful and convenient set of tools for working
149     with all NetCDF files.
150    
151     \item MNC does not (yet) provide a mechanism for reading information
152     from a single ``global'' file as can be done with the MDSIO
153     package.
154    
155     \end{itemize}
156    
157    
158     \subsection{MNC Internals}
159 edhill 1.1
160 edhill 1.3 The \texttt{mnc} package is a two-level convenience library (or
161     ``wrapper'') for most of the NetCDF Fortran API. Its purpose is to
162     streamline the user interface to NetCDF by maintaining internal
163 edhill 1.6 relations (look-up tables) keyed with strings (or names) and entities
164     such as NetCDF files, variables, and attributes.
165 edhill 1.3
166     The two levels of the \texttt{mnc} package are:
167     \begin{description}
168    
169     \item[Upper level] \
170    
171     The upper level contains information about two kinds of
172     associations:
173     \begin{description}
174 edhill 1.6 \item[grid type] is lookup table indexed with a grid type name.
175 edhill 1.3 Each grid type name is associated with a number of dimensions, the
176     dimension sizes (one of which may be unlimited), and starting and
177     ending index arrays. The intent is to store all the necessary
178     size and shape information for the Fortran arrays containing
179     MITgcm--style ``tile'' variables (that is, a central region
180     surrounded by a variably-sized ``halo'' or exchange region as
181     shown in Figures \ref{fig:communication_primitives} and
182     \ref{fig:tiling-strategy}).
183    
184 edhill 1.6 \item[variable type] is a lookup table indexed by a variable type
185 edhill 1.3 name. For each name, the table contains a reference to a grid
186     type for the variable and the names and values of various
187     attributes.
188     \end{description}
189    
190     Within the upper level, these associations are not permanently tied
191     to any particular NetCDF file. This allows the information to be
192     re-used over multiple file reads and writes.
193    
194     \item[Lower level] \
195    
196     In the lower (or internal) level, associations are stored for NetCDF
197     files and many of the entities that they contain including
198     dimensions, variables, and global attributes. All associations are
199     on a per-file basis. Thus, each entity is tied to a unique NetCDF
200     file and will be created or destroyed when files are, respectively,
201     opened or closed.
202    
203     \end{description}
204 edhill 1.1
205    
206 edhill 1.12 \subsubsection{MNC Grid--Types and Variable--Types}
207 edhill 1.5
208     As a convenience for users, the MNC package includes numerous routines
209     to aid in the writing of data to NetCDF format. Probably the biggest
210     convenience is the use of pre-defined ``grid types'' and ``variable
211     types''. These ``types'' are simply look-up tables that store
212     dimensions, indicies, attributes, and other information that can all
213     be retrieved using a single character string.
214    
215     The ``grid types'' are a way of mapping variables within MITgcm to
216     NetCDF arrays. Within MITgcm, most spatial variables are defined
217     using two-- or three--dimensional arrays with ``overlap'' regions (see
218     Figures \ref{fig:communication_primitives}, a possible vertical index,
219     and \ref{fig:tiling-strategy}) and tile indicies such as the following
220     ``U'' velocity:
221     \begin{verbatim}
222     _RL uVel (1-OLx:sNx+OLx,1-OLy:sNy+OLy,Nr,nSx,nSy)
223     \end{verbatim}
224     as defined in \filelink{model/inc/DYNVARS.h}{model-inc-DYNVARS.h}
225    
226     The grid type is a character string that encodes the presence and
227     types associated with the four possible dimensions. The character
228     string follows the format
229     \begin{center}
230     \texttt{H0\_H1\_H2\_\_V\_\_T}
231     \end{center}
232 edhill 1.6 where the terms \textit{H0}, \textit{H1}, \textit{H2}, \textit{V},
233     \textit{T} can be almost any combination of the following:
234 edhill 1.5 \begin{center}
235     \begin{tabular}[h]{|ccc|c|c|}\hline
236 edhill 1.6 \multicolumn{3}{|c|}{Horizontal} & Vertical & Time \\
237 edhill 1.7 \textbf{H0}: location & \textbf{H1}: dimensions & \textbf{H2}: halo
238     & \textbf{V}: location & \textbf{T}: level \\\hline
239 edhill 1.5 \texttt{-} & xy & Hn & \texttt{-} & \texttt{-} \\
240     U & x & Hy & i & t \\
241     V & y & & c & \\
242     Cen & & & & \\
243     Cor & & & & \\\hline
244     \end{tabular}
245     \end{center}
246     A example list of all pre-defined combinations is contained in the
247     file
248     \begin{center}
249     \texttt{pkg/mnc/pre-defined\_grids.txt}.
250     \end{center}
251 edhill 1.7
252     The variable type is an association between a variable type name and the
253     following items:
254     \begin{center}
255 edhill 1.9 \begin{tabular}[h]{|l|l|}\hline
256 edhill 1.7 \textbf{Item} & \textbf{Purpose} \\\hline
257     grid type & defines the in-memory arrangement \\
258     \texttt{bi,bj} dimensions & tiling indices, if present \\\hline
259     \end{tabular}
260     \end{center}
261     and is used by the \texttt{mnc\_cw\_*\_[R|W]} subroutines for reading
262     and writing variables.
263 edhill 1.5
264    
265 edhill 1.12 \subsubsection{Using MNC: Examples}
266 edhill 1.5
267     Writing variables to NetCDF files can be accomplished in as few as two
268     function calls. The first function call defines a variable type,
269     associates it with a name (character string), and provides additional
270 edhill 1.6 information about the indicies for the tile (\texttt{bi},\texttt{bj})
271     dimensions. The second function call will write the data at, if
272     necessary, the current time level within the model.
273 edhill 1.5
274     Examples of the initialization calls can be found in the file
275 edhill 1.8 \filelink{model/src/ini\_mnc\_io.F}{model-src-ini_mnc_io.F}
276 edhill 1.9 where these function calls:
277 edhill 1.6 {\footnotesize
278 edhill 1.5 \begin{verbatim}
279 edhill 1.8 C Create MNC definitions for DYNVARS.h variables
280     CALL MNC_CW_ADD_VNAME('iter', '-_-_--__-__t', 0,0, myThid)
281     CALL MNC_CW_ADD_VATTR_TEXT('iter',1,
282     & 'long_name','iteration_count', myThid)
283    
284     CALL MNC_CW_ADD_VNAME('model_time', '-_-_--__-__t', 0,0, myThid)
285     CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,
286     & 'long_name','Model Time', myThid)
287     CALL MNC_CW_ADD_VATTR_TEXT('model_time',1,'units','s', myThid)
288    
289     CALL MNC_CW_ADD_VNAME('U', 'U_xy_Hn__C__t', 4,5, myThid)
290     CALL MNC_CW_ADD_VATTR_TEXT('U',1,'units','m/s', myThid)
291     CALL MNC_CW_ADD_VATTR_TEXT('U',1,
292     & 'coordinates','XU YU RC iter', myThid)
293    
294     CALL MNC_CW_ADD_VNAME('T', 'Cen_xy_Hn__C__t', 4,5, myThid)
295     CALL MNC_CW_ADD_VATTR_TEXT('T',1,'units','degC', myThid)
296     CALL MNC_CW_ADD_VATTR_TEXT('T',1,'long_name',
297     & 'potential_temperature', myThid)
298     CALL MNC_CW_ADD_VATTR_TEXT('T',1,
299     & 'coordinates','XC YC RC iter', myThid)
300 edhill 1.6 \end{verbatim}
301     }
302 edhill 1.9 {\noindent initialize four \texttt{VNAME}s and add one or more NetCDF
303     attributes to each.}
304 edhill 1.5
305 edhill 1.9 The four variables defined above are subsequently written at specific
306 edhill 1.6 time steps within
307     \filelink{model/src/write\_state.F}{model-src-write_state.F}
308     using the function calls:
309     {\footnotesize
310 edhill 1.5 \begin{verbatim}
311 edhill 1.8 C Write dynvars using the MNC package
312     CALL MNC_CW_SET_UDIM('state', -1, myThid)
313 edhill 1.10 CALL MNC_CW_I_W('I','state',0,0,'iter', myIter, myThid)
314 edhill 1.8 CALL MNC_CW_SET_UDIM('state', 0, myThid)
315     CALL MNC_CW_RL_W('D','state',0,0,'model_time',myTime, myThid)
316     CALL MNC_CW_RL_W('D','state',0,0,'U', uVel, myThid)
317     CALL MNC_CW_RL_W('D','state',0,0,'T', theta, myThid)
318 edhill 1.6 \end{verbatim}
319     }
320 edhill 1.5
321 edhill 1.12 While it is easiest to write variables within typical 2D and 3D fields
322     where all data is known at a given time, it is also possible to write
323     fields where only a portion (\textit{eg.} a ``slab'' or ``slice'') is
324     known at a given instant. An example is provided within
325     \filelink{pkg/mom\_vecinv/mom\_vecinv.F}{pkg-mom_vecinv-mom_vecinv.F}
326     where an offset vector is used: {\footnotesize
327     \begin{verbatim}
328     IF (useMNC .AND. snapshot_mnc) THEN
329     CALL MNC_CW_RL_W_OFFSET('D','mom_vi',bi,bj, 'fV', uCf,
330     & offsets, myThid)
331     CALL MNC_CW_RL_W_OFFSET('D','mom_vi',bi,bj, 'fU', vCf,
332     & offsets, myThid)
333     ENDIF
334     \end{verbatim}
335     }
336     to write a 3D field one depth slice at a time.
337 edhill 1.1
338 edhill 1.12 Each element in the offset vector corresponds (in order) to the
339     dimensions of the ``full'' (or virtual) array and specifies which are
340     known at the time of the call. A zero within the offset array means
341     that all values along that dimension are available while a positive
342     integer means that only values along that index of the dimension are
343     available. In all cases, the matrix passed is assumed to start (that
344     is, have an in-memory structure) coinciding with the start of the
345     specified slice. Thus, using this offset array mechanism, a slice
346     can be written along any single dimension or combinations of
347     dimensions.
348 edhill 1.9

  ViewVC Help
Powered by ViewVC 1.1.22