| 20 |
the \texttt{rw} package. |
the \texttt{rw} package. |
| 21 |
|
|
| 22 |
\subsubsection{Using MDSIO} |
\subsubsection{Using MDSIO} |
| 23 |
|
The \texttt{mdsio} package is geared toward the reading and writing of |
| 24 |
|
floating point (Fortran \texttt{REAL*4} or \texttt{REAL*8}) arrays. |
| 25 |
|
It assumes that the in-memory layout of all arrays follows the per-tile |
| 26 |
|
MITgcm convention |
| 27 |
|
\begin{verbatim} |
| 28 |
|
C Example of a "2D" array |
| 29 |
|
_RL anArray(1-OLx:sNx+OLx,1-OLy:sNy+OLy,nSx,nSy) |
| 30 |
|
|
| 31 |
|
C Example of a "3D" array |
| 32 |
|
_RL anArray(1-OLx:sNx+OLx,1-OLy:sNy+OLy,1:Nr,nSx,nSy) |
| 33 |
|
\end{verbatim} |
| 34 |
|
where the first two dimensions are spatial or ``horizontal'' indicies |
| 35 |
|
that include a ``halo'' or exchange region (please see |
| 36 |
|
Chapters \ref{chap:sarch} and \ref{sec:exch2} which describe domain |
| 37 |
|
decomposition), and the remaining indicies (\texttt{Nr},\texttt{nSx}, |
| 38 |
|
and \texttt{nSx}) are often present but not required. |
| 39 |
|
|
| 40 |
|
In order to write output, the \texttt{mdsio} package is called with a |
| 41 |
|
function such as: |
| 42 |
|
\begin{verbatim} |
| 43 |
|
CALL MDSWRITEFIELD(fn,prec,lgf,typ,Nr,arr,irec,myIter,myThid) |
| 44 |
|
\end{verbatim} |
| 45 |
|
where: |
| 46 |
|
\begin{quote} |
| 47 |
|
\begin{description} |
| 48 |
|
\item[\texttt{fn}] is a \texttt{CHARACTER} string containing a file |
| 49 |
|
``base'' name which will then be used to create file names that |
| 50 |
|
contain tile and/or model iteration indicies |
| 51 |
|
\item[\texttt{prec}] is an integer that contains one of two globally |
| 52 |
|
defined values (\texttt{precFloat64} or \texttt{precFloat32}) |
| 53 |
|
\item[\texttt{lgf}] is a \texttt{LOGICAL} that typically contains |
| 54 |
|
the globally defined \texttt{globalFile} option which specifies |
| 55 |
|
the creation of globally (spatially) concatenated files |
| 56 |
|
\item[\texttt{typ}] is a \texttt{CHARACTER} string that specifies |
| 57 |
|
the type of the variable being written (\texttt{'RL'} or |
| 58 |
|
\texttt{'RS'}) |
| 59 |
|
\item[\texttt{Nr}] is an integer that specifies the number of |
| 60 |
|
vertical levels within the variable being written |
| 61 |
|
\item[\texttt{arr}] is the variable (array) to be written |
| 62 |
|
\item[\texttt{irec}] is the starting record within the output file |
| 63 |
|
that will contain the array |
| 64 |
|
\item[\texttt{myIter,myThid}] are integers containing, respectively, |
| 65 |
|
the current model iteration count and the unique thread ID for the |
| 66 |
|
current context of execution |
| 67 |
|
\end{description} |
| 68 |
|
\end{quote} |
| 69 |
|
As one can see from the above (generic) example, enough information is |
| 70 |
|
made available (through both the argument list and through common blocks) |
| 71 |
|
for the \texttt{mdsio} package to perform the following tasks: |
| 72 |
|
\begin{enumerate} |
| 73 |
|
\item open either a per-tile file such as: |
| 74 |
|
\begin{center} |
| 75 |
|
\texttt{uVel.0000302400.003.001.data} |
| 76 |
|
\end{center} |
| 77 |
|
or a ``global'' file such as |
| 78 |
|
\begin{center} |
| 79 |
|
\texttt{uVel.0000302400.data} |
| 80 |
|
\end{center} |
| 81 |
|
\item byte-swap (as necessary) the input array and write its contents |
| 82 |
|
(minus any halo information) to the binary file -- or to the correct |
| 83 |
|
location within the binary file if the globalfile option is used, and |
| 84 |
|
\item create an ASCII--text metadata file (same name as the binary but |
| 85 |
|
with a \texttt{.meta} extension) describing the binary file contents |
| 86 |
|
(often, for later use with the MatLAB \texttt{rdmds()} utility). |
| 87 |
|
\end{enumerate} |
| 88 |
|
|
| 89 |
|
Reading output with \texttt{mdsio} is very similar to writing it. A |
| 90 |
|
typical function call is |
| 91 |
|
\begin{verbatim} |
| 92 |
|
CALL MDSREADFIELD(fn,prec,typ,Nr,arr,irec,myThid) |
| 93 |
|
\end{verbatim} |
| 94 |
|
where variables are exactly the same as the \texttt{MDSWRITEFIELD} |
| 95 |
|
example provided above. It is important to note that the \texttt{lgf} |
| 96 |
|
argument is missing from the \texttt{MDSREADFIELD} function. By |
| 97 |
|
default, \texttt{mdsio} will first try to read from an appropriately |
| 98 |
|
named global file and, failing that, will try to read from a per-tile |
| 99 |
|
file. |
| 100 |
|
|
| 101 |
|
|
| 102 |
|
\subsubsection{Important Considerations} |
| 103 |
|
When using \texttt{mdsio}, one should be aware of the following |
| 104 |
|
package features and limitations: |
| 105 |
|
\begin{description} |
| 106 |
|
\item[Byte-swapping] is, for the most part, gracefully handled. All |
| 107 |
|
files intended for reading/writing by \texttt{mdsio} should contain |
| 108 |
|
big-endian (sometimes called ``network byte order'') data. By |
| 109 |
|
handling byte-swapping within the model, MITgcm output is more |
| 110 |
|
easily ported between different machines, architectures, compilers, |
| 111 |
|
etc. Byteswapping can be turned on/off at compile time within |
| 112 |
|
\texttt{mdsio} using the \texttt{\_BYTESWAPIO} CPP macro which is |
| 113 |
|
usually set within a \texttt{genmake2} options file or |
| 114 |
|
``\texttt{optfile}'' which are located in |
| 115 |
|
\begin{verbatim} |
| 116 |
|
MITgcm/tools/build_options |
| 117 |
|
\end{verbatim} |
| 118 |
|
Additionally, some compilers may have byte-swap options that are |
| 119 |
|
speedier or more convenient to use. |
| 120 |
|
|
| 121 |
|
\item[Types] are currently limited to single-- or double--precision |
| 122 |
|
floating point values. These values can be converted, on-the-fly, |
| 123 |
|
from one to the other so that any combination of either single-- or |
| 124 |
|
double--precision variables can be read from or written to files |
| 125 |
|
containing either single-- or double--precision data. |
| 126 |
|
|
| 127 |
|
\item[Array sizes] are limited. The \texttt{mdsio} package is very |
| 128 |
|
much geared towards the reading/writing of per-tile (that is, |
| 129 |
|
domain-decomposed and halo-ed) arrays. Data that cannot be made to |
| 130 |
|
``fit'' within these assumed sizes can be challenging to read or |
| 131 |
|
write with \texttt{mdsio}. |
| 132 |
|
|
| 133 |
|
\item[Tiling] or domain decomposition is, for logically rectangular |
| 134 |
|
grid topologies and ``standard'' cubesphere topologies, gracefully |
| 135 |
|
handled by \texttt{mdsio}. The \texttt{mdsio} package can, without |
| 136 |
|
any coding changes, read and write to/from files that were run on |
| 137 |
|
the same global grid but with different tiling (grid decomposition) |
| 138 |
|
schemes. For example, \texttt{mdsio} can use and/or create |
| 139 |
|
identical input/output files for a ``C32'' cube when the model is |
| 140 |
|
run with either 6, 12, or 24 tiles (corresponding to 1, 2 or 4 tiles |
| 141 |
|
per cubesphere face). Currently, this is one of the primary |
| 142 |
|
advantages that the \texttt{mdsio} package has over \texttt{mnc}. |
| 143 |
|
|
| 144 |
|
\item[Meta-data] is written on a per-file basis using a second file |
| 145 |
|
with a \texttt{.meta} extension as described above. One should be |
| 146 |
|
careful not to delete the metadata files when using convenient |
| 147 |
|
MatLAB post-processing scripts such as \texttt{rdmds()}. |
| 148 |
|
|
| 149 |
|
\item[Numerous files] can be written by \texttt{mdsio} due to its |
| 150 |
|
typically per-time-step and per-variable orientation. The creation of |
| 151 |
|
both a binary (\texttt{*.data}) and ASCII text meta--data |
| 152 |
|
(\texttt{*.meta}) file for each output type step tends to exacerbate |
| 153 |
|
the problem. Some (mostly, older) operating systems do not |
| 154 |
|
gracefully handle large numbers (\textit{eg.} many thousands) of |
| 155 |
|
files within one directory. So care should be taken to split output |
| 156 |
|
into smaller groups using subdirectories. |
| 157 |
|
|
| 158 |
|
\item[Overwriting] is the \textbf{default behavior} for |
| 159 |
|
\texttt{mdsio}. If a model tries to write to a file name that |
| 160 |
|
already exists, the older file \textbf{will be deleted}. For this |
| 161 |
|
reason, MITgcm users should be careful to move output that that wish |
| 162 |
|
to keep into, for instance, subdirectories before performing |
| 163 |
|
subsequent runs that may over--lap in time or otherwise produce |
| 164 |
|
files with identical names (\textit{eg.} Monte-Carlo simulations). |
| 165 |
|
|
| 166 |
|
\item[No ``halo'' information] is written or read by \texttt{mdsio}. |
| 167 |
|
Along the horizontal dimensions, all variables are written in an |
| 168 |
|
\texttt{sNx}--by--\texttt{sNy} fashion. So, although variables |
| 169 |
|
(arrays) may be defined at different locations on Arakawa grids [U |
| 170 |
|
(right/left horizontal edges), V (top/bottom horizontal edges), M |
| 171 |
|
(mass or cell center), or Z (vorticity or cell corner) points], they |
| 172 |
|
are all written using only interior (\texttt{1:sNx} and |
| 173 |
|
\texttt{1:sNy}) values. For quantities defined at U, V, and M |
| 174 |
|
points, writing \texttt{1:sNx} and \texttt{1:sNy} for every tile is |
| 175 |
|
sufficient to ensure that all values are written globally for some |
| 176 |
|
grids (eg. cubesphere, re-entrant channels, and doubly-periodic |
| 177 |
|
rectangular regions). For Z points, failing to write values at the |
| 178 |
|
\texttt{sNx+1} and \texttt{sNy+1} locations means that, for some |
| 179 |
|
tile topologies, not all values are written. For instance, with a |
| 180 |
|
cubesphere topology at least two corner values are ``lost'' (fail to |
| 181 |
|
be written for any tile) if the \texttt{sNx+1} and \texttt{sNy+1} |
| 182 |
|
values are ignored. To fix this problem, the \texttt{mnc} package |
| 183 |
|
writes the \texttt{sNx+1} and \texttt{sNy+1} grid values for the U, |
| 184 |
|
V, and Z locations. Also, the \texttt{mnc} package is capable of |
| 185 |
|
reading and/or writing entire halo regions and more complicated |
| 186 |
|
array shapes which can be helpful when debugging--features that |
| 187 |
|
do not exist within \texttt{mdsio}. |
| 188 |
|
\end{description} |
| 189 |
|
|
| 190 |
|
|
| 191 |
\subsection{RW Basic binary I/O utilities} |
\subsection{RW Basic binary I/O utilities} |
Parent Directory
| 
Revision Log
| 
Revision Graph
| 
Patch