--- manual/s_software/text/sarch.tex	2001/11/13 20:13:55	1.6
+++ manual/s_software/text/sarch.tex	2004/01/28 19:33:04	1.11
@@ -1,10 +1,20 @@
-% $Header: /home/ubuntu/mnt/e9_copy/manual/s_software/text/sarch.tex,v 1.6 2001/11/13 20:13:55 adcroft Exp $
+% $Header: /home/ubuntu/mnt/e9_copy/manual/s_software/text/sarch.tex,v 1.11 2004/01/28 19:33:04 afe Exp $
 
-In this chapter we describe the software architecture and
-implementation strategy for the MITgcm code. The first part of this
-chapter discusses the MITgcm architecture at an abstract level. In the second 
-part of the chapter we described practical details of the MITgcm implementation 
-and of current tools and operating system features that are employed.
+This chapter focuses on describing the {\bf WRAPPER} environment within which
+both the core numerics and the pluggable packages operate. The description
+presented here is intended to be a detailed exposition and contains significant
+background material, as well as advanced details on working with the WRAPPER. 
+The tutorial sections of this manual (see sections
+\ref{sect:tutorials}  and \ref{sect:tutorialIII}) 
+contain more succinct, step-by-step instructions on running basic numerical
+experiments, of varous types, both sequentially and in parallel. For many 
+projects simply starting from an example code and adapting it to suit a 
+particular situation 
+will be all that is required.
+The first part of this chapter discusses the MITgcm architecture at an 
+abstract level. In the second part of the chapter we described practical 
+details of the MITgcm implementation and of current tools and operating system 
+features that are employed.
 
 \section{Overall architectural goals}
 
@@ -87,7 +97,7 @@
 \resizebox{!}{4.5in}{\includegraphics{part4/fit_in_wrapper.eps}}
 \end{center}
 \caption{
-Numerical code is written too fit within a software support
+Numerical code is written to fit within a software support
 infrastructure called WRAPPER. The WRAPPER is portable and
 can be specialized for a wide range of specific target hardware and 
 programming environments, without impacting numerical code that fits
@@ -110,7 +120,7 @@
 (UMA) and non-uniform memory access (NUMA) designs. Significant work has also 
 been undertaken on x86 cluster systems, Alpha processor based clustered SMP 
 systems, and on cache-coherent NUMA (CC-NUMA) systems from Silicon Graphics. 
-The MITgcm code, operating within the WRAPPER, is also used routinely used on 
+The MITgcm code, operating within the WRAPPER, is also routinely used on 
 large scale MPP systems (for example T3E systems and IBM SP systems). In all 
 cases numerical code, operating within the WRAPPER, performs and scales very 
 competitively with equivalent numerical code that has been modified to contain 
@@ -651,6 +661,12 @@
 computation is performed concurrently over as many processes and threads
 as there are physical processors available to compute.
 
+An exception to the the use of {\em bi} and {\em bj} in loops arises in the
+exchange routines used when the exch2 package is used with the cubed 
+sphere.  In this case {\em bj} is generally set to 1 and the loop runs from 
+1,{\em bi}.  Within the loop {\em bi} is used to retrieve the tile number,
+which is then used to reference exchange parameters.
+
 The amount of computation that can be embedded
 a single loop over {\em bi} and {\em bj} varies for different parts of the
 MITgcm algorithm. Figure \ref{fig:bibj_extract} shows a code extract
@@ -771,7 +787,7 @@
 forty grid points in y. The two sub-domains in each process will be computed 
 sequentially if they are given to a single thread within a single process.
 Alternatively if the code is invoked with multiple threads per process
-the two domains in y may be computed on concurrently.
+the two domains in y may be computed concurrently.
 \item
 \begin{verbatim}
       PARAMETER (
@@ -807,6 +823,7 @@
 WRAPPER is shown in figure \ref{fig:wrapper_startup}.
 
 \begin{figure}
+{\footnotesize
 \begin{verbatim}
 
        MAIN  
@@ -835,6 +852,7 @@
 
 
 \end{verbatim}
+}
 \caption{Main stages of the WRAPPER startup procedure.
 This process proceeds transfer of control to application code, which
 occurs through the procedure {\em THE\_MODEL\_MAIN()}.
@@ -917,7 +935,7 @@
 File: {\em eesupp/inc/MAIN\_PDIRECTIVES2.h}\\
 File: {\em model/src/THE\_MODEL\_MAIN.F}\\
 File: {\em eesupp/src/MAIN.F}\\
-File: {\em tools/genmake}\\
+File: {\em tools/genmake2}\\
 File: {\em eedata}\\
 CPP:  {\em TARGET\_SUN}\\
 CPP:  {\em TARGET\_DEC}\\
@@ -972,7 +990,7 @@
 
 Additionally, compile time options are required to link in the 
 MPI libraries and header files. Examples of these options 
-can be found in the {\em genmake} script that creates makefiles
+can be found in the {\em genmake2} script that creates makefiles
 for compilation. When this script is executed with the {bf -mpi}
 flag it will generate a makefile that includes
 paths for search for MPI head files and for linking in 
@@ -994,7 +1012,7 @@
 
 \fbox{ 
 \begin{minipage}{4.75in}
-File: {\em tools/genmake}
+File: {\em tools/genmake2}
 \end{minipage}
 } \\
 \paragraph{\bf Execution} The mechanics of starting a program in 
@@ -1414,6 +1432,7 @@
 
 WRAPPER layer.
 
+{\footnotesize
 \begin{verbatim}
 
        MAIN  
@@ -1441,9 +1460,11 @@
        |--THE_MODEL_MAIN   :: Numerical code top-level driver routine
 
 \end{verbatim}
+}
 
 Core equations plus packages.
 
+{\footnotesize
 \begin{verbatim}
 C
 C
@@ -1782,6 +1803,7 @@
 C                     :: events.
 C 
 \end{verbatim}
+}
 
 \subsection{Measuring and Characterizing Performance}