/[MITgcm]/MITgcm_contrib/llc_hires/llc_90/code-async/readme.txt
ViewVC logotype

Diff of /MITgcm_contrib/llc_hires/llc_90/code-async/readme.txt

Parent Directory Parent Directory | Revision Log Revision Log | View Revision Graph Revision Graph | View Patch Patch

revision 1.2 by dimitri, Tue Oct 3 02:33:26 2017 UTC revision 1.4 by dimitri, Sun Mar 3 13:59:35 2019 UTC
# Line 1  Line 1 
1  Right now some sizes need to be configured manually:  From Bron, February 28, 2019
2    
3  recvTask.c   lines 79-82  In the newest version, it is no longer necessary to hand-edit the
4    constants in "recvTask.c" and "readtile_mpiio.c".  Instead, the file
5  #define NUM_X   90  "SIZE.h" has been modified in two ways:
6  #define NUM_Y   1170L                     // get rid of this someday    (1) SIZE.h  now includes the constant "sFacet"
7  #define NUM_Z   50    (2) SIZE.h  may now be #include in both C and Fortran files
8  #define MULTDIM  7  This means that "recvTask.c" and "readtile_mpiio.c" now get the
9    information they need directly from "SIZE.h", so the magic constants
10    for the run only need to be edited in one place (namely, SIZE.h).
 and  
   
 readtile_mpiio.c    lines 115-119  
   
     facetElements1D = 90;  
     tileSizeX = 30;  
     tileSizeY = 30;  
     xGhosts = 8;  
     yGhosts = 8;  
11    
12  One tile per rank is recommended, mostly for pickup input performance,  One tile per rank is recommended, mostly for pickup input performance,
13  but it is not strictly necessary.  but it is not strictly necessary.  A minimum of one full node of I/O
14    ranks is required.  The async I/O does allocate whole nodes to be
15    either an I/O node, or a compute node.  It is permitted for the last
16    *compute* node to have a "ragged edge", i.e., have fewer MPI processes
17    on it than the other nodes do.  But the I/O nodes are all "full size".
18    
19    The other minimum value the I/O code requires is that there must
20    be at least one core for each field you want to write, e.g., if you
21    are dumping 20 different fields, there must be at least 20 cores
22    allocated to the I/O.  Note that the 20 (or whatever) number is
23    *aggregate* across all the I/O nodes, NOT a "per node" number.
24    
25  Choose dumpFreq and pChkptFreq as usual. We're not set up  Choose dumpFreq and pChkptFreq as usual. We're not set up
26  to do the rolling checkpoints yet. It'll dump u,v,t, and etan now -  to do the rolling checkpoints yet. It'll dump u,v,t, and etan now -
27  send me a list of other fields you want, as it is rather involved  send me a list of other fields you want, as it is rather involved
28  to change them. But this should be enough to see if it works.  to change them. But this should be enough to see if it works.
29    
30    Set run-time parameter: useSingleCPUio=.FALSE.
31    
32    Only a couple of files are different from previous version.
33    But note in particular that  "SIZE.h"  is a new file in that directory,
34    and "recvTask.c" has a huge number of changes.
35    
36    The input scheme implemented here is only invoked on
37    the 64bit pickup files.  It is specific to the LLC decomposition and will
38    not work on e.g. the Monterey high-res simulations we did a couple years
39    ago.  (Although, the code should work for any facet size as specified
40    in SIZE.h)  The format of SIZE.h was changed so that it can be included
41    in both C and Fortran files, and I also added the "sFacet" constant that
42    specifies the base facet size (e.g. 1080).  So SIZE.h will probably look
43    kinda weird at first, but shouldn't be hard to figure out.  The major
44    advantage is that now you no longer need to edit any magic constants in
45    recvTask.c and readtile_mpiio.c - they now derive the info they need by
46    directly including SIZE.h
47    
48    The code now automatically figures out how many ranks are running per node.
49    You can run with whatever number of ranks per node that you want, but the
50    number needs to be consistent for all nodes (except possibly the last node,
51    which can be short).
52    
53    The initial burst of output generated by recvTask.c (the "map" describing
54    the way the I/O processes are allocated) is now somewhat longer and more
55    detailed, but can continue to be ignored.
56    
57    I did NOT try to cure the "integer" problem.  It seems that the code is
58    getting fairly close to bumping into the 2G (i.e. 2^31) limit on numbers
59    that fit into a default integer.  I *think* you can probably do one more
60    doubling of the resolution (to 8640), but I'm also pretty sure that going
61    past that will break the code.

Legend:
Removed from v.1.2  
changed lines
  Added in v.1.4

  ViewVC Help
Powered by ViewVC 1.1.22