1 |
|
============================================ |
2 |
Example: "4x4 Global Simulation with Seasonal Forcing" |
Example: "4x4 Global Simulation with Seasonal Forcing" |
3 |
============================================ |
============================================ |
4 |
(see also similar set-up in: verification/tutorial_global_oce_latlon/) |
(see also similar set-up in: verification/tutorial_global_oce_latlon/) |
50 |
../build/mitgcmuv > output.dwnslp.txt |
../build/mitgcmuv > output.dwnslp.txt |
51 |
cd .. |
cd .. |
52 |
|
|
53 |
|
|
54 |
|
============================================ |
55 |
|
Use of "blank tiles" in conjunction with exch2 package: |
56 |
|
============================================ |
57 |
|
|
58 |
|
This verification experiment also demonstrate the omission of tiles |
59 |
|
(or processors) for tiles that are fully land cover and don't need computation. |
60 |
|
The relevant config. files to be manipulated are: |
61 |
|
* at compile time, in dir. code/: packages.conf, SIZE.h |
62 |
|
* at run time, in dir. input/: data.exch2 |
63 |
|
|
64 |
|
To enable this feature requires package "exch2" |
65 |
|
(see Section 6.2.4 "exch2: Extended Cubed Sphere Topology" of online manual), |
66 |
|
i.e. in code/packages.conf add "exch2" |
67 |
|
|
68 |
|
The basic layout of the experiment is Nx*Ny = 90x40. |
69 |
|
In a single-processor config. with very small tile sizes (sNx*sNy=10*10) |
70 |
|
this can be represented, e.g. via |
71 |
|
& sNx = 10, |
72 |
|
& sNy = 10, |
73 |
|
& OLx = 3, |
74 |
|
& OLy = 3, |
75 |
|
& nSx = 9, |
76 |
|
& nSy = 4, |
77 |
|
& nPx = 1, |
78 |
|
& nPy = 1, |
79 |
|
i.e. we use nSx*nSy=9*4=36 virtual processors. |
80 |
|
|
81 |
|
An equivalent parallel setup using 9 virtual and 4 real processors |
82 |
|
would look like: |
83 |
|
& sNx = 10, |
84 |
|
& sNy = 10, |
85 |
|
& OLx = 3, |
86 |
|
& OLy = 3, |
87 |
|
& nSx = 9, |
88 |
|
& nSy = 1, |
89 |
|
& nPx = 1, |
90 |
|
& nPy = 4, |
91 |
|
|
92 |
|
In this layout it turns out that tile number 30 is "empty", i.e. fully land covered. |
93 |
|
We wish to remove this tile from our calculation. How to proceed? |
94 |
|
|
95 |
|
1. Find out which tiles to eliminate via a config. that uses all tiles |
96 |
|
--- |
97 |
|
1.1. At compile time: |
98 |
|
* add line "exch2" to packages.conf |
99 |
|
* configure SIZE.h using your desired individual tile size, e.g. sNx*sNy=10*10, as follows: |
100 |
|
& sNx = 10, |
101 |
|
& sNy = 10, |
102 |
|
& OLx = 3, |
103 |
|
& OLy = 3, |
104 |
|
& nSx = 9, |
105 |
|
& nSy = 1, |
106 |
|
& nPx = 1, |
107 |
|
& nPy = 4, |
108 |
|
As described above you are using 4 real processors with 9 virtual tiles per processor. |
109 |
|
* compile (don't forget to compile via -mpi or similar) |
110 |
|
|
111 |
|
1.2 At runtime: |
112 |
|
* need to reflect your basic layout in data.exch2 |
113 |
|
This is simple: since you are not using any non-trivial topology with |
114 |
|
multiple facets (such as cubed-sphere), |
115 |
|
you only need to specify one basic facet (Nx,Ny)=(90,40) layout via following line: |
116 |
|
dimsFacets = 90, 40, |
117 |
|
Then run the moodel, e.g. via: |
118 |
|
mpirun -np 4 ./mitgcmuv |
119 |
|
|
120 |
|
1.3 Diagnose which tile numbers are empty: |
121 |
|
* from STDOUT.000[0-3] you can infer which tiles are empty using following grep: |
122 |
|
grep "Empty tile" STDOUT.* | awk '{print " " $6 ","}' > empty_tiles.txt |
123 |
|
* In this example there's only one empty tile, and it is #30. |
124 |
|
|
125 |
|
2. Configuring with empty tiles removed |
126 |
|
--- |
127 |
|
2.1 At compile time: |
128 |
|
We've determined that 1 out of nSx*nPx*nSy*nPy=36 tiles is empty and can be removed, |
129 |
|
leaving 36-1=35 non-empty tiles. We are free to re-order nSx,nPx,nSy,nPy in any way, |
130 |
|
as long as nSx*nPx*nSy*nPy=35. Here's how it's chosen in the verif. experiment |
131 |
|
(see file SIZE.h_mpi) |
132 |
|
& sNx = 10, |
133 |
|
& sNy = 10, |
134 |
|
& OLx = 3, |
135 |
|
& OLy = 3, |
136 |
|
& nSx = 7, |
137 |
|
& nSy = 1, |
138 |
|
& nPx = 1, |
139 |
|
& nPy = 5, |
140 |
|
and for which, as required nSx*nPx*nSy*nPy=35 |
141 |
|
|
142 |
|
2.2 At runtime: |
143 |
|
We now need to specify in data.exch2 the number of the empty tile to be removed. |
144 |
|
This is done as follows (see file data.exch2.mpi): |
145 |
|
blankList = 30, |
146 |
|
If there were more empty tiles, this would be a more extensive index array, e.g. |
147 |
|
blankList = tile1, tile2, tile3, ... |
148 |
|
Now run the model (note that we've selected nPy=5, so in this example we actually |
149 |
|
*increase* the number of "real" processors used, despite reducing the number of |
150 |
|
total, i.e. "real" plus virtual tiles from 36 to 35): |
151 |
|
mpirun -np 5 ./mitgcmuv |
152 |
|
|
153 |
|
|
154 |
============================================ |
============================================ |
155 |
Adjoint set-up example: |
Adjoint set-up example: |
156 |
|
============================================ |
157 |
|
|
158 |
Configure and compile the code: |
Configure and compile the code: |
159 |
cd build |
cd build |