| 1 |
mlosch |
1.1 |
#!/bin/bash |
| 2 |
mlosch |
1.2 |
########################################################### |
| 3 |
|
|
# on May, 22, 2019 I copied the content of this directory |
| 4 |
|
|
# github.com/mjlosch/optim_m1qn3.git |
| 5 |
|
|
# I will not update this version any longer, but only the |
| 6 |
|
|
# git-version |
| 7 |
|
|
########################################################### |
| 8 |
mlosch |
1.1 |
# these are some basic instructions to an optimization of |
| 9 |
|
|
# MITgcm/verification/tutorial_global_oce_optim with |
| 10 |
|
|
# MITgcm_offline/mlosch/optim_m1qn3 |
| 11 |
|
|
# Some tweaking is definitely possible and not described here. |
| 12 |
|
|
# |
| 13 |
|
|
# |
| 14 |
|
|
cd MITgcm/verification |
| 15 |
|
|
# this is just to compile and run the model for testing. |
| 16 |
|
|
# I use TAF (just because I have it and OpenAD is a pain to compile on a Mac), |
| 17 |
|
|
# but that should not make any difference |
| 18 |
|
|
./testreport -t tutorial_global_oce_optim -adm -j 4 -ncad |
| 19 |
|
|
# this is the result: |
| 20 |
|
|
# G D M C A F |
| 21 |
|
|
# e p a R o d D |
| 22 |
|
|
# n n k u s G G |
| 23 |
|
|
# 2 d e n t r r |
| 24 |
|
|
|
| 25 |
|
|
# Y Y Y Y 15>16< 7 pass tutorial_global_oce_optim (e=0, w=2) |
| 26 |
|
|
|
| 27 |
|
|
# here is the cost function value that I get |
| 28 |
|
|
# (PID.TID 0000.0001) local fc = 0.620023228182336D+01 |
| 29 |
|
|
# (PID.TID 0000.0001) global fc = 0.620023228182336D+01 |
| 30 |
|
|
# now I download and compile optim_m1qn3 |
| 31 |
|
|
cd ../../ |
| 32 |
|
|
cvs co MITgcm_contrib/mlosch/optim_m1qn3 |
| 33 |
|
|
cd MITgcm_contrib/mlosch/optim_m1qn3 |
| 34 |
|
|
# edit Makefile to adjust to your platform. For me this involves choosing |
| 35 |
|
|
# the correct CPP command and setting SUFF=for (because MacOS is |
| 36 |
|
|
# case-insensitive in my case) |
| 37 |
|
|
make depend |
| 38 |
|
|
make |
| 39 |
|
|
# then I get optim.x |
| 40 |
|
|
cd ../../../MITgcm/verification/tutorial_global_oce_optim/run |
| 41 |
|
|
cp ../../../../MITgcm_contrib/mlosch/optim_m1qn3/optim.x . |
| 42 |
|
|
# - turn off the gradient check (in data.pkg: useGrdchk = .FALSE.) |
| 43 |
|
|
# - tweak the namelist files data.ctrl and data.optim to the compiler needs |
| 44 |
|
|
# (I need a "/" to terminate a namelist) |
| 45 |
|
|
# - I would replace fmin with dfminFrac = 0.1 (expected reduction of 10%) |
| 46 |
|
|
# in data.optim&OPTIM to be independent of the absolute value of the |
| 47 |
|
|
# cost function. When you run ./mitgcmuv_ad with this you need to comment |
| 48 |
|
|
# out dfminFrac, because mitgcmuv_ad does not know about this namelist |
| 49 |
|
|
# parameter (something to be fixed) |
| 50 |
|
|
# - set numiter=100, nfunc=10, or some other large value. nfunc*numiter is |
| 51 |
|
|
# the number of simulations that are allowed in total. This number should be |
| 52 |
|
|
# much larger than numiter, because you may need more than one function call |
| 53 |
|
|
# (= run of mitgcmuv_ad) per iteration, see m1qn3 docs for details |
| 54 |
|
|
# - add an empty namelist &M1QN3 to data.optim |
| 55 |
|
|
# &M1QN3 |
| 56 |
|
|
# / |
| 57 |
|
|
./optim.x > opt0.txt |
| 58 |
|
|
# I get a lot of output in opt0.txt, which is easier to read with |
| 59 |
|
|
# less -S opt0.out (to truncate long lines) |
| 60 |
|
|
# essentially I have the same value for the cost function |
| 61 |
|
|
|
| 62 |
|
|
# ================================================== |
| 63 |
|
|
# Large Scale Optimization with off-line capability. |
| 64 |
|
|
# ================================================== |
| 65 |
|
|
|
| 66 |
|
|
# OPTIM_READPARMS: Control options have been read. |
| 67 |
|
|
# OPTIM_READPARMS: Minimization options have been read. |
| 68 |
|
|
|
| 69 |
|
|
# OPTIM_READDATA: Reading cost function and gradient of cost function |
| 70 |
|
|
# for optimization cycle: 0 |
| 71 |
|
|
|
| 72 |
|
|
# OPTIM_READDATA: opened file ecco_cost_MIT_CE_000.opt0000 |
| 73 |
|
|
# OPTIM_READDATA: nvartype 1 |
| 74 |
|
|
# OPTIM_READDATA: nvarlength 2315 |
| 75 |
|
|
# OPTIM_READDATA: yctrlid MIT_CE_000 |
| 76 |
|
|
# OPTIM_READDATA: filenopt 0 |
| 77 |
|
|
# OPTIM_READDATA: fileff 6.2002322818233591 |
| 78 |
|
|
# OPTIM_READDATA: fileiG 1 |
| 79 |
|
|
# OPTIM_READDATA: filejG 1 |
| 80 |
|
|
# OPTIM_READDATA: filensx 2 |
| 81 |
|
|
# OPTIM_READDATA: filensy 2 |
| 82 |
|
|
# [...] |
| 83 |
|
|
# OPTIM_READDATA: end of optim_readdata |
| 84 |
|
|
|
| 85 |
|
|
|
| 86 |
|
|
# OPTIM_READPARMS: Iteration number = 0 |
| 87 |
|
|
# number of control variables = 2315 |
| 88 |
|
|
# cost function value in ecco_ctrl = 6.2002322818233591 |
| 89 |
|
|
# expected cost function minimum = 5.5802090536410232 |
| 90 |
|
|
# expected cost function decrease = 0.62002322818233591 |
| 91 |
|
|
# Data will be read from the following file: ecco_ctrl_MIT_CE_000.opt0000 |
| 92 |
|
|
|
| 93 |
|
|
# OPTIM_SUB: Calling m1qn3_optim for iteration: 0 |
| 94 |
|
|
# OPTIM_SUB: with nn, REAL_BYTE = 2315 4 |
| 95 |
|
|
# OPTIM_SUB: read model state |
| 96 |
|
|
|
| 97 |
|
|
# OPTIM_READDATA: Reading control vector |
| 98 |
|
|
# for optimization cycle: 0 |
| 99 |
|
|
|
| 100 |
|
|
# OPTIM_READDATA: opened file ecco_ctrl_MIT_CE_000.opt0000 |
| 101 |
|
|
# OPTIM_READDATA: nvartype 1 |
| 102 |
|
|
# OPTIM_READDATA: nvarlength 2315 |
| 103 |
|
|
# OPTIM_READDATA: yctrlid MIT_CE_000 |
| 104 |
|
|
# OPTIM_READDATA: filenopt 0 |
| 105 |
|
|
# OPTIM_READDATA: fileff 6.2002322818233591 |
| 106 |
|
|
# OPTIM_READDATA: fileiG 1 |
| 107 |
|
|
# OPTIM_READDATA: filejG 1 |
| 108 |
|
|
# OPTIM_READDATA: filensx 2 |
| 109 |
|
|
# OPTIM_READDATA: filensy 2 |
| 110 |
|
|
# OPTIM_READDATA: end of optim_readdata |
| 111 |
|
|
|
| 112 |
|
|
|
| 113 |
|
|
# OPTIM_READDATA: Reading cost function and gradient of cost function |
| 114 |
|
|
# for optimization cycle: 0 |
| 115 |
|
|
|
| 116 |
|
|
# OPTIM_READDATA: opened file ecco_cost_MIT_CE_000.opt0000 |
| 117 |
|
|
# OPTIM_READDATA: nvartype 1 |
| 118 |
|
|
# OPTIM_READDATA: nvarlength 2315 |
| 119 |
|
|
# OPTIM_READDATA: yctrlid MIT_CE_000 |
| 120 |
|
|
# OPTIM_READDATA: filenopt 0 |
| 121 |
|
|
# OPTIM_READDATA: fileff 6.2002322818233591 |
| 122 |
|
|
# OPTIM_READDATA: fileiG 1 |
| 123 |
|
|
# OPTIM_READDATA: filejG 1 |
| 124 |
|
|
# OPTIM_READDATA: filensx 2 |
| 125 |
|
|
# OPTIM_READDATA: filensy 2 |
| 126 |
|
|
# OPTIM_READDATA: end of optim_readdata |
| 127 |
|
|
|
| 128 |
|
|
# OPTIM_SUB after reading ecco_ctrl and ecco_cost: |
| 129 |
|
|
# OPTIM_SUB nn = 2315 |
| 130 |
|
|
# OPTIM_SUB objf = 6.2002322818233591 |
| 131 |
|
|
# OPTIM_SUB xx(1) = 0.0000000000000000 |
| 132 |
|
|
# OPTIM_SUB adxx(1) = -6.7879882408306003E-005 |
| 133 |
|
|
# OPTIM_SUB: cold start, optimcycle = 0 |
| 134 |
|
|
# OPTIM_SUB: call m1qn3_offline ........ |
| 135 |
|
|
# OPTIM_SUB: ........................... |
| 136 |
|
|
# OPTIM_SUB: returned from m1qn3_offline |
| 137 |
|
|
# OPTIM_SUB: nn = 2315 |
| 138 |
|
|
# OPTIM_SUB: xx(1) = 0.51934864251896229 0.73000729032300782 |
| 139 |
|
|
# OPTIM_SUB: adxx(1) = -6.7879882408306003E-005 -9.5413379312958568E-005 |
| 140 |
|
|
# OPTIM_SUB: omode = -1 |
| 141 |
|
|
# OPTIM_SUB: niter = 1 |
| 142 |
|
|
# OPTIM_SUB: nsim = 10000 |
| 143 |
|
|
# OPTIM_SUB: reverse = 1 |
| 144 |
|
|
|
| 145 |
|
|
# OPTIM_SUB: mean(xx) = 0.16365068483545642 |
| 146 |
|
|
# OPTIM_SUB: max(xx) = 4.6525355815045790 |
| 147 |
|
|
# OPTIM_SUB: min(xx) = -9.3326211896764324 |
| 148 |
|
|
# OPTIM_SUB: std(xx) = 15.613450260548481 |
| 149 |
|
|
|
| 150 |
|
|
|
| 151 |
|
|
# OPTIM_STORE_M1QN3: saving the state of m1qn3 in OPWARM.opt0001 |
| 152 |
|
|
|
| 153 |
|
|
# OPTIM_SUB: writing 2315 sized control to file ecco_ctrl |
| 154 |
|
|
|
| 155 |
|
|
# OPTIM_WRITEDATA: Writing new control vector to file(s) |
| 156 |
|
|
# for optimization cycle: 1 |
| 157 |
|
|
|
| 158 |
|
|
# OPTIM_WRITEDATA: nvartype 1 |
| 159 |
|
|
# OPTIM_WRITEDATA: nvarlength 2315 |
| 160 |
|
|
# OPTIM_WRITEDATA: yctrlid MIT_CE_000 |
| 161 |
|
|
# OPTIM_WRITEDATA: nopt 1 |
| 162 |
|
|
# OPTIM_WRITEDATA: ff -9999.0000000000000 |
| 163 |
|
|
# OPTIM_WRITEDATA: iG 1 |
| 164 |
|
|
# OPTIM_WRITEDATA: jG 1 |
| 165 |
|
|
# OPTIM_WRITEDATA: nsx 2 |
| 166 |
|
|
# OPTIM_WRITEDATA: nsy 2 |
| 167 |
|
|
# OPTIM_WRITEDATA: end of optim_writedata, icvoffset 2315 |
| 168 |
|
|
|
| 169 |
|
|
|
| 170 |
|
|
# ====================================== |
| 171 |
|
|
# Large Scale Optimization run finished. |
| 172 |
|
|
# ====================================== |
| 173 |
|
|
|
| 174 |
|
|
# ff = -9999 is an intentional dummy value. |
| 175 |
|
|
|
| 176 |
|
|
# now you can organize the rest in a loop. In bash, it could look like this: |
| 177 |
|
|
# first comment out dfminFrac, because mitgcmuv_ad does not know about this, |
| 178 |
|
|
# dfminFrac is really only needed in the zeroth iteration |
| 179 |
|
|
# |
| 180 |
|
|
# tabula rasa |
| 181 |
|
|
rm ecco_c* OPWARM.* m1qn3_output.txt |
| 182 |
|
|
myiter=0 |
| 183 |
|
|
cat > data.optim <<EOF # |
| 184 |
|
|
# ******************************** |
| 185 |
|
|
# Off-line optimization parameters |
| 186 |
|
|
# ******************************** |
| 187 |
|
|
&OPTIM |
| 188 |
|
|
optimcycle=${myiter}, |
| 189 |
|
|
numiter=100, |
| 190 |
|
|
nfunc=100, |
| 191 |
|
|
#dfminFrac = 0.1, |
| 192 |
|
|
iprint=10, |
| 193 |
|
|
nupdate=8, |
| 194 |
|
|
/ |
| 195 |
|
|
|
| 196 |
|
|
&M1QN3 |
| 197 |
|
|
/ |
| 198 |
|
|
EOF |
| 199 |
|
|
|
| 200 |
|
|
while (( $myiter < 20 )) |
| 201 |
|
|
do |
| 202 |
|
|
# formatter iteration count |
| 203 |
|
|
it=`echo $myiter | awk '{printf "%03i",$1}'` |
| 204 |
|
|
echo "iteration ${myiter}" |
| 205 |
|
|
# comment out dfminFrac from data.optim |
| 206 |
|
|
sed -i '' 's/.*dfminFrac.*/#dfminFrac = 0.1,/' data.optim |
| 207 |
|
|
# increment counter in data.optim |
| 208 |
|
|
sed -i .${it} "s/.*optimcycle.*/ optimcycle=${myiter},/" data.optim |
| 209 |
|
|
./mitgcmuv_ad > output${it}.txt |
| 210 |
|
|
# add dfminFrac = 0.1, |
| 211 |
|
|
sed -i '' 's/#dfminFrac.*/ dfminFrac = 0.1,/' data.optim |
| 212 |
|
|
./optim.x > opt${it}.txt |
| 213 |
|
|
# increase counter for next iteration |
| 214 |
|
|
((myiter++)) |
| 215 |
|
|
done |
| 216 |
|
|
|
| 217 |
|
|
# grep "fc " output001.txt |
| 218 |
|
|
# (PID.TID 0000.0001) early fc = 0.000000000000000D+00 |
| 219 |
|
|
# (PID.TID 0000.0001) local fc = 0.132325873958879D+02 |
| 220 |
|
|
# (PID.TID 0000.0001) global fc = 0.132325873958879D+02 |
| 221 |
|
|
|
| 222 |
|
|
# grep "global fc " output???.txt |
| 223 |
|
|
# output000.txt:(PID.TID 0000.0001) global fc = 0.620023228182336D+01 |
| 224 |
|
|
# output001.txt:(PID.TID 0000.0001) global fc = 0.132325873958879D+02 |
| 225 |
|
|
# output002.txt:(PID.TID 0000.0001) global fc = 0.615567811433600D+01 |
| 226 |
|
|
# output003.txt:(PID.TID 0000.0001) global fc = 0.615556878869932D+01 |
| 227 |
|
|
# output004.txt:(PID.TID 0000.0001) global fc = 0.615547009131471D+01 |
| 228 |
|
|
|
| 229 |
|
|
# as you can see, the improvement is not very good after the initial steps. |
| 230 |
|
|
# and the optimization will not be successful in the end, i.e. satisfy |
| 231 |
|
|
# the tolerance set by epsg. |
| 232 |
|
|
# This is likely because the number of timesteps in data is small (20) |
| 233 |
|
|
# for this test. Try longer intergations (e.g. 1 year) and wait longer |
| 234 |
|
|
|
| 235 |
|
|
# check out m1qn3_output.txt, which records the output of m1qn3 |
| 236 |
|
|
# the python script plotfc.py greps the cost function values out of |
| 237 |
|
|
# m1qn3_ouput.txt and plots them and the number of simulations per iteration |
| 238 |
|
|
# which typically increase with decreasing cost function |
| 239 |
|
|
|
| 240 |
|
|
|
Parent Directory
| 
Revision Log
| 
Revision Graph