Parent Directory
| 
Revision Log
| 
Revision Graph
| Links to HEAD: | (view) (download) (annotate) |
| Links to checkpoint58b_post: | (view) (download) (annotate) |
| Sticky Tag: |
Updated code to provide timing information per timestep to include cases where there is no pressure solve by moving the code to the main loop. Modified the timing in the pressure solver for the case where TAF needs to be fooled. Added support for PAPI summary MFlop/s information per timestep along side the simple (user/system/wallclock) timing. To use PAPI one currently needs to have declared PAPIINC and PAPILIB in the optfile for the include and library linking stings.
First step for a NLFS adjoint o initially suppress rStar (new flag DISABLE_RSTAR_CODE) o new init. routines for calc_r_star, calc_surf_dr o still need to deal with ini_masks_etc o testreport seemed happy
remove the include CG3D.h (not needed)
remove unused variables (reduces number of compiler warning)
Implement 4-level checkpointing. Needed for high-res. runs with large global files to be able to limit tapelev size (otherwise a potential show-stopper).
New package for controlling termination of the model based on the wall clock:
This is handy for those of us who keep losing 5000 cpuhours because the model
took 5 minutes longer than it did the day before.
To use:
In package.conf, add runclock
In data.pkg, add useRunClock=.TRUE.
Add data.runclock to read
&RUNCLOCK
RC_maxtime_hr=2,
RC_maxtime_mi=30,
RC_maxtime_sc=0,
&
The package also write the delta Wall Clock between each model step.
Who can explain a 30% slow down after 60 iterations? Who even knew about it?
Now you can find out lots of dark secrets about the EM EYE TEE GEE SEE EM...
use baseTime as time origin ; DIFF_BASE_MULTIPLE replaces DIFFERENT_MULTIPLE
Adding calls to AUTODIFF_INADMODE_SET/UNSET
Add hooks for inAdExact.
include GMREDI_OPTIONS.h and SEAICE_OPTIONS.h
o Fixing directives for non-ECCO forward_step, after ECCO directives have been fixed to avoid recomp. of do_oceanic_phys o Works well for global_ocean.90x40x15, but recomp. issue remains for carbon
switch btw. 2-level and 3-level checkpointing
extend simple cost functions to treat time mean quantities o cost_tile called from forward_step o cost_accumulate_state called from cost_tile
o bringing adjoint up to date for sheduled c55
o adding offline package
max number of passive tracers is now defined in PTRACERS_SIZE.h
o Adjoint for EBM package (needs further testing) o bug fix for parallel EBM in ebm_zonalmean.F
o couple of small AD fixes
Really weird thing! call do_the_model_io from the_main_loop with parameters starttime, niter0 instead of mytime, myiter screws up the adjoint: TAF places a faulty re-call of initialise_varia after cost call which shouldn't be there!
For some unknown reason calls to MONITOR and DO_THE_MODEL_IO now change the ADTHE_MAIN_LOOP parameter list. Weird! This fixes it.
call DO_THE_MODEL_IO (replaces "call WRITE_STATE") just before starting
o sort out interplay between tamc.h and PTRACERS_OPTIONS.h (need info on NUMBER_OF_PTRACERS) o missing PTRACERS headers in some routines o default tamc.h in pkg/autodiff supposed to crash to ensure proper customization
o merging from ecco-branch o minor CPP options update
o cleanup: convert '#ifndef DISABLE_DEBUGMODE"' to '#ifdef ALLOW_DEBUG"'
o convert all the '#ifndef "EXCLUDE_MONITOR"'-isms to the now-standard
'#ifdef "ALLOW_MONITOR"' format as is done with all the other packages
- note that this also means removal of the monitor package from the
packages.conf list for the AD examples
o convert all "INCLUDE_EXTERNAL_FORCING_PACKAGE" defines to the more consistent ALLOW_EXF o passed all the basic verification tests on shelley
modified pkg/cd_code o moved cd_scheme.F -> cd_code_scheme.F o separate read_checkpoint from cd_code_ini_vars.F o separated cd_code part from write_checkpoint o updated AD_SOURCE, generated .flow o added CD_CODE_VARS.h to the_main_loop
o first check-in for the "branch-genmake2" merge
o verification suite as run on shelley (gcc 3.2.2):
Wed Oct 8 23:42:29 EDT 2003
T S U V
G D M c m s m s m s m s
E p a R g m m e . m m e . m m e . m m e .
N n k u 2 i a a d i a a d i a a d i a a d
2 d e n d n x n . n x n . n x n . n x n .
OPTFILE=NONE
Y Y Y Y 13 16 16 16 0 16 16 16 16 16 16 16 16 13 12 0 0 pass adjustment.128x64x1
Y Y Y Y 16 16 16 16 0 16 16 16 16 16 16 0 0 16 16 0 0 pass adjustment.cs-32x32x1
Y Y Y Y 16 16 16 16 0 16 16 16 16 16 16 22 0 16 16 22 0 pass adjust_nlfs.cs-32x32x1
Y Y Y Y -- 13 13 16 16 13 13 13 13 16 16 16 16 16 16 16 16 N/O advect_cs
Y Y Y Y -- 22 16 16 16 16 16 16 13 16 16 16 16 16 16 16 16 N/O advect_xy
Y Y Y Y -- 13 16 13 16 16 16 16 16 16 16 22 16 16 16 16 16 N/O advect_xz
Y Y Y Y 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 pass aim.5l_cs
Y Y Y Y 14 16 16 16 16 16 16 16 16 13 16 16 16 16 16 13 16 pass aim.5l_Equatorial_Channel
Y Y Y Y 16 16 16 16 16 16 16 16 16 16 16 13 16 16 13 13 16 pass aim.5l_LatLon
Y Y Y Y 13 16 16 16 16 16 16 16 16 16 13 12 13 13 16 13 16 pass exp0
Y Y Y Y 14 16 16 16 16 16 16 16 22 16 16 16 13 16 16 22 16 pass exp1
Y Y Y Y 13 13 16 13 16 16 16 16 16 13 13 16 16 13 13 13 13 pass exp2
Y Y Y Y 16 16 16 16 16 16 16 16 22 16 16 16 16 16 16 16 16 pass exp4
Y Y Y Y 16 16 16 16 16 16 16 16 16 16 16 22 16 16 16 22 16 pass exp5
Y Y Y Y 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 pass front_relax
Y Y Y Y 14 16 16 13 13 16 16 13 13 16 13 13 16 12 13 13 16 pass global_ocean.90x40x15
Y Y Y Y 10 16 16 13 13 16 13 16 16 13 13 13 13 16 16 13 16 FAIL global_ocean.cs32x15
Y Y Y Y 6 11 12 13 13 12 13 16 13 9 9 9 9 10 9 9 11 FAIL global_ocean_pressure
Y Y Y Y 14 16 16 13 16 16 16 13 13 13 13 13 16 12 16 13 16 pass global_with_exf
Y Y Y Y 14 16 16 16 16 16 16 16 16 11 13 22 13 16 16 9 16 pass hs94.128x64x5
Y Y Y Y 13 16 16 16 16 16 16 16 16 11 16 16 16 13 16 22 13 pass hs94.1x64x5
Y Y Y Y 14 16 16 16 16 16 16 16 16 13 16 13 13 16 16 22 13 pass hs94.cs-32x32x5
Y Y Y Y 10 10 16 13 13 16 16 16 22 16 13 13 13 13 13 22 13 FAIL ideal_2D_oce
Y Y Y Y 8 16 16 16 16 16 16 16 16 13 13 8 16 16 16 16 16 FAIL internal_wave
Y Y Y Y 14 16 16 16 16 16 16 16 16 13 13 22 13 13 13 22 16 pass inverted_barometer
Y Y Y Y 12 16 16 16 16 16 16 16 16 16 13 12 13 13 13 13 13 FAIL lab_sea
Y Y Y Y 11 16 16 16 16 16 16 16 13 13 13 12 13 16 13 12 13 FAIL natl_box
Y Y Y Y 16 16 16 16 16 16 16 16 22 16 16 16 16 16 16 16 16 pass plume_on_slope
Y Y Y Y 13 16 16 16 16 13 16 16 16 16 16 16 16 13 16 16 16 pass solid-body.cs-32x32x1
o Mods and bug fixes to pkg/cal, pkg/exf, etc., needed for computation of tracer Green's fucntions for ocean inversion project.
o disentangled ALLOW_PTRACERS using new ALLOW_GCHEM
Extended pkg/debug and instrumented main code to help track down fatal errors.
cleanup.
Modif.s for correct DIVA handling:
Modif.s in
- ecco_the_main_loop.F
- ecco_the_model_main.F
and added cost_final_restore.F cost_final_store.F
o With these changes, it is possible to fully separate
the first forward run to compute the cost function (mdthe_main_loop)
and adjoint runs over individual DIVA intervals (adthe_main_loop)
into separate model executions.
This enables better tuning of DIVA to given queue sizes.
o Loss of cost function value in consecutive DIVA runs is
fixed by storing to 'costfinal' file.
o Appropriate call of ctrl_unpack/_pack,grdchk_main at
beginning/end of adjoint integration is accounted for.
The current changes apply to TAF-1.4.26.
TAF has one remaining bug, requiring one manual change
(see MITgcm manual).
Merging from release1_p12: o Modifications for using pkg/exf with pkg/seaice - improved description of the various forcing configurations - added basic radiation bulk formulae to pkg/exf - units/sign fix for evap computation in exf_getffields.F - updated verification/global_with_exf/results/output.txt o Added pkg/sbo for computing IERS Special Bureau for the Oceans (SBO) core products, including oceanic mass, center-of-mass, angular, and bottom pressure (see pkg/sbo/README.sbo). o Lower bound for viscosity/diffusivity in pkg/kpp/kpp_routines.F to avoid negative values in shallow regions. - updated verification/natl_box/results/output.txt - updated verification/lab_sea/results/output.txt o MPI gather, scatter: eesupp/src/gather_2d.F and scatter_2d.F o Added useSingleCpuIO option (see PARAMS.h). o Updated useSingleCpuIO option in mdsio_writefield.F to work with multi-field files, e.g., for single-file pickup. o pkg/seaice: - bug fix in growth.F: QNET for no shortwave case - added HeffFile for specifying initial sea-ice thickness - changed SEAICE_EXTERNAL_FLUXES wind stress implementation o Added missing /* */ to CPP comments in pkg/seaice, pkg/exf, kpp_transport_t.F, forward_step.F, and the_main_loop.F o pkg/seaice: - adjoint-friendly modifications - added a SEAICE_WRITE_PICKUP at end of the_model_main.F
Modifications in treatment of initial flux forcing controls to avoid extra storing of flux fields.
o few modif.'s to get latest version adjointed (mainly kick out code in ini_linear_phisurf) o modif's to run adjoint with exactConserv o case GM_BOLUS_ADVEC should be cleaned S/R gmredi_slope_psi should be cleaned o verification/carbon now has exactConserv=.TRUE.
checkpoint47f_post
Merging from release1_p10:
o modifications for using pkg/exf with pkg/seaice
- pkg/seaice CPP options SEAICE_EXTERNAL_FORCING
and SEAICE_EXTERNAL_FLUXES
- pkg/exf CPP options EXF_READ_EVAP and
EXF_NO_BULK_COMPUTATIONS
- usage examples are Experiments 8 and 9 in
verification/lab_sea/README
- verification/lab_sea default experiment now uses
pkg/gmredi, pkg/kpp, pkg/seaice, and pkg/exf
Two packages: bulk_force (Bulk forcing) and therm_seaice (thermodynamic_seaice) - adopted from LANL CICE.v2.0.2 Earlier integration from Stephaine Dutkiewicz and Patrick Heimbach. Two ifdef statements for compile time, ALLOW_THERM_SEAICE and ALLOW_BULK_FORCE Two switches in data.pkg to turn on at run-time: cat data.pkg # Packages &PACKAGES useBulkForce=.TRUE., useThermSeaIce=.TRUE., & WARNING: useSEAICE and useThermSEAICE are mutually exclusive. The bulk package requires an additional parameter file with two namelists, data.ice and data.blk. c ADAPTED FROM: c LANL CICE.v2.0.2 c----------------------------------------------------------------------- c.. thermodynamics (vertical physics) based on M. Winton 3-layer model c.. See Bitz, C. M. and W. H. Lipscomb, 1999: "An energy-conserving c.. thermodynamic sea ice model for climate study." J. Geophys. c.. Res., 104, 15669 - 15677. c.. Winton, M., 1999: "A reformulated three-layer sea ice model." c.. Submitted to J. Atmos. Ocean. Technol. c.. authors Elizabeth C. Hunke and William Lipscomb c.. Fluid Dynamics Group, Los Alamos National Laboratory c-----------------------------------------------------------------------
differentiable version of checkpoint46n_post o external_fields_load now part of differentiation list o pressure needs multiple storing; would be nice to have store_pressure at beginning or end of forward_step, e.g. by having phiHyd global (5-dim.) (NB: pressure is needed for certain cases in find_rho, which is also invoked through convective_adjustment). o recomputations in find_rho for cases 'JMD95'/'UNESCO' or 'MDJWF' are OK. o #define ATMOSPHERIC_LOADING should be differentiable o ini_forcing shifted to begining of initialise_varia
Merging from release1_p8 branch:
o GAD:
- generated new common blocks to account for call of
same gad routines with differing traceridentities
(needed to modify tracerIdentity indices in GAD.h)
- generated separate common blocks for case useCubedSphereExchange
(Department of Futurology)
- parameter lists to gmredi_?transport: added tracerIdentity
- added new key indices to tamc.h
Moved call to packages_init_varia before call convective_adjustment_ini. Corresponding adjustment of tape init. for TAF.
o merge of relevant stuff from the ecco-branch:
- genmake: removed $S64 overwrite for case SunOS
- pkg/exf: update and corrections for field swapping and obcs
- pkg/ecco: parameter lists for the_model_main, the_main_loop
harmonized between ECCO and MITgcm
- pkg/autodiff: added flow directives for obcs, mdsio_gl_slice
updated checkpointing_lev... lists for obcs
- model/src: minor changes in forward_step, plot_field
added directive for divided adjoint in the_main_loop
- pkg/mdsio: added mdsio_gl_slice
* Updated adjoint/makefile
o removed "_ecco" from names
o added/modified forward mode options
o added dependent vector option
o tried SVD matrix computation (doesn't work;
too much memory use by TAF)
* Updated flow directives related to FTL
* removed global_sum__g.F (not necessary)
* initialise forward mode anomalies in ctrl_unpack.F
(so far "crude" initialization only).
* Added flag ALLOW_TANGENTLINEAR_RUN complememting
ALLOW_ADJOINT_RUN
* driver routine the_model_main.F needs separate call
to FTL routine G_THE_MAIN_LOOP with parameter list
identical to THE_MAIN_LOOP (TAF doesn't remove arguments).
* added template file tangentlinear_model.F
for forward mode.
Transfering major checkpoint store directive blocks to separate header files. Files are included in the_main_loop (lev3, lev2) and forward_step (lev1).
#ifdef ALLOW_COST_TRACER replaced by ALLOW_PASSIVE_TRACER (required)
Preparing adjoint of Held-Suarez: - bugfix for storing in absence of CD code - adding EXACT_CONSERV to AD list - new routine ini_autodiff to add TAMC-specific initialisations - adding Shapiro filter to AD list
Switched sense of CPP macro for controlling multi-dimensional advection: o DISABLE_MULTIDIM_ADVECTION is set in GAD_OPTIONS.h o automatically set if differentiating code (comments around call to gad_advection point out how to re-enable it) o this avoids needing to add the former macro to CPP_OPTIONS.h - reason for this is there's no point in any of the new schemes without it.
Fixed AD-related problems: o Store directives up-to-date with re-arranged Adams-Bashforth (mainly thermodynamics.F) o New store directives for multi-dim. advection schemes * new CPP flag ALLOW_MULTI_DIM_ADVECTION * new common block and key passkey (mainly gad_advection.F) o Modified store directives for split of dynamics/thermodynamics for the case ALLOW_KPP o Cleaned argument list for timestep_tracer.F
Deleted single apostrophy (').
Bringing comments up to data and formatting for document extraction.
o New package pkg/rw/ needed new flow directives for TAMC. o Replacement of calc_mom_rhs by mom_fluxform and mom_vecinv needed update of adjoint/makefile and additional store directives in the_main_loop.F to cover all cases.
Moved TR1.h headers between ifdef ALLOW_PASSIVE_TRACER.
Cleanup of store directives for TAF.
Added a few re-initialisations for TAF.
o Extracted core part of the_main_loop and re-created forward_step
N.B.: Time-dependent part of cost function should remain in
the_main_loop (or contributions must be stored)
o Added some parameter recomputations of nIter0
o Added grdchk package handling o Added passive tracer handling
the default is now to call the filter after solve_for_pressure
Corrected all the timer labels (for the non-adjoint sections).
Added Kinetic energy monitoring.
Adding skeleton "monitor" routines. The default is to do it every time-step. monitorFreq=0 disables.
Modifications/fixes to support TAMC differentiability (mostly missing or wrong directives).
See doc/tag-index and doc/notes_c37_adj.txt Preparation for stand-alone autodifferentiability.
remove "include CG2D.h"
move the IO at the end of a time step
set initial Time & Iteration counter in the_model_main and pass them as argument to the_main_loop
Made sure each .F and .h file had the CVS keywords Header and Name at its start. Most had header but very few currently have Name, so lots of changes!
Restructered the calling tree between THE_MODEL_MAIN()
and DYNAMICS(). Two calling levels have been inserted
to better split the "fixed" initialization phase from
the main time-loop for easier connectivity with the adjoint
infrastructure.
The calling tree now looks like:
the_model_main:
initialise_fixed()
the_main_loop()
the_main_loop:
initialise_varia()
do i=1,nIters
forward_step()
enddo
forward_step()
forward_step:
load_external_fields()
dynamics()
do_the_model_io()
solve_for_pressure()
do_gterm_blocking_exchanges()
write_checkpoint()
This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, select a symbolic revision name using the selection box, or choose 'Use Text Field' and enter a numeric revision.
| ViewVC Help | |
| Powered by ViewVC 1.1.22 |