/[MITgcm]/MITgcm/verification/testreport |
Parent Directory | Revision Log | Revision Graph
Links to HEAD: | (view) (download) (annotate) |
Links to checkpoint62i: | (view) (download) (annotate) |
Sticky Tag: |
add "-makedepend" (short: "-md") option to pass argument to "genmake2 -makedepend"
AD-testing: no longer link files from dir "../input" (link 1rst files from "../input_ad.[xx]" if secondary test "xx" and then from "../input_ad")
switch few printing between std-out (1) and std-err (2) (more consistent).
fix previous modif (using relative path for CDIR does not work)
- use relative path for CDIR (like all other dir) - add a small "genmake_state" file in $DRESULTS dir.
-add option '-runonly' (short: '-ro') to just do the run part (without make) -more standard way to specify some options
add option (-use_r4 or -ur4) to switch "_RS" type to "real*4"
small modif in case no executable was build
add option "-norun" (in short: -nr) to skip the runmodel step
a hack for weird situation (finish the run but with error code)
remove Executable if "make" command return with an error
compatible with shorter monitor prefix for surface forcing (Qnet)
keep tar file if something went wrong when sending email
option "-clean" removes also executable (as it use to be)
little changes in "-clean" option
typo in print message
an other hack to link *.mpi files from input dir, when using -mpi option (similar to eedata.mth with -mth option)
use single "-" for genmake2 options mods & optfile (like all the other options)
mpi+mth: skip tests were number of tiles do not match
for now, uses OpenMP for multi-threaded test
make default ouput dir name from "short" hostname
modified for OpenAD verification - new flag -oad, new variable OADM, modified test conditions that use ADM, use 'eval' to run the command string for genmake or else the single quotes for the multiple mods directories are lost and genmake doesn't parse the mods argument correctly.
play with "run.log" file
cosmetic change to the CMDLINE print
- remove "check_for_add_mon_output" script. - fix the search for ptracer in secondary output file
fix syntax of awk so that testreport works smoothly even in the sun
1 minor change
minor change
add new argument "-mpackdir" (short="-mpd") for where to find/build the mpack utility
minor changes: print which C compiler is used to build mpack and to compile the comparison code.
add a -print to a find, as some find implementations require an action (on SX8!!!); let's hope that this does not break testreport on other systems (it doesn't on "mine") this is the change: < MPI_FILES=`(cd $CODE_DIR; find . -name "*_mpi")` --- > MPI_FILES=`(cd $CODE_DIR; find . -name "*_mpi" -print)`
add to summary a print of command line and system info
add a "-gsl" option to compile with "-gsl" flag.
allows to run several Adjoint tests (with input files from input_ad.[xx]) with the same executable and compare output to results/output_adm.[xx].txt
run_clean also remove AD output files
clean-up the -clean option
add 1rst line (with TAF version) of taf_ad.log in local summary.txt
forgot to remove this testing of results_ad/output_adm.txt
AD testing : check Gradient (instead of Cost) for pass/FAIL diagnostics + clean-up (remove un-used parts)
minor change
add option to test monitor output of Qnet, SeaIce Area & Heff (min,max,av,StD) and thSIce mean Area & Thick (Glob,N-H,S-H).
use same scripts for forward & adm testing (testoutput_ad no longer used) and check AD output against results/output_adm.txt
towards using same scripts for both forward & adm testing
- change format of summary : indicate between "> <" the "main" variable which decide the "pass" or "FAIL" (now OK with new "make_summary") - test if output exist before testing if it's older than executable (for hp_ux systems)
fix for SunOS : use "sed" instead of awk gsub/sub ; fix for hp-ux_ia64 : do not pine printf results to sed command
add bracket in comparison C-code tr_cmpnum.c (was producing warning ; reported by Constantinos)
- little fix (header of tr_out.txt) - disable the "printing improvement" (main var between ><) until someone fix the building of testing web-pages
Allow to choose (per experiment) the list of tested output variables and the main variable (used to issue a pass/FAIL ) ; for now, done by adding the specific list in file: [exp]/input/tr_checklist ; default is unchange: >cg2d< , (T,S,U,V,5 tracers)_[min,max,mean,StD] ; Also modify printing of summary: a) ">" NN "<" indicates the "main" variable (pass/FAIL) b) echo the list of tested variables in the individual summary.txt of each experiment.
cosmetic changes: rename some functions to make it easier to grep for
add argument "MATCH_CRIT" (option: -match $Number) = number of matching digits to get a "pass"
change comparison code to handle NAN when used for AD testing.
Added support for timing and performance per timestep reporting
add option: -skd|-skipdir to skip some test (usefull since full list is getting long); remove "show_help" (not used, not up-to-date, was copied from testscript ?).
check for "NORMAL END" in run.log
outputfile is STDOUT.0000 in MPI mode (option -mpi) ; "cp STDOUT.0000 output.txt" no longer needed in run-script command
update the "basic" group of experiment to test
change multi-test output file name (from output.txt.{name} to output.{name}.txt)
-t option: expand groups of exp. (recognized groups: basic, tutorials)
modified version of testreport: - always build in dir. build (if not there, mkdir build instead of building in input) - run in dir run (if it exists), in build otherwise. - call prepare_run (if input/prepare_run exists) - run multi-threaded test (with new option "-mth") if file input/eedata.mth exists. - added some debug print
litle modif
change scandirs: test if /results exist (or /results_ad if ad mode) (instead of /input previously)
ad testing: start to link files from ../input_ad and then from ../input (and not the reverse)
o make staf default, rather than taf o in testreport remover hard-wired adof
change "output.txt_adm" to "output_adm.txt"
add time report (from MITgcm) to each summary.txt if running in ad mode
fix adm testing (broken with previous modifs)
change the default COMMAND from "make output.txt" to "./mitgcmuv > output.txt" this improves multi-tests since Makefile is no longer needed and therefore no need for soft link of source+object files from ../build dir. this improves also MPI test (don't run again if output.txt is up-to-date)
fix "argument too long" problem reported by Martin & Jean-Michel: http://mitgcm.org/pipermail/mitgcm-devel/2006-January/001848.html
fix for experiment with multiple tests
o switch from egrep to the more portable grep
add a test for NAN & INF
add "-odir STRING" option to replace 'hostname' in output dir name.
no need to keep the tr_{hostname} dir when using testreport -clean
o add a "-postclean" or "-pc" option to testreport for machines that cannot handle the >1.5GB required to run a full set of tests
check for same number of lines in the output files
o make sure the "-mpi" flag gets passed on to genmake2
get shorter lines in tr_out.txt but keep summary.txt unchanged
o lrint() --> rint() which is more portable [thanks to Martin!]
title fix (do ln -s of just *.[fFhoc] instead of *.[fFhoc]* )
o change the way the extra-runs are handled (per JMC requests)
o make testreport do a better job with the "extra run" directories - no longer re-runs the model for every extra run every time - fixes problem introduced with last check-in
o quick one-line fix for 'ln -s' on Sun systems
o fix CC and MAKE defaults
o use '$CC' and report error if the 'cc' default does not work
try to fix tmp_cmpnum.c (return 22 only when all numbers are zero)
o remove unnecessary and confusing lines
o fix MPI error in testreport
o fix "16" vs. "22" - Martin, I hope this does it -- it seems to work correctly for me
o various fixes to the CVS repository due to the disk crash on 20041206
o fix the tmp_cmpnum program embedded in testreport - Martin found an error in the "if" logic and we (ML, JMC, EH3) agreed on a better way of doing things - "22" means comparison of "two zeros"
o testreport previously did a lousy job of comparing small numbers, so we (Ed and Martin) changed the way the cut-off of 1.e-12 is implemented in the little c-progam tmp_cmpnum.c which is embedded in testreport.
o new option added that lets you test a set of PTRACERS - example syntax is "-ptracres='2 3'" or "-ptr '1 4'"
o add PTRACERS (PTR0[1-5]) to testreport output
save STDERR.0000 when it's not empty
o add partial ("head -100") Makefile information - keeps the email size manageable - tells us the optfile name and much of the config info
modif runmodel: o add "rm -f run.log" before running (avoid accumulation of stuff when testreport is use many times) o add "tail run.log" to get usefull info on to the screen, such as: make: `output.txt' is up to date (when no need to re-run) and print warning to the screen
o replace readlink with more portable cmp (eg. no readlink on halem!) o use $MAKE instead of "make"
o reduce size of verif emails--do not include Makefile or output.txt
o add testreport "-j JOBS" for parallel builds -- gives impressive speedup on Altix, Opteron, and other SMP machines
multiple test-run done in a different directory
o fix help message
o make "-ieee" the default per JMC request o add "-deldir" option to remove the results directory per JMC request
o new "poster children" for the API reference: - generic_advdiff - mnc
o add "eval" to fix the shell syntax
o another change for JMC to test
o fix global vs. local
o small change for the "multiple test" feature
o version of testreport that now allows for "multiple tests per test" (explanation email to follow)
Hack to catch problem on cg01 with small numbers: "C" scanf doesn't properly translate formatted number below 1E-99 which the model does write out on rare occasions. Perhaps this little codelet should be in fortran?
o typo
o only build mpack if it will be used
o using "==" in the test command is not portable syntax
Boo boo from Boo's husband. Should only skip experiments without SIZE.h_mpi when testing MPI.
o when using "-mpi" option, skip those tests that don't contain SIZE.h_mpi
o AJA's link-don't-copy way of handling _mpi files
o export "CC=cc" so we have a saner default compiler for mpack
o remove most of the testtreport files from cvs attention
o added an "-adm" option that runs an adjoint test instead of the normal forward tests - confirmed to work with global_ocean.90x40x15 in checkpoint52a_post since the current HEAD seems to be broken for the adjoint - used staf for the above test - produces same results for the forward tests
o fix for the SGI O3K: - On the SGI O3K, and *not* the O2K (or *ANY* other system we use for that matter), "cat -n" adds a ":" immediately following each line number. <sarcasm> What a nice touch! Thank you, SGI. </sarcasm>
o replaced "tmp_cmpnum.f" with "tmp_cmpnum.c" for the SGI O3K
o cleanup formatting of output
o some small cleanups o re-work the way the emails ("-a" option) are sent -- should not change the way it works for anyone not using the email option
o changed the shell invocation syntax to "#! /usr/bin/env ${SHELL}" and verified that this works on the following machines: - all available Linux boxes - Mac OS X - OSF1 eg. halem - AIX SP[34] clusters eg. bluesky/blackforest/babyblue @ NCAR - SGI O[23]k clusters eg. turing/lomax/chapman @ nas.nasa.gov - Solaris [89] eg. slough.mit.edu
o fix typo
o fix just-checked-in mistake!
o tiny fix for date handling on Mac OS X
o use 'cat -n' instead of the 'nl' utility to aid portability
o create a convenient "tr.out" summary file
o add start/end time to the output
o fix the separate build directory "ln -s with CVS" problem pointed out earlier today by M. Losch o various small cleanups
o fix a bug JMC discovered: parsing absolute paths in the OPTFILE argument (eg. "-of=/a/path/to/optfile") caused errors
o first check-in for the "branch-genmake2" merge o verification suite as run on shelley (gcc 3.2.2): Wed Oct 8 23:42:29 EDT 2003 T S U V G D M c m s m s m s m s E p a R g m m e . m m e . m m e . m m e . N n k u 2 i a a d i a a d i a a d i a a d 2 d e n d n x n . n x n . n x n . n x n . OPTFILE=NONE Y Y Y Y 13 16 16 16 0 16 16 16 16 16 16 16 16 13 12 0 0 pass adjustment.128x64x1 Y Y Y Y 16 16 16 16 0 16 16 16 16 16 16 0 0 16 16 0 0 pass adjustment.cs-32x32x1 Y Y Y Y 16 16 16 16 0 16 16 16 16 16 16 22 0 16 16 22 0 pass adjust_nlfs.cs-32x32x1 Y Y Y Y -- 13 13 16 16 13 13 13 13 16 16 16 16 16 16 16 16 N/O advect_cs Y Y Y Y -- 22 16 16 16 16 16 16 13 16 16 16 16 16 16 16 16 N/O advect_xy Y Y Y Y -- 13 16 13 16 16 16 16 16 16 16 22 16 16 16 16 16 N/O advect_xz Y Y Y Y 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 pass aim.5l_cs Y Y Y Y 14 16 16 16 16 16 16 16 16 13 16 16 16 16 16 13 16 pass aim.5l_Equatorial_Channel Y Y Y Y 16 16 16 16 16 16 16 16 16 16 16 13 16 16 13 13 16 pass aim.5l_LatLon Y Y Y Y 13 16 16 16 16 16 16 16 16 16 13 12 13 13 16 13 16 pass exp0 Y Y Y Y 14 16 16 16 16 16 16 16 22 16 16 16 13 16 16 22 16 pass exp1 Y Y Y Y 13 13 16 13 16 16 16 16 16 13 13 16 16 13 13 13 13 pass exp2 Y Y Y Y 16 16 16 16 16 16 16 16 22 16 16 16 16 16 16 16 16 pass exp4 Y Y Y Y 16 16 16 16 16 16 16 16 16 16 16 22 16 16 16 22 16 pass exp5 Y Y Y Y 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 pass front_relax Y Y Y Y 14 16 16 13 13 16 16 13 13 16 13 13 16 12 13 13 16 pass global_ocean.90x40x15 Y Y Y Y 10 16 16 13 13 16 13 16 16 13 13 13 13 16 16 13 16 FAIL global_ocean.cs32x15 Y Y Y Y 6 11 12 13 13 12 13 16 13 9 9 9 9 10 9 9 11 FAIL global_ocean_pressure Y Y Y Y 14 16 16 13 16 16 16 13 13 13 13 13 16 12 16 13 16 pass global_with_exf Y Y Y Y 14 16 16 16 16 16 16 16 16 11 13 22 13 16 16 9 16 pass hs94.128x64x5 Y Y Y Y 13 16 16 16 16 16 16 16 16 11 16 16 16 13 16 22 13 pass hs94.1x64x5 Y Y Y Y 14 16 16 16 16 16 16 16 16 13 16 13 13 16 16 22 13 pass hs94.cs-32x32x5 Y Y Y Y 10 10 16 13 13 16 16 16 22 16 13 13 13 13 13 22 13 FAIL ideal_2D_oce Y Y Y Y 8 16 16 16 16 16 16 16 16 13 13 8 16 16 16 16 16 FAIL internal_wave Y Y Y Y 14 16 16 16 16 16 16 16 16 13 13 22 13 13 13 22 16 pass inverted_barometer Y Y Y Y 12 16 16 16 16 16 16 16 16 16 13 12 13 13 13 13 13 FAIL lab_sea Y Y Y Y 11 16 16 16 16 16 16 16 13 13 13 12 13 16 13 12 13 FAIL natl_box Y Y Y Y 16 16 16 16 16 16 16 16 22 16 16 16 16 16 16 16 16 pass plume_on_slope Y Y Y Y 13 16 16 16 16 13 16 16 16 16 16 16 16 13 16 16 16 pass solid-body.cs-32x32x1
fix omission from previous checkin
minor -make / -mpi cleanups
added -mpi option
minor changes for mpi
small changes to get testing working on bluesky.ucar.edu
add -bash option to deal with dreadful sysadmins who insist on installing bash in obscure locations like "/usr/ulocal/bin/bash"
fix directory creation order
simplify reports sent with mpack and convert "summary.txt" to bash syntax
Small cleanups to genmake2 and its "optfiles". Initial check-in of "testreport" which functions similarly to "testscript" but supports more genmake2 options and emails results to a central address for automated testing.
This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, select a symbolic revision name using the selection box, or choose 'Use Text Field' and enter a numeric revision.
ViewVC Help | |
Powered by ViewVC 1.1.22 |