User Tools

Site Tools



When performing calibrations that generate several simulations as results (e.g. Monte Carlo simulation) a number of best results (num_ens defined in optpar.txt) are saved to bestsims.txt, one row per simulation. The simulation with best objective function value (column CRIT) is first. The file is located in the resultdir folder given in info.txt.

For DEMC calibrations (task DE in optpar.txt), bestsims.txt contains parameter values of the last generation of all populations plus one row (first row in the file) with median values of this last generation. Note: The population parameter sets are the last ones accepted by the DEMC algorithm (column iacc == 1 in the allsim.txt result file). Note 2: The median is here, for npop>3 and even npop, equal to the smaller of the two middle values, i.e. not the midpoint of them.

File content

The first row contains column headings. The first column is the ordinal number, and the second is the value of the objective function on which the simulations are sorted. The closest following columns are a set of performance criteria (se table below and equations). When several criteria are given in info.txt to be used together as the objective function, the columns with performance information will be repeated once per such criteria.

The last columns contain parameter values. Only the parameters calibrated are included in the file. The heading of these columns is the parameter name. Thus for parameters that is not general parameters, several columns with the same name may occur. The number of those columns is the number of soil types, land uses etc. calibrated for that parameter.

The value of a performance criterion will be given if it has been calculated during the simulation. Which criterion that is calculated is determined by the choice of objective function. Criterion that can be deduced from the calculations of the objective function are saved, but no additional ones are calculated. Missing values are indicated with -9999. If the objective function is composed of several criteria comparing the same variables, e.g. both NSE and RE of discharge, there will be still be several (two for the example) sets of performance criteria columns in the file, but they will have the same values since they are comparing the same variables. Note: If several RA criteria have been selected, only the last of them will be printed to file.

The columns of bestsims.txt:

Column Description
NOrow number
CRITvalue of objective function
rr2regional Nash-Sutcliffe efficiency (data from all subbasins combined in one data series)
sr2spatial Nash-Sutcliffe efficiency, calculated using annual means for all subbasins (requires at least 5 years and 5 subbasins with data) to form one data series to calculate the Nash-Sutcliffe efficiency on
mr2average of Nash-Sutcliffe efficiencies for subbasins
rmaeregional mean absolute error (data from all subbasins combined in one data series)
srespatial relative bias (calculated on annual means for all subbasins)
rreregional relative bias (data from all subbasins combined in one data series)
mreaverage of the relative bias for all subbasins (Note: fraction, not %)
rraregional RA, similar to regional NSE, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged with a coefficient value
sraspatial RA, similar to spatial NSE, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged for a coefficient value
mraaverage value of RA for subbasins, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged with a coefficient value
tauaverage of Kendall's Tau value for subbasins
md2median of Nash-Sutcliffe efficiency for subbasins
mdamedian of all subbasins’ RA (Nash-Sutcliffe like criteria where the square is exchanged with a coefficient value)
mrsaverage of error in standard deviation for subbasins
mccPearson correlation coefficient, average of all subbasins with observations
mdkgmedian of Kling-Gupta efficiency for subbasins
akgaverage of Kling-Gupta efficiency for subbasins
asckgaverage of Kling-Gupta Efficiency rescaled to the interval [-1,1]
maraverage of absolute relative bias for subbasins (Note: fraction. not %)
mdnrmedian of normalised RMSE for subbasins
mnwaverage of Nash-Sutcliffe efficiencies adjusted for bias for subbasins
snrspatial root mean square error
smbspatial mean absolute scaled bias on natural log transformed values
numrcnumber of data points included in calculation of regional criteria
nummcnumber of areas (subbasins/outregions) which criteria is included in mean and median criteria calculations
parnameparameter(s) that has been calibrated (one or several columns)

Example of bestsims.txt:

start/hype_file_reference/bestsims.txt.txt · Last modified: 2023/11/16 14:28 (external edit)