Criteria equations
- Code to equation coupling
- Equation definitions

Criteria equations

Performance criteria are used in several files. Different criterion is given in subass.txt and simass.txt files. In addition criteria can be selected in info.txt. Below is listed the code/heading used in each file together with the equation identificator. Further down all the equations are defined.

Code to equation coupling

Equation IDs for subbasin assessment criteria (subassX.txt):

Heading	Description	Equation ID
`NSE`	Nash-Sutcliffe efficiency	NSE
`CC`	Pearson correlation coefficient (Kling-Gupta efficiency, part 1)	CC
`RE(%)`	relative bias in percent	RE%
`RSDE(%)`	relative error in standard deviation in percent	RS%
`Sim`	average of simulated variable	cm
`Rec`	average of observed variable	rm
`SDSim`	standard deviation of simulated variable	cd
`SDRec`	standard deviation of observed variable	rd
`MAE`	mean absolute error	MAE
`RMSE`	root mean square error	RMSE
`Bias`	bias	Bias
`SDE`	Error of standard deviation	ES
`KGE`	Kling-Gupta efficiency	KGE
`KGESD`	Kling-Gupta efficiency, part 2	KGESD
`KGEM`	Kling-Gupta efficiency, part 3	KGEM
`NRMSE`	normalised root mean square error	NE
`NSEW`	Nash-Sutcliffe efficiency adjusted for bias	NSEW
`MinRec`	minimum of observed variable	rmin
`MaxRec`	maximum of observed variable	rmax
`MinSim`	minimum of simulated variable	cmin
`MaxSim`	maximum of simulated variable	cmax

Equation IDs for simulation assessment criteria (simass.txt):

Name	Code	Equation ID
Regional NSE	`RR2`	REGNSE
Regional RA	`RRA`	REGRA
Regional RE	`RRE`	REGRB
Regional MAE	`-`	REGMAE
Average NSE	`MR2`	AVNSE
Average RA	`MRA`	AVRA
Average RE	`MRE`	AVRB
Average RSDE	`MRS`	AVRSB
Average CC	`MCC`	AVCC
Average ARE	`MAR`	AVARB
Average KGE	`AKG`	AVKGE
Aver scalKGE	`ASK`	ASCKGE
Spatial NSE	`SR2`	SPATNSE
Spatial RA	`RRA`	SPATRA
Spatial RE	`-`	SPATRB
Spatial Bias	`SMB`	SPATASB
Spatial RMSE	`SNR`	SPATRMSE
Kendalls Tau	`TAU`	AVTAU
Median NSE	`MD2`	MEDNSE
Median RA	`MDA`	MEDRA
Median KGE	`MKG`	MEDKGE
Median NRMSE	`MNR`	MEDNE
Mean NSEW	`MNW`	AVNSEW

Equation IDs for calibration simulation assessment criteria (bestsims.txt and allsim.txt):

Heading	Description	Equation ID
`rr2`	regional Nash-Sutcliffe efficiency (data from all subbasins combined in one data series)	REGNSE
`sr2`	spatial Nash-Sutcliffe efficiency, calculated using annual means for all subbasins (requires at least 5 years and 5 subbasins with data) to form one data series to calculate the Nash-Sutcliffe efficiency on	SPATNSE
`mr2`	average of Nash-Sutcliffe efficiency for subbasins	AVNSE
`rmae`	regional mean absolute error (data from all subbasins combined in one data series)	REGMAE
`sre`	spatial relative bias (calculated on annual means for all subbasins)	SPATRB
`rre`	regional relative bias (data from all subbasins combined in one data series)	REGRB
`mre`	average of the relative bias for all subbasins (Note: fraction, not %)	AVRB
`rra`	regional RA, similar to regional NSE, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged with a coefficient value	REGRA
`sra`	spatial RA, similar to spatial NSE, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged for a coefficient value	SPATRA
`mra`	average value of RA for subbasins, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged with a coefficient value	AVRA
`tau`	average of Kendall's Tau value for subbasins	AVTAU
`md2`	median of Nash-Sutcliffe efficiency for subbasins	MEDNSE
`mda`	median of all subbasins’ RA (Nash-Sutcliffe like criteria where the square is exchanged with a coefficient value)	MEDRA
`mrs`	average of error in standard deviation for subbasins	AVRSB
`mcc`	Pearson correlation coefficient, average of all subbasins with observations	AVCC
`mdkg`	median of Kling-Gupta efficiency (MKG in info.txt) for subbasins	MEDKGE
`akg`	average of Kling-Gupta efficiency for subbasins	AVKGE
`asckg`	average of Kling-Gupta efficiency rescaled to interval [-1,1] (C2M criteria applied to KGE, Mathevet et al. 2006)	ASCKGE
`mare`	average of absolute relative bias for subbasins (Note: fraction. not %) (MAR in info.txt)	AVARB
`mdnr`	median of normalised RMSE for subbasins	MEDNE
`mnw`	average of Nash-Sutcliffe efficiencies adjusted for bias for subbasins	AVNSEW
`snr`	spatial root mean square error	SPATRMSE
`smb`	spatial mean absolute scaled bias on natural log transformed values	SPATASB

Equation IDs for performance criteria set in info.txt are tabled here.

Equation definitions

Denotations

c	computed value
r	recorded value
cl	log transform of computed value, natural logarithm
rl	log transform of recorded value, natural logarithm
i	index for time steps with observations in a time series of a station
mi	number of values in a time series of a station
j	index of stations
mj	number of stations
ij	index over time steps with observations for all stations
mij	number of time steps with obsevations for all stations
cm	average value of $c_{i}, i=1,mi$ for a station
rm	average value of $r_{i}, i=1,mi$ for a station
cd	standard deviation of $c_{i}, i=1,mi$ for a station
rd	standard deviation of $r_{i}, i=1,mi$ for a station
cmax	maximum value of $c_{i}, i=1,mi$ for a station
rmax	maximum value of $r_{i}, i=1,mi$ for a station
cmin	minimum value of $c_{i}, i=1,mi$ for a station
rmin	minimum value of $r_{i}, i=1,mi$ for a station
w	weight of station

Basic equations

Average value for a time series of a station:

$xm = {1/mi} sum{i=1}{mi}{x_{i}}$ x=r or c

Standard deviation of a time series of a station:

$xd = sqrt{{1/mi} sum{i=1}{mi}{{x_{i}}^2}-xm^2}$ x=r or c

Natural logaritm of value:

xl = LN(x) x=r or c or rm or cm, x>0

Criteria equations for a time series of a station

Nash-Sutcliffe Efficiency (NSE or R2):

$NSE = 1-{sum{i=1}{mi}{(c_{i}-r_{i})^2}}/{sum{i=1}{mi}{(r_{i}-rm)^2}}$

Efficiency with coefficient a (RA):

$RA = 1-{sum{i=1}{mi}{delim{|} {c_{i}-r_{i}} {|}^a}}/{sum{i=1}{mi}{delim{|} {r_{i}-rm} {|}^a}}$

Bias:

$Bias = {sum{i=1}{mi}{(c_{i}-r_{i})}}/mi$

Relative bias (RB or RE):

$RB = {sum{i=1}{mi}{(c_{i}-r_{i})}}/{delim{|}{sum{i=1}{mi}{r_{i}}}{|}}$

Relative bias in percent (RE%):

$RE% = RB*100 = {{sum{i=1}{mi}{(c_{i}-r_{i})}}/{delim{|}{sum{i=1}{mi}{r_{i}}}{|}}}*100$

Error of standard deviation (ES):

ES = {cd-rd}

Relative error of standard deviation (RS):

RS = {{cd-rd}/rd}

Relative error of standard deviation in percent (RS%):

RS% = RS*100 = {{cd-rd}/rd}*100

Mean absolute error (MAE):

$MAE = {sum{i=1}{mi}{delim{|}{c_{i}-r_{i}}{|}}}/mi$

Kling-Gupta efficiency (KGE):

$KGE = 1-sqrt{(CC-1)^2+(cd/rd-1)^2+(cm/rm-1)^2}$ cm>0 and rm>0 and cd>0 and rd>0

Pearson correlation coefficient, Kling-Gupta efficiency part 1 (CC):

$CC = {{1/mi} sum{i=1}{mi}{({r_{i}}*{c_{i}})}-cm*rm}/{cd*rd}$

Kling-Gupta efficiency part 2 (KGESD):

KGESD = cd/rd

Kling-Gupta efficiency part 3 (KGEM):

KGEM = cm/rm

Root mean square error (RMSE):

$RMSE = sqrt{{1/mi sum{i=1}{mi}{({c_{i}}-{r_{i}})^2}}}$

Normalised root mean square error (NE):

$NE = sqrt{{1/mi sum{i=1}{mi}{({c_{i}}-{r_{i}})^2}}}/{max{(r_{i}})}$

Kendalls rank correlation coefficient, tau-b, with adjustments for ties (TAU):

$TAU = {n_{c}-n_{d}}/{sqrt{(n_{0}-n_{1})(n_{0}-n_{2})}}$

Nash-Sutcliffe Efficiency adjusted for bias (NSEW). Introduced in Lindström (2016):

NSEW = NSE+Bias^2/rd^2

where

$n_{c}$ = number of concordant pairs ( $c_{i}<c_{k} and r_{i}<r_{k} or c_{i}>c_{k} and r_{i}>r_{k}, i=1,mi k=1,mi$ )

$n_{d}$ = number of discordant pairs ( $c_{i}<c_{k} and r_{i}>r_{k} or c_{i}>c_{k} and r_{i}<r_{k}, i=1,mi k=1,mi$ )

$n_{0}$ = number of compared pairs

$n_{1}$ = number of compared pairs that ties in the computed values

$n_{2}$ = number of compared pairs that ties in the recorded values

Scaled bias (ScBias):

$ScBias = {sum{i=1}{mi}{delim{|}{{(c_{i}-r_{i})}/{(c_{i}+r_{i})}}{|}}}/mi$

Scaled KGE (SCKGE):

SCKGE = KGE/{2-KGE}

Criteria equations for a model domain (several stations)

Average Nash-Sutcliffe efficiency (AVNSE):

AVNSE arithmetric mean

$AVNSE = {1/mj sum{j=1}{mj}{NSE_{j}}}$

or AVNSE weighted average

$AVNSE = {sum{j=1}{mj}{w_{j}*NSE_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Median Nash-Sutcliffe efficiency (MEDNSE):

$MEDNSE = median delim{lbrace}{{NSE_{j}},{j=1..mj}}{rbrace}$

Spatial Nash-Sutcliffe efficiency (SPATNSE):

$SPATNSE = 1-{sum{j=1}{mj}{(cm_{j}-rm_{j})^2}}/{sum{j=1}{mj}{(rm_{j}-{1/mj} sum{j=1}{mj}{rm_{j}})^2}}$

Regional Nash-Sutcliffe efficiency (REGNSE):

$REGNSE = 1-{sum{ij=1}{mij}{(c_{ij}-r_{ij})^2}}/{sum{ij=1}{mij}{(r_{ij}-{1/mij} sum{ij=1}{mij}{r_{ij}})^2}}$

Average Nash-Sutcliffe efficiency adjusted for bias (AVNSEW):

AVNSEW arithmetric mean

$AVNSEW = {1/mj sum{j=1}{mj}{NSEW_{j}}}$

or AVNSEW weighted average

$AVNSEW = {sum{j=1}{mj}{w_{j}*NSEW_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Average efficiency with coefficient a (AVRA):

AVRA arithmetric mean

$AVRA = {1/mj sum{j=1}{mj}{RA_{j}}}$

or AVRA weighted average

$AVRA = {sum{j=1}{mj}{w_{j}*RA_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Median efficiency with coefficient a (MEDRA):

$MEDRA = median delim{lbrace}{{RA_{j}},{j=1..mj}}{rbrace}$

Spatial efficiency with coefficient a (SPATRA):

$SPATRA = 1-{sum{j=1}{mj}{delim{|}{cm_{j}-rm_{j}}{|}^a}}/{sum{j=1}{mj}{delim{|}{rm_{j}-{1/mj} sum{j=1}{mj}{rm_{j}}}{|}^a}}$

Regional efficiency with coefficient a (REGRA):

$REGRA = 1-{sum{ij=1}{mij}{delim{|}{c_{ij}-r_{ij}}{|}^a}}/{sum{ij=1}{mij}{delim{|}{r_{ij}-{1/mij} sum{ij=1}{mij}{r_{ij}}}{|}^a}}$

Average relative bias (AVRB):

AVRB arithmetric mean

$AVRB = {1/mj sum{j=1}{mj}{RB_{j}}}$

or AVRB weighted average

$AVRB = {sum{j=1}{mj}{w_{j}*RB_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Regional relative bias (REGRB):

$REGRB = {sum{ij=1}{mij}{(c_{ij}-r_{ij})}}/{delim{|}{sum{ij=1}{mij}{r_{ij}}}{|}}$

Spatial relative bias (SPATRB):

$SPATRB = {sum{j=1}{mj}{(cm_{j}-rm_{j})}}/{delim{|}{sum{j=1}{mj}{rm_{j}}}{|}}$

Average Kling-Gupta efficiency (AVKGE):

AVKGE arithmetric mean

$AVKGE = {1/mj sum{j=1}{mj}{KGE_{j}}}$

or AVKGE weighted average

$AVKGE = {sum{j=1}{mj}{w_{j}*KGE_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Median Kling-Gupta efficiency (MEDKGE):

$MEDKGE = median delim{lbrace}{{KGE_{j}},{j=1..mj}}{rbrace}$

Average scaled Kling-Gupta efficiency (ASCKGE):

ASCKGE arithmetric mean

$ASCKGE = {1/mj sum{j=1}{mj}{SCKGE_{j}}}$

or ASCKGE weighted average

$ASCKGE = {sum{j=1}{mj}{w_{j}*SCKGE_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Spatial root mean square error (SPATRMSE):

$SPATRMSE = sqrt{{1/mj sum{j=1}{mj}{({cm_{j}}-{rm_{j}})^2}}}$

Median of Normalised root mean square error (MEDNE):

$MEDNE = median delim{lbrace}{{NE_{j}},{j=1..mj}}{rbrace}$

Average of absolute relative bias (AVARB):

AVARB arithmetric mean

$AVARB = {1/mj sum{j=1}{mj}{delim{|}{RB_{j}}{|}}}$

or AVARB weighted average

$AVARB = {sum{j=1}{mj}{w_{j}*{delim{|}{RB_{j}}{|}}}}/{sum{j=1}{mj}{w_{j}}}$

Average Pearson correlation coefficient (AVCC):

AVCC arithmetric mean

$AVCC = {1/mj sum{j=1}{mj}{CC_{j}}}$

or AVCC weighted average

$AVCC = {sum{j=1}{mj}{w_{j}*CC_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Average relative error of standard deviation (AVRSB):

AVRSB arithmetric mean

$AVRSB = {1/mj sum{j=1}{mj}{RS_{j}}}$

or AVRSB weighted average

$AVRSB = {sum{j=1}{mj}{w_{j}*RS_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Average Kendalls rank correlation coefficient (AVTAU):

AVTAU arithmetric mean

$AVTAU = {1/mj sum{j=1}{mj}{TAU_{j}}}$

or AVTAU weighted average

$AVTAU = {sum{j=1}{mj}{w_{j}*TAU_{j}}}/{sum{j=1}{mj}{w_{j}}}$

Regional mean absolute error (REGMAE):

$REGMAE = {sum{ij=1}{mij}{delim{|}{c_{ij}-r_{ij}}{|}}}/mij$

Spatial mean absolute scaled bias on log transformed values (SPATASB):

$SPATASB = {sum{j=1}{mj}{delim{|}{{cml_{j}-rml_{j}}/{cml_{j}+rml_{j}}}{|}}}/{mj}$

References

Lindström, G., 2016. Lake water levels for calibration of the S-HYPE model. Hydrology Research 47.4:672-682. doi: 10.2166/nh.2016.019.

Mathevet et al. 2006. A bounded version of the Nash-Sutcliffe criterion for better model assessment on large sets of basins. In: Large Sample Basin Experiments for Hydrological Model Parameterization: Results of the Model Parameter Experiment–MOPEX. IAHS Publ. 307, 2006, p. 211-219.

Table of Contents