====== Criteria equations ======

Performance criteria are used in several files. Different criterion is given in [[start:HYPE_file_reference:subassX.txt|subass.txt]] and [[start:HYPE_file_reference:simass.txt|simass.txt]] files. In addition criteria can be selected in [[start:hype_file_reference:info.txt#performance_criteria_options|info.txt]]. [[start:hype_file_reference:info.txt:criteria_equations#code_to_equation_coupling|Below]] is listed the code/heading used in each file together with the equation identificator.  [[start:hype_file_reference:info.txt:criteria_equations#equation_definitions|Further down]] all the equations are defined.



===== Code to equation coupling =====

Equation IDs for subbasin assessment criteria ([[start:HYPE_file_reference:subassX.txt|subassX.txt]]):

<sortable>
^ Heading ^ Description ^ Equation ID ^
|''NSE''|Nash-Sutcliffe efficiency|//NSE//|
|''CC''|Pearson correlation coefficient (Kling-Gupta efficiency, part 1)|//CC//|
|''RE(%)''|relative bias in percent|//RE%//|
|''RSDE(%)''|relative error in standard deviation in percent|//RS%//|
|''Sim''|average of simulated variable|//cm//|
|''Rec''|average of observed variable|//rm//|
|''SDSim''|standard deviation of simulated variable|//cd//|
|''SDRec''|standard deviation of observed variable|//rd//|
|''MAE''|mean absolute error |//MAE//|
|''RMSE''|root mean square error |//RMSE//|
|''Bias''|bias|//Bias//|
|''SDE''|Error of standard deviation|//ES//|
|''KGE''|Kling-Gupta efficiency |//KGE//|
|''KGESD''|Kling-Gupta efficiency, part 2|//KGESD//|
|''KGEM''|Kling-Gupta efficiency, part 3|//KGEM//|
|''NRMSE''|normalised root mean square error |//NE//|
|''NSEW''|Nash-Sutcliffe efficiency adjusted for bias|//NSEW//|
|''MinRec''|minimum of observed variable|//rmin//|
|''MaxRec''|maximum of observed variable|//rmax//|
|''MinSim''|minimum of simulated variable|//cmin//|
|''MaxSim''|maximum of simulated variable|//cmax//|
</sortable>

Equation IDs for simulation assessment criteria ([[start:HYPE_file_reference:simass.txt|simass.txt]]):

<sortable>
^ Name          ^ Code     ^ Equation ID   ^
| Regional NSE  | ''RR2''  | //REGNSE//    |
| Regional RA   | ''RRA''  | //REGRA//     |
| Regional RE   | ''RRE''  | //REGRB//     |
| Regional MAE  | ''-''    | //REGMAE//    |
| Average NSE   | ''MR2''  | //AVNSE//     |
| Average RA    | ''MRA''  | //AVRA//      |
| Average RE    | ''MRE''  | //AVRB//      |
| Average RSDE  | ''MRS''  | //AVRSB//     |
| Average CC    | ''MCC''  | //AVCC//      |
| Average ARE   | ''MAR''  | //AVARB//     |
| Average KGE   | ''AKG''  | //AVKGE//     |
| Aver scalKGE  | ''ASK''  | //ASCKGE//    |
| Spatial NSE   | ''SR2''  | //SPATNSE//   |
| Spatial RA    | ''RRA''  | //SPATRA//    |
| Spatial RE    | ''-''    | //SPATRB//    |
| Spatial Bias  | ''SMB''  | //SPATASB//   |
| Spatial RMSE  | ''SNR''  | //SPATRMSE//  |
| Kendalls Tau  | ''TAU''  | //AVTAU//     |
| Median NSE    | ''MD2''  | //MEDNSE//    |
| Median RA     | ''MDA''  | //MEDRA//     |
| Median KGE    | ''MKG''  | //MEDKGE//    |
| Median NRMSE  | ''MNR''  | //MEDNE//     |
| Mean NSEW     | ''MNW''  | //AVNSEW//    |
</sortable>

Equation IDs for calibration simulation assessment criteria ([[start:HYPE_file_reference:bestsims.txt|bestsims.txt]] and [[start:HYPE_file_reference:allsim.txt|allsim.txt]]):

<sortable>
^ Heading ^ Description ^ Equation ID ^
|''rr2''|regional Nash-Sutcliffe efficiency (data from all subbasins combined in one data series)|//REGNSE//|
|''sr2''|spatial Nash-Sutcliffe efficiency, calculated using annual means for all subbasins (requires at least 5 years and 5 subbasins with data) to form one data series to calculate the Nash-Sutcliffe efficiency on|//SPATNSE//|
|''mr2''|average of Nash-Sutcliffe efficiency for subbasins|//AVNSE//|
|''rmae''|regional mean absolute error (data from all subbasins combined in one data series)|//REGMAE//|
|''sre''|spatial relative bias (calculated on annual means for all subbasins)|//SPATRB//|
|''rre''|regional relative bias (data from all subbasins combined in one data series)|//REGRB//|
|''mre''|average of the relative bias for all subbasins (Note: fraction, not %)|//AVRB//|
|''rra''|regional RA, similar to regional NSE, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged with a coefficient value|//REGRA//|
|''sra''|spatial RA, similar to spatial NSE, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged for a coefficient value|//SPATRA//|
|''mra''|average value of RA for subbasins, RA is a Nash-Sutcliffe like criterion where the square in the Nash-Sutcliffe formula is exchanged with a coefficient value|//AVRA//|
|''tau''|average of Kendall's Tau value for subbasins|//AVTAU//|
|''md2''|median of Nash-Sutcliffe efficiency for subbasins|//MEDNSE//|
|''mda''|median of all subbasins’ RA (Nash-Sutcliffe like criteria where the square is exchanged with a coefficient value)|//MEDRA//|
|''mrs''|average of error in standard deviation for subbasins|//AVRSB//|
|''mcc''|Pearson correlation coefficient, average of all subbasins with observations|//AVCC//|
|''mdkg''|median of Kling-Gupta efficiency (MKG in [[start:hype_file_reference:info.txt|info.txt]]) for subbasins|//MEDKGE//|
|''akg''|average of Kling-Gupta efficiency for subbasins|//AVKGE//|
|''asckg''|average of Kling-Gupta efficiency rescaled to interval [-1,1] (C2M criteria applied to KGE, Mathevet et al. 2006)|//ASCKGE//|
|''mare''|average of absolute relative bias for subbasins (Note: fraction. not %) (MAR in [[start:hype_file_reference:info.txt|info.txt]])|//AVARB//|
|''mdnr''|median of normalised RMSE for subbasins|//MEDNE//|
|''mnw''|average of Nash-Sutcliffe efficiencies adjusted for bias for subbasins|//AVNSEW//|
|''snr''|spatial root mean square error|//SPATRMSE//|
|''smb''|spatial mean absolute scaled bias on natural log transformed values|//SPATASB//|
</sortable>

Equation IDs for performance criteria set in info.txt are tabled [[start:HYPE_file_reference:info.txt:criteria|here]].


===== Equation definitions =====

==== Denotations ====
<sortable>
|//c//|computed value|
|//r//|recorded value|
|//cl//|log transform of computed value, natural logarithm|
|//rl//|log transform of recorded value, natural logarithm|
|//i//|index for time steps with observations in a time series of a station|
|//mi//|number of values in a time series of a station|
|//j//|index of stations|
|//mj//|number of stations|
|//ij//|index over time steps with observations for all stations|
|//mij//|number of time steps with obsevations for all stations|
|//cm//|average value of <m> c_{i}, i=1,mi </m> for a station|
|//rm//|average value of <m> r_{i}, i=1,mi </m> for a station|
|//cd//|standard deviation of <m> c_{i}, i=1,mi </m> for a station|
|//rd//|standard deviation of <m> r_{i}, i=1,mi </m> for a station|
|//cmax//|maximum value of <m> c_{i}, i=1,mi </m> for a station|
|//rmax//|maximum value of <m> r_{i}, i=1,mi </m> for a station|
|//cmin//|minimum value of <m> c_{i}, i=1,mi </m> for a station|
|//rmin//|minimum value of <m> r_{i}, i=1,mi </m> for a station|
|//w//|weight of station|
</sortable>

==== Basic equations ====

Average value for a time series of a station:

<m> xm = {1/mi} sum{i=1}{mi}{x_{i}} </m>  //x=r// or //c//

Standard deviation of a time series of a station:

<m> xd = sqrt{{1/mi} sum{i=1}{mi}{{x_{i}}^2}-xm^2} </m>  //x=r// or //c//

Natural logaritm of value:

<m> xl = LN(x) </m>  //x=r// or //c// or //rm// or //cm//, //x>0//


====Criteria equations for a time series of a station====

Nash-Sutcliffe Efficiency (//NSE// or R2):

<m> NSE = 1-{sum{i=1}{mi}{(c_{i}-r_{i})^2}}/{sum{i=1}{mi}{(r_{i}-rm)^2}} </m>

Efficiency with coefficient a (//RA//):

<m> RA = 1-{sum{i=1}{mi}{delim{|} {c_{i}-r_{i}} {|}^a}}/{sum{i=1}{mi}{delim{|} {r_{i}-rm} {|}^a}} </m>

Bias:

<m> Bias = {sum{i=1}{mi}{(c_{i}-r_{i})}}/mi </m>

Relative bias (//RB// or RE):

<m> RB = {sum{i=1}{mi}{(c_{i}-r_{i})}}/{delim{|}{sum{i=1}{mi}{r_{i}}}{|}} </m>

Relative bias in percent (//RE%//):

<m> RE% = RB*100 = {{sum{i=1}{mi}{(c_{i}-r_{i})}}/{delim{|}{sum{i=1}{mi}{r_{i}}}{|}}}*100 </m>

Error of standard deviation (//ES//):

<m> ES = {cd-rd} </m>

Relative error of standard deviation (//RS//):

<m> RS = {{cd-rd}/rd} </m>

Relative error of standard deviation in percent (//RS%//):

<m> RS% = RS*100 = {{cd-rd}/rd}*100 </m>

Mean absolute error (//MAE//):

<m> MAE = {sum{i=1}{mi}{delim{|}{c_{i}-r_{i}}{|}}}/mi </m>

Kling-Gupta efficiency (//KGE//):

<m> KGE = 1-sqrt{(CC-1)^2+(cd/rd-1)^2+(cm/rm-1)^2} </m>  //cm>0// and //rm>0// and //cd>0// and //rd>0//

Pearson correlation coefficient, Kling-Gupta efficiency part 1 (//CC//):

<m> CC = {{1/mi} sum{i=1}{mi}{({r_{i}}*{c_{i}})}-cm*rm}/{cd*rd} </m>

Kling-Gupta efficiency part 2 (//KGESD//):

<m> KGESD = cd/rd </m>

Kling-Gupta efficiency part 3 (//KGEM//):

<m> KGEM = cm/rm </m>

Root mean square error (//RMSE//):

<m> RMSE = sqrt{{1/mi sum{i=1}{mi}{({c_{i}}-{r_{i}})^2}}} </m>

Normalised root mean square error (//NE//):

<m> NE = sqrt{{1/mi sum{i=1}{mi}{({c_{i}}-{r_{i}})^2}}}/{max{(r_{i}})} </m>

Kendalls rank correlation coefficient, tau-b, with adjustments for ties (//TAU//):

<m> TAU = {n_{c}-n_{d}}/{sqrt{(n_{0}-n_{1})(n_{0}-n_{2})}} </m>

Nash-Sutcliffe Efficiency adjusted for bias (//NSEW//). Introduced in Lindström (2016):

<m> NSEW = NSE+Bias^2/rd^2 </m>

where 

<m> n_{c} </m> = number of concordant pairs (<m> c_{i}<c_{k} and r_{i}<r_{k} or c_{i}>c_{k} and r_{i}>r_{k}, i=1,mi k=1,mi </m>)

<m> n_{d} </m> = number of discordant pairs (<m> c_{i}<c_{k} and r_{i}>r_{k} or c_{i}>c_{k} and r_{i}<r_{k}, i=1,mi k=1,mi </m>)

<m> n_{0} </m> = number of compared pairs

<m> n_{1} </m> = number of compared pairs that ties in the computed values

<m> n_{2} </m> = number of compared pairs that ties in the recorded values

Scaled bias (//ScBias//):	

<m> ScBias = {sum{i=1}{mi}{delim{|}{{(c_{i}-r_{i})}/{(c_{i}+r_{i})}}{|}}}/mi </m>

Scaled KGE (//SCKGE//):

<m> SCKGE = KGE/{2-KGE} </m>

====Criteria equations for a model domain (several stations)====

Average Nash-Sutcliffe efficiency (//AVNSE//): 

//AVNSE// arithmetric mean

<m> AVNSE = {1/mj sum{j=1}{mj}{NSE_{j}}} </m> 

or //AVNSE// weighted average

<m> AVNSE = {sum{j=1}{mj}{w_{j}*NSE_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Median Nash-Sutcliffe efficiency (//MEDNSE//):

<m> MEDNSE = median delim{lbrace}{{NSE_{j}},{j=1..mj}}{rbrace} </m>

Spatial Nash-Sutcliffe efficiency (//SPATNSE//):

<m> SPATNSE = 1-{sum{j=1}{mj}{(cm_{j}-rm_{j})^2}}/{sum{j=1}{mj}{(rm_{j}-{1/mj} sum{j=1}{mj}{rm_{j}})^2}} </m>

Regional Nash-Sutcliffe efficiency (//REGNSE//):

<m> REGNSE = 1-{sum{ij=1}{mij}{(c_{ij}-r_{ij})^2}}/{sum{ij=1}{mij}{(r_{ij}-{1/mij} sum{ij=1}{mij}{r_{ij}})^2}} </m>

Average Nash-Sutcliffe efficiency adjusted for bias (//AVNSEW//):

//AVNSEW// arithmetric mean

<m> AVNSEW = {1/mj sum{j=1}{mj}{NSEW_{j}}} </m>

or //AVNSEW// weighted average

<m> AVNSEW = {sum{j=1}{mj}{w_{j}*NSEW_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Average efficiency with coefficient a (//AVRA//):

//AVRA// arithmetric mean

<m> AVRA = {1/mj sum{j=1}{mj}{RA_{j}}} </m>

or //AVRA// weighted average

<m> AVRA = {sum{j=1}{mj}{w_{j}*RA_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Median efficiency with coefficient a (//MEDRA//):

<m> MEDRA = median delim{lbrace}{{RA_{j}},{j=1..mj}}{rbrace} </m>

Spatial efficiency with coefficient a (//SPATRA//):

<m> SPATRA = 1-{sum{j=1}{mj}{delim{|}{cm_{j}-rm_{j}}{|}^a}}/{sum{j=1}{mj}{delim{|}{rm_{j}-{1/mj} sum{j=1}{mj}{rm_{j}}}{|}^a}} </m>

Regional efficiency with coefficient a (//REGRA//):

<m> REGRA = 1-{sum{ij=1}{mij}{delim{|}{c_{ij}-r_{ij}}{|}^a}}/{sum{ij=1}{mij}{delim{|}{r_{ij}-{1/mij} sum{ij=1}{mij}{r_{ij}}}{|}^a}} </m>

Average relative bias (//AVRB//):

//AVRB// arithmetric mean

<m> AVRB = {1/mj sum{j=1}{mj}{RB_{j}}} </m>

or //AVRB// weighted average

<m> AVRB = {sum{j=1}{mj}{w_{j}*RB_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Regional relative bias (//REGRB//):

<m> REGRB = {sum{ij=1}{mij}{(c_{ij}-r_{ij})}}/{delim{|}{sum{ij=1}{mij}{r_{ij}}}{|}} </m>

Spatial relative bias (//SPATRB//):

<m> SPATRB = {sum{j=1}{mj}{(cm_{j}-rm_{j})}}/{delim{|}{sum{j=1}{mj}{rm_{j}}}{|}} </m>

Average Kling-Gupta efficiency (//AVKGE//):

//AVKGE// arithmetric mean

<m> AVKGE = {1/mj sum{j=1}{mj}{KGE_{j}}} </m>

or //AVKGE// weighted average

<m> AVKGE = {sum{j=1}{mj}{w_{j}*KGE_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Median Kling-Gupta efficiency (//MEDKGE//):

<m> MEDKGE = median delim{lbrace}{{KGE_{j}},{j=1..mj}}{rbrace} </m>

Average scaled Kling-Gupta efficiency (//ASCKGE//):

//ASCKGE// arithmetric mean

<m> ASCKGE = {1/mj sum{j=1}{mj}{SCKGE_{j}}} </m>

or //ASCKGE// weighted average

<m> ASCKGE = {sum{j=1}{mj}{w_{j}*SCKGE_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Spatial root mean square error (//SPATRMSE//):

<m> SPATRMSE = sqrt{{1/mj sum{j=1}{mj}{({cm_{j}}-{rm_{j}})^2}}} </m>

Median of Normalised root mean square error (//MEDNE//):

<m> MEDNE = median delim{lbrace}{{NE_{j}},{j=1..mj}}{rbrace} </m>

Average of absolute relative bias (//AVARB//):

//AVARB// arithmetric mean

<m> AVARB = {1/mj sum{j=1}{mj}{delim{|}{RB_{j}}{|}}} </m>

or //AVARB// weighted average

<m> AVARB = {sum{j=1}{mj}{w_{j}*{delim{|}{RB_{j}}{|}}}}/{sum{j=1}{mj}{w_{j}}} </m>

Average Pearson correlation coefficient (//AVCC//):

//AVCC// arithmetric mean

<m> AVCC = {1/mj sum{j=1}{mj}{CC_{j}}} </m>

or //AVCC// weighted average

<m> AVCC = {sum{j=1}{mj}{w_{j}*CC_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Average relative error of standard deviation (//AVRSB//):

//AVRSB// arithmetric mean

<m> AVRSB = {1/mj sum{j=1}{mj}{RS_{j}}} </m>

or //AVRSB// weighted average

<m> AVRSB = {sum{j=1}{mj}{w_{j}*RS_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Average Kendalls rank correlation coefficient (//AVTAU//):

//AVTAU// arithmetric mean

<m> AVTAU = {1/mj sum{j=1}{mj}{TAU_{j}}} </m>

or //AVTAU// weighted average

<m> AVTAU = {sum{j=1}{mj}{w_{j}*TAU_{j}}}/{sum{j=1}{mj}{w_{j}}} </m>

Regional mean absolute error (//REGMAE//):

<m> REGMAE = {sum{ij=1}{mij}{delim{|}{c_{ij}-r_{ij}}{|}}}/mij </m>

Spatial mean absolute scaled bias on log transformed values (//SPATASB//):

<m> SPATASB = {sum{j=1}{mj}{delim{|}{{cml_{j}-rml_{j}}/{cml_{j}+rml_{j}}}{|}}}/{mj} </m>


==== References ====
Lindström, G., 2016. Lake water levels for calibration of the S-HYPE model. Hydrology Research 47.4:672-682. doi: 10.2166/nh.2016.019.

Mathevet et al. 2006. A bounded version of the Nash-Sutcliffe criterion for better model assessment on large sets of basins. In: Large Sample Basin Experiments for Hydrological Model Parameterization: Results of the Model Parameter Experiment–MOPEX. IAHS Publ. 307, 2006, p. 211-219.