start:hype_tutorials:automatic_calibration

This shows you the differences between two versions of the page.

Both sides previous revision Previous revision Next revision | Previous revision | ||

start:hype_tutorials:automatic_calibration [2018/09/06 15:24] cpers [Differential Evolution Markov Chain method (task DE)] |
start:hype_tutorials:automatic_calibration [2019/02/25 16:58] (current) cpers [Quasi-Newton methods (task Q1 and Q2)] |
||
---|---|---|---|

Line 7: | Line 7: | ||

There are in total **//nine methods of optimization//** to choose from in HYPE. The sampling methods are a basic Monte-Carlo simulation with random parameters values chosen within a user-specified parameter interval, and two progressive Monte-Carlo simulations where the Monte-Carlo simulations are made in stages with a reduced parameter space in between the stages. In addition it is possible to run an organized sampling of two parameters. The Differential Evolution Markov Chain method combines a genetic optimization algorithm with random sampling. The directional methods are the Brent method, two versions of quasi-Newton methods with different ways to calculate the gradient, and the method of steepest decent. | There are in total **//nine methods of optimization//** to choose from in HYPE. The sampling methods are a basic Monte-Carlo simulation with random parameters values chosen within a user-specified parameter interval, and two progressive Monte-Carlo simulations where the Monte-Carlo simulations are made in stages with a reduced parameter space in between the stages. In addition it is possible to run an organized sampling of two parameters. The Differential Evolution Markov Chain method combines a genetic optimization algorithm with random sampling. The directional methods are the Brent method, two versions of quasi-Newton methods with different ways to calculate the gradient, and the method of steepest decent. | ||

- | Given enough sampling points, even the simple **//sampling method//** can give a rough estimate of the optimum. An advantage of the sampling methods is that the number of function evaluations, and thus the computation time, is determined by the user. The sampling methods are useful to provide a starting point for the directional optimization methods. | + | Given enough sampling points, the simple **//sampling method//** can give a estimate of the optimum. An advantage of the sampling methods is that the number of function evaluations, and thus the computation time, is determined by the user. The sampling methods are useful to provide a starting point for the directional optimization methods. |

The **//Differential Evolution Markov Chain//** (DEMC) provides an uncertainty estimate of the optimum. The genetic algorithm (i.e. DE) works by proposing new members (parameter values) and then accepting or rejecting them. In addition to the random element of the creation of a proposal (by inheriting traits from other members and keeping some traits unchanged), in the DEMC method a random number is added to the proposed parameters and the proposal may be accepted by a certain probability even if the objective criterion is worse than for the replaced member. The advantage of DEMC versus plain DE is both the possibility to get a probability based uncertainty estimate of the global optimum and a better convergence towards it. | The **//Differential Evolution Markov Chain//** (DEMC) provides an uncertainty estimate of the optimum. The genetic algorithm (i.e. DE) works by proposing new members (parameter values) and then accepting or rejecting them. In addition to the random element of the creation of a proposal (by inheriting traits from other members and keeping some traits unchanged), in the DEMC method a random number is added to the proposed parameters and the proposal may be accepted by a certain probability even if the objective criterion is worse than for the replaced member. The advantage of DEMC versus plain DE is both the possibility to get a probability based uncertainty estimate of the global optimum and a better convergence towards it. | ||

- | The **//directional methods//** progress iteratively from one set of model parameters to a new set that have a better objective criterion. This is achieved by determining a direction of improvement, and then the optimal step length in that direction. The determination of the direction is what separates the different optimization methods. It is given by one parameter and the direction between the last two best parameter sets (for Brent method), or by a function of the gradient of the objective function. The methods using the gradient are more powerful, but require more evaluations. The directional methods depend on a starting point for their iterations. This choice of the starting point is important for the performance of the methods. It influences the calculation time and possibly which (local) optimum that is reached. | + | The **//directional methods//** progress iteratively from one set of model parameters to a new set that have a better objective criterion. This is achieved by determining a direction of improvement, and then the optimal step length in that direction. The directional methods assume there exist a minima within the space. The determination of the direction is what separates the different optimization methods. It is given by one parameter and the direction between the last two best parameter sets (for Brent method), or by a function of the gradient of the objective function. The methods using the gradient are more powerful, but require more evaluations. The directional methods depend on a starting point for their iterations. This choice of the starting point is important for the performance of the methods. It influences the calculation time and possibly which (local) optimum that is reached. |

The automatic calibration algorithm is controlled by means of two or three **//files//**: [[start:HYPE_file_reference:info.txt|info.txt]] and [[start:HYPE_file_reference:optpar.txt|optpar.txt]], and for some methods [[start:HYPE_file_reference: qnstartpar.txt|qNstartpar.txt]]. The following sections present and discuss the entries and numerical parameters of those two files, necessary and/or optional to use the automatic calibration. | The automatic calibration algorithm is controlled by means of two or three **//files//**: [[start:HYPE_file_reference:info.txt|info.txt]] and [[start:HYPE_file_reference:optpar.txt|optpar.txt]], and for some methods [[start:HYPE_file_reference: qnstartpar.txt|qNstartpar.txt]]. The following sections present and discuss the entries and numerical parameters of those two files, necessary and/or optional to use the automatic calibration. | ||

Line 175: | Line 175: | ||

|Figure 10: Example of optpar.txt file for the quasi-Newton method| | |Figure 10: Example of optpar.txt file for the quasi-Newton method| | ||

- | The quasi-Newton methods optimise all parameters at the same time. The parameter set is optimized with the line search routine starting from the point of the current best parameters. The direction of the search is determined by the gradient of the criteria surface at this point. The gradient can be estimated in three different ways in HYPE, the two quasi-Newton methods described in this section and the one called steepest descent in the next section. The optimization continues until one of several interruption criteria is fulfilled. | + | The quasi-Newton methods optimise all parameters at the same time. The direction of the search is determined by the gradient of the criteria surface at the point of the current best parameters. The parameter set is optimized with the line search routine along the line determined by the gradient. The gradient can be estimated in three different ways in HYPE, the two quasi-Newton methods described in this section and the one called steepest descent in the next section. The optimization continues until one of several interruption criteria is fulfilled. |

Calculating the gradient for the quasi-Newton method involves updating the inverse Hessian matrix. This can be done by two methods, both described in Nocedal and Wright (2006). Task Q1 uses the DFP (Davidon-Fletcher-Powell) method and task Q2 uses the BFGS (Broyden-Fletcher-Goldfarb-Shanno) method. | Calculating the gradient for the quasi-Newton method involves updating the inverse Hessian matrix. This can be done by two methods, both described in Nocedal and Wright (2006). Task Q1 uses the DFP (Davidon-Fletcher-Powell) method and task Q2 uses the BFGS (Broyden-Fletcher-Goldfarb-Shanno) method. |

start/hype_tutorials/automatic_calibration.1536240284.txt.gz ยท Last modified: 2018/09/06 15:24 by cpers

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International