Minimizer Options

This section is used to declare the minimizers to use for each fitting software. If a fitting software has been selected in Fitting Options then a default set of minimizers for that solver will be run unless alternative minimizer options have been set. All minimizers for a software are included on the default list of minimizers unless otherwise stated.

Warning

Options set in this section will only have an effect if the related software is also set in Fitting Options (either explicitly, or as a default option).

Bumps (bumps)

Bumps is a set of data fitting (and Bayesian uncertainty analysis) routines. It came out of the University of Maryland and NIST as part of the DANSE (Distributed Data Analysis of Neutron Scattering Experiments) project.

FitBenchmarking currently supports the Bumps minimizers:

Licence The main licence file for Bumps is here. Individual files have their own copyright and licence – if you plan to incorporate this in your own software you should first check that the licences used are compatible.

Links GitHub - bumps

The Bumps minimizers are set as follows:

[MINIMIZERS]
bumps: amoeba
       lm-bumps
       newton
       de
       scipy-leastsq

Warning

The additional dependency Bumps must be installed for this to be available; See Extra dependencies.

Note

de is not included in the default list of minimizers for bumps. To run this solver, you must explicitly set the minimizer as seen above.

DFO (dfo)

There are two Derivative-Free Optimization packages, DFO-LS and DFO-GN. They are derivative free optimization solvers that were developed by Lindon Roberts at the University of Oxford, in conjunction with NAG. They are particularly well suited for solving noisy problems.

FitBenchmarking currently supports the DFO minimizers:

Licence Both DFO-GN and DFO-LS are available under the GPL-3 licence. A proprietary licence is also available from NAG .

Links GitHub - DFO-GN GitHub - DFO-LS

The DFO minimizers are set as follows:

[MINIMIZERS]
dfo: dfols
     dfogn

Warning

Additional dependencies DFO-GN and DFO-LS must be installed for these to be available; See Extra dependencies.

Gradient-Free-Optimizers (gradient_free)

Gradient-Free-Optimizers are a collection of gradient-free methods capable of solving various optimization problems. Please note that Gradient-Free-Optimizers must be run with problems that have finite bounds on all parameters.

  • Hill Climbing (HillClimbingOptimizer)

  • Repulsing Hill Climbing (RepulsingHillClimbingOptimizer)

  • Simulated Annealing (SimulatedAnnealingOptimizer)

  • Random Search (RandomSearchOptimizer)

  • Random Restart Hill Climbing (RandomRestartHillClimbingOptimizer)

  • Random Annealing (RandomAnnealingOptimizer)

  • Parallel Tempering (ParallelTemperingOptimizer)

  • Particle Swarm (ParticleSwarmOptimizer)

  • Evolution Strategy (EvolutionStrategyOptimizer)

  • Bayesian (BayesianOptimizer)

  • Tree Structured Parzen Estimators (TreeStructuredParzenEstimators)

  • Decision Tree (DecisionTreeOptimizer)

Licence The Gradient-Free-Optimizers package is available under an MIT Licence .

The gradient_free minimizers are set as follows:

[MINIMIZERS]
gradient_free: HillClimbingOptimizer
               RepulsingHillClimbingOptimizer
               SimulatedAnnealingOptimizer
               RandomSearchOptimizer
               RandomRestartHillClimbingOptimizer
               RandomAnnealingOptimizer
               ParallelTemperingOptimizer
               ParticleSwarmOptimizer
               EvolutionStrategyOptimizer
               BayesianOptimizer
               TreeStructuredParzenEstimators
               DecisionTreeOptimizer

Warning

The additional dependency Gradient-Free-Optimizers must be installed for this to be available; See Extra dependencies.

Note

BayesianOptimizer, TreeStructuredParzenEstimators and DecisionTreeOptimizer may be slow running and so are not run by default when gradient_free software is selected. To run these minimizers you must explicity set them as seen above.

GSL (gsl)

The GNU Scientific Library is a numerical library that provides a wide range of mathematical routines. We call GSL using the pyGSL Python interface.

The GSL routines have a number of parameters that need to be chosen, often without default suggestions. We have taken the values as used by Mantid.

We provide implementations for the following packages in the multiminimize and multifit sections of the library:

Links SourceForge PyGSL

Licence The GNU Scientific Library is available under the GPL-3 licence .

The GSL minimizers are set as follows:

[MINIMIZERS]
gsl: lmsder
     lmder
     nmsimplex
     nmsimplex2
     conjugate_pr
     conjugate_fr
     vector_bfgs
     vector_bfgs2
     steepest_descent

Warning

The external packages GSL and pygsl must be installed to use these minimizers.

Horace (horace)

Horace is described as a suite of programs for the visiualization and analysis from time-of-flight neutron inelastic scattering spectrometers. We currently support:

  • Levenberg-Marquardt (lm-lsqr)

Licence Matlab must be installed to use Horace within FitBenchmarking, which is a proprietary product. Horace is made available under the the GPL-3 licence.

Warning

The Horace Toolbox and MATLAB must be installed for this to be available; see Installing External Software.

Mantid (mantid)

Mantid is a framework created to manipulate and analyze neutron scattering and muon spectroscopy data. It has support for a number of minimizers, most of which are from GSL.

Licence Mantid is available under the GPL-3 licence .

The Mantid minimizers are set as follows:

[MINIMIZERS]
mantid: BFGS
        Conjugate gradient (Fletcher-Reeves imp.)
        Conjugate gradient (Polak-Ribiere imp.)
        Damped GaussNewton
        FABADA
        Levenberg-Marquardt
        Levenberg-MarquardtMD
        Simplex
        SteepestDescent
        Trust Region

Warning

The external package Mantid must be installed to use these minimizers.

Levmar (levmar)

The levmar package which implements the Levenberg-Marquardt method for nonlinear least-squares. We interface via the python interface available on PyPI.

  • Levenberg-Marquardt with supplied Jacobian (levmar) - the Levenberg-Marquardt method

Licence Levmar is available under the GPL-3 licence . A paid licence for proprietary commerical use is available from the author .

The levmar minimizer is set as follows:

[MINIMIZERS]
levmar: levmar

Warning

The additional dependency levmar must be installed for this to be available; See Extra dependencies. This package also requires the BLAS and LAPACK libraries to be present on the system.

Matlab (matlab)

We call the fminsearch function from MATLAB, using the MATLAB Engine API for Python.

  • Nelder-Mead Simplex (Nelder-Mead Simplex)

Licence Matlab is a proprietary product .

The matlab minimizer is set as follows:

[MINIMIZERS]
matlab: Nelder-Mead Simplex

Warning

MATLAB must be installed for this to be available; See Installing External Software.

Matlab Curve Fitting Toolbox (matlab_curve)

We call the fit function from the MATLAB Curve Fitting Toolbox, using the MATLAB Engine API for Python.

  • Levenberg-Marquardt (Levenberg-Marquardt)

  • Trust-Region (Trust-Region)

Licence Matlab and the Curve Fitting Toolbox are both proprietary products .

The matlab_curve minimizers are set as follows:

[MINIMIZERS]
matlab_curve: Levenberg-Marquardt
              Trust-Region

Warning

MATLAB Curve Fitting Toolbox must be installed for this to be available; See Installing External Software.

Matlab Optimization Toolbox (matlab_opt)

We call the lsqcurvefit function from the MATLAB Optimization Toolbox, using the MATLAB Engine API for Python.

  • Levenberg-Marquardt (levenberg-marquardt)

  • Trust-Region-Reflective (trust-region-reflective)

Licence Matlab and the Optimization Toolbox are both proprietary products .

The matlab_opt minimizers are set as follows:

[MINIMIZERS]
matlab_opt: levenberg-marquardt
            trust-region-reflective

Warning

MATLAB Optimization Toolbox must be installed for this to be available; See Installing External Software.

Matlab Statistics Toolbox (matlab_stats)

We call the nlinfit function from the MATLAB Statistics Toolbox, using the MATLAB Engine API for Python.

  • Levenberg-Marquardt (Levenberg-Marquardt)

Licence Matlab and the Statistics Toolbox are both proprietary products .

The matlab_stats minimizer is set as follows:

[MINIMIZERS]
matlab_stats: Levenberg-Marquardt

Warning

MATLAB Statistics Toolbox must be installed for this to be available; See Installing External Software.

Minuit (minuit)

CERN developed the Minuit 2 package to find the minimum value of a multi-parameter function, and also to compute the uncertainties. We interface via the python interface iminuit with support for the 2.x series.

Links Github - iminuit

Licence iminuit is released under the MIT licence, while Minuit 2 is LGPL v2 .

The Minuit minimizers are set as follows:

[MINIMIZERS]
minuit: minuit

Warning

The additional dependency Minuit must be installed for this to be available; See Extra dependencies.

RALFit (ralfit)

RALFit is a nonlinear least-squares solver, the development of which was funded by the EPSRC grant Least-Squares: Fit for the Future. RALFit is designed to be able to take advantage of higher order derivatives, although only first order derivatives are currently utilized in FitBenchmarking.

  • Gauss-Newton, trust region method (gn)

  • Hybrid Newton/Gauss-Newton, trust region method (hybrid)

  • Gauss-Newton, regularization (gn_reg)

  • Hybrid Newton/Gauss-Newton, regularization (hybrid_reg)

Links Github - RALFit. RALFit’s Documentation on: Gauss-Newton/Hybrid models, the trust region method and The regularization method

Licence RALFit is available under a 3-clause BSD Licence

The RALFit minimizers are set as follows:

[MINIMIZERS]
ralfit: gn
        gn_reg
        hybrid
        hybrid_reg

Warning

The external package RALFit must be installed to use these minimizers.

SciPy (scipy)

SciPy is the standard python package for mathematical software. In particular, we use the minimize solver for general minimization problems from the optimization chapter of SciPy’s library. Currently we only use the algorithms that do not require Hessian information as inputs.

Links Github - SciPy minimize

Licence Scipy is available under a 3-clause BSD Licence. Individual packages may have their own (compatible) licences, as listed here.

The SciPy minimizers are set as follows:

[MINIMIZERS]
scipy: Nelder-Mead
       Powell
       CG
       BFGS
       Newton-CG
       L-BFGS-B
       TNC
       SLSQP

SciPy LS (scipy_ls)

SciPy is the standard python package for mathematical software. In particular, we use the least_squares solver for Least-Squares minimization problems from the optimization chapter of SciPy’s library.

  • Levenberg-Marquardt with supplied Jacobian (lm-scipy) - a wrapper around MINPACK

  • The Trust Region Reflective algorithm (trf)

  • A dogleg algorithm with rectangular trust regions (dogbox)

Links Github - SciPy least_squares

Licence Scipy is available under a 3-clause BSD Licence. Individual packages many have their own (compatible) licences, as listed here.

The SciPy least squares minimizers are set as follows:

[MINIMIZERS]
scipy_ls: lm-scipy
          trf
          dogbox

SciPy GO (scipy_go)

SciPy is the standard python package for mathematical software. In particular, we use the Global Optimization solvers for global optimization problems from the optimization chapter of SciPy’s library.

Links Github - SciPy optimization

Licence Scipy is available under a 3-clause BSD Licence. Individual packages may have their own (compatible) licences, as listed here.

The SciPy global optimization minimizers are set as follows:

[MINIMIZERS]
scipy_go: differential_evolution
          shgo
          dual_annealing

Note

The shgo solver is particularly slow running and should generally be avoided. As a result, this solver is not run by default when scipy_go software is selected. In order to run this minimizer, you must explicitly set it as above.