sc_asd

sc_asd

Adaptive stochastic descent optimization algorithm, building on scipy.optimize.

This algorithm is published as:

Kerr CC, Dura-Bernal S, Smolinski TG, Chadderdon GL, Wilson DP (2018). Optimization by Adaptive Stochastic Descent. PLoS ONE 13(3): e0192944. https://doi.org/10.1371/journal.pone.0192944

Functions

Name	Description
asd	Optimization using adaptive stochastic descent (ASD). Can be used as a faster

asd

sc_asd.asd(
    function,
    x,
    args=None,
    stepsize=0.1,
    sinc=2,
    sdec=2,
    pinc=2,
    pdec=2,
    pinitial=None,
    sinitial=None,
    xmin=None,
    xmax=None,
    maxiters=None,
    maxtime=None,
    abstol=1e-06,
    reltol=0.001,
    stalliters=None,
    stoppingfunc=None,
    randseed=None,
    label=None,
    verbose=1,
    minval=0,
    die=True,
    **kwargs,
)

Optimization using adaptive stochastic descent (ASD). Can be used as a faster and more powerful alternative to e.g. scipy.optimize.minimize().

ASD starts at x and attempts to find a local minimizer of the function function(). function() accepts input x and returns a scalar function value evaluated at x. x can be a scalar, list, or Numpy array of any size.

Parameters

Name	Type	Description	Default
function	`func`	The function to minimize	required
x	`arr`	The vector of initial parameters	required
args	any	List, tuple, or dictionary of additional parameters to be passed to the function	`None`
kwargs	dict	Additional keywords passed to the function	`{}`
stepsize	0.1	Initial step size as a fraction of each parameter	`0.1`
sinc	2	Step size learning rate (increase)	`2`
sdec	2	Step size learning rate (decrease)	`2`
pinc	2	Parameter selection learning rate (increase)	`2`
pdec	2	Parameter selection learning rate (decrease)	`2`
pinitial	None	Set initial parameter selection probabilities	`None`
sinitial	None	Set initial step sizes; if empty, calculated from stepsize instead	`None`
xmin	None	Min value allowed for each parameter	`None`
xmax	None	Max value allowed for each parameter	`None`
maxiters	1000	Maximum number of iterations (1 iteration = 1 function evaluation)	`None`
maxtime	3600	Maximum time allowed, in seconds	`None`
abstol	1e-06	Minimum absolute change in objective function	`1e-06`
reltol	0.001	Minimum relative change in objective function	`0.001`
stalliters	10 * `n`	Number of iterations over which to calculate TolFun (n = number of parameters)	`None`
stoppingfunc	None	External method that can be used to stop the calculation from the outside.	`None`
randseed	None	The random seed to use	`None`
label	None	A label to use to annotate the output	`None`
verbose	1	How much information to print during the run (max 3); less than one will print out once every 1/verbose steps	`1`
minval	0	Minimum value the objective function can take	`0`
die	True	If True, raise when the objective function raises; if False, treat that trial as np.inf and continue	`True`

Returns

Name	Type	Description
		objdict (see below)

The returned object is an objdict, which can be accessed by index, key, or attribute. Its keys/attributes are:

- `x`          -- The parameter set that minimizes the objective function
- `fval`       -- The value of the objective function at the final iteration
- `exitreason` -- Why the algorithm terminated;
- `details`    -- See below

The details key consists of:

- `fvals`         -- The value of the objective function at each iteration
- `xvals`         -- The parameter values at each iteration;
- `probabilities` -- The probability of each step; and
- `stepsizes`     -- The size of each step for each parameter.

Examples:

# Basic usage
import numpy as np
import sciris as sc
result = sc.asd(np.linalg.norm, [1, 2, 3])
print(result.x)

# With arguments: positional via args, or dict of keywords, or keyword arguments
def my_func(x, scale=1.0, weight=1.0):  # Example function with keywords
    return abs((x[0] - 1)) + abs(x[1] + 2)*scale + abs(x[2] + 3)*weight

result = sc.asd(my_func, x=[0, 0, 1], args=[0.5, 0.1]) # Option 1 for passing arguments
result = sc.asd(my_func, x=[0, 0, 1], args=dict(scale=0.5, weight=0.1)) # Option 2 for passing arguments
result = sc.asd(my_func, x=[0, 0, 1], scale=0.5, weight=0.1) # Option 3 for passing arguments

Please use the following citation for this method:

CC Kerr, S Dura-Bernal, TG Smolinski, GL Chadderdon, DP Wilson (2018).
Optimization by adaptive stochastic descent.
PLOS ONE 13 (3), e0192944.

New in version 3.0.0: Uses its own random number stream