SciPy Least-Sq Trouble

Reading time ~3 minutes

I noticed something weird today. The exact same inputs and code were exhibiting completely different behaviour on two different clusters. The only difference between them was SciPy versions: 0.10.1 (correct behaviour) and 0.12.0 (incorrect behaviour). Here’s the line in question:

p1, cov_p, infodict, mesg, ier = scipy.optimize.leastsq(errfunc, p0.copy()[0], args=args, full_output=True)

The correct behaviour on 0.10.1:

ipdb> scipy.__version__
'0.10.1'
ipdb> errfunc(p0.copy()[0], *args)
array([ 0.06799529,  0.07318012,  0.06680378,  0.05200964,  0.05814424,
        0.09025226,  0.09680308,  0.05702837, -0.14674592, -0.22665459,
       -0.15485406, -0.01311882,  0.08502507,  0.10292671,  0.08557168,
        0.05098229,  0.04956718,  0.06520266, -0.05950772, -0.29728424])
ipdb> scipy.optimize.leastsq(errfunc, p0.copy()[0], args=args)
(array([  4.78875656e+03,   6.67606610e-02,   6.42906789e-01]), 2)

The incorrect behaviour on 0.12.0 (after excluding all other differences and possibilities):

ipdb> scipy.__version__
'0.12.0'
ipdb> errfunc(p0.copy()[0], *args)
array([ 0.06799529,  0.07318012,  0.06680378,  0.05200964,  0.05814424,
        0.09025226,  0.09680308,  0.05702837, -0.14674592, -0.22665459,
       -0.15485406, -0.01311882,  0.08502507,  0.10292671,  0.08557168,
        0.05098229,  0.04956718,  0.06520266, -0.05950772, -0.29728424])
ipdb> scipy.optimize.leastsq(errfunc, p0.copy()[0], args=args)
(array([  4.78874773e+03,   7.96486918e-02,   4.68803543e-01]), 2)

You can see that errfunc behaves the same way, but scipy.optimize.leastsq does not. Well, if you ever have this problem too then all you need to do is edit the epsfcn flag. The epsfcn flag is described as:

A suitable step length for the forward-difference approximation of the Jacobian (for Dfun=None). If epsfcn is less than the machine precision, it is assumed that the relative errors in the functions are of the order of the machine precision.

In scipy 0.10.1 the default value is 0.0, but in 0.12.0 the default value is None. In this example, 0.0 and None are very different beasts, which makes the default behaviour for scipy.optimize.leastsq unintuitively different between versions.

On 0.12.0 (the previously ‘incorrect’ behaviour):

ipdb> scipy.__version__
'0.12.0'
ipdb> optimize.leastsq(errfunc, p0.copy()[0], args=args, epsfcn=0.0)
(array([  4.78875656e+03,   6.67606608e-02,   6.42906789e-01]), 2)
ipdb> optimize.leastsq(errfunc, p0.copy()[0], args=args, epsfcn=None)
(array([  4.78874773e+03,   7.96486918e-02,   4.68803543e-01]), 2)

So there you go. If you’re using scipy.optimize.leastsq, make sure you specify epsfcn as 0.0 (or whatever) to be sure your code is future-compatible.

Making Python GUIs

Sometimes I want to make a simple (or complex) graphical user interface (GUI) for exploratory data analysis. I use Python, but there are ...… Continue reading

Hiatus

Published on September 13, 2015

Best and Brightest EMP Stars

Published on September 18, 2014