Commit 66c6007a authored by Philipp Zumstein's avatar Philipp Zumstein Committed by GitHub
Browse files

Clip exponential in ffunc to avoid overflow

This should avoid (hopefully) some possible FloatingPointError overflow errors.

The sigmoid function ffunc is for any x<-20 and x>20 already 0 resp. 1 up to 10^-9
and cutting will therefore not change the function substantially.

This idea is from @tmbdev in
Implemented first in
Additional infos from
parent fc525cb2
...@@ -380,7 +380,8 @@ class MLP(Network): ...@@ -380,7 +380,8 @@ class MLP(Network):
def ffunc(x): def ffunc(x):
"Nonlinearity used for gates." "Nonlinearity used for gates."
return 1.0/(1.0+exp(-x)) # cliping to avoid overflows
return 1.0/(1.0+exp(clip(-x,-20,20)))
def fprime(x,y=None): def fprime(x,y=None):
"Derivative of nonlinearity used for gates." "Derivative of nonlinearity used for gates."
if y is None: y = sigmoid(x) if y is None: y = sigmoid(x)
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment