made one change to the cascade node training function (i actually made a new function that it uses called CalcSigmaO) because it looks like i might not have understood how the calculation of “the sign of the correlation between the candidate’s value and output” term (σo) worked. i was previously calculating a different σ for every o and p combination, but it seems that σ is actually just for the outputs, not different for every training pattern. now, looking at the subscript, this should have been obvious to me. i bet i made the same mistake in 2006 as well. i will leave the network running overnight tonight on the circle problem. after 5 nodes were added, here is its output:
i might want to try multiple candidate nodes and adjusting weights (the coefficient) by greater amounts the first times through the training set and then decreasing as time goes on slowly, since it seems to be rapidly adding nodes now with little change to the network’s output while still adding a few points. file cascnet7.nb has been backed up on euclid.
