Comparing networks with differing neural-node functions using transputer-based genetic algorithms

Abstract

Different neural net node computational functions are compared using feedforward backpropagation networks. Three node types are examined: the standard model, ellipsoidal nodes and quadratic nodes. After preliminary experiments on simple small problems, in which quadratic nodes performed very well, networks of differing nodes types are applied to the speech recognition 104 speaker E-task using a fixed architecture. Ellipsoidal nodes were found to work well, but not as well as the standard model. Quadratic nodes did not perform well on the larger task. To facilitate an architecture independent comparison a transputer-based genetic algorithm is then used to compare ellipsoidal and mixed ellipsoidal and standard networks with the standard model. These experiments confirmed the earlier conclusion that ellipsoidal networks could not compare favourably with the standard model on the 104 speaker E-task. In an evolutionary search in which layer node types were free to adjust ellipsoidal nodes had a tendency to become extinct or barely survive. If the presence of ellipsoidal nodes was enforced then the resulting networks again performed poorly when compared with the standard model.