Autonomy and homeostasis

Homeostatic adaptation

In the 1950’s British cybernetician W. Ross Ashby introduced the concept of an ultrastable system in his book Design for a Brain. If an organism needs to keep certain essential variables within bounds (e.g., blood temperature, pH) then a system that triggered internal parametrical changes whenever such variables approach or cross a boundary of viability would, in principle, have a chance of finding new parameters that make the challenged variables return to their viability zones. In the scheme on the left, the organism is represented in interaction with the environment: R represents the behaviour-generating sub-system of the organism which is in a closed sensorimotor loop with the environment (Env), S represents those parameters that affect R, and the meter represents the organism’s essential variables which can be directly affected by the environment and in turn affect S by introducing random step-changes in the parameter values. If these changes produce a new dynamics capable of restoring the essential variables to their equilibrium through the newly acquired behaviours, the organism will have adapted to the environmental challenge.

This principle can be applied to adaptation in evolved robots controlled by plastic neural networks. The artificial evolution of homeostatic controllers for robots was originally inspired by the work of James Garden Taylor (1962) who presented a theory based on Ashby’s concept of ultrastability to account for the fascinating results obtained in experiments on adaptation to distortions of the visual field. Taylor worked in the 1950s at Innsbruck with Erismann and Kohler on adaptation to different sorts of prismatic perturbations including full up/down and left/right inversion. These experiments involved sub jects wearing distorting goggles for a few weeks. They follow in style similar experiments by Stratton in the 1890s and Ewert in the 1930s. Participants, after several days, experienced behavioural adaptation to those behaviours that they enacted frequently (walking on busy roads, riding a bicycle, and even skiing). In some cases fragmented aspects of perceptual adaptation were reported (world did not look upside down anymore). Upon removal of the goggles a period of re-adaptation was always necessary.

Each neuron in the neurocontroller behaves homeostatically by triggering local plastic changes on incoming synapses whenever it is firing either too much or too little. A viability zone is thus defined in the space of neural activations (left). A mobile agent is evolved to produce a certain behaviour (phototaxis) and to remain as homeostatic as possible. This double fitness requirement (one behavioural and one corresponding to neural stability) generates agents capable adapting to unseen radical sensorimotor perturbations, including the left-right inversion of the visual field (by swapping the position of its sensors). Upon repeated presentations of light sources the agents approaches each new one in turn (below top). If the sensor are swapped (below bottom) the agents first moves away from the light (as expected). But this induces unstable firing in the neurons which in turn trigger local plastic changes. At some point the network finds its way into the viability zone resulting the the recovery of phototactic behaviour.

The result has interesting implications for understanding the dynamics of adaptive systems capable of long-term interactions with their environment. The homeostatic mechanism produces qualitatively natural developmental patterns such as the presence of critical periods of higher plasticity early in life.

Di Paolo, E. A., (2000). Homeostatic adaptation to inversion of the visual field and other sensorimotor disruptions. From Animals to Animals, Proc. of the Sixth International Conference on the Simulation of Adaptive Behavior, SAB’2000, Paris, J-A. Meyer, A. Berthoz, D. Floreano, H. Roitblat and S W. Wilson (eds), MIT Press, pp. 440 – 449.

Di Paolo, E. A., (2003). Organismically-inspired robotics: Homeostatic adaptation and natural teleology beyond the closed sensorimotor loop, in: K. Murase & T. Asakura (Eds) Dynamical Systems Approach to Embodiment and Sociality, Advanced Knowledge International, Adelaide, Australia, pp 19 – 42.

Harvey, I., Di Paolo, E. A., Tuci, E., Wood, R., Quinn, M., (2005). Evolutionary robotics: A new scientific tool for studying cognition. Artificial Life, 11(1/2), pp. 79 – 98.

Iizuka, H. and Di Paolo, E. A. (2008). Extended homeostatic adaptation: Improving the link between internal and behavioural stability. In From Animats to Animals 10, The Tenth International Conference on the Simulation of Adaptive Behavior, Osaka, Japan, July 7-10, 2008.

Di Paolo, E. A. (2010). Robotics inspired in the organism, Intellectica, 53-54: 129 – 162.

Dynamics of preference and goal-generation

The Ashbyan ideas behind homeostatic adaptation may be combined with an enactive view on autonomy to investigate complex conceptual issues such as spontaneous generation of new goals and the formation of behavioural preferences. In this view, a preference is not located anywhere in the agent’s cognitive architecture, but it is rather a constraining of behaviour which is in turn shaped by behaviour. Based on this idea, a minimal model of behavioural preference was developed in collaboration with Hiro Iizuka. A simulated mobile agent is modelled with a plastic neurocontroller, which holds two separate high dimensional homeostatic boxes in the space of neural dynamics used to regulated synaptic plasticity (right). An evolutionary algorithm is used for creating a link between the boxes and the performance of two different phototactic behaviours. After evolution, the agent’s performance exhibits some important aspects of behavioural preferences such as durability and spontaneous transitions. These two properties through the mutual constraining of internal and external dynamics in the flow of alternating high and low susceptibility to environmental variations.

Below: A study of “virtual” choices (what the agent might have done at a given time had the environment been different – in this case, the relative angular orientation of two light sources emitting different coloured light). Plots show the agent’s end target of approach depending on the angular orientation of each light (horizontal and vertical axes in each sub-plot) as a function of time over a presentation of 100 pairs of lights. At different points in time the agent shows the alternation between periods of “high commitment” to a goal (one colour is prevalent; agent practically ignores the other option) and periods of environmentally susceptible choice (both choices are highly represented).

Iizuka, H., and Di Paolo, E. A. (2007). Toward Spinozist robotics: Exploring the minimal dynamics of behavioural preference Adaptive Behavior, 15:359-376.

Di Paolo, E. A., and Iizuka, H., (2008). How (not) to model autonomous behaviour BioSystems 91: 409-423, Special issue on Modelling Autonomy.

A dynamical homeostat using programmable springs.

It is possible to test several of the above ideas in systems without segregated control mechanism capable of plastic adaptation. We have devised with Santosh Manicka such a real system (right) actuated by programmable springs (a real-time non-linear programmable actuation technology developed at Sussex by Bill Bigge that permits actuators to dynamically change their mechanical properties such as angle/force profiles, stiffness and damping as a function of speed, etc. thus allowing the online reconfiguration of actuator dynamics – and consequently control not by directive signals but by altering the “muscle” parameters very much like the equilibrium point theory we investigate here).

Figure: example of a programmable spring profile developed in an unsupervised manner.

We investigate this idea using a simple see-saw actuated system whose essential variable is the angle of the see-saw. We have modified important aspects of Ashby’s theory, transforming ultrastability into a local adaptation process. A way to move gradually towards an objective is by making sure at every step that there is as little deviation as possible while adapting to obstacles. The phase space on the left indicates force (red-blue gradient) as a function of angle and angular velocity for a successful “self-developed” profile. This approach has inspired us to model a local strategy to eventually attain viability (equilibrium) in a real complex dynamical system, amidst perturbations, using ultrastability to make sure that the path to viability itself is viable.

Our experiment involves a problem in adaptation similar to the pole- balancing problem. To solve it, we use ultrastability in a novel way, looking at the viability of dynamical transitions of the system in its phase space, to tweak the local properties of the actuator. Observations show that our approach is indeed effective in producing adaptive behaviour (see viable trajectories, right) although it still requires further testing in other platforms, thus supporting the original hypothesis that ultrastability can be an effective adaptive mechanism and laying a foundation for a promising new perspective in ultrastable robotics.

Manicka, S. and Di Paolo, E. A. (2009) Local ultrastability in a real system based on programmable springs in Kampis, G., Karsai, I and Szathmary, E. (eds) Advances in Artificial Life Proceedings of the 10th European Conference on Artificial Life, ECAL09, Budapest, September 13-16, 2009, LNAI 5777, Springer Verlag, pp. 87 – 94.