The Temporal Nature Of Reinforcement

The essentially temporal nature of reinforcement can be illustrated in a simple animal demonstration (Skinner, 1948). A hungry pigeon is placed in the conditioning apparatus untrained except to eat from the food dispenser. The food magazine is then presented every minute regardless of what the bird is doing. Even though no conditioning is explicitly planned, the operation of the food magazine will reinforce some performance. The particular performance conditioned will vary from bird to bird, but each will exhibit a substantial frequency of some activity. This spurious conditioning occurs because the presentation of the food magazine is likely to occur in close temporal proximity to some performance the bird may be engaged in. When the occurrence of the food magazine increases the frequency of a particular response, the bird is likely to be performing the same activity one minute later when the food magazine operates again. Continuing the periodic delivery of food will maintain the behavior. It is not crucial to the effect of the reinforcement whether or not the individual who arranged the periodic delivery of food purposefully conditioned the response. The bird's performance is being maintained by a fixed-interval schedule of intermittent reinforcement just as if it were specified. The only difference would be a slight drift in the form of the behavior, since the form of the response in the reinforcement contingency is not strictly defined. Over a long period of time, the bird's response would slowly drift from one form to another, but it would remain under the control of the fixed-interval schedule of intermittent reinforcement. This kind of reinforcement has been termed superstitious, spurious, or adventitious reinforcement in the sense that the particular form of behavior conditioned is not specified in the response-reinforcement contingency. It is also adventitious or superstitious in the sense that from the point of view of the experimental subject, this behavior is not necessary for the delivery of food. But in the strictest sense, this latter statement is simply a restatement of the non-specificity of the response-reinforcement contingency.

Much of human behavior is necessarily under this kind of control. The gas explosion occurring at the instant a salesman is about to press the front door buzzer will come to function as a preaversive stimulus, even though pressing the buzzer has no stable relationship in the physical environment to the probability of an explosion. Similarly, the gambler who shouts, "Come seven," just before the throw of the dice brings a seven, will find that his subsequent disposition to say "Come seven" will be higher, whether or not speaking has any influence on the outcome of the throw. The fact that seven appears on the dice only intermittently does not necessarily nullify the effect of the reinforcement. It simply specifies an intermittent schedule of reinforcement of the response and may, in fact, under appropriate circumstances, as in the previous discussion of intermittent reinforcement, strengthen the disposition to say, "Come seven." A similar example is the Indian medicine man praying and chanting for rain. The rain, as a reinforcing event for the praying and chanting, occurs on a variable-interval schedule: sooner or later rain will fall, thus reinforcing the praying and dancing. Once the "rain maker" has a strong disposition to pray for rain, its high frequency will make it likely that the behavior will occur when it finally rains. Superstitious behavior is likely to occur under conditions of strong deprivation or with an extremely powerful reinforcer. It is under these conditions that the initial reinforcement of a response will produce a substantial tendency to respond, and the probability is high that the same response will occur again at the time of the next reinforcement.

Both the doctor and the patient are susceptible to accidental reinforcement of therapeutic practices in ameliorating a disease. In the case of the patient the reinforced behavior is carrying out the therapy, and in the doctor, prescribing it. Many diseases have a natural course of development, regardless of the treatment, and any therapeutic procedure which might be carried out at the crucial point in the disease process might be reinforced by the recovery of the patient. Pityriasis rosea provides an excellent example of such a disease in which only symptomatic treatment is presently possible. This disease, which is severe in form and very uncomfortable for the patient, has a relatively fixed time course of seven to ten days and ends with complete recovery of the patient and life-long immunity. Any doctor treating this disease, or patient following the treatment, is susceptible to accidental reinforcement for the patient's recovery. Avoidance responses (see Chapter 15) are especially susceptible to adventitious reinforcement because the reinforcement of the avoidance response is the nonoccurrence of the aversive event. Once an avoidance response is accidentally reinforced, the subsequent nonoccurrence of the aversive stimulus will tend to continue to maintain the avoidance behavior. The person who knocks on wood on the occasion of some possibility of an aversive consequence and the person who crosses the street to avoid a black cat, or avoids walking under a ladder, are successfully avoiding aversive consequences, once these performances have some initial strength as avoidance responses. The fact that no harm occurs only serves to strengthen the avoidance behavior. The extinction of these performances would be defined inversely to positive reinforcement. To extinguish these behaviors some aversive consequences would have to occur despite the avoidance response. This is slightly difficult, however. Extinction can probably be best defined as the removing of the consequence of a response that is maintaining it. In avoidance this would be removing the postponement of the aversive stimulus, which is equivalent to punishing every response. Alternatively, we could remove aversive stimuli from the situation altogether.

Tics, phobias, or compulsions may be similar avoidance responses, as performances which once had some effect on an aversive stimulus but which continue to occur because the individual fails to "test reality" by failing to emit the avoidance behavior. Avoidance behavior is often so strongly maintained in experimental situations, particularly with higher organisms, that the avoidance response occurs successfully for hours at a stretch without the occurrence of the aversive stimulus (Sidman, 1955; Soloman and Wynne, 1953). The important experimental questions which need to be answered are the particular conditions which favor the development of these superstitious or accidentally reinforced avoidance responses.