Operant Conditioning: Videos & Practice Problems

Operant conditioning, developed by Edward Thorndike and B.F. Skinner, emphasizes how behavior is influenced by reinforcement and punishment. Reinforcement increases behavior likelihood, while punishment decreases it. Reinforcement can be positive (adding a stimulus) or negative (removing a stimulus). Schedules of reinforcement, such as fixed and variable intervals or ratios, affect learning rates. Primary reinforcers satisfy biological needs, while secondary reinforcers, like money, are learned associations. Understanding these concepts is crucial for behavior modification and effective learning strategies.

Contributions of Thorndike and Skinner

Alright. So to kick off our section on operant conditioning, we're going to go over the contributions of two researchers, Edward Thorndike and B. F. Skinner. So we're going to begin with Edward Thorndike.

Edward Thorndike coined the term law of effect, which is a very simple law but it really holds true for both human and animal behavior. This law essentially stated that behaviors that are followed by a reward are going to be repeated, whereas behaviors that are followed by a punishment usually aren't. We do have "reward" and "punishment" in quotation marks there because those are not the terms he originally used, but that is essentially what he was talking about. Thorndike discovered the law of effect by studying cats in puzzle boxes. So the way that his experimental procedures would work, he would have the cat in this little puzzle box and there would be food available to them kind of on the outside of the box.

When the cats were just doing random behaviors, like, for example, pawing at the corner of the box, they would get no reward for that. However, if the cat did a specific behavior like pulling a lever, that would open a door, and then the cat could access a reward, usually food. What he noticed is that once cats had done that one time, that behavior tended to be repeated. So the cats would be pulling the lever over and over, and they were much less likely to do things like randomly pawing at the box. Again, he called this the law of effect.

Now, B. F. Skinner was really interested in Thorndike's research and he expanded on it quite a bit. So B. F. Skinner was very interested in how behavior operates on the environment. Essentially, the fact that our behavior has an effect on our environment and the environment can respond with either reinforcement or punishment. Skinner went on to modify Thorndike's puzzle box and create what he called the operant conditioning chamber, which we often just call a Skinner box. A standard Skinner box would look something like this.

It would usually have a pigeon or a rat inside, and there would be a little lever where the animal could press. There would be a food dispensing tray to reinforce or reward the animal. Quite often a little green light would go on if the animal was being rewarded. There was also a red light to signify punishment. Sometimes there would be a speaker in the box, and that speaker could play a very loud unpleasant tone to act as a punishment, and often the floor would be equipped with the capability to be electrified, which could also punish the animal.

Obviously, a very mild electric shock, nothing that would actually hurt the animal, and Skinner did hundreds of experiments. He was very prolific, during his time, and he went on to coin the term operant conditioning. Operant conditioning is essentially a learning model in which the consequences of an organism's actions determine how likely they are to repeat that behavior again. What's really different here from classical conditioning is the fact that we're interested essentially in the consequences of an organism's behavior. Now, the organism's behavior is actually going to have an impact on the environment and the environment is going to be responding in one way or another either with reinforcement or punishment.

And we're going to be digging into all of these concepts and all of these terms much more in our upcoming videos. So I will see you guys there. Bye-bye.

Example 1

Alright. So for this one, we're going to be matching the psychologist with the experiment that they performed or the innovation that they pioneered. Our first one reads placed cats in a puzzle box and rewarded them with treats if they performed specific actions. The researcher here best known for his work with cats was Edward Thorndike. Edward Thorndike pioneered the use of these puzzle boxes and predominantly conducted research using cats to understand how rewarding or punishing a behavior would change its frequency.

Next up, we have someone who created an operant conditioning chamber, and we know that B. F. Skinner is the person who coined the term operant conditioning. This operant conditioning chamber, also known as a Skinner box, was essentially a very refined and updated version of Thorndike's puzzle boxes where they would have usually multiple different ways of reinforcing or punishing the animal. He was best known for his work using rats and pigeons.

Finally, we have Ivan Pavlov, and our answer is probably going to be option C. Ivan Pavlov studied the salivation of dogs in response to the expectation of getting food while Pavlov's peers Skinner and Thorndike were interested in operant conditioning, Ivan Pavlov was focused on classical conditioning.

There you have it, and I will see you in the next one. Bye-bye.

The Law of Effect states:

• A. The anticipation of a stimulus is sufficient to illicit a response.
• B. Responses can be generalized to gain rewards.
• C. Behaviors that are rewarded are repeated, behaviors that are not punished are not.
• D. A behavior that has been extinguished can spontaneously reappear after a few weeks.

Answer: (C)

Operant Conditioning

Like we talked about with our last video, with operant conditioning, we're really focusing on two environmental consequences: reinforcement and punishment. We're going to dig into those terms a bit more in this video here, and we're going to begin with reinforcement. So reinforcement is essentially just any event or stimulus that increases the likelihood of a behavior. This is something that strengthens the response and makes it more likely to occur again in the future. For example, if our little rat in our Skinner box there pushes the lever just like we want it to, we might reinforce that lever pushing by giving the rat some food, and that would make that response more likely to occur again in the future.

Now, punishment is basically the exact opposite. This is going to be any event or stimulus that decreases the likelihood of a behavior happening again. This is something that essentially weakens the response and makes it less likely to occur again in the future. So, if the rat were to perform a behavior that we don't want to see, we might punish that behavior by playing a very loud unpleasant tone over the speaker, for example, hopefully making that behavior less likely to occur again in the future. And we are going to go over these in a bit more detail in a future video and talk about some different types of reinforcement and punishment as well.

Now, one thing to note, and what Skinner observed and more modern researchers have confirmed, is that punishment can actually have unintended consequences, which usually look like fear or aggression in the organism. This is true in both humans and animals. So, generally speaking, behaviorists, including Skinner, recommend using reinforcement because reinforcement generally leads to more desirable outcomes. There are absolutely situations where punishment can lead to effective learning, and we don't see those negative consequences, but, generally speaking, for both humans and animals, reinforcement tends to be ideal if we can use that instead. Something else that Skinner, as well as more modern researchers, have noted is that we can modify reinforcement and punishment, and those modifications can influence how efficacious they are and can also influence the learning that takes place, for example, the speed of the learning.

So, we're going to go over three really common types of modifications, and all of these are actually fairly intuitive. Our first one is just the type of reinforcement or punishment that we use, and we always want to make sure that it is appropriate for the organism. And what I mean by that is, you know, just imagine a dolphin is probably going to be very reinforced by a bucket of raw fish whereas I, a human being, am not going to be particularly reinforced by a bucket of raw fish. Right? So we just want to make sure that the type of reinforcement or punishment we use is appropriate for that animal.

Next up, there's been a lot of research looking at time, and generally speaking, what we find is that we want consequences, either reinforcement or punishment, to follow the behavior as soon as possible. So we want to be reinforcing or punishing as closely to that behavior as we can to make sure that the animal is associating that environmental consequence with that behavior. And then finally, there has been quite a bit of research done on consistency and we have found pretty reliably that different patterns of reinforcement or punishment can actually impact learning in a variety of ways. And we're actually going to get into that in an upcoming video, so I will see you there. Bye bye.

Example 2

Alright. For all the statements here, we're going to be writing an 'r' if it applies to reinforcement, a 'p' if it applies to punishment, and a 'b' if it applies to both. So our first one reads that it weakens a response, and that would be a punishment. Right? We're trying to diminish that response, make it less likely to occur.

'B' is can be pleasant or unpleasant, and technically we want to say both. Right? We don't want to necessarily assume objective pleasantness in either direction. We want to just be thinking about them in terms of do they make a behavior more likely or less likely. 'C' is increases the likelihood of a behavior happening again.

If we are increasing that likelihood, it is reinforcement. 'D' happens in response to an action. That would be both of them. These are both types of environmental consequences. And then finally decreases the likelihood of a behavior happening again.

That is going to be punishment if we are decreasing that behavior. Alright, and there you guys have it, and I will see you guys in the next one. Bye bye.

In operant conditioning, modifications to reinforcement and punishment can influence behavior. Which of the following modifications to reinforcement would increase the likelihood of a behavior occurring again?
I. Reinforcing the behavior as quickly as possible after it occurs.
II. Providing a reward that is appropriate and desirable for the organism.
III. Punishing a desired behavior consistently.

• A. I & II.
• B. II & III.
• C. I & III
• D. I, II, & III.

Answer: (A)

Reinforcement Schedules

This video, we're going to be talking about reinforcement schedules. So unlike in classical conditioning where learning is essentially just based on repetition, something happening over and over again, research has shown that operant conditioning can change based on the patterns of reinforcement. We're going to go over all those different types of patterns in this video. Now, I do want to give you a heads-up this is going to be a very terminology-heavy video. There are a lot of different terms to learn.

However, the terms are all quite intuitive, and once you know what they mean, all of these are going to be pretty easy to understand, but it can be a bit tricky at first. So definitely take your time with this one, kind of slow it down if you need to, and I promise it will totally make sense in the end. Alright. So our first type of reinforcement is called interval reinforcement, and interval reinforcement is essentially dependent on the length of time that elapses between reinforcement. So this is completely dependent on time.

It is not dependent on the animal's behavior or responses in any way. And we have two types of interval reinforcement. Our fixed interval schedule or fixed interval reinforcement essentially means that behavior is going to be reinforced at a fixed or a set time. So, for example, a rat might get a treat every five minutes. It does not matter how many times in that five-minute span the rat pushes that little lever.

The behavior does not matter. It's only going to come after five minutes. Humans also experience this if you've ever gotten a paycheck. You probably get a paycheck once a week or twice a month or whatever it is, and it's not super dependent on your behavior, right?

Maybe one week you kind of slack off a little bit. Maybe one week you take a day off. Maybe one week you work super hard, but that paycheck is going to come at the same time no matter what your behavior looked like. So that is fixed interval. Now we can also have variable interval, and this would be when behavior is going to be reinforced at a variable amount of time since the last reinforcement.

So again, these terms are pretty intuitive once you can kind of get the hang of them. So essentially, this could be, you know, a rat getting a treat about every five minutes, maybe it comes every four minutes, the next time it comes at six minutes. The idea is that the animal cannot quite anticipate exactly when it's going to come.

That time is now variable. This is also kind of like fishing. Obviously, there is some skill and behavior involved in fishing. I don't want to imply that there isn't, but there are plenty of days where you, you know, put your line in the water and some days you get a response after one minute, some days you get a response after five minutes, and it does not depend on your behavior at all. So that would be a variable interval schedule.

Now, moving away from interval reinforcement, we can also have something called ratio reinforcement, and ratio reinforcement is more dependent on the ratio of responses to reinforcement. So now, responses and behavior are what matters. So with interval reinforcement, time is important. With ratio reinforcement, the response or the behavior is more important, and again we have two types fixed and variable. So with a fixed ratio schedule, behavior is going to be reinforced after a fixed number of repetitions.

So, for example, the rat might get a treat every five responses. Or this is also how customer loyalty programs work. So if you've ever been part of one of these, maybe you have to buy, you know, 10 coffees and then you get a free coffee, right? But this is entirely dependent on your behaviors and how many behaviors you do. It doesn't matter if it takes you a week to buy 10 coffees.

It could take you a month to buy 10 coffees, right? The time doesn't matter, just your behavior. So that's fixed ratio. Now with variable ratio schedules, you probably know what's coming here, but behavior is going to be reinforced based on a variable number of responses or repetitions. So now that rat is going to get a treat about every five responses, could come after three and then it could come after seven.

The idea again is that the animal cannot quite predict it. That is going to be a variable, responding rate. This is also how slot machines work. So you can imagine, people just keep on pulling those levers, right, and you might get a jackpot after a hundred lever pulls. It can take a really long time, but you know that that reinforcement is coming if you just do enough of the behavior.

That's kind of the idea here. So that is variable ratio schedules or variable ratio reinforcement. So to summarize this, when we're thinking about our reinforcement schedules, quite often you will see them depicted on a graph like this where we're looking at time on the x-axis and the number of responses on the y-axis. And this is not going to be real data, but the data usually comes out looking kind of like this where what we see is that ratio schedules, both variable ratio and fixed ratio, usually lead to the fastest learning and the most responses. Okay?

So and this makes sense, right? Because with these ratio schedules, behavior is what matters. Responding is the most important thing here, and so you're going to have both humans and animals learning very quickly to do more of the behaviors in order to get more of the rewards. So again, ratio schedules usually lead to faster learning and higher response rates. With interval schedules, we usually see a little bit slower learning and less responding overall.

And this totally makes sense, right, because, again, with these interval schedules, the only thing that matters is time. These are not dependent on the animal's behavior at all. So why would you have a high response rate if your responses don't actually matter? Right? There's no point in pushing the lever a thousand times if that reward is just coming after five minutes if you push it once or push it a hundred times.

It doesn't matter. Right? Now one more kind of specific thing to note is that, like we said, the ratio schedules lead to kind of the fastest learning and the most responding, and out of those variable ratio schedules tend to be the absolute most effective. So they lead to the fastest learning, the highest response rates, and they are the most resistant to extinction. And again, think of those slot machines just to kind of keep that in your mind to help you remember that these are kind of the most resistant to extinction.

We've all seen that stereotype of a person who's gambling or using slot machines and they will respond hundreds if not thousands of times just anticipating that reinforcement, right? So these schedules can lead to really high rates of response, and they're very hard to extinguish once they are learned. Alright. Great job with that one, guys, and I will see you in our next one. Bye bye.

Example 3

Alright. So for every one of these scenarios here, we're going to be determining what type of reinforcement schedule is being used. So, scenario a reads: a social media algorithm rewards a viewer with a fun video every forty-five to ninety seconds. We have two big clues here. First is that we're working with time, and if we're working with time, it's going to be some type of interval schedule.

And we can see here that the time is variable; it's not fixed, so the person cannot really predict exactly how many seconds are going to go by before they are going to get that fun video. So here we're working with a variable interval schedule. We're going to put a two here. Alright, scenario b reads: A video game gives a player an extra life after every 50 points that they earn.

So, here, this is no longer dependent on time. This is going to be dependent on that person's behavior. So this is going to be a type of ratio schedule. We can see here that it is fixed. It's happening exactly every 50 points.

It's going to be predictable for that person. So based on that, we're working with a fixed ratio schedule. We're going to put three. And then finally, scenario c reads: a boss gives a promotion after the completion of four to nine major projects. So, again, this is not dependent on time; it's dependent on behavior, so it's going to be a ratio schedule.

And we can see here that the amount is variable. Again, the person can't really predict when this promotion is going to happen. So that makes it variable. So we have a variable ratio schedule. So that is answer four. Alright, there you guys have it and I will see you in the next one.

Bye bye.

True or False: if false, choose the answer that best corrects the statement.
In ratio-based reinforcement schedules, rewards are given at regular time intervals.

• A. True.
• B. False, in ratio-based reinforcement schedules rewards are given based in the ratio of responses to reinforcement.
• C. False, in ratio-based reinforcement schedules rewards are given at irregular time intervals.
• D. False, in ratio-based reinforcement schedules rewards are given at regular time intervals.

Answer: (B)

Reinforcement & Punishment

So remember a few videos back I said that we're going to get into some more specific types of reinforcement and punishment? That is what we are doing here. So in operant conditioning, reinforcement and punishment can either be positive, which essentially means that we are adding something, or they can be negative, which means that we are removing something. Now, I do want to be really clear in this context, positive does not mean good and negative does not mean bad. In operant conditioning, positive just means that we are adding something to the environment and negative just means that we are removing something from the environment, and this essentially creates four different categories, two types of reinforcement and two types of punishment.

So we're going to go over those now. Our first type is positive reinforcement. So if it's positive, that means we're going to be adding a stimulus, and if it's reinforcement, that means that what we're trying to do is increase the likelihood of a behavior happening again. Right? So, for example, let's say a kid behaves and puts away their toys, and we want to reinforce that, we could give the child a piece of candy.

So we are adding that candy into their environment—that would be positive reinforcement. Now we could also do negative reinforcement. So now if it's negative, that means we're going to be removing a stimulus. But again, this is still reinforcement, so the goal is to increase the likelihood of the behavior in the future. So again, if our kid behaves and puts away their toys, we could potentially take away their chores for the rest of the day.

They don't have to vacuum that afternoon. Right? So now we are removing something to make this prior behavior more likely to occur again in the future. To be clear, this one is a little weird. It doesn't happen as often in real life.

Positive reinforcement is much more common, but you do need to understand what the idea of negative reinforcement is. Alright. So now we're going to get into our two types of punishment. So we can have positive punishment, and again if something is positive that means we are adding a stimulus. But now if it's punishment, that means the goal is to decrease the likelihood of a behavior.

So let's say the kid misbehaves and paints all over the walls. We don't want that, right? And so we could give the child a time out. So we are adding that time out into their environment, to decrease the likelihood of this behavior. And we can also do negative punishment.

So now if it's negative, we're going to be removing a stimulus. And again, with punishment, the goal is to decrease the likelihood of that behavior. So if the kid misbehaves, we could potentially take away their favorite toy for the rest of the day. So we're removing that stimulus from their environment again to decrease the odds of this behavior happening. Now, I do want to be clear this is not parenting advice.

Some of these can work on kids, but kids are obviously much more cognitively and emotionally complex than pigeons and rats, so we want to keep that in mind at all times. And I know that these can be a little bit confusing at first, so we are going to practice all of these again in our example. So I will see you guys there. Bye bye.

Example 4

Alright. So Lana is trying to help her son do better in school as she decides to try operant conditioning. In the boxes here, we're going to write out some examples of positive reinforcement, positive punishment, negative reinforcement, and negative punishment that she could potentially use with her son. I have written out my answers a little bit just so that you're not watching me write for the next five minutes. Alright, so we're going to begin with reinforcement and, what's important to think about is that with reinforcement, we want to focus on the behavior that we want to see again in the future.

So that would be, for example, studying here. So with positive reinforcement, we're going to be adding something to try and increase the odds of this behavior happening. So we could, for example, give him ice cream when he studies. I know that would be very reinforcing for me. Love me some strawberry ice cream.

Alright, and then with negative reinforcement, we're going to be taking something away or removing something again when he studies and does that desirable behavior. So maybe we take away his chores or we take away his least favorite chore to reinforce this idea of him studying. Now with punishment, this is going to be focusing around the behavior that we don't want to see and that we don't want repeated in the future. So essentially him, you know, failing to study. So again with positive punishment, we're going to be adding something.

So let's say we give him some extra chores if he doesn't study, and then with negative punishment, we're going to be taking something away. So let's say we take away his phone if he doesn't study, hopefully making that behavior less likely to occur or to occur again in the future. Alright. So there you have it, and I will see you guys in the next one. Bye bye.

Jerome completes a very demanding project at work, and his boss rewards him with a modest bonus on his next paycheck. This is an example of...

• A. Positive reinforcement.
• B. Negative reinforcement.
• C. Positive punishment.
• D. Negative punishment.

Answer: (A)

In a laboratory experiment, mice who fail to pull a lever when cued are given a mild electric shock. This is an example of...

• A. Positive reinforcement.
• B. Negative reinforcement.
• C. Positive punishment.
• D. Negative punishment.

Answer: (C)

Primary and Secondary Reinforcement

So in this video, we're going to be talking about primary and secondary reinforcement. Both reinforcers and punishers are essentially built on biological needs, but we classify them into two types based on how close they are to our biological needs. We can have primary reinforcement or secondary reinforcement. Primary reinforcement is based entirely on our biological needs or things that we need to survive and thrive. For example, food, shelter, avoiding pain, gaining comfort, and social interaction all serve as primary reinforcers.

Now, secondary reinforcers are things that are associated with primary reinforcement through classical conditioning; a very common example of this would be money. Other examples include grades, stickers, or praise. By themselves, none of these things satisfy biological needs, but they can through association. If you have money, you can afford shelter and buy food and water.

Even something like grades can lead to higher educational attainment, higher occupational attainment, more money, and thus more food and shelter. So, secondary reinforcers are essentially things that are associated with primary reinforcers.

Now, as you may have already guessed, primary reinforcement tends to be more commonly used in animal research. Obviously, it could be very unethical to use primary reinforcement when we're doing research on humans, and even unethical with animals in some situations. Studies that use primary reinforcement, even with animals, are very closely monitored to make sure that they're being done in an ethical way, and research done with humans usually involves using secondary reinforcement. Typically, if you participate in research, you would be getting either money for your time or class credit.

So that is primary and secondary reinforcement, and I will see you guys in the next one. Bye bye.

Example 5

Alright. So primary reinforcers and punishers are very powerful in modifying behavior because of their link to basic biological needs. So what would be some examples of ethical concerns that may come up when using primary punishers in research settings? I think most of us can agree that it would be pretty unethical to inflict pain on humans. So with rats or pigeons in a Skinner box, for example, we might do something like deliver a very mild electric shock to them as a punishment, but I think most of us can agree that it's fairly unethical to do that to a human.

And when it comes to animals, you might also feel like that is unethical as well, and that's completely valid to feel. When it comes to doing animal research, it might be considered unethical to deny animals access to food and water, especially for prolonged periods of time. So anytime we are using primary reinforcement or punishment in research, with animals or the rare cases where we might do it with humans, it's very important that we are closely monitoring these studies to make sure that they are being done in ethical ways. Alright, there you go, and I will see you in the next one. Bye bye.

Which of the following is NOT a primary reinforcer?

• A. Money.
• B. Shelter.
• C. Water.
• D. Food.

Answer: (A)

Here’s what students ask on this topic:

What is operant conditioning and how does it differ from classical conditioning?

Operant conditioning, developed by B.F. Skinner, is a learning model where the consequences of an organism's actions determine the likelihood of repeating that behavior. It focuses on reinforcement (increasing behavior likelihood) and punishment (decreasing behavior likelihood). In contrast, classical conditioning, pioneered by Ivan Pavlov, involves learning through association, where a neutral stimulus becomes associated with a significant stimulus, eliciting a conditioned response. The key difference is that operant conditioning is about the consequences of behavior affecting future actions, while classical conditioning is about forming associations between stimuli. Understanding these distinctions is crucial for applying effective learning and behavior modification strategies.

What are the different types of reinforcement and punishment in operant conditioning?

In operant conditioning, reinforcement and punishment can be positive or negative. Positive reinforcement involves adding a stimulus to increase behavior likelihood, like giving a child candy for tidying up. Negative reinforcement involves removing a stimulus to increase behavior likelihood, such as taking away chores for good behavior. Positive punishment adds a stimulus to decrease behavior likelihood, like giving a timeout for misbehavior. Negative punishment removes a stimulus to decrease behavior likelihood, such as taking away a toy for bad behavior. These concepts help in understanding how to effectively modify behavior through reinforcement and punishment strategies.

What are reinforcement schedules and how do they affect learning?

Reinforcement schedules in operant conditioning determine how and when a behavior is reinforced, affecting learning rates. There are two main types: interval and ratio schedules. Interval schedules depend on time, with fixed intervals providing reinforcement after a set time and variable intervals offering reinforcement at unpredictable times. Ratio schedules depend on behavior, with fixed ratios reinforcing after a set number of responses and variable ratios reinforcing after an unpredictable number of responses. Ratio schedules, especially variable ratios, typically lead to faster learning and higher response rates, as they encourage more frequent behavior to obtain reinforcement.

What is the difference between primary and secondary reinforcement?

Primary reinforcement involves stimuli that satisfy biological needs, such as food, water, and shelter, which are inherently rewarding. Secondary reinforcement, on the other hand, involves stimuli that have become rewarding through association with primary reinforcers. Examples include money, grades, and praise. While primary reinforcers are directly linked to survival and well-being, secondary reinforcers gain their value through learned associations, often via classical conditioning. Understanding these distinctions is important for designing effective reinforcement strategies in both educational and behavioral contexts.

How did B.F. Skinner contribute to the development of operant conditioning?

B.F. Skinner significantly advanced operant conditioning by building on Edward Thorndike's law of effect. Skinner introduced the concept of the operant conditioning chamber, or Skinner box, to study behavior in controlled environments. He emphasized the role of reinforcement and punishment in shaping behavior, coining the term 'operant conditioning.' Skinner's work highlighted how behavior operates on the environment, with consequences influencing future actions. His research on reinforcement schedules and the effects of different types of reinforcement and punishment has been foundational in understanding learning processes and behavior modification.