Saturday, July 19, 2008

Mental Associations v. Reduction of Tension II

Another article about behavioral science, and some of its flaws and inconsistencies.

Do Dogs Learn Via Mental Associations? Part II
Let me start by saying I have no problem with the idea that certain neurons in a dog’s brain are capable of forming dendritic trees with other neurons when they’re activated at the same time. Hebb’s Law sums it up: “Cells that fire together, wire together.” This energy exchange between neurons ultimately creates an actual physical, biological pathway in the brain creating a direct connection between what might otherwise be disparate, unrelated stimuli. But it starts as an energy exchange.

I also don’t have a problem with the idea that some of a dog’s memories are stored in the hippocampus, one of the “memory” centers in the dog and human brain. And if the paired stimuli also create a change in the dog’s hedonic state at the time they’re perceived or experienced, that connection is strengthened by the “pleasure and pain circuits” in the amygdalla. But one has to realize that these connections all take place in a primitive part of the brain called the basal ganglia, located near the brain stem, and are exhibited to some extent by all vertebrates, not just dogs and humans.

My point is that when some trainers say that dogs make a mental association between the word sit, the behavior itself, and the resultant reward, it all depends on what they mean by “mental association.” The common implication is that there’s an actual conscious thought process involved—which would have to take place in the frontal lobes, not just the basal ganglia—and that’s simply not the case; there is no “If I sit when I hear the command then I’ll get a reward,” construct in the dog’s mind. And according to John Staddon, the head of Duke University’s behavioral science department, anytime you have an “if-then” construct you’re using hypothetical or at the very least propositional thinking, which requires the use of language.

It seems clear to me that when dogs make associations between cues (commands), actions (obedience behaviors) and the resulting conse- quences (reinforcers) it’s all happening on a totally unconscious level (perhaps entirely within the basal ganglia). And I don’t know about you, but when I hear the words “mental association,” it always implies that some form of conscious thinking of a higher order is taking place.

Learning in Rats, Flatworms, and Protozoans
The idea of learning to produce a specific behavior in response to a command (or “discriminate stimulus”) goes back to Skinner’s lab rats who learned to press a lever to get a food pellet whenever a light was flashing (as one example). But would any dog trainer truly believe that when a rat presses a lever he’s doing it because he’s thinking logically, conceptually, or through any kind of internal linguistic-based thought process? How about planaria? Experiments done with flatworms show that they can “learn” through operant conditioning to navigate a maze. Do flatworms learn this by inductive reasoning: “If I go this way I’ll get food?” Do they make mental associations? Clearly not.

We could stick with canines and ask if Pavlov’s dogs really “learned” to salivate on cue (when the bell rang) through some higher-level cognitive process which required logic and language, or even through some lower, rudimentary thought process, such as pattern recognition. I had a well-known “positive” trainer tell me recently that when Pavlov’s dogs heard the bell it signaled to them that “food was on the way,” never mind the fact that all they ever got in conjunction with the bell ringing was a little meat powder, sprayed in the mouths while their heads and bodies were held immobile by leather straps. The fact is Pavlov simply found a trip-wire for one of a dog’s unconscious reflexes, and Madison Avenue has been using that kind of tripwire to get consumers “salivating” ever since!

Even the stentor, a single-celled organism, is supposedly capable of learning to avoid slightly noxious substance called carmine through a kind of trial and error method. This initially happens in such a way that it’s obviously a biochemical reaction; this is a protozoan, after all. (Its behavior was first observed and recorded in 1906 by H. S. Jennings.)

How a Protozoan “Learns”
When the carmine grains are introduced into the stentor’s environment, the first response is no response at all. The organism doesn’t react. Then, after a bit, the organism’s structure sort of bends away from the source of the irritant. That’s obviously a matter of biochemistry. When that doesn’t work, the stentor begins to reverse the motion of its ciliae, moving the substance away from its body. And if that doesn’t get results, the stentor contracts itself into its own feeding tube, interrupting its current “meal.” That trick usually solves the problem if none of the others haven’t.

So?

So this is a protozoa, and we wouldn’t naturally expect a single-cell organism to be changed by its experience, would we? It can’t really learn which of these behaviors—the contractions—was ultimately the most effective, can it? And yet let a little time pass, introduce the irritating substance again, and the little guy doesn’t go back to square one—ignoring the substance—or step two—bending away from it—or step three—reversing the movement of its ciliae. No, it goes straight to step four and begins contracting in on itself, doing so quite violently now, as if it had learned from its previous experience that the more violent the contractions are, the quicker and better the results. And it does this even though that particular behavioral pattern prevents it from being able to feed.

Would we for a second consider the idea that the stentor made a positive mental association with the one behavior that worked to relieve its irritation the first time, and that that’s why it immediately went straight to that behavior when the substance was re-introduced? Of course not; it’s a single-celled organism. If there was any sort of change that took place in its internal structure it has nothing to do with memory (not as we know it), or any sort of thought process. It’s got to be the result of a simple biochemical transformation of some sort. In fact, on a very simple level it’s solely about reducing tension. The carmine creates stress on the stentor’s simple biological system. Each of its behaviors are simply reactions to the stress and are designed to get rid of it and return to homeostasis. The “learning” that takes place is, again, due purely to a temporary chemical transformation*.

And how do single cells undergo chemical transformation? Through an energy exchange. In fact, as I stated up top: whenever a human or a dog learns something new, a more complicated and sophisticated form of energy exchanges takes place: one neuron sends energy to another, which sends that energy to another, and on up the line. And the main reason it’s more complicated is that there are more cells involved. On a cellular level, it’s still pretty simple.

So anyway, back to dogs…

Theory of Mind
I think one problem in applying some of these lessons (the stentor, the rat in the box, the planarian in the maze, and Pavlov’s bell) to dog training has to do with the way the dog’s closest biological relative hunts its prey: In some instances one or more wolves will chase the prey animal toward a spot where they know other members of their unit are waiting to leap out and pounce. A you watch this take place it sure seems like a carefully thought-out, deliberate, and well-orchestrated plan. The mere fact that some of the wolves are keeping themselves hidden suggests that they have a rudimentary theory of mind; one that involves the ability to know and understand the logical proposition that

a) I am able to see things because I have eyes,

b) other animals with eyes must have the same ability to see things, ergo

c) if I can prevent my prey from seeing me, I’ll catch him by surprise...

When we watch footage of wolves hunting it seems almost incontrovertible that wolves must have this first level ToM. (There are three levels: sensory, emotional, and intellectual.) But does a dog or wolf even know that he has eyes? How could he? He may have a vague sense of where his field of vision is centered—somewhere above his nose and away from his hindquarters, etc.—but they don’t have the ability to understand how they look to themselves, let alone what “look” or “themselves” mean. So how could they know what another animal's perceptions of them are, or that other animals have perceptions?

In discussing this with a +R trainer once, she said she was convinced that when a dog brings a ball back and drops it at your feet he’s placing the ball where you can see it. So in her mind dogs definitely have a first level ToM. And on first glance this sounds quite plausible. But once you go back in time and look at the way the dog first learned to bring the ball back, you’ll see that the main criteria the dog has is to put the ball where you can reach it, not where you can see it.

I know of many dogs who’ll bring a ball back and when it rolls past me, or under the bench where I might be sitting at the dog run, they’ll bark in frustration that I’m not throwing it fast enough. I have to teach them to bring the ball back to a spot that’s close enough for me to reach down and grab it. That spot will be in front of me and not under the bench, etc. If these dogs had a rudimentary ToM they would already understand naturally that they didn’t put the ball where I can see it (which would also most likely be where I could reach it). But they don’t.

If you ask me the most relevant bit of human anatomy in the dog’s experience of fetch (and a lot of other daily events) is your hands, not your eyes. Yes, the eyes are great for communicating emotion, but if a dog wants you to throw the ball he’d be pretty dumb to focus solely on your eyes. He sometimes needs you to pay attention to him, of course. That’s why the dogs I mentioned a moment ago start barking at the dog run. But knowing where your owner’s attention is focused is something separate and apart from knowing that your owner’s eyes are what provides him or her with visual input (or even what visual input is).

Speaking of attention, another proof some people give that dogs have a first-level ToM is the dog who’s been punished for making mistakes in the house. As a result whenever the urge comes on her she might develop the habit of finding a place to eliminate where her owner can’t “see her.” We’ve all known or heard of dogs who do this. It has to be ToM, right? She doesn’t want to be seen. Perhaps. But maybe the more relevant thing to the dog is that she can’t see you when she sneaks off. Or what’s even more probable, she’s simply trying to escape your attention. Attention and focus are things a dog can understand. The business of how visual input comes through these orbs in the face, then through neural pathways to the brain, etc., is way out of reach.

By the way, there are scientists who study this stuff. They set up very complex experiments to test whether chimps, for example, have this first level ToM. We’re not talking about the casual observations of misinformed dog owners. These brilliant men and women are doing cutting- edge work. And yet there’s currently no consensus that even chimps have this first-level ability. But if any animal on the planet would be expected to be able to, it would certainly be our closest living relative. (I know I’m leaving out dolphins and cetaceans, but I’m not sure science has found a way to run tests on them.)

If behavior and learning are all about making mental associations then it’s almost impossible to not see dogs as having a theory of mind. We, as dog lovers automatically make that leap. But if we see what’s really going on, and what’s really relevant to the dog’s experience—whether it’s getting his owner to play fetch, or finding a place to eliminate where his owner can’t “see him,” or even the wolf’s tactic of hiding from his prey—we might recognize that all of these behaviors are about reducing internal tension, not making mental thought processes.

Yes dogs and wolves are far more complex than stentor and flaworms, or even rats. But if you ask me, all of nature—all the changes that take place in the natural world, from plate tectonics to hurricanes to the way rivers flow, or the way seeds fall from a tree, absorb nutrients from the soil and then burst out of their casings and start to grow, reaching up toward the sunlight—all these phenomena are built around the very simple, energetic properties of tension and release. I think even the earth’s orbit around the sun is a constant dance between these two expressions of energy.

Wolves, Spiders, and Jellyfish
At any rate, let’s go back to the idea that wolves have to have a ToM to be able to exhibit this predatory behavior of “hiding” from their prey. But let’s go down the evolutionary ladder a little. There’s a type of spider who, when he’s hungry (if spiders get “hungry”), will find a hole, climb into it, cover himself with a leaf, and wait for an insect to come by. Then he’ll jump out, kill his prey, and eat it. Does the spider have a theory of mind? Does he “know” he’s invisible to that insect when he’s under that leaf? Does he think about all this? If he doesn’t need such cognitive abilities to perform these behaviors, then why do wolves and dogs need them?

“Because,” (those who see dogs as having some intellectual abilities would tell us), “the spider has a much smaller brain.”

There’s no arguing that point. So how about looking at a species of animal (in the broadest sense) that has no brain at all—the jellyfish—which is technically a type of plankton? The jellie has no brain and no sensory organs. And there is one species of jellie that actually hunts its prey in a fashion very similar to that used by wolves: one jellie will “chase” while another seemingly “hides,” or least circles around from the other side, “waiting” for the prey to be driven toward him.

And yet the jellyfish has no brain! No face, no eyes, no ears, no nose.

And no brain!

Granted their “strategy” isn’t anywhere near as complicated or nuanced as the one performed by a wolf pack. Jellyfish don’t even “swim,” really. They sort of use the ocean currents to help propel them in the direction they “want” to go. So it seems to me that when we look at such splendid underwater footage of a seeming concerted effort among two jellyfish, we have to wonder: “How the hell do they do that?”

Of course all of this is sort of beside the point I’m trying to make, which has more to do with how dogs learn to obey commands or to produce new behaviors or learn to let go of old ones than how spiders and jellyfish hunt, right? I mean do spiders even have the capacity to learn new behaviors or change old ones? Maybe so: (how some spiders learn)

What about jellyfish? Some species do tend to return to a place where they’ve successfully found food before. Does that constitute learning? If so, why don’t we say that spiders and jellyfish learn by making mental associations (I mean, besides the fact that they have no real mental capacities at all)?

Face Recognition Software
While a spider has eyes, and a tiny brain, his face, if you could call it that, is expressionless. Do spiders have emotions? It’s doubtful. Then there’s the jellyfish, who doesn’t even have eyes or a face (or a brain!) to give us any clues as to how he’s thinking or feeling. On the other hand, when we look at wolves hunting or see our dogs listening to our words we see their eyes, we see their facial expressions, we see the changes in their body language. They tilt their heads as if thinking, even though we might know logically that the head tilt is just a way a dog has of hearing us better. We also see gaps in the action. And when we see those gaps we focus on the changes, if any, taking place in the animal’s face, particularly its eyes.

Human infants do five basic, important things: eat, sleep, cry, eliminate, and look at faces. The first thing a toddler draws when given a crayon and some drawing paper is a face. Face recognition is one of the driving forces in human behavior. And so it seems to me that any time we see an animal with a face and eyes who stops for a moment in the middle of a concerted activity, our brains are both engineered and taught practically from birth to read some sort of thought process onto that momentary lacuna; we have to fill that gap. We can’t help ourselves; it’s an automatic reflex in us. After all, what is a dog doing at that moment if not thinking?

Meanwhile, if we see a gap in the action of two jellyfish using a similar hunting “strategy,” we think no such thing, partly because the jellyfish has no face. He has no body language, he has no emotions. So we see a gap in the action and think, “It’s just due to a change in the ocean current,” or “The jellie really wasn’t hunting at all; it was just an illusion.” But should the action resume, and if the jellies are then successful at hunting their prey by working in unison, we have to wonder: “How the hell do they do that?”

With dogs and wolves we’re sure we know—they’re thinking the whole thing through. They’re changing their strategy, or developing a new one on the fly. But they’re definitely thinking it all out the whole time, or so we believe.

Another part of the problem may go back to how evolutionary biologists have framed their explanations of the evolutionary process itself. Organisms are said to have adaptive and reproductive “strategies.” We don’t even question it when we hear the word strategy intoned dramatically in nature films: “The spider has a clever strategy for killing insects,” or “The call of the male meadowlark is part of his mating strategy.” Even certain viruses are supposed to have adaptive strategies. It’s like every living thing in nature is playing chess or planning to invade Poland on a regular basis! They’re all strategizing!

Still, when we see footage of a spider climbing into a hole and pulling a leaf over his head, waiting for an “unsuspecting” insect to come along, do we think he’s planned it all out in his head? Spiders do have brains, after all. Do we think Mr. Meadowlark wonders to himself which notes will best attract a female, and then choose them accordingly? No. We perceive those types of behaviors as being purely instinctual.

So why do we automatically impute thought processes onto animals?

Personally, I blame Charles Darwin and Walt Disney...

"Changing the World, One Dog at a Time"

*Not surprisingly, the stentor doesn’t retain this lesson much past four hours or so afterwards.

Monday, July 14, 2008

Something Doesn't Click

I got an e-mail yesterday from Summer's Mommy that I thought warranted a full post rather than a reply buried in the comments section. It's, once again, about some of the problems with behavioral theory.

Something Doesn't Click
Hi Lee. I didn't want to further hijack your thread on your blog.... :) But I still had some questions if you don't mind...

As I mentioned I have a good friend who is a hardcore +R clicker devotee in the Karen Pryor Academy, so she's access to all the theories etc that we've been discussing.

She talks about how +R clicker training proofs the behavior in the dog. First you teach in a no distraction zone, then as the dog showing 80% compliance (but OH they hate that word!), then you introduce distance and/or distraction, ad nauseum until the behavior is proofed.

In SchutzHund (and elsewhere) they talk about three levels of training:

Right. And here's my description of these three stages.

LEARNING – where a new behavior is introduced, and there are few or no distractions, and everything should be positive (no punishment, no force). I’ve always likened this level to “Here’s a game called sit…”

NARROWING – where nuances to the behavior are taught through not rewarding the dog unless the behavior is done exactly right (which +R trainers call “negative punishment,” meaning that what they do is not “all positive”). Narrowing could be anything from teaching the dog that she has to always sit in front of you, not to one side, that she has to face you directly, or that she has to have both front paws resting on the ground, etc. I call this level “Here are the rules of the game…”

PROOFING – where the dog has to obey no matter what else is going on, squirrels running up his backside, buckets of KFC strewn all around the park, etc. This is where corrections are allowed if necessary. And it’s at this level that the idea of “positive-training-only” tends to break down with some dogs. I call this stage: “The rules always apply.”

The problem I see with dominance trainers is they often start the force and punishment right at the 1st level. That’s not good because you don’t a dog’s first experience with any command to be a negative one. The problem with +R trainers is that their ideology prevents them from being really good at getting results at the 3rd level with some dogs.

Trust me, no +R trainer would’ve ever gotten to the same level you have with Summer…

My problem with that idea is that unless you are training ten 3-min sessions every day of the week, it's hard to get any improvement out of the dog. So I guess it's while they are proofing, +R trainers are inadvertently working in tandem with the dog's natural predatory motor pattern without knowing, as you put in your post?

No. Anytime you get the dog to produce a behavior which relates to a predatory motor pattern, or "modal-action pattern," it has its genesis in the hunting behaviors of wild wolves, you’re stimulating the prey drive, whether you’re aware of it or not. So it doesn’t matter if you’re at the final stage of proofing the stay, or teaching it right out of the box, it’s always related to the “eye-stalk” behavior.

Also there’s a bit of folklore in dog training circles that “active” behaviors like “heel” and coming when called can be taught with toys, while the dog is energized, but that “passive” behaviors like “down” and “stay” are better learned through food rewards, while the dog is calm. But as you probably know, it’s actually much easier for a dog to learn the “down,” and particularly the “stay,” when she’s been energized before the training starts.

It SOUNDS like a nice way of training, but do you think it's possible to also inadvertently dampen drive if say, your clicker timing sucks? 

Yes, but it’s not just the timing of the clicker that’s the problem. Remember, Karen Pryor doesn’t really know anything about dogs. She was a dolphin trainer, one who believes that all learning for all species is basically the same process. So she sees a ball throw or game of tug as no different, essentially, from a food reward; it’s just another form of positive reinforcement. But if dogs learn best through reduction of internal tension related to an unexpressed prey drive, then the +R model falls apart. Yes, being given an opportunity to chase and bite can be viewed as a positive reinforcement, but it’s not the same kind of reinforcement as a food reward.

However, what the clicker does well, at least in dog training, is it provides a means for isolating a specific moment in time when the dog does something right. And yes, that takes perfect timing, and yes some dogs can be driven nuts if the timing is not perfect. (There are seminars now on how to avoid hyper-anxiety in clicker-trained dogs.) And while it’s great for teaching dogs to do tricks, that’s not what dog training is really about. At least I don’t think so.

Another reason I dislike the clicker is that the “learning” happens too quickly. It doesn’t allow the dog to process the information through his instincts and emotions. It’s all run on nervous, not emotional, energy. The reason some dogs do well with it is that they have a more flexible emotional-response system, one that can accommodate any kind of training. But clicker training kind of forces the dog to do too much of the emotional work on his own.

Here’s what Bob Bailey has to say on the subject: 

I am not a clicker trainer. I have used a clicker, and quite successfully. Keller and Marian Breland were using clickers in 1943. In the modern use of the name clicker trainer, punishment, especially positive punishment, is "disallowed." I allow myself to use punishment if I believe it is necessary to accomplish the task and if the task merits the use of punishment.

I am not a fan of the "ever-clicking" approach to training. The proper application of the clicker is that akin to using a scalpel to make fine cuts. However, the increasing use of reinforcement to get behavior is good, so I guess the prevalence of sloppy "clicking" is a price paid for trainers thinking more about reinforcement rather than punishment. Most pet owners seldom have need for a clicker, in my opinion; a clicker can easily get in the way of getting good behavior.

Animal training, and animal behavior, has consumed a large part of my life. Clicker training has had little impact on what I did, or what I will do. [And it] has not taught me a whole bunch, other than that people can get wrapped up in fads and catch phrases. 

I agree with Bailey that the +R movement is good, in that it has reduced the amount of fear, force, and punishment commonly used in dog training. But there’s also been a backlash, one that Angela mentioned in the comments section of my last post: many dog owners are finding out that +R doesn’t work very well with their dogs.

“Or you practice things like avoidance JUST as a dog is displaying a high drive behavior that's inappropriate (like barking at rollerbladers), and because they're practicing avoidance, they're not providing appropriate drive release at the crucial moment?”

Of course they’re not providing drive release at critical moments, at least not intentionally. That’s because their focus is on using positive reinforcement. And “positive” in the Skinnerian argot doesn’t mean that “something good” reinforces the behavior, it simply means that something is added, like food or praise, not subtracted, like internal tension or blocked drive, etc., to reinforce the behavior. (It’s hard to remember this so just try to remember that “positive punishment“ means that pain or fear is “added” to achieve learning; there’s nothing positive/good about punishment)


And since +R trainers are so focused on the idea that food and clickers are not only “positive” but “good,” that it’s hard for them to see that reducing a dog’s internal tension by providing appropriate drive release (as you put it) - which is technically a form of negative reinforcement - is the real agent of learning. And anytime you mention negative reinforcement to these people their minds automatically go to the ear-pinch as a means of teaching a dog to drop what he’s retrieved. (I think there’s a lot of Orwellian brainwashing going on +R circles.)

I hope you don't me asking these to you. I am still trying to improve as a trainer...

As you can see, I don’t mind. These are important questions.

BTW, I've taken a few steps back and asking Summer for more HUPS on walks, whenever I see her ramp up her energy outwards, and it's helping her rebalance much quicker. Yesterday she freaked at a pittie across the street who went apeshit on her, I said, NO NOT YOUR JOB, and asked for a HUP instead, and even though she delivered the hup visibly (and audibly!) distressed, right after the HUP the distressed energy diffused and she was back to walking normally. The raised scruff was gone, no more whining. Amazing how that works... A block later when we approached a leashed dog walking towards us, I just said quietly, Not Your Job, and she snapped her head to look at me in a tight formal heeling pattern.

That’s great. And I’m not sure how your friend at the Karen Pryor Academy would interpret that. She’d probably say that Summer is making a positive mental association with the pit bull because of the rewarding feeling she gets from jumping up on you. (?)

Can't wait to show you her heeling - she's doing amazingly well. I don't think she'll ever have the precision of a SCH 3 titled dog, but I really think we will nail the CD Novice Title, and in the fall when it cools down, I'm going to be teaching her the Broad Jump. We started some preliminary tracking exercises too. I never imagined I would bring her to this level... Remember how spazzy she was?? She does a kick ass recall to Front and Finish too.

From what I hear she’s the brightest doggie star in Fort Greene Park!

"Changing the World, One Dog at a Time"