Think about it this way: if the Jolly Roger was so effective at facilitating merchant ship surrender, then why didn't the legitimate belligerents, the other guys attacking merchant ships, also want to hoist the Jolly Roger? Because it could have facilitated easier surrender for them, too. My point is that they did want to. A pooling equilibrium was threatened but was largely prevented by what is called the single-crossing property in economics—the fact that it's more costly for the one type than for the other. And it was more costly for a privateer ship, which was legitimate, because if they raised the Jolly Roger, all of a sudden their status moves from legitimate ... to criminal—they could be caught and hanged. So that's an added cost for them. But pirates, since they were already outlaws, they had already incurred that cost.
In terms of establishing the pirate brand, you discuss cruelty as a means of achieving that notoriety.
That's exactly right. Again, if we think of piracy as a business, as I think we should, their reputation was just as important as it is for any other business. So in order to institutionalize the brand name that they wanted to cultivate, what they needed to do was first work diligently in creating it. The way that they did that was through ruthlessly adhering to this idea of torturing people if they didn't comply with them once they had boarded their ship. We normally think about pirates as sort of blood-lusting, that they want to slash somebody to pieces. [It's probably more likely that] a pirate, just like a normal person, would probably rather not have killed someone, but pirates knew that if that person resisted them and they didn't do something about it, their reputation and thus their brand name would be impaired. So you can imagine a pirate rather reluctantly engaging in this behavior as a way of preserving that reputation.
In fact, you point out that to be bloodthirsty would undercut the desired result, because it would signal to potential targets that they might as well resist.
The reason that cruelty was effective is because it constituted a cost of a behavior that pirates wanted to deter. If you're a merchant crew and you know that pirates, no matter what you do, are going to try and slaughter you once they board you, well then of course there's no cost to you resisting them. You might as well try; you'll probably lose, but you're no worse off than if you had just surrendered to them peacefully in the first place. So it was critical for pirates that they only applied heinous tortures when, in fact, they were using it to penalize behavior.
And this is part of this idea of what I call the "invisible hook." It's analogous, in some ways, to Adam Smith's invisible hand [a hypothesized force by which a free-market economy naturally benefits the greater good] in the sense that, of course, pirate prey are worse off as a result of pirates attacking them, but a profit-motivated pirate crew is likely to behave better toward the people they're attacking than one that in fact was truly sadistic and didn't care about money at all. And this is a case where we can see that.
Is that how you're able to separate cause from correlation? What's to say that these pirates weren't just, say, bands of bloodthirsty marauders that lucked into this strategy?
One of the things we can do is look at testable implications of the rational-choice theory. If in fact pirates were truly madmen, we would not expect them to only display that madness in particular cases. Especially, it would be a great coincidence if it happened to be those cases in which it stood to make them money. And that's pretty much precisely what we observe.
Notice the sort of piratical paradox, if you will, that we confront. You've got these depraved and feral sea bandits living somehow by a strict pirate's code, holding judicial sessions, and regulating alcohol use and gambling. The two things just don't seem to match up. First of all, that undermines the claim that pirates were simply crazy, because when it was in their interest, they seemed to be able to behave perfectly rationally. And the rational-choice framework can really allow us to resolve that piratical paradox, in that you can take one basic core assumption about human behavior and explain two things that seemingly are at odds with each other, as opposed to positing ad hoc pirate motives as we go from practice to practice.