My heads-up matches against DeepStack

Yesterday, Andrew Brokos and I had the pleasure of playing some of the first heads-up matches against DeepStack, a product of the Computer Poker Research Group at the University of Alberta.

While I don't have time to write a full (or fully eloquent) summary of the experience, here is a quick summary of the bot's play.

The striking features of the AI are, to my meatbag mind, unified by the concept of _board coverage_. Whatever the runout, it seems that DeepStack can hit it. Good human players quite generally work to achieve board coverage preflop and, to some extent, on the flop, but DeepStack does this much more. It is the expert electronic analogue of the player who "can be holding any two cards."

More specifically:

(1) DeepStack makes some very light calls preflop, both in and out of position. So, for example, it calls at least sometimes with J2s in the big blind against a 3x raise, and it calls with weak suited hands (say, K7s) against pos-sized or larger three-bets (again, at least sometimes).

The connection here to board coverage should be obvious; one clear effect here is that DeepStack holds more threes and deuces out of position than most human opponents do.

(2) DeepStack does not overbet as much as I expected it to, or at much as I see expert players do in the training videos I watch. That said, it chooses full-pot bets instead of half-pot bets perhaps more often than most humans.

Again, there is a clear connection to board coverage. The tendency not to overbet is intelligible in terms of an expectation that both players will cover boards, because overbetting most commonly arises when a runout hits one player much better than the other. As board coverage improves, cards tend not to favor any opponent as much, and canonical overbetting spots won't arise as much.

One exception: We played a pot where DeepStack limped and I checked the BB with 92o. The flop came Ts9s5x, and I check-called a pot-sized bet. The turn was a ten and it went check-check. The river was an offsuit 5; I checked and DeepStack bet ~17k into 2400. (I folded; if I recall correctly, DeepStack had J2, and neither of us held a spade.)

(3) DeepStack tends not to inflate the pot out of position, especially with deep stacks. It doesn't three-bet very aggressively preflop; it doesn't check-raise flops often; and it rarely "donks out" on the flop. (Again, I'm talking about deep-stacked play, but these features are striking even at 30 to 40 big blinds.)

One obvious lesson here is that position matters a lot; all serious poker players know this, but there's a lot of implicit disagreement about just how much "a lot" is.

Again, when I think about this, I return to the idea of board coverage; by keeping more hands in a calling (or check-calling, or checking) range, DeepStack covers more runouts--and, crucially, strengthens these ranges in order to protect the weak hands with which one must continue if one wants to cover all runouts. So, for example, DeepStack will often peel with hands like K7 and K6 on a board like Q83, especially to half-pot bets but even for larger bets. (Perhaps it doesn't do this with 100% of the combinations of these hands, but this is a striking feature of its play.) Keeping more strong top-pair (and better) hands in a check-call range--and, farther back, in a preflop calling range--surely helps DeepStack make these calls effectively.

(4) DeepStack makes more hero-folds and -checks than I expected, and somewhat fewer hero-calls, especially in big pots. Quite often, I thought one of my betting ranges was weaker than DeepStack did.

Please note that this is relative to my expectations. DeepStack is still far more aggressive than most human players. My expectations were formed in light of some surprising results from _Mathematics of Poker_, the loose play of certain experts, and so on. DeepStack still put me in plenty of tough spots with flop check-raises, for example.

(5) DeepStack avoids jam-fold strategies even with very short stacks. It often limps with fewer than 10 big blinds, and it even minraised (before calling a jam) with J6s when the effective stack was less than five big blinds. I imagine that there is not a very big difference in EV per hand between these strategies and optimal jam-fold strategies, but these situations are so common, especially in tournaments, that I expect players to continue to question the conventional wisdom about jamming with roughly 10 to 15 big blinds. (Because of these inquiries, it is perhaps no longer quite right even to call jam-fold play the "conventional wisdom" unqualifiedly.)

I hope to expand this post as I have more time to expand on these thoughts and to study the hand histories from the matches. For now, please excuse any errors and infelicities.

UPDATE: My co-host Andrew has written up his thoughts on the matches here.