Cooperation, solution concepts and long-term dynamics in the iterated prisoner's dilemma

Solution concepts help designing co-evolutionary algorithms by interfacing search mechanisms and problems. This work analyses co-evolutionary dynamics by coupling the notion of solution concept with a Markov chain model of coevolution. It is shown that once stationarity has been reached by the Markov chain, and given a particular solution concept of interest, the dynamics can be seen as a Bernoulli process describing how the algorithm visits solution and non-solution sets. A particular analysis is presented using the Iterated Prisoner's Dilemma. By numerically computing the Markov chain transition matrices and stationary distributions, a complex and strong relation between variation and selection is observed.