Not only size, but also shape counts: Argumentation solvers are benchmark-sensitive

We test different solvers dedicated to the solution of classical problems in Argumentation, as enumeration/existence of extensions, and sceptical/credulous acceptance of arguments. We handle a subset of the solvers tested in ICCMA15, and a superset of graphs used in the same competition. The goal is to provide considerations that can help future comparisons and competitions as ICCMA15. We offer a detailed report of this comparison from the point of view of different graphs, solvers, problems and timeouts. We show that the characteristics of graphs impact on the performance of solvers and on their final ranking. In addition, we extract other general considerations, e.g., reducing the computation timeout does not change the same ranking.