
ALT Schematic diagram showing examples of punctuated change across biological change. In the centre, four different empirical patterns of punctuated change (ecological dynamics, a fitness landscape, phenotypic space and genotype space, from top to bottom) with graph showing example data linked to each one: (top right) evolution of ecosystem organization in different modules during the Miocene; (top left) fitness change in a virus; (bottom left) evolution of E. coli cell size; (bottom right) clonal evolution in cancer cells.

ALT 8 Conclusion Sabre is the second defense to be accepted at IEEE S&P in as many years that had clear signs of a flawed evaluation which should have been easily identified by qualified reviewers. As we have shown, these evaluation flaws are caused by between one and three bugs in the original evaluation that can be fixed by modifying one to three lines of code. There is no doubt that defending against adversarial examples remains an interesting and important problem. And reviewers are attracted to papers making bold claims: it is much more tempting to accept a paper that claims near-perfect robust accuracy than a paper that claims a marginal improvement in accuracy. But it is of utmost importance that papers with errors such as this are identified during review, so as to not suggest that incorrectly evaluating defenses is tolerable. Where do we go from here? Adversarial machine learning is no longer a field that only studies toy problems to develop a better scientific understanding of th

























