Semantic Adversarial Robustness with Differentiable Ray-Tracing

Reviewer 1

I think this is a really neat research direction which deserves further pursuit. It is interesting to examine how much a learned model can be tricked by only changing simple scene parameters, and I think it is smart to employ a differentiable renderer to do this. “ “-> I would like to see more information about the differentiable renderer used, and its settings. I would also be interesting to see how experiments vary with different rendering models. -> for line 123, would 20% not be .2 instead of .02? -> I feel the experiment section is lacking intuition for what you are trying to demonstrate, and is too stacked with details . I would prefer an intuitive explanation of what augmented and robust are , and what the experiment seeks to show about them.

-> I feel like this paper would be more interesting from a ““look how much we can fool the CNN using a diff renderer”” perspective , then a ““look how much we can help CNN be robust to fooling using a diff renderer””.

Reviewer 2

The paper introduces a 3D dataset for adversarial training, and uses the dataset to evaluate adversarial training strategies. Using differentiable rendering to find adversarial example is not a new idea, but I believe the experiments done here is new. The grid search experiment is interesting and shows a limitation of gradient-based adversarial training in low-dimensional cases.

Given the grid search result, I wonder if gradient-based adversarial training not that useful in low-dimensional space. It would be interesting to dig into this deeper. For example, if you do this in higher-dimensional space (e.g., with vertex positions or/and texture color), how will PGD compare to a random sampling of neighbor?

typo around L131/132: I think you want to say the ““error”” is slightly higher, not the performance.”\

Reviewer 3

The paper tackles an important problem of making deep learning models robust to semantic adversarial perturbations. Semantic perturbations correspond to modifying physical scene properties such as object pose, illumination, camera locations etc. that generated the image. In this paper, the authors proposed using a combination of local grid search and projected gradient descent to generate strong adversaries in 3d scene parameter space. They use these generated adversarial examples to learn more robust models. They show results on 3d traffic sign dataset.

The problem of perturbing 3d scene parameters to generate adversary is not new. In fact, they have missed very important related paper, “Beyond pixel norm-balls: parametric adversaries using an analytically differentiable renderer” Hsueh-Ti Derek Liu, Michael Tao, Chun-Liang Li, Derek Nowrouzezahrai, Alec Jacobson; published at ICLR 2019.

Overall, the presented work is a simple application of prior work on 3d traffic sign dataset.

Experiment setup is very simplistic. The 3d sign dataset does not look realistic. It would be good to show the effect of varying other interesting scene parameters such as the effect of changing light in the scene etc. on more challenging scenes.

Reviewer 4

This paper describes a method to improve robustness for semantic adversarial attacks using differentiable edge sampling in rendering. It is unclear to me what the main contribution of this paper is, as the proposed approach looks identical to [11] Li et al., 2018 (Section 5.4). Moreover, the scope is fairly limited as the authors only experiment on a simple traffic sign dataset, where few degrees of freedom are available (camera & light position only). The small dataset proposed could be useful but its simplicity may hinder its practicality and further adoption as a “real world” benchmark. I would recommend moving to a more advanced differentiable renderer, such as Mitsuba 2, to expand the space of parameters. Better lighting models (e.g. envmaps) and non-Lambertian, physics-based material models for the traffic sign (e.g. microfacets) could potentially enables better robustness. Moreover, directly perturbing the geometry seems like something the authors need to do, along with showcasing actual adversarial examples. Finally, I recommend to look at “Beyond Pixel Norm-Balls: Parametric Adversaries using an Analytically Differentiable Renderer” by Liu et al. (ICLR 2019).