AttackVLM: On Evaluating Adversarial Robustness of Large Vision-Language Models

Yunqing Zhao1*, Tianyu Pang2*†, Chao Du2†, Xiao Yang3, Chongxuan Li4,
Ngai-Man Cheung1†, Min Lin2
*Equal Contribution, †Corresponding authors
1Singapore University of Technology and Design
2Sea AI Lab, Singapore
3Tsinghua University     4Renmin University of China

Abstract

Visual vulnerability. Large vision-language models (VLMs) such as GPT-4 have achieved unprecedented performance in response generation, especially with visual inputs, enabling more creative and adaptable interaction than large language models such as ChatGPT. Nonetheless, multimodal generation exacerbates safety concerns, since adversaries may successfully evade the entire system by subtly manipulating the most vulnerable modality (e.g., vision).

Black-box transferability. To this end, we propose evaluating the robustness of open-source large VLMs in the most realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning the targeted responses. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP, and then transfer these adversarial examples to other VLMs such as MiniGPT-4, LLaVA, UniDiffuser, BLIP-2, and Img2Prompt. In addition, we observe that black-box queries on these VLMs can further improve the effectiveness of targeted evasion, resulting in a surprisingly high success rate for generating targeted responses.

Robustness before deployment. Overall, our investigation and findings in this work provide a quantitative understanding regarding the adversarial vulnerability of large VLMs and call for a more thorough examination of their potential security flaws before deployment in practice.

Overview of Proposed Method

AttackVLM method overview: transfer-based and query-based attacking strategies

Experiment Results

MiniGPT-4: conversations with clean and adversarial images

Effects persist across turns. Visual question-answering (VQA) task implemented by MiniGPT-4. MiniGPT-4 has capabilities for vision-language understanding and performs comparably to GPT-4 on tasks such as multi-round VQA by leveraging the knowledge of large LMs. We select images with refined details generated by Midjourney [51] and feed questions (e.g., Can you tell me what is the interesting point of this image?) into MiniGPT-4. As expected, MiniGPT-4 can return descriptions that are intuitively reasonable, and when we ask additional questions (e.g., But is this a common scene in the normal life?), MiniGPT-4 demonstrates the capacity for accurate multi-round conversation. Nevertheless, after being fed targeted adversarial images, MiniGPT-4 will return answers related to the targeted description (e.g., A robot is playing in the field). This adversarial effect can even affect multi-round conversations when we ask additional questions.

UniDiffuser: joint image-text generation with clean and adversarial images

Cross-modal propagation. Joint generation task implemented by UniDiffuser. There are generative VLMs such as UniDiffuser that model the joint distribution of image-text pairs and are capable of both image-to-text and text-to-image generation. Consequently, given an original text description (e.g., An oil painting of a bridge in rains. Monet Style), the text-to-image direction of UniDiffuser is used to generate the corresponding clean image, and its image-to-text direction can recover a text response (e.g., A painting of a bridge at night by Monet) similar to the original text description. Recovery between image and text modalities can be performed consistently on clean images. When a targeted adversarial perturbation is added to a clean image, however, the image-to-text direction of UniDiffuser will return a text (e.g., A small white dog sitting in the grass near a stream in Autumn) that semantically resembles the predefined targeted description (e.g., A small white dog sitting on the ground in autumn leaves), thereby affecting the subsequent chains of recovery processes.

BLIP-2: captions for clean and adversarial images

Targeted caption redirection. Image captioning task implemented by BLIP-2. Given an original text description, DALL-E/Midjourney/Stable Diffusion is used to generate corresponding clean images. Note that real images can also serve as clean images. BLIP-2 accurately returns captioning text (e.g., A field with yellow flowers and a sky full of clouds) that is analogous to the original text description / the content on the clean image. After the clean image is maliciously perturbed by targeted adversarial noise, the adversarial image can mislead BLIP-2 to return a caption (e.g., A cartoon drawn on the side of an old computer) that semantically resembles the predefined targeted response (e.g., A computer from the 90s in the style of vaporwave).

Quantitative Results

Image-image matching transfers better. In the six evaluated models, MF-ii achieves higher ensemble scores than MF-it. Starting from MF-ii and refining the image with black-box text feedback gives the highest target alignment among the reported strategies for every model.

Evaluation setting. Clean images come from ImageNet-1K validation; target descriptions are randomly selected MS-COCO captions. Victim models remain frozen, the input question stays fixed, and only the image is modified. The default constraint is ‖xadv − xcle‖∞ ≤ 8 on pixel values in [0, 255]. Setup details ↗

Response–target CLIP text similarity · Ensemble score ↑ · Unitless
Victim modelClean imageMF-itMF-iiMF-ii
+ MF-tt
BLIP0.4500.4830.7550.846
UniDiffuser0.4020.6560.7000.743
Img2Prompt0.4610.4700.4760.791
BLIP-20.4520.4830.5430.638
LLaVA0.3880.3970.4000.542
MiniGPT-40.4170.4430.5220.614

Source: Table 2, ensemble column. Values are reproduced from the paper. Higher scores mean that the model’s response is semantically closer to the predefined target text.

MF-it: image-text feature matching.   MF-ii: image-image feature matching.   MF-tt: query-based text-text matching.

Query feedback makes a difference. For example, Img2Prompt increases from 0.476 with MF-ii to 0.791 with MF-ii + MF-tt; LLaVA increases from 0.400 to 0.542. The comparison separates the benefit of transferable perturbations from the additional benefit of refining them against the victim’s returned text.

Further Analyses

Why Combine Transfer and Queries?

The two stages are complementary. The UniDiffuser ablation compares different allocations of a fixed perturbation budget. Combining transfer-based initialization with query-based refinement is more effective than relying on either stage alone in this experiment.

Original Figure 7: four CLIP evaluation panels comparing transfer/query perturbation-budget allocations on UniDiffuser
Original Fig. 7. Horizontal axis: transfer/query budget allocation; vertical axis: response–target CLIP text similarity (unitless, ↑). Panels use different CLIP text encoders. The dotted t+q=8 line uses full-budget transfer initialization followed by query refinement, with the final perturbation still constrained to ε = 8. Click the figure to enlarge.

Perturbation Budget and Image Quality

Larger perturbations bring diminishing returns. In this UniDiffuser example, increasing ε eventually gives captions with similar target semantics while moving the image further from the clean input. The paper uses ε = 8 as its default balance between the visual constraint and targeted generation.

Original Figure 6: increasing epsilon from 2 to 64 changes UniDiffuser captions toward a fish target while LPIPS distance rises
Original Fig. 6. ε is the pixel-wise ℓ∞ budget on the [0, 255] scale. LPIPS measures perceptual distance from the clean image (↓): 0.013, 0.019, 0.054, 0.116, and 0.158 at ε = 2, 4, 8, 16, and 64 in this example. Click the figure to enlarge.

What Does the Perturbation Reveal?

The target is hard to recognize from the perturbation alone. The full adversarial image redirects the generated caption toward the target. However, the isolated difference image and its UniDiffuser caption do not reveal the target concept in the illustrated example. The residual is amplified tenfold so that it can be inspected visually.

Original Figure 5: target image, adversarial image, clean image, and ten-times-amplified perturbation with UniDiffuser captions
Original Fig. 5. The residual is Δ = xadv − xclean, displayed at ×10 amplification. UniDiffuser supplies the captions; DALL-E supplies the reference target image. Click the figure to enlarge.

How Does Visual Grounding Change?

Question-related relevance shifts. In the Img2Prompt example, GradCAM highlights the skateboard for a question about the clean scene, but shifts toward background regions for the adversarial image. When conditioned on the target description, the adversarial image’s relevance pattern resembles that of the reference target image.

Original Figure 8: Img2Prompt GradCAM visualizations for clean, adversarial, and target images under scene-related and target-related questions
Original Fig. 8. GradCAM visualizations for Img2Prompt. These provide a feature-attribution view of the example; Sec. 4.3 also discusses the interpretation’s limitations. Click the figure to enlarge.

Failure Cases and Scope

Some responses retain the original scene. The appendix includes LLaVA examples in which the response changes but still describes concepts from the clean image, rather than following the intended target. The paper reports that additional optimization steps can improve these cases while keeping the same perturbation budget.

Original Figure 16, first example: LLaVA still describes dogs after an attack targeting a scene of people walking past bicycles
Original Fig. 16, Example 1. A failure case from the LLaVA evaluation. The target describes people and bicycles; the adversarial response retains dog-related content. See Appendix C.2. Click the figure to enlarge.

Digital-domain evaluation. The results concern the paper’s evaluated open-source checkpoints and images fed directly into those models. The discussion identifies physical-world evaluation, including camera-captured inputs, as future work.

Related Links

There are many excellent works that build large vision-language models, for example:

- LAVIS is a one-stop Library for Language-Vision Intelligence.

- MiniGPT-4 and LLaVA perform Vision Question-Answering on top of Large Language Models (LLMs).

- Unidiffuser can achieve multimodal joint generation by using a single ViT.

BibTeX

@inproceedings{zhao2023evaluate,
      title={On Evaluating Adversarial Robustness of Large Vision-Language Models},
      author={Zhao, Yunqing and Pang, Tianyu and Du, Chao and Yang, Xiao and Li, Chongxuan and Cheung, Ngai-Man and Lin, Min},
      booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
      year={2023}
    }

Meanwhile, a relevant research that aims to Embedding a Watermark to (multi-modal) Diffusion Models:

@article{zhao2023recipe,
                title={A Recipe for Watermarking Diffusion Models},
                author={Zhao, Yunqing and Pang, Tianyu and Du, Chao and Yang, Xiao and Cheung, Ngai-Man and Lin, Min},
                journal={arXiv preprint arXiv:2303.10137},
                year={2023}
              }