Yunqing Zhao

I am a Machine Learning Research Scientist in TikTok/ByteDance, Singapore. I received my PhD in computer science (AI/ML) from Singapore University of Technology and Design (SUTD), advised by Prof. Ngai-Man Cheung. My research interests lie in multimodal AI, large language models and data / parameter - efficient training of AI models.

During my PhD, I am fortunate to have research experience with Chao Du and Tianyu Pang and being advised by Prof. Shuicheng Yan and Min Lin at Sea AI Lab. I had great pleasure to collaborate with Henghui Ding and Houjing Huang at TikTok / Bytedance AI Lab. I also spent a winter at Microsoft-Research Asia Lab and explored the capabilities of multimodal foundation models.

Prior to my PhD, I did my master in University of Hong Kong (degree with Distinction) and undergraduate in China University of Geosciences (with honors).

[Open to research collaborations on Generative AI & Multimodal LLMs!]

Email  /  LinkedIn  /  Github  /  Google Scholar


Selected Publications

project_img

Evaluating adversarial robustness of large vision-language models (VLMs)
Yunqing Zhao*, Tianyu Pang*, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, Min Lin

Large VLMs such as GPT-4 achieve unprecedented performance in response generation, esp. with visual inputs, enabling more creative and adaptable interaction than LLMs like ChatGPT. However, multimodal generation exacerbates safety concerns, since adversaries may successfully evade the entire system by subtly manipulating the most vulnerable modality (e.g., vision). We evaluate the robustness of open-source large VLMs (e.g., MiniGPT-4, LLaVA, BLIP, UniDiffuser) in the most realistic and high-risk setting, where adversaries have only black-box system access and seek to deceive the model into returning the targeted responses.

NeurIPS 2023, New Orleans, Louisiana, United States.
[Paper] [Webpage] [Code]

project_img

Exploring incompatible knowledge transfer in few-shot image generation
Yunqing Zhao, Chao Du, Milad Abdollahzadeh, Tianyu Pang, Min Lin, Shuicheng Yan, Ngai-Man Cheung

Through interpretable GAN dissection tools, we demonstrate that fine-tuning based methods cannot effectively remove knowledge that is incompatible to the target domain after adaptation (e.g., trees /buildings on the sea) for few-shot image generation task. We propose Remove In-Compatible Knowledge (RICK), an efficient and dynamic algorithm that estimates the filter importance and prune those are incompatible to the target domain.

CVPR 2023, Vancouver, British Columbia, Canada.
[Paper] [Webpage] [Code]

project_img

FS-BAN: Born-Again Networks for Domain Generalization Few-shot Classification
Yunqing Zhao, Ngai-Man Cheung

We propose a method to improve the generalizability for cross-domain few-shot classification problem using born-again networks. Our algorithm does not require additional parameters and training data and can be applied readily to many exisiting FSC models. The key insight is to distill the dark knowledge from a teacher model with additional multi-task objectives designed specific for cross-domain few-shot learning.

IEEE Trans. on Image Processing (TIP) 2023
[Paper] [Webpage] [Code]

project_img

Few-shot image generation via adaptation-aware kernel modulation
Yunqing Zhao*, Keshigeyan Chandrasegaran*, Milad Abdollahzadeh*, Ngai-Man Cheung

When fine-tuning a pretrained image generator on few-shot target samples, we show that state-of-the-art algorithms perform no-better than a simple baseline method when the target samples are distant to the source domain. We propose AdAM, a parameter-efficient and target-aware method to select source knowledge important for few-shot adaptation.

NeurIPS 2022, New Orleans, Louisiana, United States.
[Paper] [Webpage] [Code]

project_img

Revisiting Label Smoothing & Knowledge Distillation Compatibility: What was Missing?
Keshigeyan Chandrasegaran, Ngoc-Trung Tran*, Yunqing Zhao*, Ngai-Man Cheung

We investigate the compatibility between label smoothing (LS) and knowledge distillation (KD), i.e., to smooth or not to smooth a teacher network? We discover, analyze and validate the proposed systematic diffusion as the missing concept which is instrumental in understanding and resolving these contradictory findings in prior works. This systematic diffusion essentially curtails the benefits of distilling from an LS-trained teacher, thereby rendering KD at increased temperatures ineffective.

ICML 2022, Baltimore, Maryland, United States.
[Paper] [Webpage] [Code]

project_img

A Closer Look at Few-shot Image Generation
Yunqing Zhao, Henghui Ding, Houjing Huang, Ngai-Man Cheung

We analyze the existing few-shot image generation algorithms in a unified testbed and find that diversity degradation is the major issue during few-shot target adaptation. Our proposed mutual information based algorithm can alleviate this issue and achieve state-of-the-art performance on few-shot image generation tasks.

CVPR 2022, New Orleans, Louisiana, United States.
[Paper] [Webpage] [Code]




Workshop & Challenge


project_img

Explanation-guided Training for Cross-domain Few-shot Classification
Jiamei Sun, Sebastian Lapuschkin, Wojciech Samek, Yunqing Zhao, Ngai-Man Cheung, Alexander Binder


ICML-2020 Workshop & ICPR-2020.
[Paper] [Code]

project_img

CIKM-2020 Alibaba-Tsinghua Adversarial Challenge on Object Detection
Honglin Li, Yunqing Zhao

Rank 10/1814 in the Challenge

CIKM-2020 Workshop.
[Paper] [Code]

denotes corresponding author
* denotes equal contribution




Research Experience


project_img

Microsoft Research - Asia
Research Intern


11.2023 - 02.2024,
Exploring the applications/capability of LLMs for multi-modal understanding and generation.

project_img

Sea AI Lab, Singapore
Research Intern


09.2022 - 11.2023,
Work with Chao Du and Tianyu Pang
Advised by Prof. Shuicheng Yan and Min Lin

project_img

TikTok / ByteDance AI Lab, Singapore
Research Intern


08.2021 - 08.2022,
Work with Henghui Ding (now @ NTU, Singapore) and Houjing Huang (now @ UZH, Switzerland)

project_img

ST Engineering - SUTD Cyber Security Lab, Singapore
Student Researcher

08.2020 - 07.2021,
Advised by Prof. Ngai-Man Cheung

project_img

University of Hong Kong
Research Assistant


Spent wonderful days in SouthLane & Pok Fu Lam Road
11.2018 - 04.2019,
Advised by Dr. Vincent Tam




Education


project_img

Singapore University of Technology and Design
8 Somapah Road, Singapore 487372

Ph.D in AI and Machine Learning.
Jan. 2020 - July 2024

About SUTD:
Founded with MIT and started in 2012, SUTD is ranked
#66 globally in AI, Computer Vision and NLP on CSRanking in 2020-2024.



project_img

The University of Hong Kong
Pok Fu Lam Road, Hong Kong

M.S in Electrical and Electronic Engineering (degree with Distinction)
I have a wonderful visiting experience in Zhejiang University, Hangzhou, China.

project_img

China University of Geosciences
388 Lumo Road, Wuhan, 430074

B.Eng in Automation
I spent a wonderful summer semester in University of British Columbia, Vancouver BC, Canada.


Teaching & Service

Active Reviewer of NeurIPS, CVPR, TPAMI, TIP, TIFS, TNNLS, TASL, TMM, TCSVT, CVIU, etc.

Graduate Teaching Assistant of 50.021 Artificial Intelligence and 50.035 Computer Vision @ SUTD


web template taken from this website