I am a Principal Researcher at Microsoft Research. My recent research focuses on large-scale natural language processing and multimodal learning, which includes:

• Building (multimodal) foundation models and vision-language assistant [1, 2, 3]
• LLM distillation for broad application classes [4, 5]
• Domain adaptation of LLMs without specialized training [6, 7]

If you are interested in working with me on any of these topics, please feel free to drop me an email.

I obtained my Ph.D. in natural language processing and machine learning at Johns Hopkins University, advised by Benjamin Van Durme and Kevin Duh. My PhD research studies transductive semantic parsing. My work has been nominated for the best paper of ACL. I have served as an Area Chair for NeurIPS, EMNLP, NAACL, AAAI & IJCNLP-AACL.

Selected Publications [See Google Scholar for full publications]

• MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding ICLR 2025
  Fei Wang^*, Xingyu Fu^*, James Y. Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, Tianyi Lorena Yan, Wenjie Jacky Mo, Hsiang-Hui Liu, Pan Lu, Chunyuan Li, Chaowei Xiao, Kai-Wei Chang, Dan Roth, Sheng Zhang, Hoifung Poon, Muhao Chen (^*equal contribution)
  [ Project Page | Data ]
• From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning NAACL 2025
  Nan Xu, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen
  
• BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs NEJM AI
  Sheng Zhang^*, Yanbo Xu^*, Naoto Usuyama^*, Hanwen Xu^*, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, Cliff Wong, Andrea Tupini, Yu Wang, Matt Mazzola, Swadheen Shukla, Lars Liden, Jianfeng Gao, Angela Crabtree, Brian Piening, Carlo Bifulco, Matthew P. Lungren, Tristan Naumann, Sheng Wang, Hoifung Poon (^*equal contribution)
  [ Model | Data ]
• A whole-slide foundation model for digital pathology from real-world data Nature
  Hanwen Xu^*, Naoto Usuyama^*, Jaspreet Bagga, Sheng Zhang, Rajesh Rao, Tristan Naumann, Cliff Wong, Zelalem Gero, Javier González, Yu Gu, Yanbo Xu, Mu Wei, Wenhui Wang, Shuming Ma, Furu Wei, Jianwei Yang, Chunyuan Li, Jianfeng Gao, Jaylen Rosemon, Tucker Bower, Soohee Lee, Roshanthi Weerasinghe, Bill J. Wright, Ari Robicsek, Brian Piening, Carlo Bifulco, Sheng Wang, Hoifung Poon
  (^*equal contribution)
  [ Model | Data ]
• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day NeurIPS 2023 Datasets & Benchmarks (Spotlight)
  Chunyuan Li^*, Cliff Wong^*, Sheng Zhang^*, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao (^*equal contribution)
  [ Project page ]
• Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
  Harsha Nori^*, Yin Tat Lee^*, Sheng Zhang^*, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, Renqian Luo, Scott Mayer McKinney, Robert Osazuwa Ness, Hoifung Poon, Tao Qin, Naoto Usuyama, Chris White, Eric Horvitz
  (^*equal contribution)
  [ MSR blog | promptbase ]
• UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition ICLR 2024
  Wenxuan Zhou^*, Sheng Zhang^*, Yu Gu, Muhao Chen, Hoifung Poon (^*equal contribution)
  [ Demo | Model | MSR Podcast ]

Tutorials

• Precision Health in the Age of Large Language Models KDD 2023
  Hoifung Poon, Tristan Naumann, Sheng Zhang, Javier González
  [ Webpage | Blog ]

Service

• Area Chair: NeurIPS 2023; ARR; ACL 2024; NAACL 2021, 2024; EMNLP 2022; IJCNLP-AACL 2023
• Tutorial: KDD 2023
• Organizer: Workshop on COmmonsense INference in NLP (COIN) at EMNLP 2019
• (S)PC Member/Reviewer: TACL; Computational Linguistics; ARR; BMC Bioinformatics; ACL 2017-2023; EMNLP 2018-2021; AAAI 2020-2024; ICCV 2023; NAACL 2018-2021; EACL 2017 2021; AACL-IJCNLP 2020; COLM 2024; COLING 2020; CoNLL 2019; IJCNLP 2017; IWCS 2017; NLE