Zilong Zheng

Research Scientist at BIGAI

Email: z.zheng[at]ucla[dot]edu

I am a research scientist and team lead at Beijing Institute for General Artificical Intelligence (BIGAI).

I received my Ph.D. degree from the Department of Computer Science at University of California, Los Angeles (UCLA), under the supervision of Prof. Song-chun Zhu. Before that, I obtained bachelor degree of Computer Science at University of Minnesota. I also received B.E. degree from University of Electronic Science and Technology of China (UESTC).

My research interests lie in the intersection of statistical machine learning, vision-language modeling and cognition. Current research themes include:

  • Generative Modeling: Statistical generative modeling (energy-based models) for vision and language.
  • Multimodal Understanding: General vision-language (VL) understanding and explainable VL modeling.
  • Cognitive Language Reasoning: Understand and reason underlying information in language and human conversations.

I am always looking for self-motivated interns and long-term collaborators. Please contact me if you have excellent background or share similar research interests with me.

News

2023/05 Three papers are accepted to ACL’23. Congratulations to Yuxuan and Zixia.
2023/02 One paper is accepted to ICLR’23. Congratulations to Xiaojian and Silong.
2022/12 I served as an organizer on the first UM-IoS workshop at EMNLP’22. I also presented a keynote speak on Vison-Language Joint Parsing.
2022/07 Our work on bidirectional value alignment is accepted to Science Robotics Volume July! :sparkles: :smile:
2022/03 One paper on fine-grained Vision-Language alignment is accepted to CVPR’22. Congratulations to Chao Lou.

Publications

*: Equal contribution, : Corresponding author

2023

  1. SQA3D: Situated Question Answering in 3D Scenes ICLR'23
    Xiaojian Ma*, Silong Yong*, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang
    The Tenth International Conference on Learning Representations (ICLR) , 2023
  2. VSTAR: A Video-grounded Dialogue Dataset for Situated Semantic Understanding with Scene and Topic Transitions ACL'23
    Yuxuan Wang, Zilong Zheng, Xueliang Zhao, Jinpeng Li, Yueqian Wang, and Dongyan Zhao
    In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) , 2023
  3. Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field ACL'23
    Zixia Jia, Zhaohui Yan, Wenjuan Han, Zilong Zheng, and Kewei Tu
    In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) , 2023
  4. Shuō Wén Jiě Zì: Rethinking Dictionaries and Glyphs for Chinese Language Pre-training ACL'23
    Yuxuan Wang, Jianghui Wang, Dongyan Zhao, and Zilong Zheng
    In Findings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL-Findings) , 2023

2022

  1. In situ bidirectional human-robot value alignment ScienceRobotics
    Science Robotics , 2022
  2. SHARP: Search-Based Adversarial Attack for Structured Prediction NAACL'22
    Liwen Zhang, Zixia Jia, Wenjuan Han, Zilong Zheng, and Kewei Tu
    In Findings of Annual Conference of the North American Chapter of the Association for Computational Linguistics (NACCL) , 2022
  3. VGStore: A Multimodal Extension to SPARQL for Querying RDF Scene Graph ISWC'22
    Yanzeng Li, Zilong Zheng, Wenjuan Han, and Lei Zou
    In The 21st International Semantic Web Conference (ISWC) Poster & Demo Track , 2022
  4. Unsupervised Vision-Language Grammar Induction with Shared Structure Modeling Oral ICLR'22
    Bo Wan, Wenjuan Han, Zilong Zheng, and Tinne Tuytelaars
    The Tenth International Conference on Learning Representations (ICLR) , 2022
  5. Unsupervised Vision-Language Parsing: Seamlessly Bridging Visual Scene Graphs with Language Structures via Dependency Relationships CVPR'22
    Chao Lou*, Wenjuan Han, Yuhuan Lin, and Zilong Zheng*
    In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , 2022
  6. Energy-Based Generative Cooperative Saliency Prediction Oral AAAI'22
    Jing Zhang, Jianwen Xie, Zilong Zheng, and Nick Barnes
    The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI) , 2022

2021

  1. Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Multi-Modal Conditional Learning TPAMI
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 2021
  2. Learning Triadic Belief Dynamics in Nonverbal Communication from Videos Oral CVPR'21
    In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , 2021
  3. Patchwise Generative ConvNet: Training Energy-Based Models from a Single Natural Image for Internal Learning Oral CVPR'21
    Zilong Zheng, Jianwen Xie, and Ping Li
    In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , 2021
  4. Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification CVPR'21
    In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , 2021
  5. GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning ACL'21
    In Findings of the Association for Computational Linguistics: ACL-IJCNLP (ACL-Findings), 2021 , 2021
  6. Learning Energy-Based Model with Variational Auto-Encoder as Amortized Sampler AAAI'21
    Jianwen Xie, Zilong Zheng, and Ping Li
    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI) , 2021
  7. Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translation AAAI'21
    The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI) , 2021

2020

  1. Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis TPAMI
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , 2020
  2. Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs ICRA'20
    In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) , 2020
  3. Motion-Based Generator Model: Unsupervised Disentanglement of Appearance, Trackable and Intrackable Motions in Dynamic Patterns Oral AAAI'20
    The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI) , 2020

2019

  1. Reasoning Visual Dialogs with Structural and Partial Observations Oral CVPR'19
    Zilong Zheng*, Wenguan Wang*, Siyuan Qi*, and Song-Chun Zhu
    In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , 2019
  2. Learning Dynamic Generator Model by Alternating Back-Propagation Through Time Spotlight AAAI'19
    The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI) , 2019

2018

  1. Learning Descriptor Networks for 3D Shape Synthesis and Analysis Oral CVPR'18
    In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) , 2018

Professional Services

  • Conference reviewer for ICML 2022-2023; ICLR 2022-2023; CVPR 2019-2022; NeurIPS 2020-2022; AAAI 2020-2022; ICCV 2019-2023; ECCV 2020-2022; BMVC 2020; WACV 2021
  • Journal reviewer for International Journal of Computer Vision (IJCV), Pattern Recognition (PR), Neurocomputing