Daqing Liu | JD Explore Academy

About Me

Currently, I am a research scientist at JD Explore Academy (JDEA), JD.com Inc, led by Dr. Xiaodong He and Prof. Dacheng Tao. Before that, I received my Ph.D. degree from the University of Science and Technology of China (USTC) in 2021, advised by Prof. Zheng-Jun Zha. From May 2018 to May 2019, I was a visiting student at Nanyang Technological University (NTU), working with Associate Prof. Hanwang Zhang.

Research Interests

My research revolves around the exciting field of Multi-Modality Learning (MML) and its practical applications. I am driven by the ambition to develop an artificial general intelligence (AGI) that can effectively represent and comprehend information from diverse modalities, enabling it to make informed decisions. Within the realm of MML, I aim to bridge the gap between different modalities, unlock the full potential of multimodal data, and explore various research areas that contribute to this overarching goal, including:

Multi-Modality Representation
- Image self-supervised learning (SemMAE[NeurIPS'22])
- Video representation learning (VSP[CVPR'23])
Multi-Modality Alignment
- Visual grounding (NMTree[ICCV'19], RvG-Tree[TPAMI'21], TransVG++[TPAMI'23])
- Cross-modal retrieval (TCP[ICCV'23], CMI[arXiv'23])
Multi-Modality Generation
- Visual captioning (CAVP[MM'18, TPAMI'21], POS-SCAN[CVPR'20], RMN[IJCAI'20], CBT[arXiv'22], Prefix-Captioning[arXiv'23])
- Image generation (TwFA[CVPR'22], DualCycleDiffsion[TCSVT'23], MMoT[arXiv'23]), Cocktail🍸[NeurIPS'23])
Multi-Modality Decision-Making
- Visual question answering (CMCL[arXiv'22])
- Vision-language navigation (ESceme[arXiv'23])

News

[2023-09-22] 🌟🌟 Our paper Cocktail🍸 is accepted to NeurIPS 2023!
[2023-07-14] 🌟🌟 One paper about cross-modal retrieval is accepted to ICCV 2023!
[2023-07-06] 🌟🌟 One paper about visual grounding is accepted to IEEE TPAMI!
[2023-06-12] One paper about image editing is accepted to IEEE TCSVT.
[2023-06-01] 🔥🔥 Our AIGC project Cocktail🍸 is avaliable!
[2023-05-10] 🔥🔥 Our AIGC project MMoT is avaliable!
[2023-03-22] 🎖🎖 Our CVPR paper VSP is selected as highlight (top 2.5%)!
[2023-02-28] One paper about video representation learning is accepted to CVPR 2023.
[2022-09-15] One paper about image self-supervised learning is accepted to NeurIPS 2022.
[2022-06-30] One paper about image deblurring is accepted to ACM MM 2022.
[2022-03-03] One paper about image generation is accepted to CVPR 2022.
Show more

Publications

*: Co-First Authors, #: Supervised Students, ✉: Corresponding Authors
I marked my name in Red if I am the first or corresponding author, or the mentor of the first author.

arXiv

arXiv

MMoT: Mixture-of-Modality-Tokens Transformer for Composed Multimodal Conditional Image Synthesis

Jianbin Zheng#, Daqing Liu*, Chaoyue Wang, Minghui Hu, Zuopeng Yang, Changxing Ding, Dacheng Tao

arXiv preprint, arXiv:2305.05992.

arXiv PDF Project Page Code Journal Version
arXiv

ESceme: Vision-and-Language Navigation with Episodic Scene Memory

Qi Zheng#, Daqing Liu, Chaoyue Wang, Jing Zhang, Dadong Wang, Dacheng Tao

arXiv preprint, arXiv:2303.01032.

arXiv PDF Code Journal Version
arXiv

OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System

Chao Xue, Wei Liu, Shuai Xie, Zhenfang Wang, Jiaxing Li, Xuyang Peng, Liang Ding, Shanshan Zhao, Qiong Cao, Yibo Yang, Fengxiang He, Bohua Cai, Rongcheng Bian, Yiyan Zhao, Heliang Zheng, Xiangyang Liu, Dongkai Liu, Daqing Liu, Li Shen, Chang Li, Shijin Zhang, Yukang Zhang, Guanpu Chen, Shixiang Chen, Yibing Zhan, Jing Zhang, Chaoyue Wang, Dacheng Tao

arXiv preprint, arXiv:2303.00501.

arXiv PDF Journal Version, technical report of our OmniForce System
arXiv

Cross-Modal Contrastive Learning for Robust Reasoning in VQA

Qi Zheng#, Chaoyue Wang, Daqing Liu, Dadong Wang, Dacheng Tao

arXiv preprint, arXiv:2211.11190.

arXiv PDF Code Journal Version

2023

NeurIPS

Cocktail🍸: Mixing Multi-Modality Controls for Text-Conditional Image Generation

Minghui Hu#, Jianbin Zheng#, Daqing Liu, Chuanxia Zheng, Chaoyue Wang, Dacheng Tao, Tat-Jen Cham

Neural Information Processing Systems (NeurIPS), 2023.

arXiv PDF Project Page Code
ICCV

Exploring Temporal Concurrency for Video-Language Representation Learning

Heng Zhang#, Daqing Liu, Bing Su, Dacheng Tao

IEEE/CVF International Conference on Computer Vision (ICCV), 2023.

Coming soon
TPAMI

TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer

Jiajun Deng, Zhengyuan Yang, Daqing Liu, Tianlang Chen, Wengang Zhou, Yanyong Zhang, Houqiang Li, Wanli Ouyang

IEEE Transactions on Pattern Analysis and Machine Intelligence.

arXiv PDF Code Impact Factor: 24.314
TCSVT

Eliminating Prior Bias for Semantic Image Editing via Dual-Cycle Diffusion

Zuopeng Yang#, Tianshu Chu, Xin Lin, Erdun Gao, Daqing Liu, Jie Yang, Chaoyue Wang

IEEE Transactions on Circuits and Systems for Video Technology.

arXiv PDF Code
CVPR

Modeling Video as Stochastic Processes for Fine-Grained Video Representation Learning

Heng Zhang#, Daqing Liu*, Qi Zheng, Bing Su

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

PDF Code Highlight, Accept Rate: 2.5%

2022

NeurIPS

SemMAE: Semantic-Guided Masking for Learning Masked Autoencoders

Gang Li, Heliang Zheng, Daqing Liu, Bing Su, Changwen Zheng

Neural Information Processing Systems (NeurIPS), 2022.

arXiv PDF Code Full, Accept Rate: 25.6%
MM

Semantically-Consistent Dynamic Blurry Image Generation for Image Deblurring

Zhaohui Jing, Youjian Zhang, Chaoyue Wang, Daqing Liu, Yong Xia

ACM International Conference on Multimedia (ACM MM), 2022.

Full, Accept Rate: 27.9%
CVPR

Modeling Image Composition for Complex Scene Generation

Zuopeng Yang#, Daqing Liu*, Chaoyue Wang, Jie Yang, Dacheng Tao

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

arXiv PDF Code Full, Accept Rate: 25.3%

2021

TPAMI

Context-Aware Visual Policy Network for Fine-Grained Image Captioning

Zheng-Jun Zha, Daqing Liu✉, Hanwang Zhang, Yongdong Zhang, Feng Wu

IEEE Transactions on Pattern Analysis and Machine Intelligence.

arXiv PDF Code Impact Factor: 24.314
TPAMI

Learning to Compose and Reason with Language Tree Structures for Visual Grounding

Richang Hong, Daqing Liu✉, Xiaoyu Mo, Xiangnan He, Hanwang Zhang

IEEE Transactions on Pattern Analysis and Machine Intelligence.

arXiv PDF Impact Factor: 24.314

2020

IJCAI

Learning to Discretely Compose Reasoning Module Networks for Video Captioning

Ganchao Tan#, Daqing Liu*, Meng Wang, Zheng-Jun Zha

International Joint Conference on Artificial Intelligence (IJCAI), 2020.

arXiv PDF Code Oral, Accept Rate: 12.6%
CVPR

More Grounded Image Captioning by Distilling Image-Text Matching Model

Yuanen Zhou#, Meng Wang, Daqing Liu, Zhenzhen Hu, Hanwang Zhang

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

arXiv PDF Code Full, Accept Rate: 22%

2019

ICCV

Learning to Assemble Neural Module Tree Networks for Visual Grounding

Daqing Liu, Hanwang Zhang, Zheng-Jun Zha, Feng Wu

IEEE/CVF International Conference on Computer Vision (ICCV), 2019.

arXiv PDF Code Talk Oral, Accept Rate: 4.3%

2018

MM

Context-Aware Visual Policy Network for Sequence-Level Image Captioning

Daqing Liu, Zheng-Jun Zha, Hanwang Zhang, Yongdong Zhang, Feng Wu

ACM International Conference on Multimedia (ACM MM), 2018.

arXiv PDF Code Oral, Accept Rate: 8.5%

Experience

Services

Conference Reviewers

Journal Reviewers

Contact

Address: Block 2-A-13, JD Building, Beijing, China, 101111
Phone: (86) 188-5698-8126 Wechat: liudaqing94