See and Think: Embodied Agent in Virtual Environment

Zhonghan Zhao^1* , Wenhao Chai^*2❤, Xuan Wang^1*, Li Boyi¹, Shengyu Hao¹, Shidong Cao¹, Tian Ye³, Jenq-Neng Hwang², Gaoang Wang^1✉ ¹ Zhejiang University ² University of Washington ³ Hong Kong University of Science and Technology (GZ) ^*Equal contribution ^❤Project lead ^✉Corresponding author

STEVE, named after the protagonist of the game Minecraft, is our proposed framework aims to build an embodied agent based on the vision model and LLMs within an open world.

Link: See and Think: Embodied Agent in Virtual Environment