About
I’m a fourth-year PhD student at Columbia University, advised by Carl Vondrick. I’m fortunate to receive mentorship from Shuran Song and Shree Nayar during my PhD.
In the past decades, we’ve figured out very effective ways to pass knowledge from humans to machines, a powerful capability branded in different names (next-token prediction, contrastive learning, behavior cloning etc.). As a community, we should think about what goes beyond. Therefore, I’m interested in developing embodied intelligent systems that can exceed human capability perceptually, intellectually, and physically.
News
[Sep 2024] Two papers accepted to CoRL, with one oral!
[Jun 2024] Three papers accepted to ECCV, with two orals and one best paper finalist!
[Mar 2024] Two papers accepted to CVPR, with one highlight!
[Jul 2023] Two recent papers accepted to ICCV!
[Mar 2023] Zero123 released! Thanks Huggingface for funding a demo!
[Feb 2023] Two recent papers accepted to CVPR!
Publication
Self-Improving Autonomous Underwater Manipulation arXiv 2024
Ruoshi Liu, Huy Ha, Mengxue Hou, Shuran Song, Carl Vondrick
Paper | Project Page | Video | Code
Differentiable Robot Rendering CoRL 2024 (Oral)
Ruoshi Liu*, Alper Canberk*, Shuran Song, Carl Vondrick
Paper | Project Page | Video | Code
Dreamitate: Real-World Visuomotor Policy Learning via Video Generation CoRL 2024
Junbang Liang*, Ruoshi Liu*, Ege Ozguroglu, Sruthi Sudhakar, Achal Dave, Pavel Tokmakov, Shuran Song, Carl Vondrick
Paper | Project Page | Video | Code
EraseDraw: Learning to Draw Step-by-Step via Erasing Objects from Images ECCV 2024
Alper Canberk, Maksym Bondarenko, Ege Ozguroglu, Ruoshi Liu, Carl Vondrick
Paper | Project Page | Code | Model Weights
Controlling the World by Sleight of Hand ECCV 2024 (Oral, Best Paper Award Candidate)
Sruthi Sudhakar, Ruoshi Liu, Basile Van Hoorick, Carl Vondrick, Richard Zemel
Paper | Project Page
Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesis ECCV 2024 (Oral)
Basile Van Hoorick, Rundi Wu, Ege Ozguroglu, Kyle Sargent, Ruoshi Liu, Pavel Tokmakov, Achal Dave, Changxi Zheng, Carl Vondrick
Paper | Project Page | Code
PaperBot: Learning to Design Real-World Tools Using Paper arXiv 2024
Ruoshi Liu, Junbang Liang, Sruthi Sudhakar, Huy Ha, Cheng Chi, Shuran Song, Carl Vondrick
Paper | Project Page | 8-min Video | New Scientist Article
GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering CVPR 2024
Abdullah Hamdi, Luke Melas-Kyriazi, Guocheng Qian, Jinjie Mai, Ruoshi Liu, Carl Vondrick, Bernard Ghanem, Andrea Vedaldi
Paper | Project Page
pix2gestalt: Amodal Segmentation by Synthesizing Wholes CVPR 2024 (Highlight)
Ege Ozguroglu, Ruoshi Liu, Dídac Surís, Dian Chen, Achal Dave, Pavel Tokmakov, Carl Vondrick
Paper | Project Page
Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape ICLR 2024
Rundi Wu, Ruoshi Liu, Carl Vondrick, Changxi Zheng
Paper | Project Page | Code | Demo
Objaverse-XL: A Colossal Universe of 3D Objects NeurIPS 2023
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, Eli VanderBilt, Aniruddha Kembhavi, Carl Vondrick, Georgia Gkioxari, Kiana Ehsani, Ludwig Schmidt, Ali Farhadi
Paper | Project Page | Code | Zero123-XL
Zero-1-to-3: Zero-shot One Image to 3D Object ICCV 2023 (#3 most impactful paper at ICCV 2023)
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, Carl Vondrick
Paper | Project Page | Code | Demo
Landscape Learning for Neural Network Inversion ICCV 2023
Ruoshi Liu, Chengzhi Mao, Purva Tendulkar, Hao Wang, Carl Vondrick
Paper Blog Post
Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflection CVPR 2023 (high-concept research)
Ruoshi Liu, Carl Vondrick
Paper | Project Page
What You Can Reconstruct from a Shadow CVPR 2023
Ruoshi Liu, Sachit Menon, Chengzhi Mao, Dennis Park, Simon Stent, Carl Vondrick
Paper Blog Post
Learning the Predictability of the Future CVPR 2021
Dídac Surís*, Ruoshi Liu*, Carl Vondrick
Paper Project Page
Machine Learning Forecasting of Active Nematics Soft Matter 2021
Zhengyang Zhou, Chaitanya Joshi, Ruoshi Liu, Michael M. Norton, Linnea Lemma, Zvonimir Dogic, Michael F. Hagan, Seth Fraden, Pengyu Hong
Paper
Contact
Invited Talks
[Oct 2024] “Towards Intelligent Robots Beyond Human Capability” @ UMichigan Vision Seminar, hosted by Andrew Owens
[Oct 2024] “Towards Intelligent Robots Beyond Human Capability” @ University of Notre Dame, hosted by Mengxue Hou
[Oct 2024] “Towards Intelligent Robots Beyond Human Capability” @ TTIC/UChicago, hosted by Anand Bhattad, recording available
[Oct 2024] “Towards Intelligent Robots Beyond Human Capability” @ UIUC Vision Seminar, hosted by Shenlong Wang
[Sep 2024] “Towards Intelligent Robots Beyond Human Capability” @ Rice ECE Seminar, hosted by Ashok Veeraraghavan
[Jun 2024] “3D Foundation Models for Physical Intelligence” @ 3DFM Workshop CVPR, hosted by Hao Su
[Jun 2024] “From Video Understanding to Embodied AI” @ HVU Workshop CVPR, hosted by Shyamal Buch
[Jun 2024] “From Human Object Interaction to Robot Object Interaction” @ RHOBIN Workshop CVPR, hosted by Xi Wang
[Jun 2024] “3D Generation for Physical Intelligence” @ AI3DG Workshop CVPR, hosted by Georgios Pavlakos
[May 2024] “Fantastic 3D Generative Models and How to Use Them” @ VALSE Webinar, talk and panel available (in Chinese)
[Apr 2024] “Learning to Design Tools in the Real World” @ NYC Vision Day
[Mar 2024] “Generating the 3D World” @ NYU, hosted by Chen Feng
[Feb 2024] “Neural Network Inversion” @ UCL, recording available, hosted by Kaan Akşit
[Dec 2023] “Seeing the Unseen” @ AI Time Large Model Workshop
[Dec 2023] “Seeing the Unseen” @ NYC Vision day, hosted by Noah Snavely
[Nov 2023] “Generating the 3D World” @ Stanford, hosted by Jiajun Wu
[Nov 2023] “Generating the 3D World” @ Berkeley, hosted by Angjoo Kanazawa
[Nov 2023] “Generating the 3D World” @ TRI, hosted by Adrien Gaidon
[Oct 2023] “Generating the 3D World” @ Brown University, recording available, hosted by James Tompkin
[Oct 2023] “Generating the 3D World” @ CMU, hosted by Shubham Tulsiani
[Jul 2023] “Generating the 3D World” @ Oxford. recording and slides available, hosted by Christian Rupprecht
[May 2023] “Analysis by Synthesis for 3D Reconstruction” @ CCVL lab @ JHU hosted by Alan Yuille
[Apr 2023] “AIGC in the 3D World” @ GAMES Webinar, recording available, hosted by Xiaoguang Han, and Guanying Chen
[Sep 2022] “Inverting Generative Models for Visual Understanding” @ MIT hosted by Wei-Chiu Ma
[Oct 2021] “Learning the Predictability of the Future” @ UPenn hosted by Dinesh Jayaraman
Reviewer
CVPR, ICCV, ECCV, ICLR, NeurIPS, CoRL, ICRA, ICML, AAAI, AISTATS, Siggraph, Siggraph Asia, TPAMI, IJCV, ACCV
Other Things
My first name Ruoshi(若石), can be translated into “Like a Rolling Stone”.
My virtual meetings are sometimes interrupted by Pattie, a 3-year-old domestic shorthair.
I like playing guitar. A few bands whose music I deeply enjoy: Omnipotent Youth Society, Radiohead, Pink Floyd, Sunset Rollercoaster.
I’m a big movie fan and most admire the work of Stanley Kubrick, Quentin Tarantino, Lee Chang-dong, and Hirokazu Kore-eda.
I enjoy performing non-convex gradient ascent and descent a.k.a hiking and skiing.
As a person, I feel that I’m so weird and complicated. But I love it.