I work on diffusion-based controllable image and video generation at UIUC, advised by Yu-Xiong Wang. I design the supervision, training data, and curricula that teach pretrained generative models new capabilities, including new visual controls, multi-view and temporal consistency, and minute-scale identity-consistent video.
My current work builds generative world models that predict what an action or event will cause: the next screen after a UI action, the continuation of a physical event, and the camera view of a robot following a commanded trajectory. My long-term goal is one model combining the understanding and reasoning of vision-language models (VLMs) with the visual generation capabilities of diffusion models, so reasoning guides generation and generation improves reasoning (research vision).
Graduating Dec. 2026|Open to Research Scientist / Research Engineer roles starting 2027
First or co-first author at NeurIPS 2025, NeurIPS 2024, CVPR 2024 (×2), 3DV 2026, and WACV 2027, across controllable image, video, 3D, and 4D generation.
Virtual Fitting Room: identity-consistent try-on video up to 90 seconds long from a single image, trained only on short clips. NeurIPS 2025
Dress&Dance: teaches pretrained video diffusion new garment, identity, and motion controls. 20–59% lower FVD than a Kling 1.6 try-on-and-animation pipeline. WACV 2027
Research internships at Meta Reality Labs (Zurich 2023, Bay Area 2025) and SpreeAI. At Meta in 2025, designed a data-processing and training pipeline at 128–256-GPU scale.
Jun-Kun Chen, Aayush Bansal, Minh Phuoc Vo, Yu-Xiong Wang
Builds on Dress&Dance (D&D). Learns reference-conditioned generation from ordinary pairs of short clips. At inference, generated 360° anchors keep identity consistent in try-on videos up to 90 seconds long. On 30–50 s hard-motion tests it scores highest on VBench subject consistency, background consistency, and motion smoothness against D&D+FramePack and D&D+Kling 2.0 (93.2 / 95.3 / 99.4 vs. 88.3 / 91.0 / 98.2 and 92.7 / 94.1 / 98.9), with comparable GPT-judged try-on quality.
Jun-Kun Chen, Aayush Bansal, Minh Phuoc Vo, Yu-Xiong Wang
A general framework for conditional video generation: CondNet maps text, layout, pose, garment, and identity into one conditioning space, and target-steered supervision with staged training adds these controls to pretrained video diffusion. Demonstrated on high-resolution virtual try-on, it generates 1152×720, 24-FPS videos and reaches FVD (lower is better) of 157 vs. 197 on Internet videos and 20 vs. 49 on captured videos against a Kling 1.6 try-on-and-animation pipeline, and lower FVD than all four evaluated modular baselines on both benchmarks. Followed by Virtual Fitting Room, which extends this setting to minute-long videos.
Jun-Kun Chen, Samuel Rota Bulò, Norman Müller, Lorenzo Porzi, Peter Kontschieder, Yu-Xiong Wang
Teaches 2D diffusion to edit 3D scenes consistently across views. Uses cross-view self-supervision and staged adaptation, with consistency losses along the full denoising trajectory. A 4-GPU asynchronous pipeline overlaps diffusion training and image generation with scene fitting, for a 10× training speed-up.
The first instruction-driven 4D editing framework. Treating a dynamic scene as pseudo-3D breaks long 4D edits into tractable problems for 2D diffusion.
Yanming Zhang*, Jun-Kun Chen*, Jipeng Lyu, Yu-Xiong Wang
General-purpose, training-free editing of videos and 3D scenes. Hierarchical task decomposition separates instruction following from content preservation.
* Equal contribution. Publications appear under Jun-Kun Chen; earlier papers may use Junkun Chen or J. Chen.
Current Research: Generative World Models Planned submissions to CVPR 2027
Three current projects develop and evaluate generative world models for computer use, physical events, and robotics, focusing on reliable predictions of actions and their visual consequences.
A training-set illustration of GUI next-screen prediction, comparing generated screens with a reference screen.
I lead ProofPass, a visual GUI world model that predicts the screen resulting from a user-interface action. It combines VLM reasoning and diffusion-based image generation, with a generation-informed training approach that improves prediction fidelity. Controlled ablations establish the contribution of the training design. On 654 AndroidWorld test transitions, the full system scores 71.86/100 in VLM-judged screen similarity, compared with 66.87 for supervised fine-tuning alone. Across 109 live Android tasks, four agents' pooled success increases by 5.73 percentage points when they consult the model. The gain also holds against a control that selects among candidate actions without predicted screens.
Diagnostic examples of physical video continuation, comparing recordings with generated videos.
Physics-aware video continuation
Video diffusion · Cosmos3-Nano · 2026 · project lead, supervising an undergraduate intern
I lead a study of where physical-outcome errors arise in plan-guided video generation. Our controlled evaluations identify planning as a decisive factor in the physical outcomes of the video models studied. We also post-trained Cosmos3-Nano for physical video continuation, gaining 8.74 Physics-IQ points over the base model under the same interface, and developed a training-free planning method that improves over single-call planning.
Robot-motion prediction in a held-out DROID scene, followed by a synthetic diagnostic.
Peek&Perform: robot world model for new setups
Motion-conditioned video diffusion · Cosmos3-Nano · 2026 · solo project
Peek&Perform is a robot video world model for predicting the visual consequences of commanded motion in new setups, without target-camera calibration or per-setup retraining. I independently designed its control and adaptation methods, built the training and evaluation pipelines, and post-trained Cosmos3-Nano on more than 100k real and simulated clips. In a comparison on one unseen robot (24 cases), with no method receiving camera calibration, it achieved a robot IoU of .647 versus .565 for the best of six published world models. On the final six frames, the scores were .575 versus .438.
Research Vision
Research direction
My long-term goal is a unified model that combines the understanding and reasoning of vision-language models (VLMs) with the visual generation capabilities of diffusion models. I want reasoning to guide what the model generates, and the visual, 3D and motion knowledge learned through generation to strengthen what it understands. The aim is a model that can reason about possible outcomes as well as generate them.
Research vision: one unified model that combines the understanding and reasoning of a VLM with the visual generation of diffusion, so that each strengthens the other. Reasoning guides generation, and the visual, 3D and motion knowledge of generation enriches understanding and reasoning.
Research Scientist InternBay Area, CA, U.S. · Zurich, Switzerland
May–Dec. 2025Bay Area · part-time continuation Aug.–Dec.
Studied joint 3D and video generation, then explored video-to-3D and 3D-to-video generation. Designed and implemented a GPU-role-separated pipeline for data processing and model training at 128–256-GPU scale on Meta's distributed infrastructure.
May–Aug. 2023Zurich
Took ConsistDreamer (CVPR 2024) from method design through implementation and evaluation. Designed cross-view self-supervision, staged diffusion adaptation, and a 4-GPU asynchronous pipeline that overlaps diffusion training and image generation with scene fitting, for a 10× training speed-up.
SpreeAI
AI Research InternRemote, U.S.
May 2024 – May 2025
Dress&Dance (WACV 2027): taught pretrained video diffusion new garment, identity, and motion controls. Designed and implemented target-steered supervision, training triplets with synthetic conditions and real targets, and staged image/video curricula. Evaluated garment and identity fidelity, motion, and video quality.
Virtual Fitting Room (NeurIPS 2025): designed training on arbitrary pairs of short clips to learn reference appearance and prefix continuation, without canonical 360° training anchors. At inference, a generated 360° reference and anchored autoregression maintain minute-scale consistency, while a refiner trained with targeted augmentation removes generator artifacts.
Mila – Quebec AI Institute
Research InternMontreal, Canada
Feb.–Aug. 2020Knowledge-graph reasoning with Prof. Jian Tang. Co-first author of RNNLogic (ICLR 2021).
Baidu, NLP Department
Research / Development InternBeijing, China
July 2019 – Jan. 2020Scene-aware dialogue generation. Top-3 system at DSTC8, presented at the DSTC8 Workshop at AAAI 2020.
Models: image/video diffusion, diffusion Transformers (DiT), video world models (NVIDIA Cosmos), unified multimodal models (mixture-of-transformers), vision-language models, LoRA
Practice: training diffusion models with rectified-flow / flow-matching objectives, diffusion post-training (supervised fine-tuning, LoRA, reward-based tuning with DiffusionNFT), multimodal supervision and model adaptation, action and multimodal conditioning, video and robot-data pipelines, multi-GPU training with exact checkpoint resume, held-out benchmark design, human and VLM-judge evaluation, paired-bootstrap statistics
Selected Honors
ICPC EC-Final — Gold Medal, 4th place (2018)
ICPC Qingdao Regional — Gold Medal, 2nd place (2018)
CCPC Harbin Regional — Gold Medal (2017)
National Olympiad in Informatics — Gold Medal (2016)
Problem setter — 9 contest tasks, including hardest-tier problems at NOI 2018 and NOI 2019 (2018–2019)
