Jun-Kun Chen

Jun-Kun Chen

Legal name: Junkun Chen · Publications as Jun-Kun Chen / J.-K. Chen

Ph.D. Candidate in Computer Science, University of Illinois Urbana-Champaign

I work on diffusion-based controllable image and video generation at UIUC, advised by Yu-Xiong Wang. I design the supervision, training data, and curricula that teach pretrained generative models new capabilities, including new visual controls, multi-view and temporal consistency, and minute-scale identity-consistent video.

I am now training visual world models through multimodal post-training. They predict the next screen after a UI action, physical events from observed video, and robot–object interactions under a prescribed motion.

Graduating Dec. 2026 Open to Research Scientist / Research Engineer roles starting 2027

Portrait of Jun-Kun Chen

Selected Publications

Google Scholar

Static 3D scenes

Spatial consistency

ConsistDreamerProEditNeuralEditor

Videos & 4D scenes

Temporal consistency

Instruct 4D-to-4DHDEdit

Human-centric video

Multimodal control, long horizon

Dress&DanceVirtual Fitting Room

Interactive worlds · current

Predicting visual outcomes

Computer usePhysicsRobotics

NeurIPS 2025First author

Virtual Fitting Room: Generating Arbitrarily Long Videos of Virtual Try-On from a Single Image

Jun-Kun Chen, Aayush Bansal, Minh Phuoc Vo, Yu-Xiong Wang

Builds on Dress&Dance (D&D). Learns reference-conditioned generation from ordinary pairs of short clips. At inference, generated 360° anchors keep identity consistent in try-on videos up to 90 seconds long. On 30–50 s hard-motion tests it scores highest on VBench subject consistency, background consistency, and motion smoothness against D&D+FramePack and D&D+Kling 2.0 (93.2 / 95.3 / 99.4 vs. 88.3 / 91.0 / 98.2 and 92.7 / 94.1 / 98.9), with comparable GPT-judged try-on quality.

ProjectPaperPreview (arXiv, Sep. 2025)

WACV 2027First author · with SpreeAI

Dress&Dance: Dress up and Dance as You Like It

Jun-Kun Chen, Aayush Bansal, Minh Phuoc Vo, Yu-Xiong Wang

A general framework for conditional video generation: CondNet maps text, layout, pose, garment, and identity into one conditioning space, and target-steered supervision with staged training adds these controls to pretrained video diffusion. Demonstrated on high-resolution virtual try-on, it generates 1152×720, 24-FPS videos and reaches FVD (lower is better) of 157 vs. 197 on Internet videos and 20 vs. 49 on captured videos against a Kling 1.6 try-on-and-animation pipeline, and lower FVD than all four evaluated modular baselines on both benchmarks. Followed by Virtual Fitting Room, which extends this setting to minute-long videos.

ProjectFull paperPreview (arXiv, Aug. 2025)

CVPR 2024First author · with Meta Reality Labs Zurich

ConsistDreamer: 3D-Consistent 2D Diffusion for High-Fidelity Scene Editing

Jun-Kun Chen, Samuel Rota Bulò, Norman Müller, Lorenzo Porzi, Peter Kontschieder, Yu-Xiong Wang

Teaches 2D diffusion to edit 3D scenes consistently across views. Uses cross-view self-supervision and staged adaptation, with consistency losses along the full denoising trajectory. A 4-GPU asynchronous pipeline overlaps diffusion training and image generation with scene fitting, for a 10× training speed-up.

ProjectPaperarXiv

Show all 14 publications Hide full publication list

  1. 2027

    1. Dress&Dance: Dress up and Dance as You Like ItWACV 2027Jun-Kun Chen, Aayush Bansal, Minh Phuoc Vo, Yu-Xiong Wang ProjectarXiv

  2. 2026

    1. HDEdit: Editing Videos and 3D Scenes with Video Diffusion Through Hierarchical Task Decomposition3DV 2026Yanming Zhang*, Jun-Kun Chen*, Jipeng Lyu, Yu-Xiong Wang ProjectarXiv

  3. 2025

    1. Virtual Fitting Room: Generating Arbitrarily Long Videos of Virtual Try-On from a Single ImageNeurIPS 2025Jun-Kun Chen, Aayush Bansal, Minh Phuoc Vo, Yu-Xiong Wang ProjectarXiv

  4. 2024

    1. ProEdit: Simple Progression is All You Need for High-Quality 3D Scene EditingNeurIPS 2024Jun-Kun Chen, Yu-Xiong Wang ProjectarXiv
    2. SceneCraft: Layout-Guided 3D Scene GenerationNeurIPS 2024Xiuyu Yang*, Yunze Man*, Jun-Kun Chen, Yu-Xiong Wang ProjectarXiv
    3. ConsistDreamer: 3D-Consistent 2D Diffusion for High-Fidelity Scene EditingCVPR 2024Jun-Kun Chen, Samuel Rota Bulò, Norman Müller, Lorenzo Porzi, Peter Kontschieder, Yu-Xiong Wang ProjectarXiv
    4. Instruct 4D-to-4D: Editing 4D Scenes as Pseudo-3D Scenes Using 2D DiffusionCVPR 2024Linzhan Mou*, Jun-Kun Chen*, Yu-Xiong Wang ProjectarXiv

  5. 2023

    1. Contrastive Learning Relies More on Spatial Inductive Bias Than Supervised Learning: An Empirical StudyICCV 2023Yuanyi Zhong*, Haoran Tang*, Jun-Kun Chen*, Yu-Xiong Wang
    2. NeuralEditor: Editing Neural Radiance Fields via Manipulating Point CloudsCVPR 2023Jun-Kun Chen*, Jipeng Lyu*, Yu-Xiong Wang

  6. 2022

    1. PointTree: Transformation-Robust Point Cloud Encoder with Relaxed K-D TreesECCV 2022Jun-Kun Chen, Yu-Xiong Wang
    2. Is Self-Supervised Contrastive Learning More Robust Than Supervised Learning?ICML 2022 WorkshopYuanyi Zhong*, Haoran Tang*, Jun-Kun Chen, Jian Peng, Yu-Xiong Wang
    3. TorchDrug: A Powerful and Flexible Machine Learning Platform for Drug DiscoveryarXiv 2022Zhaocheng Zhu, Chence Shi, Zuobai Zhang, …, Jun-Kun Chen, …, Jian Tang

  7. 2021

    1. RNNLogic: Learning Logic Rules for Reasoning on Knowledge GraphsICLR 2021Meng Qu*, Jun-Kun Chen*, Louis-Pascal Xhonneux, Yoshua Bengio, Jian Tang

  8. 2020

    1. Pre-training Assisted Scene-aware Dialogue GenerationDSTC8 Workshop at AAAI 2020Jun-Kun Chen, Huang He, Siqi Bao, Fan Wang

* Equal contribution. Publications appear under Jun-Kun Chen; earlier papers may use Junkun Chen or J. Chen.

Current Research: Generative World Models

Research direction: generative world models that predict what happens next on a screen, in a physical scene, or under a robot's motion.

Training-data example: screen + tap → predicted next screen (middle) vs. real (right).

Computer-use visual world model

Image diffusion · SenseNova MoT · 2025–26 · with collaborators

Built a next-screen world model on SenseNova MoT: a language Reasoner describes what an action changes and what stays the same, and a diffusion Generator adapted via diffusion post-training renders the next screenshot. Ongoing work trains the Reasoner on teacher corrections of the model's own rendering mistakes.

PhysicsIQ example: past video → predicted future (middle) vs. real (right).

Physical-event world model

Video diffusion · Cosmos3 · 2025–26 · with collaborators

Adapted Cosmos3 video diffusion to predict physical events from past RGB frames and structured event descriptions. Implemented teacher distillation, where a teacher with full-video access supervises a language Reasoner that sees only past frames.

Held-out DROID scene: predicted (left) vs. real (right) interaction video.

Robot-interaction world model

Motion-conditioned video diffusion · Cosmos3 · 2025–26 · lead

Trained a motion-conditioned video world model on simulated and real robot data (MimicGen, RoboTwin, DROID, RH20T, AgiBot). It predicts robot–object interaction video from a starting scene, recent frames, and rendered robot motion. It conditions on a rendered robot-motion proxy plus paired motion–video examples from the target setup. Ongoing work studies adaptation to new robot and camera setups without retraining.

Experience

Meta Reality Labs

Research Scientist Intern Bay Area, CA, U.S. · Zurich, Switzerland

May–Dec. 2025Bay Area · part-time continuation Aug.–Dec.

Studied joint 3D and video generation, then explored video-to-3D and 3D-to-video generation. Designed and implemented a GPU-role-separated pipeline for data processing and model training at 128–256-GPU scale on Meta's distributed infrastructure.

May–Aug. 2023Zurich

Took ConsistDreamer (CVPR 2024) from method design through implementation and evaluation. Designed cross-view self-supervision, staged diffusion adaptation, and a 4-GPU asynchronous pipeline that overlaps diffusion training and image generation with scene fitting, for a 10× training speed-up.

SpreeAI

AI Research Intern Remote, U.S.

May 2024 – May 2025

Dress&Dance (WACV 2027): taught pretrained video diffusion new garment, identity, and motion controls. Designed and implemented target-steered supervision, training triplets with synthetic conditions and real targets, and staged image/video curricula. Evaluated garment and identity fidelity, motion, and video quality.

Virtual Fitting Room (NeurIPS 2025): designed training on arbitrary pairs of short clips to learn reference appearance and prefix continuation, without canonical 360° training anchors. At inference, a generated 360° reference and anchored autoregression maintain minute-scale consistency, while a refiner trained with targeted augmentation removes generator artifacts.

Mila – Quebec AI Institute

Research Intern Montreal, Canada

Feb.–Aug. 2020Knowledge-graph reasoning with Prof. Jian Tang. Co-first author of RNNLogic (ICLR 2021).

Baidu, NLP Department

Research / Development Intern Beijing, China

July 2019 – Jan. 2020Scene-aware dialogue generation. Top-3 system at DSTC8, presented at the DSTC8 Workshop at AAAI 2020.

Education

University of Illinois Urbana-Champaign

Ph.D. Candidate in Computer Science · 2021 – Dec. 2026 (expected)
Advisor: Yu-Xiong Wang

Tsinghua University

B.Eng. in Computer Science, Yao Class (IIIS) · 2017 – 2021
Major GPA: 3.94/4.0

Skills

Tools: PyTorch, Python, C++, FSDP2, torch.compile, Slurm, MuJoCo

Models: image/video diffusion, diffusion Transformers (DiT), video world models (NVIDIA Cosmos), unified multimodal models (mixture-of-transformers), vision-language models, LoRA

Practice: training diffusion models with rectified-flow / flow-matching objectives, diffusion post-training (supervised fine-tuning, LoRA, reward-based tuning with DiffusionNFT), VLM-teacher distillation into a language Reasoner, action and multimodal conditioning, video and robot-data pipelines, multi-GPU training with exact checkpoint resume, held-out benchmark design, human and VLM-judge evaluation, paired-bootstrap statistics

Selected Honors

  • ICPC EC-Final — Gold Medal, 4th place (2018)
  • ICPC Qingdao Regional — Gold Medal, 2nd place (2018)
  • CCPC Harbin Regional — Gold Medal (2017)
  • National Olympiad in Informatics — Gold Medal (2016)
  • Problem setter — 9 contest tasks, including hardest-tier problems at NOI 2018 and NOI 2019 (2018–2019)