Legal name: Junkun Chen · Publications as Jun-Kun Chen / J.-K. Chen
Ph.D. Candidate in Computer Science, University of Illinois Urbana-Champaign
I work on diffusion-based controllable image and video generation at UIUC, advised by Yu-Xiong Wang. I design the supervision, training data, and curricula that teach pretrained generative models new capabilities, including new visual controls, multi-view and temporal consistency, and minute-scale identity-consistent video.
I am now training visual world models through multimodal post-training. They predict the next screen after a UI action, physical events from observed video, and robot–object interactions under a prescribed motion.
Graduating Dec. 2026 Open to Research Scientist / Research Engineer roles starting 2027
Builds on Dress&Dance (D&D). Learns reference-conditioned generation from ordinary pairs of short clips. At inference, generated 360° anchors keep identity consistent in try-on videos up to 90 seconds long. Evaluated against D&D+FramePack and D&D+Kling 2.0 pipelines.
Teaches pretrained video diffusion new garment, identity, and motion controls through target-steered supervision and staged training. Generates 1152×720, 24-FPS try-on videos, achieving 20–59% lower FVD than a Kling 1.6 try-on-and-animation pipeline on Internet and captured benchmarks. Followed by Virtual Fitting Room, which extends this setting to minute-long videos.
Edits 3D scenes in simple progressive steps, with previews, edit-strength control, and tuning-free preservation of scene geometry and appearance.
Teaches 2D diffusion to edit 3D scenes consistently across views. Uses cross-view self-supervision and staged adaptation, with consistency losses along the full denoising trajectory. An asynchronous multi-GPU pipeline overlaps diffusion training and image generation with scene fitting.
The first instruction-driven 4D editing framework. Treating a dynamic scene as pseudo-3D breaks long 4D edits into tractable problems for 2D diffusion.
General-purpose, training-free editing of videos and 3D scenes. Hierarchical task decomposition separates instruction following from content preservation.
2027
2026
2025
2024
2023
2022
2021
2020
* Equal contribution. Publications appear under Jun-Kun Chen; earlier papers may use Junkun Chen or J. Chen.
Research direction: connect language reasoning with visual generation, using the model's predicted outcomes as training feedback.
Training-data example: screen + tap → predicted next screen (middle) vs. real (right).
Image diffusion · SenseNova MoT
Built a next-screen world model on SenseNova MoT: a language Reasoner describes what an action changes and what stays the same, and a diffusion Generator adapted via diffusion post-training renders the next screenshot. Ongoing work trains the Reasoner on teacher corrections of the model's own rendering mistakes.
PhysicsIQ example: past video → predicted future (middle) vs. real (right).
Video diffusion · Cosmos3
Adapted Cosmos3 video diffusion to predict physical events from past RGB frames and structured event descriptions. Implemented teacher distillation, where a teacher with full-video access supervises a language Reasoner that sees only past frames. An earlier online variant also updated the Generator from generated-versus-real comparisons.
Held-out DROID scene: predicted (left) vs. real (right) interaction video.
Motion-conditioned video diffusion · Cosmos3
Trained a motion-conditioned video world model on simulated and real robot data (MimicGen, RoboTwin, DROID, RH20T, AgiBot). It predicts robot–object interaction video from a starting scene, recent frames, and rendered robot motion. Ongoing work studies adaptation to new robot and camera setups without retraining.
May–Dec. 2025Bay Area · part-time continuation Aug.–Dec.
Designed and implemented a GPU-role-separated pipeline for data processing and model training at 128–256-GPU scale on Meta's distributed infrastructure, supporting 3D and video generation research.
May–Aug. 2023Zurich
Took ConsistDreamer (CVPR 2024) from method design through implementation and evaluation. Designed cross-view self-supervision, staged diffusion adaptation, and an asynchronous pipeline that overlaps diffusion training and image generation with scene fitting.
May 2024 – May 2025
Dress&Dance (WACV 2027): taught pretrained video diffusion new garment, identity, and motion controls. Designed and implemented target-steered supervision, training triplets with synthetic conditions and real targets, and staged image/video curricula. Evaluated garment and identity fidelity, motion, and video quality.
Virtual Fitting Room (NeurIPS 2025): designed training on arbitrary pairs of short clips to learn reference appearance and prefix continuation, without canonical 360° training anchors. At inference, a generated 360° reference and anchored autoregression maintain minute-scale consistency, while a refiner trained with targeted augmentation removes generator artifacts.
Feb.–Aug. 2020Knowledge-graph reasoning with Prof. Jian Tang. Co-first author of RNNLogic (ICLR 2021).
July 2019 – Jan. 2020Scene-aware dialogue generation. Top-3 system at DSTC8, presented at the DSTC8 Workshop at AAAI 2020.
Ph.D. Candidate in Computer Science · 2021 – Dec. 2026 (expected)
Advisor: Yu-Xiong Wang
B.Eng. in Computer Science, Yao Class (IIIS) · 2017 – 2021
Major GPA: 3.94/4.0
Tools: PyTorch, Python, C++, FSDP2, torch.compile, Slurm, MuJoCo
Models: image/video diffusion, diffusion Transformers (DiT), video world models (NVIDIA Cosmos), unified multimodal models (mixture-of-transformers), vision-language models, LoRA
Practice: training diffusion models with rectified-flow / flow-matching objectives, diffusion post-training (supervised fine-tuning, LoRA, reward-based tuning with DiffusionNFT), VLM-teacher distillation into a language Reasoner, action and multimodal conditioning, video and robot-data pipelines, multi-GPU training with exact checkpoint resume, held-out benchmark design, human and VLM-judge evaluation, paired-bootstrap statistics