Legal name: Junkun Chen · Publications as Jun-Kun Chen / J.-K. Chen
Ph.D. Candidate in Computer Science, University of Illinois Urbana-Champaign
I work on diffusion-based controllable image and video generation at UIUC, advised by Yu-Xiong Wang. I design the supervision, training data, and curricula that teach pretrained generative models new capabilities. So far these include new visual controls, multi-view and temporal consistency, and identity-consistent video at minute scale.
I am now training visual world models through multimodal post-training. They predict the next screen after a UI action, physical events from observed video, and robot–object interactions under a prescribed motion.
Graduating Dec. 2026 Open to Research Scientist / Research Engineer roles starting 2027
Learns reference-conditioned generation from ordinary pairs of short clips. At inference, generated 360° anchors keep identity consistent in try-on videos up to 90 seconds long. Evaluated against D&D+FramePack and D&D+Kling 2.0 pipelines.
Edits 3D scenes in simple progressive steps, with previews, edit-strength control, and tuning-free preservation of scene geometry and appearance.
Teaches 2D diffusion to edit 3D scenes consistently across views. Uses cross-view self-supervision and staged adaptation, with consistency losses along the whole denoising trajectory. An asynchronous multi-GPU pipeline overlaps diffusion training and image generation with scene fitting.
The first instruction-driven 4D editing framework. It treats a dynamic scene as a pseudo-3D scene, which breaks a long 4D edit into tractable problems for 2D diffusion.
General-purpose, training-free editing of videos and 3D scenes. A hierarchical task decomposition separates following the instruction from preserving the original content.
Teaches pretrained video diffusion new garment, identity, and motion controls through target-steered supervision and staged training. Generates 1152×720, 24-FPS try-on videos. Scores 20–59% lower FVD than a Kling 1.6 try-on-and-animation pipeline on Internet and captured benchmarks.
2026
2025
2024
2023
2022
2021
2020
* Equal contribution. Publications appear under Jun-Kun Chen; earlier papers may use Junkun Chen or J. Chen.
Research direction: connect language reasoning with visual generation, and use the outcomes the model predicts as training feedback.
Training-data example: screen + tap → predicted next screen (middle) vs. real (right).
Image diffusion · SenseNova MoT
Built a next-screen world model on SenseNova MoT: a language Reasoner describes what an action changes and what must stay the same, and a diffusion Generator adapted through diffusion post-training renders the next screenshot. Ongoing work trains the Reasoner on a teacher's corrections of the model's own rendering mistakes.
PhysicsIQ example: past video → predicted future (middle) vs. real (right).
Video diffusion · Cosmos3
Adapted Cosmos3 video diffusion to predict physical events from past RGB frames, conditioned on structured event descriptions. Implemented teacher distillation: a teacher that sees the whole video supervises a language Reasoner that sees only the past frames. An earlier online variant also updated the Generator from generated-versus-real comparisons.
Held-out DROID scene: predicted (left) vs. real (right) interaction video.
Motion-conditioned video diffusion · Cosmos3
Trained a motion-conditioned video world model on simulated and real robot data (MimicGen, RoboTwin, DROID, RH20T, AgiBot). It predicts robot–object interaction video from a starting scene, recent frames, and a rendering of the prescribed robot motion. Ongoing work studies adaptation to new robot and camera setups without retraining.
May–Dec. 2025Bay Area · part-time continuation Aug.–Dec.
Designed and implemented a GPU-role-separated pipeline for data processing and model training at 128–256-GPU scale. Built on Meta's distributed infrastructure to support 3D/video-generation research.
May–Aug. 2023Zurich
Took ConsistDreamer (CVPR 2024) from method design through implementation and evaluation. Designed cross-view self-supervision and staged diffusion adaptation. Built an asynchronous pipeline that overlaps diffusion training and image generation with scene fitting.
May 2024 – May 2025
Dress&Dance: taught pretrained video diffusion new garment, identity, and motion controls. Designed and implemented target-steered supervision, training triplets with synthetic conditions and real targets, and staged image/video curricula. Evaluated garment and identity fidelity, motion, and video quality.
Virtual Fitting Room (NeurIPS 2025): designed training on arbitrary pairs of short clips. The model learns reference appearance and prefix continuation, with no canonical 360° anchors needed for training. At inference, a generated 360° reference and anchored autoregression keep the video consistent at minute scale. Targeted augmentation trains a refiner to remove the generator's typical artifacts.
Feb.–Aug. 2020Knowledge-graph reasoning with Prof. Jian Tang. Co-first author of RNNLogic (ICLR 2021).
July 2019 – Jan. 2020Scene-aware dialogue generation. Top-3 system at DSTC8, presented at the DSTC8 Workshop at AAAI 2020.
Ph.D. Candidate in Computer Science · 2021 – Dec. 2026 (expected)
Advisor: Yu-Xiong Wang
B.Eng. in Computer Science, Yao Class (IIIS) · 2017 – 2021
Major GPA: 3.94/4.0
Tools: PyTorch, Python, C++, FSDP2, torch.compile, Slurm, MuJoCo
Models: image/video diffusion, diffusion Transformers (DiT), video world models (NVIDIA Cosmos), unified multimodal models (mixture-of-transformers), vision-language models, LoRA
Practice: training diffusion models with rectified-flow / flow-matching objectives, diffusion post-training (supervised fine-tuning, LoRA, online teacher distillation, reward-based tuning with DiffusionNFT), action and multimodal conditioning, video and robot-data pipelines, multi-GPU training with exact checkpoint resume, held-out benchmark design, human and VLM-judge evaluation, paired-bootstrap statistics