CreateAI open-sources Ruyi-Mini-7B image-to-video model for consumer GPUs
- CreateAI released Ruyi, its first image-to-video model, and open-sourced the Ruyi-Mini-7B version on Hugging Face for use on consumer GPUs such as the RTX 4090.
- The roughly 7.1B-parameter DiT model was trained on about 200 million video clips and uses a causal VAE that compresses spatial resolution by 8x and temporal resolution by 4x.
- Ruyi accepts one input image and generates up to 5 seconds or 120 frames at resolutions from 384x384 to 1024x1024, including arbitrary aspect ratios.
- Users can set one of four motion amplitudes, select tilt-up, tilt-down, pan-left, pan-right, or static camera movement, and use as many as five starting and five ending keyframes.
- CreateAI reports current defects including hand deformation, facial-detail collapse in multi-person scenes, and uncontrollable scene transitions.