LivePortrait

LivePortrait

by Kuaishou
LivePortrait is an open-source framework by Kuaishou for efficiently transferring expressions and poses from videos to portraits, creating expressive animations.

What is LivePortrait?

LivePortrait is an open-source portrait animation generation framework developed by Kuaishou. It efficiently transfers expressions and poses from driving videos to static or dynamic portraits, creating expressive videos. The framework uses an implicit keypoint system, leveraging large-scale, high-quality training data and mixed training strategies to enhance generalization and motion control.

Main Features of LivePortrait

  • Expression and Pose Transfer: Transfer expressions and poses from driving videos to portraits in real-time.
  • High Efficiency: Achieves a single-frame generation speed of 12.8 milliseconds on an RTX 4090 GPU.
  • Generalization Ability: Adapts to portraits of different styles and identities.
  • Controllability: Uses implicit keypoints and a lightweight MLP network for enhanced control.
  • Multi-Style Support: Handles realistic and stylized (e.g., anime) portraits.
  • High-Resolution Animation: Supports generating high-resolution animations.
  • Alignment and Redirection Modules: Adapts to complex scenes such as cropping and group photos.

Technical Principles of LivePortrait

  • Basic Model Training: Optimizes appearance extractor, motion extractor, warping module, and decoder.
  • Alignment and Redirection Module Training: Freezes basic model components and optimizes alignment and redirection modules.
  • Video-Image Mixed Training: Treats each image as a frame of a video segment for training.
  • Upgraded Network Structure: Unifies keypoint estimation networks into a single model.
  • Keypoint-Guided Implicit Keypoint Optimization: Captures micro-expressions using 2D keypoints.
  • Cascaded Loss Function: Improves texture quality using perceptual and GAN losses.

Project Addresses of LivePortrait

Application Scenarios of LivePortrait

  • Social Media Content Creation: Convert photos or videos into dynamic content for social media.
  • Virtual Hosts and Live Streaming: Create virtual avatars for live streaming or video production.
  • Film and Animation Production: Improve production efficiency and reduce costs.
  • Advertising and Marketing: Create attractive advertising videos with dynamic character images.

Framework Features

Supported Tasks
Portrait Animation Expression Transfer Pose Transfer High-Resolution Video Generation
Tags
Animation Portrait Open-Source AI Video Generation Expression Transfer Pose Transfer High-Resolution Multi-Style Real-Time

Getting Started

Pricing
free

Screenshots & Images

Primary Screenshot
Additional Images
View Repository View Demo Documentation

Stats

0 Views
0 Favorites
14462 GitHub Stars

Community & Support

GitHub Repository

Similar Frameworks

TPO
0
Phantom by ByteDance
0
AgentSociety by Tsinghua University
0

Recently Viewed

DeepEP Framework