What is CSGO?

CSGO (Content-Style Composition in Text-to-Image Generation) is a research project on image style transfer and text-to-image generation, developed in collaboration by Nanjing University of Science and Technology, Xiaohongshu, and other institutions. The goal of CSGO is to provide users with richer and more diverse image creation tools. The project introduces an innovative data construction process for generating and cleaning stylized data triplets and builds a large-scale style transfer dataset called IMAGStyle. Based on this dataset, the CSGO framework achieves image-driven style transfer, text-driven stylized synthesis, and text-editing-driven stylized synthesis through end-to-end training, significantly enhancing style control in image generation.

Key Features of CSGO

• Image-Driven Style Transfer: Users can apply the style of one image to another, achieving a visual style transformation while maintaining the semantic content of the original image.
• Text-Driven Stylized Synthesis: By inputting text descriptions, users can generate images with specific styles, demonstrating the ability to understand natural language and convert text into visual styles.
• Text-Editing-Driven Stylized Synthesis: After generating an image, users can further adjust the style by editing the text description, providing a higher level of creative control.
• End-to-End Training Model: CSGO adopts an end-to-end training method, where the model learns from input to output in a continuous process, eliminating the need for staged processing and improving model efficiency and effectiveness.
• Feature Injection Technology: Through independent feature injection technology, CSGO explicitly decouples content and style features, extracting and fusing them into the generated image to ensure content accuracy and style consistency.

Technical Principles of CSGO

• Data Construction Process: CSGO generates and cleans stylized data triplets through an automated data construction process. The triplets include content images, style images, and corresponding stylized result images.
• End-to-End Training Model: CSGO employs an end-to-end training method, where the model learns directly from input to output, improving efficiency and effectiveness.
• Feature Injection Technology:
• Content Control: Uses pre-trained ControlNet and additional learnable cross-attention layers to inject content features into the base model, preserving the semantic and layout of the original content.
• Style Control: Extracts style features through a pre-trained image encoder and style projection layers, injecting them into the model's upsampling blocks and independent style control modules.
• Diffusion Model: CSGO utilizes a diffusion model to gradually remove noise and generate images, applying one style to a content image while maintaining content integrity.
• Content Alignment Score (CAS): CSGO introduces the Content Alignment Score to measure the consistency between the generated image and the original content image, evaluating the quality of style transfer.

Project Links

• Official Website: csgo-gen.github.io
• GitHub Repository: https://github.com/instantX-research/CSGO
• HuggingFace Model Library: https://huggingface.co/spaces/xingpng/CSGO
• arXiv Technical Paper: https://arxiv.org/pdf/2408.16766

Application Scenarios

• Artistic Creation: Artists and designers use CSGO to explore new artistic styles, create unique digital artworks, or experiment with different visual expressions while maintaining the content theme.
• Digital Entertainment: In game development and film production, CSGO is used to generate scenes and character concept art with specific styles, providing diverse visual elements for digital content creation.
• Design Industry: Designers use CSGO to quickly generate design sketches and prototypes, showcasing product designs through different stylized images or rapidly iterating and testing various visual styles during the design process.
• Advertising and Marketing: Marketers use CSGO to generate attractive visual content for advertisements, stylizing product images to appeal to target audiences or customizing unique visual styles based on brand identity.
• Social Media Content Creation: Content creators and influencers use CSGO to create stylized content for social media platforms like Instagram and Xiaohongshu, enhancing visual appeal and personalized expression.