Z-Image ncnn Vulkan

:exclamation: :exclamation: :exclamation: This software is in the early development stage, it may bite your cat

ncnn implementation of Z-Image image generator, with LanPaint inpaint/outpaint and ControlNet support.

zimage-ncnn-vulkan uses ncnn project as the universal neural network inference framework.

Download

Download Windows/Linux/macOS Executable for Intel/AMD/NVIDIA/Apple-Silicon GPU

https://github.com/nihui/zimage-ncnn-vulkan/releases

This package includes all the binaries required. It is portable, so no CUDA or PyTorch runtime environment is needed :)

prepare model files

Download the z-image-turbo and z-image model folders to the same directory as the executable file

https://huggingface.co/nihui-szyl/z-image-ncnn/tree/main

For ControlNet, download the z-image-control model folder to the same directory as z-image-turbo. For Tile ControlNet upscale, download the z-image-control-tile model folder too.

zimage-ncnn-vulkan
z-image-turbo/
z-image-control/
z-image-control-tile/

About Z-Image

Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer

https://github.com/Tongyi-MAI/Z-Image

Usages

requirements

• Minimum (Linux / macOS): 16GB RAM, any Vulkan capable GPU

• Minimum (Windows):

  Due to WDDM limitations: Vulkan applications can only use half of the system RAM.

  The following condition shall be met: (Half of system RAM) + (GPU memory) >= 16GB

  Examples of valid combinations:

  • Any amount RAM, 16GB dedicated GPU
  • 16GB RAM, 8GB dedicated GPU
  • 24GB RAM, 4GB dedicated GPU
  • 32GB RAM, any Vulkan capable GPU

• Recommended: 32GB RAM, 16GB dedicated GPU with tensorcore/matrix hardware

Example Command

Text to image

zimage-ncnn-vulkan.exe -p "風的彷徨." -o output.png

LanPaint inpaint

zimage-ncnn-vulkan.exe -i input.png -k mask.png -p "a cute girl" -o output.png

LanPaint outpaint

zimage-ncnn-vulkan.exe -i input.png -x 128,128,128,128 -p "a cute girl" -o output.png

ControlNet

zimage-ncnn-vulkan.exe -c pose.png -p "a cute girl, full body" -o output.png

Tile ControlNet upscale

zimage-ncnn-vulkan.exe -c lowres.png -t -s 2048,2048 -p "a cute girl" -o output.png

Full Usages

Usage: zimage-ncnn-vulkan -p prompt -o outfile [options]...

  -h                   show this help
  -p prompt            prompt (default=rand)
  -n negative-prompt   negative prompt (optional)
  -o output-path       output image path (default=out.png)
  -i input-image       input image for inpaint (optional)
  -k mask-image        inpaint mask, white=paint black=preserve (optional)
  -x l,t,r,b           outpaint by expanding input canvas (optional)
  -c control-image     control image for ControlNet (optional)
  -w control-scale     ControlNet scale (default=1.0)
  -t                   Tile ControlNet upscale mode (optional)
  -s image-size        image resolution (default=1024,1024)
  -l steps             denoise steps (default=auto)
  -r random-seed       random seed (default=rand)
  -m model-path        z-image model path (default=z-image-turbo)
  -g gpu-id            gpu device to use (-1=cpu, default=auto)
  -b batch-size        batched generation (default=1)

LanPaint inpaint/outpaint requires -i input-image. Inpaint uses -k mask-image, where white means repaint and black means preserve. Outpaint uses -x left,top,right,bottom.

ControlNet uses -c control-image. The control image size must match the output image size. ControlNet currently requires the z-image-turbo model and the z-image-control model folder.

Tile ControlNet upscale uses -c lowres-image -t -s target-width,target-height. The low resolution image is resized internally to the target size and used as tile control. It loads ControlNet weights from z-image-control-tile.

If you encounter a crash or error, try upgrading your GPU driver:

• Intel: https://downloadcenter.intel.com/product/80939/Graphics-Drivers
• AMD: https://www.amd.com/en/support
• NVIDIA: https://www.nvidia.com/Download/index.aspx

Build from Source

1. Clone this project with all submodules

git clone https://github.com/nihui/zimage-ncnn-vulkan.git
cd zimage-ncnn-vulkan
git submodule update --init --recursive --depth 1

2. Build with CMake

mkdir build
cd build
cmake ../src
cmake --build . -j 4

Sample Images

<table width="100%"> <tr> <td width="50%"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -p "A half-length portrait in the warm light of a convenience store late at night. An East Asian beauty, holding milk, meets your gaze in front of the freezer." -r 2065820720

</details>

</td> <td width="50%"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -p "A Suzhou garden at night under a bright autumn moon, silvery light bathing the pavilions and rocks, paper lanterns glowing warmly along the corridors, the pond reflecting the moon and lantern light, cool color palette with accents of warmth, mysterious and poetic atmosphere." -r 1593388185

</details>

</td> </tr> <tr> <td width="50%"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -p "Cinnamoroll, Sanrio character, white puppy with long floppy ears, blue eyes, chubby cheeks, plump tail like a cinnamon roll, simple blue bow on right ear, high-quality anime figure, prize figure style, seamless joints, glossy paint finish, detailed sculpting, sitting pose with head tilted, innocent expression, standing on a fluffy cloud-shaped display base, soft gradient pastel background, studio lighting, product photography, sharp focus, shallow depth of field" -r 1362178531

</details>

</td> <td width="50%"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -p "a cute girl taking a mirror selfie, full-body view in a full-length mirror, holding a large smartphone in front of her face, the phone screen is visible and glowing, slightly obscuring her eyes or lower face, long twin tails, soft makeup, blush, wearing a trendy and cute outfit, in a cozy, clean bedroom with soft lighting, mirror may have a decorative frame, shallow depth of field, digital photo style, cinematic lighting, sharp focus on the phone and reflection, natural skin texture, Instagram aesthetic, candid feeling" -r 2029158043

</details>

</td> </tr> </table> <table width="100%"> <tr> <td width="33%"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -p "風的彷徨." -r 77

</details>

</td> <td width="33%"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -p "風的彷徨." -r 777

</details>

</td> <td width="34%"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -p "Anya Forger hugging a large white dog (Bond), both looking happy, in a park, sunny day, grass field, action shot, from Spy x Family" -r 499008670

</details>

</td> </tr> <tr> <td width="100%" colspan=3> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -p "traditional Chinese handscroll painting in the style of “清明上河图”, horizontal panorama, bustling riverside cityscape in Northern Song Dynasty, Bianliang city, with Rainbow Bridge at the center, boats on the river, shops along the street, teahouses, inns, hundreds of diverse people engaged in daily activities, travelers, merchants, scholars, entertainers, children playing, ink and color on silk, fine brushwork for architecture and figures, subtle expressive brushwork for trees and landscape, soft natural colors, high aerial viewpoint, continuous narrative, lively, prosperous, detailed, harmonious, capturing a complete day of city life, extremely wide aspect ratio, vanishing perspective, Chinese seals and calligraphy on the margins, slight aged paper texture" -r 1321390901 -s 2048,512

</details>

</td> </tr> </table>

LanPaint inpaint

<table width="100%"> <tr> <td colspan="4"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -i 2065820720.jpg -p "Holding a Hachiware plushie, the friend of both Chiikawa and Usagi" -k mask.jpg

</details> </td> </tr> <tr> <td width="60%" valign="middle"><img src="images/2065820720.jpg" width="50%"><img src="images/mask.jpg" width="50%"></td> <td width="1%" align="center" valign="middle">➡️</td> <td width="30%" valign="middle"><img src="images/inpaint.jpg" width="100%"></td> </tr> </table>

LanPaint outpaint

<table width="100%"> <tr> <td colspan="4"> <details> <summary>expand for full command</summary>

zimage-ncnn-vulkan.exe -i 2065820720.jpg -p "A half-length portrait in the warm light of a convenience store late at night. An East Asian beauty, holding milk, meets your gaze in front of the freezer." -x 128,128,128,128

</details> </td> </tr> <tr> <td width="40%" valign="middle"><img src="images/2065820720.jpg" width="100%"></td> <td width="8%" align="center" valign="middle">➡️</td> <td width="50%" valign="middle"><img src="images/outpaint.jpg" width="100%"></td> </tr> </table>

ControlNet pose

<table width="100%"> <tr> <td colspan="3"> <details> <summary>expand for full command</summary>