Skip to main content

Model Settings

What is Model Settings?​

Model Settings lets you choose how the AI model runs on your system. This includes selecting a model, choosing a variant optimized for your hardware, and adjusting performance-related options.

Choosing a model​

Select the model you want to use from your installed models.

Different models offer different styles, quality levels, and performance characteristics. If you're unsure, start with a recommended model.

Model variants​

When you install a model, you install one variant at a time. Each variant is an optimized version of the model designed for specific hardware and performance needs.

In Model Settings, you will only see the variants that you have installed.

If needed, you can install additional variants of the same model from the Model Hub and switch between them.

Choosing the right variant affects:

  • Performance (speed)
  • Memory usage
  • Compatibility with your system

If you're unsure, choose a variant that is marked as compatible with your system. You can install additional variants later if needed.

If a configuration is not working well or is slow, try installing a different variant of the same model.

Common variant types​

You may see different variant types when selecting a model. These are optimized for different hardware and performance needs.

CUDA / PyTorch (NVIDIA GPUs)​

Optimized for NVIDIA graphics cards. This is usually the fastest option if you have a supported GPU.

  • Prefer BF16 over FP16 if the model and your system support it, for the best quality.
  • Enable reduced GPU memory usage if you encounter memory issues.

ONNX (DirectML / AMD GPUs)​

Optimized for AMD GPUs and compatible systems.

  • Works across a wide range of AMD hardware.
  • On some systems, increasing GPU memory allocation in the BIOS can improve performance.

OpenVINO FP16 (Intel GPUs and CPU)​

Optimized for Intel hardware and also works well on CPUs.

  • Suitable for Intel GPUs.
  • Recommended option when using CPU (Intel or AMD).

OpenVINO INT8 (select Intel CPUs)​

Optimized for performance on supported Intel CPUs.

  • Faster than FP16 in many cases.
  • May slightly reduce image quality compared to FP16.

Processing methods​

Depending on the selected variant, the app will propose available processing methods (such as CPU or GPU) and check whether your selection is compatible.

  • GPU: Faster performance, recommended when available
  • CPU: More compatible, but generally slower

Additional processing methods may be available depending on your hardware.

Compatibility and recommendations​

The app evaluates your configuration and provides guidance based on your hardware.

  • Some configurations may not be supported
  • Some may work but require more memory than available
  • Compatible options are highlighted to help you choose

If you're unsure, follow the compatible options shown in the app.

Performance and quality​

Different configurations offer different trade-offs:

  • GPU vs CPU: GPU is faster, CPU is more widely supported
  • FP16 vs INT8: FP16 typically provides higher quality, INT8 can be faster
  • Model choice: Some models prioritize speed, others prioritize quality

You can experiment with different configurations to find what works best for your needs.

Advanced options​

Some models provide additional options to improve performance or stability.

Reduce GPU memory usage​

  • Helps run models on systems with limited GPU memory
  • May reduce performance slightly

These options may be enabled automatically when needed, but you can adjust them manually.

Tips​

  • Start with compatible options if you're unsure.
  • Prefer GPU-based variants when available for better performance.
  • If something is slow or not working, try a different variant or processing method.
  • Small configuration changes can significantly affect results.