Text Generation Web UI

A comprehensive Gradio interface for running large language models locally with support for multiple backends like Transformers, llama.cpp, and ExLlamaV2.

This framework serves as the industry standard for local LLM orchestration (the AUTOMATIC1111 of text generation). It provides a unified dashboard to swap between engines like AutoGPTQ and AutoAWQ while managing complex parameters including RoPE scaling, 4-bit quantization, and LoRA integration. Users can toggle between chat, notebook, and API modes to test models such as Llama 3 or Mistral 7B with granular control over sampling techniques and character templates.

https://github.com/oobabooga/text-generation-webui