llama_flutter_android

A high-performance Flutter plugin for running Llama 2 and Llama 3 models locally on Android via the llama.cpp C++ library.

This implementation bridges Dart and C++ to execute large language models directly on Android hardware. It utilizes the llama.cpp backend to ensure memory efficiency and hardware acceleration (GGUF format support). Developers can integrate offline inference into mobile apps with minimal overhead, leveraging specific optimizations for ARM-based processors and local storage. The package includes a clean API for model loading, tokenization, and real-time text generation.

https://github.com/netdur/llama_flutter_android