llama_flutter_android
0.2.6, released 4 months ago by an unverified publisher
Run llama.cpp GGUF models on Android via a Flutter plugin
What it does
This plugin lets Flutter apps load and run llama.cpp models on Android devices. It streams generated tokens, supports GPU acceleration, and exposes full generation parameters.
AI-written summary of version 0.2.6, based on the package's README and metadata. Report an inaccuracy
Use it when
- Need on-device LLM inference without external APIs
- Require real‑time token streaming for chat UIs
- Want to leverage Vulkan GPU for faster inference
Skip it when
- Target iOS or other platforms
- Need inference to survive backgrounding without a custom foreground service
- Deploy very large models that exceed device RAM
Strengths
- Android‑only GPU acceleration via Vulkan
- Fine‑grained control of sampling, penalties, and templates
- Stable streaming with guaranteed onDone termination
- No external network dependency
Weaknesses
- Only works on Android (iOS support missing)
- Requires NDK r27+ and ARM64 ABI
- No built‑in background service for long‑running inference
- Setup requires proguard rules and native library handling
Setup
- Kind
- Plugin (native code)
- Effort
- Medium
- Code generation
- No
- Direct dependencies
- 1
Add dependency, include proguard rules, ensure Android API 26+, NDK r27+, and optional GPU detection before loading model
import 'package:llama_flutter_android/llama_flutter_android.dart';
final controller = LlamaController();
await controller.loadModel(modelPath: '/path/to/model.gguf', threads: 4, contextSize: 2048);
final sub = controller.generate(prompt: 'Hello', maxTokens: 100).listen((token) => print(token), onDone: () => print('Done'));
await controller.stop();
await controller.dispose();Alternatives
Packages that do the same job and could replace llama_flutter_android.
- llamadart
On‑device LLM inference for Dart/Flutter across native and web
93Compare - genkit_llamadart
Genkit Dart plugin for local AI through llamadart: GGUF, LiteRT-LM, chat, tool calling, and embeddings
81Compare - nobodywho
Run local LLMs, vision, embeddings, RAG, tool calling, STT, and TTS on mobile and desktop.
80Compare - whisper_flutter_new
Use whisper.cpp speech-to-text models implementation for Android and iOS
50Compare
Works with
Packages commonly used together with llama_flutter_android: ones it builds on, ones built on it, and its own family.