Skip to content
pubscout

llama_flutter_android

0.2.6, released 4 months ago by an unverified publisher

Run llama.cpp GGUF models on Android via a Flutter plugin

WASM-ready
$ flutter pub add llama_flutter_android
Source repository

What it does

This plugin lets Flutter apps load and run llama.cpp models on Android devices. It streams generated tokens, supports GPU acceleration, and exposes full generation parameters.

AI-written summary of version 0.2.6, based on the package's README and metadata. Report an inaccuracy

Use it when

  • Need on-device LLM inference without external APIs
  • Require real‑time token streaming for chat UIs
  • Want to leverage Vulkan GPU for faster inference

Skip it when

  • Target iOS or other platforms
  • Need inference to survive backgrounding without a custom foreground service
  • Deploy very large models that exceed device RAM

Strengths

  • Android‑only GPU acceleration via Vulkan
  • Fine‑grained control of sampling, penalties, and templates
  • Stable streaming with guaranteed onDone termination
  • No external network dependency

Weaknesses

  • Only works on Android (iOS support missing)
  • Requires NDK r27+ and ARM64 ABI
  • No built‑in background service for long‑running inference
  • Setup requires proguard rules and native library handling

Setup

Kind
Plugin (native code)
Effort
Medium
Code generation
No
Direct dependencies
1

Add dependency, include proguard rules, ensure Android API 26+, NDK r27+, and optional GPU detection before loading model

import 'package:llama_flutter_android/llama_flutter_android.dart';

final controller = LlamaController();
await controller.loadModel(modelPath: '/path/to/model.gguf', threads: 4, contextSize: 2048);
final sub = controller.generate(prompt: 'Hello', maxTokens: 100).listen((token) => print(token), onDone: () => print('Done'));
await controller.stop();
await controller.dispose();

Alternatives

Packages that do the same job and could replace llama_flutter_android.

  • llamadart

    On‑device LLM inference for Dart/Flutter across native and web

  • genkit_llamadart

    Genkit Dart plugin for local AI through llamadart: GGUF, LiteRT-LM, chat, tool calling, and embeddings

  • nobodywho

    Run local LLMs, vision, embeddings, RAG, tool calling, STT, and TTS on mobile and desktop.

  • whisper_flutter_new

    Use whisper.cpp speech-to-text models implementation for Android and iOS

Works with

Packages commonly used together with llama_flutter_android: ones it builds on, ones built on it, and its own family.