Skip to content
pubscout

llamadart

0.20.2, released yesterday by leehack.com

On‑device LLM inference for Dart/Flutter across native and web

WASM-ready
$ flutter pub add llamadart
Source repository

What it does

Runs GGUF (llama.cpp) and LiteRT‑LM models locally, providing streaming chat, embeddings, multimodal input, and structured JSON output. It supports native platforms and experimental web support via WebGPU.

AI-written summary of version 0.20.1, based on the package's README and metadata. Report an inaccuracy

Use it when

  • Need offline, no‑network LLM inference in a Flutter app
  • Want streaming chat or embeddings without a server
  • Require multimodal (audio, image) input or text‑to‑speech
  • Target multiple platforms (Android, iOS, macOS, Windows, Linux, web)

Skip it when

  • Relying on GPU‑heavy models on low‑end devices
  • Using web exclusively where experimental support may be unstable
  • Need cloud‑based inference or large model hosting

Strengths

  • Offline, no network required
  • Cross‑platform support (native + experimental web)
  • Streaming API and structured JSON output
  • Multimodal capabilities (audio, image, TTS)

Weaknesses

  • Complex native setup (runtime packages, build hooks)
  • Web support is experimental
  • Requires native C++ toolchain resolution on first run
  • Large model binaries increase app size

Setup

Kind
FFI (native library)
Effort
Medium
Code generation
No
Direct dependencies
12

Add companion runtime packages (llamadart_llama_cpp_flutter, llamadart_litert_lm_flutter, etc.), run `dart run llamadart:web_setup` for web, ensure iOS/macOS deployment targets, and include runtime assets via build hook.

import 'package:llamadart/llamadart.dart';

Future<void> main() async {
  final engine = await LlamaEngine.load(
    LlamaModel(
      ModelSource.parse(
        'hf://unsloth/SmolLM2-135M-Instruct-GGUF/SmolLM2-135M-Instruct-Q2_K.gguf',
      ),
    ),
  );
  try {
    final reply = await engine.create(
      const [
        LlamaChatMessage.fromText(
          role: LlamaChatRole.user,
          text: 'Explain local inference in one sentence.',
        ),
      ],
      params: const GenerationParams(maxTokens: 48),
    ).text();
    print(reply);
  } finally {
    await engine.dispose();
  }
}

Alternatives

Packages that do the same job and could replace llamadart.

Works with

Packages commonly used together with llamadart: ones it builds on, ones built on it, and its own family.