Skip to content
pubscout

flutter_litert_lm

0.3.0, released 7 months ago by an unverified publisher

Run Large Language Models on-device using Google's LiteRT-LM runtime.

WASM-ready
$ flutter pub add flutter_litert_lm
Source repository

What it does

A Flutter plugin providing local on-device inference for Large Language Models using Google's LiteRT-LM runtime. It supports hardware acceleration via CPU, GPU (OpenCL), and NPU, offering streaming chat, multi-turn conversations, multimodal inputs, and tool calling without network dependencies.

AI-written summary of version 0.3.0, based on the package's README and metadata. Report an inaccuracy

Use it when

  • You need offline LLM execution in your app.
  • Prompts and data must remain strictly on-device for privacy.
  • You want to avoid per-token API costs and rate limits.

Skip it when

  • Your target devices have insufficient RAM to load large model files.
  • You require cloud-scale models that cannot fit on mobile hardware.

Strengths

  • Fully offline execution with no API keys or token billing
  • Hardware acceleration via CPU, GPU (OpenCL), and NPU
  • Supports streaming responses, multimodal inputs, and tool calling

Weaknesses

  • High memory usage can lead to out-of-memory crashes on large models
  • iOS requires a lengthy one-time local Bazel build process
  • iOS runtime currently limited to CPU backend only

Setup

Kind
Plugin (native code)
Effort
High
Code generation
No
Direct dependencies
1

Requires downloading large model files (.litertlm) and setting up writable storage paths. iOS builds require installing Bazelisk and running a manual framework build script that takes 30-60 minutes on first run.

import 'package:flutter_litert_lm/flutter_litert_lm.dart';

final engine = await LiteLmEngine.create(
  LiteLmEngineConfig(
    modelPath: '/storage/.../model.litertlm',
    backend: LiteLmBackend.gpu,
  ),
);

final conversation = await engine.createConversation();
final reply = await conversation.sendMessage('What is Flutter?');
print(reply.text);

await conversation.dispose();
await engine.dispose();

Alternatives

Packages that do the same job and could replace flutter_litert_lm.

  • llamadart

    Alternative on-device LLM solution if you prefer llama.cpp integration.

  • firebase_ai

    If you prefer cloud-hosted Gemini models instead of running models locally.

Works with

Packages commonly used together with flutter_litert_lm: ones it builds on, ones built on it, and its own family.