flutter_litert_lm
0.3.0, released 7 months ago by an unverified publisher
Run Large Language Models on-device using Google's LiteRT-LM runtime.
What it does
A Flutter plugin providing local on-device inference for Large Language Models using Google's LiteRT-LM runtime. It supports hardware acceleration via CPU, GPU (OpenCL), and NPU, offering streaming chat, multi-turn conversations, multimodal inputs, and tool calling without network dependencies.
AI-written summary of version 0.3.0, based on the package's README and metadata. Report an inaccuracy
Use it when
- You need offline LLM execution in your app.
- Prompts and data must remain strictly on-device for privacy.
- You want to avoid per-token API costs and rate limits.
Skip it when
- Your target devices have insufficient RAM to load large model files.
- You require cloud-scale models that cannot fit on mobile hardware.
Strengths
- Fully offline execution with no API keys or token billing
- Hardware acceleration via CPU, GPU (OpenCL), and NPU
- Supports streaming responses, multimodal inputs, and tool calling
Weaknesses
- High memory usage can lead to out-of-memory crashes on large models
- iOS requires a lengthy one-time local Bazel build process
- iOS runtime currently limited to CPU backend only
Setup
- Kind
- Plugin (native code)
- Effort
- High
- Code generation
- No
- Direct dependencies
- 1
Requires downloading large model files (.litertlm) and setting up writable storage paths. iOS builds require installing Bazelisk and running a manual framework build script that takes 30-60 minutes on first run.
import 'package:flutter_litert_lm/flutter_litert_lm.dart';
final engine = await LiteLmEngine.create(
LiteLmEngineConfig(
modelPath: '/storage/.../model.litertlm',
backend: LiteLmBackend.gpu,
),
);
final conversation = await engine.createConversation();
final reply = await conversation.sendMessage('What is Flutter?');
print(reply.text);
await conversation.dispose();
await engine.dispose();Alternatives
Packages that do the same job and could replace flutter_litert_lm.
- llamadart
Alternative on-device LLM solution if you prefer llama.cpp integration.
93Compare - firebase_ai
If you prefer cloud-hosted Gemini models instead of running models locally.
86Compare
Works with
Packages commonly used together with flutter_litert_lm: ones it builds on, ones built on it, and its own family.