flutter_edge_ai_litertlm
1.20.2, released 3 days ago by sashadenisov.dev
LiteRT-LM on-device engine for flutter_edge_ai supporting .litertlm models and embeddings.
What it does
Provides a LiteRT-LM engine implementation for the flutter_edge_ai package, running .litertlm models, EmbeddingGemma 2 embeddings, and LiteRT .tflite embeddings across multiple native platforms and web via FFI. It handles sessions, streaming, structured outputs, and platform-specific execution on CPU, GPU, and NPU.
AI-written summary of version 1.20.0, based on the package's README and metadata. Report an inaccuracy
Use it when
- You are using flutter_edge_ai and need to run .litertlm language models or embeddings on-device.
- You want support for structured outputs (JSON schema) via LiteRT-LM constrained decoding.
- You need multi-platform on-device inference across Android, iOS, web, Windows, macOS, and Linux.
Skip it when
- You are not using flutter_edge_ai in your application.
- Your application relies strictly on cloud-based LLM APIs and does not require local on-device inference.
Strengths
- Runs models locally across various hardware targets including CPU, GPU, and NPU.
- Supports structured output via JSON schema validation.
- Integrated deeply with the flutter_edge_ai ecosystem and model catalog.
Weaknesses
- Requires strict version matching and upgrading of multiple dependent flutter_edge_ai packages together.
- NPU and certain platforms have specific limitations regarding session sharing, samplers, or context windows.
Setup
- Kind
- Plugin (native code)
- Effort
- High
- Code generation
- No
- Direct dependencies
- 7
Requires pairing with flutter_edge_ai, registering the LiteRtLmEngine during initialization, and adhering to strict package version requirements alongside other edge AI packages.
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';
await FlutterEdgeAi.initialize(
inferenceEngines: [LiteRtLmEngine()],
);Alternatives
Packages that do the same job and could replace flutter_edge_ai_litertlm.
- tflite_flutter
If you want direct TensorFlow Lite model execution without the specific flutter_edge_ai framework abstraction.
68Compare - llamadart
If you prefer running LLaMA models via llama.cpp bindings rather than LiteRT-LM.
93Compare - firebase_ai
If you want to use cloud-backed AI models instead of local on-device execution.
86Compare
Works with
Packages commonly used together with flutter_edge_ai_litertlm: ones it builds on, ones built on it, and its own family.
- cryptoflutter_edge_ai_litertlm depends on itProvides SHA, MD5, and HMAC hashing functions for Dart.
- hooksflutter_edge_ai_litertlm depends on itDart API for the JSON-based protocol for package build and link hooks.
- code_assetsflutter_edge_ai_litertlm depends on itProvides the hook protocol specification for bundling native code with Dart packages.
- ffiflutter_edge_ai_litertlm depends on itUtilities for working with Foreign Function Interface (FFI) code in Dart.
- path_providerflutter_edge_ai_litertlm depends on itProvides Flutter APIs to locate common filesystem directories on Android, iOS, macOS, Linux and Windows.
- mutexflutter_edge_ai_litertlm depends on itMutual exclusion locks and read-write locks for asynchronous Dart code
- flutter_edge_aiflutter_edge_ai_litertlm depends on itOn-device AI for Flutter to run Gemma and other LLMs, embeddings, and speech without a server.