Skip to content
pubscout

flutter_edge_ai_litertlm

1.20.2, released 3 days ago by sashadenisov.dev

LiteRT-LM on-device engine for flutter_edge_ai supporting .litertlm models and embeddings.

WASM-readyRock solid
$ flutter pub add flutter_edge_ai_litertlm
Source repository

What it does

Provides a LiteRT-LM engine implementation for the flutter_edge_ai package, running .litertlm models, EmbeddingGemma 2 embeddings, and LiteRT .tflite embeddings across multiple native platforms and web via FFI. It handles sessions, streaming, structured outputs, and platform-specific execution on CPU, GPU, and NPU.

AI-written summary of version 1.20.0, based on the package's README and metadata. Report an inaccuracy

Use it when

  • You are using flutter_edge_ai and need to run .litertlm language models or embeddings on-device.
  • You want support for structured outputs (JSON schema) via LiteRT-LM constrained decoding.
  • You need multi-platform on-device inference across Android, iOS, web, Windows, macOS, and Linux.

Skip it when

  • You are not using flutter_edge_ai in your application.
  • Your application relies strictly on cloud-based LLM APIs and does not require local on-device inference.

Strengths

  • Runs models locally across various hardware targets including CPU, GPU, and NPU.
  • Supports structured output via JSON schema validation.
  • Integrated deeply with the flutter_edge_ai ecosystem and model catalog.

Weaknesses

  • Requires strict version matching and upgrading of multiple dependent flutter_edge_ai packages together.
  • NPU and certain platforms have specific limitations regarding session sharing, samplers, or context windows.

Setup

Kind
Plugin (native code)
Effort
High
Code generation
No
Direct dependencies
7

Requires pairing with flutter_edge_ai, registering the LiteRtLmEngine during initialization, and adhering to strict package version requirements alongside other edge AI packages.

import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';

await FlutterEdgeAi.initialize(
  inferenceEngines: [LiteRtLmEngine()],
);

Alternatives

Packages that do the same job and could replace flutter_edge_ai_litertlm.

  • tflite_flutter

    If you want direct TensorFlow Lite model execution without the specific flutter_edge_ai framework abstraction.

  • llamadart

    If you prefer running LLaMA models via llama.cpp bindings rather than LiteRT-LM.

  • firebase_ai

    If you want to use cloud-backed AI models instead of local on-device execution.

Works with

Packages commonly used together with flutter_edge_ai_litertlm: ones it builds on, ones built on it, and its own family.