kflite

3.4.13-alphaindexed

Fresh and improved version of moko tensorflow, facilitating machine learning model integration with enhanced functionality and ongoing development for evolving needs.

AndroidNative·ShadAdman/kflite

Stars

—

Used by

dependents

Health

/ 100

About

kflite runs ml models such as TensorFlow, PyTorch, and JAX models on mobile device (support for all edge devices is the goal) with help of Kotlin Multiplatform. It abstracts platform differences, and manages model loading, tensor creation, and inference through a unified API without any performance change & completely native.

Key features:

kflite is now split into 3 main artifacts for better flexibility:

kflite: The main runtime and model management.
preprocessing: Image preprocessing utilities (Audio and Text coming soon).
postprocessing: NMS, COCO helpers, reshaping, and image normalization.

Quick Intro

For a Compose example to run with TFLite checkout here

For a Compose example to run with LiteRT checkout here

For a memmory mapped example to run with TFLite checkout here

Installation

Add dependencies

Include the required dependencies in your shared commonMain.dependencies. You can choose only what you need:

val version = "3.4.0"
implementation("io.github.shadadman:kflite:$version")
implementation("io.github.shadadman:preprocessing:$version")
implementation("io.github.shadadman:postprocessing:$version")

Configure for iOS (Using SPM should remove this necesserity)

Since KMP doesn’t automatically include CocoaPods dependencies in the consumer's iosApp, you need to manually add TensorFlow Lite for iOS. Create a Podfile inside your iosApp with following pods:

target 'iosApp' do
  use_frameworks!
  platform :ios, '16.0'
  pod 'TensorFlowLiteObjC'
  pod 'TensorFlowLiteObjC/Metal'
  pod 'TensorFlowLiteObjC/CoreML'
end

Run a Model

Step 1 - Place the model

Put your .tflite model file in composeResources/files You can view an example model placement on Kflite Sample.

kflite uses Compose Resources to manage assets in a platform independent way.Your model becomes available to all targets by converting it into a byte array.

Step 2 - init the model

kflite can be used via a global singleton (Kflite) for simple use cases or by instantiating a Class (KfliteClass) when you need to manage multiple models simultaneously or control their lifecycles independently.

Load from Bytes (Standard)

Kflite.init(
    model = Res.readBytes("files/efficientdet-lite2.tflite"),
    options = InterpreterOptions(...)
)

Load from File (Memory Mapped / High Performance)

To avoid reading large models into memory, you can load them directly from a file path. This uses memory-mapping on Android/JVM and native file-based loading on iOS.

Kflite.init(
    modelSource = KFliteModel.fromFile(modelPath),
    options = InterpreterOptions(...)
)

numThreads: number of threads allocate to CPU. Defualt is 4
delegateType: selects hardware acceleration backend.(GPU and NNAPI on Android, METAL and CoreML on iOS)
allowFp16PrecisionForFp32: speeds up inference if supported by hardware
inferencePreferenceType The preference for inference speed and accuracy.(Platform default on Android is FAST_SINGLE_ANSWER and on iOS WaitTypePassive)

[!IMPORTANT] The following properties are not customizable in LiteRT(Android), so setting them will have no effect. Maybe in feature updates:

inferencePreferenceType

allowQuantizedModels

allowFp16PrecisionForFp32

Step 3 - Prepare the input data

Kflite works with direct ByteBuffer as input, so you can feed preprocessed inputs or tensors directly.

[!IMPORTANT] Netron is a great tool for visualizing your model. You can see your model input/output details.

You can see input tensor shape of your model by calling getInputTensor or Checkout your model metadata or use Netron to check input dimensions and what each shape represents.
To see the number of input tensor that your model has, you can call or Checkout your model metadata or use to check input dimensions.

Following example shows how to prepare input data for a model that takes an image as input.

Our model has 4 shape. The batch size,image width, image height and the pixel size.

[!IMPORTANT] If you know the amount of each shape, just hard code them in a const, since each shape is constant.

val pixcelSize = Kflite.getInputTensor(0).shape[0] = 1
val inputImageWidth = Kflite.getInputTensor(0).shape[1] = 448
val inputImageHeight = Kflite.getInputTensor(0).shape[2] = 448
val floatTypeSize = Kflite.getInputTensor(0).shape[3] = 3

Calculate the input size.This will be used to allocate the size of input data.

val modelInputSize =
    floatTypeSize * inputImageWidth * inputImageHeight * pixcelSize

Create a ByteBuffer from your input data. This is for image inputs using the preprocessing module. Following example scales an image to match model input size and converts it into a normalized float array.

[!IMPORTANT] When the normalize is true, the code performs Image Normalization and Data Type Conversion on the pixel data before feeding it into the byte buffer. This changes the data type of the input in the buffer from an 8-bit integer to 32-bit floating point.True this only for models that supports input data in a range of [0.0,1.0].

val inputImage =  imageResource(Res.drawable.example_model_input)
    .toScaledByteBuffer(
        inputWidth = inputImageWidth,
        inputHeight = inputImageHeight,
        inputAllocateSize = modelInputSize,
        normalize: Boolean = false
    )

Step 4 - Prepare the output data

Create a container that matches the model’s output tensor shape. This gives you a correctly sized structure to hold the results.

You can see output tensor shape of your model by calling getOutputTensor or Checkout your model metadata or use Netron
To see the number of input tensor that your model has, you can call getOutputTensorCount or Checkout your model metadata or use

Our example shows how to prepare output data for an object detection model that outputs a 3D matrix. Our example model has 3 shape. The batch size (number of input), number of results and the bounding box locations (x,y,w,h).

[!IMPORTANT] If you know the amount of each shape, just hard code them in a const, since each shape is constant.

val batchSize = Kflite.getOutputTensor(0).shape[0] = 1
val numberOfResults = Kflite.getOutputTensor(0).shape[1] = 25
val detailsPerResult = Kflite.getOutputTensor(0).shape[2] = 4

We then create a matrix to hold the model output.

val modelOutputContainer = Array(batchSize) {
    Array(numberOfResults) {
        FloatArray(detailsPerResult)
    }
}

Step 5 - Running the model:

Kflite.run() performs inference on the model. You feed it the inputs and provide a container for outputs, and it returns predictions, detections, or classifications depending on your model type.

[!IMPORTANT] By default we assume the model have multiple inputs and outputs.

inputs: List of input tensors.
outputs: A map linking output tensor indices to the containers you created earlier.

If your model supports multiple input/output, add them to the list. The example model supports only one input and one output.

Kflite.run(
    inputs = listOf(inputImage), // The input made in step 3
    outputs = mapOf(Pair(0,modelOutputContainer)) // the output prepared in step 4
)

Step 6: Close the model:

Once inference is complete, you call close() to free up resources and safely release the interpreter to avoid memory leaks.

Kflite.close()

Normalizing Model Output

Most detection models output is in model scaled coordinates.
When you resize your image to match the model input, you need to normalize the output data into the original size.
This normalization is not necessary unless you want to match the original to the model output. For example putting a bounding box on the original image or
modify the original data based on the model output.

You can use the normalizing extensions from the postprocessing module to rescale object detection bounding boxes into the original image. The normalizedBox will be a data class contain the new ordinations.

val normalizedBox = Normalization(
    originalImageHeight = 1080f, //Original input height
    originalImageWidth = 2010f, // Original input width
    modelImagWidth = 448f, //Model input width
    modelImageHeight = 448f //Model input height
).YOLO(
    center_x = 20f, 
    center_y = ,
    width = ,  
    height =  
)