Inferkt
0.0.2indexedOffers a binding for llama.cpp, enabling API functionality for common usage across mobile platforms like Android and iOS.
32
Stars
—
Used by
dependents
—
Health
/ 100
InferKt
InferKt is a llama.cpp binding for Kotlin multiplatform API for common use on (Android, iOS).
How to use
This library is experimental and api may be subject to change.
Add the dependency
- Add the dependency to your module's
build.gradle.ktsfile:
kotlin {
sourceSets {
commonMain.dependencies{
implementation("com.dilivva:inferkt:${version}")
}
}
}
On iOS: Add Accelerate.framework and Metal.framework to your project on Xcode.
- Explore API in common code:
Acknowledgements
llama.cpp: for their awesome work on local inference.
llama.rn: inspired the build process implemented in this project.
Kotlin Multiplatform: awesome cross-platform framework.
Contributing
We welcome contributions to InferKt!
License
InferKt is licensed under the MIT License. See the LICENSE file for details.