Generates protobuf from `@Serializable` classes using KSP at compile time, offering enhanced file generation and documentation access compared to runtime alternatives. Experimental with evolving features.
Use KSP to generate protobuf from kotlin classes. It's composed of 2 parts:
K2PB is designed to have models you maintain in Kotlin or KMP. Those models define the serialization format with a couple of annotations and a minimum of boilerplate.
Kotlinx Serialization works similarly but doesn't provide a strong compatibility with protobuf official protocol, which leads to subtle issues when interoperating with other services using protoc.
Add the plugin (coming from the gradle plugin portal and maven central):
plugins {
kotlin("multiplatform")
id("com.glureau.k2pb") version "<!--$ GRADLE_PROPERTIES version -->0.9.32<!-- END $-->"
}
Annotate your classes with @ProtoMessage
@ProtoMessage
data class MyDataClass(val myInt: Int)
Create your serializer:
val serializer = K2PB {
// Those methods are generated by the plugin, by module, to collect all defined serializers
// You just need to define the modules you want to use on this serializer
registerSampleLibSerializers()
registerSampleAppSerializers()
}
Use the serializer to serialize/deserialize your objects:
val myDataClass = MyDataClass(42)
val bytes = serializer.encodeToByteArray<MyDataClass>(myDataClass)
val decoded = serializer.decodeFromByteArray<MyDataClass>(bytes)
A few settings can be customized in the k2pb block in your build.gradle.kts:
k2pb {
protoPackageName = "com.custom.protobuf"
javaPackage = "com.custom.javapackage"
javaOuterClassnameSuffix = "Proto"
}
See K2PBExtension for more details. Warning, if you work with multiple compilation modules, you may want to keep the same settings on all your modules.
You can also define some settings on classes and properties:
@ProtoMessage(name = "LegacyName") // Use LegacyName in protobuf files
data class MyDataClass(
@ProtoField(
name = "my_int", // Use my_int in protobuf files
number = 3, // Use 3 as the "proto number" in protobuf files
converter = MyCustomConverter::class // Use MyCustomConverter to serialize/deserialize this field
// and more to come
)
val myInt: Int
)
See ProtoMessage and ProtoField for more details.
Ktx Serialization is a great library, but it's oriented Kotlin first and need to support multiple formats.
As a result, some choices on kotlinx-serialization-protobuf (still experimental) have been taken that are not fully protobuf compatible.
Having a proper protobuf compatibility means that you could use K2PB on clients and protoc on servers for example.
In projects where you need to interoperate with other languages, it's critical.
Ktx Serialization uses Kotlin default values as a fallback when nothing is provided.
Protobuf defines a list of default values for everyone (ex: int32 is 0 by default).
Let's take an example:
data class MyMessage(val myInt: Int = 42)
with the equivalent protobuf message:
syntax = "proto3";
message MyMessage {
int32 myInt = 1;
}
If you deserialize a protobuf message that is empty:
myInt to 42myInt to 0This is already very problematic, but it gets worse. Protobuf explicitly states that default values should not be serialized.
Also note that if a scalar message field is set to its default, the value will not be serialized on the wire
Let's say now you create your instance with MyMessage() in KMP, now you want to serialize it, Ktx offers 2 choices:
To offer some flexibility, Ktx Serialization allows you to choose if you want to encode default values or not.
See Protobuf documentation: https://protobuf.dev/programming-guides/proto3/#default
https://github.com/Kotlin/kotlinx.serialization/issues/2831
Protobuf doesn't support nullability, but optionality:
Kotlin works based on the nullability concept, "" != null in Kotlin.
Let's say you want to add a new nullable field of type enum, the message encoded before this change won't contain this field.
Because there's no Ktx Serialization support of nullability,
the deserialization of an old message requires the default Kotlin value if present (see previous chapter why it's not good),
and because developer knows it's a new field, the dev should put null as the default value.
But a protoc generated class, not having the kotlin default value reading the same old message will decode the first value of the enum.
Eventually, Ktx Serialization & protoc have again different values.
K2PB wants to differentiate explicitly the nullability, and will add a new field to disambiguate the nullability. Protoc generated class will have a specific accessor and will need to be aware of the nullability algorithm. And for K2PB users, it should be invisible, and can be tweaked if needed via annotations.
TODO
TODO
TODO
(Eventually I hope to be able to handle upgrades from existing protobuf files, that's another reason to get this out of KotlinXSerialization/run-time limitations.)
This is an experimental tool, don't hesitate to create github issues for any question or problem you encounter.
Current known limitations:
Tests are located in the 2 samples modules. It's asserting that protobuf files are stable.
It's also using protoc on the generated protobuf files, allowing us to test that a message encoded via KotlinX Serialization will be decoded successfully by the code generated by the protobuf files.
Proto2 is not supported at the moment (please open an issue/PR if you need it). KotlinX Serialization support the Proto3 format IF all List<> are annotated with @ProtoPacked.
See sample-lib and sample-app for examples. This library is using those samples as the primary tests source.
encodeDefaults = false:
You create your instance with MyMessage() in KMP, when you want to serialize it
42 as an empty message because it's Kotlin default (and default should not be serialized)myInt = 0encodeDefaults = true:
You create your instance with MyMessage(0) in KMP, and you want to serialize it
0 (not the Kotlin default + config says to encode everything)myInt = 0Surfaced from shared tags and platforms — no rankings paid for.