Cross-platform library for allocating and managing byte arrays using an API similar to Java's ByteBuffer. Supports direct, heap, and shared memory allocation zones, enabling efficient memory management across different platforms.
See the project website for documentation and APIs.
Kotlin Multiplatform byte buffers — same code on JVM, Android, iOS, macOS, Linux, JS, and WASM with zero-copy performance.
Buffer gives you one ReadBuffer/WriteBuffer API that delegates to platform-native types — ByteBuffer on JVM, NSData on Apple, malloc on Linux, Uint8Array on JS — so data never copies between your code and the OS.
Why not just use ByteArray?
plugins {
id("com.google.devtools.ksp") version "<ksp-version>" // Only needed for codec code generation
}
dependencies {
// Core buffer library
implementation("com.ditchoom:buffer:<latest-version>")
// Optional: Protocol codecs (annotation-driven code generation)
implementation("com.ditchoom:buffer-codec:<latest-version>")
ksp()
implementation()
implementation()
}
Find the latest version on Maven Central.
// Works identically on JVM, Android, iOS, macOS, Linux, JS, WASM
val buffer = BufferFactory.Default.allocate(1024)
buffer.writeInt(42)
buffer.writeString("Hello!")
buffer.resetForRead()
val number = buffer.readInt() // 42
val text = buffer.readString(6) // "Hello!"
Tip: Always use
BufferFactoryto allocate buffers — notPlatformBuffer.allocate(). Factories compose with pooling, deterministic cleanup, and custom allocation strategies. For structured binary data, use Protocol Codecs instead of manual read/write sequences.
For structured data, always use buffer-codec instead of manual readInt()/writeInt() sequences. Hand-written encode/decode is error-prone (field order mismatches, type mismatches) and doesn't guarantee round-trip correctness. Define a data class, annotate it, and the codec writes itself:
@ProtocolMessage
data class SensorReading(
val sensorId: UShort, // 2 bytes
val temperature: Int, // 4 bytes
@LengthPrefixed val label: String, // 2-byte length prefix + UTF-8
)
The KSP processor generates a full Codec<SensorReading> — encode, decode, and wireSize — at compile time:
val buffer = BufferFactory.Default.allocate(64)
SensorReadingCodec.encode(buffer, reading, EncodeContext.Empty)
buffer.resetForRead()
val decoded = SensorReadingCodec.decode(buffer, DecodeContext.Empty)
Annotations cover common binary protocol patterns:
Generated codecs also provide peekFrameSize(stream) for zero-boilerplate stream framing and CodecContext for typed runtime configuration through codec chains.
Generated codecs implement the same Codec<T> interface used for manual codecs, so all patterns (streaming, round-trip testing, composition) work unchanged.
Note: Generated code appears after compilation (
./gradlew build). IDE features like autocomplete and navigation for generated codecs require an initial build.
See the full protocol codecs guide for details.
Because codecs encode positionally, edits like reordering an enum, inserting a field, or
changing a wire width silently break peers already on the wire — and round-trip tests can't catch
them (the same new code encodes and decodes). The com.ditchoom.buffer.codec-schema Gradle plugin
baselines your protocol's wire shape into a committed, diffable file and fails the build when a
change is wire-breaking:
// settings.gradle.kts — the plugin is on Maven Central, not the Gradle Plugin Portal
pluginManagement {
repositories {
mavenCentral()
gradlePluginPortal()
}
}
// build.gradle.kts
plugins {
id("com.ditchoom.buffer.codec-schema") version "<latest-version>"
}
codecSchema {
failOnBreaking.set(true) // fail on breaking drift (default: warn only)
}
The first check writes src/codecSchema/codec-schema.txt — commit it. After an intentional wire
change, run ./gradlew updateCodecSchema and commit the diff. See the
plugin README for the full drift-classification table and
workflow.
Allocate once, reuse for every request — no GC pressure:
withPool(defaultBufferSize = 8192) { pool ->
repeat(10_000) {
pool.withBuffer(1024) { buffer ->
buffer.writeInt(requestId)
buffer.writeString(payload)
buffer.resetForRead()
sendToNetwork(buffer)
} // buffer returned to pool, not GC'd
}
} // pool cleared
Without pooling: 10,000 requests = 10,000 allocations. With pooling: 10,000 requests = ~64 allocations (maxPoolSize).
Parse protocols that span chunk boundaries — no manual accumulator code:
val processor = StreamProcessor.create(pool)
// Chunks arrive from the network
processor.append(chunk1) // partial message
processor.append(chunk2) // rest of message + start of next
// Parse length-prefixed messages across boundaries
while (processor.available() >= 4) {
val length = processor.peekInt()
if (processor.available() < 4 + length)
processor.skip()
message = processor.readBufferScoped(length) {
MyMessage(readInt(), readString(remaining()))
}
handleMessage(message)
}
readBufferScopedvsreadBuffer: PreferreadBufferScoped— it passes the buffer to your block as a receiver and automatically releases it back to the pool.readBufferreturns an unmanaged buffer that is not returned to the pool, which can cause pool exhaustion under load.
Compress and decompress any ReadBuffer — works on all platforms:
val data = "Hello, World!".toReadBuffer()
val compressed = compress(data, CompressionAlgorithm.Gzip).getOrThrow()
val decompressed = decompress(compressed, CompressionAlgorithm.Gzip).getOrThrow()
Compose streaming transforms with Kotlin Flow:
// Transform a flow of raw buffers into processed lines
bufferFlow
.mapBuffer { decompress(it, Gzip).getOrThrow() }
.asStringFlow()
.lines()
.collect { line -> process(line) }
For explicit memory management without relying on GC:
BufferFactory.deterministic().allocate(8192).use { buffer ->
buffer.writeInt(42)
buffer.writeString("Hello")
buffer.resetForRead()
// Use buffer...
} // Memory freed immediately
Buffer is the foundation for the Socket library — cross-platform TCP + TLS that streams these same ReadBuffer/WriteBuffer types.
┌─────────────────────────────┐
│ Your Protocol │
├─────────────────────────────┤
│ buffer-codec │ ← com.ditchoom:buffer-codec (+ KSP processor)
├─────────────────────────────┤
│ socket (TCP + TLS) │ ← com.ditchoom:socket
├─────────────────────────────┤
│ buffer-compression │ ← com.ditchoom:buffer-compression
├─────────────────────────────┤
│ buffer-flow │ ← com.ditchoom:buffer-flow
├─────────────────────────────┤
│ buffer │ ← com.ditchoom:buffer
└─────────────────────────────┘
Copyright 2022 DitchOoM
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http:
Unless applicable law agreed to writing, software
distributed under the License distributed an BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express implied.
See the License the specific language governing permissions
limitations under the License.
| Concern | ByteArray | Buffer |
|---|
| Platform I/O | Copy into ByteBuffer/NSData/Uint8Array every time | Zero-copy — delegates to the native type directly |
| Memory reuse | Allocate per request, GC cleans up | BufferPool — acquire, use, release, same buffer reused |
| Fragmented data | Manual accumulator + boundary tracking | StreamProcessor — peek/read across chunk boundaries |
| Compression | Platform-specific zlib wrappers | compress()/decompress() on any ReadBuffer |
| Streaming transforms | Build your own pipeline | mapBuffer(), asStringFlow(), lines() — compose with Flow |
| Cross-platform | Only type that's truly portable | Same API on JVM, Android, iOS, macOS, Linux, JS, WASM |
| Module | Description |
|---|
buffer | Core ReadBuffer/WriteBuffer interfaces, BufferPool, StreamProcessor |
buffer-codec | Codec<T> interface (operates directly on ReadBuffer/WriteBuffer), annotations for code generation |
buffer-codec-processor | KSP processor — generates Codec implementations from @ProtocolMessage annotations |
buffer-codec-schema | Wire-format schema descriptor model, parser, and drift classifier (shared by the processor and the Gradle plugin) |
buffer-codec-gradle-plugin | Gradle plugin (com.ditchoom.buffer.codec-schema) — baselines the wire shape and fails the build on breaking drift |
buffer-compression | compress()/decompress() (gzip, deflate) on ReadBuffer |
buffer-flow | Kotlin Flow extensions: mapBuffer(), asStringFlow(), lines() |
@LengthPrefixed(prefix) — length-prefixed strings (1, 2, or 4 byte prefix)@RemainingBytes — consume all remaining bytes@LengthFrom("field") — string length from a preceding numeric field@WireBytes(n) — custom wire width for numeric fields (e.g., 3-byte integers)@WireOrder(order) — per-field byte order override (big/little endian)@When("expr") — conditional nullable fields@UseCodec(codec) — delegate to an existing Codec object@Payload — generic payload type parameter@PacketType(value, wire) on sealed interface variants — auto-dispatched decode; wire for spec-compliant encode values@DispatchOn(Discriminator::class) + @DispatchValue — custom multi-byte or bit-packed discriminator dispatch@FramedBy(codec, after) — framework-owned length framing computed from and bounding a sealed message's body@ForwardCompatible(unknown) + @UnknownVariant — skip and preserve unrecognized sealed variants byte-identically (forward-compatible relay/persistence)value class PacketId(val raw: UShort))| Platform | Native Type | Notes |
|---|
| JVM | java.nio.ByteBuffer | Direct buffers for zero-copy NIO |
| Android | ByteBuffer + SharedMemory | IPC via Parcelable |
| iOS/macOS | NSMutableData | Foundation integration |
| JavaScript | Uint8Array | SharedArrayBuffer support |
| WASM | LinearBuffer / ByteArrayBuffer | Zero-copy JS interop |
| Linux | NativeBuffer (malloc/free) | Zero-copy io_uring I/O |
Surfaced from shared tags and platforms — no rankings paid for.