Deduplicates concurrent HTTP requests so identical in-flight calls share one network response; configurable dedup methods, header exclusions, polynomial-hash cache keys, shared in-memory body, optional minWindow.
A Kotlin Multiplatform library that prevents duplicate concurrent HTTP requests in Ktor clients.
When multiple components request the same resource simultaneously, only one actual HTTP request is executed, and all callers receive the same response. This optimizes network usage and reduces server load.
All platforms from v2.x.x plus:
| If you use Ktor | Use plugin version |
|---|---|
| 2.3.0 to 2.x.x | 2.x.x ← Use this |
| 3.0.0+ | ← Use this |
Add to your build.gradle.kts:
dependencies {
implementation("io.github.tiper:ktor-client-deduplication:2.x.x")
}
Add to your build.gradle.kts:
dependencies {
implementation("io.github.tiper:ktor-client-deduplication:3.x.x")
}
val client = HttpClient {
install(RequestDeduplication)
}
// Multiple concurrent GET requests to the same URL
launch { client.get("https://api.example.com/users") }
launch { client.get("https://api.example.com/users") }
launch { client.get("https://api.example.com/users") }
// Result: Only ONE actual HTTP request is made!
// All three callers receive the same shared response
Plugin order matters! The order affects what gets included in the deduplication cache key.
Example:
val client = HttpClient {
// Install BEFORE if you want their effects in the cache key
install(DefaultRequest) { ... } // Headers add to cache key
install(Auth) { ... } // Token adds to cache key
install(RequestDeduplication) // Deduplication based on above
// Install AFTER if you don't want them affecting deduplication
install(OtherAuth) { ... } // Token that doesn't affect cache key
install(Logging) { ... } // Logs response, doesn't affect cache key
install(HttpTimeout) { ... } // Timeout applies after dedup
}
Consider your requirements and test to ensure the plugin order matches your expected behavior.
val client = HttpClient {
install(RequestDeduplication) {
// Deduplicate both GET and HEAD requests
deduplicateMethods = setOf(HttpMethod.Get, HttpMethod.Head)
// Exclude tracing/telemetry headers from cache key computation
excludeHeaders = setOf(
"X-Trace-Id",
"X-Request-Id",
"X-Correlation-Id",
"traceparent",
"tracestate"
)
// Optional: Add minimum deduplication window for fast responses
// Useful when error responses or cached data return very quickly
// minWindow = 50 // milliseconds (default: 0)
}
}
excludeHeaders).The cache key is built from:
excludeHeaders)Headers are combined using polynomial rolling hash to ensure order-independence and collision resistance:
GET:https://api.example.com/users?id=123|h=1847563829
deduplicateMethodsType: Set<HttpMethod>
Default: setOf(HttpMethod.Get)
HTTP methods to deduplicate. Typically you only want to deduplicate idempotent methods (GET, HEAD). You can add POST if your use case allows.
deduplicateMethods = setOf(HttpMethod.Get, HttpMethod.Head, HttpMethod.Post)
minWindowType: Long (milliseconds)
Default: 0 (no delay)
Minimum deduplication window (in milliseconds). This adds an artificial delay to ensure fast responses (like errors or cached responses) wait long enough for concurrent requests to join the deduplication window.
When to use:
Trade-off: Higher values increase deduplication effectiveness but can add latency to fast requests by enforcing a minimum response window (slower requests are not delayed further)
Example:
install(RequestDeduplication) {
minWindow = 50 // Wait at least 50ms before completing
}
Recommended: Start with 50-100ms if you have very fast responses and measure the impact.
excludeHeadersType: Set<String>
Default: emptySet()
Headers to exclude from cache key computation (case-sensitive). This is crucial for preventing tracing/telemetry headers from breaking deduplication.
Common headers to exclude:
Example:
excludeHeaders = setOf(
"X-Trace-Id",
"X-Request-Id",
"traceparent"
)
⚠️ Important: Header matching is case-sensitive. Make sure the casing matches exactly how your SDK sends the headers.
If deduplication isn't working as expected, check the actual header names sent by your SDK:
val client = HttpClient {
install(Logging) {
level = LogLevel.HEADERS
}
install(RequestDeduplication) {
// Add headers after verifying their exact casing
excludeHeaders = setOf("X-Custom-Header")
}
}
The plugin reads the response body once into memory and shares it (not copies) across all concurrent callers. This is highly memory-efficient:
Memory behavior:
Implications:
The implementation is thread-safe and uses Kotlin's Mutex for synchronization. It's safe to use with concurrent coroutines across all platforms.
Error responses (e.g., 404, 500) are also deduplicated and shared with all waiting callers.
The plugin handles cancellation gracefully to ensure robustness:
Individual caller cancellation:
CancellationException as expectedAll callers cancel:
Example:
// Caller 1 starts request
launch { client.get("https://api.example.com/data") }
// Caller 2 joins (waits for same request)
val job = launch { client.get("https://api.example.com/data") }
// Caller 2 cancels
job.cancel() // ✅ Caller 2 is cancelled, Caller 1 still gets response
// If both cancel before response arrives, the HTTP request is cancelled
In unit tests, add artificial latency (e.g., delay(100)) in your mock handler to ensure concurrent requests overlap and deduplication is triggered reliably:
val client = mockClient {
delay(100) // Simulate network latency
requestCount++
"response-$requestCount"
}
This library uses semantic versioning aligned with Ktor major versions:
| Plugin Version | Ktor Version | Kotlin Version | |---|---|----------------|---|---| | 2.x.x | 2.3.0+ | 1.9.20+ | | 3.x.x | 3.0.0+ | 2.0.20+ |
Both versions support the same platforms:
Version 3.x additionally supports:
Apple Ecosystem:
Mobile:
Desktop:
Web:
Copyright 2026 Tiago Pereira
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance the License.
You may obtain a copy of the License at
http:
Unless applicable law agreed to writing, software
distributed under the License distributed an BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express implied.
See the License the specific language governing permissions
limitations under the License.
Contributions are welcome! Please feel free to submit a Pull Request.
| Category | Headers |
|---|
| Distributed Tracing | X-Trace-Id, X-Request-Id, X-Correlation-Id |
| Zipkin/B3 | X-B3-TraceId, X-B3-SpanId, X-B3-ParentSpanId, X-B3-Sampled |
| W3C Trace Context | traceparent, tracestate |
| Firebase | X-Firebase-Locale, X-Firebase-Auth-Time |
| AWS | X-Amzn-Trace-Id |
| Google Cloud | X-Cloud-Trace-Context |
Surfaced from shared tags and platforms — no rankings paid for.