--- url: /playground.md --- # Playground **Basic Get/Set** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Multi-Layer Stack** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Stale-While-Revalidate** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Tag Invalidation** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Cache Policy** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Stored Nulls & getEntry** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Exact-Key APIs** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Generation Rotation** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Parallel Backfill** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Namespaces** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Stampede Prevention** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Circuit Breaker** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here **Write-Behind** Editor Edit a preset, then run it in the sandboxed worker playground. RunIdleOutput Execution logs and layer state. LogsLayersRun code to see output here --- url: /docs/api.md --- # API Reference Complete API documentation for layercache. ## Table of Contents - [CacheStack](#cachestack) - [Constructor](#constructor) - [Read Operations](#read-operations) - [Write Operations](#write-operations) - [Invalidation](#invalidation) - [Wrapping & Namespaces](#wrapping--namespaces) - [Warming & Persistence](#warming--persistence) - [Observability](#observability) - [Generation Management](#generation-management) - [Lifecycle](#lifecycle) - [Cache Layers](#cache-layers) - [MemoryLayer](#memorylayer) - [RedisLayer](#redislayer) - [DiskLayer](#disklayer) - [MemcachedLayer](#memcachedlayer) - [Custom Layers](#custom-layers) - [Options Reference](#options-reference) - [CacheStackOptions](#cachestackoptions) - [Per-Operation Options](#per-operation-options) - [Invalidation Strategies](#invalidation-strategies) - [Freshness Strategies](#freshness-strategies) - [Resilience](#resilience) - [Compression & Serialization](#compression--serialization) - [Distributed Features](#distributed-features) - [Event Hooks](#event-hooks) - [Framework Integrations](#framework-integrations) - [Admin CLI](#admin-cli) *** ## CacheStack The main class that orchestrates reads, writes, and invalidation across multiple cache layers. ### Constructor ```ts import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' const cache = new CacheStack(layers, options?) ``` **Parameters:** - `layers` - `CacheLayer[]` - Array of cache layers, ordered from fastest (L1) to slowest (Ln) - `options` - `CacheStackOptions` - Optional configuration (see [CacheStackOptions](#cachestackoptions)) *** ### Read Operations #### `cache.get(key, fetcher?, options?): Promise` Reads through all layers in order. On a partial hit (found in L2 but not L1), backfills the upper layers automatically. On a full miss, runs the fetcher if provided. ```ts // Without fetcher - returns undefined on miss const user = await cache.get('user:123') // With fetcher - runs once on miss, fills all layers const user = await cache.get('user:123', () => db.findUser(123)) // With full options const user = await cache.get('user:123', () => db.findUser(123), { ttl: { memory: 30_000, redis: 600_000 }, tags: ['user', 'user:123'], negativeCache: true, negativeTtl: 15_000, staleWhileRevalidate: 30_000, staleIfError: 300_000, ttlJitter: 5_000 }) ``` #### `cache.getOrThrow(key, fetcher?, options?): Promise` Like `get()`, but throws `CacheMissError` instead of returning `undefined`. A stored `null` is a cache hit and is returned normally. ```ts import { CacheMissError } from 'layercache' try { const config = await cache.getOrThrow('app:config') } catch (err) { if (err instanceof CacheMissError) { console.error(`Missing key: ${err.key}`) } } ``` #### `cache.mget(entries): Promise>` Concurrent multi-key fetch. Uses layer-level `getMany()` fast paths when all entries are simple reads. ```ts const [user1, user2] = await cache.mget([ { key: 'user:1', fetch: () => db.findUser(1) }, { key: 'user:2', fetch: () => db.findUser(2) }, ]) ``` #### `cache.has(key): Promise` Check if a key exists in any layer. #### `cache.ttl(key): Promise` Get the remaining TTL in milliseconds for a key in the first layer that has it. Returns `null` if the key doesn't exist. #### `cache.inspect(key): Promise` Returns detailed metadata about a cache key for debugging. ```ts const info = await cache.inspect('user:123') // { // key: 'user:123', // foundInLayers: ['memory', 'redis'], // freshTtlMs: 45, // staleTtlMs: 75, // errorTtlMs: 345, // isStale: false, // tags: ['user', 'user:123'] // } ``` #### `cache.getEntry(key): Promise | null>` Reads a key and returns entry metadata instead of only the value. Use this when `null` is a valid cached value and you need to distinguish stored nulls, negative-cache entries, stale entries, and misses. ```ts await cache.set('user:deleted', null) const entry = await cache.getEntry('user:deleted') // { // key: 'user:deleted', // value: null, // kind: 'value', // state: 'fresh', // layer: 'memory' // } const miss = await cache.getEntry('user:missing') // null ``` `cache.set()` stores `null` values directly. In v4, read-through fetchers also store `null` as a regular value by default, so misses and negative-cache entries remain distinguishable as `undefined`. Set `cacheNullValues: false` only when a fetcher uses `null` to mean absence. ```ts await cache.get('user:deleted', async () => null) ``` *** ### Write Operations #### `cache.set(key, value, options?): Promise` Writes to all layers simultaneously. ```ts await cache.set('user:123', user, { ttl: { memory: 60_000, redis: 600_000 }, tags: ['user', 'user:123'], staleWhileRevalidate: { redis: 30_000 }, staleIfError: { redis: 120_000 }, ttlJitter: { redis: 5_000 } }) // Uniform TTL across all layers await cache.set('user:123', user, { ttl: 120_000, tags: ['user'] }) ``` #### `cache.mset(entries): Promise` Concurrent multi-key write. #### `cache.delete(key): Promise` Delete one exact key from all layers. #### `cache.mdelete(keys): Promise` Bulk delete exact keys. #### `cache.clear(): Promise` Delete all keys from all layers. *** ### Invalidation #### `cache.invalidateByKey(key): Promise` Alias for `cache.delete(key)`. Use it when you want the `invalidateBy*` naming style for one exact key. ```ts await cache.invalidateByKey('user:123') // deletes only user:123 ``` #### `cache.invalidateByKeys(keys): Promise` Alias for `cache.mdelete(keys)`. Deletes only the exact keys provided. ```ts await cache.invalidateByKeys(['user:123', 'user:123:posts']) ``` #### `cache.invalidateByTag(tag): Promise` Deletes every key stored with this tag across all layers. ```ts await cache.set('user:123', user, { tags: ['user:123'] }) await cache.set('user:123:posts', posts, { tags: ['user:123'] }) await cache.invalidateByTag('user:123') // both keys gone ``` #### `cache.invalidateByTags(tags, mode?): Promise` Delete keys matching any or all of a set of tags. ```ts await cache.invalidateByTags(['tenant:a', 'users'], 'all') // keys tagged with both await cache.invalidateByTags(['users', 'posts'], 'any') // keys tagged with either ``` #### `cache.invalidateByPattern(pattern): Promise` Glob-style deletion. Patterns must be non-empty, at most 1024 characters, and free of control characters. ```ts await cache.invalidateByPattern('user:*') ``` #### `cache.invalidateByPrefix(prefix): Promise` Hierarchical prefix-based invalidation. Prefer this over glob when keys are hierarchical. ```ts await cache.invalidateByPrefix('user:123:') // deletes user:123:profile, user:123:posts, ... ``` #### `cache.expireByKey(key): Promise` Marks one exact key as no longer fresh while keeping the cached value available for stale-while-revalidate / stale-if-error windows. ```ts await cache.expireByKey('user:123') // expires only user:123 ``` #### `cache.expireByKeys(keys): Promise` Expires only the exact keys provided without deleting their stored values. ```ts await cache.expireByKeys(['user:123', 'user:123:posts']) ``` #### `cache.expireByTag(tag): Promise` Marks every key stored with this tag as no longer fresh while keeping the cached value available for stale-while-revalidate / stale-if-error windows. ```ts await cache.set('user:123', user, { ttl: 60_000, staleWhileRevalidate: 30_000, tags: ['user:123'] }) await cache.expireByTag('user:123') // value remains, next read can serve stale and refresh ``` #### `cache.expireByTags(tags, mode?): Promise` Expire keys matching any or all of a set of tags without deleting the stored values. ```ts await cache.expireByTags(['tenant:a', 'users'], 'all') await cache.expireByTags(['users', 'posts'], 'any') ``` #### `cache.expireByPattern(pattern): Promise` Glob-style expiration. Matching envelope-backed entries keep their stale windows; plain layer values that do not carry layercache freshness metadata are left unchanged. ```ts await cache.expireByPattern('user:*') ``` #### `cache.expireByPrefix(prefix): Promise` Hierarchical prefix-based expiration. Prefer this over glob when keys are hierarchical. ```ts await cache.expireByPrefix('user:123:') ``` *** ### Wrapping & Namespaces #### `cache.wrap(prefix, fetcher, options?)` Wraps an async function so every call is transparently cached. The key is derived from function arguments unless you supply a `keyResolver`. Structured arguments use the versioned `j2:` key schema. Plain objects cannot use reserved native `$type` tags (`Date`, `URL`, `RegExp`, `Map`, or `Set`); provide a `keyResolver` if your domain objects intentionally contain one of those tags. ```ts const getUser = cache.wrap('user', (id: number) => db.findUser(id)) const user = await getUser(123) // key -> "user:123" // Custom key resolver const getUser = cache.wrap( 'user', (id: number) => db.findUser(id), { keyResolver: (id) => String(id), ttl: 300_000 } ) ``` #### `cache.namespace(prefix): CacheNamespace` Returns a scoped view with the same full API. `clear()` only touches `prefix:*` keys. ```ts const users = cache.namespace('users') const posts = cache.namespace('posts') await users.set('123', userData) // stored as "users:123" await users.clear() // only deletes "users:*" // Nested namespaces const tenant = cache.namespace('tenant:abc') const tenantPosts = tenant.namespace('posts') await tenantPosts.set('1', data) // stored as "tenant:abc:posts:1" ``` Namespace prefixes must be non-empty, at most 256 characters, and free of control characters. *** ### Warming & Persistence #### `cache.warm(entries, options?)` Pre-populate layers at startup. Higher `priority` values run first. ```ts await cache.warm( [ { key: 'config', fetcher: () => db.getConfig(), priority: 10 }, { key: 'user:1', fetcher: () => db.findUser(1), priority: 5 }, { key: 'user:2', fetcher: () => db.findUser(2), priority: 5 }, ], { concurrency: 4, continueOnError: true } ) ``` #### `cache.exportState() / cache.importState(snapshot)` In-memory snapshot transfer. ```ts const snapshot = await cache.exportState() await anotherCache.importState(snapshot) ``` #### `cache.persistToFile(path) / cache.restoreFromFile(path)` Disk-based snapshot persistence. Restricted to `process.cwd()` by default (configurable via `snapshotBaseDir`). ```ts await cache.persistToFile('./cache-snapshot.json') await cache.restoreFromFile('./cache-snapshot.json') ``` Keep `snapshotBaseDir` process-owned and avoid group/world-writable parent directories. Snapshot writes validate paths and reject symlinked target parents before commit, but shared writable directories are still a poor snapshot boundary. *** ### Observability #### `cache.getMetrics(): CacheMetricsSnapshot` ```ts const { hits, misses, fetches, staleHits, refreshes, writeFailures } = cache.getMetrics() ``` #### `cache.getStats(): CacheStatsSnapshot` Returns metrics, per-layer degradation state, and background refresh count. ```ts const { metrics, layers, backgroundRefreshes } = cache.getStats() // layers: [{ name, isLocal, degradedUntil }] ``` #### `cache.captureMetrics(operation)` Runs an async operation and returns only the metrics emitted while that operation was active. Namespaces use this internally so overlapping namespace operations do not serialize on a global metrics lock. ```ts const { result, metrics } = await cache.captureMetrics(async () => { return cache.get('user:123', fetchUser) }) ``` If the operation rejects, the thrown error is annotated with a `metrics` property containing the captured `CacheMetricsSnapshot`. ```ts try { await cache.captureMetrics(async () => cache.get('user:123', fetchUser)) } catch (error) { const metrics = (error as { metrics?: CacheMetricsSnapshot }).metrics throw error } ``` #### `cache.getHitRate()` Computed hit rate overall and per-layer. #### `cache.healthCheck(): Promise` ```ts const health = await cache.healthCheck() // [{ layer: 'memory', healthy: true, latencyMs: 0.03 }, ...] ``` #### `cache.resetMetrics(): void` Resets all counters to zero. *** ### Generation Management Add a generation prefix to every key and rotate it for bulk invalidation without scanning. ```ts const cache = new CacheStack([...], { generation: 1 }) await cache.set('user:123', user) cache.bumpGeneration() // now reads use v2:user:123 // Optional: auto-cleanup old generation keys const cache = new CacheStack([...], { generation: 1, generationCleanup: { batchSize: 500, maxMatches: 10_000 } }) ``` Cleanup retains a de-duplication set while scanning layers, so `maxMatches` defaults to 10,000 unique keys per run. Exceeding the limit stops the cleanup and emits the existing `generation-cleanup-error` warning. Set `maxMatches: false` only when the deployment bounds the keyspace elsewhere. Persist the active generation outside the process when you deploy multiple instances or restart workers: ```ts import { CacheStack, RedisGenerationStore } from 'layercache' const generations = new RedisGenerationStore({ client: redis }) const generation = await generations.getOrInitialize(1) const cache = new CacheStack([...], { generation }) // Later, atomically rotate all future keys and apply that generation locally. const nextGeneration = await generations.bump() cache.bumpGeneration(nextGeneration) ``` #### `cache.bumpGeneration()` Rotate cache namespace by incrementing generation. #### `cache.getGeneration()` Get current generation number. *** ### Lifecycle #### `cache.disconnect(): Promise` Graceful shutdown (unsubscribes from invalidation bus, etc.). *** ## Cache Layers All layers implement the `CacheLayer` interface: ```ts interface CacheLayer { readonly name: string readonly defaultTtl?: number readonly isLocal?: boolean get(key: string): Promise getEntry?(key: string): Promise getMany?(keys: string[]): Promise> set(key: string, value: unknown, ttl?: number): Promise setMany?(entries: Array<{ key: string; value: unknown; ttl?: number }>): Promise delete(key: string): Promise deleteMany?(keys: string[]): Promise clear(): Promise keys?(): Promise forEachKey?(visitor: (key: string) => void | Promise): Promise has?(key: string): Promise ttl?(key: string): Promise size?(): Promise ping?(): Promise dispose?(): Promise } ``` ### MemoryLayer In-process LRU/LFU/FIFO eviction with configurable max size. ```ts new MemoryLayer({ ttl: 60_000, maxSize: 5_000, name: 'memory' // default }) ``` ### RedisLayer Distributed caching via ioredis with compression, serializers, and optional prefix. ```ts new RedisLayer({ client: redis, ttl: 300_000, prefix: 'myapp:cache:', compression: 'gzip', compressionThreshold: 1_024, commandTimeoutMs: 200, serializer: new MsgpackSerializer(), name: 'redis', allowUnprefixedClear: false }) ``` `commandTimeoutMs` applies a per-command timeout to Redis round-trips. When a Redis command exceeds this threshold, the layer surfaces an error so `CacheStack` can trigger graceful degradation instead of waiting on a slow dependency indefinitely. ### DiskLayer Persistent file-based caching with atomic writes and optional at-rest protection. ```ts import { resolve } from 'node:path' new DiskLayer({ directory: resolve('./var/cache/layercache'), maxFiles: 50_000, maxWriteQueueDepth: 10_000, name: 'disk' }) ``` `maxWriteQueueDepth` caps pending serialized `set()` / `delete()` work so a slow disk cannot accumulate unbounded writes. Defaults to 10,000. Set it to `false` to disable the guard for trusted low-volume environments. #### At-Rest Protection DiskLayer supports AES-256-GCM encryption or HMAC-SHA256 signing to protect cached data on disk: ```ts new DiskLayer({ directory: resolve('./var/cache/layercache'), encryptionKey: process.env.CACHE_ENCRYPTION_KEY, // AES-256-GCM encryption signingKey: process.env.CACHE_SIGNING_KEY, // HMAC-SHA256 signing (ignored if encryptionKey is set) name: 'disk' }) ``` Encryption also provides authenticated integrity — a separate `signingKey` is unnecessary when `encryptionKey` is provided. When `encryptionKey` or `signingKey` is configured, plaintext legacy entries are rejected by default. Use `allowLegacyPlaintext: true` only during a controlled migration window: ```ts new DiskLayer({ directory: resolve('./var/cache/layercache'), signingKey: process.env.CACHE_SIGNING_KEY, allowLegacyPlaintext: true // migration only }) ``` ### MemcachedLayer Memcached support with pluggable serializers and bulk operations. ```ts new MemcachedLayer({ client: memcachedClient, ttl: 300_000, name: 'memcached' }) ``` ### Custom Layers Implement `CacheLayer` to plug in any backend: ```ts class MyCustomLayer implements CacheLayer { readonly name = 'custom' readonly defaultTtl = 300_000 readonly isLocal = false async get(key: string): Promise { /* ... */ } async set(key: string, value: unknown, ttl?: number): Promise { /* ... */ } async delete(key: string): Promise { /* ... */ } async clear(): Promise { /* ... */ } } ``` *** ## Options Reference ### CacheStackOptions | Option | Type | Default | Description | | ----------------------------- | ----------------------------------------------------------------- | ---------------------- | ----------------------------------------------------------------------------- | | `logger` | `Logger \| boolean` | `false` | Pluggable logger interface or boolean | | `metrics` | `boolean` | `true` | Enable/disable metrics collection | | `stampedePrevention` | `boolean` | `true` | In-process request deduplication | | `stampedeMaxInFlight` | `number` | - | Max concurrent in-flight deduplicated requests | | `stampedeEntryTimeoutMs` | `number` | - | Per-entry timeout for stampede guard | | `invalidationBus` | `RedisInvalidationBus` | - | Distributed L1 invalidation | | `tagIndex` | `TagIndex \| RedisTagIndex` | in-memory | Custom tag tracking | | `generation` | `number` | - | Generation prefix for bulk invalidation | | `generationCleanup` | `boolean \| { batchSize?: number; maxMatches?: number \| false }` | - | Auto-prune stale generation keys; discovery defaults to 10,000 unique matches | | `broadcastL1Invalidation` | `boolean` | `false` | Publish writes to peer memory layers | | `negativeCaching` | `boolean` | `false` | Cache absent results as empty entries | | `cacheNullValues` | `boolean` | `true` | Cache null fetcher results as regular values | | `negativeTtl` | `number \| LayerTtlMap` | - | Global TTL for negative cache entries | | `staleWhileRevalidate` | `number \| LayerTtlMap` | - | Global stale-while-revalidate window (milliseconds) | | `staleIfError` | `number \| LayerTtlMap` | - | Global stale-if-error window (milliseconds) | | `ttlJitter` | `number \| LayerTtlMap` | - | Global TTL jitter (milliseconds) | | `refreshAhead` | `number \| LayerTtlMap` | - | Global refresh-ahead threshold (milliseconds) | | `adaptiveTtl` | `boolean \| AdaptiveTtlOptions` | - | Auto-ramp TTLs for hot keys | | `circuitBreaker` | `CircuitBreakerOptions` | - | Per-fetcher failure tracking | | `gracefulDegradation` | `boolean \| { retryAfterMs: number }` | - | Skip failed layers temporarily | | `writePolicy` | `'strict' \| 'best-effort'` | `'strict'` | Write failure behavior | | `writeStrategy` | `'write-through' \| 'write-behind'` | `'write-through'` | Write batching strategy | | `writeBehind` | `WriteBehindOptions` | - | Batch size, flush interval, max queue | | `writeCoordination` | `{ maxPendingWrites?; maxActiveKeys?; maxPendingWritesPerKey? }` | `10000 / 10000 / 1000` | Finite admission limits for per-key write ordering state | | `fetcherRateLimit` | `RateLimitOptions` | - | Global rate limiting | | `backgroundRefreshTimeoutMs` | `number` | `30000` | Max time for stale refresh attempts | | `singleFlightCoordinator` | `RedisSingleFlightCoordinator` | - | Distributed deduplication | | `singleFlightLeaseMs` | `number` | `30000` | Distributed lock duration | | `singleFlightTimeoutMs` | `number` | `5000` | Wait timeout for distributed lock | | `singleFlightPollMs` | `number` | `50` | Polling interval | | `singleFlightRenewIntervalMs` | `number` | - | Lease renewal cadence | | `snapshotBaseDir` | `string \| false` | `process.cwd()` | Base directory for file snapshots | | `snapshotMaxBytes` | `number \| false` | - | Max snapshot file size | | `snapshotMaxEntries` | `number \| false` | - | Max entries in a snapshot | | `invalidationMaxKeys` | `number \| false` | - | Safety limit for invalidation scans | | `maxProfileEntries` | `number` | `100000` | Max size before pruning internal maps | All single-key and bulk writes use the same per-key ordering boundary so stale cleanup cannot overtake newer data. When a `writeCoordination` limit is reached, the write rejects with `CacheWriteSaturationError`; treat it as backpressure or increase a limit only after sizing the expected burst. ### CircuitBreakerOptions | Option | Type | Default | Description | | ------------------ | ------------------- | ------- | ------------------------------------------------------------ | | `failureThreshold` | `number` | `3` | Consecutive failures before opening the circuit | | `cooldownMs` | `number` | `30000` | Milliseconds before another fetch attempt is allowed | | `scope` | `'key' \| 'shared'` | `'key'` | Use per-key buckets or one shared bucket for these options | | `breakerKey` | `string` | - | Explicit bucket id for grouping related backend dependencies | If `breakerKey` is provided, it selects the explicit bucket id and takes precedence over `scope`. Otherwise `scope: 'key'` creates one bucket per cache key, while `scope: 'shared'` uses one shared bucket for all keys using those options. ```ts await cache.get('user:1', fetchFromApi, { circuitBreaker: { failureThreshold: 2, cooldownMs: 60_000, scope: 'shared', breakerKey: 'users-api' } }) ``` ### RateLimitOptions | Option | Type | Default | Description | | ---------------- | -------------------------------- | ---------- | ---------------------------------------------------------- | | `maxConcurrent` | `number` | - | Maximum concurrent fetchers in the selected bucket | | `intervalMs` | `number` | - | Rate-limit window size in milliseconds | | `maxPerInterval` | `number` | - | Maximum fetches per interval | | `scope` | `'global' \| 'key' \| 'fetcher'` | `'global'` | Bucket by all fetches, cache key, or fetcher function | | `bucketKey` | `string` | - | Explicit bucket id for related work | | `queueOverflow` | `'reject' \| 'bypass'` | `'reject'` | Reject saturated queues or deliberately bypass the limiter | ### Per-Operation Options | Option | Type | Description | | ---------------------- | ------------------------------------- | ------------------------------------------------------------------------------ | | `tags` | `string[]` | Tags for tag-based invalidation | | `ttl` | `number \| LayerTtlMap` | TTL in milliseconds, or per-layer overrides | | `ttlPolicy` | `string \| object \| function` | `'until-midnight'`, `'next-hour'`, `{ alignTo }`, or custom | | `negativeCache` | `boolean` | Cache absent results as empty entries | | `cacheNullValues` | `boolean` | Cache null fetcher results as regular values (default `true`) | | `negativeTtl` | `number` | Short TTL for misses | | `staleWhileRevalidate` | `number \| LayerTtlMap` | Return stale and refresh in background | | `staleIfError` | `number \| LayerTtlMap` | Keep serving stale if refresh fails | | `ttlJitter` | `number \| LayerTtlMap` | +/- random jitter on expiry | | `slidingTtl` | `boolean` | Reset TTL on every read | | `refreshAhead` | `number` | Trigger background refresh when TTL drops below threshold | | `adaptiveTtl` | `AdaptiveTtlOptions` | Auto-ramp TTL for hot keys | | `circuitBreaker` | `CircuitBreakerOptions` | Per-operation circuit breaker | | `fetcherRateLimit` | `RateLimitOptions` | Per-operation rate limiting | | `contextOptions` | `(context) => CacheEntryWriteOptions` | Override stored entry TTLs/tags from `{ key, value, kind }` right before write | | `shouldCache` | `(value: T) => boolean` | Predicate to skip caching specific results | *** ## Invalidation Strategies ### Tag Invalidation ```ts await cache.set('user:123', user, { tags: ['user', 'user:123'] }) await cache.invalidateByTag('user:123') ``` ### Exact-Key Invalidation ```ts await cache.invalidateByKey('user:123') // alias for delete() await cache.invalidateByKeys(['user:123', 'user:456']) // alias for mdelete() ``` ### Batch Tag Invalidation ```ts await cache.invalidateByTags(['tenant:a', 'users'], 'all') await cache.invalidateByTags(['users', 'posts'], 'any') ``` ### Wildcard Invalidation ```ts await cache.invalidateByPattern('user:*') ``` ### Prefix Invalidation ```ts await cache.invalidateByPrefix('user:123:') ``` ### Expiration Without Deletion Use the `expireBy*` counterparts when stale serving is preferable to removing values immediately. ```ts await cache.expireByTag('user:123') await cache.expireByKey('user:123') await cache.expireByTags(['tenant:a', 'users'], 'all') await cache.expireByKeys(['user:123', 'user:456']) await cache.expireByPattern('user:*') await cache.expireByPrefix('user:123:') ``` ### Generation-Based Invalidation ```ts cache.bumpGeneration() // instant bulk invalidation without scanning ``` *** ## Freshness Strategies ### Stale-While-Revalidate ```ts await cache.set('config', config, { ttl: 60_000, staleWhileRevalidate: 30_000, // serve stale for 30s while refreshing staleIfError: 300_000 // serve stale for 5min if refresh fails }) ``` ### Sliding TTL ```ts await cache.get('session:abc', fetchSession, { slidingTtl: true }) ``` ### Adaptive TTL ```ts await cache.get('popular-post', fetchPost, { adaptiveTtl: { hotAfter: 5, step: 60_000, maxTtl: 3_600_000 } }) ``` Adaptive TTL counters are process-local. In multi-instance deployments, each Node.js process ramps TTLs from its own observed hits, so use explicit TTLs or a shared Redis counter when every instance must make the same TTL decision. ### Refresh-Ahead ```ts await cache.get('leaderboard', fetchLeaderboard, { ttl: 120_000, refreshAhead: 30_000 // refresh when <= 30s remain }) ``` ### TTL Policies ```ts await cache.set('daily-report', report, { ttlPolicy: 'until-midnight' }) await cache.set('hourly-rollup', rollup, { ttlPolicy: 'next-hour' }) await cache.set('aligned', value, { ttlPolicy: { alignTo: 300_000 } }) await cache.set('custom', value, { ttlPolicy: ({ key }) => key.startsWith('hot:') ? 30_000 : 300_000 }) ``` ### Context-Aware Entry Options ```ts await cache.get('oauth:token', fetchToken, { ttl: 300_000, contextOptions: ({ value }) => { const token = value as { refreshExpiresInMs: number; tenantId: string } return { ttl: Math.max(1, token.refreshExpiresInMs), tags: ['oauth', `tenant:${token.tenantId}`] } } }) ``` `contextOptions()` runs immediately before a cache write and overrides static entry settings on the same call. Use it for value-dependent `ttl`, `negativeTtl`, `staleWhileRevalidate`, `staleIfError`, `ttlJitter`, `adaptiveTtl`, or `tags`. ### Per-Layer TTL Overrides ```ts await cache.set('session:abc', data, { ttl: { memory: 30_000, redis: 3_600_000 } }) ``` ### Conditional Caching ```ts const data = await cache.get('api:response', fetchFromApi, { shouldCache: (value) => (value as any).status === 200 }) ``` *** ## Resilience ### Graceful Degradation ```ts new CacheStack([...], { gracefulDegradation: { retryAfterMs: 10_000 } }) ``` ### Circuit Breaker ```ts new CacheStack([...], { circuitBreaker: { failureThreshold: 5, cooldownMs: 30_000 } }) // Per-operation await cache.get('fragile-key', fetch, { circuitBreaker: { failureThreshold: 3, cooldownMs: 10_000 } }) ``` ### Write Policies ```ts // Strict (default): fail if any layer fails new CacheStack([...], { writePolicy: 'strict' }) // Best-effort: only fail if every layer fails new CacheStack([...], { writePolicy: 'best-effort' }) ``` ### Scoped Fetcher Rate Limiting ```ts await cache.get('user:123', fetchUser, { fetcherRateLimit: { maxConcurrent: 1, scope: 'key' } }) ``` *** ## Compression & Serialization ### Compression ```ts new RedisLayer({ client: redis, ttl: 300_000, compression: 'gzip', // or 'brotli' compressionThreshold: 1_024 // skip compression for small values }) ``` For large payloads such as HTML fragments, denormalized API responses, or MB-scale documents, prefer `compression: 'brotli'` with a threshold around `1_024 * 1_024` so small values avoid compression overhead while large values pay less network cost. ### MessagePack Serializer ```ts import { MsgpackSerializer } from 'layercache' new RedisLayer({ client: redis, ttl: 300_000, serializer: new MsgpackSerializer() }) ``` *** ## Distributed Features ### Distributed Single-Flight ```ts import { RedisSingleFlightCoordinator } from 'layercache' const coordinator = new RedisSingleFlightCoordinator({ client: redis }) new CacheStack([...], { singleFlightCoordinator: coordinator, singleFlightLeaseMs: 30_000, singleFlightRenewIntervalMs: 10_000, }) ``` ### Cross-Server L1 Invalidation ```ts import { RedisInvalidationBus } from 'layercache' const bus = new RedisInvalidationBus({ publisher: redis, subscriber: new Redis(), signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET }) new CacheStack([...], { invalidationBus: bus, broadcastL1Invalidation: true }) ``` ### Distributed Tag Index ```ts import { RedisTagIndex } from 'layercache' const tagIndex = new RedisTagIndex({ client: redis, prefix: 'myapp:tag-index', knownKeysShards: 16 }) new CacheStack([...], { tagIndex }) ``` *** ## Event Hooks `CacheStack` extends `EventEmitter`: | Event | Payload | | ----------------- | ----------------------------- | | `hit` | `{ key, layer }` | | `miss` | `{ key }` | | `set` | `{ key }` | | `delete` | `{ key }` | | `stale-serve` | `{ key, state, layer }` | | `stampede-dedupe` | `{ key }` | | `backfill` | `{ key, fromLayer, toLayer }` | | `warm` | `{ key }` | | `error` | `{ event, context }` | ```ts cache.on('hit', ({ key, layer }) => metrics.inc('cache.hit', { layer })) cache.on('miss', ({ key }) => metrics.inc('cache.miss')) cache.on('error', ({ event, context }) => logger.error(event, context)) ``` *** ## Framework Integrations ### Express ```ts import { createExpressCacheMiddleware } from 'layercache' app.get('/api/users', createExpressCacheMiddleware(cache, { ttl: 30_000, tags: ['users'], keyResolver: (req) => `user:${req.url}` }), handler) ``` ### Fastify ```ts import { createFastifyLayercachePlugin } from 'layercache' await fastify.register(createFastifyLayercachePlugin(cache, { statsPath: '/cache/stats' })) ``` ### Hono ```ts import { createHonoCacheMiddleware } from 'layercache' app.use('/api/*', createHonoCacheMiddleware(cache, { ttl: 60_000 })) ``` ### tRPC ```ts import { createTrpcCacheMiddleware } from 'layercache' // keyResolver is required: it must include every input and context attribute // that affects procedure output (including the authenticated user, if any). const cacheMiddleware = createTrpcCacheMiddleware(cache, 'trpc', { keyResolver: (input, path, _type, context) => `${context?.id ?? 'anon'}:${path}:${JSON.stringify(input)}`, ttl: 60_000 }) export const cachedProcedure = t.procedure.use(cacheMiddleware) ``` ### GraphQL ```ts import { cacheGraphqlResolver } from 'layercache' const resolvers = { Query: { user: cacheGraphqlResolver(cache, 'user', (_root, { id }) => db.findUser(id), { keyResolver: (_root, { id }) => id, ttl: 300_000 }) } } ``` ### NestJS Use `CacheStack` directly in your NestJS modules: ```ts import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' import Redis from 'ioredis' @Module({ providers: [ { provide: 'CACHE', useFactory: () => new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: new Redis(), ttl: 300_000 }) ]) } ], exports: ['CACHE'] }) export class CacheModule {} ``` > **Note:** The separate `@cachestack/nestjs` package was removed in v1.3.2. Import directly from `layercache` instead. ### OpenTelemetry ```ts import { createOpenTelemetryPlugin } from 'layercache' createOpenTelemetryPlugin(cache, tracer) ``` By default the plugin exports `layercache.key_hash` instead of raw cache keys. Raw key attributes are available only when explicitly requested: ```ts createOpenTelemetryPlugin(cache, tracer, { includeRawKeyAttributes: true }) ``` ### Stats HTTP Handler ```ts import { createCacheStatsHandler } from 'layercache' import http from 'node:http' const statsHandler = createCacheStatsHandler(cache) http.createServer(statsHandler).listen(9090) ``` *** ## Admin CLI Inspect and manage Redis-backed caches from the terminal. ```bash npx layercache stats --redis redis://localhost:6379 npx layercache keys --redis redis://localhost:6379 --pattern "user:*" npx layercache invalidate --redis redis://localhost:6379 --tag user:123 npx layercache invalidate --redis redis://localhost:6379 --pattern "session:*" ``` Full-cache invalidation requires `--force`, both for the default pattern and for every wildcard-only combination of `*` and `?` such as `*`, `**`, and `?*`: ```bash npx layercache invalidate --redis redis://localhost:6379 --pattern "*" --force npx layercache invalidate --redis redis://localhost:6379 --pattern "?*" --force ``` *** ## Debug Logging ```bash DEBUG=layercache:debug node server.js ``` Or pass a logger instance: ```ts new CacheStack([...], { logger: { debug(message, context) { myLogger.debug(message, context) } } }) ``` --- url: /docs/cli.md --- # CLI Tool Layercache provides a command-line interface for inspecting and managing Redis-backed caches. Use it to check statistics, list keys, inspect values, and invalidate data without writing code. ## Installation ### Global Install ```sh [npm] npm install -g layercache@latest ``` ```sh [yarn] yarn add -g layercache@latest ``` ```sh [pnpm] pnpm add -g layercache@latest ``` ```sh [bun] bun add -g layercache@latest ``` ```sh [deno] deno add -g npm:layercache@latest ``` ### Using npx No installation required: ```sh [npx] npx layercache stats --redis redis://localhost:6379 ``` ```sh [yarn] yarn layercache stats --redis redis://localhost:6379 ``` ```sh [pnpm] pnpm layercache stats --redis redis://localhost:6379 ``` ```sh [bun] bun layercache stats --redis redis://localhost:6379 ``` ```sh [deno] deno run -A npm:layercache stats --redis redis://localhost:6379 ``` ## Commands ### stats Display cache statistics with optional pattern filtering. #### Basic Usage ```bash layercache stats --redis redis://localhost:6379 ``` #### Response ```json { "totalKeys": 1234, "pattern": "*" } ``` #### With Pattern Filter ```bash layercache stats --redis redis://localhost:6379 --pattern "user:*" ``` #### Use Cases - Check cache size before deployment - Monitor key growth over time - Verify pattern-based key distribution ### keys List all cached keys matching a pattern. #### Basic Usage ```bash layercache keys --redis redis://localhost:6379 ``` #### Output ``` user:1 user:2 user:3 session:abc123 config:app ... ``` #### With Pattern Filter ```bash layercache keys --redis redis://localhost:6379 --pattern "user:*" ``` #### Output ``` user:1 user:2 user:3 ... ``` #### Use Cases - Find all keys for a specific entity - Debug cache key patterns - Export key lists for analysis - Verify tag index contents ### inspect Inspect a specific cache key to view metadata and value. #### Basic Usage ```bash layercache inspect --redis redis://localhost:6379 --key "user:123" ``` #### Response ```json { "key": "user:123", "exists": true, "ttlMs": 245, "sizeBytes": 1024, "isEnvelope": true, "state": "fresh", "preview": { "kind": "fresh", "value": { "id": 123, "name": "John Doe", "email": "john@example.com" }, "freshUntil": 1712848000000, "staleUntil": 1712848300000, "errorUntil": 1712848900000 } } ``` #### Non-Existent Key ```json { "key": "user:999", "exists": false, "ttlMs": null, "sizeBytes": 0, "isEnvelope": false, "state": null, "preview": null } ``` #### Field Descriptions - **exists** - Whether the key exists in Redis - **ttlMs** - Remaining TTL in milliseconds (-1 if no expiry, -2 if key does not exist) - **sizeBytes** - Size of the stored value in bytes - **isEnvelope** - Whether the value is a Layercache envelope (vs raw) - **state** - Cache state: `fresh`, `stale`, or `error` - **preview** - Value preview with metadata #### Use Cases - Debug cache contents - Verify TTL configuration - Check value state (fresh/stale/error) - Inspect envelope metadata ### invalidate Invalidate cached data by pattern or tag. #### By Pattern ```bash layercache invalidate --redis redis://localhost:6379 --pattern "user:*" ``` #### Response ```json { "deletedKeys": 123, "pattern": "user:*" } ``` #### By Tag ```bash layercache invalidate --redis redis://localhost:6379 --tag "user:123" ``` #### Response ```json { "deletedKeys": 5, "tag": "user:123" } ``` #### Safety Guard for Untargeted Invalidation Running `invalidate` without `--pattern` or `--tag` defaults to matching all keys (`*`). Every explicit pattern made only from `*` and `?`, including `*`, `**`, and `?*`, is also treated as a full-cache invalidation. These forms require `--force` to prevent accidental cache wipes: ```bash # This shows a warning and aborts layercache invalidate --redis redis://localhost:6379 # Explicitly confirm full invalidation layercache invalidate --redis redis://localhost:6379 --force layercache invalidate --redis redis://localhost:6379 --pattern "*" --force layercache invalidate --redis redis://localhost:6379 --pattern "?*" --force ``` #### Custom Tag Index Prefix ```bash layercache invalidate \ --redis redis://localhost:6379 \ --tag "tenant:abc" \ --tag-index-prefix "myapp:tag-index" ``` #### Use Cases - Bulk invalidation after data updates - Clear user-specific caches on logout - Invalidate by entity type (e.g., all products) - Clear tenant-specific data ### migrate-tag-index Migrate legacy RedisTagIndex known-key sets into the current sharded layout. #### Basic Usage ```bash layercache migrate-tag-index \ --redis rediss://prod-redis.example.com:6380 \ --tag-index-prefix "myapp:tag-index" ``` #### Response ```json { "migratedKeys": 12500 } ``` #### Custom Shard Count ```bash layercache migrate-tag-index \ --redis rediss://prod-redis.example.com:6380 \ --tag-index-prefix "myapp:tag-index" \ --known-key-shards 16 ``` Use this after upgrading an existing RedisTagIndex that previously stored known keys in a single `<prefix>:keys` set. New RedisTagIndex instances default to 16 known-key shards. ## Options ### Global Options #### `--redis <url>` Redis connection URL (required). ```bash --redis redis://localhost:6379 --redis redis://user:pass@localhost:6379/2 --redis rediss://prod-redis.example.com:6380 # TLS ``` Supported formats: - `redis://[user:pass@]host[:port][/db]` - `rediss://[user:pass@]host[:port][/db]` (TLS) - `host:port` (simplified) - `host` (defaults to port 6379) When `NODE_ENV=production`, plaintext `redis://` URLs are rejected before any Redis connection is attempted. Use `rediss://` for production Redis endpoints. ```bash NODE_ENV=production layercache stats --redis redis://prod-redis.example.com:6379 # Error: refusing plaintext redis:// connection because NODE_ENV=production. ``` ### Command-Specific Options #### `--pattern <glob>` Glob pattern for filtering keys (used with `stats`, `keys`, `invalidate`). ```bash # All user keys --pattern "user:*" # Specific user --pattern "user:123:*" # Multiple patterns --pattern "user:*" --pattern "session:*" ``` #### `--key <key>` Exact cache key to inspect (used with `inspect`). ```bash --key "user:123" --key "config:app" ``` #### `--tag <tag>` Tag for invalidation (used with `invalidate`). ```bash --tag "user:123" --tag "tenant:abc" ``` #### `--tag-index-prefix <prefix>` Redis key prefix for tag index (used with `invalidate` and `migrate-tag-index`). ```bash --tag-index-prefix "layercache:tag-index" --tag-index-prefix "myapp:cache:tags" ``` #### `--known-key-shards <count>` Shard count for `migrate-tag-index`. Defaults to the RedisTagIndex default of 16 shards. ```bash layercache migrate-tag-index \ --redis rediss://localhost:6380 \ --tag-index-prefix "myapp:tag-index" \ --known-key-shards 16 ``` #### `--limit <count>` Maximum Redis keys to scan for `stats`, `keys`, and pattern-based `invalidate`. Defaults to 100,000. ```bash layercache keys --redis redis://localhost:6379 --pattern "user:*" --limit 250000 ``` #### `--require-tls` Require TLS connection. Fails if the Redis URL uses `redis://` (plaintext) instead of `rediss://`. ```bash layercache stats --redis redis://localhost:6379 --require-tls # Error: --require-tls is set but the URL uses redis:// (plaintext). ``` #### `--allow-plaintext` Explicitly allow `redis://` when `NODE_ENV=production`. Prefer `rediss://`; this flag is only for controlled environments where plaintext Redis is intentionally accepted. ```bash NODE_ENV=production layercache stats --redis redis://localhost:6379 --allow-plaintext ``` #### `--force` Bypass the safety guard for full-cache `invalidate` operations. Required when running `invalidate` without `--pattern` or `--tag`, and when using a wildcard-only pattern made from `*` and `?`. ```bash # Without --force: warns and aborts layercache invalidate --redis redis://localhost:6379 # With --force: proceeds with full invalidation layercache invalidate --redis redis://localhost:6379 --force layercache invalidate --redis redis://localhost:6379 --pattern "*" --force layercache invalidate --redis redis://localhost:6379 --pattern "?*" --force ``` ## Usage Examples ### Check Cache Health ```bash # Total keys in cache layercache stats --redis redis://localhost:6379 # Keys by pattern layercache stats --redis redis://localhost:6379 --pattern "user:*" layercache stats --redis redis://localhost:6379 --pattern "session:*" layercache stats --redis redis://localhost:6379 --pattern "config:*" ``` ### Debug Cache Issues ```bash # Inspect a specific key layercache inspect --redis redis://localhost:6379 --key "user:123" # Check if key exists and its TTL layercache inspect --redis redis://localhost:6379 --key "config:app" # View all keys for a user layercache keys --redis redis://localhost:6379 --pattern "user:123:*" ``` ### Invalidate Data ```bash # Invalidate all user caches layercache invalidate --redis redis://localhost:6379 --pattern "user:*" # Invalidate specific user layercache invalidate --redis redis://localhost:6379 --tag "user:123" # Invalidate all sessions layercache invalidate --redis redis://localhost:6379 --pattern "session:*" # Invalidate tenant data layercache invalidate --redis redis://localhost:6379 --tag "tenant:abc" ``` ### Deployment Scripts ```bash #!/bin/bash # deploy.sh - Clear caches before deployment REDIS_URL="rediss://prod-redis.example.com:6380" # Check current state echo "Current cache size:" layercache stats --redis "$REDIS_URL" --require-tls # Clear all application caches echo "Clearing application caches..." layercache invalidate --redis "$REDIS_URL" --pattern "app:*" --require-tls # Verify cleared echo "After invalidation:" layercache stats --redis "$REDIS_URL" --pattern "app:*" --require-tls ``` ### Monitoring Scripts ```bash #!/bin/bash # monitor-cache.sh - Monitor cache growth REDIS_URL="redis://localhost:6379" while true; do echo "$(date): Total keys" layercache stats --redis "$REDIS_URL" echo "$(date): User keys" layercache stats --redis "$REDIS_URL" --pattern "user:*" echo "---" sleep 300 # Every 5 minutes done ``` ### Cron Jobs ```bash # Clear expired sessions hourly 0 * * * * layercache invalidate --redis redis://localhost:6379 --pattern "session:expired:*" # Invalidate stale configs daily 0 3 * * * layercache invalidate --redis redis://localhost:6379 --pattern "config:daily:*" # Report cache size weekly 0 9 * * 1 layercache stats --redis redis://localhost:6379 > /var/log/cache-size.log ``` ## Error Handling ### Connection Errors ```bash # Invalid URL $ layercache stats --redis "invalid-url" Error: Failed to connect to Redis at invalid-url: connect ECONNREFUSED # Missing --redis flag $ layercache stats Error: --redis requires a value (e.g. redis://localhost:6379) ``` ### Scan Limits The CLI stops scanning after 100,000 keys by default to prevent runaway scans. Raise the cap with `--limit` when you intentionally need a larger scan. ```bash $ layercache keys --redis redis://localhost:6379 --pattern "*" Warning: stopped scanning after 100000 keys. Use --limit to raise the scan cap. ``` ### Missing Keys ```bash # Inspect non-existent key $ layercache inspect --redis redis://localhost:6379 --key "missing" { "key": "missing", "exists": false, "ttlMs": null, "sizeBytes": 0, "isEnvelope": false, "state": null, "preview": null } ``` ## Best Practices ### 1. Use Specific Patterns Avoid scanning all keys: ```bash # Bad - scans entire database layercache keys --redis redis://localhost:6379 --pattern "*" # Good - specific pattern layercache keys --redis redis://localhost:6379 --pattern "user:*" ``` ### 2. Use Tag Invalidation Prefer tags over pattern matching: ```bash # Slower - scans and deletes each key layercache invalidate --redis redis://localhost:6379 --pattern "user:123:*" # Faster - uses tag index layercache invalidate --redis redis://localhost:6379 --tag "user:123" ``` ### 3. Verify Before Invalidating Check what will be invalidated: ```bash # First, check what matches layercache stats --redis redis://localhost:6379 --pattern "user:*" # Then invalidate layercache invalidate --redis redis://localhost:6379 --pattern "user:*" ``` ### 4. Use Connection Pooling For frequent CLI operations, use a connection pool or reuse connections in scripts. ### 5. Set Timeouts For slow Redis instances, set connection timeouts: ```bash # The CLI uses 5-second connect timeout # Adjust by modifying Redis client options in your scripts ``` ## Integration with CI/CD ### GitHub Actions ```yaml name: Deploy on: push: branches: [main] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Clear caches run: | npx layercache invalidate \ --redis ${{ secrets.REDIS_URL }} \ --pattern "app:*" ``` ### Docker ```dockerfile FROM node:20-alpine RUN npm install -g layercache # Use in entrypoint script COPY entrypoint.sh / RUN chmod +x /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"] ``` ```bash #!/bin/bash # entrypoint.sh # Clear caches on startup layercache invalidate --redis "$REDIS_URL" --pattern "temp:*" # Start application exec node server.js ``` ## Advanced Usage ### Piping to Other Tools ```bash # Count keys layercache keys --redis redis://localhost:6379 --pattern "user:*" | wc -l # Search for specific user layercache keys --redis redis://localhost:6379 | grep "user:123" # Export to file layercache keys --redis redis://localhost:6379 --pattern "user:*" > user-keys.txt # Analyze with jq layercache inspect --redis redis://localhost:6379 --key "user:123" | jq '.preview.value' ``` ### Batch Operations ```bash # Invalidate multiple tags for tag in user:123 user:456 user:789; do layercache invalidate --redis redis://localhost:6379 --tag "$tag" done # Check multiple patterns for pattern in user:* session:* config:*; do echo "Pattern: $pattern" layercache stats --redis redis://localhost:6379 --pattern "$pattern" done ``` ### Monitoring with Grafana Export stats to Prometheus via a cron job: ```bash #!/bin/bash # Export to Prometheus pushgateway STATS=$(layercache stats --redis redis://localhost:6379 --pattern "user:*") cat < Math.min(times * 50, 2000) }) // 1. Distributed single-flight const coordinator = new RedisSingleFlightCoordinator({ client: redis, prefix: 'myapp:singleflight' }) // 2. Cross-server invalidation const bus = new RedisInvalidationBus({ publisher: redis, subscriber: new Redis(), channel: 'myapp:invalidation', signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET, logger: console }) // 3. Shared tag index const tagIndex = new RedisTagIndex({ client: redis, prefix: 'myapp:tag-index', knownKeysShards: 16 }) // 4. Create cache stack const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ], { // Distributed single-flight singleFlightCoordinator: coordinator, singleFlightLeaseMs: 30000, singleFlightTimeoutMs: 5000, singleFlightPollMs: 50, // L1 invalidation invalidationBus: bus, broadcastL1Invalidation: true, // Tag index tagIndex }) // 5. Use the cache await cache.set('user:123', user, { tags: ['user:123'] }) const user = await cache.get('user:123', fetchUser) await cache.invalidateByTag('user:123') // 6. Graceful shutdown process.on('SIGTERM', async () => { await cache.disconnect() await redis.quit() process.exit(0) }) ``` ## Architecture Diagram ``` ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Server A │ │ Server B │ │ Redis │ ├─────────────┤ ├─────────────┤ ├─────────────┤ │ Memory L1 │ │ Memory L1 │ │ │ │ Redis L2 │ │ Redis L2 │ │ Data Store │ │ │ │ │ │ │ │ Coordinator │◄────┤ Coordinator │◄────│ Locks │ │ Bus Pub │ │ Bus Sub │◄────│ Pub/Sub │ │ Tag Index │◄────┤ Tag Index │◄────│ Tag Sets │ └─────────────┘ └─────────────┘ └─────────────┘ ``` ## Performance Considerations ### Single Flight - **Lock acquisition**: \~1-2ms (SET NX) - **Polling overhead**: \~50ms intervals - **Lease renewal**: Background, non-blocking ### Invalidation Bus - **Publish latency**: \<1ms - **Message size**: \~1KB per invalidation - **Subscriber lag**: Network-dependent ### Tag Index - **Set operations**: O(1) per key - **Tag invalidation**: O(n) where n = keys with tag - **Sharding**: Reduces contention by factor of shards ### Recommended Settings For high-traffic deployments: ```tsx const cache = new CacheStack([/* ... */], { // Single-flight: longer leases for slow fetchers singleFlightLeaseMs: 60000, singleFlightRenewIntervalMs: 10000, singleFlightPollMs: 100, // Tag index: more shards for scale tagIndex: new RedisTagIndex({ client: redis, knownKeysShards: 16 }) }) ``` ## Monitoring ### Track Coordinator Operations ```tsx cache.on('error', ({ event, context }) => { if (context.operation === 'singleFlight') { console.error('Single-flight error:', context.error) } }) ``` ### Monitor Bus Messages ```tsx cache.on('invalidation', ({ keys, sourceId }) => { console.log(`Invalidated ${keys.length} keys from ${sourceId}`) }) ``` ### Check Tag Index Size ```tsx const redis = new Redis() const keys = await redis.keys('layercache:tag-index:tag:*') console.log(`Tracking ${keys.length} tags`) ``` ## Troubleshooting ### Lock Timeouts If servers timeout waiting for locks: ```tsx // Increase lease duration singleFlightLeaseMs: 60000 // Increase wait timeout singleFlightTimeoutMs: 10000 ``` ### Stale L1 Data If memory layers aren't invalidating: ```tsx // Ensure broadcast is enabled broadcastL1Invalidation: true // Check bus connection bus.on('error', (err) => console.error('Bus error:', err)) ``` ### Slow Tag Invalidations If tag invalidations are slow: ```tsx // Increase shards for parallelization tagIndex: new RedisTagIndex({ client: redis, knownKeysShards: 16 // More shards }) ``` ### High Redis Memory If tag index uses too much memory: ```tsx // Reduce scan batch size tagIndex: new RedisTagIndex({ client: redis, scanCount: 50 // Fewer keys per SSCAN }) ``` --- url: /docs/getting-started.md --- # Getting Started ## Installation ```sh [npm] npm install layercache@latest ``` ```sh [yarn] yarn add layercache@latest ``` ```sh [pnpm] pnpm add layercache@latest ``` ```sh [bun] bun add layercache@latest ``` ```sh [deno] deno add npm:layercache@latest ``` ## Basic Setup The simplest cache stack uses a single memory layer: ```ts import { CacheStack, MemoryLayer } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }) ]) ``` ### Reading Through the Cache Use the read-through pattern to fetch data with automatic caching: ```ts const user = await cache.get('user:123', () => db.findUser(123) ) // On first call: fetcher runs, result is cached // On subsequent calls: serves from memory, no fetcher execution ``` ## Multi-Layer Setup For production, add Redis for cross-process sharing: ```ts import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' import Redis from 'ioredis' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000, maxSize: 1_000 }), // L1: in-process new RedisLayer({ client: new Redis(), ttl: 3_600_000 }), // L2: shared ]) ``` Use `rediss://` Redis URLs for production CLI operations and hosted Redis endpoints. The CLI rejects plaintext `redis://` URLs under `NODE_ENV=production` unless you explicitly pass `--allow-plaintext`. ### How Layered Reads Work When you call `cache.get()`: 1. **L1 Memory** is checked first (\~0.01ms) 2. If missing, **L2 Redis** is checked (\~0.5ms) 3. If also missing, your **fetcher** runs (\~20ms+) 4. On any partial hit, upper layers are **backfilled** automatically ## Three-Layer Setup with Disk Persistence Add disk persistence for fault tolerance: ```ts import { CacheStack, MemoryLayer, RedisLayer, DiskLayer } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000, maxSize: 5_000 }), new RedisLayer({ client: new Redis(), ttl: 3_600_000, compression: 'gzip' }), new DiskLayer({ directory: './var/cache', maxFiles: 10_000 }), ]) ``` ## Key Configuration Options ### MemoryLayer ```ts new MemoryLayer({ ttl: 60_000, // Time-to-live in milliseconds maxSize: 1000, // Max number of entries (LRU eviction) }) ``` ### RedisLayer ```ts new RedisLayer({ client: new Redis(), // ioredis client ttl: 3_600_000, // Time-to-live in milliseconds compression: 'gzip', // 'gzip' | 'brotli' | false compressionThreshold: 1024, // Min bytes to compress }) ``` ### DiskLayer ```ts new DiskLayer({ directory: './var/cache', // Storage directory maxFiles: 10_000, // Max files (LRU eviction) }) ``` ## Common Patterns ### Function Wrapping Transparently cache any function with `wrap()`: ```ts const cachedFetch = cache.wrap('users', async (id: string) => { return db.findUser(id) }) const user = await cachedFetch('123') // Uses key "users:123" automatically ``` ### Namespacing Create scoped cache views with prefixes: ```ts const userCache = cache.namespace('user') const productCache = cache.namespace('product') await userCache.set('123', data) await productCache.set('456', data) // Stored as "user:123" and "product:456" ``` ### Bulk Operations Set and get multiple keys efficiently: ```ts await cache.setMany([ { key: 'user:1', value: { name: 'Alice' } }, { key: 'user:2', value: { name: 'Bob' } }, ]) const users = await cache.getMany(['user:1', 'user:2'], (keys) => db.findUsers(keys) ) ``` ## Next Steps - **[Tutorial](/docs/tutorial.md)** — Learn stampede prevention, tag invalidation, and more - **[API Reference](/docs/api.md)** — Complete method documentation - **[Integrations](/docs/integrations.md)** — Use with Express, Fastify, NestJS, and more --- url: /docs/index.md --- # Documentation Welcome to the Layercache documentation. Layercache stacks memory, Redis, and disk behind a single API with stampede prevention, tag invalidation, stale-while-revalidate, and full observability. ## How It Works Every read follows the same path: 1. Check **L1 Memory** first (fastest, in-process). 2. If miss, check **L2 Redis** (shared across instances). 3. If miss, check **L3 Disk** (persistent fallback). 4. If all miss, run the **origin fetcher** once (single-flight). 5. Backfill upper layers so next reads return faster. Layercache keeps responses stable under pressure with stale serving, circuit breakers, and timeout guards. ## Version 5.0 Highlights Layercache 5.0 is a major release because it hardens cache isolation so authenticated responses are never served across users: - **Breaking:** `createTrpcCacheMiddleware` and `cacheGraphqlResolver` now require a `keyResolver` — the `allowImplicitContextCaching` option is removed. Implicit path+input or argument-only keys could not distinguish authenticated callers. - Express and Hono middlewares bypass implicit URL-only caching when requests carry authentication headers (`authorization`, `cookie`, `set-cookie`, `x-api-key`, `x-session-id`, `x-auth-token`, `x-forwarded-user`), so one user's authenticated response is never served to another. Provide a `keyResolver` to opt back in for safe keys. - `RedisInvalidationBus` supports `requireSignature` to fail fast when a signing secret is missing, preventing forged invalidation messages on shared Redis channels. - The test suite is split into unit and real-Redis Vitest projects, and docker-compose Redis uses the configurable `REDIS_PORT`. Read the [migration guide](/docs/migration.md#upgrading-to-50) before upgrading an existing deployment. ## Version 4.0 Highlights Layercache 4.0 makes missing values unambiguous and hardens production coordination boundaries: - Public cache reads return `undefined` on misses while preserving intentional cached `null` values. - Read-through fetchers cache `null` by default; `cacheNullValues: false` retains legacy null-as-absence behavior. - Structured `wrap()` keys use the collision-resistant `j2:` schema, and write ordering plus generation cleanup have finite limits. - Snapshot commits, signed invalidation, HTTP credential handling, destructive CLI patterns, and playground isolation are hardened. - Regression coverage includes the merged scheduler, snapshot, invalidation, and epoch rollover fixes. Read the [migration guide](/docs/migration.md#upgrading-to-40) before upgrading an existing deployment. ## Quick Links - **[Getting Started](/docs/getting-started.md)** — Install and configure your first cache stack - **[Tutorial](/docs/tutorial.md)** — 10-step walkthrough of production features - **[API Reference](/docs/api.md)** — Complete method and option documentation - **[Integrations](/docs/integrations.md)** — Express, Fastify, NestJS, Hono, tRPC, GraphQL --- url: /docs/integrations.md --- # Framework Integrations Layercache provides first-class integrations with popular Node.js frameworks and observability tools. Each integration is designed to be lightweight, type-safe, and minimal in configuration. ## Express The Express middleware caches JSON responses from your route handlers. ### Installation ```bash npm install layercache@latest ``` ### Basic Usage ```tsx import express from 'express' import { CacheStack, MemoryLayer, RedisLayer, createExpressCacheMiddleware } from 'layercache' import Redis from 'ioredis' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: new Redis(), ttl: 300_000 }) ]) const app = express() // Cache all GET requests to /api/users app.get('/api/users', createExpressCacheMiddleware(cache, { ttl: 30_000, tags: ['users'] }), async (req, res) => { const users = await fetchUsersFromDB() res.json(users) } ) app.listen(3000) ``` ### Custom Cache Keys ```tsx app.get('/api/users/:id', createExpressCacheMiddleware(cache, { keyResolver: (req) => `user:${req.params.id}`, ttl: 300_000 }), async (req, res) => { const user = await fetchUser(req.params.id) res.json(user) } ) ``` ### Options - `keyResolver` - Function to generate cache keys from requests - `methods` - HTTP methods to cache (default: `['GET']`) - `ttl` - Time-to-live in milliseconds - `tags` - Tags for invalidation - `allowPrivateCaching` - Allow implicit URL-based keys (default: false). This is shared URL-based caching, not per-user/private caching; never enable it for responses that vary by cookies, authorization headers, or authenticated user identity. Requests with common sensitive query parameters such as `access_token`, `api_key`, `apikey`, `auth`, `authorization`, `client_assertion`, `client_assertion_type`, `client_secret`, `code`, `credentials`, `id_token`, `jwt`, `password`, `private_key`, `refresh_token`, `secret`, `session`, `sessionid`, `session_id`, and `token` bypass implicit caching unless you provide a custom `keyResolver`. Requests carrying common authentication headers also bypass implicit caching unless you provide a custom `keyResolver`. Only 2xx JSON responses are written to the cache. 3xx, 4xx, and 5xx responses still return to the client but are not cached. ### Response Headers The middleware adds `x-cache: HIT` or `x-cache: MISS` headers to responses. ## Fastify The Fastify plugin decorates your app with a cache instance and provides an optional stats endpoint. ### Installation ```bash npm install layercache@latest ``` ### Basic Usage ```tsx import Fastify from 'fastify' import { CacheStack, MemoryLayer, RedisLayer, createFastifyLayercachePlugin } from 'layercache' import Redis from 'ioredis' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: new Redis(), ttl: 300_000 }) ]) const fastify = Fastify() await fastify.register(createFastifyLayercachePlugin(cache, { exposeStatsRoute: true, statsPath: '/cache/stats', allowPublicStatsRoute: false })) // Use the cache directly in routes fastify.get('/api/users', async (request, reply) => { const users = await cache.get('users', () => fetchUsersFromDB()) return users }) ``` ### Options - `exposeStatsRoute` - Enable stats endpoint (default: false) - `statsPath` - Path for stats endpoint (default: '/cache/stats') - `allowPublicStatsRoute` - Allow public access (default: false) - `authorizeStatsRoute` - Async authorization function - `unauthorizedStatusCode` - Status code for unauthorized (default: 403) ## Hono The Hono middleware caches JSON responses with minimal overhead. ### Installation ```bash npm install layercache@latest ``` ### Basic Usage ```tsx import { Hono } from 'hono' import { CacheStack, MemoryLayer, createHonoCacheMiddleware } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }) ]) const app = new Hono() app.use('/api/*', createHonoCacheMiddleware(cache, { ttl: 60_000, tags: ['api'] })) app.get('/api/users', async (c) => { const users = await fetchUsersFromDB() return c.json(users) }) ``` ### Custom Cache Keys ```tsx app.get('/api/users/:id', createHonoCacheMiddleware(cache, { keyResolver: (req) => `user:${req.path.split('/').pop()}`, ttl: 300_000 }), async (c) => { const id = c.req.param('id') const user = await fetchUser(id) return c.json(user) } ) ``` ### Options - `keyResolver` - Function to generate cache keys from Hono requests - `methods` - HTTP methods to cache (default: `['GET']`) - `ttl` - Time-to-live in milliseconds - `tags` - Tags for invalidation - `allowPrivateCaching` - Allow implicit URL-based keys (default: false). This is shared URL-based caching, not per-user/private caching; never enable it for responses that vary by cookies, authorization headers, or authenticated user identity. Requests with sensitive query parameters or common authentication headers bypass implicit caching unless you provide a custom `keyResolver`. Only 2xx JSON responses are written to the cache. The middleware also respects status set via `context.status(500)` before `context.json(body)`. ## NestJS Use `CacheStack` directly in your NestJS providers. Import from `layercache` — no separate package needed. ### Module Setup ```tsx import { Module } from '@nestjs/common' import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' import Redis from 'ioredis' @Module({ providers: [ { provide: 'CACHE_STACK', useFactory: () => new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: new Redis(), ttl: 300_000 }) ]) } ], exports: ['CACHE_STACK'] }) export class CacheModule {} ``` ### Async Configuration ```tsx import { Module } from '@nestjs/common' import { ConfigService } from '@nestjs/config' import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' import Redis from 'ioredis' @Module({ providers: [ { provide: 'CACHE_STACK', inject: [ConfigService], useFactory: (config: ConfigService) => new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: new Redis(config.get('REDIS_URL')), ttl: 300_000 }) ]) } ], exports: ['CACHE_STACK'] }) export class CacheModule {} ``` ### Using in Services ```tsx import { Injectable, Inject } from '@nestjs/common' import { CacheStack } from 'layercache' @Injectable() export class UsersService { constructor( @Inject('CACHE_STACK') private readonly cache: CacheStack ) {} async getUser(id: string): Promise { return this.cache.get(`user:${id}`, () => this.usersRepository.findOne(id) ) } } ``` > **Note:** The separate `@cachestack/nestjs` package was removed in v1.3.2. Use `CacheStack` directly from `layercache`. ## tRPC The tRPC middleware caches procedure results based on input arguments. ### Installation ```bash npm install layercache@latest ``` ### Basic Usage ```tsx import { initTRPC } from '@trpc/server' import { CacheStack, MemoryLayer, createTrpcCacheMiddleware } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }) ]) const t = initTRPC.create() const cacheMiddleware = createTrpcCacheMiddleware(cache, 'trpc', { keyResolver: (input) => JSON.stringify(input), ttl: 300_000 }) export const cachedProcedure = t.procedure.use(cacheMiddleware) export const appRouter = t.router({ user: cachedProcedure .input((val: unknown) => val as { id: string }) .query(async ({ input }) => { return fetchUser(input.id) }) }) ``` ### Context-Aware Caching ```tsx const cacheMiddleware = createTrpcCacheMiddleware(cache, 'user', { keyResolver: (input, path, type) => { return `${type}:${path}:${JSON.stringify(input)}` }, ttl: 300_000 }) ``` ## GraphQL Cache resolver results with the GraphQL wrapper. ### Installation ```bash npm install layercache@latest ``` ### Basic Usage ```tsx import { CacheStack, MemoryLayer, cacheGraphqlResolver } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }) ]) const resolvers = { Query: { user: cacheGraphqlResolver( cache, 'user', async (_root, { id }) => { return fetchUser(id) }, { keyResolver: (_root, { id }) => id, ttl: 300_000 } ) } } ``` ### With Tags ```tsx const resolvers = { Query: { user: cacheGraphqlResolver( cache, 'user', async (_root, { id }) => { return fetchUser(id) }, { keyResolver: (_root, { id }) => id, ttl: 300_000, tags: ({ id }) => ['user', `user:${id}`] } ) } } ``` ## OpenTelemetry Add distributed tracing to cache operations with OpenTelemetry integration. ### Installation ```bash npm install layercache@latest @opentelemetry/api ``` ### Basic Setup ```tsx import { trace } from '@opentelemetry/api' import { CacheStack, MemoryLayer, createOpenTelemetryPlugin } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }) ]) const tracer = trace.getTracer('layercache') const plugin = createOpenTelemetryPlugin(cache, tracer) // Cache operations are now traced await cache.get('user:123', fetchUser) // Clean up on shutdown plugin.uninstall() ``` ### Span Attributes Each cache operation creates a span with the following attributes: - `layercache.key_hash` - SHA-256 hash of the cache key, included by default for key-based operations - `layercache.success` - Whether the operation succeeded - `layercache.result` - The result type (hit, miss, etc.) - Error details if the operation failed Raw cache keys are not exported by default. Enable them only for trusted telemetry sinks: ```tsx const plugin = createOpenTelemetryPlugin(cache, tracer, { includeRawKeyAttributes: true }) ``` ### Custom Tracer ```tsx import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node' import { Resource } from '@opentelemetry/resources' const provider = new NodeTracerProvider({ resource: new Resource({ service: 'my-app' }) }) provider.register() const tracer = trace.getTracer('my-app', '1.0.0') const plugin = createOpenTelemetryPlugin(cache, tracer) ``` ## Stats HTTP Handler Expose cache statistics via HTTP for monitoring dashboards. ### Basic Usage ```tsx import { createCacheStatsHandler } from 'layercache' import http from 'node:http' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }) ]) const handler = createCacheStatsHandler(cache) const server = http.createServer(handler) server.listen(9090) ``` ### With Authorization ```tsx const handler = createCacheStatsHandler(cache, { authorize: async (req) => { const authHeader = req.headers['authorization'] return authHeader === `Bearer ${process.env.API_KEY}` }, unauthorizedStatusCode: 401 }) ``` ### Public Access ```tsx const handler = createCacheStatsHandler(cache, { allowPublicAccess: true }) ``` ### Response Format ```json { "metrics": { "hits": 1234, "misses": 56, "fetches": 56, "sets": 890, "deletes": 12, "backfills": 234, "invalidations": 45, "staleHits": 5, "refreshes": 3, "refreshErrors": 0, "writeFailures": 0, "singleFlightWaits": 7, "negativeCacheHits": 12, "circuitBreakerTrips": 0, "degradedOperations": 0 }, "layers": [ { "name": "memory", "isLocal": true, "degradedUntil": null }, { "name": "redis", "isLocal": false, "degradedUntil": null } ], "backgroundRefreshes": 3 } ``` --- url: /docs/invalidation.md --- # Cache Invalidation Cache invalidation is one of the hardest problems in distributed systems. layercache provides multiple strategies to invalidate cached data efficiently. ## Table of Contents - [Tag-Based Invalidation](#tag-based-invalidation) - [Batch Tag Invalidation](#batch-tag-invalidation) - [Pattern Invalidation](#pattern-invalidation) - [Prefix Invalidation](#prefix-invalidation) - [Generation-Based Versioning](#generation-based-versioning) - [Distributed Invalidation](#distributed-invalidation) *** ## Tag-Based Invalidation Tag keys when writing, then invalidate all keys with a given tag when data changes. ### Basic Usage ```ts import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ]) // Tag related data together await cache.set('user:123', user, { tags: ['user:123'] }) await cache.set('user:123:posts', posts, { tags: ['user:123', 'posts'] }) await cache.set('user:123:profile', profile, { tags: ['user:123'] }) // Invalidate all data for user 123 await cache.invalidateByTag('user:123') ``` ### Multi-Tag Keys A single key can have multiple tags: ```ts await cache.set('post:456', post, { tags: ['post:456', 'author:123', 'category:tech'] }) // Invalidate by any tag await cache.invalidateByTag('author:123') // Removes post:456 await cache.invalidateByTag('category:tech') // Removes post:456 ``` ### Hierarchical Tagging Use tag hierarchies for flexible invalidation: ```ts // Product data with hierarchical tags await cache.set('product:123', product, { tags: [ 'product:123', // Individual product 'category:electronics', // Category 'brand:sony', // Brand 'in-stock:true' // Stock status ] }) // Invalidate all Sony products await cache.invalidateByTag('brand:sony') // Invalidate all electronics await cache.invalidateByTag('category:electronics') ``` *** ## Batch Tag Invalidation Invalidate keys matching multiple tags with `any` or `all` semantics. ### Any Mode Delete keys tagged with **any** of the specified tags: ```ts // Invalidate keys tagged with 'users' OR 'posts' await cache.invalidateByTags(['users', 'posts'], 'any') ``` Use case: Clear multiple related caches at once. ```ts // Clear all user and post caches after database migration await cache.invalidateByTags(['users', 'posts', 'comments'], 'any') ``` ### All Mode Delete keys tagged with **all** of the specified tags: ```ts // Invalidate keys tagged with BOTH 'tenant:a' AND 'admin' await cache.invalidateByTags(['tenant:a', 'admin'], 'all') ``` Use case: Invalidate specific cross-cutting concerns. ```ts // Invalidate all admin data for tenant A await cache.set('user:a:1', data, { tags: ['tenant:a', 'admin', 'user'] }) await cache.set('user:a:2', data, { tags: ['tenant:a', 'admin', 'user'] }) await cache.set('user:b:1', data, { tags: ['tenant:b', 'admin', 'user'] }) await cache.invalidateByTags(['tenant:a', 'admin'], 'all') // Only user:a:1 and user:a:2 are deleted // user:b:1 remains (different tenant) ``` *** ## Pattern Invalidation Use glob-style patterns to match and delete keys. ### Supported Patterns - `*` - Match any sequence of characters - `?` - Match exactly one character ```ts // Delete all user keys await cache.invalidateByPattern('user:*') // Delete all keys matching a pattern await cache.invalidateByPattern('session:*') // Delete keys with specific format await cache.invalidateByPattern('report:2024-04-*') ``` ### Pattern Validation Patterns must be: - Non-empty - At most 1024 characters - Free of control characters (except `*` and `?`) ```ts // Valid patterns await cache.invalidateByPattern('user:*') await cache.invalidateByPattern('temp:?') await cache.invalidateByPattern('cache:v1:*.data') // Invalid patterns (throw) await cache.invalidateByPattern('') // Empty await cache.invalidateByPattern('a\nb') // Contains newline ``` ### Performance Considerations Pattern invalidation requires scanning all known keys. For better performance: 1. **Use prefix invalidation** when possible (faster) 2. **Use tag-based invalidation** for complex queries (more flexible) 3. **Avoid overly broad patterns** like `*` alone ```ts // SLOWER: Scans all keys await cache.invalidateByPattern('*') // FASTER: Uses trie-based prefix search await cache.invalidateByPrefix('') // BEST: Uses tag index await cache.invalidateByTag('all') ``` *** ## Prefix Invalidation Hierarchical prefix-based invalidation using efficient trie structures. Prefer this over pattern invalidation for hierarchical keys. ### Basic Usage ```ts // Set up hierarchical keys await cache.set('user:123:profile', profileData) await cache.set('user:123:posts', postsData) await cache.set('user:123:settings', settingsData) await cache.set('user:456:profile', otherProfile) // Invalidate all data for user 123 await cache.invalidateByPrefix('user:123:') // Deletes: user:123:profile, user:123:posts, user:123:settings // Keeps: user:456:profile ``` ### Namespace Hierarchy Combine with namespaces for clean organization: ```ts const tenantCache = cache.namespace('tenant:acme') await tenantCache.set('users:1', userData) await tenantCache.set('users:2', userData) await tenantCache.set('posts:1', postData) // Clear all tenant data await tenantCache.invalidateByPrefix('') ``` ### Trie-Based Performance Prefix invalidation uses a trie data structure for O(k) lookup where k is prefix length: ```ts // Fast: O(7) operations for 'user:123:' await cache.invalidateByPrefix('user:123:') // Slower: O(n) pattern matching for 'user:123:*' await cache.invalidateByPattern('user:123:*') ``` *** ## Generation-Based Versioning Add a generation prefix to every key and rotate it for instant bulk invalidation without scanning. ### Basic Usage ```ts // Create cache with generation const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ], { generation: 1 // Start at generation 1 }) // All keys are prefixed: "g1:user:123" await cache.set('user:123', userData) // Bump generation to invalidate everything cache.bumpGeneration() // Now generation 2 // Old keys are automatically ignored // New writes use "g2:user:123" await cache.set('user:123', newUserData) ``` ### Auto-Cleanup Old Generations Automatically delete keys from previous generations: ```ts const cache = new CacheStack([...], { generation: 1, generationCleanup: { batchSize: 500 // Delete 500 keys per batch } }) cache.bumpGeneration() // Cleans up g1 keys in batches ``` ### Use Cases #### Deployment Invalidation ```ts // Deploy with new generation const generation = process.env.DEPLOYMENT_ID ?? '1' const cache = new CacheStack([...], { generation: Number.parseInt(generation) }) // Every deployment starts with a fresh cache namespace ``` #### Feature Flag Rollout ```ts // Rotate cache when enabling feature let cacheGeneration = 1 async function enableNewFeature() { cacheGeneration += 1 cache.bumpGeneration() // All cached data is invalidated // Next requests use fresh data with new feature enabled } ``` #### Schema Migration ```ts // Bump generation after schema change async function migrateSchema() { await db.migrate() // Invalidate all cached data that uses old schema cache.bumpGeneration() } ``` *** ## Distributed Invalidation For multi-instance deployments, use distributed invalidation to keep memory caches in sync across servers. ### Redis Tag Index Share tag state across all servers using Redis: ```ts import { RedisTagIndex } from 'layercache' const tagIndex = new RedisTagIndex({ client: redis, prefix: 'myapp:tag-index', scanCount: 100, knownKeysShards: 16 // Default: distribute known keys across 16 shards }) const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ], { tagIndex // All servers share the same tag index }) // Any server can invalidate by tag await cache.invalidateByTag('user:123') // All servers use the same Redis-backed tag lookup ``` ### Redis Invalidation Bus Pub/sub-based L1 invalidation for real-time memory cache consistency: ```ts import { RedisInvalidationBus } from 'layercache' const bus = new RedisInvalidationBus({ publisher: redis, // Use existing Redis connection subscriber: new Redis(), // Separate connection for subscriptions signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET }) const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ], { invalidationBus: bus, broadcastL1Invalidation: true // Publish writes to peer memory layers }) // When Server A writes to cache: await cache.set('user:123', userData) // -> Publishes invalidation message to Redis // Server B's memory layer receives message: // -> Deletes 'user:123' from local memory // -> Next read fetches fresh data from Redis ``` ### Full Distributed Setup Combine all distributed features for multi-instance consistency: ```ts import { CacheStack, MemoryLayer, RedisLayer, RedisInvalidationBus, RedisTagIndex, RedisSingleFlightCoordinator } from 'layercache' const redis = new Redis() const bus = new RedisInvalidationBus({ publisher: redis, subscriber: new Redis(), signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET }) const tagIndex = new RedisTagIndex({ client: redis, prefix: 'myapp:tags', knownKeysShards: 16 }) const coordinator = new RedisSingleFlightCoordinator({ client: redis }) const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000, maxSize: 10_000 }), new RedisLayer({ client: redis, ttl: 3_600_000, prefix: 'myapp:cache:' }) ], { invalidationBus: bus, tagIndex: tagIndex, singleFlightCoordinator: coordinator, gracefulDegradation: { retryAfterMs: 10_000 } }) ``` ### Signed Invalidation Messages Set `signingSecret` on every instance that shares a Redis invalidation channel. Messages are signed with HMAC-SHA256 and rejected when the signature does not match, which prevents unrelated publishers on the same Redis from spoofing invalidation messages. ```ts const bus = new RedisInvalidationBus({ publisher: redis, subscriber: new Redis(process.env.REDIS_URL), channel: 'myapp:invalidation', signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET }) ``` All producers and consumers on the same channel must use the same secret. Omit `signingSecret` only for trusted, isolated Redis channels or for compatibility with older deployments. ### RedisTagIndex Shard Migration `RedisTagIndex` now defaults `knownKeysShards` to 16. Existing deployments that used the previous single-set layout (`<prefix>:keys`) are still read for compatibility and emit a warning, but you should migrate them into the sharded layout: ```bash npx layercache migrate-tag-index \ --redis rediss://prod-redis.example.com:6380 \ --tag-index-prefix "myapp:tags" \ --known-key-shards 16 \ --require-tls ``` ### Invalidation Flow ``` Server A Redis Server B | | | | cache.set('user:123', data) | | |------------------------------->| | | | SET user:123 | | | | | | PUBLISH invalidate user:123 | | |------------------------------->| | | | delete user:123 | | | (from memory) | | | ``` *** ## Best Practices ### 1. Use Tags for Related Data ```ts // GOOD: Tag related data await cache.set('user:123', user, { tags: ['user:123'] }) await cache.set('user:123:posts', posts, { tags: ['user:123'] }) await cache.set('user:123:profile', profile, { tags: ['user:123'] }) await cache.invalidateByTag('user:123') // Invalidates all 3 keys ``` ### 2. Use Prefixes for Hierarchical Data ```ts // GOOD: Hierarchical keys with prefix invalidation await cache.set('tenant:acme:users:1', data) await cache.set('tenant:acme:posts:1', data) await cache.invalidateByPrefix('tenant:acme:') // Fast trie lookup ``` ### 3. Use Generations for Bulk Invalidation ```ts // GOOD: Generation for instant bulk invalidation cache.bumpGeneration() // Invalidates everything immediately ``` ### 4. Avoid Overly Broad Patterns ```ts // BAD: Too broad await cache.invalidateByPattern('*') // GOOD: Specific pattern await cache.invalidateByPattern('temp:*') // BETTER: Prefix await cache.invalidateByPrefix('temp:') ``` ### 5. Clean Up Tags on Delete Tags are automatically removed when keys are deleted: ```ts await cache.set('user:123', data, { tags: ['user:123'] }) await cache.delete('user:123') // Tags automatically cleaned up await cache.invalidateByTag('user:123') // No keys found ``` --- url: /docs/layers.md --- # Cache Layers layercache provides several built-in cache layer implementations, each optimized for different use cases. You can also create custom layers by implementing the `CacheLayer` interface. ## Table of Contents - [MemoryLayer](#memorylayer) - [RedisLayer](#redislayer) - [DiskLayer](#disklayer) - [MemcachedLayer](#memcachedlayer) - [Custom Layers](#custom-layers) *** ## MemoryLayer In-process cache with configurable eviction policies. Perfect for L1 caching with sub-microsecond latency. ### Features - **Eviction policies**: LRU (default), LFU, or FIFO - **Max size enforcement**: Automatically evicts when limit reached - **TTL support**: Per-entry expiration with optional cleanup interval - **Snapshot support**: Export/import state for persistence - **Zero dependencies**: Pure in-memory storage ### Configuration ```ts import { MemoryLayer } from 'layercache' const memoryLayer = new MemoryLayer({ ttl: 60_000, // Default TTL in milliseconds maxSize: 5_000, // Maximum number of entries name: 'memory', // Layer name for metrics evictionPolicy: 'lru', // 'lru' | 'lfu' | 'fifo' cleanupIntervalMs: 60_000, // Expired entry cleanup interval onEvict: (key, value) => { // Optional eviction callback console.log(`Evicted: ${key}`) } }) ``` ### Eviction Policies #### LRU (Least Recently Used) - Default Evicts entries that haven't been accessed for the longest time. Best for general-purpose caching. ```ts new MemoryLayer({ evictionPolicy: 'lru', maxSize: 1_000 }) ``` #### LFU (Least Frequently Used) Evicts entries with the lowest access count. Best for workloads with hot spots. ```ts new MemoryLayer({ evictionPolicy: 'lfu', maxSize: 1_000 }) ``` #### FIFO (First In, First Out) Evicts the oldest entries regardless of access pattern. Best for time-series data. ```ts new MemoryLayer({ evictionPolicy: 'fifo', maxSize: 1_000 }) ``` ### State Persistence ```ts // Export current state const snapshot = memoryLayer.exportState() // [{ key: 'user:123', value: {...}, expiresAt: 1712847652000 }, ...] // Import state (e.g., after process restart) memoryLayer.importState(snapshot) ``` ### Cleanup Interval MemoryLayer periodically scans for expired entries to free memory. ```ts new MemoryLayer({ ttl: 300_000, cleanupIntervalMs: 30_000 // Cleanup every 30 seconds }) ``` Set to `0` or omit to disable periodic cleanup (entries still expire on access). *** ## RedisLayer Distributed caching layer backed by Redis with compression, serialization, and pipeline optimizations. ### Features - **Shared state**: Multiple processes/servers share the same cache - **Compression**: gzip or brotli compression with configurable threshold - **Serializer chain**: Try multiple deserializers for smooth migrations - **Pipeline batch ops**: Fast multi-key reads/writes - **Scan-based operations**: Efficient keys/clear without blocking - **Prefix support**: Isolate different applications in the same Redis instance ### Configuration ```ts import { RedisLayer } from 'layercache' import Redis from 'ioredis' const redisLayer = new RedisLayer({ client: new Redis(), // ioredis client instance ttl: 300_000, // Default TTL in milliseconds prefix: 'myapp:cache:', // Key prefix for namespacing compression: 'gzip', // 'gzip' | 'brotli' | undefined compressionThreshold: 1_024, // Min bytes to compress (default: 1024) decompressionMaxBytes: 64 * 1_024 * 1_024, // Max decompressed size serializer: new MsgpackSerializer(), // Custom serializer name: 'redis', // Layer name for metrics allowUnprefixedClear: false, // Require prefix for clear() scanCount: 100, // SCAN COUNT batch size disconnectOnDispose: false // Disconnect client on dispose }) ``` ### Compression Reduce Redis memory usage for large values: ```ts // Gzip compression (faster, moderate compression) new RedisLayer({ client: redis, compression: 'gzip', compressionThreshold: 1_024 // Only compress >1KB values }) // Brotli compression (slower, better compression) new RedisLayer({ client: redis, compression: 'brotli', compressionThreshold: 512 }) ``` ### Serialization Chain Support smooth data format migrations by trying multiple deserializers: ```ts import { JsonSerializer, MsgpackSerializer } from 'layercache' new RedisLayer({ client: redis, serializer: [ new MsgpackSerializer(), // Try MessagePack first new JsonSerializer() // Fall back to JSON ] }) // On write: always use first serializer (MessagePack) // On read: try each serializer until one succeeds // Successful reads auto-migrate to primary format ``` ### Key Prefixing Isolate different applications or environments: ```ts const productionCache = new RedisLayer({ client: redis, prefix: 'prod:cache:' }) const stagingCache = new RedisLayer({ client: redis, prefix: 'staging:cache:' }) ``` ### Clear Behavior `clear()` requires a prefix by default to prevent accidental data loss: ```ts const layer = new RedisLayer({ client: redis, prefix: 'myapp:cache:', allowUnprefixedClear: false // Default }) await layer.clear() // OK - deletes "myapp:cache:*" only const unsafeLayer = new RedisLayer({ client: redis, prefix: '', allowUnprefixedClear: true // Explicitly allow dangerous clear }) await unsafeLayer.clear() // DELETES EVERYTHING IN REDIS ``` ### Pipeline Operations RedisLayer uses ioredis pipelines for efficient bulk operations: ```ts // getMany() uses pipeline() const values = await redisLayer.getMany(['key1', 'key2', 'key3']) // setMany() uses pipeline() await redisLayer.setMany([ { key: 'key1', value: data1, ttl: 60_000 }, { key: 'key2', value: data2, ttl: 60_000 } ]) // deleteMany() uses pipeline() await redisLayer.deleteMany(['key1', 'key2', 'key3']) ``` *** ## DiskLayer Persistent file-based cache with atomic writes and LRU eviction. Perfect for caching across restarts without Redis. ### Features - **Persistent storage**: Survives process restarts - **Atomic writes**: Uses temp file + rename for crash safety - **SHA-256 hashed filenames**: Safe for any cache key - **Max files enforcement**: LRU eviction when limit exceeded - **Size limits**: Configurable max entry size - **Concurrent operations**: Safe for multi-process access ### Configuration ```ts import { DiskLayer } from 'layercache' import { resolve } from 'node:path' const diskLayer = new DiskLayer({ directory: resolve('./var/cache/layercache'), // Cache directory ttl: 3_600_000, // Default TTL in milliseconds name: 'disk', // Layer name for metrics maxFiles: 50_000, // Maximum cache files (default: 50,000) maxEntryBytes: 16 * 1_024 * 1_024, // Max 16MiB per entry maxWriteQueueDepth: 10_000, // Max pending serialized writes serializer: new JsonSerializer(), // Optional custom serializer encryptionKey: process.env.CACHE_ENCRYPTION_KEY, // Optional AES-256-GCM encryption signingKey: process.env.CACHE_SIGNING_KEY, // Optional HMAC-SHA256 signing allowLegacyPlaintext: false // Default: reject plaintext when protection is enabled }) ``` ### At-Rest Protection DiskLayer supports optional at-rest protection for cached data using AES-256-GCM encryption or HMAC-SHA256 signing: ```ts // Full encryption (recommended) new DiskLayer({ directory: './cache', encryptionKey: process.env.CACHE_ENCRYPTION_KEY // AES-256-GCM }) // Signing only (integrity verification without encryption) new DiskLayer({ directory: './cache', signingKey: process.env.CACHE_SIGNING_KEY // HMAC-SHA256 }) // Both — signingKey is ignored when encryptionKey is provided // (encryption already provides authenticated integrity) new DiskLayer({ directory: './cache', encryptionKey: process.env.CACHE_ENCRYPTION_KEY, signingKey: process.env.CACHE_SIGNING_KEY }) ``` When protection is configured, plaintext legacy entries are rejected by default. Use `allowLegacyPlaintext: true` only during a controlled migration window: ```ts new DiskLayer({ directory: './cache', signingKey: process.env.CACHE_SIGNING_KEY, allowLegacyPlaintext: true // migration only }) ``` ### File Storage Each cache entry is stored as a separate file: ```ts // Cache key -> SHA-256 hash 'user:123:profile' -> 'a1b2c3d4...lf' // File contents (JSON) { "key": "user:123:profile", "value": { "name": "Alice", ... }, "expiresAt": 1712847652000 } ``` ### Atomic Writes DiskLayer uses temp file + rename for crash-safe writes: ```ts // Write to temp file first await fs.writeFile('cache/a1b2...lf.tmp.12345.67890.tmp', data) // Atomic rename (fails if target exists) await fs.rename('cache/a1b2...lf.tmp.12345.67890.tmp', 'cache/a1b2...lf') ``` If the process crashes mid-write, the temp file is cleaned up on next access. ### Max Files Enforcement When `maxFiles` is exceeded, oldest files (by mtime) are evicted: ```ts new DiskLayer({ directory: './cache', maxFiles: 10_000 // Keep only 10k most recent entries }) ``` ### Size Limits Prevent disk space exhaustion with entry size limits: ```ts new DiskLayer({ directory: './cache', maxEntryBytes: 16 * 1_024 * 1_024 // Reject entries >16MiB }) // Disable size limit new DiskLayer({ directory: './cache', maxEntryBytes: false }) ``` Oversized entries are treated as corrupted and deleted. ### Write Queue Guard DiskLayer serializes writes to avoid corrupting files. `maxWriteQueueDepth` prevents a slow disk from accumulating unbounded pending writes: ```ts new DiskLayer({ directory: './cache', maxWriteQueueDepth: 1_000 }) // Disable only when the environment already bounds write pressure. new DiskLayer({ directory: './cache', maxWriteQueueDepth: false }) ``` *** ## MemcachedLayer Memcached-backed cache layer with key validation and pluggable serializers. ### Features - **Binary protocol**: Compatible with `memjs` and `memcache-client` - **Key validation**: Enforces 250-byte key limit - **Pluggable serializers**: JSON default, MessagePack supported - **Bulk operations**: getMany, deleteMany support ### Configuration ```ts import { MemcachedLayer } from 'layercache' import Memjs from 'memjs' const memcached = Memjs.Client.create('localhost:11211') const memcachedLayer = new MemcachedLayer({ client: memcached, // Memcached client (memjs or memcache-client) ttl: 300_000, // Default TTL in milliseconds name: 'memcached', // Layer name for metrics keyPrefix: 'myapp:', // Optional key prefix serializer: new JsonSerializer() // Optional custom serializer }) ``` ### Key Limit Validation Memcached enforces a 250-byte key limit. MemcachedLayer validates keys before sending: ```ts // This throws: key exceeds 250 bytes await cache.set('a'.repeat(251), value) // This throws: key contains invalid characters await cache.set('key with spaces', value) ``` Use a short `keyPrefix` to reserve space for dynamic keys: ```ts const layer = new MemcachedLayer({ client: memcached, keyPrefix: 'app1:' // Uses 5 bytes, leaves 245 for dynamic keys }) ``` ### Clear Not Supported Memcached doesn't support pattern-based deletion. Use key prefix rotation instead: ```ts // DON'T do this - throws await memcachedLayer.clear() // DO this instead - rotate prefix const cache = new CacheStack([ new MemcachedLayer({ client: memcached, keyPrefix: 'v1:' }) ]) // To invalidate all keys, deploy with new prefix const cache2 = new CacheStack([ new MemcachedLayer({ client: memcached, keyPrefix: 'v2:' }) ]) ``` *** ## Custom Layers Implement the `CacheLayer` interface to create custom cache backends. ### Interface ```ts interface CacheLayer { readonly name: string readonly defaultTtl?: number readonly isLocal?: boolean // Required methods get(key: string): Promise set(key: string, value: unknown, ttl?: number): Promise delete(key: string): Promise clear(): Promise // Optional optimizations getEntry?(key: string): Promise getMany?(keys: string[]): Promise> setMany?(entries: Array<{ key: string; value: unknown; ttl?: number }>): Promise deleteMany?(keys: string[]): Promise keys?(): Promise forEachKey?(visitor: (key: string) => void | Promise): Promise has?(key: string): Promise ttl?(key: string): Promise size?(): Promise ping?(): Promise dispose?(): Promise } ``` ### Example: Cloudflare KV Layer ```ts interface KVLayerOptions { namespace: KVNamespace ttl?: number name?: string } class KVLayer implements CacheLayer { readonly name: string readonly defaultTtl?: number readonly isLocal = false constructor(private options: KVLayerOptions) { this.name = options.name ?? 'kv' this.defaultTtl = options.ttl } async get(key: string): Promise { const value = await this.options.namespace.get(key, 'json') return value as T | null } async set(key: string, value: unknown, ttl?: number): Promise { await this.options.namespace.put(key, JSON.stringify(value), { expirationTtl: ttl ?? this.defaultTtl }) } async delete(key: string): Promise { await this.options.namespace.delete(key) } async clear(): Promise { // KV doesn't support clear - implement key listing if needed throw new Error('KVLayer.clear() is not supported') } // Optional optimizations async getMany(keys: string[]): Promise> { return Promise.all(keys.map(key => this.get(key))) } async deleteMany(keys: string[]): Promise { await Promise.all(keys.map(key => this.delete(key))) } } ``` ### Example: S3 Layer ```ts import { S3Client, GetObjectCommand, PutObjectCommand, DeleteObjectCommand } from '@aws-sdk/client-s3' class S3Layer implements CacheLayer { readonly name = 's3' readonly defaultTtl?: number readonly isLocal = false constructor( private s3: S3Client, private bucket: string, private prefix: string ) {} async get(key: string): Promise { try { const response = await this.s3.send(new GetObjectCommand({ Bucket: this.bucket, Key: `${this.prefix}${key}` })) const body = await response.Body.transformToString() return JSON.parse(body) as T } catch { return null } } async set(key: string, value: unknown, ttl?: number): Promise { await this.s3.send(new PutObjectCommand({ Bucket: this.bucket, Key: `${this.prefix}${key}`, Body: JSON.stringify(value), Expires: ttl ? new Date(Date.now() + ttl * 1000) : undefined })) } async delete(key: string): Promise { await this.s3.send(new DeleteObjectCommand({ Bucket: this.bucket, Key: `${this.prefix}${key}` })) } async clear(): Promise { // S3 doesn't support clear - use lifecycle rules or prefix rotation throw new Error('S3Layer.clear() is not supported') } } ``` ### Best Practices 1. **Implement optional methods**: `getMany`, `setMany`, `deleteMany` provide significant performance improvements 2. **Return correct TTL**: If your backend supports TTL, implement the `ttl()` method 3. **Handle errors gracefully**: Return `null` on network errors instead of throwing 4. **Use `getEntry` for metadata**: If your backend stores metadata (e.g., creation time), implement `getEntry` to support stale-while-revalidate 5. **Set `isLocal` correctly**: This affects distributed invalidation behavior 6. **Implement `dispose`**: Clean up resources (connections, timers) when the layer is no longer needed ```ts class MyLayer implements CacheLayer { readonly name = 'custom' readonly defaultTtl = 300 readonly isLocal = true // Set based on your backend // Implement all required methods... async dispose(): Promise { // Clean up connections, timers, etc. await this.connection.close() } } ``` --- url: /docs/migration.md --- # Migration Guide This guide helps you migrate from popular Node.js caching libraries to Layercache. Each section includes before/after code examples, API mappings, and key differences. ## Upgrading to 5.0 Layercache 5.0 is a major release that hardens cache isolation so authenticated responses are never served across users, and makes cross-instance invalidation fail closed. ### `createTrpcCacheMiddleware` and `cacheGraphqlResolver` require a `keyResolver` Both wrappers previously accepted `allowImplicitContextCaching: true` to derive cache keys from the procedure path + input (tRPC) or resolver arguments (GraphQL). Those keys did not include the authenticated request context, so caller-specific results could be cached under a shared key and served to another user. The option is removed and `keyResolver` is now required. If your code passed `allowImplicitContextCaching: true`, add a `keyResolver` that includes every input and context attribute affecting the result: ```tsx // Before (tRPC) const middleware = createTrpcCacheMiddleware(cache, 'proc', { allowImplicitContextCaching: true }) // After (tRPC) — include the authenticated context in the key const middleware = createTrpcCacheMiddleware(cache, 'proc', { keyResolver: (input, path, _type, context) => `${context?.id ?? 'anon'}:${path}:${JSON.stringify(input)}` }) ``` ```tsx // Before (GraphQL) const user = cacheGraphqlResolver(cache, 'user', resolver, { allowImplicitContextCaching: true }) // After (GraphQL) — include the caller identity in the key const user = cacheGraphqlResolver(cache, 'user', resolver, { keyResolver: (root, args) => `${root.userId}:${args.id}` }) ``` ### Express and Hono implicit caching skips authenticated requests When `allowPrivateCaching: true` is used without a `keyResolver`, the Express and Hono middlewares now bypass implicit URL-only caching for requests carrying authentication headers (`authorization`, `cookie`, `set-cookie`, `x-api-key`, `x-session-id`, `x-auth-token`, `x-forwarded-user`). This prevents one user's authenticated response from being served to another. Provide a `keyResolver` that includes the caller identity if you need to cache such responses. ### `RedisInvalidationBus.requireSignature` Add `requireSignature: true` so the bus throws at construction when `signingSecret` is missing. Zero-length secrets are treated as missing. Without signing, any client that can publish to the channel can forge invalidation messages: ```tsx const bus = new RedisInvalidationBus({ publisher: redis, subscriber: new Redis(process.env.REDIS_URL), signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET, requireSignature: process.env.NODE_ENV === 'production' }) ``` ## Upgrading to 4.0 Layercache 4.0 resolves [issue #90](https://github.com/flyingsquirrel0419/layercache/issues/90) by giving cache misses a JavaScript-native `undefined` result while preserving intentional `null` values. ### Miss and null semantics - `get()`, `getOrSet()`, `mget()`, `wrap()`, and namespace reads return `undefined` on a miss or negative-cache hit. - `getOrThrow()` throws only for `undefined`; a cached `null` is returned normally. - Read-through fetchers cache `null` as a regular value by default. Set `cacheNullValues: false` only when your fetcher uses `null` to mean absence. - `getEntry()` keeps its metadata contract: it returns `null` when no entry exists and exposes negative entries with `kind: 'empty'`. - Custom `CacheLayer` implementations keep returning `null` from their low-level read methods. Only the public stack and namespace APIs changed. Replace public miss checks such as `value === null` with `value === undefined`. If your application previously used `null` as a fetcher miss, opt out explicitly: ```tsx const user = await cache.get('user:missing', fetchUser, { cacheNullValues: false, negativeCache: true }) ``` Serializers retain their format-specific behavior: JSON omits `undefined` object properties and converts `undefined` array elements to `null`; MessagePack encodes `undefined` as nil and decodes it as `null`. Layercache does not store an `undefined` fetch result. ### Security and operational boundaries This release also deliberately changes cache and operator safety boundaries: - Automatically derived structured `wrap()` argument keys now use the `j2:` schema. Existing `j:` entries are left to expire and will be cold misses; no data migration is required. Plain objects with reserved native `$type` tags (`Date`, `URL`, `RegExp`, `Map`, or `Set`) now throw instead of colliding with native values. Use a `keyResolver` when those objects are intentional inputs. - `generationCleanup: true` now stops after discovering 10,000 unique old-generation keys in one cleanup run. Set `generationCleanup: { batchSize, maxMatches }` to choose a lower deployment-specific bound. `maxMatches: false` is an explicit opt-out and should only be used when the keyspace is bounded elsewhere. - Write-through, write-behind, single-key, and `mset()` writes share finite ordering state. The defaults admit 10,000 pending key-write units, 10,000 active keys, and 1,000 pending operations per key. Tune `writeCoordination` for known bursts and handle `CacheWriteSaturationError` as backpressure rather than retrying without a limit. Wildcard-only CLI invalidation patterns now all require `--force`, including combinations of `*` and `?` such as `**` and `?*`. ## Upgrading to 3.0 Layercache 3.0 is a major release because it changes operational defaults for Redis-backed deployments and HTTP middleware cache keys. ### RedisTagIndex known-key shards `RedisTagIndex` now defaults to 16 known-key shards. Existing deployments that used the previous single-set layout at `<prefix>:keys` are still read for compatibility, but you should migrate them before relying on the sharded layout in production: ```bash npx layercache migrate-tag-index \ --redis rediss://redis.example.com:6379 \ --tag-index-prefix myapp:tag-index \ --known-key-shards 16 ``` Use `knownKeysShards: 1` only when you intentionally need the legacy layout during a staged rollout. ### Production Redis URLs in the CLI CLI commands now reject plaintext `redis://` URLs when `NODE_ENV=production`, unless `--allow-plaintext` is passed: ```bash NODE_ENV=production npx layercache stats --redis rediss://redis.example.com:6379 NODE_ENV=production npx layercache stats --redis redis://localhost:6379 --allow-plaintext ``` ### Implicit HTTP cache keys Express and Hono middleware now bypass implicit URL-only caching when common sensitive query parameters are present. This avoids both storing secrets in cache keys and collapsing private responses into one scrubbed URL key. Provide a custom `keyResolver` when private responses are selected by query credentials. The sensitive parameter names are `access_token`, `api_key`, `apikey`, `auth`, `authorization`, `client_assertion`, `client_assertion_type`, `client_secret`, `code`, `credentials`, `id_token`, `jwt`, `password`, `private_key`, `refresh_token`, `secret`, `session`, `sessionid`, `session_id`, and `token`. The same list is documented in the [integration options](/docs/integrations.md#express). ### DiskLayer protected entries When `DiskLayer` is configured with `encryptionKey` or `signingKey`, plaintext legacy entries are now rejected by default. If you need to read old plaintext files during a migration, enable `allowLegacyPlaintext: true` temporarily and disable it after the cache directory has been rewritten. ### OpenTelemetry key attributes `createOpenTelemetryPlugin()` now exports `layercache.key_hash` by default. Pass `{ includeRawKeyAttributes: true }` only if your telemetry backend is allowed to receive raw cache keys. ### Generation persistence Persist generation rotations when several instances share the same cache: ```tsx import { CacheStack, RedisGenerationStore } from 'layercache' const generations = new RedisGenerationStore({ client: redis }) const generation = await generations.getOrInitialize(1) const cache = new CacheStack(layers, { generation }) const nextGeneration = await generations.bump() cache.bumpGeneration(nextGeneration) ``` ## From node-cache-manager ### Basic Setup **Before (node-cache-manager):** ```tsx import { caching, multiCaching } from 'cache-manager' import { redisStore } from 'cache-manager-redis-yet' const memoryCache = await caching('memory', { max: 100, ttl: 60 * 1000 }) const redisCache = await caching(redisStore, { url: 'redis://localhost:6379', ttl: 300 * 1000 }) const cache = multiCaching([memoryCache, redisCache]) ``` **After (layercache):** ```tsx import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' import Redis from 'ioredis' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000, maxSize: 100 }), new RedisLayer({ client: new Redis(), ttl: 300_000 }) ]) ``` ### Read-Through Fetch **Before:** ```tsx const user = await cache.wrap('user:123', () => db.findUser(123)) ``` **After:** ```tsx const user = await cache.get('user:123', () => db.findUser(123)) ``` ### Set with TTL **Before:** ```tsx await cache.set('user:123', user, 60000) // TTL in milliseconds ``` **After:** ```tsx await cache.set('user:123', user, { ttl: 60_000 }) // TTL in milliseconds ``` ### Delete **Before:** ```tsx await cache.del('user:123') ``` **After:** ```tsx await cache.delete('user:123') ``` ### Clear All **Before:** ```tsx await cache.reset() ``` **After:** ```tsx await cache.clear() ``` ### API Mapping | node-cache-manager | layercache | Notes | | -------------------------- | ----------------------------------------- | --------------------------------- | | `cache.wrap(key, fn)` | `cache.get(key, fn)` | Read-through fetch | | `cache.set(key, val, ttl)` | `cache.set(key, val, { ttl })` | TTL in milliseconds | | `cache.get(key)` | `cache.get(key)` | Same API | | `cache.del(key)` | `cache.delete(key)` | Renamed | | `cache.reset()` | `cache.clear()` | Renamed | | Per-store TTL | `ttl: { memory: 60_000, redis: 300_000 }` | Per-layer TTL map | | - | `cache.invalidateByTag(tag)` | New: tag invalidation | | - | `cache.wrap(prefix, fn)` | New: transparent function caching | ### Key Differences #### TTL is in Milliseconds **node-cache-manager uses milliseconds:** ```tsx await cache.set('key', value, 60000) // 60 seconds ``` **Layercache uses milliseconds:** ```tsx await cache.set('key', value, { ttl: 60_000 }) // 60 seconds ``` #### Auto Backfill **node-cache-manager requires manual warming:** ```tsx // After a cache miss in L1, L1 stays cold const value = await cache.wrap('key', fetcher) // Next request might still hit L2 instead of L1 ``` **Layercache auto-backfills L1:** ```tsx // After a cache miss in L1, L1 is automatically backfilled const value = await cache.get('key', fetcher) // Next request hits L1 immediately ``` #### Stampede Prevention **node-cache-manager requires plugins:** ```tsx // No built-in stampede prevention // Need external solutions ``` **Layercache has built-in stampede prevention:** ```tsx // Enabled by default // Multiple concurrent requests for same key share single fetcher ``` #### Tag Invalidation **node-cache-manager:** ```tsx // Manual key tracking required const userKeys = [`user:${id}`, `user:${id}:posts`, `user:${id}:profile`] await Promise.all(userKeys.map(key => cache.del(key))) ``` **Layercache:** ```tsx await cache.set('user:123', user, { tags: ['user:123'] }) await cache.set('user:123:posts', posts, { tags: ['user:123'] }) await cache.invalidateByTag('user:123') // Deletes both ``` ## From keyv ### Basic Setup **Before (keyv):** ```tsx import Keyv from 'keyv' import KeyvRedis from '@keyv/redis' const keyv = new Keyv({ store: new KeyvRedis('redis://localhost:6379') }) await keyv.set('user:123', user, 60000) const user = await keyv.get('user:123') ``` **After (layercache):** ```tsx import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: new Redis(), ttl: 300_000 }) ]) await cache.set('user:123', user, { ttl: 60_000 }) const user = await cache.get('user:123') ``` ### Read-Through Fetch **Before:** ```tsx // No built-in read-through let user = await keyv.get('user:123') if (!user) { user = await fetchUser(123) await keyv.set('user:123', user, 60000) } ``` **After:** ```tsx // Built-in read-through fetch const user = await cache.get('user:123', () => fetchUser(123)) ``` ### API Mapping | keyv | layercache | Notes | | ------------------------- | ------------------------------ | ----------------------- | | `keyv.set(key, val, ttl)` | `cache.set(key, val, { ttl })` | TTL in milliseconds | | `keyv.get(key)` | `cache.get(key)` | Same | | `keyv.delete(key)` | `cache.delete(key)` | Same | | `keyv.clear()` | `cache.clear()` | Same | | Namespace via constructor | `cache.namespace(prefix)` | Scoped views | | - | `cache.get(key, fetcher)` | New: read-through fetch | | - | `cache.wrap(prefix, fn)` | New: function caching | ### Key Differences #### Multi-Layer is Native **keyv requires plugins:** ```tsx // Multi-layer is not native // Requires custom adapters ``` **Layercache has native multi-layer:** ```tsx const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ]) // Reads cascade through layers with auto backfill ``` #### Read-Through Fetch **keyv requires manual checks:** ```tsx let value = await keyv.get('key') if (value === undefined) { value = await fetcher() await keyv.set('key', value) } ``` **Layercache has built-in read-through:** ```tsx const value = await cache.get('key', fetcher) ``` #### Namespaces **keyv:** ```tsx const userCache = new Keyv({ namespace: 'users' }) const postCache = new Keyv({ namespace: 'posts' }) ``` **Layercache:** ```tsx const userCache = cache.namespace('users') const postCache = cache.namespace('posts') // Full CacheStack API on namespaces await userCache.set('123', data) // Stored as "users:123" await userCache.clear() // Only deletes "users:*" ``` ## From cacheable ### Basic Setup **Before (cacheable):** ```tsx import { Cacheable } from 'cacheable' const cache = new Cacheable({ ttl: '1h' }) await cache.set('key', value) ``` **After (layercache):** ```tsx import { CacheStack, MemoryLayer } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 3_600_000 }) ]) await cache.set('key', value) ``` ### API Mapping | cacheable | layercache | Notes | | ------------------------------ | -------------------------------------- | ------------------- | | `new Cacheable({ ttl: '1h' })` | `new CacheStack([{ ttl: 3_600_000 }])` | TTL in milliseconds | | `cache.set(key, val)` | `cache.set(key, val)` | Same | | `cache.get(key)` | `cache.get(key)` | Same | | `cache.delete(key)` | `cache.delete(key)` | Same | | `cache.clear()` | `cache.clear()` | Same | ### Key Differences #### TTL Format **cacheable uses string durations:** ```tsx const cache = new Cacheable({ ttl: '1h' }) ``` **Layercache uses numeric milliseconds:** ```tsx const cache = new CacheStack([new MemoryLayer({ ttl: 3_600_000 })]) ``` #### Multi-Layer Orchestration **cacheable:** ```tsx // Limited multi-layer support // Manual coordination required ``` **Layercache:** ```tsx const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }), new DiskLayer({ ttl: 3_600_000 }) ]) // Automatic cascading reads and writes ``` #### Distributed Consistency **cacheable:** ```tsx // No built-in distributed features // Manual implementation required ``` **Layercache:** ```tsx import { RedisInvalidationBus, RedisTagIndex } from 'layercache' const bus = new RedisInvalidationBus({ publisher: redis }) const tagIndex = new RedisTagIndex({ client: redis }) const cache = new CacheStack([/* ... */], { invalidationBus: bus, broadcastL1Invalidation: true, tagIndex }) ``` #### Stampede Prevention **cacheable:** ```tsx // No built-in stampede prevention ``` **Layercache:** ```tsx const cache = new CacheStack([/* ... */], { stampedePrevention: true // Enabled by default }) ``` ## Operational Migration Tips ### Replace Ad-Hoc Redis Key Scans **Before:** ```bash # Manual Redis scans redis-cli --scan --pattern "user:*" | xargs redis-cli del ``` **After:** ```bash # Use Layercache CLI npx layercache keys --redis redis://localhost:6379 --pattern "user:*" npx layercache invalidate --redis redis://localhost:6379 --pattern "user:*" ``` ### Replace Manual Prefill Scripts **Before:** ```tsx // Custom warm-up script for (const key of criticalKeys) { const val = await fetch(key) await redis.set(key, JSON.stringify(val), 'EX', 300) } ``` **After:** ```tsx // Built-in warm() method await cache.warm( criticalKeys.map(key => ({ key, fetcher: () => fetchByKey(key), priority: 10 })), { concurrency: 4, continueOnError: true } ) ``` ### Replace Custom Stats Endpoints **Before:** ```tsx // Custom stats endpoint app.get('/stats', (req, res) => { res.json({ hits: myCounter.hits, misses: myCounter.misses }) }) ``` **After:** ```tsx import { createCacheStatsHandler } from 'layercache' app.get('/cache/stats', createCacheStatsHandler(cache)) ``` ### Replace Manual Tag Tracking **Before:** ```tsx // Manual tag-to-key mapping const tagMap = new Map() async function setWithTags(key, value, tags) { await redis.set(key, JSON.stringify(value)) for (const tag of tags) { const keys = tagMap.get(tag) || [] keys.push(key) tagMap.set(tag, keys) } } async function invalidateByTag(tag) { const keys = tagMap.get(tag) || [] await Promise.all(keys.map(key => redis.del(key))) tagMap.delete(tag) } ``` **After:** ```tsx // Built-in tag support await cache.set('user:123', user, { tags: ['user:123'] }) await cache.invalidateByTag('user:123') ``` ### Replace Custom Invalidation Logic **Before:** ```tsx // Manual prefix-based invalidation async function invalidatePrefix(prefix) { const keys = await redis.keys(`${prefix}*`) await Promise.all(keys.map(key => redis.del(key))) } ``` **After:** ```tsx // Built-in prefix invalidation await cache.invalidateByPrefix('user:123:') ``` ### Replace Custom Locking for Stampede Prevention **Before:** ```tsx // Manual distributed locking async function getWithLock(key, fetcher) { const lockKey = `lock:${key}` const lock = await redis.set(lockKey, '1', 'PX', 5000, 'NX') if (lock === 'OK') { try { const value = await fetcher() await redis.set(key, JSON.stringify(value)) return value } finally { await redis.del(lockKey) } } else { // Wait and retry await sleep(100) return getWithLock(key, fetcher) } } ``` **After:** ```tsx // Built-in distributed single-flight import { RedisSingleFlightCoordinator } from 'layercache' const coordinator = new RedisSingleFlightCoordinator({ client: redis }) const cache = new CacheStack([/* ... */], { singleFlightCoordinator: coordinator, singleFlightLeaseMs: 30000 }) const value = await cache.get(key, fetcher) ``` ## Migration Checklist ### Phase 1: Setup - [ ] Install Layercache: `npm install layercache@latest` - [ ] Create CacheStack with equivalent layers - [ ] Keep TTL values in milliseconds - [ ] Configure distributed features (if needed) ### Phase 2: Code Changes - [ ] Replace `cache.wrap()` with `cache.get(key, fetcher)` - [ ] Replace `cache.del()` with `cache.delete()` - [ ] Replace `cache.reset()` with `cache.clear()` - [ ] Update set operations to use options object: `{ ttl: 60_000 }` - [ ] Add tags for group invalidation ### Phase 3: Testing - [ ] Verify cache hits/misses work correctly - [ ] Test tag-based invalidation - [ ] Test multi-layer cascading - [ ] Test distributed coordination (if applicable) - [ ] Monitor metrics and hit rates ### Phase 4: Optimization - [ ] Enable stampede prevention (default) - [ ] Configure stale-while-revalidate - [ ] Set up Prometheus metrics - [ ] Configure health checks - [ ] Set up distributed single-flight (if needed) ## Common Migration Issues ### TTL Confusion **Issue:** Keeping old second-based TTL values after migrating. **Solution:** ```tsx // Wrong await cache.set('key', value, { ttl: 60 }) // 60 milliseconds // Correct await cache.set('key', value, { ttl: 60_000 }) // 60 seconds ``` ### Missing Tags **Issue:** Unable to invalidate related keys. **Solution:** ```tsx // Add tags when setting await cache.set('user:123', user, { tags: ['user:123'] }) await cache.set('user:123:posts', posts, { tags: ['user:123'] }) // Invalidate by tag await cache.invalidateByTag('user:123') ``` ### Namespace Confusion **Issue:** Not using namespaces for scoped caches. **Solution:** ```tsx // Use namespaces instead of separate caches const userCache = cache.namespace('users') const postCache = cache.namespace('posts') await userCache.set('123', data) // Stored as "users:123" ``` ### Not Using Read-Through **Issue:** Still using manual miss handling. **Solution:** ```tsx // Old way let value = await cache.get('key') if (!value) { value = await fetcher() await cache.set('key', value) } // New way const value = await cache.get('key', fetcher) ``` ## Need Help? If you run into issues during migration: 1. Check the [API Reference](/docs/api.md) for detailed method documentation 2. Review [examples](https://github.com/flyingsquirrel0419/layercache/tree/main/examples) for common patterns 3. [Open an issue](https://github.com/flyingsquirrel0419/layercache/issues) with your current setup We're happy to help you find the right approach for your use case! --- url: /docs/observability.md --- # Observability and Monitoring Layercache provides comprehensive observability features out of the box. Track cache performance, monitor layer health, and integrate with Prometheus and OpenTelemetry. ## Metrics Overview Layercache automatically tracks detailed metrics for all cache operations: ### Available Metrics - **hits** - Cache hits across all layers - **misses** - Cache misses (no layer had the key) - **fetches** - Fetcher function invocations (full misses) - **sets** - Write operations - **deletes** - Delete operations - **backfills** - L1/L2... automatic backfills from deeper layers - **invalidations** - Explicit invalidations (by tag, pattern, prefix) - **staleHits** - Stale-while-revalidate hits served - **refreshes** - Background refresh attempts - **refreshErrors** - Background refresh failures - **writeFailures** - Write operation failures - **singleFlightWaits** - Requests waiting for distributed lock - **negativeCacheHits** - Negative cache (null) results served - **circuitBreakerTrips** - Circuit breaker activations - **degradedOperations** - Operations run in degraded mode ## Getting Metrics ### getMetrics() Retrieve a snapshot of all metrics counters: ```tsx import { CacheStack, MemoryLayer } from 'layercache' const cache = new CacheStack([new MemoryLayer({ ttl: 60_000 })]) // ... use cache ... const metrics = cache.getMetrics() console.log(metrics) // { // hits: 1234, // misses: 56, // fetches: 56, // sets: 890, // deletes: 12, // backfills: 234, // invalidations: 45, // staleHits: 5, // refreshes: 3, // refreshErrors: 0, // writeFailures: 0, // singleFlightWaits: 7, // negativeCacheHits: 12, // circuitBreakerTrips: 0, // degradedOperations: 0, // hitsByLayer: { memory: 1000, redis: 234 }, // missesByLayer: { memory: 45, redis: 11 }, // latencyByLayer: { // memory: { avgMs: 0.05, maxMs: 1.2, count: 1200 }, // redis: { avgMs: 2.3, maxMs: 15.6, count: 250 } // }, // resetAt: 1712847654321 // } ``` ### getHitRate() Calculate cache hit rate overall and per-layer: ```tsx const hitRate = cache.getHitRate() console.log(hitRate) // { // overall: 0.956, // 95.6% hit rate // byLayer: { // memory: 0.957, // L1 hit rate // redis: 0.955 // L2 hit rate // } // } ``` ### getStats() Get comprehensive stats including layer health: ```tsx const stats = cache.getStats() console.log(stats) // { // metrics: { ... }, // Same as getMetrics() // layers: [ // { // name: 'memory', // isLocal: true, // degradedUntil: null // }, // { // name: 'redis', // isLocal: false, // degradedUntil: 1712848000000 // Timestamp if degraded // } // ], // backgroundRefreshes: 3 // } ``` ### resetMetrics() Reset all metric counters to zero: ```tsx cache.resetMetrics() ``` ### captureMetrics() Capture the metrics emitted by one async operation without diffing a global snapshot. This is useful for namespace-level accounting and other overlapping work where multiple operations may run at the same time. ```tsx const { result, metrics } = await cache.captureMetrics(async () => { return cache.get('user:123', fetchUser) }) console.log(result) console.log(metrics.fetches) ``` ## Health Checks Monitor the health of each cache layer with ping checks: ### healthCheck() Check connectivity and latency for all layers: ```tsx const health = await cache.healthCheck() console.log(health) // [ // { // layer: 'memory', // healthy: true, // latencyMs: 0.03 // }, // { // layer: 'redis', // healthy: true, // latencyMs: 2.45 // } // ] ``` ### Usage in Health Endpoints ```tsx import express from 'express' import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ]) const app = express() app.get('/health', async (req, res) => { const health = await cache.healthCheck() const allHealthy = health.every(h => h.healthy) res.status(allHealthy ? 200 : 503).json({ status: allHealthy ? 'healthy' : 'degraded', checks: health }) }) ``` ## Prometheus Integration Export metrics in Prometheus text format for scraping. ### createPrometheusMetricsExporter() Create a Prometheus metrics exporter: ```tsx import { createPrometheusMetricsExporter } from 'layercache' import http from 'node:http' const cache = new CacheStack([/* ... */]) const collectMetrics = createPrometheusMetricsExporter(cache) const server = http.createServer(async (_req, res) => { res.setHeader('content-type', 'text/plain; version=0.0.4; charset=utf-8') res.end(collectMetrics()) }) server.listen(9091, () => { console.log('Prometheus metrics exposed on :9091/metrics') }) ``` ### Multiple Cache Stacks ```tsx const userCache = new CacheStack([/* ... */]) const productCache = new CacheStack([/* ... */]) const collectMetrics = createPrometheusMetricsExporter([ { stack: userCache, name: 'users' }, { stack: productCache, name: 'products' } ]) ``` ### Example Output ``` # HELP layercache_hits_total Total number of cache hits # TYPE layercache_hits_total counter layercache_hits_total{cache="default"} 1234 # HELP layercache_misses_total Total number of cache misses # TYPE layercache_misses_total counter layercache_misses_total{cache="default"} 56 # HELP layercache_hit_rate Overall cache hit rate (0-1) # TYPE layercache_hit_rate gauge layercache_hit_rate{cache="default"} 0.956000 # HELP layercache_hits_by_layer_total Hits broken down by layer # TYPE layercache_hits_by_layer_total counter layercache_hits_by_layer_total{cache="default",layer="memory"} 1000 layercache_hits_by_layer_total{cache="default",layer="redis"} 234 # HELP layercache_layer_latency_avg_ms Average read latency per layer in milliseconds # TYPE layercache_layer_latency_avg_ms gauge layercache_layer_latency_avg_ms{cache="default",layer="memory"} 0.0500 layercache_layer_latency_avg_ms{cache="default",layer="redis"} 2.3000 ``` ## OpenTelemetry Integration Add distributed tracing to cache operations. ### Setup ```tsx import { trace } from '@opentelemetry/api' import { createOpenTelemetryPlugin } from 'layercache' const tracer = trace.getTracer('my-app', '1.0.0') const plugin = createOpenTelemetryPlugin(cache, tracer) ``` Key-based spans include `layercache.key_hash` by default. Raw cache keys are not exported unless you explicitly pass `{ includeRawKeyAttributes: true }`. ### Span Events The plugin emits two types of span events: #### operation-start ```typescript { id: number, name: string, // Operation name (e.g., "get", "set") attributes: object // Operation-specific attributes } ``` #### operation-end ```typescript { id: number, success: boolean, result?: string, // Result type error?: Error } ``` ### Example Traced Operations ```tsx // Each of these creates a span await cache.get('user:123', fetchUser) await cache.set('user:123', user, { ttl: 300_000 }) await cache.invalidateByTag('user:123') ``` ### Cleanup ```tsx // Uninstall when shutting down plugin.uninstall() ``` ## Event Hooks CacheStack extends EventEmitter and emits events for all operations. ### Available Events | Event | Payload | Description | | ----------------- | ----------------------------- | ----------------------------- | | `hit` | `{ key, layer }` | Cache hit in specific layer | | `miss` | `{ key }` | Full cache miss | | `set` | `{ key }` | Value written | | `delete` | `{ key }` | Key deleted | | `backfill` | `{ key, fromLayer, toLayer }` | Upper layer filled from lower | | `stale-serve` | `{ key, state, layer }` | Stale value served | | `stampede-dedupe` | `{ key }` | Request deduplicated | | `warm` | `{ key }` | Cache warmed | | `error` | `{ event, context }` | Operation error | ### Subscribing to Events ```tsx cache.on('hit', ({ key, layer }) => { console.log(`Cache hit for ${key} in ${layer}`) metrics.inc('cache.hit', { layer }) }) cache.on('miss', ({ key }) => { console.log(`Cache miss for ${key}`) metrics.inc('cache.miss') }) cache.on('error', ({ event, context }) => { console.error(`Cache error on ${event}:`, context) }) ``` ### Backfill Tracking ```tsx cache.on('backfill', ({ key, fromLayer, toLayer }) => { console.log(`Backfilled ${key} from ${fromLayer} to ${toLayer}`) }) ``` ### Error Handling ```tsx cache.on('error', ({ event, context }) => { if (event === 'set') { logger.error('Failed to set cache value', context) } }) ``` ## HTTP Stats Handler Expose cache statistics via a simple HTTP handler. ### Basic Usage ```tsx import { createCacheStatsHandler } from 'layercache' import http from 'node:http' const handler = createCacheStatsHandler(cache) const server = http.createServer(handler) server.listen(9090) ``` ### With Express ```tsx import express from 'express' import { createCacheStatsHandler } from 'layercache' const app = express() app.get('/cache/stats', createCacheStatsHandler(cache)) ``` ### With Authorization ```tsx const handler = createCacheStatsHandler(cache, { authorize: async (req) => { const apiKey = req.headers['x-api-key'] return apiKey === process.env.STATS_API_KEY }, unauthorizedStatusCode: 401 }) ``` ### Response Format ```json { "metrics": { "hits": 1234, "misses": 56, "fetches": 56, "sets": 890, "deletes": 12, "backfills": 234, "invalidations": 45, "staleHits": 5, "refreshes": 3, "refreshErrors": 0, "writeFailures": 0, "singleFlightWaits": 7, "negativeCacheHits": 12, "circuitBreakerTrips": 0, "degradedOperations": 0, "hitsByLayer": { "memory": 1000, "redis": 234 }, "missesByLayer": { "memory": 45, "redis": 11 }, "latencyByLayer": { "memory": { "avgMs": 0.05, "maxMs": 1.2, "count": 1200 }, "redis": { "avgMs": 2.3, "maxMs": 15.6, "count": 250 } }, "resetAt": 1712847654321 }, "layers": [ { "name": "memory", "isLocal": true, "degradedUntil": null }, { "name": "redis", "isLocal": false, "degradedUntil": null } ], "backgroundRefreshes": 3 } ``` ## Admin CLI Inspect and manage caches from the command line. ### Installation ```bash npm install -g layercache ``` Or use via npx: ```bash npx layercache stats --redis redis://localhost:6379 ``` ### Commands #### stats Show cache statistics: ```bash layercache stats --redis redis://localhost:6379 --pattern "user:*" ``` Response: ```json { "totalKeys": 1234, "pattern": "user:*" } ``` #### keys List cached keys: ```bash layercache keys --redis redis://localhost:6379 --pattern "user:*" ``` Output: ``` user:1 user:2 user:3 ... ``` #### inspect Inspect a specific key: ```bash layercache inspect --redis redis://localhost:6379 --key "user:123" ``` Response: ```json { "key": "user:123", "exists": true, "ttlMs": 245, "sizeBytes": 1024, "isEnvelope": true, "state": "fresh", "preview": { "kind": "fresh", "freshUntil": 1712848000000, "staleUntil": 1712848300000, "errorUntil": 1712848900000 } } ``` #### invalidate Invalidate cached data: ```bash # By pattern layercache invalidate --redis redis://localhost:6379 --pattern "user:*" # By tag layercache invalidate --redis redis://localhost:6379 --tag "user:123" ``` ### Options - `--redis ` - Redis connection URL (required) - `--pattern ` - Glob pattern for filtering keys - `--key ` - Exact key for inspect command - `--tag ` - Tag for invalidate command - `--tag-index-prefix ` - Redis key prefix for tag index ## Monitoring Best Practices ### 1. Track Hit Rate Monitor hit rate to ensure cache effectiveness: ```tsx setInterval(() => { const { overall, byLayer } = cache.getHitRate() console.log(`Overall hit rate: ${(overall * 100).toFixed(2)}%`) console.log(`Memory hit rate: ${(byLayer.memory * 100).toFixed(2)}%`) console.log(`Redis hit rate: ${(byLayer.redis * 100).toFixed(2)}%`) }, 60000) ``` ### 2. Alert on Degraded Layers ```tsx cache.on('error', ({ event, context }) => { if (context.layer === 'redis') { alerting.send(`Redis layer degraded: ${context.error}`) } }) ``` ### 3. Monitor Latency ```tsx const metrics = cache.getMetrics() for (const [layer, latency] of Object.entries(metrics.latencyByLayer)) { if (latency.maxMs > 100) { console.warn(`High latency in ${layer}: ${latency.maxMs}ms`) } } ``` ### 4. Track Circuit Breaker ```tsx cache.on('error', ({ event, context }) => { if (context.operation === 'circuitBreakerTrip') { alerting.send(`Circuit breaker tripped for ${context.key}`) } }) ``` ### 5. Export to Prometheus ```tsx // Expose metrics for Prometheus scraping const collectMetrics = createPrometheusMetricsExporter(cache) http.createServer(async (_req, res) => { res.setHeader('content-type', 'text/plain; version=0.0.4') res.end(collectMetrics()) }).listen(9091) ``` ### 6. Distributed Tracing ```tsx // Add OpenTelemetry tracing const plugin = createOpenTelemetryPlugin(cache, tracer) // Clean up on shutdown process.on('SIGTERM', async () => { plugin.uninstall() await cache.disconnect() }) ``` --- url: /docs/resilience.md --- # Resilience Features Build fault-tolerant caches that handle failures gracefully without cascading errors to your application. ## Table of Contents - [Graceful Degradation](#graceful-degradation) - [Circuit Breaker](#circuit-breaker) - [Write Policies](#write-policies) - [Write Strategies](#write-strategies) - [Fetcher Rate Limiting](#fetcher-rate-limiting) *** ## Graceful Degradation When a cache layer fails (e.g., Redis connection timeout), skip it temporarily instead of failing every request. ### Configuration ```ts import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ], { gracefulDegradation: { retryAfterMs: 10_000 // Retry failed layer after 10 seconds } }) ``` ### How It Works 1. **First failure**: Layer operation fails (e.g., Redis timeout) 2. **Mark degraded**: Layer is marked as degraded for `retryAfterMs` 3. **Skip layer**: Subsequent operations skip the degraded layer 4. **Retry after cooldown**: After `retryAfterMs`, operations retry the layer 5. **Recover on success**: First successful operation clears degraded state ### Example ```ts // Redis is healthy const user1 = await cache.get('user:1') // -> Reads from memory (miss) // -> Reads from Redis (hit) // -> Backfills memory // Redis connection times out const user2 = await cache.get('user:2') // -> Reads from memory (miss) // -> Redis fails (timeout) // -> Falls back to fetcher // -> Marks Redis as degraded for 10 seconds // Within 10 second cooldown const user3 = await cache.get('user:3') // -> Reads from memory (miss) // -> Skips Redis (degraded) // -> Falls back to fetcher // -> Writes to memory only // After 10 seconds, Redis recovers const user4 = await cache.get('user:4') // -> Retries Redis // -> If successful: clears degraded state // -> If failed: restarts cooldown ``` ### Per-Layer Degradation Each layer degrades independently: ```ts const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis1, ttl: 300_000 }), // L2 new DiskLayer({ directory: './cache', ttl: 3_600_000 }) // L3 ], { gracefulDegradation: { retryAfterMs: 10_000 } }) // Redis fails but disk is healthy const data = await cache.get('key') // -> Memory: miss // -> Redis: degraded (skip) // -> Disk: hit // -> Backfills memory ``` ### Monitoring Degradation ```ts const stats = cache.getStats() for (const layer of stats.layers) { console.log(`${layer.name}:`, { healthy: !layer.degradedUntil, degradedUntil: layer.degradedUntil ? new Date(layer.degradedUntil).toISOString() : 'N/A' }) } // Output: // memory: { healthy: true, degradedUntil: 'N/A' } // redis: { healthy: false, degradedUntil: '2024-04-11T12:34:56.789Z' } ``` ### Health Checks Use `healthCheck()` to proactively detect layer issues: ```ts const health = await cache.healthCheck() for (const result of health) { if (!result.healthy) { console.warn(`Layer ${result.layer} is unhealthy: ${result.error}`) } } // Output: // [ // { layer: 'memory', healthy: true, latencyMs: 0.03 }, // { layer: 'redis', healthy: false, latencyMs: 5000, error: 'Connection timeout' } // ] ``` *** ## Circuit Breaker Stop hammering broken upstream services after repeated failures. Prevents cascading failures and reduces load on struggling systems. ### Configuration ```ts const cache = new CacheStack([...], { circuitBreaker: { failureThreshold: 5, // Trip after 5 consecutive failures cooldownMs: 30_000 // Retry after 30 seconds } }) ``` ### How It Works 1. **Closed state**: Requests pass through normally 2. **Failures increment**: Each failure increments a counter 3. **Trip**: When failures reach `failureThreshold`, circuit trips 4. **Open state**: Requests fail immediately without calling upstream 5. **Cooldown**: After `cooldownMs`, enter half-open state 6. **Half-open**: Allow one request to test recovery 7. **Recover**: On success, close circuit (reset counter) 8. **Fail again**: On failure, reopen circuit ### Example ```ts let dbFailures = 0 const fetchUser = async (id: number) => { dbFailures++ if (dbFailures <= 7) { throw new Error('Database connection failed') } return { id, name: `User ${id}` } } const cache = new CacheStack([...], { circuitBreaker: { failureThreshold: 5, cooldownMs: 30_000 } }) // First 5 calls: fail, increment counter for (let i = 0; i < 5; i++) { await cache.get(`user:${i}`, fetchUser) // Circuit state: CLOSED -> still trying } // 6th call: trips circuit await cache.get('user:6', fetchUser) // Circuit state: OPEN -> fails immediately without calling fetcher // 7th call: circuit still open (within cooldown) await cache.get('user:7', fetchUser) // Circuit state: OPEN -> fails immediately // After 30 seconds: circuit enters half-open await cache.get('user:8', fetchUser) // Circuit state: HALF_OPEN -> tries one request // -> Fails (dbFailures = 7) // Circuit state: OPEN -> trips again // After another 30 seconds: circuit enters half-open await cache.get('user:9', fetchUser) // Circuit state: HALF_OPEN -> tries one request // -> Succeeds (dbFailures = 8, database recovered) // Circuit state: CLOSED -> circuit resets ``` ### Per-Operation Circuit Breaker Configure circuit breaker for specific operations: ```ts // Global circuit breaker const cache = new CacheStack([...], { circuitBreaker: { failureThreshold: 10, cooldownMs: 60_000 } }) // Override for fragile operation await cache.get('fragile-key', fetchFragileData, { circuitBreaker: { failureThreshold: 3, // Trip after 3 failures cooldownMs: 10_000 // Retry after 10 seconds } }) ``` Use shared scope when several cache keys depend on the same backend and should trip one circuit together: ```ts await cache.get('user:1', fetchUser, { circuitBreaker: { failureThreshold: 2, cooldownMs: 60_000, scope: 'shared', breakerKey: 'users-api' } }) ``` ### Circuit Breaker Events Monitor circuit breaker state changes: ```ts cache.on('error', ({ event, context }) => { if (event === 'circuit-breaker-trip') { console.error(`Circuit tripped for key: ${context.key}`) } if (event === 'circuit-breaker-reset') { console.log(`Circuit reset for key: ${context.key}`) } }) ``` ### Metrics ```ts const metrics = cache.getMetrics() console.log(`Circuit breaker trips: ${metrics.circuitBreakerTrips}`) ``` *** ## Write Policies Control how write failures are handled when some cache layers fail. ### Strict Mode (Default) Fail if **any** layer fails to write: ```ts const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000 }), new RedisLayer({ client: redis, ttl: 300_000 }) ], { writePolicy: 'strict' // Default }) // Redis fails -> entire write fails try { await cache.set('user:123', userData) } catch (err) { console.error('Write failed: Redis unavailable') } ``` ### Best-Effort Mode Succeed if **at least one** layer writes successfully: ```ts const cache = new CacheStack([...], { writePolicy: 'best-effort' }) // Redis fails -> memory write succeeds await cache.set('user:123', userData) // No error thrown // Data is cached in memory only ``` ### Use Cases #### Strict Mode Use when cache consistency is critical: ```ts // Financial data: must be consistent across all layers const priceCache = new CacheStack([...], { writePolicy: 'strict' }) ``` #### Best-Effort Mode Use when cache availability is more important than consistency: ```ts // Session data: better to cache in memory than not at all const sessionCache = new CacheStack([...], { writePolicy: 'best-effort' }) ``` ### Per-Operation Override ```ts // Global: best-effort const cache = new CacheStack([...], { writePolicy: 'best-effort' }) // Override: strict for critical data await cache.set('critical:config', config, { writePolicy: 'strict' // Override global setting }) ``` *** ## Write Strategies Control how writes are executed: immediately (write-through) or batched (write-behind). ### Write-Through (Default) Write to all layers immediately before returning: ```ts const cache = new CacheStack([...], { writeStrategy: 'write-through' // Default }) // Blocks until all layers confirm write await cache.set('user:123', userData) // -> Write to memory: 0.01ms // -> Write to Redis: 0.5ms // -> Total: ~0.51ms ``` ### Write-Behind Queue writes and flush them in batches: ```ts const cache = new CacheStack([...], { writeStrategy: 'write-behind', writeBehind: { maxQueueSize: 1000, // Max pending writes flushIntervalMs: 100, // Flush every 100ms batchSize: 100 // Flush when this batch size is reached } }) // Returns immediately, writes are queued await cache.set('user:123', userData) // -> Returns in <0.01ms // -> Write queued in memory // -> Flushed to all layers within 100ms ``` ### Write-Behind Configuration ```ts const cache = new CacheStack([...], { writeStrategy: 'write-behind', writeBehind: { maxQueueSize: 1000, // Maximum queued writes flushIntervalMs: 100, // Auto-flush interval batchSize: 100 // Maximum writes per flush }, writeCoordination: { maxPendingWrites: 10_000, // Pending key-write units across operations maxActiveKeys: 10_000, // Distinct keys retained for ordering maxPendingWritesPerKey: 1_000 // Operations queued behind one hot key } }) ``` Single-key and `mset()` paths share this ordering boundary because a stale write may need compensating cleanup after its backend call completes. The finite defaults prevent that promise chain from becoming an unbounded memory queue. Saturation rejects with `CacheWriteSaturationError`; handle it as backpressure. ### Use Cases #### Write-Through Use for data that must be persisted immediately: ```ts // User payments: write to cache immediately const paymentCache = new CacheStack([...], { writeStrategy: 'write-through' }) ``` #### Write-Behind Use for high-volume writes where slight delay is acceptable: ```ts // Analytics events: batch writes for performance const analyticsCache = new CacheStack([...], { writeStrategy: 'write-behind', writeBehind: { maxQueueSize: 10_000, flushIntervalMs: 1000, batchSize: 100 } }) ``` ### Manual Flush Manually trigger write-behind flush: ```ts await cache.set('user:123', userData) await cache.set('user:456', userData) // Manually flush pending writes await cache.flushWriteBehindQueue() ``` *** ## Fetcher Rate Limiting Prevent thundering herd problems by limiting concurrent fetcher executions. ### Configuration ```ts const cache = new CacheStack([...], { fetcherRateLimit: { maxConcurrent: 10, // Max 10 concurrent fetchers intervalMs: 1000, // Reset limit every second scope: 'global', // 'global' | 'key' | 'fetcher' queueOverflow: 'reject' } }) ``` ### Scope Options #### Global Scope Limit total concurrent fetchers across all keys: ```ts const cache = new CacheStack([...], { fetcherRateLimit: { maxConcurrent: 10, scope: 'global' } }) // Only 10 fetchers running at once, regardless of key await Promise.all([ cache.get('key1', fetch1), cache.get('key2', fetch2), // ... 100 more requests ]) // -> 10 fetchers run immediately // -> 90 wait in queue ``` #### Key Scope Limit concurrent fetchers **per key**: ```ts const cache = new CacheStack([...], { fetcherRateLimit: { maxConcurrent: 1, scope: 'key' } }) // Only 1 fetcher per key await Promise.all([ cache.get('user:123', fetchUser123), // Running cache.get('user:123', fetchUser123), // Queued cache.get('user:456', fetchUser456) // Running (different key) ]) ``` #### Fetcher Scope Limit concurrent fetchers **per unique fetcher function**: ```ts const fetchUser = (id: number) => db.findUser(id) const fetchPost = (id: number) => db.findPost(id) const cache = new CacheStack([...], { fetcherRateLimit: { maxConcurrent: 5, scope: 'fetcher' } }) // 5 fetchUser calls run concurrently // 5 fetchPost calls run concurrently await Promise.all([ cache.get('user:1', fetchUser), cache.get('user:2', fetchUser), // ... more fetchUser calls cache.get('post:1', fetchPost), cache.get('post:2', fetchPost), // ... more fetchPost calls ]) ``` ### Per-Operation Override ```ts // Global: no rate limiting const cache = new CacheStack([...]) // Override: rate limit specific operation await cache.get('expensive-key', fetchExpensiveData, { fetcherRateLimit: { maxConcurrent: 1, scope: 'key' } }) ``` ### Queue Behavior When rate limit is reached, requests queue and wait: ```ts const cache = new CacheStack([...], { fetcherRateLimit: { maxConcurrent: 2, scope: 'global' } }) // First 2 requests start immediately const p1 = cache.get('key1', fetch1) // Running const p2 = cache.get('key2', fetch2) // Running // Next request queues const p3 = cache.get('key3', fetch3) // Queued // When p1 or p2 completes, p3 starts ``` By default, saturated internal queues reject new work with a clear overflow error. If a caller intentionally prefers availability over strict limiting, use `queueOverflow: 'bypass'`: ```ts await cache.get('search:expensive', fetchSearch, { fetcherRateLimit: { maxConcurrent: 1, scope: 'key', queueOverflow: 'bypass' } }) ``` ### Combining with Stampede Prevention Fetcher rate limiting works with stampede prevention: ```ts const cache = new CacheStack([...], { stampedePrevention: true, // Dedupe concurrent requests for same key fetcherRateLimit: { maxConcurrent: 10, scope: 'key' } }) // 100 concurrent requests for 'user:123' await Promise.all( Array.from({ length: 100 }, () => cache.get('user:123', () => db.findUser(123)) ) ) // Result: // -> 1 fetcher runs (stampede prevention) // -> Rate limit not reached (only 1 concurrent fetcher) ``` *** ## Best Practices ### 1. Enable Graceful Degradation ```ts // GOOD: Always enable graceful degradation const cache = new CacheStack([...], { gracefulDegradation: { retryAfterMs: 10_000 } }) ``` ### 2. Use Circuit Breakers for Fragile Dependencies ```ts // GOOD: Protect fragile upstreams const cache = new CacheStack([...], { circuitBreaker: { failureThreshold: 5, cooldownMs: 30_000 } }) await cache.get('fragile-api-key', fetchFromApi, { circuitBreaker: { failureThreshold: 3, cooldownMs: 10_000 } }) ``` ### 3. Use Best-Effort for Non-Critical Data ```ts // GOOD: Use best-effort for non-critical data const analyticsCache = new CacheStack([...], { writePolicy: 'best-effort' }) ``` ### 4. Use Write-Behind for High-Volume Writes ```ts // GOOD: Batch high-volume writes const eventCache = new CacheStack([...], { writeStrategy: 'write-behind', writeBehind: { maxQueueSize: 10_000, flushIntervalMs: 1000 } }) ``` ### 5. Rate Limit Expensive Operations ```ts // GOOD: Rate limit expensive fetchers await cache.get('expensive-report', generateReport, { fetcherRateLimit: { maxConcurrent: 1, scope: 'key' } }) ``` --- url: /docs/serialization.md --- # Serialization & Compression Control how data is serialized and compressed in cache layers for optimal performance and storage efficiency. ## Table of Contents - [JSON Serializer](#json-serializer) - [MessagePack Serializer](#messagepack-serializer) - [Custom Serializers](#custom-serializers) - [Prototype Pollution Protection](#prototype-pollution-protection) - [Compression](#compression) - [Serializer Chain](#serializer-chain) *** ## JSON Serializer Default serializer for all cache layers. Uses `JSON.stringify` and `JSON.parse` with prototype pollution protection. ### Usage ```ts import { JsonSerializer } from 'layercache' const serializer = new JsonSerializer() // Serialize const payload = serializer.serialize({ foo: 'bar' }) // '{"foo":"bar"}' // Deserialize const data = serializer.deserialize('{"foo":"bar"}') // { foo: 'bar' } ``` ### Built-In Usage JSON serialization is used by default: ```ts import { MemoryLayer, RedisLayer, DiskLayer } from 'layercache' // All use JsonSerializer by default const memory = new MemoryLayer({ ttl: 60_000 }) const redis = new RedisLayer({ client: redis, ttl: 300_000 }) const disk = new DiskLayer({ directory: './cache' }) // Explicitly specify JSON serializer const redisJson = new RedisLayer({ client: redis, ttl: 300_000, serializer: new JsonSerializer() }) ``` ### Handling Special Values ```ts const serializer = new JsonSerializer() // Dates become ISO strings serializer.serialize({ date: new Date('2024-04-11') }) // '{"date":"2024-04-11T00:00:00.000Z"}' // undefined object properties are omitted serializer.serialize({ foo: undefined }) // '{}' // undefined array elements become null serializer.serialize([undefined]) // '[null]' // Functions are removed serializer.serialize({ foo: () => {} }) // '{}' ``` ### Prototype Pollution Protection All deserialized data is sanitized to prevent prototype pollution attacks: ```ts const serializer = new JsonSerializer() // Attempted prototype pollution is blocked const data = serializer.deserialize('{"__proto__":{"polluted":true}}') // -> Sanitized to: {} // -> Object.prototype.polluted remains undefined ``` *** ## MessagePack Serializer Binary serialization format that's more compact and faster than JSON. ### Installation ```bash npm install @msgpack/msgpack ``` ### Usage ```ts import { MsgpackSerializer } from 'layercache' const serializer = new MsgpackSerializer() // MessagePack nil has no undefined/null distinction serializer.deserialize(serializer.serialize(undefined)) // null // Serialize const payload = serializer.serialize({ foo: 'bar', num: 42 }) // Buffer <81 a3 66 6f 6f a3 62 61 72 a3 6e 75 6d 2a> // Deserialize const data = serializer.deserialize(payload) // { foo: 'bar', num: 42 } ``` ### Use with Cache Layers ```ts import { RedisLayer, DiskLayer } from 'layercache' const redis = new RedisLayer({ client: redis, ttl: 300_000, serializer: new MsgpackSerializer() }) const disk = new DiskLayer({ directory: './cache', serializer: new MsgpackSerializer() }) ``` ### Benefits - **Smaller size**: \~30-50% smaller than JSON for typical data - **Faster**: Binary parsing is faster than JSON - **Type preservation**: Preserves binary data, dates, etc. ### Comparison ```ts import { JsonSerializer, MsgpackSerializer } from 'layercache' const data = { id: 123, name: 'Alice', email: 'alice@example.com', tags: ['user', 'active'], created: new Date() } const jsonSerializer = new JsonSerializer() const msgpackSerializer = new MsgpackSerializer() const jsonPayload = jsonSerializer.serialize(data) // '{"id":123,"name":"Alice","email":"alice@example.com","tags":["user","active"],"created":"2024-04-11T00:00:00.000Z"}' // Length: ~140 bytes const msgpackPayload = msgpackSerializer.serialize(data) // Binary buffer // Length: ~95 bytes (32% smaller) console.log(`JSON size: ${jsonPayload.length} bytes`) console.log(`MessagePack size: ${msgpackPayload.length} bytes`) ``` *** ## Custom Serializers Implement the `CacheSerializer` interface to create custom serialization logic. ### Interface ```ts interface CacheSerializer { serialize(value: unknown): string | Buffer deserialize(payload: string | Buffer): T } ``` ### Example: CBOR Serializer ```ts import { encode, decode } from 'cbor' class CborSerializer implements CacheSerializer { serialize(value: unknown): Buffer { return Buffer.from(encode(value)) } deserialize(payload: string | Buffer): T { const buffer = Buffer.isBuffer(payload) ? payload : Buffer.from(payload, 'binary') return decode(buffer) as T } } // Usage const redis = new RedisLayer({ client: redis, ttl: 300_000, serializer: new CborSerializer() }) ``` ### Example: Base64 JSON Serializer ```ts class Base64JsonSerializer implements CacheSerializer { serialize(value: unknown): string { const json = JSON.stringify(value) return Buffer.from(json).toString('base64') } deserialize(payload: string | Buffer): T { const normalized = typeof payload === 'string' ? payload : payload.toString('binary') const json = Buffer.from(normalized, 'base64').toString('utf8') return JSON.parse(json) as T } } // Usage const memcached = new MemcachedLayer({ client: memcached, ttl: 300_000, serializer: new Base64JsonSerializer() }) ``` ### Example: Compressing Serializer ```ts import { gzip, ungzip } from 'node:zlib' import { promisify } from 'node:util' const gzipAsync = promisify(gzip) const ungzipAsync = promisify(ungzip) class GzipJsonSerializer implements CacheSerializer { async serialize(value: unknown): Promise { const json = JSON.stringify(value) return await gzipAsync(Buffer.from(json)) } async deserialize(payload: string | Buffer): Promise { const buffer = Buffer.isBuffer(payload) ? payload : Buffer.from(payload) const decompressed = await ungzipAsync(buffer) return JSON.parse(decompressed.toString('utf8')) as T } // Note: This returns Promise but interface expects sync // For async serialization, use a wrapper layer } ``` *** ## Prototype Pollution Protection All built-in serializers protect against prototype pollution attacks by sanitizing deserialized data. ### What is Prototype Pollution? Prototype pollution is a vulnerability where attackers modify `Object.prototype` to affect all objects in the application: ```json { "__proto__": { "admin": true } } ``` Without protection, this could make every object have `admin: true`. ### How layercache Protects layercache uses `StructuredDataSanitizer` to: 1. **Block dangerous keys**: Removes `__proto__`, `constructor`, `prototype` 2. **Limit depth**: Prevents deeply nested objects (default: 200 levels) 3. **Limit nodes**: Prevents excessive object size (default: 10,000 nodes) ### Configuration ```ts import { JsonSerializer } from 'layercache' const serializer = new JsonSerializer() // Default limits const data = serializer.deserialize(payload) // -> maxDepth: 200 // -> maxNodes: 10,000 // Custom limits (not exposed in API, defaults are safe) // JsonSerializer uses built-in safe defaults ``` ### Example: Blocked Attack ```ts const serializer = new JsonSerializer() const maliciousPayload = JSON.stringify({ user: 'alice', __proto__:: { admin: true } }) const data = serializer.deserialize(maliciousPayload) // -> { user: 'alice' } // -> __proto__ is removed // -> Object.prototype.admin remains undefined console.log(({} as any).admin) // undefined (safe!) ``` ### MessagePack Protection MessagePack serializer also sanitizes data: ```ts import { MsgpackSerializer } from 'layercache' const serializer = new MsgpackSerializer() // MessagePack payloads are sanitized on deserialize const data = serializer.deserialize(maliciousMsgpackBuffer) // -> Dangerous keys removed // -> Depth and size limits enforced ``` *** ## Compression Reduce memory usage and network bandwidth by compressing cached values. Supported in `RedisLayer`. ### Gzip Compression ```ts import { RedisLayer } from 'layercache' const redis = new RedisLayer({ client: redis, ttl: 300_000, compression: 'gzip', compressionThreshold: 1_024 // Only compress values >1KB }) // Large values are compressed automatically await cache.set('large-data', largeObject) // -> Serialized to JSON (or MessagePack) // -> Compressed with gzip if >1KB // -> Stored in Redis // Decompression is automatic on read const data = await cache.get('large-data') // -> Fetched from Redis // -> Decompressed // -> Deserialized ``` ### Brotli Compression ```ts const redis = new RedisLayer({ client: redis, ttl: 300_000, compression: 'brotli', compressionThreshold: 512 // Compress values >512 bytes }) ``` ### Compression Threshold Skip compression for small values (overhead isn't worth it): ```ts const redis = new RedisLayer({ client: redis, ttl: 300_000, compression: 'gzip', compressionThreshold: 1_024 // 1KB threshold }) // Small values: not compressed await cache.set('small', { id: 1 }) // Serialized: ~15 bytes // Stored: ~15 bytes (not compressed) // Large values: compressed await cache.set('large', largeArray) // Serialized: ~10KB // Stored: ~3KB (compressed) ``` ### Decompression Max Bytes Prevent decompression bomb attacks: ```ts const redis = new RedisLayer({ client: redis, ttl: 300_000, compression: 'gzip', compressionThreshold: 1_024, decompressionMaxBytes: 64 * 1_024 * 1_024 // 64MiB limit }) // If decompressed data exceeds 64MiB, read fails and key is deleted const data = await cache.get('malicious-key') // -> Throws error // -> Key is deleted from Redis ``` ### Compression Format Compressed values use a custom header format: ``` LCZ1:: Examples: LCZ1:gzip: LCZ1:brotli: ``` This header allows: - Format detection on read - Future algorithm support - Backward compatibility ### Performance Considerations **Compression tradeoffs:** | Factor | Gzip | Brotli | No Compression | | ------------------- | ----------- | ------------------- | -------------- | | Compression speed | Fast | Slow | N/A | | Decompression speed | Fast | Medium | N/A | | Compression ratio | Medium | High | N/A | | CPU usage | Low | Medium | None | | Best for | General use | Storage-constrained | Small values | **Recommendations:** ```ts // Use gzip for most cases (balanced) const redis = new RedisLayer({ client: redis, compression: 'gzip', compressionThreshold: 1_024 }) // Use brotli for storage-constrained environments const redis = new RedisLayer({ client: redis, compression: 'brotli', compressionThreshold: 512 }) // Disable compression for low-latency requirements const redis = new RedisLayer({ client: redis, compression: undefined // No compression }) ``` *** ## Serializer Chain Try multiple deserializers in sequence for smooth data format migrations. ### Configuration ```ts import { RedisLayer, JsonSerializer, MsgpackSerializer } from 'layercache' const redis = new RedisLayer({ client: redis, ttl: 300_000, serializer: [ new MsgpackSerializer(), // Try MessagePack first new JsonSerializer() // Fall back to JSON ] }) ``` ### How It Works **On write:** Always use the first serializer ```ts await cache.set('user:123', userData) // Serialized with MsgpackSerializer (first in array) ``` **On read:** Try each serializer until one succeeds ```ts const data = await cache.get('user:123') // Try MsgpackSerializer.deserialize() // -> If success: return data // -> If error: try JsonSerializer.deserialize() // -> If success: return data, migrate to MessagePack // -> If error: delete key and return null ``` ### Auto-Migration When a value is successfully deserialized with a non-primary serializer, it's automatically rewritten with the primary serializer: ```ts // Old data in Redis: JSON format const oldData = '{"id":123,"name":"Alice"}' // Read with serializer chain const data = await cache.get('user:123') // -> Try MessagePack: fails // -> Try JSON: succeeds, returns { id: 123, name: 'Alice' } // -> Rewrites with MessagePack // -> Next read is faster (MessagePack is first) ``` ### Migration Example ```ts // Phase 1: Deploy with JSON only const redis = new RedisLayer({ client: redis, serializer: new JsonSerializer() }) // Phase 2: Deploy with MessagePack, keep JSON fallback const redis = new RedisLayer({ client: redis, serializer: [ new MsgpackSerializer(), // New format new JsonSerializer() // Old format ] }) // Phase 3: Data auto-migrates on access // -> Existing JSON keys work fine // -> On first read, rewritten as MessagePack // -> New keys written as MessagePack // Phase 4: Remove JSON fallback (after all data migrated) const redis = new RedisLayer({ client: redis, serializer: new MsgpackSerializer() }) ``` ### Complex Chain ```ts import { MsgpackSerializer, JsonSerializer, CborSerializer } from 'layercache' const redis = new RedisLayer({ client: redis, serializer: [ new MsgpackSerializer(), // Current format new CborSerializer(), // Previous format new JsonSerializer() // Legacy format ] }) // Tries formats in order: // 1. MessagePack (current) // 2. CBOR (previous) // 3. JSON (legacy) // 4. Delete if all fail ``` *** ## Best Practices ### 1. Use MessagePack for Large Data ```ts // GOOD: MessagePack for large datasets const cache = new RedisLayer({ client: redis, serializer: new MsgpackSerializer(), compression: 'gzip', compressionThreshold: 1_024 }) ``` ### 2. Enable Compression for Redis ```ts // GOOD: Reduce Redis memory usage const redis = new RedisLayer({ client: redis, compression: 'gzip', compressionThreshold: 1_024 }) ``` ### 3. Use Serializer Chain for Migrations ```ts // GOOD: Support multiple formats during migration const redis = new RedisLayer({ client: redis, serializer: [ new MsgpackSerializer(), // New new JsonSerializer() // Old ] }) ``` ### 4. Set Compression Threshold Appropriately ```ts // GOOD: Avoid compressing small values const redis = new RedisLayer({ client: redis, compression: 'gzip', compressionThreshold: 1_024 // Only compress >1KB }) ``` ### 5. Protect Against Decompression Bombs ```ts // GOOD: Set decompression limit const redis = new RedisLayer({ client: redis, compression: 'gzip', decompressionMaxBytes: 64 * 1_024 * 1_024 // 64MiB }) ``` ### 6. Don't Compress Already-Compressed Data ```ts // BAD: Compressing already compressed data const cache = new RedisLayer({ client: redis, serializer: new MsgpackSerializer(), // Binary compression: 'gzip' // Compressing binary }) // GOOD: Skip compression for binary data const cache = new RedisLayer({ client: redis, serializer: new MsgpackSerializer() // No compression needed }) ``` --- url: /docs/tutorial.md --- # Tutorial: Getting Started with layercache A step-by-step guide to setting up and operating layercache in production. ## Table of Contents 1. [Create a Cache Stack](#1-create-a-cache-stack) 2. [Basic Read-Through Caching](#2-basic-read-through-caching) 3. [Warm Critical Keys at Startup](#3-warm-critical-keys-at-startup) 4. [Wrap Service Methods](#4-wrap-service-methods) 5. [Use Namespaces for Organization](#5-use-namespaces-for-organization) 6. [Set Up Tag-Based Invalidation](#6-set-up-tag-based-invalidation) 7. [Configure Stale Serving](#7-configure-stale-serving) 8. [Add Resilience](#8-add-resilience) 9. [Monitor with Stats & Metrics](#9-monitor-with-stats--metrics) 10. [Snapshot Before Deploys](#10-snapshot-before-deploys) *** ## 1. Create a Cache Stack Start with a two-layer setup: fast in-memory L1 and shared Redis L2. ```ts import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' import Redis from 'ioredis' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000, maxSize: 5_000 }), new RedisLayer({ client: new Redis(), ttl: 300_000, prefix: 'myapp:cache:', compression: 'gzip' }) ], { gracefulDegradation: { retryAfterMs: 10_000 }, stampedePrevention: true // on by default }) ``` **Why this setup?** - Memory (L1) handles repeated reads with \~0.01ms latency - Redis (L2) provides shared state across instances with \~0.5ms latency - Compression reduces Redis memory usage for large values - Graceful degradation keeps the cache working even if Redis goes down *** ## 2. Basic Read-Through Caching The simplest pattern: fetch on miss, cache automatically. ```ts // Fetcher runs once on miss, result fills all layers const user = await cache.get('user:123', () => db.findUser(123)) // Subsequent calls hit L1 (memory) - no DB or Redis call const sameUser = await cache.get('user:123') ``` With options: ```ts const user = await cache.get('user:123', () => db.findUser(123), { ttl: { memory: 30_000, redis: 600_000 }, // short L1, longer L2 tags: ['user', 'user:123'], // for bulk invalidation later ttlJitter: 5_000 // prevent synchronized expiry }) ``` *** ## 3. Warm Critical Keys at Startup Pre-populate the cache before traffic arrives: ```ts await cache.warm( [ { key: 'config:flags', fetcher: () => fetchFlags(), priority: 10 }, { key: 'catalog:top-100', fetcher: () => fetchCatalog(), priority: 5 }, { key: 'pricing:matrix', fetcher: () => fetchPricing(), priority: 5 }, ], { concurrency: 4, continueOnError: true } ) ``` Higher `priority` values load first. `continueOnError` ensures one failed fetch doesn't block the rest. *** ## 4. Wrap Service Methods Turn any async function into a cached function with automatic key derivation: ```ts const getUser = cache.wrap('user', (id: number) => db.findUser(id), { ttl: 60_000, tags: ['users'] }) // Calls are automatically cached with key "user:123" const user = await getUser(123) ``` With a custom key resolver: ```ts const searchProducts = cache.wrap( 'search', (query: string, page: number) => db.search(query, page), { keyResolver: (query, page) => `${query}:p${page}`, ttl: 30_000 } ) ``` *** ## 5. Use Namespaces for Organization Scope cache operations to avoid key collisions: ```ts const users = cache.namespace('users') const posts = cache.namespace('posts') await users.set('123', userData) // stored as "users:123" await posts.set('456', postData) // stored as "posts:456" // Clear only user cache await users.clear() // deletes "users:*" only // Nested namespaces for multi-tenancy const tenant = cache.namespace('tenant:acme') const tenantUsers = tenant.namespace('users') await tenantUsers.set('1', data) // stored as "tenant:acme:users:1" ``` *** ## 6. Set Up Tag-Based Invalidation Tag keys when writing, invalidate groups when data changes: ```ts // Tag related data together await cache.set('user:123', user, { tags: ['user:123'] }) await cache.set('user:123:posts', posts, { tags: ['user:123', 'posts'] }) await cache.set('user:123:profile', profile, { tags: ['user:123'] }) // When user 123 updates their profile, invalidate everything related await cache.invalidateByTag('user:123') // All three keys are deleted across all layers // Batch invalidation await cache.invalidateByTags(['users', 'posts'], 'any') // either tag await cache.invalidateByTags(['tenant:a', 'users'], 'all') // both tags ``` For multi-instance deployments, use `RedisTagIndex` so all servers share the same tag state: ```ts import { RedisTagIndex } from 'layercache' const tagIndex = new RedisTagIndex({ client: redis, prefix: 'myapp:tags', knownKeysShards: 16 }) const cache = new CacheStack([...], { tagIndex }) ``` *** ## 7. Configure Stale Serving Keep serving cached data even after expiry while refreshing in the background: ```ts const config = await cache.get('app:config', fetchConfig, { ttl: 60_000, staleWhileRevalidate: 30_000, // serve stale for 30s while refreshing staleIfError: 300_000 // serve stale for 5min if refresh fails }) ``` Combined with **refresh-ahead** to proactively refresh before expiry: ```ts const leaderboard = await cache.get('leaderboard', fetchLeaderboard, { ttl: 120_000, refreshAhead: 30_000 // start refreshing when <= 30s remain }) ``` *** ## 8. Add Resilience Protect your app from cascading failures: ```ts const cache = new CacheStack([...], { // Skip failed layers temporarily gracefulDegradation: { retryAfterMs: 10_000 }, // Stop hammering broken upstreams circuitBreaker: { failureThreshold: 5, cooldownMs: 30_000 }, // Rate limit fetcher calls fetcherRateLimit: { maxConcurrent: 10 }, // Don't fail writes if one layer is down writePolicy: 'best-effort' }) ``` *** ## 9. Monitor with Stats & Metrics ### Quick stats check ```ts const stats = cache.getStats() console.log(stats.metrics) // { hits, misses, fetches, staleHits, ... } console.log(stats.layers) // [{ name, isLocal, degradedUntil }] ``` ### HTTP stats endpoint ```ts import { createCacheStatsHandler } from 'layercache' app.get('/cache/stats', createCacheStatsHandler(cache)) ``` ### Health checks ```ts const health = await cache.healthCheck() // [{ layer: 'memory', healthy: true, latencyMs: 0.03 }, // { layer: 'redis', healthy: true, latencyMs: 0.41 }] ``` ### Event-based monitoring ```ts cache.on('hit', ({ key, layer }) => metrics.inc('cache.hit', { layer })) cache.on('miss', ({ key }) => metrics.inc('cache.miss')) cache.on('error', ({ event, ctx }) => logger.error(event, ctx)) ``` ### Admin CLI ```bash npx layercache stats --redis redis://localhost:6379 npx layercache keys --redis redis://localhost:6379 --pattern "user:*" ``` *** ## 10. Snapshot Before Deploys Save cache state before restarting: ```ts // Before shutdown await cache.persistToFile('./cache-snapshot.json') // After restart await cache.restoreFromFile('./cache-snapshot.json') ``` Or transfer between instances in-memory: ```ts const snapshot = await cache.exportState() await anotherCache.importState(snapshot) ``` *** ## Next Steps - [API Reference](/docs/api.md) - Full API documentation - [Migration Guide](/docs/migration.md) - Switching from another library - [Comparison](/docs/comparison.md) - Feature comparison with alternatives - [Observability](/docs/observability.md) - Performance measurement and monitoring --- url: /index.md --- # Layercache Production-ready caching for Node.js > Stack memory, Redis, disk, and Memcached behind one compact API with single-flight fetches, tag invalidation, stale serving, and operational metrics. [Get Started](./docs/getting-started) | [Playground](./playground) L1 · memory~0.1 msL2 · redis~1 msL3 · disk~5 msorigin · db50+ mshow a read travels 100 concurrent requests. *One* database call. read path · 1 of 4 ## L1 · Memory ~0.1 msAn in-process LRU answers hot keys without leaving the Node.js process. Most reads stop here and never go deeper. read path · 2 of 4 ## L2 · Redis ~1 msShared across every instance. A miss takes a single-flight lease — one caller runs the fetcher while the rest wait for its result. read path · 3 of 4 ## L3 · Disk ~5 msSurvives restarts and serves stale fallback when the layers above are cold or a fetch fails. read path · 4 of 4 ## Origin · your database 50+ msThe layer you're protecting. Watch the stampede: every few seconds 14 concurrent requests fall — exactly one reaches the database. then, on the way up ## The result backfills every layer One origin call refills disk, Redis, and memory on its way back — so the next read stops at L1 in a tenth of a millisecond. 0requests100%hit rate0origin callsscroll to follow a read↓capabilities ## Production concerns, already handled. [stack### Multi-layer stack Memory, Redis, disk, and Memcached share one read-through interface with automatic backfill. Read the docs →](/docs/layers)[flight### Single-flight fetches Concurrent callers collapse into one fetch locally, with Redis leases for distributed coordination. Read the docs →](/docs/resilience)[tags### Precise invalidation Expire by tag, prefix, pattern, or namespace without throwing away stale fallback state. Read the docs →](/docs/invalidation)[ops### Operational guardrails Circuit breakers, timeout controls, Prometheus metrics, OpenTelemetry spans, and CLI inspection. Read the docs →](/docs/observability)[adapters### Framework integrations Middleware and helpers for Express, Fastify, Hono, tRPC, GraphQL, and OpenTelemetry. Read the docs →](/docs/integrations)[sandbox### Browser playground Run real layercache examples in a worker-based sandbox, right inside this site. Read the docs →](/playground)quick start ## Two layers in nine lines. ```` import { CacheStack, MemoryLayer, RedisLayer } from 'layercache' import Redis from 'ioredis' const cache = new CacheStack([ new MemoryLayer({ ttl: 60_000, maxSize: 1_000 }), // L1: in-process new RedisLayer({ client: new Redis(), ttl: 3_600_000 }), // L2: shared ]) // Read-through: fetcher runs once, all layers filled const user = await cache.get('user:123', () => db.findUser(123)) ```` ## Put a cache stack in front of it. [Get started](/docs/getting-started)[GitHub](https://github.com/flyingsquirrel0419/layercache)Apache-2.0 · 672 tests passing · Node.js ≥ 20