Layercache

    Distributed Deployment

    Layercache provides distributed coordination features for multi-server deployments. Prevent stampedes, coordinate invalidations, and maintain consistent tag indexes across your infrastructure.

    Overview

    When running multiple instances of your application, you need to coordinate cache operations to avoid:

    1. Stampedes - Multiple servers fetching the same uncached key simultaneously
    2. Stale L1 caches - Memory layers holding outdated data after writes
    3. Inconsistent tags - Different servers with different tag-to-key mappings

    Layercache solves these with three Redis-based components:

    • RedisSingleFlightCoordinator - Distributed request deduplication
    • RedisInvalidationBus - Cross-server L1 invalidation via pub/sub
    • RedisTagIndex - Shared tag index for consistent invalidations

    RedisSingleFlightCoordinator

    Prevent multiple servers from fetching the same key simultaneously using distributed locking.

    How It Works

    1. First server to request a key acquires a Redis lock (SET NX PX)
    2. Lock holder runs the fetcher and writes the value
    3. Other servers wait and poll for the value
    4. Lock is released automatically via Lua script
    5. Waiters read the cached value and return

    Installation

    import { CacheStack, MemoryLayer, RedisLayer, RedisSingleFlightCoordinator } from 'layercache'
import Redis from 'ioredis'

const redis = new Redis()
const coordinator = new RedisSingleFlightCoordinator({ client: redis })

    Basic Setup

    const cache = new CacheStack([
  new MemoryLayer({ ttl: 60_000 }),
  new RedisLayer({ client: redis, ttl: 300_000 })
], {
  singleFlightCoordinator: coordinator,
  singleFlightLeaseMs: 30000,        // Lock duration
  singleFlightTimeoutMs: 5000,       // Max wait time
  singleFlightPollMs: 50             // Polling interval
})

    Configuration Options

    OptionTypeDefaultDescription
    singleFlightCoordinatorRedisSingleFlightCoordinator-Coordinator instance
    singleFlightLeaseMsnumber30000Lock duration in milliseconds
    singleFlightTimeoutMsnumber5000Max wait time for lock
    singleFlightPollMsnumber50Polling interval in milliseconds
    singleFlightRenewIntervalMsnumberleaseMs / 2Lease renewal interval

    Lease Renewal

    The lock holder automatically renews the lease to prevent expiration during long fetches:

    const cache = new CacheStack([/* ... */], {
  singleFlightCoordinator: coordinator,
  singleFlightLeaseMs: 60000,
  singleFlightRenewIntervalMs: 10000  // Renew every 10 seconds
})

    Custom Prefix

    const coordinator = new RedisSingleFlightCoordinator({
  client: redis,
  prefix: 'myapp:singleflight'
})

    RedisInvalidationBus

    Broadcast invalidations to all servers via Redis pub/sub.

    How It Works

    1. Server A writes a key
    2. Invalidation message is published to Redis channel
    3. All servers receive the message
    4. Each server invalidates its local memory layer
    5. Tag index is updated (if using RedisTagIndex)

    Installation

    import { CacheStack, MemoryLayer, RedisLayer, RedisInvalidationBus } from 'layercache'
import Redis from 'ioredis'

const publisher = new Redis()
const subscriber = new Redis()

const bus = new RedisInvalidationBus({
  publisher,
  subscriber,
  channel: 'myapp:invalidation',
  signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET
})

    Basic Setup

    const cache = new CacheStack([
  new MemoryLayer({ ttl: 60_000 }),
  new RedisLayer({ client: publisher, ttl: 300_000 })
], {
  invalidationBus: bus,
  broadcastL1Invalidation: true
})

    Configuration Options

    OptionTypeDefaultDescription
    invalidationBusRedisInvalidationBus-Bus instance
    broadcastL1InvalidationbooleanfalseEnable L1 invalidation broadcasts
    channelstring'layercache:invalidation'Pub/sub channel name
    signingSecretstring | Buffer-Optional HMAC-SHA256 secret for message authenticity
    loggerLogger-Optional logger for errors

    Invalidation Message Format

    {
  sourceId: string,      // Unique server ID
  scope: 'key' | 'keys' | 'clear',
  operation: 'write' | 'delete' | 'invalidate' | 'clear',
  keys: string[],        // Affected keys
  timestamp: number
}

    Example Flow

    // Server A
await cache.set('user:123', user)
// → Publishes { keys: ['user:123'], operation: 'write' }

// Server B (memory layer)
// → Receives message, deletes 'user:123' from memory

// Server B (next read)
await cache.get('user:123', fetchUser)
// → Miss in memory, hits in Redis, backfills memory

    Custom Channel

    const bus = new RedisInvalidationBus({
  publisher: redis,
  subscriber: new Redis(),
  channel: 'myapp:cache:invalidation',
  signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET,
  logger: console
})

    Signed Messages

    Use signingSecret when a Redis channel can be reached by more than one trusted application. Layercache signs published invalidation messages and verifies inbound messages before handlers run.

    const bus = new RedisInvalidationBus({
  publisher: redis,
  subscriber: new Redis(process.env.REDIS_URL),
  channel: 'myapp:cache:invalidation',
  signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET
})

    All instances on the same channel must use the same secret. Unsigned mode remains available for compatibility with older deployments.

    Cleanup

    // When shutting down, unsubscribe from Redis
await cache.disconnect()

    RedisTagIndex

    Maintain a consistent tag index across all servers using Redis sets.

    How It Works

    1. When setting a key with tags, add key-to-tags mapping to Redis
    2. Add key to tag-indexed sets for efficient lookups
    3. When invalidating by tag, fetch all keys for that tag
    4. Delete keys and publish invalidations

    Sharding for Scale

    The tag index shards known-keys sets across multiple Redis keys for horizontal scaling:

    • Default: 16 shards
    • Set knownKeysShards: 1 only when you intentionally need the legacy single-set layout
    • Sharding reduces single-key contention

    Installation

    import { CacheStack, MemoryLayer, RedisLayer, RedisTagIndex } from 'layercache'
import Redis from 'ioredis'

const redis = new Redis()

const tagIndex = new RedisTagIndex({
  client: redis,
  prefix: 'myapp:tag-index',
  knownKeysShards: 16  // Default: distribute across 16 sets
})

    Basic Setup

    const cache = new CacheStack([
  new MemoryLayer({ ttl: 60_000 }),
  new RedisLayer({ client: redis, ttl: 300_000 })
], {
  tagIndex
})

// Use tags as usual
await cache.set('user:123', user, { tags: ['user', 'user:123'] })
await cache.invalidateByTag('user:123')

    Configuration Options

    OptionTypeDefaultDescription
    clientRedis-Redis client
    prefixstring'layercache:tag-index'Key prefix for index
    scanCountnumber100SSCAN batch size
    knownKeysShardsnumber16Number of shards for known keys

    Redis Key Structure

    myapp:tag-index:tag:<tag>           → SET of keys with this tag
myapp:tag-index:key:<key>           → SET of tags for this key
myapp:tag-index:keys:<shard>        → SET of all known keys (sharded)

    Tag Invalidation Flow

    // 1. Set key with tags
await cache.set('user:123:posts', posts, {
  tags: ['user:123', 'posts']
})

// 2. Invalidate by tag
await cache.invalidateByTag('user:123')

// Behind the scenes:
// - Fetches all keys from 'myapp:tag-index:tag:user:123'
// - Deletes keys from Redis
// - Publishes invalidation to RedisInvalidationBus
// - Cleans up tag index entries

    Sharding Example

    // The default is already 16; tune explicitly for unusual workloads
const tagIndex = new RedisTagIndex({
  client: redis,
  knownKeysShards: 16  // Distribute across 16 sets
})

// Each shard handles ~1/16 of known keys
// Reduces contention on SSCAN/SADD operations

    Migrating Legacy Known Keys

    Older deployments may have a single known-key set at <prefix>:keys. Current RedisTagIndex reads that set for compatibility and logs a warning when it exists, but migration avoids repeated fallback scans:

    npx layercache migrate-tag-index \
  --redis rediss://prod-redis.example.com:6380 \
  --tag-index-prefix "myapp:tag-index" \
  --known-key-shards 16 \
  --require-tls

    Complete Distributed Setup

    Combine all three components for a production-ready distributed cache:

    import { CacheStack, MemoryLayer, RedisLayer } from 'layercache'
import {
  RedisSingleFlightCoordinator,
  RedisInvalidationBus,
  RedisTagIndex
} from 'layercache'
import Redis from 'ioredis'

const redis = new Redis({
  host: process.env.REDIS_HOST,
  port: 6379,
  retryStrategy: (times) => Math.min(times * 50, 2000)
})

// 1. Distributed single-flight
const coordinator = new RedisSingleFlightCoordinator({
  client: redis,
  prefix: 'myapp:singleflight'
})

// 2. Cross-server invalidation
const bus = new RedisInvalidationBus({
  publisher: redis,
  subscriber: new Redis(),
  channel: 'myapp:invalidation',
  signingSecret: process.env.LAYERCACHE_INVALIDATION_SECRET,
  logger: console
})

// 3. Shared tag index
const tagIndex = new RedisTagIndex({
  client: redis,
  prefix: 'myapp:tag-index',
  knownKeysShards: 16
})

// 4. Create cache stack
const cache = new CacheStack([
  new MemoryLayer({ ttl: 60_000 }),
  new RedisLayer({ client: redis, ttl: 300_000 })
], {
  // Distributed single-flight
  singleFlightCoordinator: coordinator,
  singleFlightLeaseMs: 30000,
  singleFlightTimeoutMs: 5000,
  singleFlightPollMs: 50,

  // L1 invalidation
  invalidationBus: bus,
  broadcastL1Invalidation: true,

  // Tag index
  tagIndex
})

// 5. Use the cache
await cache.set('user:123', user, { tags: ['user:123'] })
const user = await cache.get('user:123', fetchUser)
await cache.invalidateByTag('user:123')

// 6. Graceful shutdown
process.on('SIGTERM', async () => {
  await cache.disconnect()
  await redis.quit()
  process.exit(0)
})

    Architecture Diagram

    ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  Server A   │     │  Server B   │     │   Redis     │
├─────────────┤     ├─────────────┤     ├─────────────┤
│ Memory L1   │     │ Memory L1   │     │             │
│ Redis L2    │     │ Redis L2    │     │ Data Store  │
│             │     │             │     │             │
│ Coordinator │◄────┤ Coordinator │◄────│ Locks       │
│ Bus Pub     │     │ Bus Sub     │◄────│ Pub/Sub     │
│ Tag Index   │◄────┤ Tag Index   │◄────│ Tag Sets    │
└─────────────┘     └─────────────┘     └─────────────┘

    Performance Considerations

    Single Flight

    • Lock acquisition: ~1-2ms (SET NX)
    • Polling overhead: ~50ms intervals
    • Lease renewal: Background, non-blocking

    Invalidation Bus

    • Publish latency: <1ms
    • Message size: ~1KB per invalidation
    • Subscriber lag: Network-dependent

    Tag Index

    • Set operations: O(1) per key
    • Tag invalidation: O(n) where n = keys with tag
    • Sharding: Reduces contention by factor of shards

    For high-traffic deployments:

    const cache = new CacheStack([/* ... */], {
  // Single-flight: longer leases for slow fetchers
  singleFlightLeaseMs: 60000,
  singleFlightRenewIntervalMs: 10000,
  singleFlightPollMs: 100,

  // Tag index: more shards for scale
  tagIndex: new RedisTagIndex({
    client: redis,
    knownKeysShards: 16
  })
})

    Monitoring

    Track Coordinator Operations

    cache.on('error', ({ event, context }) => {
  if (context.operation === 'singleFlight') {
    console.error('Single-flight error:', context.error)
  }
})

    Monitor Bus Messages

    cache.on('invalidation', ({ keys, sourceId }) => {
  console.log(`Invalidated ${keys.length} keys from ${sourceId}`)
})

    Check Tag Index Size

    const redis = new Redis()
const keys = await redis.keys('layercache:tag-index:tag:*')
console.log(`Tracking ${keys.length} tags`)

    Troubleshooting

    Lock Timeouts

    If servers timeout waiting for locks:

    // Increase lease duration
singleFlightLeaseMs: 60000

// Increase wait timeout
singleFlightTimeoutMs: 10000

    Stale L1 Data

    If memory layers aren't invalidating:

    // Ensure broadcast is enabled
broadcastL1Invalidation: true

// Check bus connection
bus.on('error', (err) => console.error('Bus error:', err))

    Slow Tag Invalidations

    If tag invalidations are slow:

    // Increase shards for parallelization
tagIndex: new RedisTagIndex({
  client: redis,
  knownKeysShards: 16  // More shards
})

    High Redis Memory

    If tag index uses too much memory:

    // Reduce scan batch size
tagIndex: new RedisTagIndex({
  client: redis,
  scanCount: 50  // Fewer keys per SSCAN
})