Skip to main content

AttributeMap Partitioning

A single Attribute grows into one large value, and every write rewrites it. Both patterns split data into bounded AttributeMap instances so that a read or write touches only one instance.

Choose by access pattern:

  • Append-only data that must keep its order: Sequentially Chunked.
  • Lookup or update by a stable key: Hash-Partitioned.

Comparison​

DimensionSequentially ChunkedHash-Partitioned
GroupingIncreasing sequence; one chunk per 100 recordsFNV-1a of canonical key modulo 1,000
Primary goalEfficient append and ordered paginationDirect lookup and upsert by stable business key
Typical dataSubscriber history, agent messages, audit eventsCustomer profiles, account metadata, configuration directories
One readOne sequential chunkOne hash bucket
One writeRewrite the active chunk; also archive it at rolloverRewrite the target bucket
OrderingPreserves insertion order naturallyNo natural global order
Lookup by keyUsually scans chunks or needs another indexCalculates one partition directly
Updating old dataPoor fit; archived chunks should stay immutableGood fit for upsert within the target bucket
Write concurrencyAll appends contend on the active chunkDifferent partitions use independent locks
GrowthCreates new chunks without rehashingFixed partition count; each bucket grows with total data
DistributionNaturally balanced by record countUsually balanced, but hot keys can create hot partitions
Changing parametersNew data can use a versioned chunk strategyA partition-count change requires rehash and migration
Full enumerationRead chunks in sequenceRead up to 1,000 partitions
Best fitAppend-only data with time or sequence orderStructured values with a stable lookup key