Transaction Safety in Familia

Overview

Familia uses Redis transactions (MULTI/EXEC) for atomic operations. However, Redis transactions have a fundamental limitation: commands within a transaction return Redis::Future objects instead of actual values. These Future objects cannot be inspected until the transaction completes.

Core Rules

1. No Save Operations Inside Transactions or Pipelines

Rule: The following methods cannot be called within a transaction or pipeline context:

  • save
  • save!
  • save_if_not_exists!
  • create! (calls save_if_not_exists! internally)
  • build
  • atomic_write and Familia.atomic_write (they open their own MULTI/EXEC)

Rationale: These methods need to read current state for validation (checking existence, validating unique constraints), which would return uninspectable Redis::Future objects inside transactions and pipelines.

Error: Calling these methods inside a transaction or pipeline raises Familia::OperationModeError

Correct Pattern:

# ✅ GOOD: Save before transaction
customer = Customer.new(email: 'test@example.com')
customer.save  # Validates unique constraints here

customer.transaction do
  # Perform other atomic operations
  customer.increment(:login_count)
  customer.hset(:last_login, Familia.now.to_i)
end

# ❌ BAD: Save inside transaction
Customer.transaction do
  customer = Customer.new(email: 'test@example.com')
  customer.save  # Raises Familia::OperationModeError
end

2. Unique Index Guards Run Before Transactions

Rule: Unique constraint validation (guard_unique_indexes!) and the server-side claim (claim_unique_<index>!) must execute outside transaction context.

Rationale: Guards need to read current index state to check for duplicates, and the claim's CAS verdict must be inspectable. Inside a transaction, both return Redis::Future objects that cannot be inspected.

Implementation: save — and the partial writers commit_fields, save_fields, and multi_field_update — automatically guard and claim BEFORE starting their internal transaction, then re-affirm the claim inside it.

Scope note: save_fields and multi_field_update guard and claim only the indexes covering the fields being written. commit_fields writes the full hash, so it guards and claims EVERY class-level unique index — one CAS round-trip per index — even when the indexed value is unchanged (an unchanged value resolves as :owned and succeeds). If you are updating a small set of non-indexed fields on a class with several unique indexes, save_fields/multi_field_update avoid those extra round-trips.

3. Create with Success Callback

Pattern: Use the block form of create! for additional operations:

Customer.create!(email: 'new@example.com') do |customer|
  # This block only runs if creation succeeded
  customer.add_to_premium_group
  NotificationService.send_welcome_email(customer)
end

4. Command Replies Inside Transactions and Pipelines

Inside a transaction, atomic_write or pipelined block every command is queued, and redis-rb returns a Redis::Future in place of its reply. The Future's value is available only after the block completes. Each Familia method handles this in one of three ways.

Pass through

A method that issues a command and only converts its reply returns that command's Redis::Future. Converting means deserializing values, coercing a number, or testing the reply (a count, a rank, a TTL) to answer a predicate. After the block, the Future's value is the command's reply as redis-rb returns it, without Familia's conversion. Outside a block the same methods return what they always did, with two exceptions: HashKey#randfield(count, withvalues: true) now returns [field, value] pairs, and SortedSet#diff(withscores: true) returns [member, score] pairs instead of raising ArgumentError.

views = nil
empty = nil
user.atomic_write do
  user.name = 'Alice'                    # deferred scalar, queued as HMSET
  views = user.counts.increment('views') # Redis::Future
  empty = user.tags.empty?               # Redis::Future of SCARD
end
views.value        # => 6, HINCRBY replies with an Integer
empty.value.zero?  # => false, SCARD replies with the count
Method Future resolves to
HashKey#[], #values, #hgetall, #values_at; ListKey#range, #members, #pop; SortedSet#members, #range*; UnsortedSet#members, #sample the stored values, still serialized
HashKey#randfield(count, withvalues: true) [field, value] pairs, the values still serialized
empty? on any collection the count (HLEN, LLEN, SCARD, ZCARD)
The generated related-field predicates (user.tags?, User.instances?) what the field's empty? resolves to: the count, or the raw stored string
ListKey#member?, SortedSet#member?, #rank, #revrank the index or rank, or nil
The generated participation methods in_<target>_<collection>? and score_in_<target>_<collection> (domain.in_customer_domains?(customer)) for in_*?, the ZRANK or LPOS index or nil on a sorted-set or list participation, and the SISMEMBER Boolean on a set one; for score_in_*, the Float score or nil
DataType#exists?, Horreum.exists?, expires?, expired? the EXISTS count or the TTL in seconds
HashKey#increment, #decrement, #incrbyfloat; SortedSet#score, #increment, #mscore the Integer or Float, which redis-rb converts itself
StringKey#value, #size, #empty?, Counter#value, JsonStringKey#value, #char_count, #empty? the raw stored string, or nil
Lock#release true when the lock was released, false otherwise
Horreum.any?, .count, .in_instances?, .multiget, .storage_inspect the ZCARD count, the ZRANK reply, the MGET array or the HGETALL hash
Migration::Registry#applied?, #applied_at, #all_applied, #metadata the ZSCORE, ZRANGE or HGET reply

A Redis::Future is always truthy, so never test one inside the block. A predicate or conditional write used as a condition there takes the true branch whatever the reply turns out to be. That applies to the predicates in the table (including in_<target>_<collection>?), to HashKey#key?, UnsortedSet#member? and indexed_in?, and to the conditional writes HashKey#hsetnx, StringKey#setnx, JsonStringKey#setnx and Lock#release. For example, lock.release(token) inside a transaction returns a truthy Future even when token does not hold the lock. Read future.value after the block. Some Futures resolve to a count rather than a Boolean: test empty.value.zero?, not empty.value.

A top-level transaction or pipelined call returns a MultiResult whose results holds each queued command's reply in queue order. It is empty when a WATCH-guarded transaction aborted (see MultiResult#aborted?). atomic_write and Familia.atomic_write return true or false instead, so their callers read the values of the Futures they kept.

Fail fast

A method that needs a reply before it can finish raises Familia::OperationModeError before queueing anything. That covers methods that branch on a reply, raise from it, issue follow-up commands from it, iterate, load records, or derive an answer from the reply's contents. It also covers admission checks, whose answer tells the caller whether it may proceed: whether it holds a lock, has claimed a value, or is under a limit. Inside a block that answer would be a truthy Future, which admits the caller before the server has answered. Ruby's conversion methods (to_s, to_i, to_f, to_a, as_json, to_json) fail fast too, because Ruby and JSON generation call them implicitly and expect the converted type. A to_s that returns anything but a String is printed as #<Familia::StringKey:0x...> by string interpolation, Array#join and puts, with no error. Use the reader (value, members, hgetall) to get the command's Future instead.

  • Iteration: each, eachraw, eachraw_with_index, collectraw and selectraw on every collection, each_record, and scan_keys with a block
  • Conversions: to_s and to_i on StringKey (and so Counter and Lock), JsonStringKey#to_s, #to_i and #to_f, to_a on ListKey, SortedSet and UnsortedSet, and as_json and to_json on every DataType
  • Reads that decide: HashKey#fetch, HashKey#refresh! and #refresh, Horreum#refresh! and #refresh, extend_expiration, ttl_report
  • Loading: find_by_dbkey, find_by_identifier (find_by_id, find, load), load_multi, load_multi_by_keys, all, find_by_objid, find_by_extid, the index finders (find_by_*, find_all_by_*, sample_from_*), the participation readers (*_ids, *_count, *_instances, the participation predicate <target>? such as user.project_team?, current_participations, position_in_*, *_with_permission, each_*_with_permission), current_indexings, relationship_status and validate_relationships!
  • Scans and maintenance: keys_count, keys_any?, scan_count (count!), scan_any? (any!), the audit_*, health_check, repair_* and rebuild_* methods, run_chores!, the EnforceCollectionCaps chore, and the index format checks IndexDescriptor#stale_format? and #format_current?, Familia.stale_indexes and Familia.assert_indexes_current!
  • Rebuild primitives, which take their connection as an argument: AtomicOperations.atomic_swap and .sweep_orphaned_temp_keys raise when that connection is a transaction or pipeline connection, which Familia.dbclient is inside a Familia block. AtomicOperations.with_rebuild raises in that case and inside any Familia block
  • Migrations: Migration::Base.run and .check_only, and Migration::Runner#run, #run_one, #rollback, #status and #pending. The Migration::Registry methods that decide from a reply (pending, status, record_rollback, schema_changed?, schema_drift, restore_backup) raise when the registry's client is a transaction or pipeline connection, which it is inside a Familia block for a registry built without redis:. A registry built with a client passed as redis: keeps that client and keeps working inside a block
  • Writes that read first: the save methods in rule 1, commit_fields, save_fields, multi_field_update, multi_field_fast_write, class-level destroy!, instance destroy! on a class with instance-scoped indexes, the guard_unique_*! methods, the staged participation methods (stage_*, activate_*_instance, unstage_*), and the add and remove methods of a through: participation. Inside a pipeline this also covers the add_to_* and update_in_* methods of every unique_index, class-level and instance-scoped. Inside a transaction they re-affirm a claim or write without a uniqueness check instead (see Examples 3 and 4)
  • Admission checks: Lock#acquire, #locked?, #held_by? and #empty? (and so the generated predicate of a lock field), Counter#increment_if_less_than, HashKey#claim_field, and claim_unique_*!. Lock#release is deliberately not one. Its script checks the token and deletes the lock in one step when it runs, so it can be queued, and its Future resolves to true or false. It does not guard the block's other commands: a false reply does not stop them, so a lock lost before EXEC is reported only after they have run (see the Lock section of the migration guide)
  • Fast writers (field!) on fields backing a class-level index raise Familia::IndexedFieldFastWriteError, since the index claim cannot run there

Call these before or after the block.

Queue gated side effects

Some writes refresh the TTL only when they took effect. Outside a block they read the write's reply to decide. Inside a block that reply is a Future, so the decision is queued with the write, and the TTL ends up as it would outside a block:

  • HashKey#hsetnx, JsonStringKey#setnx, JsonStringKey#value with a default, and ListKey#insert can do nothing on a key that exists (the field or key is already set, or the pivot is missing). They queue one EVAL that runs the write and calls EXPIRE only when the write took effect, so a write that does nothing leaves an existing key's TTL unchanged, and a key the write creates gets its TTL. hsetnx and setnx return the EVAL Future, which resolves to the same true or false they return outside a block; insert returns one that resolves to the same Integer. Each of the four adds one reply to MultiResult#results; for JsonStringKey#value that is the true or false SETNX outcome, ahead of the GET reply. Because they send EVAL inside a block, the connection's user needs scripting permission there (ACL @scripting, or +eval). Without it the server answers NOPERM, Familia raises Redis::PermissionError, and a transaction applies none of the block's commands.
  • ListKey#pushx and #unshiftx, and SortedSet#popmin and #popmax, write whenever the key exists, and EXPIRE does nothing on a missing key. They queue the refresh after the write. A pop with a count of 0 removes nothing and queues no refresh.

A DataType with no expiration (default_expiration 0, or a relation declared with no_expiration: true) queues no refresh.

5. Handling Nested Transactions

Behavior: Familia uses reentrant transactions (see TransactionCore.execute_normal_transaction). If you're already in a transaction, a nested transaction call does not open a new MULTI/EXEC — it yields the outer transaction's connection, so the nested block's commands are queued into the outer MULTI and commit (or fail) with the outer EXEC.

Nesting itself never raises. The Familia::OperationModeError cases described elsewhere in this document come from what runs inside a transaction (save, create!, partial writes, and the other methods rule 4 lists) or from connection handlers that do not support transactions, not from nested transaction calls.

Customer.transaction do |conn|
  # Outer transaction
  customer.increment(:counter)

  customer.transaction do |inner_conn|
    # Same connection as outer - no new MULTI/EXEC
    customer.decrement(:other_counter)
  end
end

Examples

Example 1: Correct Usage with Unique Constraints

# Create with unique email constraint
begin
  customer = Customer.create!(email: 'user@example.com')
  puts "Created customer: #{customer.email}"
rescue Familia::RecordExistsError => e
  puts "Customer already exists: #{e.message}"
end

Example 2: Atomic Multi-Object Updates

# Save all objects first
order = Order.new(order_id: 'ORD-123')
order.save

inventory = Inventory.find('ITEM-456')

# Then use transaction for atomic updates
Order.transaction do
  order.hset(:status, 'confirmed')
  inventory.decrement(:quantity, 1)
  order.add_to_daily_orders
end

Example 3: Bulk Creation Pattern

# Claim each unique value outside the transaction. The claim is a
# server-side compare-and-set: the loser of a concurrent race raises
# Familia::RecordExistsError here, before anything is written.
customers = emails.map do |email|
  customer = Customer.new(email: email)
  customer.claim_unique_email_lookup!
  customer
end

# Then save atomically. The in-transaction index write is legal only
# because it re-affirms the claim taken above (see ADR-0002); without
# a claim, add_to_class_* raises Familia::OperationModeError.
Customer.transaction do
  customers.each do |customer|
    # Direct write operations only - no save!
    customer.hmset(customer.to_h_for_storage)
    customer.add_to_class_email_lookup
  end
end

Example 4: Instance-Scoped Bulk Indexing

# Instance-scoped indexes can be added within transactions
# Uniqueness validation is automatically skipped inside transactions
# Inside a pipeline, add_to_* raises Familia::OperationModeError instead
Company.transaction do
  employees.each do |employee|
    # Safe: validation skipped, direct index write only
    employee.add_to_company_badge_index(company)
  end
end

Common Pitfalls

Pitfall 1: Checking Existence in Transaction

# ❌ WRONG
Customer.transaction do
  unless customer.exists?  # Returns Redis::Future, always truthy!
    customer.save
  end
end

# ✅ CORRECT
unless customer.exists?
  customer.save
end

Pitfall 2: Creating in Transaction

# ❌ WRONG
Customer.transaction do
  Customer.create!(email: 'test@example.com')  # Raises OperationModeError
end

# ✅ CORRECT
customer = Customer.create!(email: 'test@example.com')
customer.transaction do
  # Additional operations
end

Migration Guide

If you have code that saves within transactions:

  1. Move save operations outside the transaction
  2. Use the transaction for atomic updates only
  3. Validate constraints before entering the transaction
  4. Use write-only operations inside the transaction

See Also