Transaction Safety in Familia
Overview
Familia uses Redis transactions (MULTI/EXEC) for atomic operations. However, Redis transactions have a fundamental limitation: commands within a transaction return Redis::Future objects instead of actual values. These Future objects cannot be inspected until the transaction completes.
Core Rules
1. No Save Operations Inside Transactions or Pipelines
Rule: The following methods cannot be called within a transaction or pipeline context:
savesave!save_if_not_exists!create!(callssave_if_not_exists!internally)buildatomic_writeandFamilia.atomic_write(they open their own MULTI/EXEC)
Rationale: These methods need to read current state for validation (checking existence, validating unique constraints), which would return uninspectable Redis::Future objects inside transactions and pipelines.
Error: Calling these methods inside a transaction or pipeline raises Familia::OperationModeError
Correct Pattern:
# ✅ GOOD: Save before transaction
customer = Customer.new(email: 'test@example.com')
customer.save # Validates unique constraints here
customer.transaction do
# Perform other atomic operations
customer.increment(:login_count)
customer.hset(:last_login, Familia.now.to_i)
end
# ❌ BAD: Save inside transaction
Customer.transaction do
customer = Customer.new(email: 'test@example.com')
customer.save # Raises Familia::OperationModeError
end
2. Unique Index Guards Run Before Transactions
Rule: Unique constraint validation (guard_unique_indexes!) and the server-side claim (claim_unique_<index>!) must execute outside transaction context.
Rationale: Guards need to read current index state to check for duplicates, and the claim's CAS verdict must be inspectable. Inside a transaction, both return Redis::Future objects that cannot be inspected.
Implementation: save — and the partial writers commit_fields, save_fields, and multi_field_update — automatically guard and claim BEFORE starting their internal transaction, then re-affirm the claim inside it.
Scope note: save_fields and multi_field_update guard and claim only the indexes covering the fields being written. commit_fields writes the full hash, so it guards and claims EVERY class-level unique index — one CAS round-trip per index — even when the indexed value is unchanged (an unchanged value resolves as :owned and succeeds). If you are updating a small set of non-indexed fields on a class with several unique indexes, save_fields/multi_field_update avoid those extra round-trips.
3. Create with Success Callback
Pattern: Use the block form of create! for additional operations:
Customer.create!(email: 'new@example.com') do |customer|
# This block only runs if creation succeeded
customer.add_to_premium_group
NotificationService.send_welcome_email(customer)
end
4. Command Replies Inside Transactions and Pipelines
Inside a transaction, atomic_write or pipelined block every command is
queued, and redis-rb returns a Redis::Future in place of its reply. The
Future's value is available only after the block completes. Each Familia
method handles this in one of three ways.
Pass through
A method that issues a command and only converts its reply returns that
command's Redis::Future. Converting means deserializing values, coercing a
number, or testing the reply (a count, a rank, a TTL) to answer a predicate.
After the block, the Future's value is the command's reply as redis-rb
returns it, without Familia's conversion. Outside a block the same methods
return what they always did, with two exceptions:
HashKey#randfield(count, withvalues: true) now returns [field, value] pairs,
and SortedSet#diff(withscores: true) returns [member, score] pairs instead
of raising ArgumentError.
views = nil
empty = nil
user.atomic_write do
user.name = 'Alice' # deferred scalar, queued as HMSET
views = user.counts.increment('views') # Redis::Future
empty = user..empty? # Redis::Future of SCARD
end
views.value # => 6, HINCRBY replies with an Integer
empty.value.zero? # => false, SCARD replies with the count
| Method | Future resolves to |
|---|---|
HashKey#[], #values, #hgetall, #values_at; ListKey#range, #members, #pop; SortedSet#members, #range*; UnsortedSet#members, #sample |
the stored values, still serialized |
HashKey#randfield(count, withvalues: true) |
[field, value] pairs, the values still serialized |
empty? on any collection |
the count (HLEN, LLEN, SCARD, ZCARD) |
The generated related-field predicates (user.tags?, User.instances?) |
what the field's empty? resolves to: the count, or the raw stored string |
ListKey#member?, SortedSet#member?, #rank, #revrank |
the index or rank, or nil |
The generated participation methods in_<target>_<collection>? and score_in_<target>_<collection> (domain.in_customer_domains?(customer)) |
for in_*?, the ZRANK or LPOS index or nil on a sorted-set or list participation, and the SISMEMBER Boolean on a set one; for score_in_*, the Float score or nil |
DataType#exists?, Horreum.exists?, expires?, expired? |
the EXISTS count or the TTL in seconds |
HashKey#increment, #decrement, #incrbyfloat; SortedSet#score, #increment, #mscore |
the Integer or Float, which redis-rb converts itself |
StringKey#value, #size, #empty?, Counter#value, JsonStringKey#value, #char_count, #empty? |
the raw stored string, or nil |
Lock#release |
true when the lock was released, false otherwise |
Horreum.any?, .count, .in_instances?, .multiget, .storage_inspect |
the ZCARD count, the ZRANK reply, the MGET array or the HGETALL hash |
Migration::Registry#applied?, #applied_at, #all_applied, #metadata |
the ZSCORE, ZRANGE or HGET reply |
A Redis::Future is always truthy, so never test one inside the block. A
predicate or conditional write used as a condition there takes the true
branch whatever the reply turns out to be. That applies to the predicates in
the table (including in_<target>_<collection>?), to HashKey#key?,
UnsortedSet#member? and indexed_in?, and to the conditional writes
HashKey#hsetnx, StringKey#setnx, JsonStringKey#setnx and
Lock#release. For example,
lock.release(token) inside a transaction returns a truthy Future even when
token does not hold the lock. Read future.value after the block. Some
Futures resolve to a count rather than a Boolean: test
empty.value.zero?, not empty.value.
A top-level transaction or pipelined call returns a MultiResult whose
results holds each queued command's reply in queue order. It is empty when
a WATCH-guarded transaction aborted (see MultiResult#aborted?).
atomic_write and Familia.atomic_write return true or false instead,
so their callers read the values of the Futures they kept.
Fail fast
A method that needs a reply before it can finish raises
Familia::OperationModeError before queueing anything. That covers methods
that branch on a reply, raise from it, issue follow-up commands from it,
iterate, load records, or derive an answer from the reply's contents. It
also covers admission checks, whose answer tells the caller whether it may
proceed: whether it holds a lock, has claimed a value, or is under a limit.
Inside a block that answer would be a truthy Future, which admits the caller
before the server has answered. Ruby's conversion methods (to_s, to_i,
to_f, to_a, as_json, to_json) fail fast too, because Ruby and JSON
generation call them implicitly and expect the converted type. A to_s that
returns anything but a String is printed as #<Familia::StringKey:0x...> by
string interpolation, Array#join and puts, with no error. Use the reader
(value, members, hgetall) to get the command's Future instead.
- Iteration:
each,eachraw,eachraw_with_index,collectrawandselectrawon every collection,each_record, andscan_keyswith a block - Conversions:
to_sandto_ionStringKey(and soCounterandLock),JsonStringKey#to_s,#to_iand#to_f,to_aonListKey,SortedSetandUnsortedSet, andas_jsonandto_jsonon every DataType - Reads that decide:
HashKey#fetch,HashKey#refresh!and#refresh,Horreum#refresh!and#refresh,extend_expiration,ttl_report - Loading:
find_by_dbkey,find_by_identifier(find_by_id,find,load),load_multi,load_multi_by_keys,all,find_by_objid,find_by_extid, the index finders (find_by_*,find_all_by_*,sample_from_*), the participation readers (*_ids,*_count,*_instances, the participation predicate<target>?such asuser.project_team?,current_participations,position_in_*,*_with_permission,each_*_with_permission),current_indexings,relationship_statusandvalidate_relationships! - Scans and maintenance:
keys_count,keys_any?,scan_count(count!),scan_any?(any!), theaudit_*,health_check,repair_*andrebuild_*methods,run_chores!, theEnforceCollectionCapschore, and the index format checksIndexDescriptor#stale_format?and#format_current?,Familia.stale_indexesandFamilia.assert_indexes_current! - Rebuild primitives, which take their connection as an argument:
AtomicOperations.atomic_swapand.sweep_orphaned_temp_keysraise when that connection is a transaction or pipeline connection, whichFamilia.dbclientis inside a Familia block.AtomicOperations.with_rebuildraises in that case and inside any Familia block - Migrations:
Migration::Base.runand.check_only, andMigration::Runner#run,#run_one,#rollback,#statusand#pending. TheMigration::Registrymethods that decide from a reply (pending,status,record_rollback,schema_changed?,schema_drift,restore_backup) raise when the registry's client is a transaction or pipeline connection, which it is inside a Familia block for a registry built withoutredis:. A registry built with a client passed asredis:keeps that client and keeps working inside a block - Writes that read first: the save methods in rule 1,
commit_fields,save_fields,multi_field_update,multi_field_fast_write, class-leveldestroy!, instancedestroy!on a class with instance-scoped indexes, theguard_unique_*!methods, the staged participation methods (stage_*,activate_*_instance,unstage_*), and the add and remove methods of athrough:participation. Inside a pipeline this also covers theadd_to_*andupdate_in_*methods of everyunique_index, class-level and instance-scoped. Inside a transaction they re-affirm a claim or write without a uniqueness check instead (see Examples 3 and 4) - Admission checks:
Lock#acquire,#locked?,#held_by?and#empty?(and so the generated predicate of alockfield),Counter#increment_if_less_than,HashKey#claim_field, andclaim_unique_*!.Lock#releaseis deliberately not one. Its script checks the token and deletes the lock in one step when it runs, so it can be queued, and its Future resolves totrueorfalse. It does not guard the block's other commands: afalsereply does not stop them, so a lock lost before EXEC is reported only after they have run (see theLocksection of the migration guide) - Fast writers (
field!) on fields backing a class-level index raiseFamilia::IndexedFieldFastWriteError, since the index claim cannot run there
Call these before or after the block.
Queue gated side effects
Some writes refresh the TTL only when they took effect. Outside a block they read the write's reply to decide. Inside a block that reply is a Future, so the decision is queued with the write, and the TTL ends up as it would outside a block:
HashKey#hsetnx,JsonStringKey#setnx,JsonStringKey#valuewith a default, andListKey#insertcan do nothing on a key that exists (the field or key is already set, or the pivot is missing). They queue one EVAL that runs the write and calls EXPIRE only when the write took effect, so a write that does nothing leaves an existing key's TTL unchanged, and a key the write creates gets its TTL.hsetnxandsetnxreturn the EVAL Future, which resolves to the sametrueorfalsethey return outside a block;insertreturns one that resolves to the same Integer. Each of the four adds one reply toMultiResult#results; forJsonStringKey#valuethat is thetrueorfalseSETNX outcome, ahead of the GET reply. Because they send EVAL inside a block, the connection's user needs scripting permission there (ACL@scripting, or+eval). Without it the server answersNOPERM, Familia raisesRedis::PermissionError, and a transaction applies none of the block's commands.ListKey#pushxand#unshiftx, andSortedSet#popminand#popmax, write whenever the key exists, and EXPIRE does nothing on a missing key. They queue the refresh after the write. A pop with a count of 0 removes nothing and queues no refresh.
A DataType with no expiration (default_expiration 0, or a relation declared
with no_expiration: true) queues no refresh.
5. Handling Nested Transactions
Behavior: Familia uses reentrant transactions (see
TransactionCore.execute_normal_transaction). If you're already in a
transaction, a nested transaction call does not open a new MULTI/EXEC — it
yields the outer transaction's connection, so the nested block's commands are
queued into the outer MULTI and commit (or fail) with the outer EXEC.
Nesting itself never raises. The Familia::OperationModeError cases described
elsewhere in this document come from what runs inside a transaction (save,
create!, partial writes, and the other methods rule 4 lists) or from
connection handlers that do not support transactions, not from nested
transaction calls.
Customer.transaction do |conn|
# Outer transaction
customer.increment(:counter)
customer.transaction do |inner_conn|
# Same connection as outer - no new MULTI/EXEC
customer.decrement(:other_counter)
end
end
Examples
Example 1: Correct Usage with Unique Constraints
# Create with unique email constraint
begin
customer = Customer.create!(email: 'user@example.com')
puts "Created customer: #{customer.email}"
rescue Familia::RecordExistsError => e
puts "Customer already exists: #{e.}"
end
Example 2: Atomic Multi-Object Updates
# Save all objects first
order = Order.new(order_id: 'ORD-123')
order.save
inventory = Inventory.find('ITEM-456')
# Then use transaction for atomic updates
Order.transaction do
order.hset(:status, 'confirmed')
inventory.decrement(:quantity, 1)
order.add_to_daily_orders
end
Example 3: Bulk Creation Pattern
# Claim each unique value outside the transaction. The claim is a
# server-side compare-and-set: the loser of a concurrent race raises
# Familia::RecordExistsError here, before anything is written.
customers = emails.map do |email|
customer = Customer.new(email: email)
customer.claim_unique_email_lookup!
customer
end
# Then save atomically. The in-transaction index write is legal only
# because it re-affirms the claim taken above (see ADR-0002); without
# a claim, add_to_class_* raises Familia::OperationModeError.
Customer.transaction do
customers.each do |customer|
# Direct write operations only - no save!
customer.hmset(customer.to_h_for_storage)
customer.add_to_class_email_lookup
end
end
Example 4: Instance-Scoped Bulk Indexing
# Instance-scoped indexes can be added within transactions
# Uniqueness validation is automatically skipped inside transactions
# Inside a pipeline, add_to_* raises Familia::OperationModeError instead
Company.transaction do
employees.each do |employee|
# Safe: validation skipped, direct index write only
employee.add_to_company_badge_index(company)
end
end
Common Pitfalls
Pitfall 1: Checking Existence in Transaction
# ❌ WRONG
Customer.transaction do
unless customer.exists? # Returns Redis::Future, always truthy!
customer.save
end
end
# ✅ CORRECT
unless customer.exists?
customer.save
end
Pitfall 2: Creating in Transaction
# ❌ WRONG
Customer.transaction do
Customer.create!(email: 'test@example.com') # Raises OperationModeError
end
# ✅ CORRECT
customer = Customer.create!(email: 'test@example.com')
customer.transaction do
# Additional operations
end
Migration Guide
If you have code that saves within transactions:
- Move save operations outside the transaction
- Use the transaction for atomic updates only
- Validate constraints before entering the transaction
- Use write-only operations inside the transaction