Relationships Indexing Guide
Indexing provides O(1) field-to-object lookups using Redis data structures, enabling fast attribute-based queries without relationship semantics.
Core Concepts
Indexing creates fast lookups for finding objects by field values:
- O(1) performance - Hash/Set-based constant-time access
- Automatic management - Class indexes update on every hash-writing path (save, destroy, and the partial writers — see Index Maintenance by Write Path)
- Flexible scoping - Global or parent-scoped uniqueness
- Query generation - Automatic
find_by_*methods
Index Types
| Type | Scope | Use Case | Structure |
|---|---|---|---|
unique_index |
Class | Global unique fields | Redis HashKey |
unique_index |
Instance | Parent-scoped unique | Redis HashKey |
multi_index |
Class (default) | Global non-unique groupings | Redis Set per value |
multi_index |
Instance | Parent-scoped groupings | Redis Set per value |
Class-Level Unique Indexing
Global unique field lookups with automatic management:
class User < Familia::Horreum
feature :relationships
field :email, :username
unique_index :email, :email_lookup
unique_index :username, :username_lookup
end
# Automatic indexing on save
user = User.create(email: 'alice@example.com')
User.find_by_email('alice@example.com') # => user (O(1) lookup)
# Automatic update on field change
user.update(email: 'alice.smith@example.com')
User.find_by_email('alice.smith@example.com') # => user
# Automatic cleanup on destroy
user.destroy
User.find_by_email('alice.smith@example.com') # => nil
Uniqueness is enforced server-side, before the transaction opens.
saveclaims the value with a single-key Lua compare-and-set, so two concurrent saves of the same value cannot both see it as free — the loser raisesFamilia::RecordExistsErrorcarryingexisting_id. TheHSETinside save's MULTI only re-affirms a claim the record already holds. Every save path (save,save_if_not_exists!,atomic_write,Familia.atomic_write) claims automatically; a transaction you open yourself does not, so writing an index from inside one needs an explicitclaim_unique_<index>!first — seeFamilia::OperationModeErrorunder Troubleshooting and ADR-0002.
Index Maintenance by Write Path
Every path that writes the object hash either maintains class-level indexes
(unique and multi alike) fail-closed — the claim precedes the hash write, so a
Familia::RecordExistsError leaves the hash untouched — or refuses the write
outright rather than leave lookups stale (#308):
| Write path | Class-level index maintenance |
|---|---|
save / save! / save_if_not_exists! / create! |
All class-level indexes: claim before the MULTI, update inside it |
atomic_write / Familia.atomic_write |
All class-level indexes (claims the values held when the block opens — see Troubleshooting) |
commit_fields |
All class-level indexes (writes the full hash) |
save_fields(:a, :b) |
Indexes on the named fields only |
multi_field_update(a: 1) |
Indexes on the written fields only |
multi_field_fast_write(a: 1) |
Refused: raises Familia::IndexedFieldFastWriteError if any written field backs a class-level index — its single-HMSET contract leaves no room for the out-of-transaction claim |
field!(value) fast writer |
Maintained (claim, then index update, then HSET) — but NOT as one MULTI; see below. Inside a transaction or pipeline it raises Familia::IndexedFieldFastWriteError instead |
destroy! |
All class-level index entries removed |
For multi_index, "maintained" means add-only: a value change adds the
identifier to the new bucket without retracting the old one, same as on save.
Call update_in_class_* with the old value to retract it.
Unlike save and multi_field_update, the field! fast writer does not wrap
its index update and hash write in a single MULTI — they are separate commands
(the fast writer's contract is a single hash command on the happy path). If the
HSET fails after the index entries were already updated (e.g. a dropped
connection), the writer compensates best-effort before re-raising the original
error: it restores the in-memory value, releases the just-claimed entry, and
restores the previous value's entry. A failure during compensation is logged
via Familia.warn and never masks the original error, so in the worst case a
stale index entry remains until the next save.
Generated Methods
| Method | Description |
|---|---|
User.find_by_email(email) |
O(1) lookup |
User.index_email_for(user) |
Manual index |
User.unindex_email_for(user) |
Remove from index |
User.reindex_email_for(user) |
Update index |
Instance-Scoped Unique Indexing
Unique within parent context, allowing duplicates across parents:
class Employee < Familia::Horreum
feature :relationships
field :badge_number
unique_index :badge_number, :badge_index, within: Company
end
# Manual indexing required (needs parent context)
company1 = Company.create(name: 'Acme Corp')
company2 = Company.create(name: 'Beta Inc')
emp1 = Employee.create(badge_number: '12345')
emp1.add_to_company_badge_index(company1)
emp2 = Employee.create(badge_number: '12345') # Same badge OK
emp2.add_to_company_badge_index(company2)
# Scoped lookups
company1.find_by_badge_number('12345') # => emp1
company2.find_by_badge_number('12345') # => emp2
Note: The indexed object must be persisted before it can be added to an index.
add_to_*/update_in_*(instance-scoped) andadd_to_class_*/update_in_class_*(class-level) raiseFamilia::PersistenceErrorfor unsaved objects, since the index entry would point at a record that does not exist yet. Callsavefirst — for class-level indexes,savepopulates the index automatically anyway.
Automatic Refresh and Cleanup
The first add_to_* is manual — save has no scope instance to invent one
from. After that the membership is maintained automatically: each
add_to_*/update_in_* records it in a per-object reverse index tracker — a
hash stored at <object_key>:_idx_scopes that maps
an entry key to the field value written into the index. The key's shape
follows the index's cardinality — unique indexes are 1:1 within a scope, so
the "<scope_config>\t<index_name>\t<scope_id>" triple is the whole identity;
multi indexes are 1:many, so the value is appended and each bucket the object
occupies gets its own entry:
emp1.add_to_company_badge_index(company1)
# unique: { "company\tbadge_index\t<company1 id>" => "12345" }
emp1.add_to_company_dept_index(company1)
# multi: { "company\tdept_index\t<company1 id>\tengineering" => "engineering" }
emp1.destroy!
company1.find_by_badge_number('12345') # => nil
Both save and destroy! read the tracker before opening their transaction
(reads inside MULTI return futures, not values), then replay the writes inside
it, so index maintenance commits atomically with the object hash.
Refresh on save. Changing an indexed field and saving moves the tracked entry — the old value is retracted, so it becomes reusable:
emp1.badge_number = '99999'
emp1.save
company1.find_by_badge_number('12345') # => nil (retracted)
company1.find_by_badge_number('99999') # => emp1
This runs inside save's transaction, where dirty tracking is still live — that
is what supplies the previous value to retract. Only scopes already registered
via add_to_* are refreshed. Uniqueness is validated first, outside the
transaction: if the new value is already taken in that scope, save raises
Familia::RecordExistsError and the index is left untouched, exactly as a
class-level unique_index behaves.
atomic_writedoes not refresh instance-scoped indexes. It calls the shared persistence path from inside a MULTI it has already opened, and the tracker snapshot requires a read taken before the transaction (reads inside MULTI return futures). So scalar fields are written but instance-scoped index entries keep their previous values, silently. Usesavewhen an indexed field changed, or callupdate_in_*explicitly after theatomic_write.multi_index refresh is add-only. A value change adds the identifier to the new bucket but does not remove it from the old one, matching class-level
multi_indexbehavior. The object genuinely is in both buckets, and the tracker records both — sodestroy!still clears every one of them. Callupdate_in_*explicitly when you want the old bucket retracted at the time of the change rather than at destroy.
Cleanup on destroy. The tracker stores the indexed value, not just the membership, so cleanup targets the bucket the entry actually lives in — not whatever the field happens to hold at destroy time:
Employee.new(emp_id: emp1.identifier).destroy! # identifier-only: no field
# values in memory, cleanup
# still finds the bucket
Two constraints follow from cleanup running write-only inside a transaction:
- The scope instance must have an identifier when
add_to_*/update_in_*is called, orFamilia::NoIdentifieris raised. - The scope class must use a Symbol/String
identifier_field, since the scope is rebuilt from its identifier alone during cleanup. A Proc identifier raisesArgumentErrorat index time.
Both are checked before the index write, so a rejected call leaves nothing behind.
delete!does not clean up the tracker — the next save does.delete!removes only the main object hash; related keys — the_idx_scopestracker included — survive, per its documented contract. Tracker entries that outlive the hash can only describe a dead incarnation of the identifier, so when a record is later saved under it (plainsaveorsave_if_not_exists!), the save prunes them instead of replaying them: the previous record's index entries are removed inside the save transaction and the tracker is cleared. The new record starts with no instance-scoped memberships —add_to_*it explicitly where membership is intended. The same reconciliation applies when the hash expired via TTL while the tracker survived. Other related keys (sets, lists, counters) are still inherited as-is; usedestroy!for the full lifecycle. One escape hatch to avoid:atomic_writeperforms no staleness detection (it has no pre-transaction point to probe from), and once it has recreated the hash a latersavecan no longer tell the inherited entries are stale — so make the first write to a reused identifier asave, not anatomic_write.
Generated Methods
On scope class (Company):
| Method | Description |
|--------|-------------|
| find_by_badge_number(badge) | Find within scope |
| index_badge_number_for(emp) | Add to index |
| unindex_badge_number_for(emp) | Remove from index |
On indexed class (Employee):
| Method | Description |
|--------|-------------|
| add_to_company_badge_index(company) | Add to company's index |
| remove_from_company_badge_index(company) | Remove from index |
| in_company_badge_index?(company) | Check if indexed |
Class-Level Multi-Value Indexing
Class-level multi-value indexes group objects by field values at the class level. This is the default behavior when no within: parameter is specified.
class Customer < Familia::Horreum
feature :relationships
field :role
# Class-level multi_index (within: :class is the default)
multi_index :role, :role_index
end
# Create customers with various roles
alice = Customer.create(custid: 'cust_001', role: 'admin')
bob = Customer.create(custid: 'cust_002', role: 'user')
charlie = Customer.create(custid: 'cust_003', role: 'admin')
# Manually add to index (or use auto-indexing via save hooks)
alice.add_to_class_role_index
bob.add_to_class_role_index
charlie.add_to_class_role_index
# Query all customers with a specific role
admins = Customer.find_all_by_role('admin') # => [alice, charlie]
users = Customer.find_all_by_role('user') # => [bob]
# Random sampling
sample = Customer.sample_from_role('admin', 1) # => [random admin]
Redis Key Pattern
Class-level multi-indexes use the pattern: {classname}:{index_name}:{field_value}
Customer.role_index_for('admin').dbkey # => "customer:role_index:admin"
Customer.role_index_for('user').dbkey # => "customer:role_index:user"
Generated Class Methods
| Method | Description |
|---|---|
Customer.role_index_for(value) |
Factory returning Familia::UnsortedSet for the field value |
Customer.find_all_by_role(value) |
Find all objects with that field value |
Customer.sample_from_role(value, count) |
Random sample of objects |
Customer.rebuild_role_index |
Rebuild the entire index from source data |
Generated Instance Methods
| Method | Description |
|---|---|
customer.add_to_class_role_index |
Add this object to its field value's index |
customer.remove_from_class_role_index |
Remove this object from its field value's index |
customer.update_in_class_role_index(old_value) |
Move object from old index to new index |
Update Operations
When a field value changes, use the update method to atomically move the object between indexes:
old_role = customer.role
customer.role = 'superadmin'
customer.update_in_class_role_index(old_role)
# Customer is now in 'superadmin' index, removed from old 'admin' index
Customer.find_all_by_role('superadmin') # => includes customer
Customer.find_all_by_role('admin') # => no longer includes customer
Instance-Scoped Multi-Value Indexing
For indexes scoped to a parent object, use within: to specify the scope class. This allows the same field values across different parent contexts.
class Employee < Familia::Horreum
feature :relationships
field :department
multi_index :department, :dept_index, within: Company
end
company = Company.create(name: 'TechCorp')
# Multiple employees in same department
[
Employee.create(department: 'engineering'),
Employee.create(department: 'engineering'),
Employee.create(department: 'sales')
].each { |emp| emp.add_to_company_dept_index(company) }
# Query all in department
engineers = company.find_all_by_department('engineering') # => [emp1, emp2]
sales_team = company.find_all_by_department('sales') # => [emp3]
# Random sampling
sample = company.sample_from_department('engineering', 1) # => [random engineer]
Generated Methods (Instance-Scoped)
On scope class (Company):
| Method | Description |
|--------|-------------|
| company.dept_index_for(value) | Factory returning UnsortedSet for value |
| company.find_all_by_department(dept) | Find all in department |
| company.sample_from_department(dept, count) | Random sample |
| company.rebuild_dept_index | Rebuild index from participation |
On indexed class (Employee):
| Method | Description |
|--------|-------------|
| employee.add_to_company_dept_index(company) | Add to company's index |
| employee.remove_from_company_dept_index(company) | Remove from index |
| employee.update_in_company_dept_index(company, old_dept) | Move between indexes |
Instance-scoped multi-indexes are tracked, refreshed on save, and cleaned up
on destroy! the same way unique ones are — except that refresh is add-only.
See Automatic Refresh and Cleanup.
Blank field values
A multi_index does not index a blank field value: nil, an empty string or a
string of whitespace only. This holds for class-level and instance-scoped
indexes alike. add_to_* adds the object to no bucket for a blank value,
update_in_* removes it from no bucket for a blank old value and adds it to
none for a blank new value, remove_from_* removes nothing for a blank value,
and the rebuild skips objects whose value is blank. find_all_by_* and
sample_from_* return [] for a blank value, and
Familia::IndexDescriptor#each_record raises ArgumentError for a blank
value:, as it does for nil. The instance-scoped add_to_*
and update_in_* still record the membership for a blank value, so save
indexes the object once its value is no longer blank.
The <index>_for factory names the key of a blank value
<owner key>:<index>, without the trailing delimiter of a bucket, for
example company:c-1:dept_index or customer:role_index. That key is not
exclusive to the index: it is also the key of a related field with the same
name as the index, such as a set :tags on the scope class beside
multi_index :tag, :tags, within: Company, or a class_set :roles beside
multi_index :role, :roles. So the index methods of that scope, or of the
class for a class-level index, do not read, write, delete or audit that key
as theirs. Code that calls <index>_for itself with a blank value gets a set
on that key.
That key can also be a bucket of another index of the scope class:
- When the identifier contains the delimiter, the key can be a bucket of an
instance-scoped index within the scope class in another, shorter scope.
With companies
c-1andc-1:dept_index:x, the keycompany:c-1:dept_index:x:dept_indexof a blank value inc-1:dept_index:xis also the bucket of valuex:dept_indexinc-1. - When the identifier is the name of a class-level index of the scope class,
or starts with that name and the delimiter, the key is a bucket of that
class-level index. This needs no delimiter in the identifier. With a
class-level
multi_index :region, :region_indexonCompany, the keycompany:region_index:dept_indexof a blank value in a companyregion_indexis also theregion_indexbucket of valuedept_index.
audit_multi_indexes of that other index then reads the key as its bucket,
and the rebuild of that index deletes it or replaces its members with the
objects that have that value. This does not hold when the scope class
declares a set field named dept_index, without a dbkey: option of its
own, and the hash key of the company with that identifier holds a hash: the
key is then that record's set field (see
Multi-value index rebuilds).
Before this version, the instance-scoped add_to_* and update_in_* added
objects with a blank value to <scope key>:<index>, such as
company:c-1:dept_index. Such keys stay in place after upgrading. The index
methods of their scope do not read them as data, so they can be left in
place. Familia has no check that tells such a key apart from a related field
of the same name, a live bucket of another scope's index or a live bucket of
a class-level index of the scope class, so inspect a key before removing it
by hand.
When the scope class declares a related field named <index>, such as
set :tags beside multi_index :tag, :tags, within: Company, the
identifiers that earlier versions added to it for objects with a blank value
stay in that field after upgrading. The index wrote each one as the raw
identifier, while a plain set stores each value JSON-encoded. The members
that are identifiers of existing records of the indexed class are the
candidates:
key = Company.load('c-1')..dbkey # => "company:c-1:tags"
Familia.dbclient.smembers(key).select { |m| Employee.exists?(m) }
# => ["e-1", "e-2"]
Familia cannot tell such a member from one the application added itself, so
review the candidates before removing any. An object whose value is no longer
blank can still be among them, and the identifier of a record deleted since
then does not pass the exists? check. Use the scope class's dbclient when
it sets logical_database.
Advanced Patterns
Composite Keys
class ApiKey < Familia::Horreum
field :environment, :key_type
unique_index :environment_and_type, :env_type_index, within: Customer
private
def environment_and_type
"#{environment}:#{key_type}" # e.g., "production:read_write"
end
end
customer.find_by_environment_and_type("production:read_only")
Conditional Indexing
class Document < Familia::Horreum
field :status, :slug
unique_index :slug, :slug_index, within: Project
def add_to_project_slug_index(project)
return unless status == 'published' # Only index published
super
end
end
Time Partitioning
class Event < Familia::Horreum
field :timestamp
multi_index :daily_partition, :daily_events, within: User
private
def daily_partition
Time.at().strftime('%Y%m%d') # e.g., "20241215"
end
end
today = Familia.now.strftime('%Y%m%d')
todays_events = user.find_all_by_daily_partition(today)
Key Differences
Class vs Instance Scoping
Class-level unique (unique_index :email, :email_lookup):
- Automatic indexing on save/destroy
- System-wide uniqueness
- No parent context needed
- Examples: emails, usernames, API keys
Class-level multi (multi_index :role, :role_index):
- Default behavior (no
within:needed) - Groups all objects by field value at class level
- Manual indexing via instance methods
- Examples: roles, categories, statuses
Instance-scoped (unique_index :badge, :badge_index, within: Company):
- Manual indexing required
- Unique within parent only
- Requires parent context
- Examples: employee IDs, project names per team
Instance-scoped multi (multi_index :dept, :dept_index, within: Company):
- Groups objects by field value within parent scope
- Same field value allowed across different parents
- Manual indexing with parent context
- Examples: departments per company, tags per project
Unique vs Multi Indexing
Unique index (unique_index):
- 1:1 field-to-object mapping
- Returns single object or nil
- Enforces uniqueness within scope
Multi index (multi_index):
- 1:many field-to-objects mapping
- Returns array of objects
- Allows duplicate values
- Default: class-level scope (use
within:for instance scope)
Rebuilding Indexes
Indexes can be automatically rebuilt from source data using auto-generated rebuild methods:
# Class-level indexes
User.rebuild_email_lookup # Rebuilds from all User.email values
User.rebuild_username_lookup # Rebuilds from all User.username values
# Instance-scoped indexes
company.rebuild_badge_index # Rebuilds from all Employee.badge_number values
These methods work because indexes are derived data - they're computed from object field values.
Important: Participation data (like
@team.members) cannot be rebuilt automatically because participations represent business decisions, not derived data. See Why Participations Can't Be Rebuilt for the critical distinction between indexes and participations.
When to rebuild indexes:
- After data migrations or bulk imports
- Recovering from index corruption
- Adding indexes to existing data
Concurrency and temporary keys
A rebuild writes into a temporary key and swaps it over the live index, so
readers always see either the old index or the new one. Each rebuild first
takes an advisory lock at <index_key>:rebuild-lock. If another rebuild of the
same index already holds it, the call raises
Familia::RebuildInProgressError immediately; rebuilds never wait or retry,
because a second concurrent rebuild can only duplicate work and risk swapping a
stale snapshot over a newer one. Catch that error if a scheduled job may overlap
with a manual rebuild. A rebuild cannot run inside an enclosing
Familia.transaction or pipeline, because the lock's SET NX would only be
queued there; calling it that way raises Familia::OperationModeError.
The temporary key carries a TTL of 300 seconds
(Familia::AtomicOperations::DEFAULT_REBUILD_TTL), refreshed after each batch,
so a crashed rebuild cleans up after itself. A single batch must therefore
complete within that window. The live index never inherits this TTL. If the
final swap fails, the temporary key is retained for 3600 seconds
(DEFAULT_PRESERVE_TTL) so it can be inspected, then expires.
If a batch overruns the lock TTL, another rebuild can take the lock over. The
lock is refreshed by compare-and-extend, so the overrunning rebuild cannot
reclaim it. It raises Familia::RebuildLockLostError (a subclass of
Familia::RebuildInProgressError) at the next batch boundary or immediately
before the swap, deletes its own temporary key, and leaves the live index
untouched. Reduce batch_size if this happens routinely.
The swap fails closed. It runs as one Lua script that checks the lock still holds this rebuild's token, then renames the temporary key over the live index, so a rebuild whose lock expired or changed hands cannot publish a stale snapshot over its successor's work. If the temporary key is evicted, expires, or is swept between the rebuild and the swap, the rebuild raises rather than reporting success, and the live index keeps its previous contents. A rebuild that returns normally has swapped its full contents into place.
A rebuild is all-or-nothing. If any batch raises, the whole temporary key is discarded, not just the failing batch, and the live index is left as it was. The next rebuild starts from scratch.
Temporary keys written before this behavior existed have no TTL. Remove them with:
# Preview first
Familia::AtomicOperations.sweep_orphaned_temp_keys(Familia.dbclient, dry_run: true)
# Then delete
Familia::AtomicOperations.sweep_orphaned_temp_keys(Familia.dbclient)
The sweep only deletes keys matching *:rebuild:* that have no TTL, are
older than older_than: (3600 seconds by default), and whose index has no
live :rebuild-lock, so a rebuild started by this version is never swept while
it runs. Lock keys do not match the pattern and are never touched.
Rebuilds started by older versions hold no lock and put no TTL on their
temporary key, so the sweep cannot tell one that is still running from one that
was abandoned. Past older_than: it deletes both, and the old code then treats
the missing temporary key as an empty rebuild and deletes the live index. Run
the sweep only after every process is on the new version and no rebuild started
by older code is still running. Do not run it during a rolling upgrade.
Familia.dbclient reaches the default logical database only. For a model that
sets logical_database, run the sweep against that model's dbclient instead.
Multi-value index rebuilds
The rebuild_<index> method of a multi_index, such as
Customer.rebuild_role_index or company.rebuild_dept_index, deletes the
index's per-value bucket sets and then adds every object with a non-blank
field value to the bucket of that value. It does not touch its own key of a
blank value, though the key of a blank value of another scope or index can be
one of its buckets (see Blank field values). It finds
the buckets with a SCAN over the bucket prefix, such as customer:role_index:
or company:c-1:dept_index:, with any glob characters in the prefix escaped,
so a scope identifier such as c-* matches only itself. It deletes only keys
that start with that literal prefix and hold a set. Keys of other types under
the prefix stay in place.
An identifier may contain the delimiter or be an index name, so a set under
the prefix can also be a key of another record of the same class. With
companies c-1 and c-1:dept_index:x, the key
company:c-1:dept_index:x:dept_index:ops is both a bucket of c-1 and the
ops bucket of c-1:dept_index:x. Likewise
customer:role_index:tags is both the bucket of value tags in the
class-level role_index and the tags set field of a customer whose
identifier is role_index. Such a key belongs to the existing record with the
longest identifier, and a class-level bucket has no identifier. So rebuilding
c-1 leaves that key in place, rebuilding c-1:dept_index:x clears it, and
Customer.rebuild_role_index leaves the tags set in place.
audit_multi_indexes reads each such key for the record it belongs to. Only a
set field or multi_index bucket of a record whose hash key holds a hash
counts. Without that hash, the key belongs to the shorter identifier. A bucket
is a set, so the bucket of value object at customer:role_index:object does
not make role_index a record.
The candidate owners of a key are the splits whose identifier is followed by a
set field name that ends the key, or by an instance-scoped index name and the
delimiter. An index name that ends the key, as in the key of a blank value,
makes no candidate. So with companies c-1 and c-1:dept_index:x, the bucket
company:c-1:dept_index:x:dept_index of value x:dept_index belongs to
c-1, unless the company class declares a set field named dept_index.
Likewise, with companies c-1 and c-1:dept_index, the bucket
company:c-1:dept_index:dept_index of value dept_index belongs to c-1
under the same condition. The
rebuild and the audit check at most
Familia::Features::Relationships::Indexing::RecordKeyOwnership::MAX_CANDIDATE_SPLITS
(8) of them per key. The candidates of a batch of keys are checked in
pipelines that each hold at most RecordKeyOwnership::MAX_CANDIDATE_BATCH_BYTES
(1 MiB) of candidate identifiers, and an identifier longer than that gets a
pipeline of its own. A key with more candidates than
MAX_CANDIDATE_SPLITS, such as the bucket of a field value that repeats
:dept_index: many times, is not checked.
Before it deletes any bucket, the rebuild checks every bucket key it is about
to write. If one of them holds a type other than set, belongs to another record
as described above, or has more candidate owners than the rebuild checks, the
rebuild raises Familia::IndexBucketConflictError without deleting or writing
anything. It raises the same way for a stale bucket under the prefix whose
owner it did not check. Such a key may be another record's data, such as a list
of a record whose identifier equals the index name. The error's conflicts
maps each key to the type it holds, or to :too_many_candidate_owners for a key
whose owner was not checked, and its owners maps each key of another record
to that record's hash key. Inspect those keys. The rebuild can run once none of
them holds a type other than set, belongs to another record or has too many
candidate owners.
audit_multi_indexes lists the sets under a bucket prefix that the rebuild
leaves alone in foreign_owned_keys. Each entry has key, field_value,
owner_key and reason, plus scope_id for an instance-scoped index. With
reason: :other_record, owner_key is the hash key of the record that owns
the key; such an entry does not change the audit status. With
reason: :too_many_candidate_owners, owner_key is nil; such an entry sets the
status to :issues_found, and health_check reports the model as unhealthy.
repair_multi_indexes! calls the same rebuild. When it raises
Familia::IndexBucketConflictError, the repair records the index, or for an
instance-scoped index the scope, in :skipped with reason: :bucket_conflict,
conflicts, owners and, for a scope, scope_id, and goes on with the next
scope or index. repair_all! then returns status :partial_failure; its
errors does not list the conflict.
Bucket names in other key patterns
A bucket's key ends with a field value, so it can match a key pattern that
other methods build for other keys. With a customer whose role is notes, the
bucket customer:role_index:notes matches customer:*:notes, the pattern for
the notes field of every customer. With role object, the bucket
customer:role_index:object matches customer:*:object, the pattern for
customer objects. These methods leave such buckets alone:
audit_related_fieldsandaudit_participationslook only at keys of the type the field or collection stores. They also skip a key that is a bucket of a class-levelmulti_indexof the class, or of an instance-scopedmulti_indexwhose scope record exists, unless the hash key of the record the key names holds a hash. They skip a key whose candidate owners were not checked the same way. Sorepair_related_fields!does not delete such a bucket andrepair_participations!does not remove its members, also for atype: :setcollection and with a record whose field value isobject.all,keys_count,scan_count,keys_any?,scan_any?,scan_keys,audit_instances,rebuild_instances, the SCAN fallback of a unique index rebuild and model migrations with the default@scan_patternlook only at keys that hold a hash (Familia::Horreum::OBJECT_KEY_TYPE).audit_multi_indexesdoes not read a class-level bucket whose value contains:<index>:as a bucket of an instance-scoped index, unless the hash key of the scope record the key names holds a hash.
Performance Tips
Bulk Operations
# Efficient bulk indexing
employees.each_slice(100) do |batch|
company.transaction do
batch.each { |emp| emp.add_to_company_dept_index(company) }
end
end
Index Monitoring
# Check index sizes
company.dept_index_engineering.size # Count in engineering
User.email_lookup.size # Total indexed emails
# Index distribution
%w[engineering sales marketing].map { |dept|
[dept, company.send("dept_index_#{dept}").size]
}.to_h
Cleanup
# Remove orphaned entries
company.badge_index.to_h.each do |badge, emp_id|
unless Employee.exists?(emp_id)
company.badge_index.delete(badge)
end
end
Index Storage Format
Index values (the object identifiers stored in hash keys and sets) are raw strings, not JSON-encoded. This is a deliberate design choice shared across all Familia collections that store object references — it ensures that lookups, membership checks, and key construction all operate on the same byte representation. See Collection Member Serialization for the underlying serialization rules.
Redis Key Patterns
| Type | Pattern | Example |
|---|---|---|
| Class unique | {class}:{index_name} |
user:email_lookup |
| Class multi | {class}:{index_name}:{value} |
customer:role_index:admin |
| Instance unique | {scope}:{id}:{index_name} |
company:123:badge_index |
| Instance multi | {scope}:{id}:{index_name}:{value} |
company:123:dept_index:engineering |
Troubleshooting
Common Issues
Query methods not generated:
- Check
query: true(default) or explicitly set - Verify
feature :relationshipsdeclared
Index not updating:
- Class indexes: automatic on save/destroy and the partial writers (see Index Maintenance by Write Path)
- Instance indexes: require an initial manual
add_to_*call on a saved object. After that, refresh onsaveand removal ondestroy!are both automatic (see Automatic Refresh and Cleanup). If amulti_indexvalue changed, the old bucket is retained by design — useupdate_in_*to retract it at change time;destroy!clears it either way.
Familia::PersistenceError from add_to_* / update_in_* (including *_class_* variants):
- The object has never been saved; index writers reject unsaved objects to
prevent dangling entries. Call
savebefore indexing (class-level indexes populate automatically on save).
Familia::OperationModeError from add_to_class_* / update_in_class_* (unique_index only):
- The write is inside a transaction you opened and the record holds no claim
on the exact value being written. That in-MULTI
HSETis only sound as a re-affirmation of a server-side claim; without one it is a blind write that can silently overwrite another record's ownership. Ordinarysavenever hits this — every save path claims first. Two shapes reach it:
# 1. Reassigning the indexed field inside atomic_write. The block runs
# INSIDE the MULTI, after the claim was taken on the OLD value.
user.atomic_write { user.email = 'new@example.com' } # raises
# Fix: set the field before the block, so the claim covers the new value.
user.email = 'new@example.com'
user.atomic_write { user.name = 'Alice' }
# 2. Opening your own transaction after changing the field.
old_email = user.email
user.email = 'new@example.com'
User.transaction { user.update_in_class_email_lookup(old_email) } # raises
# Fix: claim outside the MULTI, then write inside it.
user.claim_unique_email_lookup!
User.transaction { user.update_in_class_email_lookup(old_email) }
The message names whichever mutator ran, so an atomic_write that changed a
field reports update_in_class_* (the path dirty tracking routes to), not
add_to_class_*. update_all_indexes routes there too and raises the same
way. Instance-scoped (within:) unique indexes are not claim-enforced —
inside a transaction they write blindly instead of raising, and the two paths
differ in how loudly: add_to_* says so via Familia.debug, which is silent
unless debug logging is on, while update_in_* skips the claim with no notice
at all.
Inside a pipelined block, add_to_*, update_in_* and claim_unique_*!
raise this error for both kinds of unique index, and the message names the
method you called. A pipeline has no claim verdict to check and no MULTI to
write into, so call these outside the block.
The error propagates out of the transaction block, so the MULTI is discarded
whole: neither the index entry nor any scalar field queued alongside it is
written. Only the stored state is rolled back, though — the in-memory record
keeps the assignments that triggered the error and stays dirty, so a plain
save afterwards is a valid recovery (it claims, then persists the pending
change). Call refresh! instead to discard it and reload what is stored.
- Two neighbouring causes of the same error class, both pre-existing:
saveitself refuses to run inside a transaction, andHashKey#claim_fieldrefuses too — a CAS verdict inside MULTI would come back as aFuture, with nothing left to abort.
Familia::IndexedFieldFastWriteError from multi_field_fast_write / field!:
- The field backs a class-level index (
unique_indexormulti_index), and the write cannot take the out-of-transaction claim that index maintenance requires:multi_field_fast_writenever can (its contract is a single HMSET),field!only when called inside a transaction or pipeline. Nothing is written. Formulti_field_fast_write, usemulti_field_update,save_fields, orsave. Forfield!, call it before or after the transaction or pipeline, where it claims and maintains the index itself;multi_field_update,save_fieldsandsaveraiseFamilia::OperationModeErrorinside the block, so they too belong outside it.
Familia::NoIdentifier / ArgumentError from add_to_* / update_in_*:
- The scope instance has no identifier, uses a Proc
identifier_field, or its identifier contains a tab. In each casedestroy!could not later rebuild the scope and clean the entry up safely, so the write is refused. See Automatic Refresh and Cleanup.
Familia::RecordExistsError from save (instance-scoped index):
- Saving changed an indexed field to a value another record already holds in that scope. Instance-scoped uniqueness is validated on save just as class-level uniqueness is; the index is left untouched.
Duplicate key errors:
- Use
multi_indexfor non-unique values - Consider instance-scoped for contextual uniqueness
Debugging
# Check configuration — returns Array<IndexingRelationship> (Data objects),
# distinguished by .cardinality (:unique vs :multi)
User.indexing_relationships
# => [#<data IndexingRelationship field=:email, index_name=:email_lookup,
# cardinality=:unique, within=nil, ...>]
User.indexing_relationships.select { |r| r.cardinality == :unique }
# Inspect index contents
User.email_lookup.to_h
# => {"alice@example.com" => "user_123", ...}
# Verify membership
employee.in_company_badge_index?(company) # => true/false
user.indexed_in?(:email_lookup) # => true/false (class-level indexes)
For the full introspection API — the IndexingRelationship fields, a
project-wide sweep over Familia.members, per-instance membership state, and
the audit/repair layer — see Introspection.
See Also
- Relationships Overview - Core concepts
- Methods Reference - Complete API
- Participation Guide - Associations