Relationships Indexing Guide

Indexing provides O(1) field-to-object lookups using Redis data structures, enabling fast attribute-based queries without relationship semantics.

Core Concepts

Indexing creates fast lookups for finding objects by field values:

  • O(1) performance - Hash/Set-based constant-time access
  • Automatic management - Class indexes update on every hash-writing path (save, destroy, and the partial writers — see Index Maintenance by Write Path)
  • Flexible scoping - Global or parent-scoped uniqueness
  • Query generation - Automatic find_by_* methods

Index Types

Type Scope Use Case Structure
unique_index Class Global unique fields Redis HashKey
unique_index Instance Parent-scoped unique Redis HashKey
multi_index Class (default) Global non-unique groupings Redis Set per value
multi_index Instance Parent-scoped groupings Redis Set per value

Class-Level Unique Indexing

Global unique field lookups with automatic management:

class User < Familia::Horreum
  feature :relationships
  field :email, :username

  unique_index :email, :email_lookup
  unique_index :username, :username_lookup
end

# Automatic indexing on save
user = User.create(email: 'alice@example.com')
User.find_by_email('alice@example.com')  # => user (O(1) lookup)

# Automatic update on field change
user.update(email: 'alice.smith@example.com')
User.find_by_email('alice.smith@example.com')  # => user

# Automatic cleanup on destroy
user.destroy
User.find_by_email('alice.smith@example.com')  # => nil

Uniqueness is enforced server-side, before the transaction opens. save claims the value with a single-key Lua compare-and-set, so two concurrent saves of the same value cannot both see it as free — the loser raises Familia::RecordExistsError carrying existing_id. The HSET inside save's MULTI only re-affirms a claim the record already holds. Every save path (save, save_if_not_exists!, atomic_write, Familia.atomic_write) claims automatically; a transaction you open yourself does not, so writing an index from inside one needs an explicit claim_unique_<index>! first — see Familia::OperationModeError under Troubleshooting and ADR-0002.

Index Maintenance by Write Path

Every path that writes the object hash either maintains class-level indexes (unique and multi alike) fail-closed — the claim precedes the hash write, so a Familia::RecordExistsError leaves the hash untouched — or refuses the write outright rather than leave lookups stale (#308):

Write path Class-level index maintenance
save / save! / save_if_not_exists! / create! All class-level indexes: claim before the MULTI, update inside it
atomic_write / Familia.atomic_write All class-level indexes (claims the values held when the block opens — see Troubleshooting)
commit_fields All class-level indexes (writes the full hash)
save_fields(:a, :b) Indexes on the named fields only
multi_field_update(a: 1) Indexes on the written fields only
multi_field_fast_write(a: 1) Refused: raises Familia::IndexedFieldFastWriteError if any written field backs a class-level index — its single-HMSET contract leaves no room for the out-of-transaction claim
field!(value) fast writer Maintained (claim, then index update, then HSET) — but NOT as one MULTI; see below. Inside a transaction or pipeline it raises Familia::IndexedFieldFastWriteError instead
destroy! All class-level index entries removed

For multi_index, "maintained" means add-only: a value change adds the identifier to the new bucket without retracting the old one, same as on save. Call update_in_class_* with the old value to retract it.

Unlike save and multi_field_update, the field! fast writer does not wrap its index update and hash write in a single MULTI — they are separate commands (the fast writer's contract is a single hash command on the happy path). If the HSET fails after the index entries were already updated (e.g. a dropped connection), the writer compensates best-effort before re-raising the original error: it restores the in-memory value, releases the just-claimed entry, and restores the previous value's entry. A failure during compensation is logged via Familia.warn and never masks the original error, so in the worst case a stale index entry remains until the next save.

Generated Methods

Method Description
User.find_by_email(email) O(1) lookup
User.index_email_for(user) Manual index
User.unindex_email_for(user) Remove from index
User.reindex_email_for(user) Update index

Instance-Scoped Unique Indexing

Unique within parent context, allowing duplicates across parents:

class Employee < Familia::Horreum
  feature :relationships
  field :badge_number

  unique_index :badge_number, :badge_index, within: Company
end

# Manual indexing required (needs parent context)
company1 = Company.create(name: 'Acme Corp')
company2 = Company.create(name: 'Beta Inc')

emp1 = Employee.create(badge_number: '12345')
emp1.add_to_company_badge_index(company1)

emp2 = Employee.create(badge_number: '12345')  # Same badge OK
emp2.add_to_company_badge_index(company2)

# Scoped lookups
company1.find_by_badge_number('12345')  # => emp1
company2.find_by_badge_number('12345')  # => emp2

Note: The indexed object must be persisted before it can be added to an index. add_to_*/update_in_* (instance-scoped) and add_to_class_*/update_in_class_* (class-level) raise Familia::PersistenceError for unsaved objects, since the index entry would point at a record that does not exist yet. Call save first — for class-level indexes, save populates the index automatically anyway.

Automatic Refresh and Cleanup

The first add_to_* is manual — save has no scope instance to invent one from. After that the membership is maintained automatically: each add_to_*/update_in_* records it in a per-object reverse index tracker — a hash stored at <object_key>:_idx_scopes that maps an entry key to the field value written into the index. The key's shape follows the index's cardinality — unique indexes are 1:1 within a scope, so the "<scope_config>\t<index_name>\t<scope_id>" triple is the whole identity; multi indexes are 1:many, so the value is appended and each bucket the object occupies gets its own entry:

emp1.add_to_company_badge_index(company1)
# unique: { "company\tbadge_index\t<company1 id>" => "12345" }

emp1.add_to_company_dept_index(company1)
# multi:  { "company\tdept_index\t<company1 id>\tengineering" => "engineering" }

emp1.destroy!
company1.find_by_badge_number('12345')  # => nil

Both save and destroy! read the tracker before opening their transaction (reads inside MULTI return futures, not values), then replay the writes inside it, so index maintenance commits atomically with the object hash.

Refresh on save. Changing an indexed field and saving moves the tracked entry — the old value is retracted, so it becomes reusable:

emp1.badge_number = '99999'
emp1.save

company1.find_by_badge_number('12345')  # => nil  (retracted)
company1.find_by_badge_number('99999')  # => emp1

This runs inside save's transaction, where dirty tracking is still live — that is what supplies the previous value to retract. Only scopes already registered via add_to_* are refreshed. Uniqueness is validated first, outside the transaction: if the new value is already taken in that scope, save raises Familia::RecordExistsError and the index is left untouched, exactly as a class-level unique_index behaves.

atomic_write does not refresh instance-scoped indexes. It calls the shared persistence path from inside a MULTI it has already opened, and the tracker snapshot requires a read taken before the transaction (reads inside MULTI return futures). So scalar fields are written but instance-scoped index entries keep their previous values, silently. Use save when an indexed field changed, or call update_in_* explicitly after the atomic_write.

multi_index refresh is add-only. A value change adds the identifier to the new bucket but does not remove it from the old one, matching class-level multi_index behavior. The object genuinely is in both buckets, and the tracker records both — so destroy! still clears every one of them. Call update_in_* explicitly when you want the old bucket retracted at the time of the change rather than at destroy.

Cleanup on destroy. The tracker stores the indexed value, not just the membership, so cleanup targets the bucket the entry actually lives in — not whatever the field happens to hold at destroy time:

Employee.new(emp_id: emp1.identifier).destroy!  # identifier-only: no field
                                                # values in memory, cleanup
                                                # still finds the bucket

Two constraints follow from cleanup running write-only inside a transaction:

  • The scope instance must have an identifier when add_to_*/update_in_* is called, or Familia::NoIdentifier is raised.
  • The scope class must use a Symbol/String identifier_field, since the scope is rebuilt from its identifier alone during cleanup. A Proc identifier raises ArgumentError at index time.

Both are checked before the index write, so a rejected call leaves nothing behind.

delete! does not clean up the tracker — the next save does. delete! removes only the main object hash; related keys — the _idx_scopes tracker included — survive, per its documented contract. Tracker entries that outlive the hash can only describe a dead incarnation of the identifier, so when a record is later saved under it (plain save or save_if_not_exists!), the save prunes them instead of replaying them: the previous record's index entries are removed inside the save transaction and the tracker is cleared. The new record starts with no instance-scoped memberships — add_to_* it explicitly where membership is intended. The same reconciliation applies when the hash expired via TTL while the tracker survived. Other related keys (sets, lists, counters) are still inherited as-is; use destroy! for the full lifecycle. One escape hatch to avoid: atomic_write performs no staleness detection (it has no pre-transaction point to probe from), and once it has recreated the hash a later save can no longer tell the inherited entries are stale — so make the first write to a reused identifier a save, not an atomic_write.

Generated Methods

On scope class (Company): | Method | Description | |--------|-------------| | find_by_badge_number(badge) | Find within scope | | index_badge_number_for(emp) | Add to index | | unindex_badge_number_for(emp) | Remove from index |

On indexed class (Employee): | Method | Description | |--------|-------------| | add_to_company_badge_index(company) | Add to company's index | | remove_from_company_badge_index(company) | Remove from index | | in_company_badge_index?(company) | Check if indexed |

Class-Level Multi-Value Indexing

Class-level multi-value indexes group objects by field values at the class level. This is the default behavior when no within: parameter is specified.

class Customer < Familia::Horreum
  feature :relationships
  field :role

  # Class-level multi_index (within: :class is the default)
  multi_index :role, :role_index
end

# Create customers with various roles
alice = Customer.create(custid: 'cust_001', role: 'admin')
bob = Customer.create(custid: 'cust_002', role: 'user')
charlie = Customer.create(custid: 'cust_003', role: 'admin')

# Manually add to index (or use auto-indexing via save hooks)
alice.add_to_class_role_index
bob.add_to_class_role_index
charlie.add_to_class_role_index

# Query all customers with a specific role
admins = Customer.find_all_by_role('admin')  # => [alice, charlie]
users = Customer.find_all_by_role('user')    # => [bob]

# Random sampling
sample = Customer.sample_from_role('admin', 1)  # => [random admin]

Redis Key Pattern

Class-level multi-indexes use the pattern: {classname}:{index_name}:{field_value}

Customer.role_index_for('admin').dbkey  # => "customer:role_index:admin"
Customer.role_index_for('user').dbkey   # => "customer:role_index:user"

Generated Class Methods

Method Description
Customer.role_index_for(value) Factory returning Familia::UnsortedSet for the field value
Customer.find_all_by_role(value) Find all objects with that field value
Customer.sample_from_role(value, count) Random sample of objects
Customer.rebuild_role_index Rebuild the entire index from source data

Generated Instance Methods

Method Description
customer.add_to_class_role_index Add this object to its field value's index
customer.remove_from_class_role_index Remove this object from its field value's index
customer.update_in_class_role_index(old_value) Move object from old index to new index

Update Operations

When a field value changes, use the update method to atomically move the object between indexes:

old_role = customer.role
customer.role = 'superadmin'
customer.update_in_class_role_index(old_role)

# Customer is now in 'superadmin' index, removed from old 'admin' index
Customer.find_all_by_role('superadmin')  # => includes customer
Customer.find_all_by_role('admin')       # => no longer includes customer

Instance-Scoped Multi-Value Indexing

For indexes scoped to a parent object, use within: to specify the scope class. This allows the same field values across different parent contexts.

class Employee < Familia::Horreum
  feature :relationships
  field :department

  multi_index :department, :dept_index, within: Company
end

company = Company.create(name: 'TechCorp')

# Multiple employees in same department
[
  Employee.create(department: 'engineering'),
  Employee.create(department: 'engineering'),
  Employee.create(department: 'sales')
].each { |emp| emp.add_to_company_dept_index(company) }

# Query all in department
engineers = company.find_all_by_department('engineering')  # => [emp1, emp2]
sales_team = company.find_all_by_department('sales')       # => [emp3]

# Random sampling
sample = company.sample_from_department('engineering', 1)  # => [random engineer]

Generated Methods (Instance-Scoped)

On scope class (Company): | Method | Description | |--------|-------------| | company.dept_index_for(value) | Factory returning UnsortedSet for value | | company.find_all_by_department(dept) | Find all in department | | company.sample_from_department(dept, count) | Random sample | | company.rebuild_dept_index | Rebuild index from participation |

On indexed class (Employee): | Method | Description | |--------|-------------| | employee.add_to_company_dept_index(company) | Add to company's index | | employee.remove_from_company_dept_index(company) | Remove from index | | employee.update_in_company_dept_index(company, old_dept) | Move between indexes |

Instance-scoped multi-indexes are tracked, refreshed on save, and cleaned up on destroy! the same way unique ones are — except that refresh is add-only. See Automatic Refresh and Cleanup.

Blank field values

A multi_index does not index a blank field value: nil, an empty string or a string of whitespace only. This holds for class-level and instance-scoped indexes alike. add_to_* adds the object to no bucket for a blank value, update_in_* removes it from no bucket for a blank old value and adds it to none for a blank new value, remove_from_* removes nothing for a blank value, and the rebuild skips objects whose value is blank. find_all_by_* and sample_from_* return [] for a blank value, and Familia::IndexDescriptor#each_record raises ArgumentError for a blank value:, as it does for nil. The instance-scoped add_to_* and update_in_* still record the membership for a blank value, so save indexes the object once its value is no longer blank.

The <index>_for factory names the key of a blank value <owner key>:<index>, without the trailing delimiter of a bucket, for example company:c-1:dept_index or customer:role_index. That key is not exclusive to the index: it is also the key of a related field with the same name as the index, such as a set :tags on the scope class beside multi_index :tag, :tags, within: Company, or a class_set :roles beside multi_index :role, :roles. So the index methods of that scope, or of the class for a class-level index, do not read, write, delete or audit that key as theirs. Code that calls <index>_for itself with a blank value gets a set on that key.

That key can also be a bucket of another index of the scope class:

  • When the identifier contains the delimiter, the key can be a bucket of an instance-scoped index within the scope class in another, shorter scope. With companies c-1 and c-1:dept_index:x, the key company:c-1:dept_index:x:dept_index of a blank value in c-1:dept_index:x is also the bucket of value x:dept_index in c-1.
  • When the identifier is the name of a class-level index of the scope class, or starts with that name and the delimiter, the key is a bucket of that class-level index. This needs no delimiter in the identifier. With a class-level multi_index :region, :region_index on Company, the key company:region_index:dept_index of a blank value in a company region_index is also the region_index bucket of value dept_index.

audit_multi_indexes of that other index then reads the key as its bucket, and the rebuild of that index deletes it or replaces its members with the objects that have that value. This does not hold when the scope class declares a set field named dept_index, without a dbkey: option of its own, and the hash key of the company with that identifier holds a hash: the key is then that record's set field (see Multi-value index rebuilds).

Before this version, the instance-scoped add_to_* and update_in_* added objects with a blank value to <scope key>:<index>, such as company:c-1:dept_index. Such keys stay in place after upgrading. The index methods of their scope do not read them as data, so they can be left in place. Familia has no check that tells such a key apart from a related field of the same name, a live bucket of another scope's index or a live bucket of a class-level index of the scope class, so inspect a key before removing it by hand.

When the scope class declares a related field named <index>, such as set :tags beside multi_index :tag, :tags, within: Company, the identifiers that earlier versions added to it for objects with a blank value stay in that field after upgrading. The index wrote each one as the raw identifier, while a plain set stores each value JSON-encoded. The members that are identifiers of existing records of the indexed class are the candidates:

key = Company.load('c-1').tags.dbkey   # => "company:c-1:tags"
Familia.dbclient.smembers(key).select { |m| Employee.exists?(m) }
# => ["e-1", "e-2"]

Familia cannot tell such a member from one the application added itself, so review the candidates before removing any. An object whose value is no longer blank can still be among them, and the identifier of a record deleted since then does not pass the exists? check. Use the scope class's dbclient when it sets logical_database.

Advanced Patterns

Composite Keys

class ApiKey < Familia::Horreum
  field :environment, :key_type

  unique_index :environment_and_type, :env_type_index, within: Customer

  private
  def environment_and_type
    "#{environment}:#{key_type}"  # e.g., "production:read_write"
  end
end

customer.find_by_environment_and_type("production:read_only")

Conditional Indexing

class Document < Familia::Horreum
  field :status, :slug

  unique_index :slug, :slug_index, within: Project

  def add_to_project_slug_index(project)
    return unless status == 'published'  # Only index published
    super
  end
end

Time Partitioning

class Event < Familia::Horreum
  field :timestamp

  multi_index :daily_partition, :daily_events, within: User

  private
  def daily_partition
    Time.at(timestamp).strftime('%Y%m%d')  # e.g., "20241215"
  end
end

today = Familia.now.strftime('%Y%m%d')
todays_events = user.find_all_by_daily_partition(today)

Key Differences

Class vs Instance Scoping

Class-level unique (unique_index :email, :email_lookup):

  • Automatic indexing on save/destroy
  • System-wide uniqueness
  • No parent context needed
  • Examples: emails, usernames, API keys

Class-level multi (multi_index :role, :role_index):

  • Default behavior (no within: needed)
  • Groups all objects by field value at class level
  • Manual indexing via instance methods
  • Examples: roles, categories, statuses

Instance-scoped (unique_index :badge, :badge_index, within: Company):

  • Manual indexing required
  • Unique within parent only
  • Requires parent context
  • Examples: employee IDs, project names per team

Instance-scoped multi (multi_index :dept, :dept_index, within: Company):

  • Groups objects by field value within parent scope
  • Same field value allowed across different parents
  • Manual indexing with parent context
  • Examples: departments per company, tags per project

Unique vs Multi Indexing

Unique index (unique_index):

  • 1:1 field-to-object mapping
  • Returns single object or nil
  • Enforces uniqueness within scope

Multi index (multi_index):

  • 1:many field-to-objects mapping
  • Returns array of objects
  • Allows duplicate values
  • Default: class-level scope (use within: for instance scope)

Rebuilding Indexes

Indexes can be automatically rebuilt from source data using auto-generated rebuild methods:

# Class-level indexes
User.rebuild_email_lookup      # Rebuilds from all User.email values
User.rebuild_username_lookup   # Rebuilds from all User.username values

# Instance-scoped indexes
company.rebuild_badge_index    # Rebuilds from all Employee.badge_number values

These methods work because indexes are derived data - they're computed from object field values.

Important: Participation data (like @team.members) cannot be rebuilt automatically because participations represent business decisions, not derived data. See Why Participations Can't Be Rebuilt for the critical distinction between indexes and participations.

When to rebuild indexes:

  • After data migrations or bulk imports
  • Recovering from index corruption
  • Adding indexes to existing data

Concurrency and temporary keys

A rebuild writes into a temporary key and swaps it over the live index, so readers always see either the old index or the new one. Each rebuild first takes an advisory lock at <index_key>:rebuild-lock. If another rebuild of the same index already holds it, the call raises Familia::RebuildInProgressError immediately; rebuilds never wait or retry, because a second concurrent rebuild can only duplicate work and risk swapping a stale snapshot over a newer one. Catch that error if a scheduled job may overlap with a manual rebuild. A rebuild cannot run inside an enclosing Familia.transaction or pipeline, because the lock's SET NX would only be queued there; calling it that way raises Familia::OperationModeError.

The temporary key carries a TTL of 300 seconds (Familia::AtomicOperations::DEFAULT_REBUILD_TTL), refreshed after each batch, so a crashed rebuild cleans up after itself. A single batch must therefore complete within that window. The live index never inherits this TTL. If the final swap fails, the temporary key is retained for 3600 seconds (DEFAULT_PRESERVE_TTL) so it can be inspected, then expires.

If a batch overruns the lock TTL, another rebuild can take the lock over. The lock is refreshed by compare-and-extend, so the overrunning rebuild cannot reclaim it. It raises Familia::RebuildLockLostError (a subclass of Familia::RebuildInProgressError) at the next batch boundary or immediately before the swap, deletes its own temporary key, and leaves the live index untouched. Reduce batch_size if this happens routinely.

The swap fails closed. It runs as one Lua script that checks the lock still holds this rebuild's token, then renames the temporary key over the live index, so a rebuild whose lock expired or changed hands cannot publish a stale snapshot over its successor's work. If the temporary key is evicted, expires, or is swept between the rebuild and the swap, the rebuild raises rather than reporting success, and the live index keeps its previous contents. A rebuild that returns normally has swapped its full contents into place.

A rebuild is all-or-nothing. If any batch raises, the whole temporary key is discarded, not just the failing batch, and the live index is left as it was. The next rebuild starts from scratch.

Temporary keys written before this behavior existed have no TTL. Remove them with:

# Preview first
Familia::AtomicOperations.sweep_orphaned_temp_keys(Familia.dbclient, dry_run: true)

# Then delete
Familia::AtomicOperations.sweep_orphaned_temp_keys(Familia.dbclient)

The sweep only deletes keys matching *:rebuild:* that have no TTL, are older than older_than: (3600 seconds by default), and whose index has no live :rebuild-lock, so a rebuild started by this version is never swept while it runs. Lock keys do not match the pattern and are never touched.

Rebuilds started by older versions hold no lock and put no TTL on their temporary key, so the sweep cannot tell one that is still running from one that was abandoned. Past older_than: it deletes both, and the old code then treats the missing temporary key as an empty rebuild and deletes the live index. Run the sweep only after every process is on the new version and no rebuild started by older code is still running. Do not run it during a rolling upgrade.

Familia.dbclient reaches the default logical database only. For a model that sets logical_database, run the sweep against that model's dbclient instead.

Multi-value index rebuilds

The rebuild_<index> method of a multi_index, such as Customer.rebuild_role_index or company.rebuild_dept_index, deletes the index's per-value bucket sets and then adds every object with a non-blank field value to the bucket of that value. It does not touch its own key of a blank value, though the key of a blank value of another scope or index can be one of its buckets (see Blank field values). It finds the buckets with a SCAN over the bucket prefix, such as customer:role_index: or company:c-1:dept_index:, with any glob characters in the prefix escaped, so a scope identifier such as c-* matches only itself. It deletes only keys that start with that literal prefix and hold a set. Keys of other types under the prefix stay in place.

An identifier may contain the delimiter or be an index name, so a set under the prefix can also be a key of another record of the same class. With companies c-1 and c-1:dept_index:x, the key company:c-1:dept_index:x:dept_index:ops is both a bucket of c-1 and the ops bucket of c-1:dept_index:x. Likewise customer:role_index:tags is both the bucket of value tags in the class-level role_index and the tags set field of a customer whose identifier is role_index. Such a key belongs to the existing record with the longest identifier, and a class-level bucket has no identifier. So rebuilding c-1 leaves that key in place, rebuilding c-1:dept_index:x clears it, and Customer.rebuild_role_index leaves the tags set in place. audit_multi_indexes reads each such key for the record it belongs to. Only a set field or multi_index bucket of a record whose hash key holds a hash counts. Without that hash, the key belongs to the shorter identifier. A bucket is a set, so the bucket of value object at customer:role_index:object does not make role_index a record.

The candidate owners of a key are the splits whose identifier is followed by a set field name that ends the key, or by an instance-scoped index name and the delimiter. An index name that ends the key, as in the key of a blank value, makes no candidate. So with companies c-1 and c-1:dept_index:x, the bucket company:c-1:dept_index:x:dept_index of value x:dept_index belongs to c-1, unless the company class declares a set field named dept_index. Likewise, with companies c-1 and c-1:dept_index, the bucket company:c-1:dept_index:dept_index of value dept_index belongs to c-1 under the same condition. The rebuild and the audit check at most Familia::Features::Relationships::Indexing::RecordKeyOwnership::MAX_CANDIDATE_SPLITS (8) of them per key. The candidates of a batch of keys are checked in pipelines that each hold at most RecordKeyOwnership::MAX_CANDIDATE_BATCH_BYTES (1 MiB) of candidate identifiers, and an identifier longer than that gets a pipeline of its own. A key with more candidates than MAX_CANDIDATE_SPLITS, such as the bucket of a field value that repeats :dept_index: many times, is not checked.

Before it deletes any bucket, the rebuild checks every bucket key it is about to write. If one of them holds a type other than set, belongs to another record as described above, or has more candidate owners than the rebuild checks, the rebuild raises Familia::IndexBucketConflictError without deleting or writing anything. It raises the same way for a stale bucket under the prefix whose owner it did not check. Such a key may be another record's data, such as a list of a record whose identifier equals the index name. The error's conflicts maps each key to the type it holds, or to :too_many_candidate_owners for a key whose owner was not checked, and its owners maps each key of another record to that record's hash key. Inspect those keys. The rebuild can run once none of them holds a type other than set, belongs to another record or has too many candidate owners.

audit_multi_indexes lists the sets under a bucket prefix that the rebuild leaves alone in foreign_owned_keys. Each entry has key, field_value, owner_key and reason, plus scope_id for an instance-scoped index. With reason: :other_record, owner_key is the hash key of the record that owns the key; such an entry does not change the audit status. With reason: :too_many_candidate_owners, owner_key is nil; such an entry sets the status to :issues_found, and health_check reports the model as unhealthy.

repair_multi_indexes! calls the same rebuild. When it raises Familia::IndexBucketConflictError, the repair records the index, or for an instance-scoped index the scope, in :skipped with reason: :bucket_conflict, conflicts, owners and, for a scope, scope_id, and goes on with the next scope or index. repair_all! then returns status :partial_failure; its errors does not list the conflict.

Bucket names in other key patterns

A bucket's key ends with a field value, so it can match a key pattern that other methods build for other keys. With a customer whose role is notes, the bucket customer:role_index:notes matches customer:*:notes, the pattern for the notes field of every customer. With role object, the bucket customer:role_index:object matches customer:*:object, the pattern for customer objects. These methods leave such buckets alone:

  • audit_related_fields and audit_participations look only at keys of the type the field or collection stores. They also skip a key that is a bucket of a class-level multi_index of the class, or of an instance-scoped multi_index whose scope record exists, unless the hash key of the record the key names holds a hash. They skip a key whose candidate owners were not checked the same way. So repair_related_fields! does not delete such a bucket and repair_participations! does not remove its members, also for a type: :set collection and with a record whose field value is object.
  • all, keys_count, scan_count, keys_any?, scan_any?, scan_keys, audit_instances, rebuild_instances, the SCAN fallback of a unique index rebuild and model migrations with the default @scan_pattern look only at keys that hold a hash (Familia::Horreum::OBJECT_KEY_TYPE).
  • audit_multi_indexes does not read a class-level bucket whose value contains :<index>: as a bucket of an instance-scoped index, unless the hash key of the scope record the key names holds a hash.

Performance Tips

Bulk Operations

# Efficient bulk indexing
employees.each_slice(100) do |batch|
  company.transaction do
    batch.each { |emp| emp.add_to_company_dept_index(company) }
  end
end

Index Monitoring

# Check index sizes
company.dept_index_engineering.size  # Count in engineering
User.email_lookup.size               # Total indexed emails

# Index distribution
%w[engineering sales marketing].map { |dept|
  [dept, company.send("dept_index_#{dept}").size]
}.to_h

Cleanup

# Remove orphaned entries
company.badge_index.to_h.each do |badge, emp_id|
  unless Employee.exists?(emp_id)
    company.badge_index.delete(badge)
  end
end

Index Storage Format

Index values (the object identifiers stored in hash keys and sets) are raw strings, not JSON-encoded. This is a deliberate design choice shared across all Familia collections that store object references — it ensures that lookups, membership checks, and key construction all operate on the same byte representation. See Collection Member Serialization for the underlying serialization rules.

Redis Key Patterns

Type Pattern Example
Class unique {class}:{index_name} user:email_lookup
Class multi {class}:{index_name}:{value} customer:role_index:admin
Instance unique {scope}:{id}:{index_name} company:123:badge_index
Instance multi {scope}:{id}:{index_name}:{value} company:123:dept_index:engineering

Troubleshooting

Common Issues

Query methods not generated:

  • Check query: true (default) or explicitly set
  • Verify feature :relationships declared

Index not updating:

  • Class indexes: automatic on save/destroy and the partial writers (see Index Maintenance by Write Path)
  • Instance indexes: require an initial manual add_to_* call on a saved object. After that, refresh on save and removal on destroy! are both automatic (see Automatic Refresh and Cleanup). If a multi_index value changed, the old bucket is retained by design — use update_in_* to retract it at change time; destroy! clears it either way.

Familia::PersistenceError from add_to_* / update_in_* (including *_class_* variants):

  • The object has never been saved; index writers reject unsaved objects to prevent dangling entries. Call save before indexing (class-level indexes populate automatically on save).

Familia::OperationModeError from add_to_class_* / update_in_class_* (unique_index only):

  • The write is inside a transaction you opened and the record holds no claim on the exact value being written. That in-MULTI HSET is only sound as a re-affirmation of a server-side claim; without one it is a blind write that can silently overwrite another record's ownership. Ordinary save never hits this — every save path claims first. Two shapes reach it:
  # 1. Reassigning the indexed field inside atomic_write. The block runs
  #    INSIDE the MULTI, after the claim was taken on the OLD value.
  user.atomic_write { user.email = 'new@example.com' }   # raises

  # Fix: set the field before the block, so the claim covers the new value.
  user.email = 'new@example.com'
  user.atomic_write { user.name = 'Alice' }
  # 2. Opening your own transaction after changing the field.
  old_email = user.email
  user.email = 'new@example.com'
  User.transaction { user.update_in_class_email_lookup(old_email) }  # raises

  # Fix: claim outside the MULTI, then write inside it.
  user.claim_unique_email_lookup!
  User.transaction { user.update_in_class_email_lookup(old_email) }

The message names whichever mutator ran, so an atomic_write that changed a field reports update_in_class_* (the path dirty tracking routes to), not add_to_class_*. update_all_indexes routes there too and raises the same way. Instance-scoped (within:) unique indexes are not claim-enforced — inside a transaction they write blindly instead of raising, and the two paths differ in how loudly: add_to_* says so via Familia.debug, which is silent unless debug logging is on, while update_in_* skips the claim with no notice at all.

Inside a pipelined block, add_to_*, update_in_* and claim_unique_*! raise this error for both kinds of unique index, and the message names the method you called. A pipeline has no claim verdict to check and no MULTI to write into, so call these outside the block.

The error propagates out of the transaction block, so the MULTI is discarded whole: neither the index entry nor any scalar field queued alongside it is written. Only the stored state is rolled back, though — the in-memory record keeps the assignments that triggered the error and stays dirty, so a plain save afterwards is a valid recovery (it claims, then persists the pending change). Call refresh! instead to discard it and reload what is stored.

  • Two neighbouring causes of the same error class, both pre-existing: save itself refuses to run inside a transaction, and HashKey#claim_field refuses too — a CAS verdict inside MULTI would come back as a Future, with nothing left to abort.

Familia::IndexedFieldFastWriteError from multi_field_fast_write / field!:

  • The field backs a class-level index (unique_index or multi_index), and the write cannot take the out-of-transaction claim that index maintenance requires: multi_field_fast_write never can (its contract is a single HMSET), field! only when called inside a transaction or pipeline. Nothing is written. For multi_field_fast_write, use multi_field_update, save_fields, or save. For field!, call it before or after the transaction or pipeline, where it claims and maintains the index itself; multi_field_update, save_fields and save raise Familia::OperationModeError inside the block, so they too belong outside it.

Familia::NoIdentifier / ArgumentError from add_to_* / update_in_*:

  • The scope instance has no identifier, uses a Proc identifier_field, or its identifier contains a tab. In each case destroy! could not later rebuild the scope and clean the entry up safely, so the write is refused. See Automatic Refresh and Cleanup.

Familia::RecordExistsError from save (instance-scoped index):

  • Saving changed an indexed field to a value another record already holds in that scope. Instance-scoped uniqueness is validated on save just as class-level uniqueness is; the index is left untouched.

Duplicate key errors:

  • Use multi_index for non-unique values
  • Consider instance-scoped for contextual uniqueness

Debugging

# Check configuration — returns Array<IndexingRelationship> (Data objects),
# distinguished by .cardinality (:unique vs :multi)
User.indexing_relationships
# => [#<data IndexingRelationship field=:email, index_name=:email_lookup,
#            cardinality=:unique, within=nil, ...>]
User.indexing_relationships.select { |r| r.cardinality == :unique }

# Inspect index contents
User.email_lookup.to_h
# => {"alice@example.com" => "user_123", ...}

# Verify membership
employee.in_company_badge_index?(company)  # => true/false
user.indexed_in?(:email_lookup)            # => true/false (class-level indexes)

For the full introspection API — the IndexingRelationship fields, a project-wide sweep over Familia.members, per-instance membership state, and the audit/repair layer — see Introspection.

See Also