Module: Familia::Horreum::AuditMethods
- Included in:
- ManagementMethods
- Defined in:
- lib/familia/horreum/management/audit.rb
Overview
AuditMethods provides proactive consistency detection for Horreum models.
Included in ManagementMethods so every Horreum subclass gets these as class methods (e.g. Customer.audit_instances, Customer.health_check).
Constant Summary collapse
- PARTICIPATION_KEY_TYPES =
Redis type of the collection key of each participation type, for SCAN's TYPE option.
{ sorted_set: 'zset', set: 'set', list: 'list' }.freeze
Instance Method Summary collapse
-
#audit_cross_references(batch_size: 100) {|Hash| ... } ⇒ Hash
Audits drift between the
instancesZSET and class-level unique indexes that per-registry audits cannot surface alone. -
#audit_instances(batch_size: 100) {|Hash| ... } ⇒ Hash
Compares the instances timeline against actual DB keys via SCAN.
-
#audit_multi_indexes(scanned_identifiers: nil, loaded_objects: nil) ⇒ Array<Hash>
Audits all multi indexes.
-
#audit_participations(sample_size: nil) ⇒ Array<Hash>
Audits participation collections for stale members.
-
#audit_related_fields ⇒ Array<Hash>
Audits instance-level related_fields (list/set/zset/hashkey) for orphaned collection keys whose parent Horreum hash no longer exists.
-
#audit_unique_indexes(scanned_identifiers: nil, loaded_objects: nil) ⇒ Array<Hash>
Audits all unique indexes (class-level only, where within is nil).
-
#health_check(batch_size: 100, sample_size: nil, audit_collections: false, check_cross_refs: false) {|Hash| ... } ⇒ AuditReport
Runs all audits and wraps results in an AuditReport.
Instance Method Details
#audit_cross_references(batch_size: 100) {|Hash| ... } ⇒ Hash
Audits drift between the instances ZSET and class-level unique
indexes that per-registry audits cannot surface alone.
For every live identifier in instances, verifies that each
class-level unique index has an entry keyed by the object's current
field value and that entry points back to the same identifier.
Two failure modes are detected:
in_instances_missing_unique_index: live object has a populated indexed field but no corresponding entry exists in the index.index_points_to_wrong_identifier: entry exists but references a different identifier (split-brain between two objects).
Scope is limited to class-level unique indexes (within nil or
:class). Multi-indexes are covered by audit_multi_indexes;
instance-scoped unique indexes are out of scope for this audit.
183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 |
# File 'lib/familia/horreum/management/audit.rb', line 183 def audit_cross_references(batch_size: 100, &progress) Familia.assert_replies_available!("#{self}.audit_cross_references") empty_result = { in_instances_missing_unique_index: [], index_points_to_wrong_identifier: [], status: :ok, } return empty_result unless respond_to?(:indexing_relationships) class_unique_rels = indexing_relationships.select do |rel| rel.cardinality == :unique && (rel.within.nil? || rel.within == :class) end return empty_result if class_unique_rels.empty? instance_ids = instances.members total = instance_ids.size processed = 0 in_instances_missing_unique_index = [] index_points_to_wrong_identifier = [] instance_ids.each_slice(batch_size) do |batch| objects = load_multi(batch) processed += batch.size # Per unique index, collect (identifier, field_value) pairs from live # objects in this batch and resolve them with a single HMGET round # trip instead of one HGET per (object x index) combination. class_unique_rels.each do |rel| next unless respond_to?(rel.index_name) lookups = [] batch.zip(objects).each do |identifier, obj| next unless obj field_value = obj.send(rel.field) next if field_value.nil? || Familia.blank_text?(field_value) lookups << [identifier, field_value.to_s] end next if lookups.empty? index_dbkey = send(rel.index_name).dbkey raw_values = dbclient.hmget(index_dbkey, *lookups.map(&:last)) lookups.each_with_index do |(identifier, field_value_str), idx| indexed_id = deserialize_index_value(raw_values[idx]) if indexed_id.nil? in_instances_missing_unique_index << { identifier: identifier, index_name: rel.index_name, field_value: field_value_str, existing_index_value: nil, } elsif indexed_id != identifier index_points_to_wrong_identifier << { index_name: rel.index_name, field_value: field_value_str, expected_id: identifier, index_id: indexed_id, } end end end progress&.call(phase: :cross_references, current: processed, total: total) end status = if in_instances_missing_unique_index.empty? && index_points_to_wrong_identifier.empty? :ok else :issues_found end { in_instances_missing_unique_index: in_instances_missing_unique_index, index_points_to_wrong_identifier: index_points_to_wrong_identifier, status: status, } end |
#audit_instances(batch_size: 100) {|Hash| ... } ⇒ Hash
Compares the instances timeline against actual DB keys via SCAN.
Detects:
- Phantoms: identifiers in timeline but no corresponding hash key
- Missing: hash keys in DB but not in timeline
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
# File 'lib/familia/horreum/management/audit.rb', line 29 def audit_instances(batch_size: 100, &progress) Familia.assert_replies_available!("#{self}.audit_instances") # Phase 1: Collect identifiers from timeline timeline_ids = Set.new(instances.members) progress&.call(phase: :timeline_collected, current: timeline_ids.size, total: nil) # Phase 2: SCAN keys and extract identifiers (source of truth) scan_ids = scan_identifiers(batch_size: batch_size, &progress) # Phase 3: Set differences phantoms = (timeline_ids - scan_ids).to_a missing = (scan_ids - timeline_ids).to_a { phantoms: phantoms, missing: missing, count_timeline: timeline_ids.size, count_scan: scan_ids.size, } end |
#audit_multi_indexes(scanned_identifiers: nil, loaded_objects: nil) ⇒ Array<Hash>
Audits all multi indexes.
For each multi index:
- SCANs for per-value set keys
- Checks that each member exists and field value matches
- Detects orphaned set keys (sets for values no object has)
97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 |
# File 'lib/familia/horreum/management/audit.rb', line 97 def audit_multi_indexes(scanned_identifiers: nil, loaded_objects: nil) Familia.assert_replies_available!("#{self}.audit_multi_indexes") return [] unless respond_to?(:indexing_relationships) indexing_relationships.select do |r| r.cardinality == :multi end.map do |rel| audit_single_multi_index( rel, scanned_identifiers: scanned_identifiers, loaded_objects: loaded_objects, ) end end |
#audit_participations(sample_size: nil) ⇒ Array<Hash>
Audits participation collections for stale members.
For each participation relationship defined on this class:
- Class-level: checks the single class collection directly
- Instance-level: SCANs for collection keys on the target class
- Enumerates raw members of each collection
- Verifies each referenced participant object still exists
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 |
# File 'lib/familia/horreum/management/audit.rb', line 124 def audit_participations(sample_size: nil) Familia.assert_replies_available!("#{self}.audit_participations") return [] unless respond_to?(:participation_relationships) participation_relationships.flat_map do |rel| if rel.target_class == self # Class-level participation (class_participates_in) [audit_class_participation(rel, sample_size: sample_size)] else # Instance-level participation (participates_in TargetClass, :collection) audit_instance_participations(rel, sample_size: sample_size) end end end |
#audit_related_fields ⇒ Array<Hash>
Audits instance-level related_fields (list/set/zset/hashkey) for orphaned collection keys whose parent Horreum hash no longer exists.
destroy! cleans related fields inside a transaction, so orphans only arise when destroy! is interrupted (process crash, manual Redis tampering, bugs in older code paths). This audit surfaces those cases.
Class-level related fields (class_list/class_set/class_hashkey) are intentionally skipped: their keys are Settings#prefix:field_name with no identifier segment, so they cannot be orphaned by instance destruction.
154 155 156 157 158 159 160 |
# File 'lib/familia/horreum/management/audit.rb', line 154 def Familia.assert_replies_available!("#{self}.audit_related_fields") return [] unless relations? .values.map { |definition| (definition) } end |
#audit_unique_indexes(scanned_identifiers: nil, loaded_objects: nil) ⇒ Array<Hash>
Audits all unique indexes (class-level only, where within is nil).
For each unique index:
- Reads all entries from the index HashKey
- Checks that each indexed object exists and its field value matches
- Checks for objects that should be indexed but aren't
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 |
# File 'lib/familia/horreum/management/audit.rb', line 67 def audit_unique_indexes(scanned_identifiers: nil, loaded_objects: nil) Familia.assert_replies_available!("#{self}.audit_unique_indexes") return [] unless respond_to?(:indexing_relationships) indexing_relationships.select do |r| r.cardinality == :unique && r.within.nil? end.map do |rel| audit_single_unique_index( rel, scanned_identifiers: scanned_identifiers, loaded_objects: loaded_objects, ) end end |
#health_check(batch_size: 100, sample_size: nil, audit_collections: false, check_cross_refs: false) {|Hash| ... } ⇒ AuditReport
Runs all audits and wraps results in an AuditReport.
The related_fields audit is opt-in via audit_collections: true
because it performs an additional SCAN per instance-level field.
When omitted (or false), AuditReport#related_fields is nil which
signals "not checked" rather than "checked and clean".
The cross-references audit is opt-in via check_cross_refs: true.
It walks every identifier in the instances ZSET and cross-checks
each class-level unique index; skipping it keeps the default
health_check fast. When omitted (or false), AuditReport#cross_references
is nil, signalling "not checked".
287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 |
# File 'lib/familia/horreum/management/audit.rb', line 287 def health_check(batch_size: 100, sample_size: nil, audit_collections: false, check_cross_refs: false, &progress) Familia.assert_replies_available!("#{self}.health_check") start_time = Familia.now inst = audit_instances(batch_size: batch_size, &progress) # Reuse the SCAN pass and the HGETALL pipeline across both index # audits. Without this, a model with N unique indexes and M multi # indexes would trigger N+M additional SCANs and load_multi round # trips during their "missing" phases. has_indexes = respond_to?(:indexing_relationships) && indexing_relationships.any? do |r| (r.cardinality == :unique && r.within.nil?) || r.cardinality == :multi end if has_indexes shared_ids = scan_identifiers(batch_size: batch_size).to_a shared_objects = load_multi(shared_ids) else shared_ids = nil shared_objects = nil end uniq = audit_unique_indexes( scanned_identifiers: shared_ids, loaded_objects: shared_objects, ) multi = audit_multi_indexes( scanned_identifiers: shared_ids, loaded_objects: shared_objects, ) parts = audit_participations(sample_size: sample_size) = audit_collections ? : nil cross_refs = check_cross_refs ? audit_cross_references(batch_size: batch_size, &progress) : nil duration = Familia.now - start_time AuditReport.new( model_class: name, audited_at: start_time, instances: inst, unique_indexes: uniq, multi_indexes: multi, participations: parts, related_fields: , cross_references: cross_refs, duration: duration, ) end |