Class: Familia::Migration::Model Abstract
- Defined in:
- lib/familia/migration/model.rb
Overview
Subclass and implement #prepare and #process_record
Base class for individual record migrations on Familia::Horreum models
Provides Redis SCAN-based iteration with progress tracking, error handling, and dry-run/actual-run modes for processing records one at a time.
When to Use Model vs Pipeline
Use Model when:
- Complex logic is needed for each record
- Error handling per record is important
- Records need individual validation
- Updates vary significantly between records
Use Pipeline when:
- Simple bulk updates across many records
- Performance is critical for large datasets
- All records get similar field updates
- Redis pipelining can be utilized effectively
Subclassing Requirements
Subclasses must implement:
- #prepare - Set @model_class and optionally @batch_size
- #process_record - Handle individual record processing
Subclasses may override:
- #migration_needed? - Default returns true (always migrate)
- #load_from_key - Custom object loading from database keys
Usage Example
class CustomerEmailMigration < Familia::Migration::Model def prepare @model_class = Customer @batch_size = 1000 # optional, defaults to config end
def process_record(obj, key)
return unless obj.email.blank?
for_realsies_this_time? do
obj.email = "#{obj.custid}@example.com"
obj.save
end
track_stat(:emails_updated)
end
end
Development Rule
IMPORTANT: Deploy schema changes and logic changes separately. This prevents new model logic from breaking migration logic and reduces debugging complexity.
Direct Known Subclasses
Instance Attribute Summary collapse
-
#batch_size ⇒ Integer
readonly
Number of keys to scan per Redis SCAN operation.
-
#error_count ⇒ Integer
readonly
Number of processing errors encountered.
-
#interactive ⇒ Boolean
readonly
Interactive debugging mode flag.
-
#model_class ⇒ Class
readonly
Model class being migrated.
-
#records_needing_update ⇒ Integer
readonly
Records that passed through process_record.
-
#records_updated ⇒ Integer
readonly
Records successfully updated.
-
#scan_pattern ⇒ String
readonly
Redis SCAN pattern for finding records.
-
#scan_type ⇒ String?
readonly
Redis SCAN TYPE option, or nil to scan keys of every type.
-
#total_records ⇒ Integer
readonly
Total number of indexed records in the model.
-
#total_scanned ⇒ Integer
readonly
Number of keys found by Redis SCAN.
Instance Method Summary collapse
-
#initialize(options = {}) ⇒ Model
constructor
A new instance of Model.
-
#load_from_key(key) ⇒ Familia::Horreum, Familia::DataType
Load Familia::Horreum object instance from database key.
-
#migrate ⇒ Boolean
Main migration entry point.
-
#migration_needed? ⇒ Boolean
Default migration check - always returns true.
-
#prepare ⇒ void
protected
abstract
Set @model_class and optionally @batch_size.
-
#process_record(obj, key) ⇒ void
protected
abstract
Process a single record.
-
#process_record_with_validation(obj, key) ⇒ void
protected
Wrapper that applies validation hooks around process_record.
-
#track_stat(statname, increment = 1) ⇒ void
protected
Track statistics and auto-increment records_updated counter.
-
#track_stat_and_log_reason(obj, decision, field) ⇒ nil
protected
Track stat and log decision reason in one call.
-
#validate_after_transform? ⇒ Boolean
protected
Override in subclass to enable post-transform validation.
-
#validate_before_transform? ⇒ Boolean
protected
Override in subclass to enable pre-transform validation.
Constructor Details
#initialize(options = {}) ⇒ Model
Returns a new instance of Model.
119 120 121 122 123 |
# File 'lib/familia/migration/model.rb', line 119 def initialize( = {}) super reset_counters set_defaults end |
Instance Attribute Details
#batch_size ⇒ Integer (readonly)
Number of keys to scan per Redis SCAN operation
72 73 74 |
# File 'lib/familia/migration/model.rb', line 72 def batch_size @batch_size end |
#error_count ⇒ Integer (readonly)
Number of processing errors encountered
92 93 94 |
# File 'lib/familia/migration/model.rb', line 92 def error_count @error_count end |
#interactive ⇒ Boolean (readonly)
Interactive debugging mode flag
When true, #migrate loads pry-byebug before it scans, and each record that raises opens a pry session in #handle_record_error. familia does not depend on pry-byebug, so the application's bundle must include it; otherwise #migrate raises Errors::PreconditionFailed.
102 103 104 |
# File 'lib/familia/migration/model.rb', line 102 def interactive @interactive end |
#model_class ⇒ Class (readonly)
Model class being migrated
68 69 70 |
# File 'lib/familia/migration/model.rb', line 68 def model_class @model_class end |
#records_needing_update ⇒ Integer (readonly)
Records that passed through process_record
84 85 86 |
# File 'lib/familia/migration/model.rb', line 84 def records_needing_update @records_needing_update end |
#records_updated ⇒ Integer (readonly)
Records successfully updated
88 89 90 |
# File 'lib/familia/migration/model.rb', line 88 def records_updated @records_updated end |
#scan_pattern ⇒ String (readonly)
Redis SCAN pattern for finding records. Defaults to the model class's scan_pattern, which escapes the class prefix, delimiter and suffix.
108 109 110 |
# File 'lib/familia/migration/model.rb', line 108 def scan_pattern @scan_pattern end |
#scan_type ⇒ String? (readonly)
Redis SCAN TYPE option, or nil to scan keys of every type. When
prepare leaves @scan_pattern unset, it defaults to
Familia::Horreum::OBJECT_KEY_TYPE, so the default pattern yields only object hashes and not a multi_index bucket set whose name ends with the class suffix. With a custom @scan_pattern it stays nil unless #prepare sets it.
117 118 119 |
# File 'lib/familia/migration/model.rb', line 117 def scan_type @scan_type end |
#total_records ⇒ Integer (readonly)
Total number of indexed records in the model
76 77 78 |
# File 'lib/familia/migration/model.rb', line 76 def total_records @total_records end |
#total_scanned ⇒ Integer (readonly)
Number of keys found by Redis SCAN
80 81 82 |
# File 'lib/familia/migration/model.rb', line 80 def total_scanned @total_scanned end |
Instance Method Details
#load_from_key(key) ⇒ Familia::Horreum, Familia::DataType
Load Familia::Horreum object instance from database key
Override this method to customize loading behavior. For example, with a custom @scan_pattern, the migration might loop through relation keys of a horreum model (e.g. customer:ID:custom_domain).
Typically migrations iterate over objects themselves, but this won't work if there are dangling "orphan" keys without corresponding objects. Override this method to handle such cases.
188 189 190 |
# File 'lib/familia/migration/model.rb', line 188 def load_from_key(key) model_class.find_by_key(key) end |
#migrate ⇒ Boolean
Main migration entry point
Validates configuration, displays run mode information, executes the SCAN-based record processing, and displays a comprehensive summary.
An error raised while processing one record is logged and counted, and the scan continues. The errors listed below end the run instead.
143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 |
# File 'lib/familia/migration/model.rb', line 143 def migrate validate_model_class! # Set `@interactive = true` in the implementing migration class # for an interactive debug session on a per-record basis. require_interactive_debugger if interactive print_database_details info("[#{self.class.name.split('::').last}] Starting #{model_class.name} migration") info("Processing up to #{total_records} records") info('Will show progress every 100 records and log each update') scan_and_process_records print_database_details print_migration_summary @error_count == 0 end |
#migration_needed? ⇒ Boolean
Default migration check - always returns true
Always return true to allow re-running for error recovery. The migration should be idempotent - it won't overwrite existing values. Override if you need conditional migration logic.
171 172 173 174 |
# File 'lib/familia/migration/model.rb', line 171 def migration_needed? debug("[#{self.class.name.split('::').last}] Checking if migration is needed...") true end |
#prepare ⇒ void (protected)
Subclasses must implement this method
This method returns an undefined value.
Set @model_class and optionally @batch_size
Required for subclasses - must set @model_class to a Familia::Horreum subclass. Can optionally set @batch_size to override the default.
203 204 205 |
# File 'lib/familia/migration/model.rb', line 203 def prepare raise NotImplementedError, "#{self.class} must set @model_class in #prepare" end |
#process_record(obj, key) ⇒ void (protected)
Subclasses must implement this method
This method returns an undefined value.
Process a single record
Required for subclasses - implement the core logic for processing each record. Use #track_stat to count operations and Base#for_realsies_this_time? to wrap actual changes.
218 219 220 |
# File 'lib/familia/migration/model.rb', line 218 def process_record(obj, key) raise NotImplementedError, "#{self.class} must implement #process_record" end |
#process_record_with_validation(obj, key) ⇒ void (protected)
This method returns an undefined value.
Wrapper that applies validation hooks around process_record
Called internally by #process_single_record when validation is enabled. Validates the object before and/or after the transform based on #validate_before_transform? and #validate_after_transform?.
A Familia::SchemaValidatorLoadError raised here ends the run instead of counting as an error on the record. With only #validate_after_transform? enabled, the record it is raised for has already been transformed.
291 292 293 294 295 296 297 298 299 300 301 302 303 |
# File 'lib/familia/migration/model.rb', line 291 def process_record_with_validation(obj, key) if validate_before_transform? result = validate_schema(obj, context: 'before transform') track_stat(:schema_errors_before) unless result[:valid] end process_record(obj, key) if validate_after_transform? result = validate_schema(obj, context: 'after transform') track_stat(:schema_errors_after) unless result[:valid] end end |
#track_stat(statname, increment = 1) ⇒ void (protected)
This method returns an undefined value.
Track statistics and auto-increment records_updated counter
Automatically increments @records_updated when statname is :records_updated. Use this to maintain consistent counting across migrations.
230 231 232 233 |
# File 'lib/familia/migration/model.rb', line 230 def track_stat(statname, increment = 1) super @records_updated += increment if statname == :records_updated end |
#track_stat_and_log_reason(obj, decision, field) ⇒ nil (protected)
Track stat and log decision reason in one call
Convenience method for logging migration decisions with consistent formatting and automatic statistic tracking.
244 245 246 247 248 249 |
# File 'lib/familia/migration/model.rb', line 244 def track_stat_and_log_reason(obj, decision, field) track_stat(:decision) track_stat("#{decision}_#{field}") info("#{decision} objid=#{obj.respond_to?(:objid) ? obj.objid : 'N/A'} #{field}=#{obj.send(field)}") nil end |
#validate_after_transform? ⇒ Boolean (protected)
Override in subclass to enable post-transform validation
When enabled, validates each record against its schema after #process_record completes. Validation failures are tracked via the :schema_errors_after stat.
271 272 273 |
# File 'lib/familia/migration/model.rb', line 271 def validate_after_transform? false end |
#validate_before_transform? ⇒ Boolean (protected)
Override in subclass to enable pre-transform validation
When enabled, validates each record against its schema before #process_record is called. Validation failures are tracked via the :schema_errors_before stat.
260 261 262 |
# File 'lib/familia/migration/model.rb', line 260 def validate_before_transform? false end |