Class: Familia::Migration::Model Abstract

Inherits:
Base
  • Object
show all
Defined in:
lib/familia/migration/model.rb

Overview

This class is abstract.

Subclass and implement #prepare and #process_record

Base class for individual record migrations on Familia::Horreum models

Provides Redis SCAN-based iteration with progress tracking, error handling, and dry-run/actual-run modes for processing records one at a time.

When to Use Model vs Pipeline

Use Model when:

  • Complex logic is needed for each record
  • Error handling per record is important
  • Records need individual validation
  • Updates vary significantly between records

Use Pipeline when:

  • Simple bulk updates across many records
  • Performance is critical for large datasets
  • All records get similar field updates
  • Redis pipelining can be utilized effectively

Subclassing Requirements

Subclasses must implement:

Subclasses may override:

Usage Example

class CustomerEmailMigration < Familia::Migration::Model def prepare @model_class = Customer @batch_size = 1000 # optional, defaults to config end

def process_record(obj, key)
  return unless obj.email.blank?

  for_realsies_this_time? do
    obj.email = "#{obj.custid}@example.com"
    obj.save
  end
  track_stat(:emails_updated)
end

end

Development Rule

IMPORTANT: Deploy schema changes and logic changes separately. This prevents new model logic from breaking migration logic and reduces debugging complexity.

See Also:

Direct Known Subclasses

Pipeline

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Model

Returns a new instance of Model.



102
103
104
105
106
 
# File 'lib/familia/migration/model.rb', line 102

def initialize(options = {})
  super
  reset_counters
  set_defaults
end

Instance Attribute Details

#batch_size ⇒ Integer (readonly)

Number of keys to scan per Redis SCAN operation

Returns:

  • (Integer) —

    batch size for scanning



72
73
74
 
# File 'lib/familia/migration/model.rb', line 72

def batch_size
  @batch_size
end

#error_count ⇒ Integer (readonly)

Number of processing errors encountered

Returns:

  • (Integer) —

    error count



92
93
94
 
# File 'lib/familia/migration/model.rb', line 92

def error_count
  @error_count
end

#interactive ⇒ Boolean (readonly)

Interactive debugging mode flag

Returns:

  • (Boolean) —

    whether to drop into pry on errors



96
97
98
 
# File 'lib/familia/migration/model.rb', line 96

def interactive
  @interactive
end

#model_class ⇒ Class (readonly)

Model class being migrated

Returns:

  • (Class) —

    Familia::Horreum subclass



68
69
70
 
# File 'lib/familia/migration/model.rb', line 68

def model_class
  @model_class
end

#records_needing_update ⇒ Integer (readonly)

Records that passed through process_record

Returns:

  • (Integer) —

    count of records needing updates



84
85
86
 
# File 'lib/familia/migration/model.rb', line 84

def records_needing_update
  @records_needing_update
end

#records_updated ⇒ Integer (readonly)

Records successfully updated

Returns:

  • (Integer) —

    count of records modified



88
89
90
 
# File 'lib/familia/migration/model.rb', line 88

def records_updated
  @records_updated
end

#scan_pattern ⇒ String (readonly)

Redis SCAN pattern for finding records

Returns:

  • (String) —

    pattern like "customer:*:object"



100
101
102
 
# File 'lib/familia/migration/model.rb', line 100

def scan_pattern
  @scan_pattern
end

#total_records ⇒ Integer (readonly)

Total number of indexed records in the model

Returns:

  • (Integer) —

    count from model_class.instances



76
77
78
 
# File 'lib/familia/migration/model.rb', line 76

def total_records
  @total_records
end

#total_scanned ⇒ Integer (readonly)

Number of keys found by Redis SCAN

Returns:

  • (Integer) —

    actual keys discovered



80
81
82
 
# File 'lib/familia/migration/model.rb', line 80

def total_scanned
  @total_scanned
end

Instance Method Details

#load_from_key(key) ⇒ Familia::Horreum, Familia::DataType

Load Familia::Horreum object instance from database key

Override this method to customize loading behavior. For example, with a custom @scan_pattern, the migration might loop through relation keys of a horreum model (e.g. customer:ID:custom_domain).

Typically migrations iterate over objects themselves, but this won't work if there are dangling "orphan" keys without corresponding objects. Override this method to handle such cases.

Parameters:

  • key (String) —

    database key to load from

Returns:



160
161
162
 
# File 'lib/familia/migration/model.rb', line 160

def load_from_key(key)
  model_class.find_by_key(key)
end

#migrate ⇒ Boolean

Main migration entry point

Validates configuration, displays run mode information, executes the SCAN-based record processing, and displays a comprehensive summary.

Returns:

  • (Boolean) —

    true if no errors occurred



115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
 
# File 'lib/familia/migration/model.rb', line 115

def migrate
  validate_model_class!

  # Set `@interactive = true` in the implementing migration class
  # for an interactive debug session on a per-record basis.
  require 'pry-byebug' if interactive

  print_database_details
  run_mode_banner

  info("[#{self.class.name.split('::').last}] Starting #{model_class.name} migration")
  info("Processing up to #{total_records} records")
  info('Will show progress every 100 records and log each update')

  scan_and_process_records
  print_database_details
  print_migration_summary

  @error_count == 0
end

#migration_needed? ⇒ Boolean

Default migration check - always returns true

Always return true to allow re-running for error recovery. The migration should be idempotent - it won't overwrite existing values. Override if you need conditional migration logic.

Returns:

  • (Boolean) —

    true to proceed with migration



143
144
145
146
 
# File 'lib/familia/migration/model.rb', line 143

def migration_needed?
  debug("[#{self.class.name.split('::').last}] Checking if migration is needed...")
  true
end

#prepare ⇒ void (protected)

This method is abstract.

Subclasses must implement this method

This method returns an undefined value.

Set @model_class and optionally @batch_size

Required for subclasses - must set @model_class to a Familia::Horreum subclass. Can optionally set @batch_size to override the default.

Raises:

  • (NotImplementedError) —

    if not implemented



175
176
177
 
# File 'lib/familia/migration/model.rb', line 175

def prepare
  raise NotImplementedError, "#{self.class} must set @model_class in #prepare"
end

#process_record(obj, key) ⇒ void (protected)

This method is abstract.

Subclasses must implement this method

This method returns an undefined value.

Process a single record

Required for subclasses - implement the core logic for processing each record. Use #track_stat to count operations and Base#for_realsies_this_time? to wrap actual changes.

Parameters:

Raises:

  • (NotImplementedError) —

    if not implemented



190
191
192
 
# File 'lib/familia/migration/model.rb', line 190

def process_record(obj, key)
  raise NotImplementedError, "#{self.class} must implement #process_record"
end

#process_record_with_validation(obj, key) ⇒ void (protected)

This method returns an undefined value.

Wrapper that applies validation hooks around process_record

Called internally by #process_single_record when validation is enabled. Validates the object before and/or after the transform based on #validate_before_transform? and #validate_after_transform?.

Parameters:

  • obj (Familia::Horreum) —

    the object to process

  • key (String) —

    the database key of the record



256
257
258
259
260
261
262
263
264
265
266
267
268
 
# File 'lib/familia/migration/model.rb', line 256

def process_record_with_validation(obj, key)
  if validate_before_transform?
    result = validate_schema(obj, context: 'before transform')
    track_stat(:schema_errors_before) unless result[:valid]
  end

  process_record(obj, key)

  if validate_after_transform?
    result = validate_schema(obj, context: 'after transform')
    track_stat(:schema_errors_after) unless result[:valid]
  end
end

#track_stat(statname, increment = 1) ⇒ void (protected)

This method returns an undefined value.

Track statistics and auto-increment records_updated counter

Automatically increments @records_updated when statname is :records_updated. Use this to maintain consistent counting across migrations.

Parameters:

  • statname (Symbol) —

    The name of the statistic to track

  • increment (Integer) (defaults to: 1) —

    The amount to increment by



202
203
204
205
 
# File 'lib/familia/migration/model.rb', line 202

def track_stat(statname, increment = 1)
  super
  @records_updated += increment if statname == :records_updated
end

#track_stat_and_log_reason(obj, decision, field) ⇒ nil (protected)

Track stat and log decision reason in one call

Convenience method for logging migration decisions with consistent formatting and automatic statistic tracking.

Parameters:

  • obj (Familia::Horreum) —

    object being processed

  • decision (String) —

    decision made (e.g., 'skipped', 'updated')

  • field (String) —

    field name involved in decision

Returns:

  • (nil) 


216
217
218
219
220
221
 
# File 'lib/familia/migration/model.rb', line 216

def track_stat_and_log_reason(obj, decision, field)
  track_stat(:decision)
  track_stat("#{decision}_#{field}")
  info("#{decision} objid=#{obj.respond_to?(:objid) ? obj.objid : 'N/A'} #{field}=#{obj.send(field)}")
  nil
end

#validate_after_transform? ⇒ Boolean (protected)

Override in subclass to enable post-transform validation

When enabled, validates each record against its schema after #process_record completes. Validation failures are tracked via the :schema_errors_after stat.

Returns:

  • (Boolean) —

    true to validate after transform



243
244
245
 
# File 'lib/familia/migration/model.rb', line 243

def validate_after_transform?
  false
end

#validate_before_transform? ⇒ Boolean (protected)

Override in subclass to enable pre-transform validation

When enabled, validates each record against its schema before #process_record is called. Validation failures are tracked via the :schema_errors_before stat.

Returns:

  • (Boolean) —

    true to validate before transform



232
233
234
 
# File 'lib/familia/migration/model.rb', line 232

def validate_before_transform?
  false
end