Encrypted Fields Guide

💡 Quick Reference

Add persistent encrypted storage to any Familia model:

class User < Familia::Horreum
  feature :encrypted_fields
  encrypted_field :sensitive_data
end

Overview

The Encrypted Fields feature provides transparent field-level encryption for sensitive data stored in Redis/Valkey. It combines industry-standard encryption algorithms with Ruby-friendly APIs, ensuring your sensitive data is protected at rest while maintaining the performance and simplicity you expect from Familia.

Why Use Encrypted Fields?

Compliance: Meet regulatory requirements (GDPR, HIPAA, PCI-DSS) for sensitive data protection.

Defense in Depth: Protect against database breaches, memory dumps, and unauthorized Valkey/Redis access.

Transparent Security: Encryption and decryption happen automatically - no changes to your application logic.

Performance Focused: Optimized for Ruby's memory model with request-level caching and efficient key derivation.

Future Proof: Modular provider system supports algorithm upgrades and key rotation.

⚠️ Security Consideration

Encrypted fields protect data at rest in Redis/Valkey. Consider your threat model: encryption keys are held in Ruby memory and may be visible in memory dumps.

Quick Start

Basic Encrypted Storage

class Customer < Familia::Horreum
  feature :encrypted_fields

  # Regular fields (stored as plaintext)
  field :email, :company_name, :created_at

  # Encrypted fields (automatically encrypted/decrypted)
  encrypted_field :api_key
  encrypted_field :notes
  encrypted_field :credit_card_last_four
end

# Configure encryption keys
Familia.configure do |config|
  config.encryption_keys = {
    v1: ENV['FAMILIA_ENCRYPTION_KEY']
  }
  config.current_key_version = :v1
end

# Usage is identical to regular fields
customer = Customer.new(
  email: 'contact@acme.com',
  company_name: 'Acme Corporation',
  api_key: 'sk-1234567890abcdef',
  notes: 'VIP customer - handle with care'
)

customer.save

# Access returns ConcealedString for safety
customer.api_key.class          # => ConcealedString
customer.api_key.to_s           # => "[CONCEALED]" (safe for logging)
customer.api_key.reveal         # => "sk-1234567890abcdef" (actual value)

Key Generation and Setup

Generate secure encryption keys for your environment:

# Generate a new 32-byte key
export FAMILIA_ENCRYPTION_KEY=$(openssl rand -base64 32)

# For production, use a secure key management service
export FAMILIA_ENCRYPTION_KEY_V1="your-secure-key-from-vault"
export FAMILIA_ENCRYPTION_KEY_V2="new-key-for-rotation"

🔒 Key Management Best Practices

• Use different keys for each environment (development, staging, production)
• Store keys in a secure key management service (AWS KMS, HashiCorp Vault)
• Never commit keys to source control
• Rotate keys regularly (recommended: every 90-180 days)

Configuration Deep Dive

Basic Configuration

Familia.configure do |config|
  # Single key setup (simplest)
  config.encryption_keys = {
    v1: ENV['FAMILIA_ENCRYPTION_KEY']
  }
  config.current_key_version = :v1

  # Optional: application-specific key derivation
  config.encryption_personalization = 'MyApp-Production-2024'
end

# Validate configuration before use
Familia::Encryption.validate_configuration!

Multi-Key Configuration (Key Rotation)

Familia.configure do |config|
  # Multiple keys for rotation
  config.encryption_keys = {
    v1: ENV['FAMILIA_ENCRYPTION_KEY_V1'],  # Legacy key
    v2: ENV['FAMILIA_ENCRYPTION_KEY_V2'],  # Current key
    v3: ENV['FAMILIA_ENCRYPTION_KEY_V3']   # New key for rotation
  }

  # New data encrypted with v3, old data readable with v1/v2
  config.current_key_version = :v3

  # Provider-specific configuration
  config.encryption_providers = {
    xchacha20_poly1305: {
      priority: 100,
      require_gem: 'rbnacl'
    },
    aes_gcm: {
      priority: 50,
      always_available: true
    }
  }
end

💡 Key Rotation Strategy

1. Add new key version to configuration
2. Update current_key_version to new version
3. Deploy application (new writes use new key)
4. Re-encrypt existing data with re_encrypt_fields!
5. Remove old key version after migration complete

Encryption Providers

Familia supports multiple encryption algorithms with automatic provider selection:

XChaCha20-Poly1305 (Recommended)

The preferred encryption algorithm offering excellent security and performance:

# Requires rbnacl gem
gem 'rbnacl', '~> 7.1'

# Automatic selection when available
class SecureVault < Familia::Horreum
  feature :encrypted_fields

  # Will use XChaCha20-Poly1305 if rbnacl is available
  encrypted_field :master_password
  encrypted_field :recovery_codes
end

Characteristics:

• Algorithm: XChaCha20-Poly1305 AEAD
• Key Size: 32 bytes (256 bits)
• Nonce Size: 24 bytes (192 bits) - collision resistant
• Authentication: Built-in with Poly1305 MAC
• Performance: Excellent on modern CPUs

🚀 Performance Tip

XChaCha20-Poly1305 is typically 20-30% faster than AES-GCM and provides better security margins.

AES-256-GCM (Fallback)

Standard AES encryption using OpenSSL (always available):

# No additional gems required
class StandardVault < Familia::Horreum
  feature :encrypted_fields

  # Explicitly specify AES-GCM
  encrypted_field :secret_data, provider: :aes_gcm
end

Characteristics:

• Algorithm: AES-256-GCM AEAD
• Key Size: 32 bytes (256 bits)
• IV Size: 12 bytes (96 bits)
• Authentication: Built-in with GCM mode
• Availability: Always available via OpenSSL

Provider Selection Logic

# Check available providers
providers = Familia::Encryption.available_providers
# => [
#   { name: :xchacha20_poly1305, priority: 100, available: true },
#   { name: :aes_gcm, priority: 50, available: true }
# ]

# Force specific provider for testing
class TestVault < Familia::Horreum
  feature :encrypted_fields

  encrypted_field :test_data, provider: :aes_gcm  # Force AES-GCM
end

Advanced Field Configuration

Additional Authenticated Data (AAD)

Protect against field tampering by including related fields in authentication:

class SecureDocument < Familia::Horreum
  feature :encrypted_fields

  field :document_id, :owner_id, :created_at

  # Include document_id and owner_id in authentication
  encrypted_field :content, aad_fields: [:document_id, :owner_id]
  encrypted_field :metadata, aad_fields: [:document_id, :created_at]
end

# AAD fields are included in encryption but not encrypted themselves
doc = SecureDocument.new(
  document_id: 'doc123',
  owner_id: 'user456',
  content: 'Sensitive document content',
  created_at: Time.now.to_i
)

doc.save

# If someone modifies document_id or owner_id, decryption will fail
# This prevents attacks where encrypted data is moved between records

🛡️ Security Enhancement

AAD prevents encrypted fields from being moved between objects. Use it for high-security scenarios where data integrity is critical.

Per-Field Provider Selection

class MultiAlgorithmVault < Familia::Horreum
  feature :encrypted_fields

  # Use best available algorithm (XChaCha20-Poly1305 preferred)
  encrypted_field :general_secret

  # Force AES-GCM for compliance requirements
  encrypted_field :compliance_data, provider: :aes_gcm

  # High-security field with AAD
  encrypted_field :ultra_secure,
                  provider: :xchacha20_poly1305,
                  aad_fields: [:vault_id, :owner_id]
end

ConcealedString Security

Encrypted fields return ConcealedString objects to prevent accidental exposure:

Safe Handling

class User < Familia::Horreum
  feature :encrypted_fields
  encrypted_field :api_key
end

user = User.create(api_key: "sk-1234567890abcdef")

# Safe operations (won't expose actual value)
puts user.api_key.to_s                    # => "[CONCEALED]"
puts user.api_key.inspect                 # => "[CONCEALED]"
logger.info("User key: #{user.api_key}")  # => "User key: [CONCEALED]"

# JSON serialization safety
user_json = user.to_json
# All encrypted fields appear as "[CONCEALED]" in JSON

# Explicit access when needed (requires block)
user.api_key.reveal do |actual_key|
  # Use actual_key here: "sk-1234567890abcdef"
  process_key(actual_key)
end

String Operations

api_key = user.api_key

# Length operations work on concealed representation
api_key.length        # => 11 ("[CONCEALED]".length)
api_key.size          # => 11

# Comparison operations
api_key == "[CONCEALED]"              # => true
api_key.start_with?("[CONCEALED]")    # => true

# Reveal for actual operations (requires block)
api_key.reveal do |actual_key|
  actual_key.length                   # => 17 (actual key length)
  actual_key.start_with?("sk-")       # => true
end

⚠️ Important

Always use .reveal { |value| ... } with a block when you need the actual value. This makes it obvious in code reviews where sensitive data is being accessed and prevents accidental copies.

Performance Optimization

Request-Level Caching

For applications that perform many encryption operations:

class BulkDataProcessor
  def process_sensitive_batch(records)
    # Enable key caching for the entire batch
    Familia::Encryption.with_request_cache do
      records.each do |record|
        # Key derivation happens once per field type
        record.encrypted_field1 = process_data(record.raw_data1)
        record.encrypted_field2 = process_data(record.raw_data2)
        record.save
      end
    end
    # Cache automatically cleared at end of block
  end
end

Performance Improvements:

• Key derivation: 1x per field type instead of per operation
• Typical improvement: 40-60% faster for batch operations
• Memory usage: Minimal (keys cached temporarily)

Benchmarking Your Setup

# Test encryption performance with your data
def benchmark_encryption
  require 'benchmark'

  test_data = {
    small: "x" * 100,
    medium: "x" * 1000,
    large: "x" * 10000
  }

  Benchmark.bm(15) do |x|
    test_data.each do |size, data|
      x.report("#{size} (#{data.length}b)") do
        1000.times do
          TestModel.create(encrypted_field: data)
        end
      end
    end
  end
end

# Provider comparison
providers = Familia::Encryption.benchmark_providers(iterations: 1000)
providers.each do |name, stats|
  puts "#{name}: #{stats[:ops_per_sec]} ops/sec"
end

Memory Management

class MemoryAwareModel < Familia::Horreum
  feature :encrypted_fields
  encrypted_field :large_data

  def process_and_clear
    # Process encrypted data
    result = expensive_operation(large_data.reveal)

    # Clear sensitive data from memory
    clear_encrypted_fields!

    result
  end

  def self.bulk_process_with_cleanup(ids)
    ids.each_slice(100) do |batch|
      objects = multiget(batch)

      objects.each(&:process_and_clear)

      # Force garbage collection periodically
      GC.start if batch.first % 1000 == 0
    end
  end
end

Key Rotation and Migration

Planned Key Rotation

# 1. Add new key to configuration
Familia.configure do |config|
  config.encryption_keys = {
    v1: ENV['OLD_KEY'],
    v2: ENV['CURRENT_KEY'],
    v3: ENV['NEW_KEY']        # Add new key
  }
  config.current_key_version = :v3  # Switch to new key
end

# 2. Deploy application (new data uses v3)

# 3. Migrate existing data
class KeyRotationTask
  def self.rotate_all_encrypted_data
    model_classes = [User, Document, Vault, SecretData]

    model_classes.each do |model_class|
      puts "Rotating keys for #{model_class.name}..."

      model_class.all.each_slice(100) do |batch|
        batch.each do |record|
          begin
            record.re_encrypt_fields!
            record.save
          rescue => e
            puts "Failed to rotate #{record.identifier}: #{e.message}"
          end
        end

        print "."
        sleep 0.1  # Rate limiting
      end

      puts "\nCompleted #{model_class.name}"
    end
  end
end

# 4. Remove old key after migration

Emergency Key Rotation

# For compromised keys, rotate immediately
class EmergencyRotation
  def self.emergency_key_rotation
    # 1. Generate new key immediately
    new_key = SecureRandom.base64(32)

    # 2. Update configuration
    Familia.configure do |config|
      config.encryption_keys[:emergency] = new_key
      config.current_key_version = :emergency
    end

    # 3. Re-encrypt all data immediately
    KeyRotationTask.rotate_all_encrypted_data

    # 4. Notify security team
    SecurityNotifier.alert_key_rotated(reason: 'emergency')
  end
end

Error Handling and Debugging

Common Configuration Errors

begin
  Familia::Encryption.validate_configuration!
rescue Familia::EncryptionError => e
  case e.message
  when /No encryption keys configured/
    puts "Add encryption keys to Familia.configure block"
  when /Invalid key format/
    puts "Keys must be base64-encoded 32-byte strings"
  when /Current key version not found/
    puts "current_key_version must exist in encryption_keys"
  else
    puts "Configuration error: #{e.message}"
  end
end

Debugging Encryption Issues

class DebugVault < Familia::Horreum
  feature :encrypted_fields
  encrypted_field :debug_data

  def debug_encryption_status
    status = {
      feature_enabled: self.class.features_enabled.include?(:encrypted_fields),
      field_encrypted: self.class.encrypted_field?(:debug_data),
      data_encrypted: encrypted_data?,
      fields_cleared: encrypted_fields_cleared?,
      current_provider: Familia::Encryption.current_provider,
      available_providers: Familia::Encryption.available_providers
    }

    puts JSON.pretty_generate(status)
    status
  end
end

# Debug individual field encryption
vault = DebugVault.new(debug_data: "test")
vault.save

field_status = vault.encrypted_fields_status
puts "Field status: #{field_status}"
# => {debug_data: {encrypted: true, key_version: :v2, provider: :xchacha20_poly1305}}

Performance Debugging

# Monitor encryption performance
class EncryptionMonitor
  def self.monitor_encryption_calls
    original_encrypt = Familia::Encryption.method(:encrypt)

    call_count = 0
    total_time = 0

    Familia::Encryption.define_singleton_method(:encrypt) do |data, **opts|
      start_time = Time.now
      result = original_encrypt.call(data, **opts)
      total_time += (Time.now - start_time)
      call_count += 1

      if call_count % 100 == 0
        avg_time = (total_time / call_count * 1000).round(2)
        puts "Encryption calls: #{call_count}, avg: #{avg_time}ms"
      end

      result
    end
  end
end

Testing Strategies

Test Configuration

# test/test_helper.rb
require 'familia'

# Use predictable test keys
test_keys = {
  v1: Base64.strict_encode64('a' * 32),
  v2: Base64.strict_encode64('b' * 32)
}

Familia.configure do |config|
  config.encryption_keys = test_keys
  config.current_key_version = :v1
  config.encryption_personalization = 'TestApp-Test'
end

# Validate test configuration
Familia::Encryption.validate_configuration!

Testing Encrypted Fields

# test/models/encrypted_model_test.rb
require 'test_helper'

class EncryptedModelTest < Minitest::Test
  def setup
    @model = EncryptedModel.new(
      name: "Test Model",
      secret_data: "sensitive information"
    )
    @model.save
  end

  def test_encryption_concealment
    # Field should return ConcealedString
    assert_instance_of Familia::Features::EncryptedFields::ConcealedString, @model.secret_data

    # String representation should be concealed
    assert_equal "[CONCEALED]", @model.secret_data.to_s

    # Reveal should return actual value
    assert_equal "sensitive information", @model.secret_data.reveal
  end

  def test_json_serialization_safety
    json_data = @model.to_json
    parsed = JSON.parse(json_data)

    # Encrypted fields should be concealed in JSON
    assert_equal "[CONCEALED]", parsed['secret_data']

    # Regular fields should be normal
    assert_equal "Test Model", parsed['name']
  end

  def test_encryption_persistence
    # Reload from database
    reloaded = EncryptedModel.load(@model.identifier)

    # Should still be able to decrypt
    assert_equal "sensitive information", reloaded.secret_data.reveal
  end

  def test_key_rotation
    original_data = @model.secret_data.reveal

    # Simulate key rotation
    @model.re_encrypt_fields!
    @model.save

    # Should still decrypt to same value
    reloaded = EncryptedModel.load(@model.identifier)
    assert_equal original_data, reloaded.secret_data.reveal
  end
end

Mock Encryption for Fast Tests

# test/support/mock_encryption.rb
module MockEncryption
  def self.setup
    # Replace encryption with reversible encoding for speed
    Familia::Encryption.define_singleton_method(:encrypt) do |data, **opts|
      Base64.strict_encode64("MOCK:#{data}")
    end

    Familia::Encryption.define_singleton_method(:decrypt) do |encrypted_data, **opts|
      decoded = Base64.strict_decode64(encrypted_data)
      decoded.sub(/^MOCK:/, '')
    end
  end

  def self.teardown
    # Restore original encryption methods
    load 'familia/encryption.rb'
  end
end

# Use in fast test suite
class FastEncryptedModelTest < Minitest::Test
  def setup
    MockEncryption.setup
  end

  def teardown
    MockEncryption.teardown
  end

  # Tests run much faster with mock encryption
end

Instance Methods

Core Encrypted Field Methods

encrypted_data?

Check if instance has any encrypted fields with values.

vault = Vault.new(secret_key: "value")
vault.encrypted_data?  # => true

empty_vault = Vault.new
empty_vault.encrypted_data?  # => false

clear_encrypted_fields!

Clear all encrypted field values from memory.

vault.clear_encrypted_fields!
vault.encrypted_fields_cleared?  # => true

re_encrypt_fields!

Re-encrypt all encrypted fields with current encryption settings (useful for key rotation).

vault.re_encrypt_fields!
vault.save  # Persists re-encrypted data

encrypted_fields_status

Get encryption status for all encrypted fields.

vault.encrypted_fields_status
# => {
#   secret_key: { encrypted: true, algorithm: "xchacha20poly1305", cleared: false },
#   api_token: { encrypted: true, cleared: true }
# }

Class Methods

encrypted_field?(field_name)

Check if a field is encrypted.

Vault.encrypted_field?(:secret_key)  # => true
Vault.encrypted_field?(:name)        # => false

encryption_info

Get encryption algorithm information.

Vault.encryption_info
# => {
#   algorithm: "xchacha20poly1305",
#   key_size: 32,
#   nonce_size: 24,
#   tag_size: 16
# }

ConcealedString Methods

reveal { |plaintext| ... }

Primary API for accessing decrypted values (requires block).

user.api_token.reveal do |token|
  HTTP.post('/api', headers: { 'X-Token' => token })
end

belongs_to_context?(record, field_name)

Validate that ConcealedString belongs to the given record context.

concealed.belongs_to_context?(user, :api_token)  # => true/false

cleared? and clear!

Memory management for encrypted data.

concealed.clear!
concealed.cleared?  # => true

Production Considerations

Monitoring and Alerting

# Monitor encryption health in production
class EncryptionHealthCheck
  def self.check
    results = {
      configuration_valid: false,
      providers_available: [],
      key_versions_accessible: [],
      sample_encrypt_decrypt: false
    }

    begin
      # Test configuration
      Familia::Encryption.validate_configuration!
      results[:configuration_valid] = true

      # Test providers
      results[:providers_available] = Familia::Encryption.available_providers.map { |p| p[:name] }

      # Test key access
      Familia.config.encryption_keys.each do |version, key|
        begin
          # Test key derivation
          Familia::Encryption.derive_key_for_field('test_field', version)
          results[:key_versions_accessible] << version
        rescue => e
          puts "Key version #{version} error: #{e.message}"
        end
      end

      # Test encrypt/decrypt cycle
      test_data = "health_check_#{Time.now.to_i}"
      encrypted = Familia::Encryption.encrypt(test_data)
      decrypted = Familia::Encryption.decrypt(encrypted)
      results[:sample_encrypt_decrypt] = (decrypted == test_data)

    rescue => e
      results[:error] = e.message
    end

    results
  end
end

# Set up monitoring
# Nagios, DataDog, or other monitoring
results = EncryptionHealthCheck.check
if results[:configuration_valid] && results[:sample_encrypt_decrypt]
  exit 0  # OK
else
  puts "Encryption health check failed: #{results}"
  exit 2  # Critical
end

Backup and Recovery

# Backup encryption keys securely
class EncryptionKeyBackup
  def self.backup_keys_to_vault
    keys = Familia.config.encryption_keys

    keys.each do |version, key|
      # Store in HashiCorp Vault, AWS KMS, etc.
      VaultClient.store_secret(
        path: "familia/encryption_keys/#{version}",
        data: { key: key },
        lease_duration: '8760h'  # 1 year
      )
    end
  end

  def self.restore_keys_from_vault
    versions = VaultClient.list_secrets('familia/encryption_keys/')

    restored_keys = {}
    versions.each do |version|
      secret = VaultClient.read_secret("familia/encryption_keys/#{version}")
      restored_keys[version.to_sym] = secret['key']
    end

    restored_keys
  end
end

---

Configuration Reference

Additional Configuration Options

Familia.configure do |config|
  config.encryption_keys = { v1: key, v2: new_key }
  config.current_key_version = :v2
  config.encryption_personalization = 'MyApp-2024'  # XChaCha20 only
end

# Validate configuration
Familia::Encryption.validate_configuration!

Error Types

• Familia::EncryptionError - General encryption/decryption failures
• Familia::SerializerError - Serialization safety violations
• SecurityError - Context validation or cleared data access

See Also

• Overview - Conceptual introduction to encrypted fields
• Technical Reference - Implementation details and advanced patterns
• Security Model Guide - Cryptographic design and threat model considerations
• Feature System Guide - Understanding Familia's feature architecture
• Implementation Guide - Production deployment and configuration patterns