External Identifiers Guide

💡 Quick Reference

Generate deterministic, public-facing identifiers from internal objid:

class User < Familia::Horreum
  feature :object_identifier
  feature :external_identifier
  field :email, :name
end

Overview

The External Identifier feature provides deterministic, public-facing identifiers that are securely derived from internal objid values. These shorter, URL-safe identifiers are perfect for APIs, public URLs, and external integrations where you want to hide internal implementation details while maintaining deterministic lookups.

Why Use External Identifiers?

Security Through Obscurity: Hide internal UUID structure and potential timestamp information from public interfaces.

URL-Friendly: Generate compact, base36-encoded identifiers safe for use in URLs and APIs.

Deterministic Generation: Same objid always produces the same extid for reliable lookups.

Bidirectional Mapping: Automatic lookup tables enable finding objects by external ID.

Customizable Format: Configure prefix and format patterns to match your naming conventions.

Dependencies

External identifiers require the Object Identifier feature:

class MyModel < Familia::Horreum
  feature :object_identifier    # Required first
  feature :external_identifier # Then enable external IDs
end

Basic Usage

Default External Identifier

class User < Familia::Horreum
  feature :object_identifier
  feature :external_identifier

  field :email, :name
end

user = User.new(email: 'alice@example.com', name: 'Alice')
user.save

# Internal identifier (long, detailed)
user.objid  # => "01234567-89ab-cdef-1234-567890abcdef"

# External identifier (short, public-safe, 29 chars total: 4 prefix + 25 ID)
user.extid  # => "ext_abc123def456ghi789jkl012" (deterministic from objid)

# Always produces the same extid from the same objid
user2 = User.new(objid: user.objid, email: 'alice@example.com')
user2.extid  # => "ext_abc123def456ghi789jkl012" (identical)

Finding by External ID

# Create and save user
user = User.new(email: 'bob@example.com')
user.save
puts user.extid  # => "ext_xyz789abc123def456ghi123"

# Find by external identifier
found_user = User.find_by_extid('ext_xyz789abc123def456ghi123')
found_user.email  # => "bob@example.com"

# Returns nil if not found
missing = User.find_by_extid('ext_nonexistent')  # => nil

Long-form Methods

# Aliases for clarity
user.external_identifier           # Same as user.extid
user.external_identifier = 'new'   # Same as user.extid = 'new'

Custom Format Templates

Custom Prefix

class Customer < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, format: 'cust_%{id}'

  field :company_name, :email
end

customer = Customer.new(company_name: 'Acme Corp')
customer.save
customer.extid  # => "cust_abc123def456ghi789jkl012" (30 chars: 5 prefix + 25 ID)

Hyphen Separator

class APIKey < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, format: 'api-%{id}'

  field :name, :permissions
end

key = APIKey.new(name: 'Production API Key')
key.save
key.extid  # => "api-abc123def456ghi789jkl012" (29 chars: 4 prefix + 25 ID)

Version Prefix

class Resource < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, format: 'v2/%{id}'

  field :resource_type, :data
end

resource = Resource.new(resource_type: 'document')
resource.save
resource.extid  # => "v2/abc123def456ghi789jkl012" (28 chars: 3 prefix + 25 ID)

No Prefix

class SimpleModel < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, format: '%{id}'

  field :data
end

model = SimpleModel.new(data: 'test')
model.save
model.extid  # => "abc123def456ghi789jkl012" (25 chars: just the ID)

Security Model

Deterministic but Obscured

External identifiers use cryptographic techniques to obscure the relationship between objid and extid:

1. SHA-256 Seeding: The objid is hashed to create a uniform seed
2. PRNG Function: A pseudorandom number generator acts as a deterministic transformation
3. Base36 Encoding: Result is encoded as URL-safe, compact string

This ensures:

• Same objid always produces same extid (deterministic)
• No discernible mathematical correlation between objid and extid (secure)
• Cannot reverse-engineer objid from extid (one-way)

Generated Format Details

The ID portion (after any prefix) is always:

• Length: Exactly 25 characters
• Characters: Lowercase alphanumeric only (0-9a-z)
• Encoding: Base36 representation of 128-bit random data
• Pattern: /[0-9a-z]{25}/

Provenance Validation

External identifiers can only be derived from objid values with known provenance:

# ✅ Valid - objid from ObjectIdentifier feature
user = User.new  # objid generated by UUID v7
user.extid       # => Works fine

# ❌ Invalid - objid of unknown origin
user = User.new
user.instance_variable_set(:@objid, 'unknown-source-id')
user.extid  # => ExternalIdentifierError: objid provenance unknown

Automatic Lookup Management

Bidirectional Mapping

The feature automatically maintains lookup tables:

class Product < Familia::Horreum
  feature :object_identifier
  feature :external_identifier

  field :name, :price
end

product = Product.new(name: 'Widget', price: 29.99)
product.save

# Lookup table is automatically updated
Product.extid_lookup.class    # => Familia::DataType::HashKey
Product.extid_lookup.length   # => 1

# Mapping: extid -> objid
extid = product.extid
Product.extid_lookup[extid]   # => product.objid

Cleanup on Destroy

product.destroy!

# Lookup entry is automatically cleaned up
Product.extid_lookup[extid]   # => nil

Implementation Details

Lazy Generation

External identifiers are generated lazily when first accessed:

user = User.new(email: 'test@example.com')

# extid is not generated until accessed
user.instance_variable_get(:@extid)  # => nil

# First access triggers generation
user.extid  # => "ext_abc123..." (generated from objid)

# Subsequent access returns cached value
user.extid  # => "ext_abc123..." (same value)

Preservation During Initialization

Values provided during initialization are preserved:

# Loading from database with existing extid
user = User.new(
  objid: existing_objid,
  extid: existing_extid,
  email: 'test@example.com'
)

# Existing extid is preserved, not regenerated
user.extid  # => existing_extid (not derived)

Database Persistence

External identifiers are automatically stored in the object's hash:

user = User.new(email: 'test@example.com')
user.save

# extid is stored alongside other fields
user.to_h  # => { objid: "...", extid: "ext_abc123...", email: "..." }

Error Handling

Missing ObjectIdentifier Feature

class BadModel < Familia::Horreum
  feature :external_identifier  # Missing :object_identifier dependency
end

# => Error during class definition

Unknown Provenance

user = User.new
# Manually set objid with unknown provenance
user.instance_variable_set(:@objid, 'manual-id')

user.extid
# => ExternalIdentifierError: Cannot derive external identifier: objid provenance unknown

Invalid Objid Format

# For custom generators, objid must be hexadecimal
user.instance_variable_set(:@objid, 'not-hex-format!')
user.extid
# => ExternalIdentifierError: Cannot normalize objid from custom generator

Testing Strategies

Basic Functionality

class ExternalIdentifierTest < Minitest::Test
  def setup
    @user = User.new(email: 'test@example.com')
    @user.save
  end

  def test_deterministic_generation
    extid1 = @user.extid
    extid2 = @user.extid

    assert_equal extid1, extid2
  end

  def test_same_objid_produces_same_extid
    user2 = User.new(objid: @user.objid, email: 'other@example.com')

    assert_equal @user.extid, user2.extid
  end

  def test_find_by_extid
    found_user = User.find_by_extid(@user.extid)

    assert_equal @user.objid, found_user.objid
    assert_equal @user.email, found_user.email
  end

  def test_format_validation
    # Default format: 'ext_' prefix + 25 character base36 ID
    assert_match(/\Aext_[0-9a-z]{25}\z/, @user.extid)
  end
end

Custom Format Testing

class CustomFormatTest < Minitest::Test
  def test_custom_prefix
    customer = Customer.new(company_name: 'Test Corp')
    customer.save

    # Custom format: 'cust_' prefix + 25 character base36 ID
    assert_match(/\Acust_[0-9a-z]{25}\z/, customer.extid)
  end

  def test_no_prefix_format
    model = SimpleModel.new(data: 'test')
    model.save

    # No prefix format: just 25 character base36 ID
    assert_match(/\A[0-9a-z]{25}\z/, model.extid)
  end

  def test_hyphen_separator
    key = APIKey.new(name: 'Test Key')
    key.save

    # Hyphen format: 'api-' prefix + 25 character base36 ID
    assert_match(/\Aapi-[0-9a-z]{25}\z/, key.extid)
  end
end

Security Testing

class SecurityTest < Minitest::Test
  def test_no_correlation_between_objid_and_extid
    users = 10.times.map do
      User.new(email: "user#{rand(1000)}@example.com").tap(&:save)
    end

    objids = users.map(&:objid).sort
    extids = users.map(&:extid).sort

    # Sorting should not preserve any correlation
    # (This is a statistical test - might rarely fail by chance)
    correlations = objids.zip(extids).count { |objid, extid|
      objid[0..2] == extid[-3..-1] # Check if patterns match
    }

    assert correlations < 3, "Too many correlations detected: #{correlations}"
  end

  def test_cannot_reverse_engineer_objid
    user = User.new(email: 'test@example.com')
    user.save

    # Should not be able to derive objid from extid
    # This test ensures no obvious mathematical relationship
    refute_match user.objid[0..8], user.extid
  end

  def test_base36_format_consistency
    user = User.new(email: 'test@example.com')
    user.save

    extid_suffix = user.extid.sub(/^ext_/, '')

    # Must be exactly 25 characters of base36
    assert_equal 25, extid_suffix.length
    assert_match(/\A[0-9a-z]{25}\z/, extid_suffix)
  end
end

Performance Considerations

Lookup Table Size

Each class with external identifiers maintains its own lookup table:

# Monitor lookup table growth
puts User.extid_lookup.length        # Number of extid mappings
puts Customer.extid_lookup.length    # Separate table per class

Batch Operations

For bulk operations, consider the lookup table overhead:

# Each save updates the lookup table
1000.times do |i|
  User.new(email: "user#{i}@example.com").save
end

# Consider batch cleanup if needed
# User.extid_lookup.clear if rebuilding from scratch

Integration Patterns

API Controllers

class UsersController < ApplicationController
  # Use external IDs in URLs
  def show
    @user = User.find_by_extid(params[:id])

    if @user.nil?
      render json: { error: 'User not found' }, status: 404
    else
      render json: user_json(@user)
    end
  end

  private

  def user_json(user)
    {
      id: user.extid,  # Use extid in API responses
      email: user.email,
      name: user.name
    }
  end
end

URL Generation

# In views/helpers
def user_path(user)
  "/users/#{user.extid}"  # Short, clean URLs
end

# Instead of:
# "/users/01234567-89ab-cdef-1234-567890abcdef"
#
# Generate:
# "/users/ext_abc123def456ghi789jkl012"

External System Integration

class WebhookHandler
  def handle_external_callback(payload)
    external_id = payload['user_id']  # From external system

    user = User.find_by_extid(external_id)
    return unless user

    # Process callback for identified user
    process_user_event(user, payload)
  end
end

Best Practices

Use in Public APIs

# ✅ Good - external IDs in public API
GET /api/users/ext_abc123def456ghi789jkl012

# ❌ Avoid - internal UUIDs in public API
GET /api/users/01234567-89ab-cdef-1234-567890abcdef

Consistent Format Across Models

class User < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, format: 'user_%{id}'
end

class Order < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, format: 'order_%{id}'
end

class Product < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, format: 'prod_%{id}'
end

Error Handling in Controllers

def find_user_by_external_id(extid)
  User.find_by_extid(extid) || raise(ActiveRecord::RecordNotFound)
rescue Familia::ExternalIdentifierError => e
  Rails.logger.warn "Invalid external ID format for '#{extid}': #{e.message}"
  raise(ActiveRecord::RecordNotFound)
end

Troubleshooting

Extid Returns Nil

Check that object has valid objid and required features:

user = User.new(email: 'test@example.com')

# ❌ No objid yet - extid will be nil
user.extid  # => nil

# ✅ Save first to generate objid
user.save
user.extid  # => "ext_abc123..."

Lookup Not Working

Ensure object was saved to populate lookup table:

user = User.new(email: 'test@example.com')
extid = user.extid  # Generates extid but doesn't save lookup

User.find_by_extid(extid)  # => nil (lookup not saved)

user.save  # Saves lookup mapping
User.find_by_extid(extid)  # => user (now works)

Format Not Applied

Check feature options syntax:

# ❌ Wrong - options after comma
class User < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, { format: 'user_%{id}' }
end

# ✅ Correct - options as keyword arguments
class User < Familia::Horreum
  feature :object_identifier
  feature :external_identifier, format: 'user_%{id}'
end

Invalid Format Regex

When testing external ID format, use exact character counts:

# ❌ Wrong - uses + quantifier
extid.match(/\Aext_[0-9a-z]+\z/)

# ✅ Correct - uses exact count {25}
extid.match(/\Aext_[0-9a-z]{25}\z/)

See Also

• Object Identifiers - Required dependency for external identifiers
• Feature System - Understanding Familia's feature architecture
• Relationships - Using external IDs with relationships