Module: Familia::Utils

Included in:
Familia
Defined in:
lib/familia/utils.rb

Overview

Family-related utility methods

Constant Summary collapse

GLOB_SPECIAL_CHARACTERS =

Glob pattern characters escaped by #escape_glob: the wildcards, both brackets and the escape character itself. The documented patterns use the caret and the hyphen only inside brackets (h[^e]llo, h[a-b]llo), and no bracket expression can open once every opening bracket is escaped, so those two need no escape.

/[*?\[\]\\]/

Instance Method Summary collapse

Instance Method Details

#assert_replies_available!(operation, conn: nil) ⇒ void

This method returns an undefined value.

Refuses to run an operation that needs command replies inside a transaction or pipeline block.

Some methods must read a reply before they can finish: they branch on it, raise from it, issue follow-up commands based on it, iterate over it, or load records from it. Inside a transaction or pipeline every reply is a Redis::Future until the block completes, so such a method would act on a placeholder. It raises instead, before queueing anything.

A method that takes its connection as an argument passes it as +conn+. The check then looks at that connection instead of the current fiber: a transaction or pipeline connection (a MULTI or pipeline block's yield, or Familia.dbclient inside a Familia block) queues its commands, while a separate plain client answers even inside a block.

Parameters:

  • operation (String) —

    The method name for the error message, such as "HashKey#fetch"

  • conn (Redis, Redis::PipelinedConnection, nil) (defaults to: nil) —

    the connection the operation will issue its commands on, when it takes one

Raises:

  • (Familia::OperationModeError) —

    without +conn+, inside a transaction or pipeline; with +conn+, only when +conn+ is a transaction or pipeline connection



116
117
118
119
120
121
122
123
124
# File 'lib/familia/utils.rb', line 116

def assert_replies_available!(operation, conn: nil)
  queued = conn.nil? ? transaction_or_pipeline? : conn.is_a?(Redis::PipelinedConnection)
  return unless queued

  raise Familia::OperationModeError,
        "#{operation} cannot run inside a transaction or pipeline: it needs " \
        'command replies, which are Redis::Future objects until the block ' \
        'completes. Call it outside the block.'
end

#blank_text?(value) ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Whether +value.to_s+ is empty or holds only whitespace and NULs, the field values the indexes skip. String#strip raises ArgumentError or Encoding::CompatibilityError on a String whose bytes are not valid in its encoding, such as a field value given as BINARY "r\xFEle" and loaded back as UTF-8 with the same bytes, so such a String is checked as bytes. The characters strip removes are all ASCII, so the answer is the same for a valid String.

Parameters:

  • value (#to_s) —

    field value

Returns:

  • (Boolean)


152
153
154
155
156
# File 'lib/familia/utils.rb', line 152

def blank_text?(value)
  text = value.to_s
  text = text.b unless text.valid_encoding?
  text.strip.empty?
end

#dbkey(*val) ⇒ String

Creates a dbkey from given values

Parameters:

  • val (Array) —

    elements to join for the key

Returns:

  • (String) —

    dbkey



246
247
248
# File 'lib/familia/utils.rb', line 246

def dbkey(*val)
  join(*val)
end

#escape_glob(str) ⇒ String

Escapes +str+ so that it matches only itself inside a key pattern.

Use it for every literal part of a KEYS or SCAN MATCH pattern that is built from data (identifiers, field values, prefixes, suffixes, the delimiter) and leave only the intended wildcards unescaped. The Valkey and Redis KEYS documentation (https://valkey.io/commands/keys/ and https://redis.io/docs/latest/commands/keys/) both state: "Use \ to escape special characters if you want to match them verbatim." The Valkey SCAN documentation (https://valkey.io/commands/scan/) describes the MATCH option as a glob-style pattern that works "similarly to the behavior of the KEYS command".

The escaping works on the bytes: it puts an ASCII backslash before each ASCII special character and leaves every other byte as it is. So a String that is not valid in its encoding, such as an identifier with invalid UTF-8 bytes or binary data, is escaped without raising, and the result keeps the encoding of the input.

Examples:

Familia.escape_glob('c-*')                      #=> "c-\\*"
"#{Familia.escape_glob('company:c-*:dept:')}*"  #=> "company:c-\\*:dept:*"

Parameters:

  • str (#to_s) —

    the literal text

Returns:

  • (String) —

    the text with each glob special character escaped



282
283
284
285
# File 'lib/familia/utils.rb', line 282

def escape_glob(str)
  text = str.to_s
  text.b.gsub(GLOB_SPECIAL_CHARACTERS) { |char| "\\#{char}" }.force_encoding(text.encoding)
end

#join(*val) ⇒ String

Joins array elements with Familia delimiter

The result has the encoding #normalize_key_encoding gives it: UTF-8 when its bytes are valid UTF-8, BINARY otherwise. So a key built from an identifier is == to the same key read back from the server, whatever encoding the identifier was given in.

When the parts carry non-ASCII bytes in incompatible encodings, such as a non-ASCII class prefix (e.g. 'équipe') with an identifier that redis-client returned as BINARY because it is not valid UTF-8, the parts are joined as bytes.

Parameters:

  • val (Array) —

    elements to join

Returns:

  • (String) —

    joined string



216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
# File 'lib/familia/utils.rb', line 216

def join(*val)
  parts = val.compact
  key = begin
    parts.join(Familia.delim)
  rescue Encoding::CompatibilityError
    # Join as bytes, recursing into nested arrays as Array#join does, so
    # an empty nested array keeps its separators and the key does not
    # depend on the parts' encodings.
    delim = Familia.delim.to_s.b
    join_bytes = lambda do |list|
      list.map { |part| part.is_a?(Array) ? join_bytes.call(part) : part.to_s.b }.join(delim)
    end
    join_bytes.call(parts)
  end
  # Array#join returns a new String that nothing else holds, so it is
  # relabelled in place, as normalize_key_encoding would label a copy.
  key.force_encoding(Encoding::UTF_8) unless key.encoding == Encoding::UTF_8
  key.valid_encoding? ? key : key.force_encoding(Encoding::BINARY)
end

#legacy_json_encoded?(val) ⇒ Boolean

Detects the legacy JSON-encoded string format written by pre-2.10.0 reference collections (e.g. unique_index hashkeys stored "\"u1\"" instead of the raw "u1").

This is the single source of truth for the format check: the read path (DataType::Serialization#strip_legacy_json_encoding) strips such values, and the index introspection layer (IndexDescriptor#stale_format?) samples for them to flag indexes that need a rebuild. Side-effect free.

Parameters:

  • val (Object) —

    the raw stored value

Returns:

  • (Boolean) —

    true when val is a JSON-encoded string identifier



137
138
139
# File 'lib/familia/utils.rb', line 137

def legacy_json_encoded?(val)
  val.is_a?(String) && val.length > 2 && val.start_with?('"') && val.end_with?('"')
end

#loggable_text(value) ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns +value+ as text that a log line or error message can put next to any other UTF-8 text: +value.to_s+ itself when it is ASCII-only or valid UTF-8, otherwise a UTF-8 copy of its bytes with each byte that is not valid UTF-8 written as \xNN.

A key #join builds from an identifier that is not valid UTF-8 is BINARY, as is a key or member redis-client reads with such bytes. Interpolating one into a message that also holds non-ASCII UTF-8 text, such as a member, a field value or an exception message, raises Encoding::CompatibilityError. Messages pass keys, and any value they hold as is, through here.

Parameters:

  • value (#to_s) —

    key, identifier or other value to show

Returns:

  • (String) —

    ASCII-only or valid UTF-8 text



193
194
195
196
197
198
199
200
# File 'lib/familia/utils.rb', line 193

def loggable_text(value)
  text = value.to_s
  return text if text.ascii_only? || (text.encoding == Encoding::UTF_8 && text.valid_encoding?)

  text.dup.force_encoding(Encoding::UTF_8).scrub do |bytes|
    bytes.each_byte.map { |byte| format('\\x%02X', byte) }.join
  end
end

#normalize_key_encoding(str) ⇒ String

Returns +str+ in the encoding redis-client gives a key or member with the same bytes: UTF-8 when the bytes are valid UTF-8, BINARY otherwise. (redis-client labels each bulk string it reads as UTF-8 and relabels it BINARY unless it is valid.) The bytes are not changed.

Every key #join builds goes through here, so a key is one String value wherever it comes from. Before, a key built from a UTF-8 identifier with invalid bytes was UTF-8 while the same key built from the member read back was BINARY, and the two were neither == nor equal as Hash keys.

Parameters:

  • str (#to_s) —

    key, key part or identifier

Returns:

  • (String) —

    a UTF-8 String when the bytes are valid UTF-8, else a BINARY String; +str+ itself when it is already a valid UTF-8 String



172
173
174
175
176
# File 'lib/familia/utils.rb', line 172

def normalize_key_encoding(str)
  text = str.to_s
  text = text.dup.force_encoding(Encoding::UTF_8) unless text.encoding == Encoding::UTF_8
  text.valid_encoding? ? text : text.b
end

#now(current_time = Time.now) ⇒ Float

Returns current time in UTC as a float

Parameters:

  • current_time (Time) (defaults to: Time.now) —

    time object (default: current time)

Returns:

  • (Float) —

    time in seconds since epoch



298
299
300
# File 'lib/familia/utils.rb', line 298

def now(current_time = Time.now)
  current_time.utc.to_f
end

#now_in_μs ⇒ Integer Also known as: now_in_microseconds

Returns the current time in microseconds. This is used to measure the duration of Database commands.

Alias: now_in_microseconds

Returns:

  • (Integer) —

    The current time in microseconds.



308
309
310
# File 'lib/familia/utils.rb', line 308

def now_in_μs
  Process.clock_gettime(Process::CLOCK_MONOTONIC, :microsecond)
end

#positive?(ret) ⇒ Boolean, Redis::Future

Future-aware positive check for Redis command return values.

For commands where 0 means "nothing happened" and positive means "something happened" (e.g., EXISTS, DEL, LINSERT, TTL checks).

Examples:

Check if key exists

ret = dbclient.exists(key)
Familia.positive?(ret)  #=> true if key exists

Check TTL inside pipeline

pipelined do
  ret = dbclient.ttl(key)
  Familia.positive?(ret)  #=> Redis::Future (passthrough)
end

Parameters:

  • ret (Integer, Redis::Future) —

    The return value from a Redis command

Returns:

  • (Boolean, Redis::Future) —

    true/false for concrete values, passthrough for Futures



55
56
57
# File 'lib/familia/utils.rb', line 55

def positive?(ret)
  ret.is_a?(Redis::Future) ? ret : ret.positive?
end

#pretty_path(filepath) ⇒ Pathname, ...

Converts an absolute file path to a path relative to the current working directory. This simplifies logging and error reporting by showing only the relevant parts of file paths instead of lengthy absolute paths.

Examples:

Using current directory as base

Utils.pretty_path("/home/dev/project/lib/config.rb") # => "lib/config.rb"

Path outside current directory

Utils.pretty_path("/etc/hosts") # => "hosts"

Nil input

Utils.pretty_path(nil) # => nil

Parameters:

  • filepath (String, Pathname) —

    The file path to convert

Returns:

  • (Pathname, String, nil) —

    A relative path from current directory, basename if path goes outside current directory, or nil if filepath is nil

See Also:

  • Ruby standard library documentation


358
359
360
361
362
363
364
365
366
367
368
# File 'lib/familia/utils.rb', line 358

def pretty_path(filepath)
  return nil if filepath.nil?

  basepath = Dir.pwd
  relative_path = Pathname.new(filepath).relative_path_from(basepath)
  if relative_path.to_s.start_with?('..')
    File.basename(filepath)
  else
    relative_path
  end
end

#pretty_stack(skip: 1, limit: 5) ⇒ String

Formats a stack trace with pretty file paths for improved readability

Examples:

Utils.pretty_stack(limit: 10)
# => "lib/models/user.rb:25:in `save'\n lib/controllers/app.rb:45:in `create'"

Parameters:

  • limit (Integer) (defaults to: 5) —

    Maximum number of stack frames to include (default: 3)

Returns:

  • (String) —

    Formatted stack trace with relative paths joined by newlines



378
379
380
# File 'lib/familia/utils.rb', line 378

def pretty_stack(skip: 1, limit: 5)
  caller(skip..(skip + limit + 1)).first(limit).map { |frame| pretty_path(frame) }.join("\n")
end

#qstamp(quantum = 10.minutes, pattern: nil, time: nil) ⇒ Integer, String

A quantized timestamp

Examples:

Familia.qstamp # Returns an integer timestamp rounded to the nearest 10 minutes

Familia.qstamp(1.hour)  # Uses 1 hour quantum
Familia.qstamp(10.minutes, pattern: '%H:%M')  # Returns a formatted string like "12:30"
Familia.qstamp(10.minutes, time: 1302468980)  # Quantizes the given Unix timestamp
Familia.qstamp(10.minutes, time: Familia.now)  # Quantizes the given Time object
Familia.qstamp(10.minutes, pattern: '%H:%M', time: 1302468980)  # Formats a specific time

Parameters:

  • quantum (Integer) (defaults to: 10.minutes) —

    The time quantum in seconds (default: 10 minutes).

  • pattern (String, nil) (defaults to: nil) —

    The strftime pattern to format the timestamp.

  • time (Integer, Float, Time, nil) (defaults to: nil) —

    A specific time to quantize (default: current time).

Returns:

  • (Integer, String) —

    A unix timestamp or formatted timestamp string.



327
328
329
330
331
332
333
334
335
336
337
338
# File 'lib/familia/utils.rb', line 327

def qstamp(quantum = 10.minutes, pattern: nil, time: nil)
  time ||= Familia.now
  time = time.to_f if time.is_a?(Time)

  rounded = time - (time % quantum)

  if pattern
    Time.at(rounded).utc.strftime(pattern)
  else
    Time.at(rounded).utc.to_i
  end
end

#serverid(uri) ⇒ Object

Gets server ID without DB component for pool identification



288
289
290
291
292
293
# File 'lib/familia/utils.rb', line 288

def serverid(uri)
  # Create a copy of URI without DB for server identification
  uri = uri.dup
  uri.db = nil
  uri.serverid
end

#split(val) ⇒ Array

Splits a string using Familia delimiter

Parameters:

  • val (String) —

    string to split

Returns:

  • (Array) —

    split elements



239
240
241
# File 'lib/familia/utils.rb', line 239

def split(val)
  val.split(Familia.delim)
end

#success?(ret) ⇒ Boolean, Redis::Future

Future-aware success check for Redis command return values.

Redis commands like HSET return 0 (updated existing) or 1 (created new). Both are success states. Inside a pipeline or transaction, the return value is a Redis::Future which cannot be inspected until the block completes.

Examples:

Normal usage

ret = dbclient.hset(key, field, value)
Familia.success?(ret)  #=> true (for 0 or 1)

Inside pipeline

pipelined do
  ret = dbclient.hset(key, field, value)
  Familia.success?(ret)  #=> Redis::Future (passthrough)
end

Parameters:

  • ret (Integer, Redis::Future) —

    The return value from a Redis command

Returns:

  • (Boolean, Redis::Future) —

    true/false for concrete values, passthrough for Futures



32
33
34
# File 'lib/familia/utils.rb', line 32

def success?(ret)
  ret.is_a?(Redis::Future) ? ret : (ret.zero? || ret.positive?)
end

#transaction_or_pipeline? ⇒ Boolean

Whether the current fiber is inside a Familia transaction or pipeline block. Commands issued there are queued, and each returns a Redis::Future instead of its reply.

Returns:

  • (Boolean)


87
88
89
# File 'lib/familia/utils.rb', line 87

def transaction_or_pipeline?
  !(Fiber[:familia_transaction].nil? && Fiber[:familia_pipeline].nil?)
end

#transform_reply(ret) {|reply| ... } ⇒ Object, Redis::Future

Future-aware conversion of a Redis command reply.

Yields a concrete reply to the block and returns the block's result. Returns a Redis::Future untouched: inside a pipeline or transaction the reply is not available until the block completes, so the conversion cannot run. The Future then resolves to the reply as redis-rb returns it, without the block's conversion. See docs/reference/transaction_safety.md.

Examples:

Deserialize a reply

Familia.transform_reply(dbclient.hvals(key)) do |vals|
  vals.map { |v| deserialize_value(v) }
end

Parameters:

  • ret (Object, Redis::Future) —

    The return value from a Redis command

Yield Parameters:

  • reply (Object) —

    The concrete reply

Returns:

  • (Object, Redis::Future) —

    The block's result for concrete values, passthrough for Futures



77
78
79
# File 'lib/familia/utils.rb', line 77

def transform_reply(ret)
  ret.is_a?(Redis::Future) ? ret : yield(ret)
end