Module: Familia::Utils
- Included in:
- Familia
- Defined in:
- lib/familia/utils.rb
Overview
Family-related utility methods
Constant Summary collapse
- GLOB_SPECIAL_CHARACTERS =
Glob pattern characters escaped by #escape_glob: the wildcards, both brackets and the escape character itself. The documented patterns use the caret and the hyphen only inside brackets (h[^e]llo, h[a-b]llo), and no bracket expression can open once every opening bracket is escaped, so those two need no escape.
/[*?\[\]\\]/
Instance Method Summary collapse
-
#assert_replies_available!(operation, conn: nil) ⇒ void
Refuses to run an operation that needs command replies inside a transaction or pipeline block.
-
#blank_text?(value) ⇒ Boolean
private
Whether +value.to_s+ is empty or holds only whitespace and NULs, the field values the indexes skip.
-
#dbkey(*val) ⇒ String
Creates a dbkey from given values.
-
#escape_glob(str) ⇒ String
Escapes +str+ so that it matches only itself inside a key pattern.
-
#join(*val) ⇒ String
Joins array elements with Familia delimiter.
-
#legacy_json_encoded?(val) ⇒ Boolean
Detects the legacy JSON-encoded string format written by pre-2.10.0 reference collections (e.g. unique_index hashkeys stored
"\"u1\""instead of the raw"u1"). -
#loggable_text(value) ⇒ String
private
Returns +value+ as text that a log line or error message can put next to any other UTF-8 text: +value.to_s+ itself when it is ASCII-only or valid UTF-8, otherwise a UTF-8 copy of its bytes with each byte that is not valid UTF-8 written as \xNN.
-
#normalize_key_encoding(str) ⇒ String
Returns +str+ in the encoding redis-client gives a key or member with the same bytes: UTF-8 when the bytes are valid UTF-8, BINARY otherwise.
-
#now(current_time = Time.now) ⇒ Float
Returns current time in UTC as a float.
-
#now_in_μs ⇒ Integer
(also: #now_in_microseconds)
Returns the current time in microseconds.
-
#positive?(ret) ⇒ Boolean, Redis::Future
Future-aware positive check for Redis command return values.
-
#pretty_path(filepath) ⇒ Pathname, ...
Converts an absolute file path to a path relative to the current working directory.
-
#pretty_stack(skip: 1, limit: 5) ⇒ String
Formats a stack trace with pretty file paths for improved readability.
-
#qstamp(quantum = 10.minutes, pattern: nil, time: nil) ⇒ Integer, String
A quantized timestamp.
-
#serverid(uri) ⇒ Object
Gets server ID without DB component for pool identification.
-
#split(val) ⇒ Array
Splits a string using Familia delimiter.
-
#success?(ret) ⇒ Boolean, Redis::Future
Future-aware success check for Redis command return values.
-
#transaction_or_pipeline? ⇒ Boolean
Whether the current fiber is inside a Familia transaction or pipeline block.
-
#transform_reply(ret) {|reply| ... } ⇒ Object, Redis::Future
Future-aware conversion of a Redis command reply.
Instance Method Details
#assert_replies_available!(operation, conn: nil) ⇒ void
This method returns an undefined value.
Refuses to run an operation that needs command replies inside a transaction or pipeline block.
Some methods must read a reply before they can finish: they branch on it, raise from it, issue follow-up commands based on it, iterate over it, or load records from it. Inside a transaction or pipeline every reply is a Redis::Future until the block completes, so such a method would act on a placeholder. It raises instead, before queueing anything.
A method that takes its connection as an argument passes it as +conn+. The check then looks at that connection instead of the current fiber: a transaction or pipeline connection (a MULTI or pipeline block's yield, or Familia.dbclient inside a Familia block) queues its commands, while a separate plain client answers even inside a block.
116 117 118 119 120 121 122 123 124 |
# File 'lib/familia/utils.rb', line 116 def assert_replies_available!(operation, conn: nil) queued = conn.nil? ? transaction_or_pipeline? : conn.is_a?(Redis::PipelinedConnection) return unless queued raise Familia::OperationModeError, "#{operation} cannot run inside a transaction or pipeline: it needs " \ 'command replies, which are Redis::Future objects until the block ' \ 'completes. Call it outside the block.' end |
#blank_text?(value) ⇒ Boolean
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
Whether +value.to_s+ is empty or holds only whitespace and NULs, the field values the indexes skip. String#strip raises ArgumentError or Encoding::CompatibilityError on a String whose bytes are not valid in its encoding, such as a field value given as BINARY "r\xFEle" and loaded back as UTF-8 with the same bytes, so such a String is checked as bytes. The characters strip removes are all ASCII, so the answer is the same for a valid String.
152 153 154 155 156 |
# File 'lib/familia/utils.rb', line 152 def blank_text?(value) text = value.to_s text = text.b unless text.valid_encoding? text.strip.empty? end |
#dbkey(*val) ⇒ String
Creates a dbkey from given values
246 247 248 |
# File 'lib/familia/utils.rb', line 246 def dbkey(*val) join(*val) end |
#escape_glob(str) ⇒ String
Escapes +str+ so that it matches only itself inside a key pattern.
Use it for every literal part of a KEYS or SCAN MATCH pattern that is
built from data (identifiers, field values, prefixes, suffixes, the
delimiter) and leave only the intended wildcards unescaped. The Valkey
and Redis KEYS documentation (https://valkey.io/commands/keys/ and
https://redis.io/docs/latest/commands/keys/) both state: "Use \ to
escape special characters if you want to match them verbatim." The
Valkey SCAN documentation (https://valkey.io/commands/scan/) describes
the MATCH option as a glob-style pattern that works "similarly to the
behavior of the KEYS command".
The escaping works on the bytes: it puts an ASCII backslash before each ASCII special character and leaves every other byte as it is. So a String that is not valid in its encoding, such as an identifier with invalid UTF-8 bytes or binary data, is escaped without raising, and the result keeps the encoding of the input.
282 283 284 285 |
# File 'lib/familia/utils.rb', line 282 def escape_glob(str) text = str.to_s text.b.gsub(GLOB_SPECIAL_CHARACTERS) { |char| "\\#{char}" }.force_encoding(text.encoding) end |
#join(*val) ⇒ String
Joins array elements with Familia delimiter
The result has the encoding #normalize_key_encoding gives it: UTF-8 when its bytes are valid UTF-8, BINARY otherwise. So a key built from an identifier is == to the same key read back from the server, whatever encoding the identifier was given in.
When the parts carry non-ASCII bytes in incompatible encodings, such as a non-ASCII class prefix (e.g. 'équipe') with an identifier that redis-client returned as BINARY because it is not valid UTF-8, the parts are joined as bytes.
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 |
# File 'lib/familia/utils.rb', line 216 def join(*val) parts = val.compact key = begin parts.join(Familia.delim) rescue Encoding::CompatibilityError # Join as bytes, recursing into nested arrays as Array#join does, so # an empty nested array keeps its separators and the key does not # depend on the parts' encodings. delim = Familia.delim.to_s.b join_bytes = lambda do |list| list.map { |part| part.is_a?(Array) ? join_bytes.call(part) : part.to_s.b }.join(delim) end join_bytes.call(parts) end # Array#join returns a new String that nothing else holds, so it is # relabelled in place, as normalize_key_encoding would label a copy. key.force_encoding(Encoding::UTF_8) unless key.encoding == Encoding::UTF_8 key.valid_encoding? ? key : key.force_encoding(Encoding::BINARY) end |
#legacy_json_encoded?(val) ⇒ Boolean
Detects the legacy JSON-encoded string format written by pre-2.10.0
reference collections (e.g. unique_index hashkeys stored "\"u1\""
instead of the raw "u1").
This is the single source of truth for the format check: the read path (DataType::Serialization#strip_legacy_json_encoding) strips such values, and the index introspection layer (IndexDescriptor#stale_format?) samples for them to flag indexes that need a rebuild. Side-effect free.
137 138 139 |
# File 'lib/familia/utils.rb', line 137 def legacy_json_encoded?(val) val.is_a?(String) && val.length > 2 && val.start_with?('"') && val.end_with?('"') end |
#loggable_text(value) ⇒ String
This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.
Returns +value+ as text that a log line or error message can put next to any other UTF-8 text: +value.to_s+ itself when it is ASCII-only or valid UTF-8, otherwise a UTF-8 copy of its bytes with each byte that is not valid UTF-8 written as \xNN.
A key #join builds from an identifier that is not valid UTF-8 is BINARY, as is a key or member redis-client reads with such bytes. Interpolating one into a message that also holds non-ASCII UTF-8 text, such as a member, a field value or an exception message, raises Encoding::CompatibilityError. Messages pass keys, and any value they hold as is, through here.
193 194 195 196 197 198 199 200 |
# File 'lib/familia/utils.rb', line 193 def loggable_text(value) text = value.to_s return text if text.ascii_only? || (text.encoding == Encoding::UTF_8 && text.valid_encoding?) text.dup.force_encoding(Encoding::UTF_8).scrub do |bytes| bytes.each_byte.map { |byte| format('\\x%02X', byte) }.join end end |
#normalize_key_encoding(str) ⇒ String
Returns +str+ in the encoding redis-client gives a key or member with the same bytes: UTF-8 when the bytes are valid UTF-8, BINARY otherwise. (redis-client labels each bulk string it reads as UTF-8 and relabels it BINARY unless it is valid.) The bytes are not changed.
Every key #join builds goes through here, so a key is one String value wherever it comes from. Before, a key built from a UTF-8 identifier with invalid bytes was UTF-8 while the same key built from the member read back was BINARY, and the two were neither == nor equal as Hash keys.
172 173 174 175 176 |
# File 'lib/familia/utils.rb', line 172 def normalize_key_encoding(str) text = str.to_s text = text.dup.force_encoding(Encoding::UTF_8) unless text.encoding == Encoding::UTF_8 text.valid_encoding? ? text : text.b end |
#now(current_time = Time.now) ⇒ Float
Returns current time in UTC as a float
298 299 300 |
# File 'lib/familia/utils.rb', line 298 def now(current_time = Time.now) current_time.utc.to_f end |
#now_in_μs ⇒ Integer Also known as: now_in_microseconds
Returns the current time in microseconds. This is used to measure the duration of Database commands.
Alias: now_in_microseconds
308 309 310 |
# File 'lib/familia/utils.rb', line 308 def now_in_μs Process.clock_gettime(Process::CLOCK_MONOTONIC, :microsecond) end |
#positive?(ret) ⇒ Boolean, Redis::Future
Future-aware positive check for Redis command return values.
For commands where 0 means "nothing happened" and positive means "something happened" (e.g., EXISTS, DEL, LINSERT, TTL checks).
55 56 57 |
# File 'lib/familia/utils.rb', line 55 def positive?(ret) ret.is_a?(Redis::Future) ? ret : ret.positive? end |
#pretty_path(filepath) ⇒ Pathname, ...
Converts an absolute file path to a path relative to the current working directory. This simplifies logging and error reporting by showing only the relevant parts of file paths instead of lengthy absolute paths.
358 359 360 361 362 363 364 365 366 367 368 |
# File 'lib/familia/utils.rb', line 358 def pretty_path(filepath) return nil if filepath.nil? basepath = Dir.pwd relative_path = Pathname.new(filepath).relative_path_from(basepath) if relative_path.to_s.start_with?('..') File.basename(filepath) else relative_path end end |
#pretty_stack(skip: 1, limit: 5) ⇒ String
Formats a stack trace with pretty file paths for improved readability
378 379 380 |
# File 'lib/familia/utils.rb', line 378 def pretty_stack(skip: 1, limit: 5) caller(skip..(skip + limit + 1)).first(limit).map { |frame| pretty_path(frame) }.join("\n") end |
#qstamp(quantum = 10.minutes, pattern: nil, time: nil) ⇒ Integer, String
A quantized timestamp
327 328 329 330 331 332 333 334 335 336 337 338 |
# File 'lib/familia/utils.rb', line 327 def qstamp(quantum = 10.minutes, pattern: nil, time: nil) time ||= Familia.now time = time.to_f if time.is_a?(Time) rounded = time - (time % quantum) if pattern Time.at(rounded).utc.strftime(pattern) else Time.at(rounded).utc.to_i end end |
#serverid(uri) ⇒ Object
Gets server ID without DB component for pool identification
288 289 290 291 292 293 |
# File 'lib/familia/utils.rb', line 288 def serverid(uri) # Create a copy of URI without DB for server identification uri = uri.dup uri.db = nil uri.serverid end |
#split(val) ⇒ Array
Splits a string using Familia delimiter
239 240 241 |
# File 'lib/familia/utils.rb', line 239 def split(val) val.split(Familia.delim) end |
#success?(ret) ⇒ Boolean, Redis::Future
Future-aware success check for Redis command return values.
Redis commands like HSET return 0 (updated existing) or 1 (created new). Both are success states. Inside a pipeline or transaction, the return value is a Redis::Future which cannot be inspected until the block completes.
32 33 34 |
# File 'lib/familia/utils.rb', line 32 def success?(ret) ret.is_a?(Redis::Future) ? ret : (ret.zero? || ret.positive?) end |
#transaction_or_pipeline? ⇒ Boolean
Whether the current fiber is inside a Familia transaction or pipeline block. Commands issued there are queued, and each returns a Redis::Future instead of its reply.
87 88 89 |
# File 'lib/familia/utils.rb', line 87 def transaction_or_pipeline? !(Fiber[:familia_transaction].nil? && Fiber[:familia_pipeline].nil?) end |
#transform_reply(ret) {|reply| ... } ⇒ Object, Redis::Future
Future-aware conversion of a Redis command reply.
Yields a concrete reply to the block and returns the block's result. Returns a Redis::Future untouched: inside a pipeline or transaction the reply is not available until the block completes, so the conversion cannot run. The Future then resolves to the reply as redis-rb returns it, without the block's conversion. See docs/reference/transaction_safety.md.
77 78 79 |
# File 'lib/familia/utils.rb', line 77 def transform_reply(ret) ret.is_a?(Redis::Future) ? ret : yield(ret) end |