hyped.core.ops.string module

This module defines a collection of data processors that implement common string operations.

Each processor is designed to handle a specific string transformation or query, such as finding substrings, replacing patterns, splitting strings, or trimming characters. These processors are intended for use in data processing pipelines, where they can be applied in a batched and efficient manner using Apache Arrow as the backend.

These processors are registered as methods on the StringFeature class, allowing them to be applied directly to string features. The operations are batched and optimized for high performance using Arrow, making them suitable for large-scale data processing tasks.

class hyped.core.ops.string.StringAdd(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringAddConfig]

Data processor for string concatenation.

process(ctx: RunContext, a: StringFeature | str | list[str] | StringScalar | StringArray, b: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Concatenates two strings element-wise.

Parameters:

  • ctx (RunContext) – The execution context.

  • a (String) – The first string column.

  • b (String) – The second string column.

Returns:

The concatenated string column.

Return type:

StringFeature

class hyped.core.ops.string.StringAddConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringAdd processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringCapitalize(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringCapitalizeConfig]

Data processor for capitalizing strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Capitalizes the first letter of each string in the column.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to capitalize.

Returns:

A column of strings with the first letter capitalized.

Return type:

StringFeature

class hyped.core.ops.string.StringCapitalizeConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringCapitalize processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringContains(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringContainsConfig]

Data processor for finding the position of a pattern in strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → BoolFeature[source]

Check if the pattern is contained in each string.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to search.

Returns:

A column of boolean values indicating whether the string contains the pattern.

Return type:

BoolFeature

class hyped.core.ops.string.StringContainsConfig(*, pattern: str)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringContains processor.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pattern: str

The pattern to match in each string.

class hyped.core.ops.string.StringEndsWith(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringEndsWithConfig]

Data processor for checking if strings end with a given pattern.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → BoolFeature[source]

Checks if the input string column ends with the specified pattern.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to check.

Returns:

A column of boolean values indicating whether each string ends with the pattern.

Return type:

BoolFeature

class hyped.core.ops.string.StringEndsWithConfig(*, pattern: str)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringEndsWith processor.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pattern: str

The pattern to check for at the end of each string.

class hyped.core.ops.string.StringFind(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringFindConfig]

Data processor for finding the position of a pattern in strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → Int32Feature[source]

Finds the position of the specified pattern in each string.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to search.

Returns:

A column of integer values representing the position of the pattern in each string.

Return type:

Int32Feature

class hyped.core.ops.string.StringFindConfig(*, pattern: str)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringFind processor.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pattern: str

The pattern to search for in each string.

class hyped.core.ops.string.StringFormat(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringFormatConfig]

Data processor for string formatting.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray, **kwargs: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Formats strings in the string column using keyword arguments.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to format.

  • **kwargs (String) – Keyword arguments for formatting.

Returns:

The formatted string column.

Return type:

StringFeature

class hyped.core.ops.string.StringFormatConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringFormat processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringGetSlice(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringGetSliceConfig]

Data processor for slicing strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Extracts slices of strings according to the configuration.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to slice.

Returns:

The sliced string column.

Return type:

StringFeature

class hyped.core.ops.string.StringGetSliceConfig(*, start: int, stop: None | int = None, step: int = 1)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringGetSlice processor.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

start: int

The starting index of the slice.

step: int

The step size for slicing (default is 1).

stop: None | int

The ending index of the slice.

class hyped.core.ops.string.StringLeftStrip(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringLeftStripConfig]

Data processor for stripping characters from the left end of strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Removes specified characters from the left end of the input string column.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to left-strip.

Returns:

A column of strings with the specified characters removed from the left end.

Return type:

StringFeature

class hyped.core.ops.string.StringLeftStripConfig(*, characters: str)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringLeftStrip processor.

characters: str

The set of characters to remove from the left end of each string.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringLength(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringLengthConfig]

Data processor for calculating the length of strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → Int32Feature[source]

Calculates the length of the input string column.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to calculate lengths for.

Returns:

A column of integers representing the length of each string.

Return type:

Int32Feature

class hyped.core.ops.string.StringLengthConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringLength processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringLower(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringLowerConfig]

Data processor for converting strings to lowercase.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Converts the input string column to lowercase.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to convert to lowercase.

Returns:

A column of lowercase strings.

Return type:

StringFeature

class hyped.core.ops.string.StringLowerConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringLower processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringMultiply(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringMultiplyConfig]

Data processor for string multiplication.

process(ctx: RunContext, a: StringFeature | str | list[str] | StringScalar | StringArray, b: Int64Feature | Int32Feature | Int16Feature | Int8Feature | int | list[int] | Int64Scalar | Int32Scalar | Int16Scalar | Int8Scalar | Int64Array | Int32Array | Int16Array | Int8Array | UInt64Feature | UInt32Feature | UInt16Feature | UInt8Feature | UInt64Scalar | UInt32Scalar | UInt16Scalar | UInt8Scalar | UInt64Array | UInt32Array | UInt16Array | UInt8Array) → StringFeature[source]

Repeats strings in column a according to values in column b.

Parameters:

  • ctx (RunContext) – The execution context.

  • a (String) – The string column.

  • b (Int | UInt) – The column specifying the number of repetitions.

Returns:

The resulting column with repeated strings.

Return type:

StringFeature

class hyped.core.ops.string.StringMultiplyConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringMultiply processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringReplace(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringReplaceConfig]

Data processor for replacing substrings in strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Replaces occurrences of the pattern with the replacement string.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to perform replacements on.

Returns:

A column of strings with the specified replacements.

Return type:

StringFeature

class hyped.core.ops.string.StringReplaceConfig(*, pattern: str, replacement: str, count: None | int = None)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringReplace processor.

count: None | int

The maximum number of replacements to make (default is no limit).

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pattern: str

The pattern to search for in each string.

replacement: str

The string to replace the pattern with.

class hyped.core.ops.string.StringRightStrip(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringRightStripConfig]

Data processor for stripping characters from the right end of strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Removes specified characters from the right end of the input string column.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to right-strip.

Returns:

A column of strings with the specified characters removed from the right end.

Return type:

StringFeature

class hyped.core.ops.string.StringRightStripConfig(*, characters: str)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringRightStrip processor.

characters: str

The set of characters to remove from the right end of each string.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringSetSlice(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringSetSliceConfig]

Data processor for setting slices of strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Replaces slices of strings with the specified replacement string.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to modify.

Returns:

The modified string column with replaced slices.

Return type:

StringFeature

class hyped.core.ops.string.StringSetSliceConfig(*, start: int, stop: int, replacement: str)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringSetSlice processor.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

replacement: str

The replacement string.

start: int

The starting index of the slice to replace.

stop: int

The ending index of the slice to replace.

class hyped.core.ops.string.StringSplit(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringSplitConfig]

Data processor for splitting strings by a pattern.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → SequenceFeature[StringFeature] | list[StringFeature] | list[list[StringFeature]] | ListScalar | ListArray[source]

Splits each string in the column by the specified pattern.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to split.

Returns:

A column of string sequences, resulting from the splits.

Return type:

Sequence[StringFeature]

class hyped.core.ops.string.StringSplitConfig(*, pattern: str, max_splits: None | int = None, reverse: bool = False)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringSplit processor.

max_splits: None | int

The maximum number of splits to perform (default is no limit).

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pattern: str

The pattern to split each string on.

reverse: bool

Whether to perform the splits in reverse order.

class hyped.core.ops.string.StringStartsWith(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringStartsWithConfig]

Data processor for checking if strings start with a given pattern.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → BoolFeature[source]

Checks if the input string column starts with the specified pattern.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to check.

Returns:

A column of boolean values indicating whether each string starts with the pattern.

Return type:

BoolFeature

class hyped.core.ops.string.StringStartsWithConfig(*, pattern: str)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringStartsWith processor.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pattern: str

The pattern to check for at the start of each string.

class hyped.core.ops.string.StringStrip(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringStripConfig]

Data processor for stripping characters from both ends of strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Removes specified characters from both ends of the input string column.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to strip.

Returns:

A column of strings with the specified characters removed from both ends.

Return type:

StringFeature

class hyped.core.ops.string.StringStripConfig(*, characters: str)[source]

Bases: BaseDataProcessorConfig

Configuration for the StringStrip processor.

characters: str

The set of characters to remove from both ends of each string.

model_config = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringSwapCase(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringSwapCaseConfig]

Data processor for swapping the case of strings.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Swaps the case of each character in the input string column.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to swap the case for.

Returns:

A column with swapped case strings.

Return type:

StringFeature

class hyped.core.ops.string.StringSwapCaseConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringSwapCase processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringTitle(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringTitleConfig]

Data processor for converting strings to title case.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Converts the input string column to title case.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to convert to title case.

Returns:

A column of strings in title case.

Return type:

StringFeature

class hyped.core.ops.string.StringTitleConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringTitle processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class hyped.core.ops.string.StringUpper(*args: Any, **kwargs: Any)[source]

Bases: BaseDataProcessor[StringUpperConfig]

Data processor for converting strings to uppercase.

process(ctx: RunContext, string: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Converts the input string column to uppercase.

Parameters:

  • ctx (RunContext) – The execution context.

  • string (String) – The string column to convert to uppercase.

Returns:

A column of uppercase strings.

Return type:

StringFeature

class hyped.core.ops.string.StringUpperConfig[source]

Bases: BaseDataProcessorConfig

Configuration for the StringUpper processor.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'validate_default': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

hyped.core.ops.string.string_contains(string: StringFeature | str | list[str] | StringScalar | StringArray, pattern: str) → BoolFeature[source]

Check if a specific pattern is contained in each string.

Parameters:

  • string (String) – The string column to search.

  • pattern (str) – The pattern to match in each string.

Returns:

A column of boolean values indicating whether the string contains the pattern.

Return type:

BoolFeature

hyped.core.ops.string.string_endswith(string: StringFeature | str | list[str] | StringScalar | StringArray, pattern: str) → BoolFeature[source]

Checks if each string in the column ends with the specified pattern.

Parameters:

  • string (String) – The string column to check.

  • pattern (str) – The pattern to check for at the end of each string.

Returns:

A column of boolean values indicating whether each string ends with the pattern.

Return type:

BoolFeature

hyped.core.ops.string.string_find(string: StringFeature | str | list[str] | StringScalar | StringArray, pattern: str) → Int32Feature[source]

Finds the position of the specified pattern in each string.

Parameters:

  • string (String) – The string column to search.

  • pattern (str) – The pattern to find in each string.

Returns:

A column of integer values representing the position of the pattern in each string.

Return type:

Int32Feature

hyped.core.ops.string.string_format(string: StringFeature | str | list[str] | StringScalar | StringArray, **kwargs: StringFeature | str | list[str] | StringScalar | StringArray) → StringFeature[source]

Formats each string in the column using the provided keyword arguments.

Parameters:

  • string (String) – The string column to format.

  • **kwargs (String) – The keyword arguments to substitute into the string.

Returns:

A column of formatted strings.

Return type:

StringFeature

hyped.core.ops.string.string_getitem(string: StringFeature | str | list[str] | StringScalar | StringArray, idx: int | slice) → StringFeature[source]

Gets a slice of each string in the column using the specified index or slice.

Parameters:

  • string (String) – The string column to slice.

  • idx (int | slice) – The index or slice specifying the range of characters to retrieve.

Returns:

A column of strings representing the sliced portion.

Return type:

StringFeature

hyped.core.ops.string.string_lstrip(string: StringFeature | str | list[str] | StringScalar | StringArray, characters: str) → StringFeature[source]

Removes specified characters from the left end of each string.

Parameters:

  • string (String) – The string column to left-strip.

  • characters (str) – The set of characters to remove from the left end of each string.

Returns:

A column of strings with the specified characters removed from the left end.

Return type:

StringFeature

hyped.core.ops.string.string_replace(string: StringFeature | str | list[str] | StringScalar | StringArray, pattern: str, replacement: str, count: None | int = None) → StringFeature[source]

Replaces occurrences of the specified pattern in each string with the replacement string.

Parameters:

  • string (String) – The string column to modify.

  • pattern (str) – The pattern to replace.

  • replacement (str) – The string to replace the pattern with.

  • count (None | int, optional) – The maximum number of occurrences to replace. If None, all occurrences are replaced.

Returns:

A column of strings with the specified pattern replaced by the replacement string.

Return type:

StringFeature

hyped.core.ops.string.string_rstrip(string: StringFeature | str | list[str] | StringScalar | StringArray, characters: str) → StringFeature[source]

Removes specified characters from the right end of each string.

Parameters:

  • string (String) – The string column to right-strip.

  • characters (str) – The set of characters to remove from the right end of each string.

Returns:

A column of strings with the specified characters removed from the right end.

Return type:

StringFeature

hyped.core.ops.string.string_setitem(string: StringFeature | str | list[str] | StringScalar | StringArray, idx: int | slice, replacement: str) → StringFeature[source]

Sets a slice of each string in the column to the specified replacement string.

Parameters:

  • string (String) – The string column to modify.

  • idx (int | slice) – The index or slice specifying the range of characters to replace.

  • replacement (str) – The string to replace the sliced portion with.

Returns:

A column of strings with the specified portion replaced by the replacement string.

Return type:

StringFeature

Raises:

  • ValueError – If the slice step is not 1.

  • ValueError – If the length of the replacement string does not match the slice range.

hyped.core.ops.string.string_split(string: StringFeature | str | list[str] | StringScalar | StringArray, pattern: str, max_splits: None | int = None, reverse: bool = False) → StringFeature[source]

Splits each string in the column by the specified pattern.

Parameters:

  • string (String) – The string column to split.

  • pattern (str) – The pattern to split each string on.

  • max_splits (None | int, optional) – The maximum number of splits to perform. If None, no limit is applied.

  • reverse (bool, optional) – Whether to split in reverse order.

Returns:

A column of string sequences resulting from splitting each string.

Return type:

StringFeature

hyped.core.ops.string.string_startswith(string: StringFeature | str | list[str] | StringScalar | StringArray, pattern: str) → BoolFeature[source]

Checks if each string in the column starts with the specified pattern.

Parameters:

  • string (String) – The string column to check.

  • pattern (str) – The pattern to check for at the start of each string.

Returns:

A column of boolean values indicating whether each string starts with the pattern.

Return type:

BoolFeature

hyped.core.ops.string.string_strip(string: StringFeature | str | list[str] | StringScalar | StringArray, characters: str) → StringFeature[source]

Removes specified characters from both ends of each string.

Parameters:

  • string (String) – The string column to strip.

  • characters (str) – The set of characters to remove from both ends of each string.

Returns:

A column of strings with the specified characters removed from both ends.

Return type:

StringFeature