hyped.core.utils module

Utility functions and type mappings used throughout the core module.

This module provides various helper functions for working with data types, feature mappings, and nested structures in the core module.

hyped.core.utils.build_annotation_from_dtype(dtype: DType) → Any[source]

Convert a dtype instance to a type annotation.

Maps a DType object into a Python type annotation suitable for describing the structure of a dataset feature.

Parameters:

dtype (DType) – The data type to convert.

Returns:

A type annotation corresponding to the provided dtype.

Return type:

Any

hyped.core.utils.map_recursive(fn: Callable[[tuple[int | str], dict[str, dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | list[dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | tuple[dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | Any], None | dict[str, dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | list[dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | tuple[dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | Any], obj: dict[str, dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | list[dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | tuple[dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | Any, path: tuple[str | int] = ()) → dict[str, dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | list[dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | tuple[dict[str, NestedType] | list[NestedType] | tuple[NestedType] | T] | Any[source]

Apply a function recursively on a nested object.

Parameters:

  • fn (Callable[[tuple[int, str], NestedType[Any]], None | NestedType[Any]]) – The function to apply.

  • obj (NestedType[Any]) – The nested object to apply the function to.

  • path (tuple[str | int], optional) – The current path in the object (default is an empty tuple).

Returns:

The object after the function has been applied recursively.

Return type:

NestedType[Any]

hyped.core.utils.random_uuid() → UUID[source]

Generates a random UUID using a numpy random number generator.

This function uses the random number generator to produce 16 random bytes, which are then used to create a UUID (version 4). The resulting UUID is unique and reproducible based on the underlying random byte generation.

Returns:

A randomly generated UUID created from 16 random bytes.

Return type:

UUID

hyped.core.utils.validate_hf_feature(hf_feature: dict | list | tuple | Value | ClassLabel | Translation | TranslationVariableLanguages | LargeList | List | Array2D | Array3D | Array4D | Array5D | Audio | Image | Video | Pdf, annotation: Any, session: ValidationSession = ValidationSession(_session_id=None, _contexts={}, _ref_count=0), context: dict[str, Any] = {'config': None}) → Feature[source]

Validate HuggingFace features against a type annotation.

This function checks if the given HuggingFace feature structure is compatible with the provided type annotation and returns a validated feature instance.

Parameters:

  • hf_feature (FeatureType) – The HuggingFace feature structure to validate.

  • annotation (Any) – The type annotation to validate against.

  • session (ValidationSession, optional) – The validation session instance to manage context and validation state. Defaults to a new session.

  • context (dict[str, Any], optional) – Additional context for validation. Defaults to an empty dictionary.

Returns:

A validated feature instance compatible with the provided annotation.

Return type:

Feature

Raises:

RuntimeError – If the HuggingFace features are incompatible with the annotation.