motion

Motion data structure and related functions.

class xrfeitoria.utils.anim.motion.Motion(transl: ndarray, body_poses: ndarray, n_frames: int | None = None, fps: float = 30.0)[source]

Wrap motion data. Provide methods to get transform info for 3D calculations.

The motion data will be used along with Skeleton instance in retargeting, and the local spaces of bones are all defined in such skeletons.

__init__(transl: ndarray, body_poses: ndarray, n_frames: int | None = None, fps: float = 30.0) → None[source]: Transl & body_poses are in the space of corresponding Skeleton instance.

convert_fps(fps: float)[source]

Converts the frames per second (fps) of the animation to the specified value.

Parameters:

fps (float) – The desired frames per second.

Raises:

NotImplementedError – If the fps is greater than the current fps.
NotImplementedError – If the fps is less than the current fps when undividable.

copy() → Motion[source]: Return a copy of the motion instance.

cut_motion(start_frame: int | None = None, end_frame: int | None = None)[source]

Cut the motion sequence to a given number of frames (to [start_frame, end_frame])

Parameters:

start_frame (Optional[int], optional) – The start frame to cut to. Defaults to None.
end_frame (Optional[int], optional) – The end frame to cut to. Defaults to None.

Raises:

AssertionError – If the start frame is less than 0.
AssertionError – If the end frame is greater than the number of frames in the motion sequence.
AssertionError – If the start frame is greater than or equal to the end frame.

cut_transl()[source]

Cut the transl to zero.

This will make the animation stay in place, like root motion.

get_bone_matrix_basis(bone_name: str, frame=0) → ndarray[source]

pose2rest: relative to the bone space at rest pose.

Parameters:

bone_name (str) – bone name
frame (int, optional) – frame index. Defaults to 0.

Returns:

np.ndarray – transform matrix like [ [R, T], [0, 1] ]

get_motion_data() → List[Dict[str, Dict[str, float | List[float]]]][source]

Returns a list of dictionaries containing rotation and location for each bone of each frame in the animation.

Each dictionary contains bone names as keys and a nested dictionary as values. The nested dictionary contains ‘rotation’ and ‘location’ keys, which correspond to the rotation and location of the bone in that frame.

Returns:: List[MotionFrame] – A list of dictionaries containing motion data for each frame of the animation.

insert_rest_pose()[source]: Insert rest pose to the first frame.

sample_motion(n_frames: int)[source]

Randomly sample motions, picking n_frames from the original motion sequence. The indices are totally random using np.random.choice.

Parameters:: n_frames (int) – The number of frames to sample. Randomly sampled from the original motion sequence.
Raises:: AssertionError – If the number of frames to sample is less than or equal to 0.

slice_motion(frame_interval: int)[source]

Slice the motion sequence by a given frame interval.

Parameters:: frame_interval (int) – The frame interval to use for slicing the motion sequence.
Raises:: TypeError – If the frame interval is not an integer.

BONE_NAMES: List[str]

BONE_NAME_TO_IDX: Dict[str, int]

PARENTS: List[int]

class xrfeitoria.utils.anim.motion.SMPLMotion(transl: ndarray, body_poses: ndarray, n_frames: int | None = None, fps: float = 30.0)[source]

__init__(transl: ndarray, body_poses: ndarray, n_frames: int | None = None, fps: float = 30.0) → None[source]: Transl & body_poses are in the space of corresponding Skeleton instance.

copy() → SMPLMotion[source]: Return a copy of the motion instance.

dump_humandata(filepath: str | Path, betas: ndarray, meta: Dict[str, Any] | None = None, global_orient_offset: ndarray = array([0., 0., 0.]), transl_offset: ndarray = array([0., 0., 0.]), root_location_t0: ndarray | None = None, pelvis_location_t0: ndarray | None = None) → None[source]

Dump the motion data to a humandata file at the given filepath.

Parameters:

filepath (PathLike) – The filepath to dump the motion data to.
betas (np.ndarray) – The betas array.
meta (Optional[Dict[str, Any]]) – Additional metadata. Defaults to None.
global_orient_offset (np.ndarray) – The global orientation offset. Defaults to np.zeros(3).
transl_offset (np.ndarray) – The translation offset. Defaults to np.zeros(3).
root_location_t0 (Optional[np.ndarray]) – The root location at time 0. Defaults to None.
pelvis_location_t0 (Optional[np.ndarray]) – The pelvis location at time 0. Defaults to None.

Note

HumanData is a structure of smpl/smplx data defined in https://github.com/open-mmlab/mmhuman3d/blob/main/docs/human_data.md

The humandata file is a npz file containing the following keys:

motion_data = {
    '__data_len__': n_frames,
    'smpl': {
        'betas': betas,  # (1, 10)
        'transl': transl,  # (n_frames, 3)
        'global_orient': global_orient,  # (n_frames, 3)
        'body_pose': body_pose,  # (n_frames, 69)
    },
    'meta': {'gender': 'neutral'},  # optional
}

classmethod from_amass_data(amass_data, insert_rest_pose: bool) → SMPLMotion[source]

Create a Motion instance from AMASS data (SMPL)

Parameters:

amass_data (dict) – A dictionary containing the AMASS data.
insert_rest_pose (bool) – Whether to insert a rest pose at the beginning of the motion.

Returns:

SMPLMotion – A SMPLMotion instance containing the AMASS data.

classmethod from_smpl_data(smpl_data: ~typing.Dict[str, ~numpy.ndarray], fps: float = 30.0, insert_rest_pose: bool = False, global_orient_adj: ~scipy.spatial.transform._rotation.Rotation | None = <scipy.spatial.transform._rotation.Rotation object>, vector_convertor: ~typing.Callable[[~numpy.ndarray], ~numpy.ndarray] | None = <bound method ConverterMotion.vec_humandata2smplx of <class 'xrfeitoria.utils.converter.ConverterMotion'>>) → SMPLMotion[source]

Create SMPLMotion instance from smpl_data.

smpl_data should be a dict like object, with required keys: [‘betas’, ‘body_pose’, ‘global_orient’] and optional key: [‘transl’]

Parameters:

smpl_data – dict with require keys [“body_pose”, “global_orient”] and optional key [“transl”]
insert_rest_pose (bool) – whether to insert a rest pose at the 0th-frame.

Returns:

SMPLMotion – An instance of SMPLMotion containing the smpl_data.

BONE_NAMES: List[str] = ['pelvis', 'left_hip', 'right_hip', 'spine1', 'left_knee', 'right_knee', 'spine2', 'left_ankle', 'right_ankle', 'spine3', 'left_foot', 'right_foot', 'neck', 'left_collar', 'right_collar', 'head', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'jaw']

BONE_NAME_TO_IDX: Dict[str, int] = {'head': 15, 'jaw': 22, 'left_ankle': 7, 'left_collar': 13, 'left_elbow': 18, 'left_foot': 10, 'left_hip': 1, 'left_knee': 4, 'left_shoulder': 16, 'left_wrist': 20, 'neck': 12, 'pelvis': 0, 'right_ankle': 8, 'right_collar': 14, 'right_elbow': 19, 'right_foot': 11, 'right_hip': 2, 'right_knee': 5, 'right_shoulder': 17, 'right_wrist': 21, 'spine1': 3, 'spine2': 6, 'spine3': 9}

GLOBAL_ORIENT_ADJUSTMENT = <scipy.spatial.transform._rotation.Rotation object>

NAMES = ['Pelvis', 'Hip_L', 'Hip_R', 'Spine1', 'Knee_L', 'Knee_R', 'Spine2', 'Ankle_L', 'Ankle_R', 'Chest', 'Toes_L', 'Toes_R', 'Neck', 'Scapula_L', 'Scapula_R', 'Head', 'Shoulder_L', 'Shoulder_R', 'Elbow_L', 'Elbow_R', 'Wrist_L', 'Wrist_R']

NAME_TO_SMPL_IDX = {'Ankle_L': 7, 'Ankle_R': 8, 'Chest': 9, 'Elbow_L': 18, 'Elbow_R': 19, 'Head': 15, 'Hip_L': 1, 'Hip_R': 2, 'Knee_L': 4, 'Knee_R': 5, 'Neck': 12, 'Pelvis': 0, 'Scapula_L': 13, 'Scapula_R': 14, 'Shoulder_L': 16, 'Shoulder_R': 17, 'Spine1': 3, 'Spine2': 6, 'Toes_L': 10, 'Toes_R': 11, 'Wrist_L': 20, 'Wrist_R': 21}

PARENTS: List[int] = [-1, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 12, 12, 13, 14, 16, 17, 18, 19, 20, 21]

SMPL_IDX_TO_NAME: Dict[int, str] = {0: 'Pelvis', 1: 'Hip_L', 2: 'Hip_R', 3: 'Spine1', 4: 'Knee_L', 5: 'Knee_R', 6: 'Spine2', 7: 'Ankle_L', 8: 'Ankle_R', 9: 'Chest', 10: 'Toes_L', 11: 'Toes_R', 12: 'Neck', 13: 'Scapula_L', 14: 'Scapula_R', 15: 'Head', 16: 'Shoulder_L', 17: 'Shoulder_R', 18: 'Elbow_L', 19: 'Elbow_R', 20: 'Wrist_L', 21: 'Wrist_R', 22: '', 23: ''}

class xrfeitoria.utils.anim.motion.SMPLXMotion(transl: ndarray, body_poses: ndarray, n_frames: int | None = None, fps: float = 30.0)[source]

__init__(transl: ndarray, body_poses: ndarray, n_frames: int | None = None, fps: float = 30.0) → None[source]: Transl & body_poses are in the space of corresponding Skeleton instance.

copy() → SMPLXMotion[source]: Return a copy of the motion instance.

dump_humandata(filepath: str | Path, betas: ndarray, meta: Dict[str, Any] | None = None, global_orient_offset: ndarray = array([0., 0., 0.]), transl_offset: ndarray = array([0., 0., 0.]), root_location_t0: ndarray | None = None, pelvis_location_t0: ndarray | None = None) → None[source]

Dump the motion data to a humandata file at the given filepath.

Parameters:

filepath (PathLike) – The filepath to dump the motion data to.
betas (np.ndarray) – The betas array.
meta (Optional[Dict[str, Any]]) – Additional metadata. Defaults to None.
global_orient_offset (np.ndarray) – The global orientation offset. Defaults to np.zeros(3).
transl_offset (np.ndarray) – The translation offset. Defaults to np.zeros(3).
root_location_t0 (Optional[np.ndarray]) – The root location at time 0. Defaults to None.
pelvis_location_t0 (Optional[np.ndarray]) – The pelvis location at time 0. Defaults to None.

Note

HumanData is a structure of smpl/smplx data defined in https://github.com/open-mmlab/mmhuman3d/blob/main/docs/human_data.md

The humandata file is a npz file containing the following keys:

humandata = {
    '__data_len__': n_frames,
    'smplx': {
        'betas': betas,  # (1, 10)
        'transl': transl,  # (n_frames, 3)
        'global_orient': global_orient,  # (n_frames, 3)
        'body_pose': body_pose,  # (n_frames, 63)
        'jaw_pose': jaw_pose,  # (n_frames, 3)
        'leye_pose': leye_pose,  # (n_frames, 3)
        'reye_pose': reye_pose,  # (n_frames, 3)
        'left_hand_pose': left_hand_pose,  # (n_frames, 45)
        'right_hand_pose': right_hand_pose,  # (n_frames, 45)
        'expression': expression,  # (n_frames, 10)
    },
    'meta': {'gender': 'neutral'},  # optional
}

classmethod from_amass_data(amass_data, insert_rest_pose: bool, flat_hand_mean: bool = True) → SMPLXMotion[source]

Create a Motion instance from AMASS data (SMPLX)

Parameters:

amass_data (dict) – A dictionary containing the AMASS data.
insert_rest_pose (bool) – Whether to insert a rest pose at the beginning of the motion.
flat_hand_mean (bool) – Whether to use the flat hand mean pose.

Returns:

SMPLXMotion – A SMPLXMotion instance containing the AMASS data.

Raises:

AssertionError – If the surface model type in the AMASS data is not ‘smplx’.

classmethod from_smplx_data(smplx_data: ~typing.Dict[str, ~numpy.ndarray], fps: float = 30.0, insert_rest_pose: bool = False, flat_hand_mean: bool = False, global_orient_adj: ~scipy.spatial.transform._rotation.Rotation | None = <scipy.spatial.transform._rotation.Rotation object>, vector_convertor: ~typing.Callable[[~numpy.ndarray], ~numpy.ndarray] | None = <bound method ConverterMotion.vec_humandata2smplx of <class 'xrfeitoria.utils.converter.ConverterMotion'>>) → SMPLXMotion[source]

Create SMPLXMotion instance from smplx_data.

smplx_data should be a dict like object, with required keys: [‘betas’, “body_pose”, “global_orient”] and optional key: [‘transl’, ‘jaw_pose’, ‘leye_pose’, ‘reye_pose’, ‘left_hand_pose’, ‘right_hand_pose’, ‘expression’]

Parameters:

smplx_data – require keys [“body_pose”, “global_orient”] and optional key [“transl”]
fps (float) – the motion’s FPS. Defaults to 30.0.
insert_rest_pose (bool) – whether to insert a rest pose at the 0th-frame. Defaults to False.
flat_hand_mean (bool) – whether the hands with zero rotations are flat hands. Defaults to False.
global_orient_adj (spRotation, None) –
vector_convertor – a function applies to smplx_data’s translation.

Returns:

SMPLXMotion – An instance of SMPLXMotion containing the smplx_data.

BONE_NAMES: List[str] = ['pelvis', 'left_hip', 'right_hip', 'spine1', 'left_knee', 'right_knee', 'spine2', 'left_ankle', 'right_ankle', 'spine3', 'left_foot', 'right_foot', 'neck', 'left_collar', 'right_collar', 'head', 'left_shoulder', 'right_shoulder', 'left_elbow', 'right_elbow', 'left_wrist', 'right_wrist', 'jaw', 'left_eye_smplhf', 'right_eye_smplhf', 'left_index1', 'left_index2', 'left_index3', 'left_middle1', 'left_middle2', 'left_middle3', 'left_pinky1', 'left_pinky2', 'left_pinky3', 'left_ring1', 'left_ring2', 'left_ring3', 'left_thumb1', 'left_thumb2', 'left_thumb3', 'right_index1', 'right_index2', 'right_index3', 'right_middle1', 'right_middle2', 'right_middle3', 'right_pinky1', 'right_pinky2', 'right_pinky3', 'right_ring1', 'right_ring2', 'right_ring3', 'right_thumb1', 'right_thumb2', 'right_thumb3']

BONE_NAME_TO_IDX: Dict[str, int] = {'head': 15, 'jaw': 22, 'left_ankle': 7, 'left_collar': 13, 'left_elbow': 18, 'left_eye_smplhf': 23, 'left_foot': 10, 'left_hip': 1, 'left_index1': 25, 'left_index2': 26, 'left_index3': 27, 'left_knee': 4, 'left_middle1': 28, 'left_middle2': 29, 'left_middle3': 30, 'left_pinky1': 31, 'left_pinky2': 32, 'left_pinky3': 33, 'left_ring1': 34, 'left_ring2': 35, 'left_ring3': 36, 'left_shoulder': 16, 'left_thumb1': 37, 'left_thumb2': 38, 'left_thumb3': 39, 'left_wrist': 20, 'neck': 12, 'pelvis': 0, 'right_ankle': 8, 'right_collar': 14, 'right_elbow': 19, 'right_eye_smplhf': 24, 'right_foot': 11, 'right_hip': 2, 'right_index1': 40, 'right_index2': 41, 'right_index3': 42, 'right_knee': 5, 'right_middle1': 43, 'right_middle2': 44, 'right_middle3': 45, 'right_pinky1': 46, 'right_pinky2': 47, 'right_pinky3': 48, 'right_ring1': 49, 'right_ring2': 50, 'right_ring3': 51, 'right_shoulder': 17, 'right_thumb1': 52, 'right_thumb2': 53, 'right_thumb3': 54, 'right_wrist': 21, 'spine1': 3, 'spine2': 6, 'spine3': 9}

GLOBAL_ORIENT_ADJUSTMENT = <scipy.spatial.transform._rotation.Rotation object>

NAMES = ['Pelvis', 'Hip_L', 'Hip_R', 'Spine1', 'Knee_L', 'Knee_R', 'Spine2', 'Ankle_L', 'Ankle_R', 'Chest', 'Toes_L', 'Toes_R', 'Neck', 'Scapula_L', 'Scapula_R', 'Head', 'Shoulder_L', 'Shoulder_R', 'Elbow_L', 'Elbow_R', 'Wrist_L', 'Wrist_R', 'index_A_L', 'index_B_L', 'index_C_L', 'middle_A_L', 'middle_B_L', 'middle_C_L', 'pinky_A_L', 'pinky_B_L', 'pinky_C_L', 'ring_A_L', 'ring_B_L', 'ring_C_L', 'thumb_A_L', 'thumb_B_L', 'thumb_C_L', 'index_A_R', 'index_B_R', 'index_C_R', 'middle_A_R', 'middle_B_R', 'middle_C_R', 'pinky_A_R', 'pinky_B_R', 'pinky_C_R', 'ring_A_R', 'ring_B_R', 'ring_C_R', 'thumb_A_R', 'thumb_B_R', 'thumb_C_R']

NAME_TO_SMPL_IDX = {'Ankle_L': 7, 'Ankle_R': 8, 'Chest': 9, 'Elbow_L': 18, 'Elbow_R': 19, 'Head': 15, 'Hip_L': 1, 'Hip_R': 2, 'Knee_L': 4, 'Knee_R': 5, 'Neck': 12, 'Pelvis': 0, 'Scapula_L': 13, 'Scapula_R': 14, 'Shoulder_L': 16, 'Shoulder_R': 17, 'Spine1': 3, 'Spine2': 6, 'Toes_L': 10, 'Toes_R': 11, 'Wrist_L': 20, 'Wrist_R': 21, 'index_A_L': 22, 'index_A_R': 37, 'index_B_L': 23, 'index_B_R': 38, 'index_C_L': 24, 'index_C_R': 39, 'middle_A_L': 25, 'middle_A_R': 40, 'middle_B_L': 26, 'middle_B_R': 41, 'middle_C_L': 27, 'middle_C_R': 42, 'pinky_A_L': 28, 'pinky_A_R': 43, 'pinky_B_L': 29, 'pinky_B_R': 44, 'pinky_C_L': 30, 'pinky_C_R': 45, 'ring_A_L': 31, 'ring_A_R': 46, 'ring_B_L': 32, 'ring_B_R': 47, 'ring_C_L': 33, 'ring_C_R': 48, 'thumb_A_L': 34, 'thumb_A_R': 49, 'thumb_B_L': 35, 'thumb_B_R': 50, 'thumb_C_L': 36, 'thumb_C_R': 51}

PARENTS: List[int] = [-1, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 12, 12, 13, 14, 16, 17, 18, 19, 20, 21, 20, 22, 23, 20, 25, 26, 20, 28, 29, 20, 31, 32, 20, 34, 35, 21, 37, 38, 21, 40, 41, 21, 43, 44, 21, 46, 47, 21, 49, 50]

SMPLX_IDX_TO_NAME: Dict[int, str] = {0: 'Pelvis', 1: 'Hip_L', 2: 'Hip_R', 3: 'Spine1', 4: 'Knee_L', 5: 'Knee_R', 6: 'Spine2', 7: 'Ankle_L', 8: 'Ankle_R', 9: 'Chest', 10: 'Toes_L', 11: 'Toes_R', 12: 'Neck', 13: 'Scapula_L', 14: 'Scapula_R', 15: 'Head', 16: 'Shoulder_L', 17: 'Shoulder_R', 18: 'Elbow_L', 19: 'Elbow_R', 20: 'Wrist_L', 21: 'Wrist_R', 22: 'index_A_L', 23: 'index_B_L', 24: 'index_C_L', 25: 'middle_A_L', 26: 'middle_B_L', 27: 'middle_C_L', 28: 'pinky_A_L', 29: 'pinky_B_L', 30: 'pinky_C_L', 31: 'ring_A_L', 32: 'ring_B_L', 33: 'ring_C_L', 34: 'thumb_A_L', 35: 'thumb_B_L', 36: 'thumb_C_L', 37: 'index_A_R', 38: 'index_B_R', 39: 'index_C_R', 40: 'middle_A_R', 41: 'middle_B_R', 42: 'middle_C_R', 43: 'pinky_A_R', 44: 'pinky_B_R', 45: 'pinky_C_R', 46: 'ring_A_R', 47: 'ring_B_R', 48: 'ring_C_R', 49: 'thumb_A_R', 50: 'thumb_B_R', 51: 'thumb_C_R'}

xrfeitoria.utils.anim.motion.get_humandata(smpl_x_data: Dict[str, ndarray], smpl_x_type: Literal['smpl', 'smplx'], betas: ndarray, meta: Dict[str, Any] | None = None, global_orient_offset: ndarray = array([0., 0., 0.]), transl_offset: ndarray = array([0., 0., 0.]), root_location_t0: ndarray | None = None, pelvis_location_t0: ndarray | None = None) → Dict[str, Any][source]

Get human data for a given set of parameters.

Parameters:

smpl_x_data (Dict[str, np.ndarray]) – Dictionary containing the SMPL-X data.
smpl_x_type (Literal['smpl', 'smplx']) – Type of SMPL-X model.
betas (np.ndarray) – Array of shape (n, 10) representing the shape parameters.
meta (Optional[Dict[str, Any]], optional) – Additional metadata. Defaults to None.
global_orient_offset (np.ndarray) – Array of shape (n, 3) representing the global orientation offset.
transl_offset (np.ndarray) – Array of shape (3,) representing the translation offset.
root_location_t0 (Optional[np.ndarray], optional) – Array of shape (3,) representing the root location at time t=0. Defaults to None.
pelvis_location_t0 (Optional[np.ndarray], optional) – Array of shape (3,) representing the pelvis location at time t=0. Defaults to None.

Returns:

dict – Dictionary containing the human data.