Unreal Engine Interaction SDK

1. Basic introduction

1.1. Environmental requirements

• This SDK needs to be used with UnrealEngine 5.0 or above, as well as HandDriver gloves.

1.2. File download

• SDK download address:

• Release date:

1.3. Feature introduction

The UnrealEngine Interaction SDK is a Software Development Kit designed for developers to enhance the hand interaction experience in Unreal Engine with gloves. This SDK includes two core components: UdxHandControl(interaction component) and UdGrab(grip component). In addition, the SDK also provides a static class UdeInteractionFuncLib for providing auxiliary features and static methods.

2. Development readiness

2.1. Check the file

Check if there is a HandDriver plugin and make sure the project project supports UDEXREAL HandDriver UE5 SDK streaming. If it is not supported yet, please check the UDEXREAL HandDriver UE5 Plugin Tutorial and configure the streaming environment according to the documentation.

2.2. Plugin installation

2.2.1. Import plugin

Unpack the plugin from Compressed Packet and drag it to the Plugins directory of the project project (if there is no Plugins folder, you need to manually create one with the same name).

2.2.2. Enable plugin

If not enabled, please enable the plugin first. If already enabled, please skip this step.

3. Quick start

3.1. Add components

3.1.1. Pawn Interactive Component

• Add interactive components

Add interactive components to pawn and use them together with crawling components. Two components are needed here, one for the left hand and one for the right hand, to be added according to actual needs.

• Interaction component settings

1️⃣Init interaction components.

2️⃣Add update Anim Event

3️⃣Add SimGrabOnTick func

4️⃣Update Anim for hand

5️⃣Add Grab/Release func

Name	Type	Defult	Explain
MotionControl	USeceneComponent	nullptr	The MotionControllerComponent of the controlled object
OtherHandComp	UdxGrabComponent	nullptr	The UdxGrabComponent obtained by the other hand
ThisHand	Enum	Left	Trigger the interaction of the left hand
TraceType	Enum	ForOneFrame	Display one frame of the detected sphere
Radius	float	6.0	Measure the radius of the sphere
ReturnValue	UdxGrabComponent	nullptr	Return the UdxGrabComponent obtained by the current hand.

3.1.2. Add anim blueprint

• Create an anim blueprint. If it has already been created, proceed to the next step directly.

  • Create details: omitted

• Ceate variables bool bIsPlayGrab

  • TArray<UdxHandBoneRotations> SHandBones

• Create the following blueprint for each finger joint

• Adjust the transformation nodes in the animation blueprint, and finally achieve the fist-clenching pose.

• Add the Blend animation node to switch between the motion capture state and the interaction state.

3.1.3. Add grabbing component

• Add Grab component

  • Add grabbing components to props and use them with interactive components. Add them as needed, and when interacting, recognize grabbing components with the same handType as the interactive component and grab the one closest to the grabbing point.

• Grab component settings

Name	Type	Explain
GrabType	Enum	The capture type enables the implementation of different capture logic by selecting different types.
GrabOrientType	Enum	Prompt whether to align and grasp the props at the correct positions
GrabOrientAddCustomRotation	FRotator	Add custom rotation offset to the props after interaction
GrabOrientAddCustomOffset	FVector	After interaction, add custom position offsets to the hand model
Enable	bool	Do you want to enable this crawling component
ThisGrabComponentFor	Enum	Which hand should be used to grasp the prop on which this component is hung
SnapTypeOnOverlap	Enum	Decide whether the interactive props are static or can be picked up and moved.
BreakPenetrationConstraintDst	float	Not in use yet
SimulateOnDrop	bool	Enable item physics simulation
SimulatePhysCollision	bool	Enable item physical collision
ShowGrabPointCircleFX	bool	Not in use yet
StopFingerSimAfterDelay	bool	Not in use yet
WhenGrabbedReleaseFromOtherHands	bool	Prevent irreversible situations from occurring when both hands are grasping the same object.

• Add component for grab/release events

3.2. Transfer parameters to interactive components

3.2.1. Initialize interactive components

When the event starts running, pass in the grab point and hand bone model (currently only supports hand bones).

3.2.2. Grab and release

1. PC development

1. Android development

Implement capture and release through event triggering in Android SDK. Note: The blueprint interface in the illustration needs to be created by yourself and then called at the grip event trigger of the Android SDK or using the PC development method.

1. Android development requires the use of interface calls.

4. Example of Interactive Usage

4.1. Interaction cannot move objects (such as doors, levers). Let's take the lever as an example.

1. Create a blueprint with the base class set as "Actor"

1. The component with a push rod added

1. Add the grasping component (⚠️The grasping component should be placed under the part that needs to have its transformation modified. For example, if the rod of the push rod needs to rotate, place it under the part).

1. Write the logic for the rotation of the push rod

Note: This logic does not involve optimization. It merely provides a tutorial on how to use this plugin and the approach for creating such objects. Please understand.

5. Demo video display (temporary)

To be added...