Usage for UDEXREAL Robotics Products
1. Overview
UDEXREAL products in the field of robotics primarily fall into two major categories.
UDCap DataGlove Teleoperation: A product centered around UDCap DataGlove (UDCap DataGlove, referred to as UDEXREAL Data Glove) for teleoperation control of DexHands.
UDEXREAL Robotics Teleoperation Simulation System: This system can connect to UDEXREAL DataGloves and upper-body/full-body motion capture devices to enable teleoperation of robots. It supports simulation operations for URDF models of specific robots.
2. UDCap DataGlove Teleoperation
2.1. User Manual
First, you need to understand how to use the DataGlove. This is the main entry of the user manual, where all information can be accessed.
UDEXREAL UDCAP DataGlove Product Introduction2.2. Quick Guide
If you want to get started quickly, refer to the corresponding sections in the user manual:
2.3. Obtaining the HandDriver Customized Version
Because there are significant differences between human hands and robotic DexHands, the supporting software HandDriver for the gloves needs to be adapted for DexHands.
Adaptation is a core and essential task for robot teleoperation, which UDEXREAL typically provides free of charge, but it requires the DexHands to be supplied to UDEXREAL.
2.3.1. Versions of the Software
HandDriver supports both Windows and Linux Ubuntu.
Click here to download HandDriver
HandDriver is available in two versions. It is recommended to use the HandDriver Web version, which adopts a B/S architecture and supports accessing the graphical interface via a browser (the graphical interface can be disabled). For detailed usage instructions, please visit: HandDriver Web Deployment and Usage Guide (if you need to perform firmware updates, it is advised to download the Windows standalone software version. Please note: always upgrade the glove firmware first, then upgrade the receiver).
2.3.2. HandDriver Regular Version Supports DexHand Adaptation
2.3.2.1. DexHand String Matching
HandDriver v2.2.1 Windows and Linux Web versions have added the "DexHand String Matching" feature. After entering the designated string for a specific DexHand in HandDriver, the software will output data compatible with that DexHand model. The codes for the DexHands will be provided by UDEXREAL once cooperation is confirmed.
After entering the DexHand string, click the "√" button.
After successfully matching the DexHand string, the string will remain in the dropdown list. The system will provide corresponding prompts to indicate whether the string matching succeeded or failed.
2.3.2.2. DexHand Data Transmission and Action Display
The action display in HandDriver uses human hand data (the "Data" page also shows human hand data), while the data sent externally is determined by the "DexHand String Matching" feature.
2.3.3. HandDriver Customized Version
Previously, there existed a HandDriver customized version for specific brand and model DexHands. If needed, please contact UDEXTREAL sales to obtain it. Note that the customized version does not support the "DexHand String Matching" feature.
2.4. Core Steps for Using the DataGlove
Using the HandDriver to Connect the DataGlove → Calibration → Configuring Data Transmission. These three steps are the core procedures for using the DataGlove. Please refer to the corresponding sections in the user manual.
Calibration is crucial for action accuracy. Please refer to the reference video in the Calibration section.
2.5. How to Operate the DexHand
2.5.1. DexHand Motion Control
To make the DexHand operate, a control program for the DexHand needs to be implemented. This program primarily consists of three parts:
Receiving and parsing data sent by HandDriver
After calibration with the gloves on, HandDriver sends real-time (120Hz) angle data for each joint of the human hand via UDP through its data transmission function.
The data transmission formats are divided into JSON and Protobuf, each containing two types of data: joint angles (Euler angles) and quaternions. It is generally recommended to use joint angles (Euler angles) to actuate the DexHands.
The following documents provide detailed explanations of the data format and how to receive and parse the data sent by HandDriver. The SDK for this is open-source.
JSON C++/Python SDK_Quaternion ManualJSON C++/Python SDK_Angle ManualProtobuf C++/Python SDK ManualRetargeting HandDriver's joint angle data to the corresponding joints of the DexHand
Select the required data based on the joints of the DexHand and process it into a format suitable for the DexHand, such as converting to radian mapping. An example of the mapping is shown below:
Joint 0
output [20]
Joint 1
output [2]
Joint 2
output [1]
Joint 3
output [7]
Joint 4
output [6]
Joint 5
output [10]
Invoke the DexHand SDK to control the movement of the DexHand
Send the processed joint angle data to the DexHand SDK (by calling the SDK) to perform angle control.
2.5.2. Business Collaboration
In practical collaboration, UDEXREAL does not provide the DexHand motion control by default. The client is required to implement this program independently, but UDEXREAL will supply the aforementioned mapping relationships. If the DexHand motion control is needed, please contact the UDEXREAL sales.
2.5.3. Explanation of Hand Motion Spatial Pose
The UDEXREAL DataGlove operates on the principle of flexible sensors. The data it acquires consists of joint angle measurements for each finger, which differs from optical or inertial sensors. It does not provide spatial pose information. If a spatial pose data pathway is required, it must be implemented based on the quaternion documentation mentioned above. These quaternions are derived by converting the joint angle data.
UDEXREAL also provides the initial pose data for the glove model. Please refer to:
Hand Model and Data SpecificationThe final section of this document specifies the position and rotation of each joint point (including fingertips), with the origin located at the midpoint between the bases of the two gloves.
2.6. Data Acquisition
In the software settings, there is a toggle to record the joint angles of the glove for each frame. It is turned off by default.
Function entry: Settings → Shortcut Keys → Data Acquisition
File save path:
The Windows version uses the same "File Save Path" as specified in the settings.
For the HandDriver Web version, data files are stored in the server installation directory.
File naming rule: DeviceName_YYYYMMDDHHMMSS.csv. Example: UDXST4007L_20241210100233.csv. The timestamp indicates the specific time when recording starts. A new file is generated each time HandDriver is launched.
There is no file size limit, and files will not be automatically cleared.
3. UDEXREAL Robotics Teleoperation Simulation System
This system enables teleoperation of robots, either upper-body or full-body. It is offered as a fee-based customized implementation service, tailored to specific robots, DexHand, and robotic arms. The joystick and buttons on the UDEXREAL DataGlove can be customized for teleoperation interactions.
Upper-body/full-body motion capture equipment is also selected based on customer's specific situation. UDEXREAL primarily recommends using the PICO headset with PICO motion trackers for teleoperation. The motion trackers are used to acquire the spatial position of the hands.
This system currently supports the direct use of specific robotic arms and DexHands, such as the Realman RM65-B with the OYMotion ROH-A001. UDEXREAL welcomes collaborations from all companies.
The customized implementation service process is as follows: Initial Technical Consultation -> Solution Design -> Project Evaluation -> Equipment Procurement -> Project Implementation -> Post-project Technical Support.
4. FAQs
4.1. Why is there latency with the DexHand?
From the moment the human hand starts moving to the point where the DexHand's SDK is called to initiate movement (excluding network transmission delay), the process takes approximately 20ms. The time from calling the SDK to the DexHand physically moving depends on the product from the DexHand manufacturer. This includes both software and hardware drivers, with hardware driver performance being related to the physical implementation technology of the DexHand.
Therefore, some DexHands can maintain near real-time synchronization with the human hand, while others exhibit noticeable latency. The minimum latency perceptible to an average person is around 20ms.
Last updated