Usage for UDEXREAL Robotics Products
1. Overview
UDEXREAL products in the field of robotics primarily fall into two major categories.
UDCap DataGlove Teleoperation: A product centered around UDCap DataGlove (UDCap DataGlove, referred to as UDEXREAL Data Glove) for teleoperation control of dexterous hands.
UDEXREAL Robotics Teleoperation Simulation System: This system can connect to UDEXREAL DataGloves and upper-body/full-body motion capture devices to enable teleoperation of robots. It supports simulation operations for URDF models of specific robots.
2. UDCap DataGlove Teleoperation
2.1. User Manual
First, you need to understand how to use the DataGlove. This is the main entry of the user manual, where all information can be accessed.
UDEXREAL UDCAP DataGlove Product Introduction2.2. Quick Guide
If you want to get started quickly, refer to the corresponding sections in the user manual:
2.3. Obtaining the HandDriver Customized Version
Since human hands differ significantly from robotic dexterous hands, the HandDriver Customized Version corresponding to the brand and model of the dexterous hand is required.
Adaptation is a core and essential task for robot teleoperation. UDEXREAL typically performs this service free of charge, but it requires providing the dexterous hand to UDEXREAL.
If providing the physical dexterous hand is not feasible, the URDF model of the dexterous hand can be supplied instead. However, as the model may differ from the actual dexterous hand, this approach is not recommended.
2.3.1. Versions of the Software
The HandDriver is available for both Windows and Linux Ubuntu. Download Here
For Linux Ubuntu, there are two versions available. The HandDriver Web version is recommended, which adopts a B/S architecture and supports access via a browser with a graphical interface (the interface can be closed). For detailed usage instructions, please visit:
HandDriver Web Deployment and Usage Guide2.3.2. HandDriver Regular Version Supports Dexterous Hand Adaptation
2.3.2.1. Dexterous Hand Character Matching
HandDriver v2.2.1 Windows and Linux Web versions have added the "Dexterous Hand Character Matching" feature. After entering the designated character for a specific dexterous hand in HandDriver, the software will output data compatible with that dexterous hand model. The character codes for the dexterous hands will be provided by UDEXREAL once cooperation is confirmed between both parties.
After entering the dexterous hand character, click the "√" button.
After successfully matching the dexterous hand character, the character will remain in the dropdown list. The system will provide corresponding prompts to indicate whether the character matching succeeded or failed.
2.3.2.2. Dexterous Hand Data Transmission and Action Display
The action display in HandDriver uses human hand data (the "Data" page also shows human hand data), while the data sent externally is determined by the "Dexterous Hand Character Matching" feature.
2.3.3. HandDriver Customized Version
Previously, there existed a HandDriver customized version for specific brand and model dexterous hands. If needed, please contact UDEXTREAL sales to obtain it. Note that the customized version does not support the "Dexterous Hand Character Matching" feature.
2.4. Core Steps for Using the DataGlove
Using the HandDriver to Connect the DataGlove → Calibration → Configuring Data Transmission. These three steps are the core procedures for using the DataGlove. Please refer to the corresponding sections in the user manual.
Calibration is crucial for action accuracy. Please refer to the reference video in the Calibration section.
2.5. How to Operate the Dexterous Hand
2.5.1. Dexterous Hand Motion Control
To make the dexterous hand operate, a control program for the dexterous hand needs to be implemented. This program primarily consists of three parts:
Receiving and parsing data sent by HandDriver
After calibration with the gloves on, HandDriver sends real-time (120Hz) angle data for each joint of the human hand via UDP through its data transmission function. The data format is available in two types: joint angles (Euler angles) and quaternions. Typically, joint angles are used to drive the dexterous hand.
The following two documents provide detailed explanations of the data format and how to receive and parse the data sent by HandDriver. The SDK for this is open-source.
C++/Python SDK_Quaternion ManualC++/Python SDK_Angle ManualRetargeting HandDriver's joint angle data to the corresponding joints of the dexterous hand
Select the required data based on the joints of the dexterous hand and process it into a format suitable for the dexterous hand, such as converting to radian mapping. An example of the mapping is shown below:
Joint 0
output [20]
Joint 1
output [2]
Joint 2
output [1]
Joint 3
output [7]
Joint 4
output [6]
Joint 5
output [10]
Invoke the Dexterous Hand SDK to control the movement of the dexterous hand
Send the processed joint angle data to the dexterous hand SDK (by calling the SDK) to perform angle control.
2.5.2. Business Collaboration
In practical collaboration, UDEXREAL does not provide the dexterous hand motion control by default. The client is required to implement this program independently, but UDEXREAL will supply the aforementioned mapping relationships. If the dexterous hand motion control is needed, please contact the UDEXREAL sales.
2.5.3. Explanation of Hand Motion Spatial Pose
The UDEXREAL DataGlove operates on the principle of flexible sensors. The data it acquires consists of joint angle measurements for each finger, which differs from optical or inertial sensors. It does not provide spatial pose information. If a spatial pose data pathway is required, it must be implemented based on the quaternion documentation mentioned above. These quaternions are derived by converting the joint angle data.
UDEXREAL also provides the initial pose data for the glove model. Please refer to:
Hand Model and Data SpecificationThe final section of this document specifies the position and rotation of each joint point (including fingertips), with the origin located at the midpoint between the bases of the two gloves.
2.6. Data Acquisition
In the software settings, there is a toggle to record the joint angles of the glove for each frame. It is turned off by default.
Function entry: Settings → Shortcut Keys → Data Acquisition
File save path:
The Windows version uses the same "File Save Path" as specified in the settings.
For the HandDriver Web version, data files are stored in the server installation directory.
File naming rule: DeviceName_YYYYMMDDHHMMSS.csv. Example: UDXST4007L_20241210100233.csv. The timestamp indicates the specific time when recording starts. A new file is generated each time HandDriver is launched.
There is no file size limit, and files will not be automatically cleared.
3. UDEXREAL Robotics Teleoperation Simulation System
This system enables teleoperation of robots, either upper-body or full-body. It is offered as a fee-based customized implementation service, tailored to specific robots, dexterous hands, and robotic arms. The joystick and buttons on the UDEXREAL DataGlove can be customized for teleoperation interactions.
Upper-body/full-body motion capture equipment is also selected based on customer's specific situation. UDEXREAL primarily recommends using the PICO headset with PICO motion trackers for teleoperation. The motion trackers are used to acquire the spatial position of the hands.
This system currently supports the direct use of specific robotic arms and dexterous hands, such as the Realman RM65-B with the OYMotion ROH-A001. UDEXREAL welcomes collaborations from all companies.
The customized implementation service process is as follows: Initial Technical Consultation -> Solution Design -> Project Evaluation -> Equipment Procurement -> Project Implementation -> Post-project Technical Support.
4. FAQs
4.1. Why is there latency with the dexterous hand?
From the moment the human hand starts moving to the point where the dexterous hand's SDK is called to initiate movement (excluding network transmission delay), the process takes approximately 20ms. The time from calling the SDK to the dexterous hand physically moving depends on the product from the dexterous hand manufacturer. This includes both software and hardware drivers, with hardware driver performance being related to the physical implementation technology of the dexterous hand.
Therefore, some dexterous hands can maintain near real-time synchronization with the human hand, while others exhibit noticeable latency. The minimum latency perceptible to an average person is around 20ms.
Last updated