Usage for UDEXREAL Robotics Products
1. Overview
UDEXREAL products in the field of robotics primarily fall into two major categories.
UDEXREAL Intelligent Ultra-sensitive Data Glove Teleoperation: A product centered around UDEXREAL Intelligent Ultra-sensitive Data Glove (UDCap DataGlove, referred to as UDEXREAL Data Glove) for teleoperation control of dexterous hands.
UDEXREAL Robotics Teleoperation Simulation System: This system can connect to UDEXREAL DataGloves and upper-body/full-body motion capture devices to enable teleoperation of robots. It supports simulation operations for URDF models of specific robots.
2. UDEXREAL Intelligent Ultra-sensitive Data Glove Teleoperation
2.1. User Manual
First, you need to understand how to use the DataGlove. This is the main entry of the user manual, where all information can be accessed.
UDEXREAL UDCAP DataGlove Product Introduction2.2. Quick Guide
If you want to get started quickly, refer to the corresponding sections in the user manual:
2.3. Obtaining the HandDriver Customized Version
2.3.1. Obtaining the Software
Since human hands differ significantly from robotic dexterous hands, the HandDriver Customized Version corresponding to the brand and model of the dexterous hand is required. This customized version must be obtained by contacting UDEXREAL's business department.
Adaptation is a core and essential task for robot teleoperation. UDEXREAL typically performs this service free of charge, but it requires providing the dexterous hand to UDEXREAL.
If providing the physical dexterous hand is not feasible, the URDF model of the dexterous hand can be supplied instead. However, as the model may differ from the actual dexterous hand, this approach is not recommended.
2.3.2. Versions of the Software
The HandDriver is available for both Windows and Linux Ubuntu.
The Ubuntu version is currently a standalone application and should be used by referring to The Guide of HandDriver Linux.
A B/S architecture version will be released in the future for easier use—please stay tuned.
2.4. Core Steps for Using the DataGlove
Using the HandDriver customized version to Connect the DataGlove → Calibration → Configuring Data Transmission. These three steps are the core procedures for using the DataGlove. Please refer to the corresponding sections in the user manual.
Calibration is crucial for action accuracy. Please refer to the reference video in the Calibration section.
2.5. How to Operate the Dexterous Hand
2.5.1. Dexterous Hand Control Program
To make the dexterous hand operate, a control program for the dexterous hand needs to be implemented. This program primarily consists of three parts:
Receiving and parsing data sent by HandDriver
After calibration with the gloves on, HandDriver sends real-time (120Hz) angle data for each joint of the human hand via UDP through its data transmission function. The data format is available in two types: joint angles (Euler angles) and quaternions. Typically, joint angles are used to drive the dexterous hand.
The following two documents provide detailed explanations of the data format and how to receive and parse the data sent by HandDriver. The SDK for this is open-source.
C++/Python SDK_Quaternion ManualC++/Python SDK_Angle ManualMapping HandDriver's joint angle data to the corresponding joints of the dexterous hand
Select the required data based on the joints of the dexterous hand and process it into a format suitable for the dexterous hand, such as converting to radian mapping. An example of the mapping is shown below:
Joint 0
output [20]
Joint 1
output [2]
Joint 2
output [1]
Joint 3
output [7]
Joint 4
output [6]
Joint 5
output [10]
Invoke the Dexterous Hand SDK to control the movement of the dexterous hand
Send the processed joint angle data to the dexterous hand SDK (by calling the SDK) to perform angle control.
2.5.2. Collaboration in Practice
In practical collaboration, UDEXREAL does not provide the dexterous hand control program by default. The client is required to implement this program independently, but UDEXREAL will supply the aforementioned mapping relationships. If the dexterous hand control program is needed, please contact the UDEXREAL business department.
2.5.3. Explanation of Hand Motion Spatial Pose
The UDEXREAL DataGlove operates on the principle of flexible sensors. The data it acquires consists of joint angle measurements for each finger, which differs from optical or inertial sensors. It does not provide spatial pose information. If a spatial pose data pathway is required, it must be implemented based on the quaternion documentation mentioned above. These quaternions are derived by converting the joint angle data.
UDEXREAL also provides the initial pose data for the glove model. Please refer to:
Hand Model and Data SpecificationThe final section of this document specifies the position and rotation of each joint point (including fingertips), with the origin located at the midpoint between the bases of the two gloves.
3. UDEXREAL Robotics Teleoperation Simulation System
This system enables teleoperation of robots, either upper-body or full-body. It is offered as a fee-based customized implementation service, tailored to specific robots, dexterous hands, and robotic arms. The joystick and buttons on the UDEXREAL DataGlove can be customized for teleoperation interactions.
Upper-body/full-body motion capture equipment is also selected based on customer's specific situation. UDEXREAL primarily recommends using the PICO headset with PICO motion trackers for teleoperation. The motion trackers are used to acquire the spatial position of the hands.
This system currently supports the direct use of specific robotic arms and dexterous hands, such as the Realman RM65-B with the OYMotion ROH-A001. UDEXREAL welcomes collaborations from all companies.
The customized implementation service process is as follows: Initial Technical Consultation -> Solution Design -> Project Evaluation -> Equipment Procurement -> Project Implementation -> Post-project Technical Support.
4. FAQs
4.1. Why is there latency with the dexterous hand?
From the moment the human hand starts moving to the point where the dexterous hand's SDK is called to initiate movement (excluding network transmission delay), the process takes approximately 20ms. The time from calling the SDK to the dexterous hand physically moving depends on the product from the dexterous hand manufacturer. This includes both software and hardware drivers, with hardware driver performance being related to the physical implementation technology of the dexterous hand.
Therefore, some dexterous hands can maintain near real-time synchronization with the human hand, while others exhibit noticeable latency. The minimum latency perceptible to an average person is around 20ms.
Last updated