Results
Metrics
- Force Profile Accuracy (FPAcc): Measures how closely the predicted force profile \( \hat{x}_f \) matches the ground truth \( x_f \) using Mean Squared Error (MSE). $$ \text{FPAcc} = \text{MSE}(\hat{x}_f, x_f) $$
- Force Direction Accuracy (FDAcc): Evaluates directional alignment between predicted impulse \( \hat{J}(T) \) and ground truth impulse \( J(T) \) using cosine similarity. $$ \text{FDAcc} = \frac{\hat{J}(T) \cdot J(T)}{||\hat{J}(T)|| \cdot ||J(T)||} $$
- Modifier Similarity (ModSim): Cosine similarity between the predicted modifier \( \hat{w}_m \) and the ground truth \( w_m \) using SBERT embeddings \( E \). $$ \text{ModSim} = \frac{E(\hat{w}_m) \cdot E(w_m)}{||E(\hat{w}_m)|| \cdot ||E(w_m)||} $$
- Direction Similarity (DirSim): Cosine similarity between predicted direction words \( \hat{w}_d \) and ground truth \( w_d \). $$ \text{DirSim} = \frac{E(\hat{w}_d) \cdot E(w_d)}{||E(\hat{w}_d)|| \cdot ||E(w_d)||} $$
- Full Phrase Similarity (PhraseSim): Average of \( \text{ModSim} \) and \( \text{DirSim} \). $$ \text{PhraseSim} = \frac{1}{2} \left(\text{ModSim} + \text{DirSim} \right) $$
Baseline Models
- $\text{SVM/KNN}$: Uses SVM to classify force signals into phrases and KNN to predict force profiles from phrases.
- $\text{DMLP}_B$: Direct MLP mapping between force profiles and binary phrase vectors without a shared latent space.
- $\text{DMLP}_G$: Direct MLP mapping between force profiles and GloVe-embedded phrases without a shared latent space.
- $\text{DAE}_B$ (Ours): Dual autoencoder with binary phrase vectors.
- $\text{DAE}G$ (Ours): Dual autoencoder with GloVe embeddings.
Experiments
To evaluate the effectiveness of our cross-modality embedding framework, we conducted three main experiments:
- In-Distribution Evaluation: The dataset was randomly split into 90% training and 10% testing sets. Models were trained and tested on the same distribution of force–phrase pairs and averaged across 30 independent seeds to reduce variability in the results.
- Out-of-Distribution Modifiers: To test generalization, we systematically held out one modifier word (e.g., “slowly”) from the training set while keeping it in the test set. This experiment measured how well the models handled unseen adverbial variations in force descriptions.
- Out-of-Distribution Directions: Similar to the modifier experiment, we excluded a single direction word (e.g., “up”) from training and evaluated the model’s ability to generalize to new directional inputs. This tested whether the models could infer force trajectories for previously unseen movement directions.
Overall Results
Our framework outperforms baseline models in both force-to-language and language-to-force translation, demonstrating the effectiveness of a shared latent space.



Analysis
Our results showed that the dual autoencoder (DAE) models significantly outperformed baseline approaches in both force-to-language and language-to-force translation. The DAE models achieved 20-30% better performance than simpler SVM and MLP-based baselines, demonstrating that a shared latent space effectively captures the bidirectional relationship between physical force and linguistic descriptions.
Additionally, the models successfully generalized to unseen modifiers and directions, particularly when using GloVe embeddings, which improved force profile reconstruction. However, we observed a trade-off between force reconstruction accuracy and linguistic precision: the GloVe-based model (\( \text{DAE}_G \)) was better at estimating realistic force trajectories, while the binary phrase vector model (\( \text{DAE}_B \)) provided more precise textual descriptions.
These findings confirm that integrating force and language into a unified representation enhances natural human-robot communication, with different representation choices offering advantages depending on whether force accuracy or linguistic fidelity is prioritized.
Additional Results
Translation Examples
Example predictions from both $\text{DAE}_B$ and $\text{DAE}_G$ variants for corresponding force profile—phrase pairs. We generated 10 different 90-10 training-test splits, ensuring that the union of all test sets covers the full dataset. Each data point was randomly assigned to one of 10 bins, with each bin serving as a distinct test set in one of the splits. The model was trained on the remaining 9 bins and evaluated on the held-out test bin. The examples below show model predictions for cases where the corresponding force-phrase pair was not explicitly seen during training.
Phrase-To-Force TrialsCorresponding force profile and phrase pairs collected from the Phrase-To-Force procedure.
User ID | Trial # | Phrase Text | Modifier | Direction | Trial Card |
---|---|---|---|---|---|
0 | 0 | down | (None) | down | View |
0 | 1 | forward | (None) | forward | View |
0 | 2 | up | (None) | up | View |
0 | 3 | backward | (None) | backward | View |
0 | 4 | left | (None) | left | View |
0 | 5 | right | (None) | right | View |
0 | 6 | below sharply | sharply | down | View |
0 | 7 | sharply left and downward | sharply | down-left | View |
0 | 8 | leftward and behind | (None) | backward-left | View |
0 | 9 | up and to the right immediately | immediately | right-up | View |
0 | 10 | to the left and above significantly | significantly | left-up | View |
0 | 11 | sharply behind and right | sharply | backward-right | View |
0 | 12 | slowly leftward | slowly | left | View |
0 | 13 | right and backward | (None) | backward-right | View |
0 | 14 | immediately to the right | immediately | right | View |
0 | 15 | immediately ahead and up | immediately | forward-up | View |
0 | 16 | ahead smoothly | smoothly | forward | View |
0 | 17 | to the right and forward | (None) | forward-right | View |
0 | 18 | leftward and to the front | (None) | forward-left | View |
0 | 19 | backward and leftward | (None) | backward-left | View |
0 | 20 | to the front and down harshly | harshly | down-forward | View |
0 | 21 | harshly to the front and right | harshly | forward-right | View |
0 | 22 | softly below and right | softly | down-right | View |
0 | 23 | forward lightly | lightly | forward | View |
0 | 24 | to the front and to the left sharply | sharply | forward-left | View |
0 | 25 | sharply downward | sharply | down | View |
0 | 26 | lightly backward | lightly | backward | View |
0 | 27 | downward and right | (None) | down-right | View |
0 | 28 | to the left and backward | (None) | backward-left | View |
0 | 29 | ahead softly | softly | forward | View |
0 | 30 | down and left immediately | immediately | down-left | View |
0 | 31 | forward and below | (None) | down-forward | View |
0 | 32 | lightly backward | lightly | backward | View |
0 | 33 | left sharply | sharply | left | View |
0 | 34 | above and ahead | (None) | forward-up | View |
0 | 35 | left and to the back | (None) | backward-left | View |
0 | 36 | ahead slowly | slowly | forward | View |
0 | 37 | downward and left | (None) | down-left | View |
0 | 38 | immediately forward and up | immediately | forward-up | View |
0 | 39 | left and forward greatly | greatly | forward-left | View |
0 | 40 | slowly above | slowly | up | View |
0 | 41 | downward and behind | (None) | backward-down | View |
1 | 0 | left | (None) | left | View |
1 | 1 | forward | (None) | forward | View |
1 | 2 | up | (None) | up | View |
1 | 3 | right | (None) | right | View |
1 | 4 | down | (None) | down | View |
1 | 5 | backward | (None) | backward | View |
1 | 6 | to the front and downward | (None) | down-forward | View |
1 | 7 | to the back quickly | quickly | backward | View |
1 | 8 | smoothly right | smoothly | right | View |
1 | 9 | quickly rightward and downward | quickly | down-right | View |
1 | 10 | to the front and left slightly | slightly | forward-left | View |
1 | 11 | below significantly | significantly | down | View |
1 | 12 | ahead softly | softly | forward | View |
1 | 13 | quickly above | quickly | up | View |
1 | 14 | above and rightward greatly | greatly | right-up | View |
1 | 15 | rightward and up | (None) | right-up | View |
1 | 16 | forward softly | softly | forward | View |
1 | 17 | greatly forward | greatly | forward | View |
1 | 18 | below and to the front | (None) | down-forward | View |
1 | 19 | to the back and up | (None) | backward-up | View |
1 | 20 | lightly up and to the left | lightly | left-up | View |
1 | 21 | slowly below | slowly | down | View |
1 | 22 | above slightly | slightly | up | View |
1 | 23 | smoothly behind | smoothly | backward | View |
1 | 24 | slightly behind | slightly | backward | View |
1 | 25 | slowly down and forward | slowly | down-forward | View |
1 | 26 | quickly to the right and to the front | quickly | forward-right | View |
1 | 27 | immediately to the left and backward | immediately | backward-left | View |
1 | 28 | to the right and upward significantly | significantly | right-up | View |
1 | 29 | rightward and ahead | (None) | forward-right | View |
1 | 30 | below and to the left | (None) | down-left | View |
1 | 31 | upward and to the back | (None) | backward-up | View |
1 | 32 | significantly downward and to the right | significantly | down-right | View |
1 | 33 | to the front and to the left quickly | quickly | forward-left | View |
1 | 34 | below and forward | (None) | down-forward | View |
1 | 35 | downward and behind | (None) | backward-down | View |
1 | 36 | rightward and below | (None) | down-right | View |
1 | 37 | upward and to the right | (None) | right-up | View |
1 | 38 | backward and below | (None) | backward-down | View |
1 | 39 | to the back and left quickly | quickly | backward-left | View |
1 | 40 | forward and leftward sharply | sharply | forward-left | View |
1 | 41 | above smoothly | smoothly | up | View |
2 | 0 | down | (None) | down | View |
2 | 1 | forward | (None) | forward | View |
2 | 2 | right | (None) | right | View |
2 | 3 | up | (None) | up | View |
2 | 4 | backward | (None) | backward | View |
2 | 5 | left | (None) | left | View |
2 | 6 | slowly ahead and upward | slowly | forward-up | View |
2 | 7 | softly upward | softly | up | View |
2 | 8 | below and leftward | (None) | down-left | View |
2 | 9 | significantly down | significantly | down | View |
2 | 10 | to the back immediately | immediately | backward | View |
2 | 11 | ahead sharply | sharply | forward | View |
2 | 12 | leftward and backward immediately | immediately | backward-left | View |
2 | 13 | greatly backward and downward | greatly | backward-down | View |
2 | 14 | quickly to the left | quickly | left | View |
2 | 15 | to the front and upward | (None) | forward-up | View |
2 | 16 | to the back and upward | (None) | backward-up | View |
2 | 17 | backward and down slowly | slowly | backward-down | View |
2 | 18 | left and backward | (None) | backward-left | View |
2 | 19 | to the right and down | (None) | down-right | View |
2 | 20 | left and downward | (None) | down-left | View |
2 | 21 | slowly downward and to the back | slowly | backward-down | View |
2 | 22 | softly backward | softly | backward | View |
2 | 23 | below and to the right | (None) | down-right | View |
2 | 24 | downward and ahead | (None) | down-forward | View |
2 | 25 | smoothly left and behind | smoothly | backward-left | View |
2 | 26 | upward and leftward slowly | slowly | left-up | View |
2 | 27 | softly behind | softly | backward | View |
2 | 28 | backward and right quickly | quickly | backward-right | View |
2 | 29 | upward slowly | slowly | up | View |
2 | 30 | downward gradually | gradually | down | View |
2 | 31 | to the back and to the left | (None) | backward-left | View |
2 | 32 | to the back and rightward slowly | slowly | backward-right | View |
2 | 33 | to the front and downward | (None) | down-forward | View |
2 | 34 | lightly down and forward | lightly | down-forward | View |
2 | 35 | slightly to the back | slightly | backward | View |
2 | 36 | softly above and backward | softly | backward-up | View |
2 | 37 | left and down | (None) | down-left | View |
2 | 38 | to the left significantly | significantly | left | View |
2 | 39 | behind and above immediately | immediately | backward-up | View |
2 | 40 | behind and above | (None) | backward-up | View |
2 | 41 | above gradually | gradually | up | View |
3 | 0 | down | (None) | down | View |
3 | 1 | up | (None) | up | View |
3 | 2 | forward | (None) | forward | View |
3 | 3 | backward | (None) | backward | View |
3 | 4 | left | (None) | left | View |
3 | 5 | right | (None) | right | View |
3 | 6 | harshly backward and leftward | harshly | backward-left | View |
3 | 7 | leftward and below | (None) | down-left | View |
3 | 8 | down and left harshly | harshly | down-left | View |
3 | 9 | right and upward softly | softly | right-up | View |
3 | 10 | significantly to the back | significantly | backward | View |
3 | 11 | slightly to the front | slightly | forward | View |
3 | 12 | quickly down and backward | quickly | backward-down | View |
3 | 13 | forward sharply | sharply | forward | View |
3 | 14 | down and to the right | (None) | down-right | View |
3 | 15 | slowly to the back | slowly | backward | View |
3 | 16 | above and right quickly | quickly | right-up | View |
3 | 17 | downward and to the left | (None) | down-left | View |
3 | 18 | gradually behind and left | gradually | backward-left | View |
3 | 19 | to the left lightly | lightly | left | View |
3 | 20 | behind and left | (None) | backward-left | View |
3 | 21 | leftward and below lightly | lightly | down-left | View |
3 | 22 | leftward smoothly | smoothly | left | View |
3 | 23 | rightward and to the back | (None) | backward-right | View |
3 | 24 | up and to the back immediately | immediately | backward-up | View |
3 | 25 | upward and to the right | (None) | right-up | View |
3 | 26 | gradually rightward and below | gradually | down-right | View |
3 | 27 | slowly down and rightward | slowly | down-right | View |
3 | 28 | slowly up | slowly | up | View |
3 | 29 | down quickly | quickly | down | View |
3 | 30 | behind and downward | (None) | backward-down | View |
3 | 31 | below and behind | (None) | backward-down | View |
3 | 32 | leftward and below | (None) | down-left | View |
3 | 33 | slowly to the front | slowly | forward | View |
3 | 34 | slightly above | slightly | up | View |
3 | 35 | above slightly | slightly | up | View |
3 | 36 | ahead and below | (None) | down-forward | View |
3 | 37 | down and forward slowly | slowly | down-forward | View |
3 | 38 | to the front softly | softly | forward | View |
3 | 39 | right and above | (None) | right-up | View |
3 | 40 | softly up and to the back | softly | backward-up | View |
3 | 41 | downward and backward | (None) | backward-down | View |
4 | 0 | down | (None) | down | View |
4 | 1 | forward | (None) | forward | View |
4 | 2 | left | (None) | left | View |
4 | 3 | backward | (None) | backward | View |
4 | 4 | right | (None) | right | View |
4 | 5 | up | (None) | up | View |
4 | 6 | greatly ahead and leftward | greatly | forward-left | View |
4 | 7 | right harshly | harshly | right | View |
4 | 8 | forward and rightward | (None) | forward-right | View |
4 | 9 | slightly right | slightly | right | View |
4 | 10 | rightward and down | (None) | down-right | View |
4 | 11 | backward significantly | significantly | backward | View |
4 | 12 | quickly up | quickly | up | View |
4 | 13 | up greatly | greatly | up | View |
4 | 14 | below and backward immediately | immediately | backward-down | View |
4 | 15 | quickly up and to the right | quickly | right-up | View |
4 | 16 | quickly behind and to the right | quickly | backward-right | View |
4 | 17 | to the back and below | (None) | backward-down | View |
4 | 18 | gradually up | gradually | up | View |
4 | 19 | lightly left | lightly | left | View |
4 | 20 | behind and up greatly | greatly | backward-up | View |
4 | 21 | downward and to the front | (None) | down-forward | View |
4 | 22 | significantly ahead | significantly | forward | View |
4 | 23 | backward immediately | immediately | backward | View |
4 | 24 | leftward and to the back | (None) | backward-left | View |
4 | 25 | above and leftward gradually | gradually | left-up | View |
4 | 26 | to the left and upward | (None) | left-up | View |
4 | 27 | to the front and leftward | (None) | forward-left | View |
4 | 28 | upward and to the front | (None) | forward-up | View |
4 | 29 | leftward and forward | (None) | forward-left | View |
4 | 30 | gradually to the back and down | gradually | backward-down | View |
4 | 31 | quickly leftward | quickly | left | View |
4 | 32 | quickly ahead and left | quickly | forward-left | View |
4 | 33 | immediately backward and right | immediately | backward-right | View |
4 | 34 | ahead and right | (None) | forward-right | View |
4 | 35 | ahead and leftward gradually | gradually | forward-left | View |
4 | 36 | right and backward | (None) | backward-right | View |
4 | 37 | upward and to the back | (None) | backward-up | View |
4 | 38 | to the right immediately | immediately | right | View |
4 | 39 | to the right and backward softly | softly | backward-right | View |
4 | 40 | to the back and to the right greatly | greatly | backward-right | View |
4 | 41 | ahead harshly | harshly | forward | View |
5 | 0 | left | (None) | left | View |
5 | 1 | backward | (None) | backward | View |
5 | 2 | up | (None) | up | View |
5 | 3 | right | (None) | right | View |
5 | 4 | forward | (None) | forward | View |
5 | 5 | down | (None) | down | View |
5 | 6 | upward and leftward | (None) | left-up | View |
5 | 7 | upward and left slightly | slightly | left-up | View |
5 | 8 | to the front and upward | (None) | forward-up | View |
5 | 9 | gradually to the front and above | gradually | forward-up | View |
5 | 10 | ahead and right | (None) | forward-right | View |
5 | 11 | to the front greatly | greatly | forward | View |
5 | 12 | downward and right gradually | gradually | down-right | View |
5 | 13 | up and to the back | (None) | backward-up | View |
5 | 14 | slightly to the back and upward | slightly | backward-up | View |
5 | 15 | to the back and leftward | (None) | backward-left | View |
5 | 16 | smoothly left | smoothly | left | View |
5 | 17 | downward and ahead | (None) | down-forward | View |
5 | 18 | leftward and forward | (None) | forward-left | View |
5 | 19 | greatly behind | greatly | backward | View |
5 | 20 | sharply to the front | sharply | forward | View |
5 | 21 | up softly | softly | up | View |
5 | 22 | to the back and to the left | (None) | backward-left | View |
5 | 23 | ahead and right | (None) | forward-right | View |
5 | 24 | harshly ahead | harshly | forward | View |
5 | 25 | backward immediately | immediately | backward | View |
5 | 26 | downward and left gradually | gradually | down-left | View |
5 | 27 | backward greatly | greatly | backward | View |
5 | 28 | harshly up | harshly | up | View |
5 | 29 | significantly backward and below | significantly | backward-down | View |
5 | 30 | greatly down and to the front | greatly | down-forward | View |
5 | 31 | to the left lightly | lightly | left | View |
5 | 32 | ahead and downward | (None) | down-forward | View |
5 | 33 | upward and behind sharply | sharply | backward-up | View |
5 | 34 | sharply rightward | sharply | right | View |
5 | 35 | downward and forward | (None) | down-forward | View |
5 | 36 | behind quickly | quickly | backward | View |
5 | 37 | smoothly leftward and behind | smoothly | backward-left | View |
5 | 38 | forward and upward quickly | quickly | forward-up | View |
5 | 39 | softly below and left | softly | down-left | View |
5 | 40 | above and to the left | (None) | left-up | View |
5 | 41 | leftward and below lightly | lightly | down-left | View |
6 | 0 | left | (None) | left | View |
6 | 1 | right | (None) | right | View |
6 | 2 | forward | (None) | forward | View |
6 | 3 | up | (None) | up | View |
6 | 4 | backward | (None) | backward | View |
6 | 5 | down | (None) | down | View |
6 | 6 | sharply upward | sharply | up | View |
6 | 7 | to the right and behind | (None) | backward-right | View |
6 | 8 | up and backward greatly | greatly | backward-up | View |
6 | 9 | greatly below | greatly | down | View |
6 | 10 | ahead and downward | (None) | down-forward | View |
6 | 11 | downward and backward | (None) | backward-down | View |
6 | 12 | right and downward | (None) | down-right | View |
6 | 13 | to the left and down gradually | gradually | down-left | View |
6 | 14 | ahead lightly | lightly | forward | View |
6 | 15 | above and behind | (None) | backward-up | View |
6 | 16 | softly rightward and to the back | softly | backward-right | View |
6 | 17 | above quickly | quickly | up | View |
6 | 18 | behind and above | (None) | backward-up | View |
6 | 19 | to the back harshly | harshly | backward | View |
6 | 20 | rightward and up | (None) | right-up | View |
6 | 21 | quickly to the back | quickly | backward | View |
6 | 22 | to the left and above harshly | harshly | left-up | View |
6 | 23 | softly right and down | softly | down-right | View |
6 | 24 | rightward and backward slowly | slowly | backward-right | View |
6 | 25 | lightly down and left | lightly | down-left | View |
6 | 26 | slowly to the right and to the front | slowly | forward-right | View |
6 | 27 | significantly ahead and up | significantly | forward-up | View |
6 | 28 | to the left and to the back smoothly | smoothly | backward-left | View |
6 | 29 | slightly left | slightly | left | View |
6 | 30 | to the front and right | (None) | forward-right | View |
6 | 31 | ahead immediately | immediately | forward | View |
6 | 32 | downward and left quickly | quickly | down-left | View |
6 | 33 | left smoothly | smoothly | left | View |
6 | 34 | to the front and to the right | (None) | forward-right | View |
6 | 35 | leftward and up | (None) | left-up | View |
6 | 36 | slightly to the back | slightly | backward | View |
6 | 37 | greatly rightward | greatly | right | View |
6 | 38 | upward and right | (None) | right-up | View |
6 | 39 | slightly above and backward | slightly | backward-up | View |
6 | 40 | to the left and ahead | (None) | forward-left | View |
6 | 41 | above immediately | immediately | up | View |
7 | 0 | right | (None) | right | View |
7 | 1 | down | (None) | down | View |
7 | 2 | forward | (None) | forward | View |
7 | 3 | backward | (None) | backward | View |
7 | 4 | left | (None) | left | View |
7 | 5 | up | (None) | up | View |
7 | 6 | harshly to the left | harshly | left | View |
7 | 7 | backward and above | (None) | backward-up | View |
7 | 8 | to the right slightly | slightly | right | View |
7 | 9 | smoothly to the back | smoothly | backward | View |
7 | 10 | below and forward | (None) | down-forward | View |
7 | 11 | gradually below | gradually | down | View |
7 | 12 | downward slightly | slightly | down | View |
7 | 13 | backward and rightward | (None) | backward-right | View |
7 | 14 | down and right | (None) | down-right | View |
7 | 15 | ahead and down lightly | lightly | down-forward | View |
7 | 16 | sharply rightward and ahead | sharply | forward-right | View |
7 | 17 | backward quickly | quickly | backward | View |
7 | 18 | downward lightly | lightly | down | View |
7 | 19 | above and to the right gradually | gradually | right-up | View |
7 | 20 | to the back and upward | (None) | backward-up | View |
7 | 21 | above and leftward | (None) | left-up | View |
7 | 22 | up and to the left sharply | sharply | left-up | View |
7 | 23 | quickly left and forward | quickly | forward-left | View |
7 | 24 | harshly downward | harshly | down | View |
7 | 25 | ahead and downward | (None) | down-forward | View |
7 | 26 | up harshly | harshly | up | View |
7 | 27 | gradually to the left and backward | gradually | backward-left | View |
7 | 28 | lightly leftward | lightly | left | View |
7 | 29 | up and rightward smoothly | smoothly | right-up | View |
7 | 30 | immediately leftward | immediately | left | View |
7 | 31 | to the right and forward | (None) | forward-right | View |
7 | 32 | down and behind | (None) | backward-down | View |
7 | 33 | forward sharply | sharply | forward | View |
7 | 34 | forward and to the right quickly | quickly | forward-right | View |
7 | 35 | softly to the front and leftward | softly | forward-left | View |
7 | 36 | upward and left | (None) | left-up | View |
7 | 37 | gradually downward and rightward | gradually | down-right | View |
7 | 38 | left and forward | (None) | forward-left | View |
7 | 39 | smoothly forward and down | smoothly | down-forward | View |
7 | 40 | rightward and down | (None) | down-right | View |
7 | 41 | to the right and ahead slowly | slowly | forward-right | View |
8 | 0 | up | (None) | up | View |
8 | 1 | forward | (None) | forward | View |
8 | 2 | left | (None) | left | View |
8 | 3 | backward | (None) | backward | View |
8 | 4 | down | (None) | down | View |
8 | 5 | right | (None) | right | View |
8 | 6 | downward smoothly | smoothly | down | View |
8 | 7 | upward and ahead | (None) | forward-up | View |
8 | 8 | right lightly | lightly | right | View |
8 | 9 | rightward and ahead | (None) | forward-right | View |
8 | 10 | harshly down and to the left | harshly | down-left | View |
8 | 11 | backward and below | (None) | backward-down | View |
8 | 12 | right softly | softly | right | View |
8 | 13 | right and downward | (None) | down-right | View |
8 | 14 | smoothly to the left | smoothly | left | View |
8 | 15 | to the right immediately | immediately | right | View |
8 | 16 | below and to the back slowly | slowly | backward-down | View |
8 | 17 | leftward and downward | (None) | down-left | View |
8 | 18 | downward and rightward | (None) | down-right | View |
8 | 19 | greatly downward and rightward | greatly | down-right | View |
8 | 20 | leftward and behind | (None) | backward-left | View |
8 | 21 | quickly forward | quickly | forward | View |
8 | 22 | below and to the left | (None) | down-left | View |
8 | 23 | slowly below and right | slowly | down-right | View |
8 | 24 | to the right immediately | immediately | right | View |
8 | 25 | above and to the front slowly | slowly | forward-up | View |
8 | 26 | below and to the left | (None) | down-left | View |
8 | 27 | ahead slightly | slightly | forward | View |
8 | 28 | ahead and to the right | (None) | forward-right | View |
8 | 29 | left and down slightly | slightly | down-left | View |
8 | 30 | significantly to the back | significantly | backward | View |
8 | 31 | downward and ahead | (None) | down-forward | View |
8 | 32 | forward and to the right sharply | sharply | forward-right | View |
8 | 33 | quickly above | quickly | up | View |
8 | 34 | left and backward significantly | significantly | backward-left | View |
8 | 35 | right and upward | (None) | right-up | View |
8 | 36 | harshly up and rightward | harshly | right-up | View |
8 | 37 | immediately to the front and right | immediately | forward-right | View |
8 | 38 | smoothly ahead | smoothly | forward | View |
8 | 39 | sharply below | sharply | down | View |
8 | 40 | immediately to the left and up | immediately | left-up | View |
8 | 41 | upward and to the front gradually | gradually | forward-up | View |
9 | 0 | left | (None) | left | View |
9 | 1 | right | (None) | right | View |
9 | 2 | down | (None) | down | View |
9 | 3 | backward | (None) | backward | View |
9 | 4 | forward | (None) | forward | View |
9 | 5 | up | (None) | up | View |
9 | 6 | leftward and to the back | (None) | backward-left | View |
9 | 7 | greatly rightward | greatly | right | View |
9 | 8 | to the front and rightward | (None) | forward-right | View |
9 | 9 | behind and down | (None) | backward-down | View |
9 | 10 | upward quickly | quickly | up | View |
9 | 11 | upward and ahead quickly | quickly | forward-up | View |
9 | 12 | to the back and above gradually | gradually | backward-up | View |
9 | 13 | harshly below | harshly | down | View |
9 | 14 | up and backward | (None) | backward-up | View |
9 | 15 | forward and up | (None) | forward-up | View |
9 | 16 | right and up | (None) | right-up | View |
9 | 17 | behind gradually | gradually | backward | View |
9 | 18 | above gradually | gradually | up | View |
9 | 19 | leftward and up | (None) | left-up | View |
9 | 20 | harshly right | harshly | right | View |
9 | 21 | down and to the front | (None) | down-forward | View |
9 | 22 | gradually below | gradually | down | View |
9 | 23 | to the right and up | (None) | right-up | View |
9 | 24 | immediately to the right and down | immediately | down-right | View |
9 | 25 | right and forward | (None) | forward-right | View |
9 | 26 | significantly to the left and to the front | significantly | forward-left | View |
9 | 27 | slowly below and ahead | slowly | down-forward | View |
9 | 28 | down and right | (None) | down-right | View |
9 | 29 | to the left sharply | sharply | left | View |
9 | 30 | right smoothly | smoothly | right | View |
9 | 31 | slightly to the front and rightward | slightly | forward-right | View |
9 | 32 | above and rightward | (None) | right-up | View |
9 | 33 | smoothly downward | smoothly | down | View |
9 | 34 | significantly to the right and down | significantly | down-right | View |
9 | 35 | up and forward greatly | greatly | forward-up | View |
9 | 36 | right and downward slightly | slightly | down-right | View |
9 | 37 | smoothly below and forward | smoothly | down-forward | View |
9 | 38 | backward and leftward immediately | immediately | backward-left | View |
9 | 39 | up slightly | slightly | up | View |
9 | 40 | smoothly up | smoothly | up | View |
9 | 41 | leftward and to the front immediately | immediately | forward-left | View |
Out-of-Distribution Modifiers
The model successfully generalizes to unseen modifiers, with GloVe embeddings improving force profile reconstruction.












Out-of-Distribution Directions
Our approach effectively handles unseen directional inputs, capturing force-language relationships even for novel movement directions.

















