Dataset DOI: 10.5061/dryad.xksn02vv9

Description of the data and file structure

Experimental Protocol and Data Recording

We recruited twenty violinist dyads (ten amateur and ten professional dyads) who performed a custom musical piece across four different sensory feedback conditions—auditory only (A), auditory-visual (AV), auditory-haptic (AH), auditory-visual-haptic (AVH)—at two different tempi (72, 100 bpm). In the haptic conditions (AH, AVH), the coupled exoskeletons delivered real-time torques based on the joint angular difference between partners, while in the non-haptic conditions (A, AV) they delivered torques to only compensate for friction and inertia. Moreover, before and after the sensory feedback conditions, dyads also performed the same musical piece in conventional settings, i.e., without the exoskeletons and with auditory and visual feedback (baseline assessment BL, post-test assessment PT).

We recorded motion data from a motion capture (MoCap) system at 120 Hz using a Qualisys Motion Capture system (Qualisys AB, Sweden) that included 18 qualisys 6+ infrared cameras and one RGB camera. Audio data of the performance was recorded with a Y-pair of condenser microphones at 48 kHz and a bit depth of 24 bits. Data recorded by the exoskeleton, i.e., measured and desired joint angles and torques, were sampled at 100 Hz from the real-time loop and stored in the internal memory of the robots. All the questionnaire data were collected via Qualtrics software (www.qualtrics.com).

Data Structure

Data are separated in different files:

- data_joint_angles.mat contains the joint angles data extracted from the MoCap recordings.

- data_bow.mat contains the relative position of each pair of violin and bow from the MoCap recordings.

- data_coherenceRMS.csv contains the data of the coherence analysis of the audio recordings.

- data_spectral_centroid.csv contains the data of the spectral centroids of the audio recordings.

- data_questionnaire.csv contains the data from the questionnaires.

Data from the MoCap (data_joint_angles and data_bow) are organized as follows. A main structure stores all the data from each dyad, and within each dyad the data from all the blocks are stored. The blocks are stored in a pre-defined order for all the dyads: B_72_1, B_72_2, B_72_3, B_100_1, B_100_2, B_100_3, BL_72, PT_72, where 72/100 represents the tempo of the block and, if present, the last number represents the block number for each tempo. Finally, BL and PT refer to the baseline (BL) and post-test (PT) assessments (both executed at 72 bpm without the exoskeletons). Each block in turns contains all the corresponding trials: A, AV, AH, AVH for the experimental blocks, and T1 and T2 for the no-exoskeleton conditions.

For the joint angles data (data_joint_angles), each trial contains the joint angles data (shoulder, elbow and wrist) for both members of the dyad (p1 and p2), extracted using Euler angle decomposition and the following rotation sequences: i) shoulder – ZYX: Z (flexion/extension), Y (abduction/adduction), X (internal/external rotation); elbow – ZXY: Z (flexion/extension), X (not assigned), Y (not assigned); wrist – XYZ: X (pronation/supination), Y (flexion/extension), Z (radial/ulnar deviation). Moreover, the structure contains the time variables (seconds).

For relative violin/bow position (data_bow), each trial contains the relative positions of the centroids for each participant.

Data from the audio recordings (data_coherenceRMS, data_spectral_centroid) are organized in tables with the following headers:

- data_coherenceRMS: trial_id, Dyad, Block, Trial, Condition, Expertise, mean_coherence

- data_spectral_centroid: expertise, Dyad, Condition, Part, Tempo, mean_br

The variable "mean_coherence" represents the average coherence (a.u) derived from wavelet coherence analysis, whereas the variable "mean_br" represents the average spectral brightness (a.u.).

Data from the questionnaires (data_questionnaire) are organized in a table with the following headers: Placeholder, Dyad, Participant, DyadParticipant, Block, Trial, Condition, Expertise, Dyad_sex, Tempo, JointP, OwnP, QualityInt, EaseCor. The four variables assessed through the questions are Joint Performance (JointP), Own Performance (OwnP), Quality of Interaction (QualityInt) and Ease of Coordination (EaseCor).

Data Analysis

The assessment of spatiotemporal coordination was conducted by considering the movements of the end-effectors of the human upper-limb kinematic chain in the task space, i.e., the movement of the bow with respect to the violin, for each member of the dyads, and joint angular trajectories (shoulder, elbow, wrist) of the right upper limb, which holds the bow. The coordination was then quantified by comparing the movements of the virtually coupled end-effectors and right upper limbs.

Spatial bow coordination was assessed by tracking the position of each bow relative to its respective violin, extracting principal movement patterns using Principal Component Analysis (PCA), and computing spatial coordination as the Mean Absolute Error between the two violinists. Temporal synchronization was estimated as the relative phase of the Hilbert-transformed signals.

Joint coordination was evaluated by extracting upper-limb joint angles and applying Partial Least Squares Correlation (PLSC) to identify the most strongly correlated components between participants. Spatial coupling was calculated as the Euclidean distance between these components, and temporal correlation was computed using the Pearson’s correlation coefficient. These metrics captured the level of spatial and temporal coordination between musicians across conditions, both in task space and joint space.

As defined here, bow/joint spatial coordination corresponds to the average error between the dominant components of the movements—thus, lower values indicate higher spatial coordination. In contrast, bow/joint temporal coordination reflects the temporal correlation between the dominant components— hence, higher values represent stronger temporal coordination.

Kinematic data were parsed and processed in MATLAB R2024b. Specifically, two scripts were produced:

- scirob_data_processing_bow.m: this script allows to process the mocap data and calculate spatiotemporal bow coordination. This script utilizes the MATLAB built-in functions "pca" and "canoncorr" (Statistics and Machine Learning Toolbox), and "hilbert" (Signal Processing Toolbox);

- scirob_data_processing_joints.m: this script allows to process the joint angle data and calculate spatiotemporal joint coordination. This script utilizes the MATLAB built-in functions "canoncorr" and "corr" (Statistics and Machine Learning Toolbox).

Statistical analysis was performed in IBM SPSS Statistics 26.0 and the log output of the analysis is stored in the file "IBM_SPSS_log_output_kinematic_analysis.txt".

To evaluate whether haptic feedback and wearing a robotic exoskeleton influenced the acoustic quality of violin performance, we analyzed two distinct audio-based parameters: spectral brightness, indexed by the spectral centroid—a measure commonly associated with perceived timbre and bow pressure in violin performance—and dynamic alignment—defined as the temporal synchronization of loudness changes (RMS amplitude) between players.

Audio data were processed in R (version 4.3.2). Specifically, four scripts were generated:

- spectral_centroid_feature_extraction.R: this script allows to extract the main features of the spectral centroid analysis.

- spectral_centroid_statistics.R: this script allows to run a statistical analysis on the spectral centroid data and requires the packages "lmerTest" and "emmeans".

- wavelet_coherence_rms_feature_extraction.R: this script allows to extract the main features of the wavelet coherence analysis.

- wavelet_coherence_statistics.R: this script allows to run a statistical analysis on the wavelet coherence data and requires the packages "lmerTest" and "emmeans".

Finally, questionnaire data were parsed and processed in R (version 4.3.2) with the script questionnaires_statistics.R.

Human subjects data

The study was approved by the Ghent University Ethics committee (ref. 2023–29) and conducted according to the Declaration of Helsinki. Informed consent was obtained from all participants prior to the study.