We conducted a realism evaluation to focusing on how well the pitch expression and overall timbre of the synthesized samples matched those of actual violin performances.
The evaluation was carried out using the MUSHRA {1} protocol, a standardized tool for subjective audio quality assessment. Participants rated multiple models, including:
ViolinDiff: Our Proposed model
NoBend Model: Against a version of Synthesis Module trained without pitch bend information
Hawthorne et al. {2}: A multi-instrument diffusion-based model focused on synthesizing various instruments in polyphonic contexts.
Maman et al. {3}: A model using performer embeddings to better capture timbre and style in orchestral instrument synthesis.
GM Soundfont (Low Anchor): The lowest quality baseline, serving as an anchor point for comparison.
References
ITU, “Method for the subjective assessment of intermediate quality level of audio systems,” BS.1534, 2014.
Hawthorne et al.“Multi-instrument music synthesis with spectrogram diffusion,” in Proceedings of the ISMIR, 2022
Maman et al. "Performance conditioning for diffusion-based multi-instrument music synthesis,” in ICASSP, 2024
We would like to extend our gratitude to Ben Maman for providing the essential test files, which greatly contributed to this evaluation.