Search papers, labs, and topics across Lattice.
This study examines how different architectural and training objective choices in a variational autoencoder (VAE) affect the latent space structure for sign pose encoding in diffusion-based sign language production. By analyzing the Phoenix14T dataset, the authors reveal that the performance of a latent diffusion model for text-to-sign generation is more closely related to latent space properties than to traditional reconstruction quality metrics. The findings indicate that optimizing latent space characteristics can lead to improved generative performance, as evidenced by enhanced back-translation BLEU scores.
Variations in VAE design can significantly influence latent space properties, leading to better generative performance in sign language production than reconstruction accuracy alone.
Latent diffusion approaches to sign language production (SLP) rely on an initial stage that learns an encoding of sign pose sequences, enabling generative modeling in the resulting latent space. The autoencoder used in this stage is typically evaluated in terms of reconstruction quality using geometric metrics common in SLP. While informative, these metrics do not fully capture latent space properties that may influence the training and performance of the downstream generative model. In this work, we investigate how architectural and training objective design choices in a variational autoencoder (VAE) for sign pose encoding affect latent space structure, and how these differences translate into the performance of a latent diffusion model for text-to-sign generation. Our experiments on Phoenix14T dataset show that variations in generative performance, measured through back-translation BLEU scores, can sometimes be better explained by differences in latent space properties than by VAE reconstruction accuracy alone.
