Unveiling Hidden Vulnerabilities in Digital Human Generation via Adversarial Attacks

Expressive human pose and shape estimation (EHPS) is crucial for digital human generation, especially in applications like live streaming. While existing research primarily focuses on reducing estimation errors, it largely neglects robustness and security aspects, leaving these systems vulnerable to adversarial attacks. To address this significant challenge, we propose the $\mathbf{\text{Tangible}}\mathbf{\text{Attack}}\mathbf{\text{(TBA)}}$, a novel framework designed to generate adversarial examples capable of effectively compromising any digital human generation model. Our approach introduces a $\mathbf{\text{Dual}}\mathbf{\text{Heterogeneous}}\mathbf{\text{Noise}}\mathbf{\text{Generator}}\mathbf{\text{(DHNG)}}$, which leverages Variational Autoencoders (VAE) and ControlNet to produce diverse, targeted noise tailored to the original image features. Additionally, we design a custom $\mathbf{\text{adversarial}}\mathbf{\text{loss}}\mathbf{\text{function}}$ to optimize the noise, ensuring both high controllability and potent disruption. By iteratively refining the adversarial sample through multi-gradient signals from both the noise and the state-of-the-art EHPS model, TBA substantially improves the effectiveness of adversarial attacks. Extensive experiments demonstrate TBA’s superiority, achieving a remarkable 41.0% increase in estimation error, with an average improvement of approximately 17.0%. These findings expose significant security vulnerabilities in current EHPS models and highlight the need for stronger defenses in digital human generation systems.