We propose to adapt a virtual agent called ‘Zara the Supergirl’ to user personality. User personality is deducted through two models, one based on raw audio and the other based on speech transcription text. Both models show good performance, with an average F-score of 69.6 for personality perception from audio, and an average F-score of 71.0 for recognition from text. Both models deploy a Convolutional Neural Network. Through a Human-Agent Interaction study we find correlations between user personality and preferred agent personality. The study suggests that especially the Openness user personality trait correlates with a stronger preference for agents with more gentle personality. People also sense more empathy and enjoy better conversations when agents adapt their personalities.