In this master thesis, we propose a novel framework dedicated to virtual design based on natural gestures of the hands. User hands represent a natural way for interacting with objects in our everyday life. However, they have been rarely used directly in Virtual Reality (VR) setups. In this thesis, we present a framework composed of three main parts: hand tracking, object material modeling and 3D interaction techniques. Our contributions are first the real-time simulation of deformable material based on hand movements, and second the introduction of innovative 3D interaction techniques dedicated to the virtual design of deformable objects using the user hands. Our complete framework paves the way for future VR systems where natural gestures of the hand could be directly taken into account in the interaction process.