During recent years deep learning techniques haven proven extremely powerful in many different applications and have successfully been applied to high energy physics data. The aim of this project is to make these techniques available to the HEP community by integrating them into TMVA/Root. Since the training of deep neural networks is computationally highly demanding, the main focus of this project will be to enable GPU-accelerated training of deep neural networks with TMVA. This will be achieved by integrating readily available GPU deep learning frameworks as back-ends into the current implementation. The targeted hardware are OpenCL capable devices as well as CUDA capable devices. The resulting implementation will allow for platform- and vendor-independent accelerated training of deep neural networks. In addition to that, the currently available network features will be extended with convolutional and pooling layers.