Deep neural networks with applications from computer vision to medical diagnosis^1-5 are commonly implemented using clock-based processors^6-14, where computation speed is mainly limited by the clock frequency and the memory access time. In the optical domain, despite advances in photonic computation^15-17, the lack of scalable on-chip optical nonlinearity and the loss of photonic devices limit the scalability of optical deep networks. Here we report the first integrated end-to-end photonic deep neural network (PDNN) that performs sub-nanosecond image classification through direct processing of the optical waves impinging on the on-chip pixel array as they propagate through layers of neurons. Within each neuron, linear computation is performed optically and the nonlinear activation function is realised opto-electronically, enabling a classification time of under 570 ps, which is comparable with a single clock-cycle of state-of-the-art digital platforms. A uniformly distributed supply light provides the same per-neuron optical output range enabling scalability to large-scale PDNNs. Two- and four-class classification of handwritten letters with accuracies of higher than 93.8% and 89.8% are demonstrated, respectively. Direct clock-less processing of optical data eliminates analogue-to-digital conversion and the requirement for a large memory module, enabling faster and more energy-efficient neural networks for the next generations of deep learning systems.