A research team at the Radboud University in the Netherlands are using deep neural networks to convert hand-drawn face sketches into photorealistic images. The success and accuracy of this process could lead it to have applications in forensics, converting sketches of potential suspects into realistic photographs.
By using a set of 200,000 facial images, the researchers produced a set of training data for the 11-layer network by converting the images into grayscale sketches and line drawings through standard image processing algorithms. The network was then set to the task of reversing the process to turn a generated sketch back into the photograph of a face. After successfully working with the sample data, the researchers used the now-trained network to perform the same process with a new set of images. The results were promising. ‘Synthesised face images matched the ground truth photographs closely, and persons in the images were easily recognisable in most cases,’ say the team in the research paper. They found that the best results were produced when the network was working with a line sketch (produced by an algorithm) as the starting image. ‘We found that the line model performed impressively in terms of matching the hair and skin colour of the individuals even when the line sketches did not contain any colour information’ the team commented.
The next task faced by the trained network was synthesising photorealistic images from a hand drawn sketches, rather than those generated by an algorithm. The technique proved once again a success. ‘We found that the line sketch inversion model can solve this inductive transfer task almost as good as it can solve the task that it was trained on. Once again, the model synthesised photorealistic face images’. With the results, it was easy for the team to recognise the advantages of this model over those that came previously. ‘Hair texture and style, which posed a problem in most previous studies, was very well handled by our CSI model.’
The Radboud University research team believe the process has uses in both forensics and fine arts. They see the network being used to compare sketches of potential suspects in crime investigations to a database of photographs in order to identify them, and using it to turn historic portraits of artists into photographs, producing what would be the first photographic records from that time.
Further Information:
