When you think of a movie from the early 1900s you imagine it in black-and-white. This happens because in most cases at the time the productions could still afford only that. However, there are exceptions: some 19th and 20th-century directors have decided to explore the presence of color in their works. In the case of the director Georges Méliès, we are talking about a colorization done manually frame by frame, thus introducing a great arbitrariness in the choice of the colors used, always with excessively lively tones. On the other hand, thirty years later, we find a Hollywood movie with sepia and colored tones. In the first half of the 20th century, there was no large equipment available to plausibly color a film. And even Disney cartoons of that time could never have rendered so many chromatic shades like those of our days. And this is where our analysis comes into play: using specific deep learning algorithms, based on Generative Adversarial Networks (GANs) or Convolutional Neural Networks (CNNs), we have experienced how the same period films would have been colored automatically by a computer. Numerous experiments have been carried out on vintage movies and cartoons, in particular using SIGGRAPH 2016, ECCV 2016, and Pix2Pix models. The obtained results have shown a good quality colorization and flexibility in the case of Pix2Pix algorithm, especially once the model has been adapted to the presence of an additional element to be provided as input to the architecture: a Reference Image. It has been extracted from the images used for the training phase, and, from this frame, the model could take useful information about lightness and tones to reconstruct the target image. Obviously, it is a methodology that can still be improved in the future to make it usable for any type of film, trying to color each frame more precisely, with references that are increasingly careful to be derived from the scene of interest in the movie or cartoon.
Analisi e sviluppo di algoritmi di deep learning generativi per la colorization di filmati d'epoca. Quando si pensa a un film dei primi del '900 lo si immagina in bianco e nero. Questo accade perché nella maggior parte dei casi all'epoca le produzioni ancora potevano permettersi solo quello. Tuttavia, ci sono delle eccezioni: alcuni registi di diciannovesimo e ventesimo secolo hanno deciso di sperimentare la presenza del colore nelle loro opere. Nel caso del regista Georges Méliès parliamo di una colorazione fatta manualmente fotogramma per fotogramma, introducendo dunque una grande arbitrarietà nella scelta delle tinte usate, sempre dai toni eccessivamente vivaci. Dall'altra, invece, troviamo, trentanni più tardi, una pellicola hollywoodiana dai torni seppia e colorati. Nella prima metà del '900, non si avevano grandi strumentazioni a disposizione per colorare in modo plausibile un film. E anche i cartoni animati Disney di quell'epoca non avrebbero mai potuto rendere così tante sfumature cromatiche come quelli dei nostri giorni. Ed è qui che entra in gioco la nostra analisi: facendo uso di specifici algoritmi di deep learning, basati sulle Generative Adversarial Networks (GANs) o sulle Convolutional Neural Networks (CNNs), abbiamo sperimentato come i medesimi filmati d'epoca sarebbero stati colorati in automatico da un computer. Sono stati numerosi gli esperimenti eseguiti sulle pellicole vintage, film o cartoni animati che fossero, facendo in particolare uso dei modelli SIGGRAPH 2016, ECCV 2016, e Pix2Pix. I risultati ottenuti hanno mostrato una buona qualità di colorizzazione e flessibilità nel caso dell'algoritmo Pix2Pix, specialmente una volta adattato l'algoritmo alla presenza di un ulteriore elemento da fornire in input all'architettura: un'immagine di riferimento. Essa è stata estratta dalle immagini utilizzate per la fase di training, da cui il modello potesse prelevare informazioni utili circa luminosità e tonalità per ricostruire l'immagine d'interesse. Ovviamente, si tratta di una metodologia che ancora potrà essere migliorata in futuro, in modo da renderla fruibile a ogni tipo di filmato, cercando di eseguire la colorazione di ogni fotogramma in modo più preciso, con riferimenti sempre più attenti a fare parte della scena d'interesse del film.
Analysis and development of generative deep learning algorithms for vintage movie colorization
MODICA, CAMILLA
2020/2021
Abstract
When you think of a movie from the early 1900s you imagine it in black-and-white. This happens because in most cases at the time the productions could still afford only that. However, there are exceptions: some 19th and 20th-century directors have decided to explore the presence of color in their works. In the case of the director Georges Méliès, we are talking about a colorization done manually frame by frame, thus introducing a great arbitrariness in the choice of the colors used, always with excessively lively tones. On the other hand, thirty years later, we find a Hollywood movie with sepia and colored tones. In the first half of the 20th century, there was no large equipment available to plausibly color a film. And even Disney cartoons of that time could never have rendered so many chromatic shades like those of our days. And this is where our analysis comes into play: using specific deep learning algorithms, based on Generative Adversarial Networks (GANs) or Convolutional Neural Networks (CNNs), we have experienced how the same period films would have been colored automatically by a computer. Numerous experiments have been carried out on vintage movies and cartoons, in particular using SIGGRAPH 2016, ECCV 2016, and Pix2Pix models. The obtained results have shown a good quality colorization and flexibility in the case of Pix2Pix algorithm, especially once the model has been adapted to the presence of an additional element to be provided as input to the architecture: a Reference Image. It has been extracted from the images used for the training phase, and, from this frame, the model could take useful information about lightness and tones to reconstruct the target image. Obviously, it is a methodology that can still be improved in the future to make it usable for any type of film, trying to color each frame more precisely, with references that are increasingly careful to be derived from the scene of interest in the movie or cartoon.È consentito all'utente scaricare e condividere i documenti disponibili a testo pieno in UNITESI UNIPV nel rispetto della licenza Creative Commons del tipo CC BY NC ND.
Per maggiori informazioni e per verifiche sull'eventuale disponibilità del file scrivere a: unitesi@unipv.it.
https://hdl.handle.net/20.500.14239/13515