- Pretrained nets
- User supplied training set (one could call the simulation tools discussed below to generate simulated data from observation parameters)
A deep learning generative model has been used by the CADC team to remove moving object trails; it uses the open source tensorflow library. However, designing the model is not easy. A deep learning model can be trained, with relevant data sets, to detect and model various objects in images, such as satellite traces in astronomical images. The models can then be used to remove the trails from the images using deep learning techniques. Tensorflow, an open-source machine learning platform, can provide a foundation for training deep learning models. Keras, a deep learning API (applications programming interface) written in Python, runs on top of TensorFlow's machine learning platform, focusing on enabling fast experimentation and easy implementation. With Keras, a trained model can be used easily as Python code, standalone, or a part of a pipeline.
The deep models that are trained on a single instrument are likely not generically applicable but they can be used as pre-trained models for trail detection on new instruments. Training for the new instrument can then be achieved using a smaller training data set. Two items needed to enable this transfer in learning are a database with some uniformity in accessibility and metadata associated with the training data.
Deep learning methods are also highly effective at learning in lower-dimensional representations, known as latent space representation. Images with similar characteristics lie near each other in latent space. The vector length is considerably smaller than the input image size, providing a compressed representation of the original image by removing complex dimensionality associated with astrophysically uninformative parts of the image space. The latent space vector can also be mapped back into the original space, restoring the original image (i.e., preserving fluxes, with possible random losses). The deep learning model that creates the latent representation is trained such that the compressed latent representation contains astrophysically meaningful information. Simultaneously, the latent vector outputs of deep models provide a homogenous input for deep downstream models trained to solve astrophysical problems. A set of deep probabilistic deep models can learn "what is in the images".
Beyond use as homogenous inputs for deep learning, the latent space representations of the images themselves are also highly useful. A remarkable application of using latent space in deep models is to perform arithmetic methods on the latent space data. The algebraic manipulation has a visible manifestation when latent data are decoded back into the original image domain. For example, suppose we have a set of latent space vectors of particular sky images containing contamination from satellite tracks and another set of images for the same sky but without these tracks. The latent space vectors can be subtracted from each other. Once subtracted, the leftover latent data will only represent the satellite tracks. These new latent satellite vectors can act as versatile models; for example, they can be subtracted from other contaminated images to remove undesirable tracks.
As an example of deep learning methods (that can also be generalized to the satellite problems), the top panel of Figure 4 shows two different images with different pixel problems. The blue and red marks show two moving objects. The bottom panel shows the same images where the images have been denoised, and the problems have also been removed without damaging the sources in the images.
