Shopt

Table of Contents

• About
  • Who Should Use ShOpt
  • Analytic Profile Fits
  • Pixel Grid Fits
  • Interpolation Across the Field of View
• Inputs and Outputs
  • Inputs
  • Outputs
• Running
  • Command
  • Dependencies
  • Set Up
  • Multi-Threading
  • Testing
• Program Architecture
• How Can I Contribute?
• Config / YAML Information
• Known Issues
• Contributors
• Further Acknowledgements

About

Shear Optimization with ShOpt.jl, a julia library for empirical point spread function characterizations. We aim to improve upon the current state of Point Spread Function Modeling by using Julia to leverage performance gains, use a different mathematical formulation than the literature to provide more robust analytic and pixel grid fits, improve the diagnostic plots, and add features such as wavelets and shapelets. At this projects conclusion we will compare to existing software such as PIFF and PSFex. Work done under McCleary's Group.

We release the related benchmarking code at https://github.com/mcclearyj/cweb_psf

Start by Cloning This Repository. Then see TutorialNotebook.ipynb or follow along the rest of this README.md to get started! Note that the commands in the tutorial notebook are meant to give a sense of procedure and can be executed with the Julia REPL itself.

Who Should Use

Users looking for empirical point spread function characterization software tailored for the data coming from the James Webb Space Telescope, or on a dataset with the similar characteristics. For example, the point spread function spans 100s of pixels because of the pixel scale of your camera, the point spread function is not well approximated by an analytic profile, or the point spread function varies alot across the field of view. For any of these reasons, you should consider using ShOpt.jl. ShOpt.jl is not a single function package, and we would encourage the user to explore the full functionality of ShOpt.jl in the sample config to tailor the software to their needs.

JWST Data is now publicly available at: https://cosmos.astro.caltech.edu/page/cosmosweb-dr. ShOpt was evaluated across all of the wavelengths in the 30mas pixel scale at this link.

Analytic Profile Fits

ShOpt.jl's analytic profile fitting takes inspiration from a number of algorithms outside of astronomy, notably SE-Sync, an algorithm that solves the robotic mapping problem by considering the manifold properties of the data. With sufficiently clean data, the SE-Sync algorithm will descend to a global minimum constrained to the manifold $SE(d)^n / SE(d)$. Following suit, we are able to put a constraint on the solutions we obtain to $[s, g_1, g_2]$ to a manifold. The solution space to $[s, g_1, g_2]$ is constrained to the manifold $$B_2(r) \times \mathbb{R}_{+}$$. The existence of the constraint on shear is well known; nevertheless, the parameter estimation task is usually framed as an unconstrained problem.

Path to $[s, g_1, g_2]$ in $B_2(r) \times \mathbb{R}_+$

Pixel Grid Fits

PCA Mode

We used the first n weights of a Principal Component Analysis and use that to construct our PSF in addition to a smoothing kernel to account for aliasing

Autoencoder Mode

For doing Pixel Grid Fits we use an autoencoder model to reconstruct the Star
PCA mode

function pca_image(image, ncomponents)    
  #Load img Matrix
  img_matrix = image

  # Perform PCA    
  M = fit(PCA, img_matrix; maxoutdim=ncomponents)    

  # Transform the image into the PCA space    
  transformed = MultivariateStats.transform(M, img_matrix)    

  # Reconstruct the image    
  reconstructed = reconstruct(M, transformed)    

  # Reshape the image back to its original shape    
  reconstructed_image = reshape(reconstructed, size(img_matrix)...)    
end    

Autoencoder mode

# Encoder    
encoder = Chain(    
                Dense(r*c, 128, leakyrelu),    
                Dense(128, 64, leakyrelu),    
                Dense(64, 32, leakyrelu),    
               )    
#Decoder
decoder = Chain(    
                Dense(32, 64, leakyrelu),    
                Dense(64, 128, leakyrelu),    
                Dense(128, r*c, tanh),    
               )    
#Full autoencoder
autoencoder = Chain(encoder, decoder)    

#x_hat = autoencoder(x)    
loss(x) = mse(autoencoder(x), x)    

# Define the optimizer    
optimizer = ADAM()    

Interpolation Across the Field of View

[s, g1, g2] are all interpolated across the field of view. Each Pixel is also given an interpolation across the field of view for an nth degree polynomial in (u,v), where n is supplied by the user

The plot on the left shows the average cutout of all stars in a supplied catalog. The plot in the middle shows the average point spread function model for each star. The plot on the right shows the average normalized error between the observed star cutouts and the point spread function model.

Inputs and Outputs

Currently, the inputs are JWST Point Spread Functions source catalogs. The current outputs are images of these Point Spread Functions, Learned Analytic Fits, Learned Pixel Grid Fits, Residual Maps, Loss versus iteration charts, and p-value statisitcs. Not all functionality is working in its current state. Planned functionality for more Shear checkplots.

Inputs

| Image | Description | |-----------------------------------|------------------------------------| | | Star Taken From Input Catalog | | shopt.yml | Config File for Tunable Parameters | | * starcat.fits | Star Catalog to take vignets from |

Outputs

| Image | Description | |----------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------| | summary.shopt | Fits File containing summary statistics and information to reconstruct the PSF | | | Pixel Grid Fit for the Star Above | | | Residual Map for Above Model and Fit | | | s varying across the field of view | | | g1 varying across the field of view | | | g2 varying across the field of view | | | Histogram for learned profiles for each star in an analytic fit with their residuals | | | Same data recorded as a scatterplot with and without outliers removed and with error bars |

NB: This is not a comprehensive list, only a few cechkplots are presented. See the shopt.yml to configure which plots you want to see and save!

Running

Command

To run shopt.jl

First use Source Extractor to create a catalog for ShOpt to accept and save this catalog in the appropriate directory

Run julia shopt.jl [configdir] [outdir] [catalog]

There is also a shell script that runs this command so that the user may call shopt from a larger program they are running

Dependencies

Not all of these will be strictly necessary depending on the checkplots you produce, but for full functionality of ShOpt the following are necessary. Source Extractor (or Source Extractor ++) is also not a strict dependency, but in practice one will inevitably install to generate a catalog.

| Julia | Python | Binaries | Julia | Julia | |------------------|------------|----------|----------------|----------------| | Plots | matplotlib | SEx | ProgressBars | BenchmarkTools | | ForwardDiff | astropy | | UnicodePlots | Measures | | LinearAlgebra | numpy | | CSV | Dates | | Random