Machine Learning and Data Analysis for Nuclear Physics, European ERASMUS+ Master of Science program

Why a course on Machine Learning for Nuclear Physics?

Probability theory and statistical methods play a central role in science. Nowadays we are surrounded by huge amounts of data. For example, there are about one trillion web pages; more than one hour of video is uploaded to YouTube every second, amounting to 10 years of content every day; the genomes of 1000s of people, each of which has a length of more than a billion base pairs, have been sequenced by various labs and so on. This deluge of data calls for automated methods of data analysis, which is exactly what machine learning provides. The purpose of thiscourse is to provide an introduction to the core concepts and tools of machine learning in a manner easily understood and intuitive to physicists and nuclear physicists in particular. We will start with some of the basic methods from supervised learning and statistical data analysis, such as various regression methods before we move into deep learning methods for both supervised and unsupervised learning, with an emphasis on the analysis of nuclear physics experiments and theoretical nuclear physics. The students will work on hands-on daily examples as well as projects than can result in final credits. Exercises and projects will be provided and the aim is to give the participants an overview on how machine learning can be used to analyze and study nuclear physics problems (experiment and theory). The major scope is to give the participants a deeper understanding on what Machine learning and Data Analysis are and how they can be used to analyze data from nuclear physics experiments and perform theoretical calculations of nuclear many-body systems.

The goals of this course on Machine Learning and Data Analysis are to give the participants a deeper understanding and critical view of several widely popular Machine Learning algorithms, covering both supervised and unsupervised learning. The learning outcomes involve an understanding of the following central methods:

• Basic concepts of machine learning and data analysis and statistical concepts like expectation values, variance, covariance, correlation functions and errors;
• Estimation of errors using cross-validation, blocking, bootstrapping and jackknife methods;
• Optimization of functions
• Linear Regression and Logistic Regression;
• Dimensionality reductions, from PCA to clustering
• Neural networks and deep learning;
• Convolutional Neural Networks
• Recurrent Neureal Networks and Autoencoders
• Decisions trees and random forests
• Support vector machines and kernel transformations

The course has two central parts

1. Statistical analysis and optimization of data
2. Machine learning

These topics will be scattered thorughout the course and may not necessarily be taught separately. Rather, we will often take an approach (during the lectures and project/exercise sessions) where say elements from statistical data analysis are mixed with specific Machine Learning algorithms.

Statistical analysis and optimization of data

The following topics will be covered

• Basic concepts, expectation values, variance, covariance, correlation functions and errors;
• Simpler models, binomial distribution, the Poisson distribution, simple and multivariate normal distributions;
• Central elements of Bayesian statistics and modeling;
• Gradient methods for data optimization,
• Estimation of errors and resampling techniques such as the cross-validation, blocking, bootstrapping and jackknife methods;
• Principal Component Analysis (PCA) and its mathematical foundation

Machine learning

The following topics will be covered:

• Linear Regression and Logistic Regression;
• Neural networks and deep learning, including convolutional and recurrent neural networks
• Decisions trees, Random Forests, Bagging and Boosting
• Support vector machines
• Unsupervised learning Dimensionality reduction, from PCA to cluster models

Hands-on demonstrations, exercises and projects aim at deepening your understanding of these topics.

Computational aspects play a central role and you are expected to work on numerical examples and projects which illustrate the theory and varous algorithms discussed during the lectures. We recommend strongly to form small project groups of 2-3 participants, if possible.

Prerequisites

Basic knowledge in programming and mathematics, with an emphasis on linear algebra. Knowledge of Python or/and C++ as programming languages is strongly recommended and experience with Jupiter notebook is recommended.

Practicalities

1. Lectures are Mondays from 815am-10am. Start September 25, 2022
2. Group meeting and work on projects and exercises to be determined
3. Grading scale: Grades are awarded on a scale from A to F, where A is the best grade and F is a fail. We are aiming at having two projects to be handed in. These will be graded and should be finalized not later than two weeks after the course is over. Both projects count 50% each of the final grade. We plan to make the grades available not later than February 1, hopefully the grades will be available before that.

Lecture material

The link https://compphysics.github.io/MLErasmus/doc/web/course.html gives you direct access to the learning material with lectures slides and jupyter notebooks. Videos of the lectures will be added.

Detailed Lecture Notes

Detailed notes at the link https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/intro.html

Teaching schedule, topics and teacher

Teachers: Morten Hjorth-Jensen (MHJ)

| | | |-|-| | Monday September 25 | - Lecture 815am-1000am: Introduction to Machine Learning and linear regression (MHJ) | | Recommended readings | Goodfellow et al chapters 2 and 3; Bishop Sections 1.1, 1.2 and 1.3 | | | Lecture material at https://compphysics.github.io/MLErasmus/doc/web/course.html session 1 | | | Video of Lecture TBA at https://youtu.be/NtHBoMoIBP8 | | | Handwritten notes at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesSep252023.pdf | | | Video on using Git and GitHub https://mediaspace.msu.edu/media/t/1_8mgx3cyf | | Monday October 2 | - Lecture 815am-10am: Linear Regression, from ordinary least squares to Lasso and Ridge (MHJ) | | Recommended readings | Goodfellow et al, Deep Learning, chapter 2 on linear algebra. Hastie et al, The elements of statistical learning, sections 3.1-3.4. Deisenroth et al, Mathematics for Machine Learning, see chapter 6, see https://mml-book.github.io/book/mml-book.pdf | | | Lecture material at https://compphysics.github.io/MLErasmus/doc/web/course.html session 2 | | | Video of Lecture at https://youtu.be/ | | | Handwritten notes at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesOct22023.pdf | | Monday October 9 | - Lecture 815am-10am: Linear Regression, Ridge and Lasso regression (MHJ) | | Recommended readings | Bishop Section 3.2 and Hastie et al chapter 3 | | | Lecture material at https://compphysics.github.io/MLErasmus/doc/web/course.html session 3 | | | Video of Lecture at https://youtu.be/SaQ1I-yyvIo | | | Handwritten notes at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesOct92023.pdf | | Monday October 16 | - Lecture 815am-10am: Ridge and Lasso regression and statistical interpretations (MHJ) | | Recommended readings | Hastie et al Chapter 3 | | | Lecture material at https://compphysics.github.io/MLErasmus/doc/web/course.html sessions 3 and 4 | | | Video of Lecture at https://youtu.be/iqRKUPJr_bY | | | Handwritten notes at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesOct162023.pdf | | Monday October 23 | - Lecture 815am-10am: Resampling Methods and Bias-Variance tradeoff (MHJ) | | Recommended readings | Hastie et al chapter 7 | | | Lecture material at https://compphysics.github.io/MLErasmus/doc/web/course.html session 4 material | | | Video of Lecture at https://youtu.be/3DyqMQaMgvQ | | | Handwritten notes at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesOct232023.pdf | | Monday October 30 | - Lecture 815am-10am: Resampling methods and Bias-variance tradeoff | | Recommended readings | Hastie et al chapter 7 | | | Lecture material at https://compphysics.github.io/MLErasmus/doc/web/course.html session 4 material | | | Video of Lecture at https://youtu.be/-6Hpyj0dwC0 | | | Handwritten notes at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesOct302023.pdf | | Monday November 6 | - Lecture 815am-10am: Logistic Regression, Optimization and Gradient Descent, material from sessions 5 and 6 | | Recommended readings | Bishop Sections 5.1-5.5 and Hastie et al chapter 11, Goodfellow et al, chapter 4 and 8 | | | Video of Lecture, Morning session 8am-10am, https://youtu.be/NwVnnQ3SHD8 | | | Handwritten notes at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesNov62023.pdf | | | Lecture material at https://compphysics.github.io/MLErasmus/doc/web/course.html sessions 4 and 5 | | Sunday and Monday November 12-13 | - Lecture 815am-10am: Optimization problems and start discussion of project 1 | | Recommended readings | Goodfellow et al, chaptes 4 and 8| | | Video of Lecture from November 12 at https://youtu.be/e4bF7ztE0ic | | | Handwritten notes November 12 at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesNov122023.pdf | | | Video of Lecture from November 13 at https://youtu.be/8angq0E9-DA | | | Handwritten notes November 13 at https://github.com/CompPhysics/MLErasmus/blob/master/doc/HandwrittenNotes/2023/NotesNov132023.pdf | | | Lecture material at https://compphysics.github.io/MLErasmus/doc/web/course.html session 5 | | Monday November 20 | - _Lecture 815am-10