EXO-200 is a neutrino physics experiment that I worked on from 2012-2014. This project is something I worked on in the Fall of 2012 with a friend as our final project for Andrew Ng's Machine Learning class at Stanford (CS 229). I'll first give a brief explanation of the physical problem, and then I'll discuss how the machine learning problem grew out of it and how it was approached.
EXO-200 searches for double beta decay events that occur inside a cylindrical detector filled with liquid Xenon-136 (which undergoes this double beta decay process). When a decay event occurs, electrons are ionized off of the surrounding Xenon atoms, and the cloud of electrons is pushed to the end of the cylinder by an electric field. At the end of the cylinder, there are two sets of parallel wires, with the two sets being rotated about 45 degrees with respect to each other. The electrons drift past the first set of wires and are collected on the second set of wires. Double beta decay events are "single-site", meaning that only a single cluster of electrons is released in the liquid Xenon and that single cluster is often collected on a single wire in the second set of wires. Other types of events occur (background), which are often gamma decay events, which is "multi-site", i.e. many clusters of electrons are produced and are then collected on different wires. This "single-site" vs "multi-site" distinction is the first step in trying to separate signal from background in the experiment. An issue arises, though, when a very large single-site charge cluster induces a current in wires near the one it is collected on. These induction signals can be mistaken for actual charge collection, resulting in a true single-site event being mislabeled as a multi-site event. Solving this misclassification problem leads to a significant improvement in the experiment's sensitivity to the double beta decay signal.
Now for the machine learning classification. The goal was to produce a classifier that could identify when a waveform on the charge collection wires was a true charge collection, or one of these troublesome induction signals.
A set of 14 real-valued features were chosen to be calculated for each wire waveform. A data set of 500,000 real detector events (a mix of single-site and multi-site) was collected with the features calculated, but since real detector events obviously don't have true labels as to whether or not they contain the troublesome induction signals, a data set of Monte Carlo generated events was used for training. The Monte Carlo data set contained 120,000 simulated events, half with induction signals and half without. The real event data set was used as a validation data set, by hand-classifying wire waveforms from the data set as either true charge collection or induction. The usefulness of the hand-labeled real event data set was due to the fact that although the Monte Carlo data set is larger, the simulation is not a perfect representation of what actually happens in the detector, so it was important to validate our classifier with real detector data.
The training/testing data sets prepared, we simply went on to experiment with a bunch of different classifiers and see how they performed in terms of accuracy when applied to the Monte Carlo data set and the hand-labeled data set. We tested logistic regression classifiers with different regularization parameters, SVM classifiers with different kernels, single Decision Tree classifiers and Random Forest ensemble classifiers, as well as an unsupervised K-Means clustering. Testing each of these different classifiers involved either a manual or grid-search-based manipulating of classifier parameters. For each one of these configurations, a 5-fold cross-validation was performed on the Monte Carlo data set, and then the classifier was trained on the whole Monte Carlo data set before being tested on the hand-labelled data set. For each test, a confusion matrix was calculated to evaluate performance. Ultimately, our Random Forest classifier performed the best, perfectly classifying all Monte Carlo waveforms, while correctly classifying 99.4% of hand-labelled charge collection waveforms and 80.7% of hand-labelled induction waveforms.
This project was great because it allowed me to combine my physics research with my machine learning coursework. I really enjoy computational problems that come out of physical projects, such as a neutrino physics detector in this case. I got to gain experience with Python's scikit-learn package, experiment with a good number of different classifiers, and ultimately succeed in solving a real-world machine learning/classification problem.
A link to our paper for this project can be found here.
No comments:
Post a Comment