Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This work probes the robustness of a CNN-based radio technosignature detector by injecting signals with unknown morphologies and realistic terrestrial interference into real astronomical backgrounds. Despite training on a standard narrowband signal, the detector exhibited significant recall degradation for brighter signals and missed detections across various morphologies, with one interference type achieving a false positive rate comparable to the detector's baseline recall. Grad-CAM analysis revealed attention patterns that correlate with missed detections, and a simple ON-OFF filtering strategy proved ineffective at distinguishing true signals from interference, highlighting a fundamental limitation in the detector's reliance on ON-panel data.
A standard CNN technosignature detector misses 91% of genuine signals when they deviate slightly from the training morphology, and is fooled by terrestrial interference as often as it finds real signals.
Radio technosignature searches rely on machine learning detectors to shrink the candidate pool before a human looks at it. They are validated on held-out data from the training distribution, which excludes the cases where they are most likely to fail: a technosignature has unknown morphology, and terrestrial interference is unbounded in variety. I trained a standard competition detector once, froze it, and probed it with 15,000 injections across ten morphologies and five amplitudes, placed into real Green Bank backgrounds. Two directions were tested: genuine signals with morphologies other than a straight narrowband line, and interference resembling the training data. Recall did not follow the predicted ordering, and it fell as signals brightened, even for a needle of the kind the detector was trained on (from 0.590 to 0.090). One interference morphology reached a false positive rate of 0.627, comparable to the detector's own baseline recall of 0.5525 on genuine signals. Gradient-weighted Class Activation Mapping (Grad-CAM) showed attention along the injected path in one missed family by a pre-specified test, and visually in a second. A pre-specified, time-averaged ON-OFF filter removed false alarms and genuine detections at the same rate. The detector reads only the ON panels. Since the only evidence distinguishing a genuine candidate from terrestrial interference lies in the discarded OFF panels, it cannot tell if a signal originates from the target star; it can only tell if it resembles the training data. This explains the false alarm directly; the missed signals reflect what the detector learned to recognise, which reading the OFF panels would not by itself change.