Critics of the latest report of the WSPRnet tracking technology, which has given a new location for MH370, have been answered by its lead author Richard Godfrey.
Subscribe to the Airlineratings.com newsletter to get the relevant news first
Video of stunning supersonic flyovers
MH370: New research paper confirms revolutionary location technology
DJ Khaled Goes Shopping For Boeing VIP Jet
Mr Godfrey on this website has published a rebuttal which we re-print for the record.
“Victor Iannello, Mick Gilbert and Steve Kent all claim to have read our latest case study but somehow miss one of the most important points.
“The plot of the True Positive Rate against the False Positive Rate for the Receiver Operating Characteristic (ROC) curve during the cruise phase of the flight shows an improved level of aircraft detection. The ROC curve is based on the 70 observations during the cruise and the area under the curve increases from 58.66% to 60.43%.” – Discussion of the Results – Section 7.2 – (please see Figure 52).
Victor Iannello concludes:
“So by my count, with a threshold of 0.75, after including the discarded data, the True Positive rate remains at 52% and the False Positive rate increases from 37% to 44%, versus a rate of 45% of detections due to chance (assuming that SNRs are normally distributed).”
“The utility of the GDTAAA plane detector just went from weak to almost imperceptible, just as physics would predict. The prospect of using this detector to reconstruct the path of MH370 is once again proven to be junk science.”
Mick Gilbert concludes:
“Victor, doubtless Professor Maskell is familiar with the traditional academic point system for classifying the utility of a methodology based on its area under the ROC curve results:
.90-1.0 = excellent (A)
.80-.90 = good (B)
.70-.80 = fair (C)
.60-.70 = poor (D)
.50-.60 = fail (F)
With an overall AUROC of 58.66%, the authors get an F.”
“We might well ask whether anyone in any position of authority on either side of the negotiations to fund a renewed search would treat a methodology with essentially no predictive capability seriously. Who could possibly say that a methodology with such manifestly poor results clears the “credible” part of the 2017 ‘credible new information’ test?”
Steve Kent concludes:
“Godfrey takes the unjustified step of discarding duplicates, retaining only the links that cross the true aircraft position. Since the majority of the discarded links are positive detections (21 out of 28), this means that the retained links are mainly true positives while the discarded links would have been false positives had they been kept. This process introduces a bias such that the WSPR hypothesis will always be favoured, even if the null hypothesis were true. A better-designed test would have avoided this issue altogether”.
“This analysis ignores the fact that rays do not follow precise great circles (as assumed by Godfrey) due to the spheroidal shape of the earth, introducing cross-track offsets of order 10-15 nm and thus making the entire exercise total nonsense.”
In response we would like to point out the following:
1. Independent Review
Prof. Simon Maskell is on the record as stating: “the report is logical and statistically correct.”
2. Discarding Duplicates
The results in the cruise even with the duplicates is the same at 58%.
The duplicates arise mostly from the spiral dive, where the aircraft in the last 4 minutes of flight is all within a circle with a radius of 2 nm, and also in part from the alignment of the aircraft track with the WSPRnet link azimuth.
3. Methodology Classification
Gilbert invents a new 5-point Likert rating scale. He probably meant “Excellent, Very Good, Good, Fair, Poor”. There is no “F” for “Fail” in a Likert scale. We also note the next letter in the alphabet after “D” is in fact “E” and not “F”. Neither Prof. Maskell nor we, have heard of this Gilbert rating scale.
In fact, a result of 60.43% is “Fair” and not “Fail”.
4. Poorly Designed Test
The tragic flight of a Cessna 551 Citation II/SP registration OE-FGR, was not a poorly designed test.
In fact, the aircraft that crashed after fuel exhaustion into the Baltic Sea off the coast of Latvia on Sunday 4th September 2022 was not a test at all.
Like MH370 it was an incident that should not have happened and any investigation into the incident has the prime goal to prevent such an incident from ever happening again.
5. Cross-Track Errors
Steve Kent states: “I computed the rms cross-track error for each of the 183 links in the data table. The median rms is about 2.1 nm, which is the maximum cross-track error allowed by Godfrey.”
Steve Kent omits to state that the average number of SNR anomalous WSPRnet links at each point in the cruise of the aircraft is 4.3 and not a single link as he implies. When three or more anomalous links intersect at the aircraft position, as is the case at each point in the timeframe analysed, then the cross-track error is minimal.”
