HOW
SCIENCE “WORKS”
http://backreaction.blogspot.com/2018/11/story-about-ligo-noise-resurfaces-in.html?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+blogspot%2Fermku+%28Backreaction%29
Thursday,
November 01, 2018
|
Cover of New Scientist
Nov 3rd 2018. |
The current New Scientist issue
has an “exclusive feature” under the headline: “DID WE REALLY
FIND GRAVITATONAL WAVES? Breakthrough physics result questioned.”
The article is by Michael Brooks and it’s a summary of a claim I wrote about last year, that the original 2015 gravitational wave detection by the LIGO collaboration was not a real signal.
This claim was made by a Danish group around the physicist Andrew Jackson. This group tried to reproduce the data analysis of the LIGO collaboration with the publicly available data and could not.
The New Scientist article quotes Duncan Brown at Syracuse, who until recently was a member of LIGO, with reassuring the reader that the Danes are “credible scientists,” and Slava Mukhanov who likewise emphasizes that the Danes are people “with a high reputation.” Slava is also on record stating that “There is no mistake” in the analysis of the Danish group. Peter Coles chimes in to say that “I think their paper is a good one and it’s a shame that some of the LIGO team have been so churlish in response.”
The New Scientist article then draws a comparison between the LIGO case and the BICEP case. BICEP looked for the so-called primordial gravitational waves, which are in a different wavelength regime than LIGOs. Their supposed signal turned out to be merely noise.
The two measurements, however, work entirely differently because BICEP did not (attempt to) directly measure gravitational waves. Instead, it looked for a secondary signal that is the imprint of the primordial gravitational waves in the cosmic microwave background. The BICEP signal was contaminated by foreground from the Milky Way. The same problem does not exist for LIGO.
Michael Brooks in the New Scientist article then points out that this is the first time we are analyzing gravitational wave signals and it’s still early days, so if an independent analysis cannot reproduce the result that’s a problem.
Interestingly, Brooks seems to have found out that the key figure in the LIGO paper about the first discovery does not actually show the quantity that they used in the data analysis. I had been told about this previously, though I cannot now recall the details. (I believe it was something about the plotted quantity not actually showing the relevant significance. Anyone knows better, pls leave a comment.)
The way that I heard about it was that some members of the collaboration wanted a pretty plot that “could be printed on a T-shirt,” ie they opted for beauty over scientific relevance. I don’t know if that’s what really happened, but it sounds plausible enough. I recall thinking at the time that if that’s true it was a dumb decision; clearly this move pissed off some people in the collaboration and those had no reason to keep their mouth shut forever.
For me the issue with the Danish group’s criticism was not whether the signal is real. LIGO people pointed out problems with the Danes’ analysis to me that even I could understand. No, the issue for me was that the collaboration didn’t make an effort helping others to reproduce their analysis. They also did not put out an official response, indeed have not done so until today. I thought then – and still think – this is entirely inappropriate of a scientific collaboration. It has not improved my opinion that whenever I raised the issue LIGO folks would tell me they have better things to do.
We have here a group of researchers not associated with the collaboration which tries to follow the analysis methods that the collaboration reported and they cannot confirm the collaboration’s results. This should not happen. If the collaboration is not able to explain their procedures so that other scientists can find out what they’re doing, that is a problem that must be fixed. This is the first time anyone analyses data for gravitational wave signals and the methodology needs to be clearly documented. Evidently, this is not presently the case.
The Danes btw haven’t been the only ones who tried to redo the LIGO analysis and didn’t manage to. I know this not because I’m obsessed with LIGO, but because people send me references about this. I also get plenty of emails and comments from cranks who think that LIGO is a fraud and just wasting tax-money and so on. All this is reason why I think the LIGO collaboration is doing a disservice to science by ignoring the matter.
I was thus happy to read in the New Scientist article that some people from the LIGO collaboration are at least working on a response. But, well, it’s certainly taking some time.
What happened after the Danish group made their claim in June last year is that the VIRGO collaboration joined LIGO’s search for gravitational waves. So now the analysis draws on data from three detection sites. They have since seen a gravitational wave event with an optical counterpart recorded in several telescopes. Brooks reports that the Danish groups still doubts the detection because this event, which happened in August 2017, was originally labelled a “glitch”. The story about the glitches is indeed peculiar. The glitches are occasional false alarms in the detectors. They tend to not have the frequency spectrum of the real events however. So it seems to me like a stretch that the Danes are holding on to their claim, and I am not sure why New Scientist dug this up now.
If you cannot (or do not want to) access the New Scientist piece, Jennifer Ouellette has an excellent summary on Ars Technica.
The article is by Michael Brooks and it’s a summary of a claim I wrote about last year, that the original 2015 gravitational wave detection by the LIGO collaboration was not a real signal.
This claim was made by a Danish group around the physicist Andrew Jackson. This group tried to reproduce the data analysis of the LIGO collaboration with the publicly available data and could not.
The New Scientist article quotes Duncan Brown at Syracuse, who until recently was a member of LIGO, with reassuring the reader that the Danes are “credible scientists,” and Slava Mukhanov who likewise emphasizes that the Danes are people “with a high reputation.” Slava is also on record stating that “There is no mistake” in the analysis of the Danish group. Peter Coles chimes in to say that “I think their paper is a good one and it’s a shame that some of the LIGO team have been so churlish in response.”
The New Scientist article then draws a comparison between the LIGO case and the BICEP case. BICEP looked for the so-called primordial gravitational waves, which are in a different wavelength regime than LIGOs. Their supposed signal turned out to be merely noise.
The two measurements, however, work entirely differently because BICEP did not (attempt to) directly measure gravitational waves. Instead, it looked for a secondary signal that is the imprint of the primordial gravitational waves in the cosmic microwave background. The BICEP signal was contaminated by foreground from the Milky Way. The same problem does not exist for LIGO.
Michael Brooks in the New Scientist article then points out that this is the first time we are analyzing gravitational wave signals and it’s still early days, so if an independent analysis cannot reproduce the result that’s a problem.
Interestingly, Brooks seems to have found out that the key figure in the LIGO paper about the first discovery does not actually show the quantity that they used in the data analysis. I had been told about this previously, though I cannot now recall the details. (I believe it was something about the plotted quantity not actually showing the relevant significance. Anyone knows better, pls leave a comment.)
The way that I heard about it was that some members of the collaboration wanted a pretty plot that “could be printed on a T-shirt,” ie they opted for beauty over scientific relevance. I don’t know if that’s what really happened, but it sounds plausible enough. I recall thinking at the time that if that’s true it was a dumb decision; clearly this move pissed off some people in the collaboration and those had no reason to keep their mouth shut forever.
For me the issue with the Danish group’s criticism was not whether the signal is real. LIGO people pointed out problems with the Danes’ analysis to me that even I could understand. No, the issue for me was that the collaboration didn’t make an effort helping others to reproduce their analysis. They also did not put out an official response, indeed have not done so until today. I thought then – and still think – this is entirely inappropriate of a scientific collaboration. It has not improved my opinion that whenever I raised the issue LIGO folks would tell me they have better things to do.
We have here a group of researchers not associated with the collaboration which tries to follow the analysis methods that the collaboration reported and they cannot confirm the collaboration’s results. This should not happen. If the collaboration is not able to explain their procedures so that other scientists can find out what they’re doing, that is a problem that must be fixed. This is the first time anyone analyses data for gravitational wave signals and the methodology needs to be clearly documented. Evidently, this is not presently the case.
The Danes btw haven’t been the only ones who tried to redo the LIGO analysis and didn’t manage to. I know this not because I’m obsessed with LIGO, but because people send me references about this. I also get plenty of emails and comments from cranks who think that LIGO is a fraud and just wasting tax-money and so on. All this is reason why I think the LIGO collaboration is doing a disservice to science by ignoring the matter.
I was thus happy to read in the New Scientist article that some people from the LIGO collaboration are at least working on a response. But, well, it’s certainly taking some time.
What happened after the Danish group made their claim in June last year is that the VIRGO collaboration joined LIGO’s search for gravitational waves. So now the analysis draws on data from three detection sites. They have since seen a gravitational wave event with an optical counterpart recorded in several telescopes. Brooks reports that the Danish groups still doubts the detection because this event, which happened in August 2017, was originally labelled a “glitch”. The story about the glitches is indeed peculiar. The glitches are occasional false alarms in the detectors. They tend to not have the frequency spectrum of the real events however. So it seems to me like a stretch that the Danes are holding on to their claim, and I am not sure why New Scientist dug this up now.
If you cannot (or do not want to) access the New Scientist piece, Jennifer Ouellette has an excellent summary on Ars Technica.
Update: The LIGO collab has published a brief response to the New Scientist piece on their website:
“1
Nov 2018 -- Claims in a paper by Creswell et al. of puzzling correlations in
LIGO data have broadened interest in understanding the publicly available LIGO
data around the times of the detected gravitational-wave events. The features
presented in Creswell et al. arose from misunderstandings of public data
products and the ways that the LIGO data need to be treated. The LIGO
Scientific Collaboration and Virgo Collaboration (LVC) have full confidence in
our published results. We are preparing a paper that will provide more details
about LIGO detector noise properties and the data analysis techniques used by
the LVC to detect gravitational-wave signals and infer their source
properties.”
Posted
by Sabine Hossenfelder at 8:46 AM
Posts
