The Second Apocalypse
Miscellaneous Chatter => Philosophy & Science => Topic started by: sciborg2 on September 09, 2014, 06:03:49 am
-
"Everything We Know Is Wrong" (http://www.bbc.co.uk/programmes/b04f9r4k)
...Again and again, researchers are finding the same things, whether it's with observational studies, or even the "gold standard" Randomised Controlled Studies, whether it's medicine or economics. Nobody bothers to try to replicate most studies, and when they do try, the majority of findings don't stack up. The awkward truth is that, taken as a whole, the scientific literature is full of falsehoods.
Jolyon Jenkins reports on the factors that lie behind this. How researchers who are obliged for career reasons to produce studies that have "impact"; of small teams who produce headline-grabbing studies that are too statistically underpowered to produce meaningful results; of the way that scientists are under pressure to spin their findings and pretend that things they discovered by chance are what they were looking for in the first place. It's not exactly fraud, but it's not completely honest either. And he reports on new initiatives to go through the literature systematically trying to reproduce published findings, and of the bitter and personalised battles that can occur as a result...
-
Quite the polemic paper, eh?! ;)
On the other hand i do sometimes wonder. Just look at all the contradictory results in something like "what food/beverages are healthy/not healthy". Sometimes they say "Coffee is unhealthy" sometimes they say "Coffee is healthy" etc etc . I guess in such studies it is always important to know where the money to finance the study came from, right?! ;)
On a more serious note though, this is indeed a problem...underpowered statistics, studies that have only been done once or twice (so are not proven to be reproducable etc) are not that reliable. But as research is, unfortunately, still totally dependant on money, and money is most often only given to study-areas that are useful to the big companies, i guess we will have the problem for the time being (especially in areas that are pure science with no direct application for the economy).
Although, as i hinted at, money being around is not always the problemsolver one thinks....if you take the money of entity X they kind of want to...influence...what you find. Which begs the question if we really want to have such "studies".
As the saying goes: "Only believe in the statistics you falsified yourself..." ;D
-
These biases would effect government funded research as much as private. If this paper is to be believed, neither of them have a vested interest in disproving anything, so replicating results is kind of an anemic part of the division of labor in science. In a way it can be viewed as spending more money for the privilege of knowing you wasted your money, and an irrational human being that's really loss averse is not gonna want to explore that option. Plus, spending your time attempting to reproduce the results of others' studies just isn't particularly glamorous work that most scientists aspire too.
That said this article isn't even linked in the BBC article so I can't study it in detail to find out if it's accurate or if it's just another one of the headline grabbing articles it maligns. Science reporting sucks.
-
Well i guess the onus of reproducable results would lie on the group that first attempts a study. So it's not the task of a new group to verify that. At least imo. Although if you can reproduce an earlier study and then add to it, that's surely not a bad thing.
I agree though, it's not very glamorous. But i guess if you are only in research for the glamour, than you're in the wrong place anyway ;)
-
Well i guess the onus of reproducable results would lie on the group that first attempts a study. So it's not the task of a new group to verify that. At least imo.
There is a clear conflict of interest there. I suspect if you didn't place the burden of reproducing the results on another group of people, you'd mysteriously very rarely see the results fail to be reproduced.
-
Well, if you have to publish your data, you can't cheat. So i guess one could avert that problem. But i admit, it's not without work. And yes, if you just can say "yes, we were able to reproduce our results" without giving proof, it will be easy to do so.
But in the end, if you are deep into real science, you can't hide for long that you faked your results...because you will just not get anywhere. Or the people who come after you to work on the same or similar stuff will uncover your lie.
It's kind of sad that there are branches of science/research where there are people who feel the need to cheat (i guess that's what you're implying, right?). But i guess it's the usual problem...where there's real money, there are cheaters/liars.
-
There have been a number of famous cases of faked data, in particular I remember some doctored Stem Cell research coming out of South Korea.
If we're not striving to replicate data at an especially fast clip, there could be many more erroneous or outright false studies not being detected.
-
Well, as for underpowered studies or messing with statistics, basically that's what peer review process is for. The reviewers should be able to recognize these flaws and stop the study from being published in the journal. Unfortunately, there are many pseudo-scientific journals that would publish anything, as long as the authors pay. And peer review is not without faults (Sokal comes to mind).
-
I posted this in a prior thread, makes you wonder how much of science is just pseudoscience:
Why Most Published Research Findings Are False (http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.0020124)
There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.
-
As Jed Mckenna says, science should come with a disclaimer. WARNING: The scientific findings contained herein are based on the uncritical acceptance of consensus reality as true reality, and must therefore be viewed in the same light as mythology, folklore, superstition and religion.
Scientists, due to years of highly specialized training, are probably the most narrowly focused and effectively indoctrinated people in the modern world.
-
Oh boy!
Is that your view, Royce? Maybe that'd be worth an own thread.
Suffice it to say i have strong reservations with a lot of the parts of your post. Not the least of which are putting science in the same fantasy land as superstition and religion. And also thinking that scientists are narrowly focused and/or indoctrinated just because they are highly trained.
-
Oh boy!
Is that your view, Royce? Maybe that'd be worth an own thread.
Suffice it to say i have strong reservations with a lot of the parts of your post. Not the least of which are putting science in the same fantasy land as superstition and religion. And also thinking that scientists are narrowly focused and/or indoctrinated just because they are highly trained
Lol:) Is it my view? No not really. I borrow views all the time, change them around like underwear really. If you don`t you are a fundamentalist!:)
If I am 100% honest, and 100% skeptic, "science" is also a matter of belief. It is consensus, just taken a bit more seriously then religion. It is not a popular view, but it is true, if you are honest with yourself.
That being said, I am not anti-science or anything. It works, it definitely has its uses, but that does not make it truth. Just sayin:)
-
Well i also look at views of other people ;) But i am not sure one has to borrow them to not be a fundamentalist ;D
I am not sure that science and religion are on the same level of "belief". Religion is nothing but belief. Science is more substantial than that. Sure, depending on which field you look at, a lot of it is not something you can touch for real, so to speak. But it has real world applications that are reproducable and demonstrably true.
And the thing with the truth...well we talked about that at great length in another thread, didn't we ;) As long as there is no real difference for almost everyone on this planet if it is "real truth" (TM) or just "something like truth", it doesn't matter...and we can just call it truth. Even if technical definitions of quantum physics or neuroscience seem to have theories for the contrary, usefull is what finds an application in our lives, be they real or imaginative dreams of...whatever ;D
-
My fiance is in medicine and I often help her with literature reviews and the like. Most recently, we looked at studies assessing the benefits of vitamin supplements and it was shocking to see just how fraught with confirmation bias even a seemingly non-controversial topic could be.
A friend of mine who does business analysis pointed out that most scientists are not great statisticians, so their ability to distill the significance of their research is severely handicapped. I feel like the gap between our abstract ideas and the actual data it is supposedly based on is getting wider, creating a culture of certainty that is as unfounded as it is often pretentious.
-
Well i also look at views of other people ;) But i am not sure one has to borrow them to not be a fundamentalist ;D
I mean it literally though. I have no view or opinion which I can say is "mine". All my views are borrowed, always has been and always will be. I think that is the case for everyone. Unless you invent a new language, this must be the case:)
-
Not all science is created equal (http://www.rsc.org/chemistryworld/2014/10/not-all-science-created-equal)
With over 15 million publishing scientists nowadays and millions of papers published every year, meta-research is a data-rich, fertile discipline. It also offers plenty of opportunities to conduct experiments, such as randomised studies where new research practices can be compared head-to-head against established ones in controlled settings.
The Meta-Research Innovation Center at Stanford (Metrics), launched in April 2014 with generous support by the Laura and John Arnold Foundation, aims to perform such rigorous empirical research on research to identify, probe and validate best practices.
We are also eager to identify the best way to disseminate these practices across diverse scientific disciplines. This will require contributions from and collaborations with the global scientific community and stakeholders who want to promote the best possible science. Metrics aims to become a facilitating hub for such activities and to create an international web of investigators and methodologists who can stand up for good science.
Science is the best conception of the human mind and there is nothing in the physical world that cannot or should not be subjected to the scientific method. This includes the tools and practices of science itself. I invite investigators to consider this prospect and share in the Metrics initiative.
-
I mean it literally though. I have no view or opinion which I can say is "mine". All my views are borrowed, always has been and always will be. I think that is the case for everyone. Unless you invent a new language, this must be the case:)
I decidedly disagree. Just because some of us have similar or even the same views/opinions, does not mean you have to borrow/copy them from someone. You can come up with an opinion/view all on your own. And "this opinion is not new" is not the same as "you copied the opinion from someone else". You have a logical fallacy here, Royce.
From "this already exists out there" does not follow "if someone uses this, he must have copied/borrowed it".
For example: calculus was more or less developed by two mathematicians simultaneously (Leibniz and Newton). Which proves that ideas/views are not singular.
-
@Kellais
I am not sure we are talking about the same thing. Probably my fault though:)
I will give it another go. I agree that you can come up with a "new" idea. There is indeed a difference between a direct copy of an idea and a creative use of concepts and language which undoubtedly will seem "new".
My point here is only that we are born into a world of pre- existing ideas/views etc, and through learning language we get to understand these pre-existing ideas/views. We can twist and turn as much as we like with semantics, but it all boils down to learning. We learn, which means we take someone elses wisdom and twist and turn a bit, and in the end make it ours.
As a consequence of this I see it as improbable that there is any room for "my" idea. We all borrow from each other by sharing the same language. "Me" is just the sum total of all the input that has been coming my way through different mediums, forming my persona.
-
I think i see what you are getting at. And you are right, that we are shaped by our culture and the learning we receive/do. I still hold that even out of this, you can bring forth something new...at least as far as something like widsom/knowledge goes *. If you as a person can count as something "new" (and/or your views as a whole) ... i guess that's a very philosophical question. I cite the "you are more than the summ of your parts" quote and be done with it... i guess it shows in what direction i lean ** ;)
I think you also underestimate the value of "twist and turn" in that context? A lot of new stuff came out of starting by twisting and turning old stuff and then, suddenly, discover something totally differen/new.
*if that were not the case, science would be forever stuck in a circle where we discover nothing new...and that is demonstrably not the case.
** so from that angle, there IS room for "your" idea. Because it is very much probable that the exact "composition" of what makes your view/idea/etc. is unique and has never been here before.