Classes have started up again, and I am now once again a grad student. My blog's byline has been altered back to its original to reflect this. Expect a bit more free-ranging commentary from me, as I once again use my various coursework as inspiration for blog posts.
Also, due to some recent events in the online autistic community, I'm preemptively adding a policy against personal attacks in the comments here.
Friday, May 7, 2010
Tuesday, May 4, 2010
Autism's False Prophets
As most of my readers are probably aware, Paul Offit's famous (or infamous, depending on who you ask) book, Autism's False Prophets, came out in paperback last month. As most of my readers are probably unaware, this finally gave me the opportunity to buy a copy and read it in its entirety.
This was actually my first time doing so. I hadn't done so until now for reasons that had nothing to do with a lack of desire -- I simply haven't had the time or money to do so until now (and getting a copy of the hardback edition would stretch my budget a bit too much).
That said, it was well worth reading. Even though I already knew most of the story it told, having read many of the original sources that Offit cited, I still managed to learn some new things (e.g. the true story behind Dr. Geier's claim that testosterone binds to mercury). By and large, the information contained within the book is good, and the writing is highly accurate.
That said, I did find one factual error in the book. Specifically, on page three (of the paperback edition; it may be different in the hardcover), Offit refers to Bettelheim as "the first to offer a cure for autism". This is not only false, but pretty blatantly so... and even if the statement was true, it is something which would lie beyond Offit's ability to establish. It is true that Bettelheim was a the first highly visible, highly influential person to do so who is remembered (with much venom) today, but that's about it.
First off, America has a very long history of medical con-artistry and quackery. Establishing that Bettelheim was the first would require establishing that no snake-oil salesman had ever approached the parents of an autistic child and offered a faux cure prior to Bettelheim's publication of his book.
Secondly, Bettelheim initially published The Empty Fortress in 1967. This is significant -- falsifying the claim that Bettelheim was the first to offer a cure for autism would simply require demonstrating that someone else had tried an allegedly curative treatment on an autistic child prior to this.
Of course, autism was conceived of as a form of schizophrenia at the time; the realization that this belief is drastically wrong is only a relatively recent development... and the belief that they're the same thing (or related) is still periodically revived in a wide variety of forms. Moreover, a large number of treatments have been hailed as curative for schizophrenia, and many of these were tried on autistic children. In fact, you don't even have to look past Kanner's original sample to see this phenomenon.
Eisenberg's 1956 followup of the children treated at Johns Hopkins (a superset of Kanner's sample) reveals a similar pattern. Eisenberg refers to a "full range of psychiatric treatment" having been used, including electroconvulsive therapy (ECT). Even a cursory review of the literature available at the time shows that ECT was hailed as curative for schizophrenia by many of its practitioners and supporters. Personally, I suggest reading the relevant chapters of Whitaker's Mad in America for review.
That's even without getting into the matter of the orgone box which was used on one child.
As the error is understandable and this is only one clause in an otherwise accurate book, the matter can be viewed as an extremely minor issue. The larger problem lies not within how the book is inaccurate, but rather in how the book is incomplete.
First off, the discussion of ways in which the anti-vaccine/quack movement impacts and has impacted research is missing a major factor. While the book wonderfully describes the personal attacks on researchers and the wasted research efforts which have characterized the movement, it misses more indirect and pervasive harms. For one example: what has the effect been on recruitment for treatment research? I once had the distinct pleasure of speaking to a research psychiatrist about why sample sizes in trials of psychiatric treatments of autistic children are so low. His answer was that -- among other factors -- that it was extremely difficult to get families to participate... and he blamed the quack industry for this. After all, researchers need to get informed consent -- which means, among other things, a realistic picture of the potential impact of the drug being studied -- and there's a 50% chance of being assigned to the placebo arm of a RCT. By contrast, there's a 100% chance of receiving a quack's latest "miracle cure". Assuming you trust both sources of information, which would you choose?
Never mind the question of which is actually the better choice -- decision-making is based on perception, not reality.
Secondly, and more conspicuously, the book utterly ignores both the autistic rights movement and the fledgling autistic community... and the anti-vaccination movement's impact on them. This means that a very large portion of the issue -- such as autistic people's perceptions of the entire affair -- is utterly ignored. The anti-autistic stigma created by the movement is brushed off at best. The damage inherently caused by a view of autistic people as mercury poisoned is only briefly covered... by quotes from Kathleen Sidel and Camille Clark, who are parents.
Thus, the very real hardships, dismissals, and stigma faced by autistics on a regular basis because of these people is largely dismissed throughout the book, only to be specifically covered in one chapter... and even then it is only from the perspective of parents. The work and views of Jim Sinclair, of Amanda Baggs, of Ari Ne'eman, of Michelle Dawson, among others... are ignored. This is -- simply put -- not acceptable. Not only does this serve to marginalize us and exclude us from consideration in a discussion about us, but it also detracts considerably from the book's message.
By failing to take into account a large part of the story, Dr. Offit also manages to exclude a large portion of the harm and damage caused by the anti-vaccine movement. I really don't understand why he'd do this in a book about the anti-vaccine movement and the harms it's caused.
To be fair, Camille is autistic. She, however, is one person... and is invoked largely as a parent. The story of the community is discarded.
In short, Autism's False Prophets is a very good book... with one glaring flaw. It could be so much better if it wasn't for that one thing... and I cannot help but mourn the book it could have been even as I enjoy the book it is.
This was actually my first time doing so. I hadn't done so until now for reasons that had nothing to do with a lack of desire -- I simply haven't had the time or money to do so until now (and getting a copy of the hardback edition would stretch my budget a bit too much).
That said, it was well worth reading. Even though I already knew most of the story it told, having read many of the original sources that Offit cited, I still managed to learn some new things (e.g. the true story behind Dr. Geier's claim that testosterone binds to mercury). By and large, the information contained within the book is good, and the writing is highly accurate.
That said, I did find one factual error in the book. Specifically, on page three (of the paperback edition; it may be different in the hardcover), Offit refers to Bettelheim as "the first to offer a cure for autism". This is not only false, but pretty blatantly so... and even if the statement was true, it is something which would lie beyond Offit's ability to establish. It is true that Bettelheim was a the first highly visible, highly influential person to do so who is remembered (with much venom) today, but that's about it.
First off, America has a very long history of medical con-artistry and quackery. Establishing that Bettelheim was the first would require establishing that no snake-oil salesman had ever approached the parents of an autistic child and offered a faux cure prior to Bettelheim's publication of his book.
Secondly, Bettelheim initially published The Empty Fortress in 1967. This is significant -- falsifying the claim that Bettelheim was the first to offer a cure for autism would simply require demonstrating that someone else had tried an allegedly curative treatment on an autistic child prior to this.
Of course, autism was conceived of as a form of schizophrenia at the time; the realization that this belief is drastically wrong is only a relatively recent development... and the belief that they're the same thing (or related) is still periodically revived in a wide variety of forms. Moreover, a large number of treatments have been hailed as curative for schizophrenia, and many of these were tried on autistic children. In fact, you don't even have to look past Kanner's original sample to see this phenomenon.
Eisenberg's 1956 followup of the children treated at Johns Hopkins (a superset of Kanner's sample) reveals a similar pattern. Eisenberg refers to a "full range of psychiatric treatment" having been used, including electroconvulsive therapy (ECT). Even a cursory review of the literature available at the time shows that ECT was hailed as curative for schizophrenia by many of its practitioners and supporters. Personally, I suggest reading the relevant chapters of Whitaker's Mad in America for review.
That's even without getting into the matter of the orgone box which was used on one child.
As the error is understandable and this is only one clause in an otherwise accurate book, the matter can be viewed as an extremely minor issue. The larger problem lies not within how the book is inaccurate, but rather in how the book is incomplete.
First off, the discussion of ways in which the anti-vaccine/quack movement impacts and has impacted research is missing a major factor. While the book wonderfully describes the personal attacks on researchers and the wasted research efforts which have characterized the movement, it misses more indirect and pervasive harms. For one example: what has the effect been on recruitment for treatment research? I once had the distinct pleasure of speaking to a research psychiatrist about why sample sizes in trials of psychiatric treatments of autistic children are so low. His answer was that -- among other factors -- that it was extremely difficult to get families to participate... and he blamed the quack industry for this. After all, researchers need to get informed consent -- which means, among other things, a realistic picture of the potential impact of the drug being studied -- and there's a 50% chance of being assigned to the placebo arm of a RCT. By contrast, there's a 100% chance of receiving a quack's latest "miracle cure". Assuming you trust both sources of information, which would you choose?
Never mind the question of which is actually the better choice -- decision-making is based on perception, not reality.
Secondly, and more conspicuously, the book utterly ignores both the autistic rights movement and the fledgling autistic community... and the anti-vaccination movement's impact on them. This means that a very large portion of the issue -- such as autistic people's perceptions of the entire affair -- is utterly ignored. The anti-autistic stigma created by the movement is brushed off at best. The damage inherently caused by a view of autistic people as mercury poisoned is only briefly covered... by quotes from Kathleen Sidel and Camille Clark, who are parents.
Thus, the very real hardships, dismissals, and stigma faced by autistics on a regular basis because of these people is largely dismissed throughout the book, only to be specifically covered in one chapter... and even then it is only from the perspective of parents. The work and views of Jim Sinclair, of Amanda Baggs, of Ari Ne'eman, of Michelle Dawson, among others... are ignored. This is -- simply put -- not acceptable. Not only does this serve to marginalize us and exclude us from consideration in a discussion about us, but it also detracts considerably from the book's message.
By failing to take into account a large part of the story, Dr. Offit also manages to exclude a large portion of the harm and damage caused by the anti-vaccine movement. I really don't understand why he'd do this in a book about the anti-vaccine movement and the harms it's caused.
To be fair, Camille is autistic. She, however, is one person... and is invoked largely as a parent. The story of the community is discarded.
In short, Autism's False Prophets is a very good book... with one glaring flaw. It could be so much better if it wasn't for that one thing... and I cannot help but mourn the book it could have been even as I enjoy the book it is.
Wednesday, April 14, 2010
On Yet Another Stupidly Harmful Therapy
"Helminthic therapy" is the treatment of autoimmune disorders via deliberate exposure to parasitic worms. I know this for two reasons: One, I live within easy driving distance of what is apparently the quack capitol of the United States. Two, the parents of the kids I work with often have no common sense whatsoever.
If you've reached the obvious conclusion, you are entirely correct. I now know more than I ever wanted to about this, including the name of a company that will ship whipworm eggs from Thailand.
Ugh.
There really isn't much on the 'net about helminthic therapy and autism as of yet, but what I have found is suitably disturbing. The rationale is discussed here and here. The first part of a three-part video lecture is available here (with parts two and three available here and here, respectively). A very disgusting blog post can be found here.
Any other sources would be greatly appreciated.
If you've reached the obvious conclusion, you are entirely correct. I now know more than I ever wanted to about this, including the name of a company that will ship whipworm eggs from Thailand.
Ugh.
There really isn't much on the 'net about helminthic therapy and autism as of yet, but what I have found is suitably disturbing. The rationale is discussed here and here. The first part of a three-part video lecture is available here (with parts two and three available here and here, respectively). A very disgusting blog post can be found here.
Any other sources would be greatly appreciated.
Wednesday, April 7, 2010
On a Recent Huffington Post Article
Around a week ago, this article was posted to the Huffington post.
Quite frankly, my initial reaction to it was to think that it was an April Fool's joke. The sheer absurdity of the juxtaposition involved in talking about healing divides while spewing hate speech is striking.
Yes, hate speech. It's pretty unambiguous. Reading through the comments, however, it quickly becomes apparent that people don't get it.
Well, some commentators do. Thanks, Kim! (And, of course, all of the other people who I don't know...)
At the moment, the commentators seem to be divided between those who engage in reification error and those who don't. This is... pretty typical, really.
In any case... let's see. The post characterizes the neurodiversity movement as being made up of "people with Asperger's Syndrome or higher functioning autism" (this is nowhere near true, and "high functioning" is considered insulting), describes ASAN as "a self advocacy movement for people with high functioning autism" (ditto), and ends by stating that "We aren't the enemies. Autism is." (Which qualifies as blatant hate speech).
I really don't know what to say here...
Quite frankly, my initial reaction to it was to think that it was an April Fool's joke. The sheer absurdity of the juxtaposition involved in talking about healing divides while spewing hate speech is striking.
Yes, hate speech. It's pretty unambiguous. Reading through the comments, however, it quickly becomes apparent that people don't get it.
Well, some commentators do. Thanks, Kim! (And, of course, all of the other people who I don't know...)
At the moment, the commentators seem to be divided between those who engage in reification error and those who don't. This is... pretty typical, really.
In any case... let's see. The post characterizes the neurodiversity movement as being made up of "people with Asperger's Syndrome or higher functioning autism" (this is nowhere near true, and "high functioning" is considered insulting), describes ASAN as "a self advocacy movement for people with high functioning autism" (ditto), and ends by stating that "We aren't the enemies. Autism is." (Which qualifies as blatant hate speech).
I really don't know what to say here...
Sunday, March 21, 2010
A Bit More on Translational Research
Translational research is a phenomenally complex topic, but generally refers to the basic idea of taking discoveries made in basic research and researching how they apply "in the real world". It's worth noting that this (usually) does not mean applied research, although the basic concepts are pretty similar in a lot of ways. The main difference is that transitional research paradigms reject the dichotomy between basic and applied research when doing so. "Basic" research within a translational research paradigm both informs and is informed by "applied" research, blurring the lines between the two.
Any more complex explanation of research paradigms in this context, however, would require an explanation of the field of medical informatics. Frankly, I don't want to go there.
I will admit that I have some reservations and concerns regarding the entire translational research paradigm. I will also admit that there is a good chance that this is because of the limitations to my understanding of it. It is quite possible -- even probable -- that my concerns and reservations have been addressed.
Of course, it also doesn't help that there isn't a standardized definition of "translational research", and some definitions conflict -- often in major ways -- with the above (e.g. this one).
Most people, however, don't need to really understand research paradigms. What they need to understand -- even if only in general terms -- are the challenges that those paradigms were designed to address.
Simply put -- as impressive as modern medical science is, we really don't understand that much about how the human body works. This is why most "promising new treatments" turn out to be worthless -- or, all too often, worse than worthless. It's the aspects of biology that we don't understand that keep tripping us up, time after time after time.
This is why any new treatment has to be tested -- thoroughly. This is why rushing the process is a very bad idea. It's also why many of the medications we use have nasty side effects.
Simply put: as much as we'd like to believe otherwise, modern doctors, pharmacists, and so on don't really know what they're doing.
Any more complex explanation of research paradigms in this context, however, would require an explanation of the field of medical informatics. Frankly, I don't want to go there.
I will admit that I have some reservations and concerns regarding the entire translational research paradigm. I will also admit that there is a good chance that this is because of the limitations to my understanding of it. It is quite possible -- even probable -- that my concerns and reservations have been addressed.
Of course, it also doesn't help that there isn't a standardized definition of "translational research", and some definitions conflict -- often in major ways -- with the above (e.g. this one).
Most people, however, don't need to really understand research paradigms. What they need to understand -- even if only in general terms -- are the challenges that those paradigms were designed to address.
Simply put -- as impressive as modern medical science is, we really don't understand that much about how the human body works. This is why most "promising new treatments" turn out to be worthless -- or, all too often, worse than worthless. It's the aspects of biology that we don't understand that keep tripping us up, time after time after time.
This is why any new treatment has to be tested -- thoroughly. This is why rushing the process is a very bad idea. It's also why many of the medications we use have nasty side effects.
Simply put: as much as we'd like to believe otherwise, modern doctors, pharmacists, and so on don't really know what they're doing.
Saturday, March 20, 2010
Ten Research Methods Articles Every Parent Of an Autistic Child Should Understand
Perhaps understandably, many parents of autistic children keep an eye on (or attempt to keep an eye on) the latest research and treatments. Parents' lists are regularly flooded with discussion of various treatment methods, medical research studies, psychological studies, and just about anything else one could imagine... most of it related to research.
At the same time, the average parent knows around as much about how research is conducted and about what study findings really mean as does the average sixth-grader. This combination is very much not a good thing.
Simply put, many of the conclusions parents reach when reading the literature are not anywhere near accurate. Even many of the most basic aspects of research are commonly misunderstood in truly dramatic fashions.
The ideal solution, of course, would be to sit every parent of an autistic child down and give them a series of college-level (undergraduate or postgraduate) classes on the scientific process. Unfortunately, this is pretty spectacularly unrealistic. Frankly, many parents could use classes on critical thinking skills, too, but that's every bit as unrealistic.
As such, I've compiled a "Top Ten" list of papers which cover things that most parents don't get. Of course, like any such list, there are a number of biases operating in how I've constructed this. Perhaps a reader will be able to spot some of these... and, as an exercise for my readers, I've tried to make a few of them as blatantly obvious as possible. Narrowing this down to ten papers was emphatically not easy, and there are a number of papers which almost (but did not quite) make the cut for a variety of reasons. I may blog on a few of them later.
To finish: for the purposes of this list, I've defined "research methods articles" as any peer-reviewed writing dealing primarily with the design, conducting, and interpretation of research.
Edit: Also note that there's usually more than one good article on any of these topics, and I excluded "duplicate" articles. I often had to drop very good articles which deserved to be on this list because of that.
10. Strech & Tilburt (2008). Value judgements in the analysis and synthesis of evidence.
One of the (many) reasons why conflicts of interest are so important when dealing with research is the fact that there is a lot of wiggle-room in experimental design. Scientists routinely make value judgments in designing and interpreting research, and this paper serves to highlight many of the ways in which this impacts the process of research.
Just as a quick illustration: how do my value judgments impact the content of this top 10 list?
9. Rutter (2008). Epidemiological methods to tackle causal questions.
In this paper, Matthew Rutter (who I admittedly have issues with relating to other works) discusses the issue of attempting to determine the cause of something without being able to manipulate it in a lab (or clinical trial, etc.). Of course, in the modern autism world, it's more important to understand how it's possible to establish the reverse -- that something is not the cause of something else -- without an experiment... and that is, admittedly, often far simpler.
8. Lesaffre (2008). Use and misuse of the p-value.
Statistical significance testing is one of the most ubiquitous aspects of the modern scientific process. Unfortunately, it's also the source of many of the problems with it. As scientists haven't been able to find (or settle on) a better alternative, however, it still pops up just about everywhere.
One statistic -- the p-value -- is central to this process. Unfortunately, many people (including scientists) misunderstand just what the p-value is, what it means, and what it represents. Lesaffre's paper discusses this and the issues surrounding it.
7. The PRISMA statement.
One of a number of "reporting standards" documents which standardize scientific reporting in the medical literature, the PRISMA statement deals with systematic reviews and meta-analyses, and specifically with which items of the review's process and methodology most need to be reported.
The other documents of this type (e.g. the CONSORT and STROBE statements) are also very important, but the PRISMA statement deals with ways to document the possibility of biases that effect the process of drawing a conclusion from the entire body of available literature. By contrast, the others deal with the conclusion of single studies.
Of course, since what's important is understanding, most interested parents should read the explanation and elaboration document, not the PRISMA statement itself.
Were I doing a longer list, the CONSORT statement, at least, would be in here. As is, however, I believe that the biases covered by the PRISMA statement to be more important for parents of autistic children to understand... and, frankly, I felt that one major standards document was enough for this list.
6. Manchikanti (2008). Evidence-based medicine, systematic reviews, and guidelines in interventional pain management, part I: Introduction and general considerations.
The concept of evidence-based medicine has revolutionized clinical practice over the past few decades. This article discusses the concept of evidence-based medicine, its history, its tools, and countless other related topics, providing a great introduction to the medical literature... and provides a basic foundation for understanding it.
Best of all, it's available for free.
5. Ioannidis (2008). Perfect study, poor evidence: Interpretation of biases preceding study design.
Even if a study is designed, conducted, analyzed, and reported perfectly, it can still be biased or otherwise flawed in a large number of ways. This paper reviews and discusses this phenomenon, including (but not even close to limited to) such factors as poor scientific relevance, straw man effects, and the importance of the analysis of the geometry of a research field.
4. Ioannidis (2008). Why most discovered true associations are inflated.
When something is first discovered, researchers' estimates of its importance are generally exaggerated. This article discusses this phenomenon and the reasons for it, painting an unusually frank and readable picture of just why this happens.
3. Ioannidis (2005). Why most published research findings are false.
One of the more annoying aspects of science is the fact that we know that most of our discoveries are simply wrong. The problem, however, is that we don't usually know which ones until far later. This is one of many reasons why replication is so important in the sciences and why the habit of interpreting individual studies, taken in isolation, as "definitive" is really, really problematic. "False positive" findings abound in science -- especially the social and medical sciences -- and often lead armchair scientists or doctors astray.
This paper, one of the most influential papers published in the last decade, discusses this phenomenon and the reasons for it. If you have time to read the responses and the discussion that followed the publication of this article, that is also very much worth the effort of doing. I particularly recommend Mooneshinghe, Khoury, & Jannssens's (2007) essay, Most published research findings are false—But a little replication goes a long way.
The full text is available for free. I really love open-access scientific literature. Long live PLoS!
2. Altman (2002). Poor-quality medical research: What can journals do?
Poor-quality research is a problem in any field. Simply put, it's possible for a poorly-designed study to find anything, no matter how absurd. If I really wanted, I could easily design a study that, while looking legitimate to uninformed non-experts, would conclude that the Rocky mountains are flatter than a random pancake from IHOP. There is even precedent for this.
This is why expertise in experimental design and research methods is so important... both for designing and interpreting studies. Critical appraisal of any research is key, and you can never just trust the author's interpretation of his own work. It's also a large part of why a number of processes (e.g. peer review) are in place and why doctors get so up in arms about irresponsible media reporting.
And this article is available for free from JAMA. Have I yet mentioned that I love open-access literature?
Where to begin? This one article manages to cover about half of what's wrong with modern clinical autism research and with autism research funding priorities. Forget the political issues involved and the tie-ins between genetic research and prenatal testing. Forget the issue of whether a medical model is appropriate for autism or not. Forget even the normocentric bias which pervades most autism research and the question of whether or not it is appropriate to view autism as a disease.
Today, most funding into basic research into autism goes into attempts to understand the underlying biological processes that differentiate autistic and non-autistic individuals. This is a tremendously complicated task, one which has countless problems which I could rant about for hours. The sheer amount of money which this task has already gone into this task (and which it will likely require in the future) is mindboggling... not to mention researcher time and effort, etc.
This doesn't mean that I think that the task is worthless. Basic understanding of biological factors and processes is rarely worthless. There are, however, a phenomenal number of difficulties in taking these (usually incomplete) understandings and doing anything useful with them. At the same time, other approaches (what Simon refers to as focusing on "predictive laws rather than on trying to understand the [biological] why of those laws", p. 2 of the author's manuscript, parenthetical word added) offer far better cost-effectiveness... and neatly avoid a lot of the convolutions in the process which Simon spends the rest of the paper explaining.
Edit (3/21/10): An excellent discussion of some of the factors I'm trying to talk about can be found here. A dissenting -- but still valid -- opinion regarding that specific application can be found here. Many thanks to Tyler Cowen and Michelle Dawson for highlighting these and pointing me in their directions. Additionally, to avoid a misreading of the above: focusing on predictive laws does not avoid the need for translational research; it simply makes the process thereof less convoluted. One example of this would be in finding and using valid and robust surrogate endpoints within studies... but this is a much, much longer discussion.
This paper (the author's manuscript of which is available for free here) is an excellent discussion these problems... albeit in a different context. This context actually represents the paper's largest flaw, one which annoys the heck out of me: Simon is a cancer researcher and the paper was published in a cancer research journal. If, however, we are to understand the issues involved with taking the medical research models applied to cancer and applying them to autism (as Autism Speaks and others repeatedly insist on doing), we must first understand the issues with those models in general -- and they very much have them, even when they're used appropriately. This paper does an excellent job of highlighting those.
Edit: Corrected a spelling error.
At the same time, the average parent knows around as much about how research is conducted and about what study findings really mean as does the average sixth-grader. This combination is very much not a good thing.
Simply put, many of the conclusions parents reach when reading the literature are not anywhere near accurate. Even many of the most basic aspects of research are commonly misunderstood in truly dramatic fashions.
The ideal solution, of course, would be to sit every parent of an autistic child down and give them a series of college-level (undergraduate or postgraduate) classes on the scientific process. Unfortunately, this is pretty spectacularly unrealistic. Frankly, many parents could use classes on critical thinking skills, too, but that's every bit as unrealistic.
As such, I've compiled a "Top Ten" list of papers which cover things that most parents don't get. Of course, like any such list, there are a number of biases operating in how I've constructed this. Perhaps a reader will be able to spot some of these... and, as an exercise for my readers, I've tried to make a few of them as blatantly obvious as possible. Narrowing this down to ten papers was emphatically not easy, and there are a number of papers which almost (but did not quite) make the cut for a variety of reasons. I may blog on a few of them later.
To finish: for the purposes of this list, I've defined "research methods articles" as any peer-reviewed writing dealing primarily with the design, conducting, and interpretation of research.
Edit: Also note that there's usually more than one good article on any of these topics, and I excluded "duplicate" articles. I often had to drop very good articles which deserved to be on this list because of that.
10. Strech & Tilburt (2008). Value judgements in the analysis and synthesis of evidence.
One of the (many) reasons why conflicts of interest are so important when dealing with research is the fact that there is a lot of wiggle-room in experimental design. Scientists routinely make value judgments in designing and interpreting research, and this paper serves to highlight many of the ways in which this impacts the process of research.
Just as a quick illustration: how do my value judgments impact the content of this top 10 list?
9. Rutter (2008). Epidemiological methods to tackle causal questions.
In this paper, Matthew Rutter (who I admittedly have issues with relating to other works) discusses the issue of attempting to determine the cause of something without being able to manipulate it in a lab (or clinical trial, etc.). Of course, in the modern autism world, it's more important to understand how it's possible to establish the reverse -- that something is not the cause of something else -- without an experiment... and that is, admittedly, often far simpler.
8. Lesaffre (2008). Use and misuse of the p-value.
Statistical significance testing is one of the most ubiquitous aspects of the modern scientific process. Unfortunately, it's also the source of many of the problems with it. As scientists haven't been able to find (or settle on) a better alternative, however, it still pops up just about everywhere.
One statistic -- the p-value -- is central to this process. Unfortunately, many people (including scientists) misunderstand just what the p-value is, what it means, and what it represents. Lesaffre's paper discusses this and the issues surrounding it.
7. The PRISMA statement.
One of a number of "reporting standards" documents which standardize scientific reporting in the medical literature, the PRISMA statement deals with systematic reviews and meta-analyses, and specifically with which items of the review's process and methodology most need to be reported.
The other documents of this type (e.g. the CONSORT and STROBE statements) are also very important, but the PRISMA statement deals with ways to document the possibility of biases that effect the process of drawing a conclusion from the entire body of available literature. By contrast, the others deal with the conclusion of single studies.
Of course, since what's important is understanding, most interested parents should read the explanation and elaboration document, not the PRISMA statement itself.
Were I doing a longer list, the CONSORT statement, at least, would be in here. As is, however, I believe that the biases covered by the PRISMA statement to be more important for parents of autistic children to understand... and, frankly, I felt that one major standards document was enough for this list.
6. Manchikanti (2008). Evidence-based medicine, systematic reviews, and guidelines in interventional pain management, part I: Introduction and general considerations.
The concept of evidence-based medicine has revolutionized clinical practice over the past few decades. This article discusses the concept of evidence-based medicine, its history, its tools, and countless other related topics, providing a great introduction to the medical literature... and provides a basic foundation for understanding it.
Best of all, it's available for free.
5. Ioannidis (2008). Perfect study, poor evidence: Interpretation of biases preceding study design.
Even if a study is designed, conducted, analyzed, and reported perfectly, it can still be biased or otherwise flawed in a large number of ways. This paper reviews and discusses this phenomenon, including (but not even close to limited to) such factors as poor scientific relevance, straw man effects, and the importance of the analysis of the geometry of a research field.
4. Ioannidis (2008). Why most discovered true associations are inflated.
When something is first discovered, researchers' estimates of its importance are generally exaggerated. This article discusses this phenomenon and the reasons for it, painting an unusually frank and readable picture of just why this happens.
3. Ioannidis (2005). Why most published research findings are false.
One of the more annoying aspects of science is the fact that we know that most of our discoveries are simply wrong. The problem, however, is that we don't usually know which ones until far later. This is one of many reasons why replication is so important in the sciences and why the habit of interpreting individual studies, taken in isolation, as "definitive" is really, really problematic. "False positive" findings abound in science -- especially the social and medical sciences -- and often lead armchair scientists or doctors astray.
This paper, one of the most influential papers published in the last decade, discusses this phenomenon and the reasons for it. If you have time to read the responses and the discussion that followed the publication of this article, that is also very much worth the effort of doing. I particularly recommend Mooneshinghe, Khoury, & Jannssens's (2007) essay, Most published research findings are false—But a little replication goes a long way.
The full text is available for free. I really love open-access scientific literature. Long live PLoS!
2. Altman (2002). Poor-quality medical research: What can journals do?
Poor-quality research is a problem in any field. Simply put, it's possible for a poorly-designed study to find anything, no matter how absurd. If I really wanted, I could easily design a study that, while looking legitimate to uninformed non-experts, would conclude that the Rocky mountains are flatter than a random pancake from IHOP. There is even precedent for this.
This is why expertise in experimental design and research methods is so important... both for designing and interpreting studies. Critical appraisal of any research is key, and you can never just trust the author's interpretation of his own work. It's also a large part of why a number of processes (e.g. peer review) are in place and why doctors get so up in arms about irresponsible media reporting.
And this article is available for free from JAMA. Have I yet mentioned that I love open-access literature?
Where to begin? This one article manages to cover about half of what's wrong with modern clinical autism research and with autism research funding priorities. Forget the political issues involved and the tie-ins between genetic research and prenatal testing. Forget the issue of whether a medical model is appropriate for autism or not. Forget even the normocentric bias which pervades most autism research and the question of whether or not it is appropriate to view autism as a disease.
Today, most funding into basic research into autism goes into attempts to understand the underlying biological processes that differentiate autistic and non-autistic individuals. This is a tremendously complicated task, one which has countless problems which I could rant about for hours. The sheer amount of money which this task has already gone into this task (and which it will likely require in the future) is mindboggling... not to mention researcher time and effort, etc.
This doesn't mean that I think that the task is worthless. Basic understanding of biological factors and processes is rarely worthless. There are, however, a phenomenal number of difficulties in taking these (usually incomplete) understandings and doing anything useful with them. At the same time, other approaches (what Simon refers to as focusing on "predictive laws rather than on trying to understand the [biological] why of those laws", p. 2 of the author's manuscript, parenthetical word added) offer far better cost-effectiveness... and neatly avoid a lot of the convolutions in the process which Simon spends the rest of the paper explaining.
Edit (3/21/10): An excellent discussion of some of the factors I'm trying to talk about can be found here. A dissenting -- but still valid -- opinion regarding that specific application can be found here. Many thanks to Tyler Cowen and Michelle Dawson for highlighting these and pointing me in their directions. Additionally, to avoid a misreading of the above: focusing on predictive laws does not avoid the need for translational research; it simply makes the process thereof less convoluted. One example of this would be in finding and using valid and robust surrogate endpoints within studies... but this is a much, much longer discussion.
This paper (the author's manuscript of which is available for free here) is an excellent discussion these problems... albeit in a different context. This context actually represents the paper's largest flaw, one which annoys the heck out of me: Simon is a cancer researcher and the paper was published in a cancer research journal. If, however, we are to understand the issues involved with taking the medical research models applied to cancer and applying them to autism (as Autism Speaks and others repeatedly insist on doing), we must first understand the issues with those models in general -- and they very much have them, even when they're used appropriately. This paper does an excellent job of highlighting those.
Edit: Corrected a spelling error.
Thursday, March 18, 2010
On Humor
Taken yesterday from my Facebook profile:
I believe this speaks for itself.
Alexander Cheezem just spent a while chasing after his pet sixteen-year-old, three-legged dog after she ran out of the house. I'm also still on crutches and can't put weight on my left leg. I'm fairly certain that some sort of disability humor can be derived from this, but I'm too exhausted to do so myself.
I believe this speaks for itself.
Subscribe to:
Posts (Atom)
Posts
