Sunday, March 21, 2010

A Bit More on Translational Research

Translational research is a phenomenally complex topic, but generally refers to the basic idea of taking discoveries made in basic research and researching how they apply "in the real world". It's worth noting that this (usually) does not mean applied research, although the basic concepts are pretty similar in a lot of ways. The main difference is that transitional research paradigms reject the dichotomy between basic and applied research when doing so. "Basic" research within a translational research paradigm both informs and is informed by "applied" research, blurring the lines between the two.

Any more complex explanation of research paradigms in this context, however, would require an explanation of the field of medical informatics. Frankly, I don't want to go there.

I will admit that I have some reservations and concerns regarding the entire translational research paradigm. I will also admit that there is a good chance that this is because of the limitations to my understanding of it. It is quite possible -- even probable -- that my concerns and reservations have been addressed.

Of course, it also doesn't help that there isn't a standardized definition of "translational research", and some definitions conflict -- often in major ways -- with the above (e.g. this one).

Most people, however, don't need to really understand research paradigms. What they need to understand -- even if only in general terms -- are the challenges that those paradigms were designed to address.

Simply put -- as impressive as modern medical science is, we really don't understand that much about how the human body works. This is why most "promising new treatments" turn out to be worthless -- or, all too often, worse than worthless. It's the aspects of biology that we don't understand that keep tripping us up, time after time after time.

This is why any new treatment has to be tested -- thoroughly. This is why rushing the process is a very bad idea. It's also why many of the medications we use have nasty side effects.

Simply put: as much as we'd like to believe otherwise, modern doctors, pharmacists, and so on don't really know what they're doing.

Saturday, March 20, 2010

Ten Research Methods Articles Every Parent Of an Autistic Child Should Understand

Perhaps understandably, many parents of autistic children keep an eye on (or attempt to keep an eye on) the latest research and treatments. Parents' lists are regularly flooded with discussion of various treatment methods, medical research studies, psychological studies, and just about anything else one could imagine... most of it related to research.

At the same time, the average parent knows around as much about how research is conducted and about what study findings really mean as does the average sixth-grader. This combination is very much not a good thing.

Simply put, many of the conclusions parents reach when reading the literature are not anywhere near accurate. Even many of the most basic aspects of research are commonly misunderstood in truly dramatic fashions.

The ideal solution, of course, would be to sit every parent of an autistic child down and give them a series of college-level (undergraduate or postgraduate) classes on the scientific process. Unfortunately, this is pretty spectacularly unrealistic. Frankly, many parents could use classes on critical thinking skills, too, but that's every bit as unrealistic.

As such, I've compiled a "Top Ten" list of papers which cover things that most parents don't get. Of course, like any such list, there are a number of biases operating in how I've constructed this. Perhaps a reader will be able to spot some of these... and, as an exercise for my readers, I've tried to make a few of them as blatantly obvious as possible. Narrowing this down to ten papers was emphatically not easy, and there are a number of papers which almost (but did not quite) make the cut for a variety of reasons. I may blog on a few of them later.

To finish: for the purposes of this list, I've defined "research methods articles" as any peer-reviewed writing dealing primarily with the design, conducting, and interpretation of research.

Edit: Also note that there's usually more than one good article on any of these topics, and I excluded "duplicate" articles. I often had to drop very good articles which deserved to be on this list because of that.

10. Strech & Tilburt (2008). Value judgements in the analysis and synthesis of evidence.

One of the (many) reasons why conflicts of interest are so important when dealing with research is the fact that there is a lot of wiggle-room in experimental design. Scientists routinely make value judgments in designing and interpreting research, and this paper serves to highlight many of the ways in which this impacts the process of research.

Just as a quick illustration: how do my value judgments impact the content of this top 10 list?

9. Rutter (2008). Epidemiological methods to tackle causal questions.

In this paper, Matthew Rutter (who I admittedly have issues with relating to other works) discusses the issue of attempting to determine the cause of something without being able to manipulate it in a lab (or clinical trial, etc.). Of course, in the modern autism world, it's more important to understand how it's possible to establish the reverse -- that something is not the cause of something else -- without an experiment... and that is, admittedly, often far simpler.

8. Lesaffre (2008). Use and misuse of the p-value.

Statistical significance testing is one of the most ubiquitous aspects of the modern scientific process. Unfortunately, it's also the source of many of the problems with it. As scientists haven't been able to find (or settle on) a better alternative, however, it still pops up just about everywhere.

One statistic -- the p-value -- is central to this process. Unfortunately, many people (including scientists) misunderstand just what the p-value is, what it means, and what it represents. Lesaffre's paper discusses this and the issues surrounding it.

7. The PRISMA statement.

One of a number of "reporting standards" documents which standardize scientific reporting in the medical literature, the PRISMA statement deals with systematic reviews and meta-analyses, and specifically with which items of the review's process and methodology most need to be reported.

The other documents of this type (e.g. the CONSORT and STROBE statements) are also very important, but the PRISMA statement deals with ways to document the possibility of biases that effect the process of drawing a conclusion from the entire body of available literature. By contrast, the others deal with the conclusion of single studies.

Of course, since what's important is understanding, most interested parents should read the explanation and elaboration document, not the PRISMA statement itself.

Were I doing a longer list, the CONSORT statement, at least, would be in here. As is, however, I believe that the biases covered by the PRISMA statement to be more important for parents of autistic children to understand... and, frankly, I felt that one major standards document was enough for this list.

6. Manchikanti (2008).
Evidence-based medicine, systematic reviews, and guidelines in interventional pain management, part I: Introduction and general considerations.

The concept of evidence-based medicine has revolutionized clinical practice over the past few decades. This article discusses the concept of evidence-based medicine, its history, its tools, and countless other related topics, providing a great introduction to the medical literature... and provides a basic foundation for understanding it.

Best of all, it's available for free.

5. Ioannidis (2008). Perfect study, poor evidence: Interpretation of biases preceding study design.

Even if a study is designed, conducted, analyzed, and reported perfectly, it can still be biased or otherwise flawed in a large number of ways. This paper reviews and discusses this phenomenon, including (but not even close to limited to) such factors as poor scientific relevance, straw man effects, and the importance of the analysis of the geometry of a research field.

4. Ioannidis (2008). Why most discovered true associations are inflated.

When something is first discovered, researchers' estimates of its importance are generally exaggerated. This article discusses this phenomenon and the reasons for it, painting an unusually frank and readable picture of just why this happens.

3. Ioannidis (2005). Why most published research findings are false.

One of the more annoying aspects of science is the fact that we know that most of our discoveries are simply wrong. The problem, however, is that we don't usually know which ones until far later. This is one of many reasons why replication is so important in the sciences and why the habit of interpreting individual studies, taken in isolation, as "definitive" is really, really problematic. "False positive" findings abound in science -- especially the social and medical sciences -- and often lead armchair scientists or doctors astray.

This paper, one of the most influential papers published in the last decade, discusses this phenomenon and the reasons for it. If you have time to read the responses and the discussion that followed the publication of this article, that is also very much worth the effort of doing. I particularly recommend Mooneshinghe, Khoury, & Jannssens's (2007) essay, Most published research findings are false—But a little replication goes a long way.

The full text is available for free. I really love open-access scientific literature. Long live PLoS!

2. Altman (2002). Poor-quality medical research: What can journals do?

Poor-quality research is a problem in any field. Simply put, it's possible for a poorly-designed study to find anything, no matter how absurd. If I really wanted, I could easily design a study that, while looking legitimate to uninformed non-experts, would conclude that the Rocky mountains are flatter than a random pancake from IHOP. There is even precedent for this.

This is why expertise in experimental design and research methods is so important... both for designing and interpreting studies. Critical appraisal of any research is key, and you can never just trust the author's interpretation of his own work. It's also a large part of why a number of processes (e.g. peer review) are in place and why doctors get so up in arms about irresponsible media reporting.

And this article is available for free from JAMA. Have I yet mentioned that I love open-access literature?

1. Simon (2008). Lost in translation: problems and pitfalls in translating laboratory observations to clinical utility.

Where to begin? This one article manages to cover about half of what's wrong with modern clinical autism research and with autism research funding priorities. Forget the political issues involved and the tie-ins between genetic research and prenatal testing. Forget the issue of whether a medical model is appropriate for autism or not. Forget even the normocentric bias which pervades most autism research and the question of whether or not it is appropriate to view autism as a disease.

Today, most funding into basic research into autism goes into attempts to understand the underlying biological processes that differentiate autistic and non-autistic individuals. This is a tremendously complicated task, one which has countless problems which I could rant about for hours. The sheer amount of money which this task has already gone into this task (and which it will likely require in the future) is mindboggling... not to mention researcher time and effort, etc.

This doesn't mean that I think that the task is worthless. Basic understanding of biological factors and processes is rarely worthless. There are, however, a phenomenal number of difficulties in taking these (usually incomplete) understandings and doing anything useful with them. At the same time, other approaches (what Simon refers to as focusing on "predictive laws rather than on trying to understand the [biological] why of those laws", p. 2 of the author's manuscript, parenthetical word added) offer far better cost-effectiveness... and neatly avoid a lot of the convolutions in the process which Simon spends the rest of the paper explaining.

Edit (3/21/10): An excellent discussion of some of the factors I'm trying to talk about can be found here. A dissenting -- but still valid -- opinion regarding that specific application can be found here. Many thanks to Tyler Cowen and Michelle Dawson for highlighting these and pointing me in their directions. Additionally, to avoid a misreading of the above: focusing on predictive laws does not avoid the need for translational research; it simply makes the process thereof less convoluted. One example of this would be in finding and using valid and robust surrogate endpoints within studies... but this is a much, much longer discussion.

This paper (the author's manuscript of which is available for free here) is an excellent discussion these problems... albeit in a different context. This context actually represents the paper's largest flaw, one which annoys the heck out of me: Simon is a cancer researcher and the paper was published in a cancer research journal. If, however, we are to understand the issues involved with taking the medical research models applied to cancer and applying them to autism (as Autism Speaks and others repeatedly insist on doing), we must first understand the issues with those models in general -- and they very much have them, even when they're used appropriately. This paper does an excellent job of highlighting those.

Edit: Corrected a spelling error.

Thursday, March 18, 2010

On Humor

Taken yesterday from my Facebook profile:

Alexander Cheezem just spent a while chasing after his pet sixteen-year-old, three-legged dog after she ran out of the house. I'm also still on crutches and can't put weight on my left leg. I'm fairly certain that some sort of disability humor can be derived from this, but I'm too exhausted to do so myself.

I believe this speaks for itself.

Wednesday, March 17, 2010

On the Autism Health and Wellness Expo, Part Two

It is difficult to convey exactly how charismatic, convincing, and generally smooth the Geiers are. I've been trying to put the matter to words, and failing repeatedly. I have deleted the text of this message several times and started over just as many.

I suppose one anecdote from meeting them -- while minor -- will help. I knew several of the parents there. One of them was a highly qualified clinician who I've had a degree of experience with. She has always been among the more reasonable of the professionals in the area (in certain respects, anyway -- I've never worked with her clinically) and is a highly intelligent speech-language pathologist with a nineteen-year-old autistic son.

Said son apparently showed an interest in pornography for the first time rather recently. After attending the Geiers' presentation, she is now looking into getting him chemically castrated.

Sunday, March 14, 2010

On the Autism Health and Wellness Expo, Part One

Perhaps it is unsurprising that the quacks came out in force for an event entitled the "Autism Health and Wellness Expo". On the other hand, it says something that Mark and David Geier were not the biggest quacks there.

And yes, they were there. I listened to one of their talks and wound up meeting the father in their father/son castration team. It was an... interesting experience.

The Geiers' talk involved some truly spectacular fearmongering. Among other things, they claimed that they believe that autism's prevalence was at one in 20 to 30 during the "peak years" (whatever that means). They claimed that most autistics have mitochondrial dysfunction of some kind or another. They engaged in some truly insipid anti-vaccine fearmongering.

They also defined aggression as including meltdowns and frustration.

To Dr. Geier's credit, however, he did (in a subsequent discussion) criticize DAN! -- basically for being frauds, although he also called them pawns of Big Pharma. He acknowledged that hyperbaric oxygen "therapy" for autism is bunk (at least how they practice it). He gave me some rather interesting information regarding vitamin manufacturers. He stopped just short of actively calling Bradstreet a quack (and did strongly imply this). He repeatedly emphasized the need for using a reputable lab to conduct legitimate tests (of course, if you don't interpret them correctly...).

The Geiers' science was every bit as bad as I'd been expecting. There are major parts of their presentation that I really wish I could still remember -- such as the details of how they concluded that autism is really mercury poisoning (it was shoddy logic based on urinary porphyrin testing, but I don't remember several key details).

They were also pretty blatantly recruiting for CoMeD, even going so far as to hand out pamphlets. The less said about this, the better.

Until today, I wondered how a parent could ever fall for their quackery. I no longer wonder. The Geiers are a truly polished act, con artists extraordinaire. Some of the tactics they used... the liberal use of the foot-in-the-door phenomenon... the polished presentation of their pseudoscience (it was often extremely difficult to detect this without extensive knowledge of the topics -- and studies -- they were talking about)... the polished use of "reasonable" stances towards various groups (no one was perfect in his eyes, everyone had flaws and strengths)...

It scares me. It really does. I'll have to write more later, but keep in mind that they weren't the quackiest people to speak at that expo. The honor of "biggest quack" goes to Dr. Phil Bate. More on him later.

Sunday, March 7, 2010

A Recent Blog Post

It's somewhat ironic, given my recent decision to post my discussion topic for the project, that the subject of that discussion would promptly come up in one of the autistic blogosphere's better blogs. This, however, is precisely what happened.

Amanda's post on the topic is (among other things) a highly eloquent and effective explanation of one aspect of what I felt -- and still feel -- needs much more exploration by everyone involved in the field of autism. I highly recommend reading it in its entirety.

Wednesday, March 3, 2010

A Discussion Topic

Around a year ago, I recorded a discussion topic for the Dan Marino Foundation's project. They have yet to use it, much to my disappointment... as I view it as a very important question that doesn't get asked enough.

Fortunately, I'd written out my question... and I still have the transcript. As such, I'm able to post it here.

Hi, I'm Alex Cheezem, a graduate student at Nova Southeastern University and an adult on the autism spectrum. One issue that surrounds any discussion of autism is the matter of language. Words often have different meanings depending on context -- for instance, the technical language of behavior analysis defines an aversive as anything that an organism moves to avoid, while the term often brings up images of ammonia squirts to the face, electrical shock, and other such painful and humiliating "treatments".

At the same time, the spoken languages of our cultures are often a second language of sorts to us. Countless accounts are available of our thought processes and the ways in which we think, but these ways are often alien to those people who think in a more typical manner. The underlying assumptions of these modes of thought are often different from those of what we call the English language.

Finally, much human communication is in the form of subtle implcations and connotations. While people on the spectrum often have difficulty perceiving these, people not on the spectrum often have trouble with correctly interpreting our statements in their absence. Many people on the spectrum have invested a considerable amount of effort into learning this language only to come across in a manner completely unlike what we'd intended.

Do these problems obstruct meaningful discussion of autism-related issues? If so, how can we go about opening a meaningful dialogue? Is there any way that we can keep important points from being "lost in translation"? Let's hear from some voices in our community that can help us frame this conversation.

While not everything in this remains true (specifically, I'm no longer a student at Nova), the question is no less important now then it was a year ago.