Leaving Out The Important Questions On Autism Research
This month, the Simons Foundation’s SPARK had autism researcher J. Kiely Law, M.D., MPH answer questions about autism research, and it should come as no surprise that I have something to say about it, especially since the very first question is about the phrase “evidence-based”.
Q: The terms “evidence-based medicine and treatment” or “science-based medicine” are used a lot. What do they mean?
A: It means that an idea or treatment has been scientifically tested and, based on those test results, the idea is valid. In the case of a treatment, it means that the treatment works. The more times the test is repeated with the same result, the more evidence there is that the idea or treatment is valid. For example, cars go through a series of tests to make sure they are safe, and medical science does the same thing. Science uses a specific way of testing an idea, called a hypothesis. This way of testing is called the scientific method, and its purpose is to remove anything from the test that could bias the results or make the results incorrect.
You can’t answer a question about “evidence-based medicine and treatment” in autism research without once mentioning Applied Behavior Analysis, which of course is the context in which that phrase most often appears, and yet SPARK and Law do exactly that: not mention it once.
That’s a problem, because ABA relies heavily upon the fiction that it’s the “only” evidence-based treatment for autism. Not addressing ABA specifically is a disservice. Arguably, “evidence-based” is a marketing term more than anything else, and researchers too often focus on a narrow set of questions to the exclusion of the more important ones such as “any consideration of the moral or ethical value of the interventions themselves” or “is the intervention being studied in fact not even in the best interests of the autistic person as an autistic person”.
The shorter: in the context of autism research, “evidence-based” usually is said of behaviorist treatments which seek to render an autistic child more outwardly neurotypical, regardless of what it’s doing to the child within.
Further, in answering a question about the idea that research is meant to be “objective and unbiased”, Law and SPARK say the following:
In studies that rely on people answering surveys, researchers have to design the questions so that they don’t lead the person to answer in a certain way. For example, if a question asks a person how much they “liked” a treatment, that’s a leading question that assumes they liked it. It’s better to ask participants if their impression of the treatment was negative, neutral, or positive. That removes the bias that the treatment should be liked.
That’s good, so far as it goes, but it doesn’t go far enough. Just this week I once again encountered an autism-related research survey which asked participants to rank to what degree certain experiences were true for them. For instance, do “others” find you or your behavior peculiar. The obvious counter here is: which “others”?
Naturally, we can tell what they’re getting at. They mean normal people. What do normal people think of you? If you surround yourself with other neurodivergent people, or with a broad range of accepting and accommodating people, your answer to this question might very well be dramatically different than if you’re surrounded almost exclusively by neurotypical people who don’t deviate from normative expectations and expect everyone around them to do the same.
These sorts of nuances and distinctions are almost never captured by research surveys, and of course I’ve discussed it before. Two years ago I even wrote about a research paper that looked into “autistic people’s experiences of decision making and research questionnaires”. It found that “questions needed context, often questions themselves were unclear and difficult to understand, […] and how measures could have questionable validity for autistic people”.
I’ve taken to using any space provided in research surveys for additional comments or feedback to relate exactly these sorts of flaws and concerns.
Superficially, it’s great that SPARK wanted to publish something about questions people have about autism research. Unfortunately for all concerned, I’m not sure this particular something doesn’t do more harm than good. At the very least, it muddles more than it clarifies what I’d consider the important questions, mostly by the sin of omission.