PDF version of: This Article (52 KB) | This Issue (842 KB)

		Proceedings of the 19th annual conference on prevention, research, and treatment of problem gambling. June 23–25, 2005, in New Orleans, Louisiana. National Council on Problem Gambling, Washington, DC. Session III: Critical issues in treatment Improving the measurement of outcomes in problem gambling and treatment research Presenter: Tony Toneatto (Introduction.) Ken Winters: It's apropos that our next speaker is going to try to give us a bit of an overview of outcomes, in general, with various treatment approaches. Dr. Tony Toneatto comes to us from Toronto, Canada, from the Centre for Addiction and Mental Health, and has been a leading researcher and publisher in this area of summarizing outcome evaluation research. Tony Toneatto: But my title is misleading, it should say: "Improving the measurement of outcomes in problem gambling and treatment research." My focus is going to be on something I hope that all of you will be able to take back with you, rather than being a talk directed only at research. But as a context for that, let me just tell you that a couple of years ago, Rob Ladouceur and I were embarking on the review of literature, which he alluded to earlier, and in doing so we identified some of the better studies. Actually, most of them were done by people like Ladouceur, Dave Hodgins, and Alex Blaszczynski. They made up most of the studies in the 12 or so that we identified. In doing this we came up with an analysis of what needed to be done if the treatment field were to progress. And it really revolved around assessment. A lot of the recommendations were around measuring, assessing, what you do, how you do it, when you do it, why you do it. From that, about a year later, last May, a bunch of us met in Banff, Alberta, including people like Alex, David, Robert, Nancy Petry, Marc Potenza, Michael Walker, and several others, to further discuss the issue of measurement assessment in gambling research. The purpose of this was to advance the field and to not have to wait too long before we get information that is going to be useful and effective and guide our treatment practices. The downside of that is that most of the research that we were discussing and doing was efficacy research, where highly controlled research excludes a lot of populations and the end results don't generalize very well. That's not actually the best kind of research, probably, for this audience. It's really about effectiveness research and how does this research work in the real world in the clinics or settings where you work? So as I'm speaking about my material today, it's really meant to be applied in any setting that you work in, whether it's residential or outpatient, whether it's a brief or long-term program, whether it's mandated or nonmandated programs, whether it's a case study—one single subject—versus a group you're involved in. I'd like to make the issues I'll be raising applicable not only to good efficacy research, but also to any kind of clinical intervention, because the assessment piece that underlies all that, the issues that underlie a good assessment, cut across any type of research and any type of intervention. That's the context for my material. It's also ideal for a program that wants to do any kind of qualitative evaluation, program evaluation, quality assurance—not just for any kind of controlled research. Even with the best studies, there's a wide variety of assessment methodologies. People do things very differently. Surprisingly, some people don't do much with assessment. It's not unusual to find many good studies where the assessment is lacking or too poor to allow any kind of meaningful statement to be made. So we realize that with that kind of array, with that poor quality of assessment methodology, it would be very difficult to really compare studies and to make good conclusions. We also realize that by not improving our assessment procedure, we were going to retard the progress towards developing effective treatments. And as I mentioned in the article I wrote, we met in Banff to improve on that. Out of that came the following principles that apply in any setting you're in: whether treatment, research, or not. First, we need to measure problem gambling behavior. Believe it or not, not all studies measure gambling behavior. They may often measure attitude towards gambling or they measure it in a very overall simplistic sense. We would encourage people to measure the frequency of gambling behaviors. That is, when, how often, in what context, over a period of time—30 days, 60 days, 90 days—to allow you to compare what happens after the person's gone through treatment, in any context of the received treatment. Without that information, it's really hard to know and hard to actually argue that your program is effective or that you're achieving what you want to achieve. Second, in addition to the frequency of the behavior, the financial aspects of gambling are important to measure. An issue that's been very difficult for us to get around is how to best measure net financial loss. We don't have the final answer on that, but one way to do it—which I'm going to show here—is that you measure the money that you have available to gamble at the beginning of the session, including any kind of withdrawals or borrowings you make during that session, minus the money left at the end of the session. A third aspect of measuring problem gambling behavior is to describe what kind of gambling you're actually treating. It's amazing how many studies don't say what kind of gamblers are being studied or treated. That's like reporting a study on addiction including, let's say, smokers. The word "addiction" won't tell you whether it's about smokers or about cocaine users. You need to actually specify the kind of gambling. In our case, it would be specifying whether it's slot machine gamblers or racetrack gamblers or lottery players. Without that critical information, it's really hard to compare studies that may have an unknown mix of subjects. But having fairly detailed measures of frequency, the amount of money lost, and so on, you're then able to present the findings in a variety of ways that are not there if you measure outcomes just in terms of abstinence, nonabstinence, using a lot, using a little. The fourth aspect that needs to be measured—this is obvious, but we're not yet ready to do this study because it's invasive—is to measure problem gambling related consequences. This, as Dr. Ladouceur alluded to earlier, is the idea of measuring the pure gambling psychopathology, the phenomenon of gambling, the core of gambling pathology, which may be impaired control. The reason why it's important to measure that, versus consequences, is that most consequences of gambling treatment, which involves effects on the person's psychosocial functioning, may take a long time to take effect. You may as a clinician get the client to be abstinent. But it doesn't mean that they're going to be happy or that their life will be any better. They may have a lot of resolution of problems for a long time—divorces, loss of jobs, financial problems. In terms of evaluating your intervention you want to be able to show that, "Yes, I helped the person with their gambling behavior," but in a study that may not translate into better functioning. It may take years before they recover their life, and you don't want to take responsibility for that, necessarily. It also allows you to provide the additional counseling and resources to deal with the consequences that come from the gambling, besides just the gambling itself, and there are many ways to do that. The fifth thing that we thought was important to measure, when you're measuring gambling, was how much time people spend thinking or preoccupied about gambling. I know it's a symptom in DSM, but we often don't think a lot about that aspect of it, their cognitive thinking about gambling. But many people who are caught in gambling pathology spend a lot of time just thinking about gambling, not just in an obsessive way, whether you want to gamble, but thinking about the consequences of gambling, how to improve their gambling, systems of gambling, and recovering from gambling-related consequences, "How will I lie to my wife? How will I deal with this issue or that issue?" and there's a lot of mental activity that will go with the gambling. Getting a sense of how much of that is going on is actually a good measure of the impact of gambling on someone's life. And that's a variable that's often not easily measured or measured at all. Dr. Ladouceur also mentioned that we need to measure why we think people are getting better. All of us will have our pet theories about why our clients get better—education about cravings, depression, medication, self-esteem, impulsivity—but how do we know that unless we measure it? We can easily get into useless disagreements between treatments and treatment studies that could be resolved if we measured why we think someone gets better. In addition to measuring just gambling behavior, the sixth issue is measuring the important predictors or constructs that we think explain it. So if you think that impulsivity is a core mediating factor, measure impulsivity before and after so that you can say, "Well, I measured that; it did go down" or "it did go up" or whatever. Without that kind of information about the process of change, we often don't know why our clients get better. It may be for all the nontherapeutic factors that are often present in treatment, like motivation, group social support, the role of the therapist, and things that are not part of the treatment, per se, but are part of the therapeutic environment. So to convince your program head or government funding agency or anyone else that your treatment is effective, you need to have some measure of the key things that you think are effective. The example I give is that it would be strange to say that cognitive therapy is effective when there's no evidence that cognizance was modified. And so, researchers like Dr. Ladouceur include measures of cognitive functioning in order to see if that occurs so a link can be made. The seventh issue involves measuring what happens to your clients as they go through therapy. Whether with research or your own clinic it's misleading to include the people who don't attend assessment, who don't attend treatment, in your success rate. It helps to know exactly what happens to these people. In fact, where I work, we're starting to embark on actually calling up people who drop out, or who don't even come for treatment, to find out what happened to them. We often assume they're doing poorly; often that's not the case. They may be doing quite well. The assessment may have been therapeutic and we are able to use those data. But otherwise, without that information, our results are misleading, so we need to know how many people are seeking treatment—if you're doing it in a clinic or a clinical setting—how many attend the assessment, and, out of those, how many begin treatment, how many complete treatment, and how much of the treatment they complete, and how many of the clients are followed up. That way, you can actually begin to get a sense of how strong your program is and how meaningful the results might be. Then following treatment we need to measure longer-term outcomes, other than end-of-treatment and posttreatment. Almost all clients will get better with almost any treatment. That's pretty well clear. With the effort of coming, the motivation of being there, and the attention they're going to get, rarely will people get worse while they're in treatment. But the key thing is to ask, "Do they retain those gains in the short term, in the medium term—about a year later—and the long term?" And that's when you can begin to make links between your program, your intervention, your study, and a lasting change in a client. And the last point, my final point: if possible—and it's not always possible—get others that would know about the client's gambling to corroborate or validate or provide some information as a way of feeling more comfortable about the reports that clients give. Know that for many gamblers it's a great difficulty to be honest, especially if there's something riding on it—some other secondary gain—and when that is possible, include that as a way of having confidence in the results that you collect. Ken Winters: Let’s change gears a little. During break, Jeanette asked two questions. One is for Tony. Can you give us clinicians some examples of assessment tools that we could use if we want to do pre–post analysis? Tony Toneatto: That brings to mind a project that I'm involved in now in Ontario where we're evaluating a residential treatment program in a pilot project and, in doing that, I have developed a core set of questioner's instruments that are going to be administered before treatment, after treatment, and for the follow-up. That core set of instruments has to be pretty short, fairly easy to use, self-administered. If it takes too long, people aren't going to use it, so it takes about 20 minutes, half an hour, to do. It includes measures on gambling behavior, severity of gambling, DSM criteria, high-risk situations, cognitive distortions, gambling-related consequences, quality of life—which we think is important to put the gambling in context of—treatment goals, psychiatric histories, substance abuse histories, and treatment history. In addition, we ask questions around the overall program, to rate the program evaluation piece, and then we have a specific list of questions about the actual treatment components. This goes back to a comment, actually, Dr. Ladouceur mentioned about treatment manuals. Treatment manuals are often developed for the treatment studies that we have been describing. But in actual practice, most programs don't use anything that comes close to one. But what you should be able to do is to actually describe what you do. So that if somebody asks, "How did you get those results in your program?" you can say, "Well, we did this, this, this, this, and this." So what we ask the people in our program to do is to come up with about 15 key interventions that they will be administering in their residential program and we convert that into a questionnaire that the clients will get at the end of treatment to find out from the client whether they were effective, whether they were desirable, and how they felt about it. That way the program gets direct feedback from the clients as to the efficacy of the program and then they can develop the program further. I can make available to you the set of questionnaires that I just described. My e-mail is up there. It's Tony_Toneatto@camh.net. A subset of those questionnaires are then re-administered at the end of treatment, a smaller set, and then at any follow-up that one would desire. In our case, it's three months, six months, and twelve months. That way we're able to quickly and validly get information that allows each program to find out not only if what they did works, but also how to improve it because clients will be giving individual feedback. This can be used in any kind of new program being developed. You can use it just to see whether your program is working well. You can use it to see whether changing your program will make it better. You can even use it in individual cases you're seeing and just monitor the client's experience pre- and postintervention. You can use it if you're trying a new type of therapy, and so on. So that core package is something that we developed and I'd be happy to share with you. None of it is something that has to be secret or bought. It would definitely be available that way. And, along that same question, it addresses the interesting acronym that Joanna was saying, the YCTs and the YCJTs. You can just tell. Now you don't have to say that. You can say, "No, I know they got better because these cores changed and these cores didn't change and I know why they got better because they said these interventions helped and these didn't." And that, then, is good for morale for therapists, it's good for advancing a program, and if you're going to be training other people in your particular programmatic approach, you can now say what you're doing and how effective it is. So it has many, many functions that go beyond just simply outcomes, to also enhance programs. [End of session.] For correspondence: Tony_Toneatto@camh.net

		issue 15 — december 2005