OVERVIEW

AGENDA

MATERIALS AND PRODUCTS / PRESENTATION SLIDES

HOW TO REGISTER

REGISTRATION FORM

HOW TO PARTICIPATE ON AUDIO CONFERENCE

Safety Net Data Collection Strategies

The Second in a Series of Three Free Web-assisted Audio Conferences
for State and Local Health Officials

September 24, 2003

TRANSCRIPT

Cindy DiBiasi: Good afternoon. Welcome to Safety Net Data Collection Strategies. This is the second event in a series of three web-assisted audio conferences on monitoring the healthcare safety nets. These events are designed for state and local health officials. The series is sponsored by the U.S. Department of Health and Human Services Agency for Healthcare Research and Quality, often referred to by the acronym AHRQ or AHRQ and the Health Resources and Services Administration or HRSA. My name is Cindy DiBiasi and I will be your moderator for today’s session.

In 2000, the Institute of Medicine released a report describing the healthcare safety net as “in tact but in danger.” The safety net, as you know, is the nation’s system of providing healthcare to low income and other vulnerable populations. In particular, the report emphasizes the precarious financial situation of many institutions that provide care to Medicaid, uninsured, and other vulnerable patients. It also examines the changing financial, economic and social environment in which these institutions operate and it looks at the highly localized patchwork structure of the safety net. One of the five key recommendations in the report focused on the need for data systems and measures to assess the performance of the safety net and health outcomes of vulnerable population.

In response, AHRQ and HRSA are leading a joint safety net monitoring initiative. This initiative involves a three-part strategy focusing on both safety net providers and the populations they serve. As a result, they resolved to create two data books that describe baseline information on a wide variety of local safety nets, developed a tool kit for state and local policymakers, planners and analysts to assist them in monitoring the status of their local safety nets, and identify the data elements that would be needed to effectively monitor the capacity and performance of local safety nets. Information related to the AHRQ and HRSA initiative is available on AHRQ’s Web site at www.ahrq.gov/data/safetynet.

As we talk about the safety net, it is important to make sure that there is a common understanding regarding what it encompasses. The healthcare safety net consists of a wide variety of providers delivering care to low income and other vulnerable populations. These include the uninsured and those covered by Medicaid. Many of these providers have either a legal mandate or an explicit policy to provide services regardless of a patient’s ability to pay. Major safety net providers include public hospitals and community health centers, as well as teaching and community hospitals, private physicians, and other providers who deliver a substantial amount of care to these populations.

In addition to today’s event, one more call will be conducted as part of this series. The next call, scheduled for tomorrow, Thursday, September 25th from 2-3:30 PM Eastern Time, will focus on how data will be used to tell the safety net story. I will tell you more about this call later in the broadcast, but right now let’s turn to today’s call.

Yesterday we discussed the new data books and the range of measures they provide for monitoring the status of local safety nets and the populations they serve. Today we will examine three chapters that are included in the forthcoming book entitled Monitoring the Healthcare Safety Nets, Book III, Tools for Monitoring the Healthcare Safety Nets. It will be available later this fall. We will tell our listeners how to get this new book at the end of today’s discussion.

Similar to the data book, the tool kit is designed to help policy analysts and planners at the state and local levels assess the performance and needs of their local safety nets. Chapters included in this book are written by experts in the field covering a wide variety of topics and today we will examine three reports focused on data collection strategies.

Let me begin by introducing today’s panelists. In the studio with me I have Lynn Blewett, assistant professor at the University of Minnesota, School of Public Health. Joel Cantor, director for the Center for State Health Policy of Rutgers University. Timothy Clouse, agricultural economist and statistician at the U.S. Department of Agriculture and Vickie Gates, senior consultant to the State Coverage Initiatives Program. Also here with us in the studio is Robin Weinick, senior research scientist and senior advisor on safety nets and low-income populations at AHRQ. As the lead of the AHRQ Safety Net Monitoring Initiative, she will be with us today in the studio and join us during the question and answer session. Welcome everyone.

Before we begin our discussion, I would like to tell the audience a bit about the format of this web-assisted audio conference. First we will talk with our three panelists, then open up the lines to take your questions. We will give instructions on how to send your questions to us later in the program. In the meantime, if you experience any web-related technical difficulties during this event, please click the “help” function in your window to trouble-shoot your web connection. If it appears that the slides are not advancing, you may need to restart your browser and log on again. If you are on the phone, dial “*0” to be connected to technical assistance. Also, if you have difficulty with the audio stream or if you experience an uncomfortable lag time between the streamed audio and slide presentation, we encourage you to access the audio by your phone. The number is 1-888-469-5316. This is the same number to call to ask questions when we get to the question and answer portion of the program.

Now I think we are ready to tackle today’s topic, Safety Net Data Collection Strategies, and Lynn Blewett, let’s begin with you, assistant professor at the University of Minnesota School of Public Health. Lynn and her colleagues from the State Health Access Data Assistance Center at the University of Minnesota prepared a report for the safety net tool kit. It is entitled, Estimating the Size of the Uninsured and Other Vulnerable Populations in a Local Area. Lynn, what are the mechanisms to measure levels of insurance at the state and local levels?

Lynn Blewett: Well, the first mechanism is a survey tool to get a direct estimate of uninsured. You can either do that through a household survey where you ask people directly, do they have health insurance coverage or not or often researchers look at employer surveys to look at whether employers offer insurance and whether their employees take up insurance. Joel is going to talk a little bit about in the next presentation about how to do a local survey. The other thing you can do is develop either a proxy measure for uninsurance or a model-based estimate for uninsurance.

Cindy DiBiasi: Are there national data sources that provide state-level estimates of uninsurance rates?

Lynn Blewett: Yes, it is very important for local researchers and analysts to understand that there are national survey data that provide mostly state-level estimates, but some local estimates of uninsurance. Those are the current population surveys conducted by the Census Bureau. The Medical Expenditure Panel Survey, which is an employer survey, conducted by the Agency for Healthcare Research and Quality and there is a recent survey that is just being released this fall, the State and Local Area Integrated Telephone Survey or SLAITS that is conducted by the National Center for Health Statistics.

It is important to know that the current population survey does have estimates of uninsurance for the larger counties and metropolitan areas and you may be able to get information from the current population surveys.

Cindy DiBiasi: What about state surveys?

Lynn Blewett: Well, there are, because there is a lack of estimates at the local level, states have developed their own surveys to estimate health insurance coverage. We have documented at least 37 states who conduct their own household surveys. Many of these surveys are funded by the Health Resources Services Administration’s State Planning Grant Program. The state surveys typically have larger sample size and targeted populations so you can actually get estimates at the regional and local county levels. So it is important that if your state has a survey, you should contact your state survey folks to see if they have estimates of your county or region.

Cindy DiBiasi: Now are there national surveys that provide information on health insurance coverage at the local level?

Lynn Blewett: There are a couple privately funded surveys that analysts should be aware of. There is the National Survey of American Families that is conducted by the Urban Institute. They have comprehensive survey data on health insurance coverage and income and poverty for 13 states. Those data are available in public use files available to state analysts.

The Community Tracking Study, conducted by the Center for Health System Change, also have comprehensive survey data on providers, households, and other survey information. Again, only for 12 communities. But if you are one of those communities, you ought to be aware that that data is available and the lists of those states and communities are in the paper, the Took Kit, Book III, which will be published later this fall.

Cindy DiBiasi: When direct measures aren’t available, how can state and local analysts develop community-level estimates of uninsurance rates?

Lynn Blewett: Well, there are a couple of things you can do. One is to use an available measure to serve as a proxy for health insurance coverage. So while you may not have health insurance coverage, there may be a number of different variables that you could look at in your local area. For example, you could look at the self-paid variable from your hospital administrative records to estimate local areas of uninsurance. That self-paid variable is available in the data book that was talked about in yesterday’s phone call. That would be used as a proxy for health insurance coverage when you don’t have access to direct measures.

Another way is to develop a model-based approach, which is basically a statistical model that would predict health insurance coverage using different variables that are correlated with health insurance coverage. An example of this is looking at the state unemployment rate and levels of uninsurance at the state level and then applying this at the local level. So again, you use available data that you have to develop an estimate of uninsurance at the local levels.

Cindy DiBiasi: Let’s talk about the strengths and weaknesses of the different approaches that you have mentioned such as estimating levels of uninsurance at the local level.

Lynn Blewett: Well, the best way to get a good, precise estimate of health insurance coverage is to conduct a survey with a large enough sample size. The downside of that is that it is very costly to do and what you really want to do is at least two points in time so you have information to look at change over time. Many local areas cannot afford to do a direct estimate or a household survey at the local level. Then you need to go and look at what available data you do have. The proxy measure is a pretty fairly easy way to get access to that information. It is low cost, data is available, but really you are not measuring uninsurance, you are looking at a proxy for uninsurance. So there may be some (unclear) in terms of what you are measuring.

The model-based approach has greater predictive accuracy than the proxy measure that is quite complex and you are likely will need some statistician or partner with your local university to help you develop a model that can be used. It could incur some costs in terms of getting the expertise needed to develop a good model-based estimate.

Cindy DiBiasi: Are there researchers working on these models?

Lynn Blewett: Yes, there are a couple of national federal agencies that are spending quite a few resources and expertise trying to develop local, basically county level estimates of uninsurance. The Census Bureau is working with current population surveys that currently provide state-level estimates and they are trying to develop a model-based estimate to look at county-level health insurance coverage estimates.

The Agency for Healthcare Research and Quality is also working with their employer survey, which I mentioned earlier, the MEPS employer survey, to look at local estimates of employer offer and take-up rates. Those will likely be available within the next year.

There are a number of states that have also developed models using their state survey data, including the State of Minnesota who has developed model-based estimates at the county level. The State of California has also developed estimates and there are other states that are working on their own model-based estimates.

Cindy DiBiasi: So keeping all of this in mind, what is your advice to people who are interested in trying to estimate levels of uninsurance?

Lynn Blewett: Well, my advice would be to just be knowledgeable and aware of the different data sources that are available for your communities. To use existing resources and data and build on your state and local survey activities. Many states have surveys and if you can add additional samples for your local area, you can get a lot of information just by piggybacking off ongoing activities.

It is also important if you are going to use a proxy or a model-based estimate that you use multiple approaches to look at the different estimates and try to benchmark against either the state or the national survey data. Also to be flexible as new data become available. The Federal Government is always looking at new ways to estimate levels of uninsurance. There is always new information available. You need to be aware of those activities and be flexible in understanding your local markets.

Cindy DiBiasi: OK Lynn, thank you. We will be back to you during the question and answer period. Now I would like to turn to Joel Cantor from the Center for State Health Policy at Rutgers University. Joel is the author of a chapter for the Safety Net Tool Kit entitled Local Data Collection Strategies for Safety Net Assessments. Joel, how should a local group decide whether to sponsor its own safety net assessment survey?

Joel Cantor: I am going to start really where Lynn left off. That is first and foremost to look at what is already available. It is very costly and very difficult to reinvent the wheel in this case. So if there are data available for your local level, go ahead and use it. Also look into the proxy method. Look into the small-area estimation methods. If those don’t meet your needs, then you might need to do your own local survey. Local surveys can also be a useful community-organizing tool for local coalitions. They are useful for engaging stakeholders in their design and of course in going through the findings. Local surveys can also be extraordinarily useful if you have locale-specific questions to ask. If you want to know, for example, about the performance of individual providers in your community, the only way you are going to be able to ask those questions is to do your own survey.

Next you have to ask yourself, can the population in the survey actually report what I want to know? Household members of general populations, for instance, can’t report for example the specialty of the physicians they see. But a survey of providers might be useful in understanding the availability of specialty care. Likewise, people can’t necessarily report characteristics of, detailed characteristics of their employers. So in some instances employer surveys are necessary.

Finally, and perhaps most importantly, look at the resources you have available. Do you have the time to design a survey? Do you have the resources, the dollars available to collect the data, analyze the data and prepare the findings?

Cindy DiBiasi: Let’s say my local access coalition decides to sponsor a survey. What is the first step?

Joel Cantor: The first step is to be absolutely clear about what your objectives are. You might want to focus on primary care. You might want to focus on specialty care. Satisfaction with services. There is a whole range of possible goals for a survey engaged with the local stakeholders. You could go through a consensus process to find what your objectives are. Those objectives should be written down and in essence serve as your manifesto for moving forward as you design and implement the survey.

Cindy DiBiasi: How can the survey objectives be translated into a questionnaire?

Joel Cantor: My first advice here is not to reinvent the wheel again. Use existing questions. There are ample surveys that have been developed that address healthcare access questions. Lynn mentioned several in her presentation a moment ago. Others, the Medical Expenditure Panel Survey, sponsored by AHRQ, are an excellent source of questions not only for general populations but also employers. There are others as well. I would also refer you to the SHADAC Web site which has a wonderful bank of questions that can be adapted for local surveys.

But again, my first advice here is don’t reinvent. Use existing resources and refer back to your manifesto, to your survey objectives, in selecting your questions.

Cindy DiBiasi: What kinds of questions do safety net access surveys typically ask?

Joel Cantor: Typically, they ask about health coverage, about health status, whether a specific condition such as diabetes or asthma or general perceived health status, various access utilization, usual source of care, attitudes and beliefs can be important. Characteristics of employers, for example whether or not they offer coverage. The size of an employer can be important. Wage levels, socioeconomic status, education, demographics, languages spoken and so forth.

Again, don’t forget to refer back to your objectives and also don’t forget to tailor your questions to the local circumstances, the local populations and the local questions you have, the local objectives you have for your survey.

Cindy DiBiasi: So besides the questionnaire, what other things do survey sponsors need to consider?

Joel Cantor: There are a myriad considerations in designing a survey. The sample size and the sampling strategy, issues of confidentiality, and how what promises will be made, and how you will follow through on those promises. Response rate issues, follow-up strategies. It is never enough to invite people to respond once. Unfortunately, people are hurried and busy and it takes a lot of effort, often multiple contacts, and sometimes a dozen contacts to get even a 50% response rate in some instances.

Interview mode, that is will it be a phone survey, in-person interviewing, through the mail are also questions. You will need to know whether the households in the area you would like to survey have a low telephone coverage rate. The Census Bureau has information on the proportion of households without phones. Increasingly, surveys of this kind are using mixed modes. They do a lot of phone interviews, but then they supplement them with in-person interviews in census tracts with low phone coverage. There are also some other techniques available for adjusting for a lack of phones in a household.

The next issue is selecting and training interviewers. Interviewing is a complicated skill that requires training and experienced interviewers are really invaluable. Then once you have your data, the story is not over. There are questions of data management, data editing, preparing analysis files, doing the statistical analysis. Even if it is not complicated, fancy regression analysis like Lynn was talking about, it is worth really thinking through what the analysis plan is and having the skills and talents to conduct the analysis.

Finally and absolutely not least, is reporting of findings. Too many times I have seen local groups conduct a survey, put a huge amount of resources and energy into collecting the data, then come to the end of the grant period or whatever it might be and the findings kind of trickle out. Put a huge amount of energy into reporting the findings. Briefings, papers, whatever means you want to use takes a lot of energy.

Cindy DiBiasi: OK, Joel, if we haven’t scared everybody off by now (Laughs.).

Joel Cantor: That is my goal. (Laughs.)

Cindy DiBiasi: It sounds pretty complex. What are the biggest kinds of problems that novice survey sponsors should watch for?

Joel Cantor: Sure. First of all, remember why you are doing it. Work closely with the community that you are conducting the surveys for. Work with your stakeholders. Work with the population. Keep your eye on those objectives. Keep in mind why you are doing it, from beginning to end. It is very easy to get lost in the details.

The second pitfall is running out of money, time or just plain old running out of steam. As I said, very often, too often, survey sponsors get to the end and don’t have the energy or the time to really get their findings out. So always really think about spending the time at the back end. But also think about spending the time in the development at the front end. Designing questionnaires, even if you are following my advice and drawing on existing questions takes a lot of time.

Next, pay attention to data quality, sampling problems. There is an instance in a group that I worked with in the past where they were really eager to get some quick data on the health status of the community so they used the local sanitarium and gave them clip boards and sent them out to the local malls to do interviews. Well none of them spoke Spanish and there were a lot of folks in this community who spoke Spanish. It wasn’t a scientific sampling strategy so they really got quite a biased view of healthy kids visiting the mall.

Also, the next pitfall is losing your focus. Again, write down those objectives and stick to them. Then lastly, as I have said, really save enough time and energy for the analysis and reporting.

Cindy DiBiasi: OK, Joel, we will come back to you during the question and answer. Let’s now turn to Timothy Clouse, an agricultural economist and statistician at the U.S. Department of Agriculture. Tim wrote the chapter Assessing Safety Net Provider Financial Help, A Simple Measurement Tool, in conjunction with two colleagues from the Health Resources and Services Administration. Tim, how do you assess what is good financial help for a community-based healthcare organization?

Timothy Clouse: Well, we ran into this problem when I was with HRSA in the southeast regional office. We have 150-odd community-funded health centers. Unfortunately, there is no way to say is this a good clinic or a bad clinic? It was kind of a subjective process. It was a matter of “Well, I know it when I see it,” said in another context. Well, the problem arises in that it depends on who the observer is. So what we had to do was come up with a way of coming up with a sort of objective, verifiable, independent mechanism for saying this is or is not good financial help.

Cindy DiBiasi: So how does the measurement tool that you developed actually work?

Timothy Clouse: Well, in essence we decided that we wanted everybody to be above average. It doesn’t make a whole lot of sense at the outset, but work with me on this. What we had initially was a sample, national sample of community-based health centers and we decided as the (unclear) there is no objective way of saying given this set of financial measurements, this is good, this is bad, this is indifferent. So instead what we went with was a sort of a relative ranking system. We knew we couldn’t establish what they were objectively, but we did know that well, this clinic is probably worse than this other one. That is what we went with. We went with a sample. We selected a series of variables, financial measures that we thought were important and were closely related to them, and were not closely related to one another, actually measuring different things.

Cindy DiBiasi: What kind of data is required for the Financial Risk Assessment Worksheet?

Timothy Clouse: Well, it uses what should be readily available financial data, which is things that show up in a community-based healthcare organization’s financial data things like a CPA report. So it would have things like current assets, current liabilities, cash on hand, things of that nature. When you take that and you plug it into this simple worksheet, well, relatively simple. It works sort of like the 1040 for your IRS paperwork. It gives you numbers and...

Cindy DiBiasi: Let’s talk exactly about what that worksheet looks like.

Timothy Clouse: OK. Here is part of it. This is both in the document and there is an XLS version that you can download off the Internet, the HRQ Web site. This is like the first page of it, which has got the year. One of the crucial things is, instructions is if you don’t have it, assume, we allow you to assume the best. However, one important thing which doesn’t show up here which is on the worksheet and in the document. This is not a substitute for a professional judgment. This is not a cookbook. All this does is give you an indication of where the issues are.

Cindy DiBiasi: Now let’s talk about how this tool can identify problems.

Timothy Clouse: OK. There are a couple of ways it works. As you can see on the spreadsheet here, it shows some of the measures we have got to fill those out. When you run through the spreadsheet, it gives you an indication of where the problem areas are. We have actually used this. For example, there is one case in the southeast where we ran the model and looked at it. Turned out that this particular organization may have had some problems. We did some additional work with it, talked to them, and helped them out a little bit. It turned out that the issue was that their fee structure hadn’t changed in several years. So we took that and said OK, based on the model and the additional work we have done, you need to look at what your fee structure is. They went back in and said, “Oh, well we haven’t done anything with this for a while.” So they revised it and the result was their financial health improved dramatically after that.

Cindy DiBiasi: So you could adopt it to local use.

Timothy Clouse: Yes, you can. That is one thing, one of the important things about it is once again, and it is situational. It is not a cookbook. If you know, based on what is going on in your community, that particular measure is not as important or more important than what you actually think it is, then you can adjust it. You can adjust the weights, as in the relative importance of each measure. If you think, for example, we have got the depreciation in there and we don’t think it is very important, well, if the local community thinks that the age of the building is really important, they can make that, they can increase the importance of that measure.

Cindy DiBiasi: Change the weighting of it.

Timothy Clouse: Yes.

Cindy DiBiasi: How else can the worksheet be used?

Timothy Clouse: Well, actually you can also kind of use it backwards in the sense that you can use it to set up the community-based healthcare organization using these measures to say, “Well, if we want this organization to be viable”, what you can do is say, “Well, we need to be able to pay off our bills within this particular time period. We need to collect from our insurance company within this particular time period. We need to have this dollar amount available on hand stashed to pay off any unexpected bills.” You can also use it to, once you have got the organization up and running, the plan on how to keep it going over the future.

Cindy DiBiasi: OK. Timothy, we will be back to you. In a moment we are going to open up the discussion for questions from our listening audience. But first let’s go to Vickie Gates. As a senior consultant for the State Coverage Initiatives Program funded by the Robert Wood Johnson Foundation, Vickie has seen various state and local safety net initiatives first hand and she has also worked extensively on safety net issues during her tenure as a state official in Oregon. Vickie, we heard lots of discussion today about data and surveys. Based on your years of experience, do these data and survey efforts really make a difference?

Vickie Gates: Well, I would never argue that there are not a lot of decisions made without the benefit of data, but when it comes to big investments today, I think increasingly people are not only asking for data, but they are also asking about the quality of that data. One of the things particularly as we have watched the HRSA State Planning Grant states over the last three years, Lynn, Joel and I all have seen situations where policymakers have literally changed the way they understood a problem and changed their position on an issue because of the data that was made available.

Now I really want to reiterate a point that both speakers have made. Data is expensive. One of the values I think of this work that we have seen today is that it is realistic. It really puts you with what is already a base to the information and if you are thinking about a survey, read the two papers from Joel and from Lynn. It is going to help you make sure that in fact your results are not going to be disappointing, that you are going to be clear about the objectives. You are going to have realistic objectives in terms of what data actually is going to be able to do for you. I think it is an incredibly valuable starting point.

The last point that I think I really want to reiterate from the speakers is this kind of what I would call “the softer side of data.” It is that ability to use these processes to bring consensus to a situation, to put everybody on the same diagnostic level and in many cases actually put an issue on the policy agenda.

Cindy DiBiasi: Now you (unclear) as we said and most recently have collaborated with many states on solutions for the uninsured. Is a tool for assessing safety net provider health really of interest to state policymakers?

Vickie Gates: I think the answer to that is yes. I think it actually benefits the state policymakers for quite a period of time. There is no question. All states agree there is a need for the safety net. The difficulty is understanding what its actual status is and then trying to figure out once you have some way of knowing what is going on with that system, what should the role of the state be? How should the state be responding? In most cases, states are dealing with (unclear) when it comes to the system.

So what this may offer is a set of benchmarks in a way of looking at financial health. States do many things that have a major influence on the health of the safety net and their financial viability. They need a way to understand the possible implications. They need a way to understand what follows, what may actually may happen once they have made a change.

I think it is also this ability to benchmark for the clinics themselves to look across the state, to have that consistency. For policymakers at all levels, all of whom are concerned. The clinics themselves, the local government who is often a major funder, and say this could be a very valuable piece of the picture.

Cindy DiBiasi: What other opportunities do these new tools create for state and local policymakers?

Vickie Gates: Well, I think that at this time we may be getting enough of a picture to say we can talk about both sides of the equation, that we can talk about supply side issues and we can talk about demand issues and we can have good data and an ability to have some common understanding and benchmarks on both of them. As that ability to produce a credible picture that people can agree on, I think is one of the first steps for communities and states to find an effective way to work together in preserving access and expanding access to low income and uninsureds.

There is another point in the discussion that I also think is worth reiterating. Point in time investments are not the whole story. We really have to find more ways of monitoring over a period of time on both these issues. That is where a lot of the policy (unclear) is going to come.

The safety net has been characterized as fragile. I think that if you look today you will see states making really significant decisions about coverage. Trying to understand what the implication is for uninsured and for safety nets. This ability to turn to these tool kits I think is going to make it easier for states to feel at least knowledgeable about the social impact of these really difficult decisions in very hard financial times.

Cindy DiBiasi: OK, Vickie, thank you. We will be back to you.

In a moment we are going to open up the discussion to questions from our listening audience. There are two ways you can send in your questions. We encourage you to ask your question by phone. If you are already listening by phone, press “*1” to indicate that you have a question. If you are listening through your computer and want to call in with questions, dial 1-888-469-5316 and then press “*1”. Again that number is 1-888-469-5316. While asking your question on the air, please do not use a speakerphone to ask your question. If you are listening to the audio through your computer, please turn down your computer volume after speaking with the operator. There is a significant time delay between the web and telephone audio.

If you want to send a question via the Internet, simply click on the button marked “Q&A” on the event window on your computer screen, type in your question and then click the “Send” button. One important thing, if you prefer not to use your name when you communicate with us, that is fine, but we would like to know what state you are from and the name of your department or organization. So please provide those details regardless of the way in which you transmit your question.

As you are formulating your questions or queuing up on the phone lines, I would like to say a few words about our sponsors. The mission of AHRQ is to support and conduct health services research designed to improve the outcomes and quality of healthcare, reduce its cost, address patient safety and medical errors and broaden access to effective services. AHRQ’s User Liaison Program serves as a bridge between researchers and state and local policymakers. ULP not only brings research-based information to policymakers so you are better informed, but we also take your questions back to AHRQ’s researchers so they are aware of priorities at the state and local levels. Hundreds of state and local officials participate in ULP workshops every year.

The audio conferences are being co-sponsored by the Center for Health Services Financing and Managed Care and the Department of Health Resources and Services Administration or HRSA. HRSA is the Department of Health and Human Services Access Agency. It assures the availability of quality healthcare to low-income, uninsured, isolated, vulnerable and special needs populations. Its mission is to improve and expand access to healthcare quality for all Americans.

I’d like to take a quick moment to thank Rhoda Abrams, the director of HRSA’s Center for Health Services Financing and Managed Care. Rhoda has been instrumental in helping to develop and produce these safety net products.

We’d appreciate any feedback you have on this web-assisted audio conference and at the end of today’s broadcast a brief evaluation form will appear on your screen. There are easy-to-follow instructions included on how to fill it out. Be sure to take the time to complete the form. For those of you who have been listening by telephone only and not using your computer, we ask that you stay on the line. The operator will ask you to respond to the same evaluation questions using the telephone keypad.

Your comments on this audio conference will provide us with a valuable tool in planning future events that better suit your needs. You could also email your comment to the AHRQ User Liaison Program at ulp@ahrq.gov.

Now for our first question. This one comes from David Keys. He would like to know, “What is considered a good response rate for mail surveys?”

Joel Cantor: Unfortunately my answer is going to be “it depends.” I will tell you better than that. It depends on many things. It depends on the population you are surveying. It depends on the weight of scientific strength you want in your estimates. I know I am not satisfying you. Typically, if you are doing a mail survey of a general population and they are not expecting to get the survey, if you do a single mailing you will get a very low response rate. If you are lucky you will get 10%. It is virtually impossible to say you represent anything when your response rate is that low. A rule of thumb is with each successive mailing, you will add roughly half a percentage point. So with the example I gave, a second mailing will get you up to 15%. Still not good enough. To really get to the range of representative sample sizes, typically in mail surveys of general populations, you would also need to do some telephone follow up or perhaps offer financial incentives or other kinds of incentives to get folks to respond. I think you would want to get your response rate at or above 50%. There is no general consensus in the field what is adequate. One standard is what would a scientific peer-reviewed journal accept in surveys? I rarely see anything published under 50%. Things in 60%, 65% are more common. I hope that, at least that is the best I can do.

Cindy DiBiasi: We have a caller. Brenda from Illinois is on the phone. Hello?

Brenda: Yes. Hi. My question is there was mention made of the importance of receiving training for interviewing skills. How important is it for individuals to acquire training on analyzing data prior to attempting to do so?

Joel Cantor: That is a great question. There are software packages available that require very little training to analyze data. Those prevent grave risks for the novice data analyst because it is fairly easy to do the analysis and get the wrong answer and not know it. I would strongly recommend that survey developers use experienced analysts. It doesn’t mean you need to find a PhD. statistician. But go to your local university. There may be grad students with sufficient experience and they generally will work fairly cheaply. But it does take some training and experience to do the data analytics.

Brenda: Thank you.

Cindy DiBiasi: Tim, do you want to add something?

Timothy Clouse: Yes. It has been my experience the packages are easy to use and will give you all sorts of wonderful numbers. The difficulty arises in trying to basically explain to somebody else what those numbers mean. Like Joel said, that is where your graduate student intern becomes very useful because they need this for their own training so they become a very good useful source of labor.

Vickie Gates: Cindy, I would also like to say that we really do encourage folks to use unique resources that are available and Lynn Blewett’s State Health Access Data, it is certainly one of those resources for all of the states that we have worked with in the HRSA program. One of our first recommendations is take advantage of this unique resource and Lynn may want to comment on some of the things that they actually have been able to do in helping people do a higher level of analysis and know that they have got a much more credible product.

Lynn Blewett: The State Health Access Data Assistance Center was funded by the Robert Wood Johnson Foundation and we are funded to provide technical assistance and policy analysis to states who are looking at surveys and developing surveys. So we can help you with your survey design or your sampling strategy and provide some technical input as you are developing your plans.

Cindy DiBiasi: Lynn, how do they get in touch with you?

Lynn Blewett: Well, they can, I think the contact information is on my slides, but it is www.shadac.org.

Cindy DiBiasi: OK. A caller wants to know, “Can anyone in the group discuss trends in the area of the underinsured? Are you starting to monitor underinsurance and would you consider Medicare’s lack of drug coverage an underinsurance issue?”

Joel Cantor: Lynn? (Laughter.) There have been a couple of good studies of underinsurance, one by Pamela Short; I guess she has updated it at least once. It is a difficult thing to define. It can get technically complex. To my knowledge, and maybe some of the other panelists know differently, there is no standard, generally accepted way of measuring underinsurance. It is in the eye of the beholder in some ways. To answer the last part of the question I think probably I will speak for myself.
Certainly lack of prescription drug coverage for a big segment of the elderly population is a huge underinsurance problem.

Lynn Blewett: I think there is more interest in trying to define and measure underinsurance. There are like three different ways to look at it. One is what we call the economic approach. That is how much out of pocket do you pay for your health insurance coverage? As Joel said, that is what Pamela Short has looked at and the range there again is almost subjective in terms of what value you select. But if you pay 10% or 20% more of your income out of pocket to pay for coverage or services, that that may be considered underinsurance. So that is one way to look at it. If you have income information and out-of-pocket cost information. Again, it is very difficult to do to get that level of detail in a survey.

Another approach to look at is what benefits are covered. So your mention of the Medicare lack of prescription drug coverage, it could be considered underinsurance. Another approach is to ask people is there something that you need a coverage for that you weren’t able to get? That is sort of an attitude of what the perception of people are. Do you have coverage for a needed care or not? So there are three kind of different ways to approach underinsurance. As Joel said, very difficult to measure and monitor it, but again as employers are shifting a lot more costs to the employees in terms of out-of-pocket co-pays and deductibles, premiums, I think there is going to be more research and interest in developing more refined measurements.

Cindy DiBiasi: And a question on uninsured from Shari Isaac from Denver Health wants to know, “What is the standard definition being used for uninsured? Is it not having insurance for six months or twelve months? When do we draw the line on uninsured?”

Joel Cantor: Well if this were a videoconference you would see us all rolling our eyes. This is a perennial problem. The different surveys measure uninsurance in different ways. It gets technical quickly, although I think the way most people think about it and refer to it is at a given point in time, last month or today, how many people lack coverage. The main national survey, the current population survey that tracks this annually measures it a little bit differently. But if you are doing your own survey, I would recommend using the point-in-time measure. You might then also ask questions about when you last had coverage to be able to measure duration. Because obviously it is a bigger problem for people who are uninsured for longer.

Cindy DiBiasi: Tim?

Timothy Clouse: A second option, which is used by national health examination surveys, actually asks two questions. First one, do you currently have health insurance and then secondly, did you have health insurance any time within the preceding twelve months?

Cindy DiBiasi: Lynn?

Lynn Blewett: SHADAC has a tool that might be useful which basically summarizes the different ways of asking about health insurance coverage and we look at the CPS in different ways to do that and make recommendations. We call them our survey guidelines and we have one on how to ask income questions. We have one on how to ask health insurance coverage and we, I can’t think of the third one. Those again are available online and I think might be useful as you are exploring these different tools, what are pros and cons of asking different questions.

Cindy DiBiasi: I am just going to repeat your Web site since I know what is going to happen is we are going to be inundated with calls saying how to get in touch with her. It is www.shadac.org.

A question from Laurie Olson. She said, “Lynn mentioned that 37 states have done surveys. Is Nevada one of them?”

Lynn Blewett: I think Nevada is not one of them. I don’t have the list in front of me, but I am pretty sure Nevada is not one of those states.

Cindy DiBiasi: OK. A question from William Mogg. “Safety net providers have limited ability to cost shift to the uninsured population. What do you think about modifying financial statistics to measure the ability of the safety net’s provider’s ability to cost shift?”

Timothy Clouse: Well, cost shift is a little, not really a good one in this context because that kind of implies that you are basically jacking up your billing rates to people who have got insurance, which is really not what you should be doing. In theory, what you should be doing is getting access to a federal or state grant program, which would pay for that. In fact, there is a HRSA third-party reimbursement Web site which actually covers some of that where there the focus is on trying to make sure that HRSA-funded programs for grantees are maximizing use of their grants. The Web site for that is www.hrsa.gov/tpr.

Cindy DiBiasi: Joel?

Joel Cantor: One thing I would add is that it may be useful to present some what I will call the markers of the ability to cover one’s cost. I won’t call it cost shifting for the reasons Tim mentioned, but payer mix, simply put, can be very useful. What proportion of a given provider’s revenue is from private sources? How much from Medicaid? How much from Medicare and how much from self-pay or uninsured? That will tell you, that speaks volumes about the ability of the provider to cover its costs with its revenue stream. Which I think is what you are trying to get at.

Cindy DiBiasi: Question from Nancy Wilbur. She wants to know, “What are some ways to address the issue of people excluded from direct-dial phone surveys due to a lack of phones or use of cell phones only if you are primarily conducting your survey by phone?”

Joel Cantor: Good question. The number of households without phones, without landline phones has grown and is becoming a problem. There is also the problem of people using technology such as answering machines and caller ID to screen out calls, so response rates are going down. It is very difficult. To deal with the problem of under coverage by phone, there are really two techniques. One is to supplement the sample with an in-person interview. Typically what is done is to select census tracts with low telephone coverage that is, there may be 10% of households or 15% of households that don’t have phones. Those would be considered high non-phone tracts. To send interviewers in and enumerate the households, conduct a sampling of those households. Knock on doors and conduct interviews.

We are conducting a survey right now in New Brunswick, New Jersey where we are using that technique. We are actually enumerating the households, recruiting the families to participate and then handing them our own cell phone so that they can do the interview by phone like the other 90% of the sample.

Cindy DiBiasi: What is the advantage of that?

Joel Cantor: The advantage is you are using the same methodology to collect the data so you don’t end up with a method-effect bias.

Cindy DiBiasi: OK.

Joel Cantor: You also can use the computerized interviewing technology without carrying laptops around in very poor neighborhoods with high crime rates. So there are some real advantages to using the cell phone method.

The second methodology involves conducting a telephone-only sample, but then asking as part of your questionnaire for a telephone coverage history. In other words, measure for each household in your sample. Obviously they have a phone on the day of the survey, but did they not have a phone last month? For how many of those months in the last year did they not have a phone? Then statisticians can use weighting strategies, effectively up weighting households that have a history of lacking of phones to balance the survey and reduce the telephone coverage bias. It turns out that households that lacked a phone last month but have a phone this month are a lot like households that lack a phone on the day of the survey. So the weighting strategy is effective by its reduction strategy.

Cindy DiBiasi: From a pure consumer point of view here, I can’t imagine with all of this overabundance of telemarketers and everything else that as soon as somebody picks up the phone and hears a voice they don’t recognize that they either hang up or say I’m sorry or they are on the data, the new call registry which is blocking all of this. It must be very difficult now.

Joel Cantor: The Do Not Call Registries don’t block legitimate surveys. They block telemarketing. So that is not a problem. But you are right, there is a problem of people screening calls and being very reluctant to take calls from strangers. This is especially true in the big media market. Response rates in places like New York, Philadelphia, L.A. are really abysmal. They are much better in places like North Dakota and so on where there is just not the same inundation of telemarketers. Perhaps folks in North Dakota would disagree with me, I’m not sure.

But there are a number of things that survey sponsors can do to minimize this problem. One is if the survey is sponsored by a government agency, that can help or if you can get a letter of endorsement from a government agency that legitimizes the survey. A university-based study generally gets better response rates than a no-name commercial vendor survey or even a brand name commercial vendor survey. Sending out letters in advance of calling can help. As I mentioned earlier, paying people even a modest incentive tells them that you are serious and you take them seriously. That can help your response rate.

Cindy DiBiasi: Tim?

Timothy Clouse: Just a comment on sending it out basically on government letterhead is depending upon the population you are surveying, they may view the government regardless of what level, as all being the same and if they see something that has government letterhead on, that may have some negative connotations.

Cindy DiBiasi: I was going to ask you, does that increase your chances that it is going to be answered or decrease them?

Timothy Clouse: It depends on the population you are surveying. In my experience, I have had it both ways. Actually like Joel said, I think for most people if it comes from the University of South Florida or whatever, that is a little more neutral, a little bit, well, less threatening. That might work better. My experience is that public sponsorship actually helps and there are populations that are, for example, immigrants who may not be here legally who would really be concerned. But they are going to be concerned regardless. But in general, government endorsement helps I think, as a general rule.

Cindy DiBiasi: Lynn, why haven’t we heard more about the State and Local Area Integrated Telephone Survey called SLAITS? When is that data going to be released?

Lynn Blewett: Well, that survey was initially, it is a survey of children with special health needs so it was funded by the Maternal and Child Health Bureau, the Department of Health and Human Services, and it was targeted for children with special health needs. Another federal agency actually funded an additional part of that survey to look at health insurance coverage for children in general. So it was targeted for one population; they added a different funding source to get children without special health needs and so I think it has been a complicated health survey that has taken a while. We expect the data to be released really within the next month or so. We are working with the, SHADAC is working with the SLAITS people to try to make it more visible and have people access to the data and information because it will provide some new information on levels of estimates of uninsured children for state and local areas.

It is not a survey; it is a very interesting mechanism because it is not a survey that has ongoing consistent funding every year like CPS or MEPS. It is what they call a survey module that you can fund, if they can find other agencies that are interested in funding, they will be able to field the survey. But it is not an ongoing (unclear) which may be related to why more people don’t know about it. It is the first time that this information will be released.

Cindy DiBiasi: OK. A question from Beth Baliot. “We were reminded again how important data quality is as you present it to the community to gain interest and support. What factors should we pay attention to protect ourselves from folks questioning quality?”

Vickie Gates: Well, I think we could make, all make a couple of comments about that, but I think one of the things that is really key is the source of the data and thinking through who actually is going to be responsible for this data, the quality of the provider that you have worked with to produce the data and your ability to present it as objective and neutral. So if you in fact are an advocate on an issue, it is even more important that you work with a contractor who has credibility and who can bring that objectivity and neutrality to the issue.

It also helps sometimes when you are working with policymakers to think about who on their particular group of advisors has the time and energy and is worth the investment for you spending time and energy on really going through where this data came from, how this data was put together, what is was designed to do.

The other thing that I think I want to reiterate is worry about many of these issues on the front end, not the back end. So as you begin to contemplate what the questions are and what the issues are that you are trying to design data for, that is when in fact you want to go and think very carefully about who will be part of that process, how do they understand the issue, how do you get them involved? It is sometimes more difficult. It will take you a little longer, but when it comes to credibility issues I believe it will pay off for you.

Cindy DiBiasi: When it comes to being realistic about surveys and Joel I have a feeling I am going to get a “that depends” answer to this question, but is there a range of how much time you should expect it to take and how much money you should expect it to cost? Just to be realistic, you are saying think it through before you start, but experience is often our best teacher and often we don’t know what is going to happen once we are in the process.

Joel Cantor: Cindy, the answer is it depends. (Laughter.) Another obnoxious answer is make your best estimate and then double it. The truth is of course it depends on many factors. If you are, for example, trying to conduct a survey in a local area, say 600 interviews, 30-minute interview, this is the kind of parameters that we are talking about in these sorts of surveys. You are talking about spending perhaps $100,000 just to have a vendor collect the data for you. That doesn’t count the time you put in developing the questionnaire, which should be counted in months, not weeks. The time you spend testing and fielding. One of the best ways to get a decent response rate is to keep your survey in the field for a long time so that you don’t go back to people repeatedly in one week, then they will turn you down. But if they are reluctant to participate because they are busy, don’t call them again for three weeks. Then maybe they won’t be so busy. So our surveys generally stay in the field between four and six months. Then you need several months to clean and prepare the data. Another several months to do the analysis and then spend as much time as you can presenting the findings and publishing the findings as you possibly can. So it is kind of a cobbled-together answer, but those are general parameters.

Cindy DiBiasi: Robin?

Robin Weinick: I want to ask a follow-up question for both Joel and Lynn. Given the long time horizon for collecting data, and particularly when you look at some of these national surveys, you have talked about uninsurance, can often be giving you data from a year or two or even three ago. Given the long time horizon, how do we make these data really relevant to policymakers today because they need the most current information that they can possibly get?

Lynn Blewett: That is a very good question. I think the current population survey actually is released every fall and it is probably the most up to date and it is a year lag time. So that is kind of what you are having to deal with. The HRSA state planning grant states who have conducted surveys, they have been under a timeframe that they are required to get their surveys in the field and the analysis done within a year. So that data is really timely and relevant and having that sort of time pressure has actually helped make the data more relevant. So if you are able to do your own survey and you have got a tight timeframe and you have got enough people helping you to get it done, you have used their existing resources, you can get more timely information. But most of the national data that we, 18 months out. It is just something that you sort of have to live with.

Joel Cantor: I think that is the best you can do. Survey collections such as the Current Population Survey, such as the Behavioral Risk Factor Surveillance surveys, which are sponsored by the Centers for Disease Control, and conducted by the states, those surveys are done routinely. They are kind of a well oiled a machine as there is going to be. So you are never going to beat their timeframes. So don’t think that doing your own survey is a way of shortcutting the timeframes for available data.

Cindy DiBiasi: You are possibly sacrificing the quality of the results, right?

Joel Cantor: that is right.

Cindy DiBiasi: Tim?

Timothy Clouse: I will just put on my economist hat for a moment and say that when you are dealing with these types of issues, you are dealing with time series data and it is certainly that they do change over time. But for policymaking purposes, they don’t change very much. Government is basically kind of a blunt instrument, so the fact that it is a year old is probably unfortunately close enough for government work.

Cindy DiBiasi: So that is where that phrase came from. (Laughs.) Joel, a question for you. “Are there good sources of information about the care that is available in the local safety net?”

Joel Cantor: Sure, we have talked a lot about general population surveys and family surveys, household surveys. Tim talked about financial data from healthcare providers. Of course that is one important piece of understanding the safety nets. There are some data sources available about what resources are available in a local community. For the HRSA-funded community health centers, there is the UDS Data System about the federally qualified health (unclear). The American Hospital Association conducts an annual survey, which collects information about services available through hospitals including emergency departments and outpatient departments. In the data book, some of these measures are summarized, aggregated up to the local level. There is also physician data available through the area resource file, through the AMA, so there are some existing sources.

Sometimes, again using my rule of thumb of use existing data whenever possible, do that. But when it is not possible, you can construct a survey of providers. Individual private practice physicians are notoriously difficult to survey, very hard to get a good response rate. They are very busy folks and they are inundated, given interest of pharmaceutical companies and others with marketing surveys and other things.

Institutional providers, if you can motivate them to work with you, might be a more ready source of information for the safety nets since many safety net services are delivered through institutions and my chapter in the tool kit does go into a little bit about what kind of information you can get from surveys of institutional providers. Things like understanding enabling services, physical plant capacity, and other sorts of measures that can be very valuable in assessing capacity.

Cindy DiBiasi: We talk so much about people being busy and everything else. How long does a typical survey, if there is such thing as a typical survey time, how long does it take to actually fill out a survey or answer questions posed to you in a survey?

Joel Cantor: You can collect an enormous amount of information in a 15-minute interview. You would be really surprised how much data you can collect. One rule of thumb is you can ask six or seven questions in a minute. It depends on the question, of course. It depends on the audience and so on. So do the math. In 15 minutes you can ask a lot of questions. Some surveys, such as the National Health Interview Survey, such as the Medical Expenditure Panel Survey, use much longer instruments. There is a real depth of information there.

For telephone surveys, typically they run between 20 and 40 minutes. After that, people can’t stand to be on the phone and you really lose data quality.

Cindy DiBiasi: That is a long time.

Joel Cantor: That is a long time, but people generally seem willing. In our surveys, which run that long, people don’t hang up on us. They complete the interview. If you can motivate them, it is an important topic, a subject they want to talk about, people like to talk about themselves, and they are willing to stay on.

Cindy DiBiasi: Do you explain upfront why you are doing this? The importance of it and putting it into some context?

Joel Cantor: Yes and I think as researchers we are ethically bound to tell people what we are doing, why we are doing it, how long it will take and then ask their permission to proceed.

Cindy DiBiasi: Robin?

Robin Weinick: I actually wanted to come back to a slightly bigger picture. We have been talking a lot about the technical details about data collection and how to really get into the nitty-gritty. I wanted to pose a question for Vickie which is the policymaking process is so complex. How is it that people can actually start to use database information collected from the kinds of things that Tim and Lynn and Joel have been talking about today, to really inform that process?

Vickie Gates: Part of, I think, the thing that you do is you think about where your policy levers are in a given environment. What people often forget to think about is that your levers are not simply political leadership; they are often also the administrative leadership that is part of problem solving. They are, in many cases, groups that may have a specific interest in an issue and who may be a good voice. So I think one of the classic things if you have good data, if you have information that you feel will add value to the policy process, then begin to catalog the environment, the players and the way they interact. Then begin to think about what it is that they need and how they like to access information. Because it is going to vary. There are certain types of tools that you will want to develop for legislators. There are other types of tools that you will develop for administrative leaders. There are other types of tools that you may develop for key stakeholder groups around an issue.

I think Joel made a comment about when you think about putting out information on a survey, to think about white papers. To think about one-page recaps. It is the same situation. Think about your audience. Get it split into groups. Think about the way they like to access information, how to get that information across and be willing to use a diversity of tools.

Cindy DiBiasi: A question from Kim Barrick wants to know how we can access a copy of the full worksheet. I believe Tim that is your worksheet.

Robin Weinick: Yes it is, but let me answer that question. (Laughter.) We actually have all three of these chapters that we have been discussing today from the school kit are available on our Web site, which is www.ahrq.gov/data/safetynet. That is www.ahrq.gov/data/safetynet. What you will find there right now are copies of these three papers as well as a few others in addition. You will find the interactive version of the worksheet that Tim is describing.

Cindy DiBiasi: We have Rene from Arizona on the telephone. Hello?

Rene: Hi. We have a question. What advice can you give about connecting with human subjects committees and institutional review boards before doing surveys?

Joel Cantor: You have to. (Laughter.) Of course it depends on your institutional setting. But typically each IRB will have a formal process that you have to go through and send them your research protocol. They will often want to look at your questionnaire. Am I answering your question?

Lynn Blewett: Yes. I was thinking this is why it is good to partner with a university that has that infrastructure already in place. Most local universities will have an institutional review board. Many hospitals will have an institutional review board or health plans who do research will have a research arm. If you are not familiar with those, you need to find one. I think there are actually some health departments that have their own review structures as well. It is an important part of the process and can provide you with a lot of information that you will need to make sure that you meet the privacy and consent requirements.

Cindy DiBiasi: Joel, you were being a little facetious but not really when you say you have to. Why...?

Joel Cantor: Well, it depends on your institutional setting. You would need to speak to somebody who is more expert than I in the legalities of it. But if you are doing research with human subjects in an institution that is bound by the IRB rules, then you have to have IRB approval to move ahead.

Cindy DiBiasi: Robin?

Robin Weinick: I am curious what you have to say about the role of the new Health Insurance Portability and Accountability Act that went into effect this April on data collection issues in terms of thinking about when you are going out and collecting survey data. What do you need to do that is different? I am not asking you to give anybody legal advice on HIPAA but just sort of your sense of what is added into the process?

Joel Cantor: My sense is that if you are a HIPAA covered entity or you are working with one, you are bound by the HIPAA rules and it has a slightly different but conceptually-similar process to the IRB process so there will be privacy boards that you will need to go through. The issues are different and now you have totally exhausted my expertise.

Cindy DiBiasi: Rene...

Robin Weinick: I think that one good follow up is to point out that the Institutional Review Board is responsible for human subject protection and HIPAA is responsible for the protection of the privacy of the data. So these actually do complement one another.

Cindy DiBiasi: Rene, does that answer your question or do you need to follow up? We lost Rene. OK. Sorry. Go ahead.

Lynn Blewett: I just want to say that the HIPAA requirements pertain to the data and it is really important that if you have individual data which includes information about their health status and their health insurance coverage, that that is considered private data and there are mechanisms that if you are a researcher or a state agency on how you store that and where you file it and where it is kept. Those are again really important things that if you are not in expertise, you need to find somebody who can help you sort of sort through that, those requirements. There are severe penalties.

Cindy DiBiasi: Antonio Laci wants to know, “Is there data available that summarizes the different safety net programs?”

Robin Weinick: There have been a bunch of different reviews that have bee written that really sort of cover some of the bit picture issues. I can’t present any of them off the top of my head, but certainly the kinds of things you want to be thinking about are of course the Medicaid programs. You want to be thinking about the issues (unclear) hospital program and Joel has just kindly pointed out that I should remember the core source of the reason we are doing this whole thing is you go to look at the Institute of Medicine Report, which was published in 2000 called the American Healthcare Safety Net: Intact or In Danger? You will find the full list there.

Cindy DiBiasi: Great. Tim?

Timothy Clouse: I just wanted to mention, this information is available on, like I said, a catalog of federal domestic assistance, but that is probably too detailed. But I realized just about any basic introduction to the U.S. Public Health or the U.S. Healthcare System would have it as well. So just any university of public health bookstore.

Cindy DiBiasi: Vickie?

Vickie Gates: I was also going to suggest that people might be interested in a couple of tools that are on the State Coverage Initiative’s Web site, www.statecoverage.net. One of them is a coverage matrix that will give you at least some idea of what states are doing. The other is a very unique resource. There are over 700 searchable reports, all produced at the state level, that you can have access to on that site. So if you are in a particular state and you are curious as to whether your state, like some, have done a review of their safety net programs. For example, in Oregon you would find at least two fairly expensive reports on their safety net. You may be able to accept that type of material there.

Cindy DiBiasi: Tim, a question for you. What can the financial model tell us about the number of patients needed to support a community-based clinic?

Timothy Clouse: Well, it won’t tell you directly, unfortunately because it is only dealing with dollar amounts. Theoretically, if you had one person who made quite a lot of money and was exceptionally ill, then that might provide enough dollar volume right there. It is not going to happen, but instead you can say well, this is the dollar volume we need. So you can say in order to support this dollar volume, we need roughly this many patients assuming that each patient visits two or three times a year and each visit is going to cost however many dollars.

On the federal side, it is a very, very approximate rule of thumb. The designation for a health professional shortage area, which is one of the designations required for certain types of federal programs is if an area has more than one, a ratio of greater than one primary care provider per 3,500 people, that area is considered medically needy. So you can use the 1:3,500 ratio as a very approximate rule of thumb for the target population.

Cindy DiBiasi: A question from Victor Alice. “Is there a place for convenience sample versus a probability sample for pilot testing a survey tool?”

Joel Cantor: Yes, absolutely. If your goal is to see how well the survey tool is working, there is no reason why you need to go through the machinations of doing probability sampling. Typically, pilot tests used to be (unclear) samples. It is generally accepted. You learn a lot from just a handful of interviews, too.

Cindy DiBiasi: Is there, back to my question of how to sort of get your mind around whether you really can do this or how to do it. You were saying to make sure there is a lot of thought in the beginning of the survey so you are not dealing with all those issues sort of after the horse is out of the barn. Is there a percentage? Do you basically divide it into before, during and after and give equal time and weight and attention to all part of it? Do people generally make mistakes about not putting enough thought upfront?

Joel Cantor: Yes they do. I would say that, I am making this up on the fly, but that you should spend probably double the amount of time that you spend collecting data actually preparing to do it and then close to that in the post-data collection phase working your data and your findings and getting the word out.

Cindy DiBiasi: As my carpenter father used to say, “Measure twice and cut once.” (Laughs.)

Joel Cantor: That is right.

Lynn Blewett: I was going to say too, for health insurance coverage there are so many good survey tools and questions that have been fielded and validated that you really should use an existing tool just because of the cost in terms of developing a new tool and field testing and all of that that requires. Either through the MEPS or CPS or our tool, which is the Coordinated State Coverage Survey, I would encourage you to sort use a tool that has already been field tested and validated.

Cindy DiBiasi: On the phone with us from Texas, Kimberly is waiting. Kimberly, hello?

Bill: This isn’t Kimberly. This is her associate, Bill with Texas Health and Human Services Commission. In several of my jobs I have worked extensively with special populations. One that I had a lot of headaches was working with the limited English-speaking population trying to find physicians, clinics, hospitals, who had language resources so they could do consulting examinations for people who did not speak English. Has this sort of issue been studied by anyone?

Robin Weinick: Well, we have done some work that has been funded by HRSA looking in rural areas at sort of safety net and the ability to meet the needs of a growing Latino population. This has been mostly in rural areas and have documented some of the unique safety net response to a growing Latino population. As you can imagine, there isn’t sort of a rule of thumb of how you do that. These safety net services kind of grow from the grassroots level and a lot of piecing together of different funding sources. I think that is one of the things that the IOM study documented is just the fragile nature of the safety net and responding to sort of this growing Latino population.

Bill: Yes, for a while I was excused from the Latino population and worked primarily with the Indo-Chinese refugees and later with other refugee populations that finally extended to the Latino population because we had problems even in large urban areas to find the qualified sources. Even when they were available, no one had cataloged them.

Cindy DiBiasi: There is not a repository of information?

Bill: No. In some communities, the county medial association published directories that not only included the pictures, the names of the physicians and their specialties, but showed where they graduated from medical school and that was extremely valuable. If, for instance, the doctor wasn’t the right specialty but he did speak the language, he might well know a nurse from that same country who could interpret, but most of the counties didn’t do anything like that. They just had simple names and numbers and the specialty and that was it.

Robin Weinick: I think it is a really important question and I think one of the challenges that AHRQ has in terms of trying to understand the local safety net and documenting and articulating some of these issues, because there isn’t a repository. There is just no systematic source of information that I know of that documents that. You may find that at the local level and we certainly have found that. But it is again, really community, locally-based information.

Bill: Right. OK, thank you.

Vickie Gates: I would just like to make a comment that in fact one of the things that is interesting about these types of discussions is they sometimes bring up work that would be useful. There are places where you have got some leadership in identifying with better ways to deal with very small populations to make sure that there are either interpreters or practitioners who understand the language and who can communicate. As we become more sensitive to this, I think there is going to be a need to identify those model areas and take the best of their technology and make it available for others.

Cindy DiBiasi: We are going to go around this table now just to get a final wrap up from each of the participants. Lynn, why don’t we start with you?

Lynn Blewett: I would encourage all of you who are interested in monitoring the safety net to use your local resources and the available data. There is a lot of information available and you ought to be aware of it before you go ahead and do your own data collection. I am encouraged by the number of people on this call. SHADAC is interested in helping in any way we can in providing you access or links to information and I wish you all good luck.

Cindy DiBiasi: Joel, she stole what you were going to say, didn’t she?

Joel Cantor: That is exactly what I was going to say. (Laughter.) I am also heartened by the interest and really the great questions. Probably the most important thing to come out today was the list of resources that we have all sort of come up with. The SHADAC Web site, the coverage initiatives, the information on the AHRQ Web site, the tool kit and so on. So I hope we are notching things up a little bit in terms of providing support to the communities.

Cindy DiBiasi: Tim?

Timothy Clouse: I have been really encouraged with the questions. It has raised issues, which quite simply I hadn’t thought about when we were writing this paper. I don’t know if it is a good thing or a bad thing, but...

Cindy DiBiasi: It gives you information for your own paper.

Timothy Clouse: Right. If there is one. But the crucial issue with this paper, which I mentioned before, is this is not a cookbook. It is not a substitute for professional judgment. You can’t run the numbers and then say, “Well, these guys at AHRQ said that we are in good shape or bad shape.” All this is is an indicator and it is designed to provide you with some information or you can take this and say, “Well, more research is needed.”

Vickie Gates: I would say roughly the same thing the group has said in terms of being encouraged about the number of people who are interested in the issue. But also I really would like to say that I think AHRQ has brought together some valuable new resources. I was serious when I said read this material. Take it seriously. There is very good advice here.

Cindy DiBiasi: Robin?

Robin Weinick: In just a minute, Cindy is going to tell you how to find copies of those materials, but what I also wanted to encourage you to do is to let us know what other topics, what other types of tools would be useful to you as we think about our future works, that we can best meet your needs.

Cindy DiBiasi: Thank you all for joining us this afternoon. If you have any unanswered questions, please send us an email to ulp@ahrq.gov. Depending on the number of questions, we will try to answer you directly. As Robin mentioned, we encourage you to send us any researchable questions that you are facing at the state or local levels for AHRQ’s consideration as the agency plans its future research priorities.

As we wind down, let me mention that a number of products from the audio conference will be available at a later date. Around the middle of November, an audio streamed archive of today’s call, a written transcript and all of the presenter’s slides, including those used in the question and answer session, will be posted to the AHRQ Web site. PowerPoint and text versions of the site are available at www.academyhealth.org/ahrq/ulp/safetynet. An audiotape of this event will be available for purchase in several week’s time and the cost for a set of three audiotapes from this series will be $10. To order a copy, call the AHRQ Publications Clearinghouse. The phone number is 1-800-358-9295. Again, that number is 1-800-358-9295. Ask for AHRQ03-AV12A. It is entitled Monitoring the Healthcare Safety Net.

Also, information related to the AHRQ and HRSA initiative to monitor the healthcare safety net is available on the AHRQ Web site. Copies of electronic versions of the tool kit chapters will be put on the safety net Web site at www.ahrq.gov/data/safetynet. That will go up as they become available. In fact, five of the nine chapters are already online. You can also access an interactive version of Tim Clouse’s financial model on that site. This interactive version performs calculations for users after entering their own information. Hardcopies of the Book III: Tools for Monitoring the Healthcare Safety Net are not yet available. To request a copy to be mailed to you when they become available, please send an email to SAFENET@ahrq.gov. The data books introduced in yesterday’s web-assisted audio conference are not available. To request a copy of the data books, please send an email to the clearinghouse at ahrqpubs@ahrq.gov or you can call 1-800-358-9295. Again that number is 1-800-358-9295. You are going to be asking for publications ahrq03-0025 and ahrq03-0026.

Please mark your calendars for the final event in our safety net series. That will be tomorrow, Thursday, September 25th from 2:00-3:30 pm eastern time. We will be focusing on how to use data to tell the safety net story.

Finally, before you log off, don’t forget to take a few minutes to fill out the brief evaluation form that will appear on your screen at the end of the broadcast. Easy to follow instructions are included. For those of you who have been listening by phone only and not using your computer, please stay on the line. The operator will ask you to respond to the same evaluation questions by using the keypad on your telephone. You may email us your comments to ulp@ahrq.gov. Thanks for joining us this afternoon. We look forward to you joining us tomorrow for the final call in our series. Have a good day.

Back to Top