Beta Research Corporation >> Beta Insights

Following the attacks that occurred on September 11th and the subsequent anthrax scares, Beta Research and McPheters & Company entered into testing of various methods to lift respondent confidence in the validity of the survey instrument and, hence, improve response rates. The first test involved an 8-cell design that looked at the effects of various combinations of variables on response rates to mail surveys. The second involved use of a telephone pre-recruit to obtain consent to fill out the same mail questionnaire also used in Test I.

Test I involved use of a cover letter, $1 incentive, and 1-page (front and back) questionnaire, which was mailed to 8 nationally representative sample replicates of 250 each. Advance postcards were mailed on November 9th, and surveys were mailed on November 14th. Combinations of three variables were tested. The variables were:

The control cell did not make use of any of the methods tested; i.e., there was no advance postcard and no contact name. It included a response envelope and, as was the case with every cell, live postage was affixed to the envelope. All fieldwork was closed on December 19th.

The advance postcard in combination with the regular envelope and the name and phone number of a Beta contact generated the highest response rate 40.5%. This response rate was significantly higher than all others involving the use of a regular envelope and more than a third higher than the control cell. Interestingly, not only did a smaller proportion of the sample respond to the control cell, but respondents in this cell were significantly less affluent than respondents in the remaining cells.

However, the addition of the advance postcard and use of a contact had negligible impact on response rate for those cells involving the use of the Peel 'N Seal envelope. None of the cells in which the Peel 'N Seal envelope was used differed significantly from either the control group or from each other.

RESPONSE RATES - PEEL 'N SEaL ENVELOPES
Cell 5	No advance postcard No contact	33.2 %
Cell 6	No advance postcard Contact	31.8 %
Cell 7	Advance postcard No contact	33.1 %
Cell 8	Advance postcard Contact	34.4 %

Our conclusion is that advance postcard notification and the name and phone number of a contact with whom survey validity can be verified, in combination with a response envelope with live postage, increases respondent confidence and results in higher response rates. We expect to be using this method to generate higher response rates for our clients in the months ahead.

TEST II

Prior to November 14th, we used an RDD sample to get up-front commitment to fill-out a mail questionnaire. We called 7008 numbers, of which 1831 were ineligible by virtue of being business or government offices, fax/modem lines, disconnected, beepers, etc. The remaining 5177 eligible numbers yielded 993 respondents who agreed to participate or a yield of 19.2%.

On November 14th, the questionnaire was then mailed out to the 993 who agreed to fill it out. The package mailed was the same as that used for the control cell in Test I. As was the case with Test I, the fieldwork was concluded on December 19th. Of the 993 mailed, we received completed questionnaires from 659 or 66.4%. The demographic profile of the respondents was similar to that for the other test cells. While the net response rate was a low 12.7%, we believe that for certain low incidence populations this method may have applicability, in that it provides a way to undertake at reasonable cost, research that would otherwise be cost prohibitive.

SUMMARY OF TESTS

Use of advance post cards and identification of a contact with whom survey validity can be confirmed increase response to mail surveys. While a telephone pre-recruit generates relatively low response rates in the initial phase, it yields high rates of cooperation in the second phase, and provides a relatively cost-effective way to survey low incidence populations or to conduct surveys where the cost/survey administered is unusually high.

We expect to conduct further experimental designs over the course of this year in an effort to enhance both telephone and mail response. If you would like to receive copies of earlier write-ups regarding other steps that can be used to enhance response rates, please do not hesitate to call, write, or email us.

For further information, please contact: