Subscribe to our RSS feed

Survey Methods, Pros & Cons

By Better Office LLC on 04/17/2014

Sources: Arlene Fink, "How to Conduct Surveys", 3rd Edition; Alan Aldridge; Kevin Levine, "Surveying the Social World"

Recovered from original nedarc.org article

Self-Completion Questionnaires

  • Postal / E-Mail Questionnaire

    General Description:

    Paper & pencil survey, sent in the mail; e-mail attachment, completed individually

    Example Uses Pros Cons Considerations

    Pediatric Equipment on an ambulance - i.e. someone needs to physically stand in front of the ambulance...

    • Can reach a large geographical area
    • People are used to completing paper-and-pencil surveys
    • Can take the survey with you and complete it anywhere and anytime
    • Great for sensitive issues
    • No clarification available during completion.
    • Need a motivated population to return the survey (people have too much to do)
    • Respondents must be able to read, see, and write
    • Need an up-to-date address list
    • Visual cues and illustrative figures

    COSTS:

    • Follow-up mailings
    • Printing, paper, envelopes
    • Stamps
    • Incentives

    Email considerations: include it as an ATTACHMENT (Word Document, PDF); could fill-out electronically or fax back.

  • On-the-Spot Questionnaire

    General Description:

    Paper & pencil survey, administered in person but completed individually

    Example Uses Pros Cons Considerations

    After an advisory committee meeting, you survey the attendees about the current meeting compared to past meetings

    • Captive Population
    • Information is obtained immediately
    • Questions can be asked as they arise
    • In some cases, surveys can be done with groups of people
    • Can gauge an immediate response to an event, etc.
    • Limited to responses from just those who are on site - possible bias
    • Respondents must be able to read, see, and write
    • Possible bias from the administrator
    • Annoyance factor
    • Visual cues and illustrative figures
    • Fill-out & return on site or take home & return

    COSTS:

    • Requires on-site staff/supervisor
    • Space and privacy for respondent
    • Printing & paper
    • Incentives
  • Internet/Web Based

    General Description:

    Online survey; email or distribute a link (URL) to a survey which is designed and completed online

    Example Uses Pros Cons Considerations

    Determine the percentage of hospitals in the State/Territory that have written pediatric inter-facility transfer agreements

    • Possible worldwide population
    • Negligible distribution costs
    • Order of questions can be preprogrammed
    • Only "acceptable" answers are allowed (validation)
    • Can give respondent links that give additional explanation
    • Data is automatically entered in a database and can be automatically analyzed OR exported to other software programs
    • Branching, Piping & Skipping can be automatically programmed
    • Can easily track users response rate OR choose anonymous
    • Automatic Reminder, Thank You, Spell Check, and Math Calculations
    • Easy pilot testing (discover where people are abandoning the survey)
    • Progress bar to inform respondents of the percentage completed
    • Require the question to be answered
    • Need reliable access to Internet
    • Respondent must be able to use a browser
    • Respondent must be "online"
    • Respondent must be able to use a computer, a mouse, and/or keyboard
    • Technical expertise in-house vs. respondent expertise
    • Link to many different media types
    • Does the software import pre-written questions?
    • Does the hosting company have a policy for back-ups?
    • Make sure the company has a solid "confidentiality" agreement
    • Is the survey taken and submitted on a secure URL(SSL) - security
    • Make sure you can export the survey data into various formats
    • Survey customization (branding)

    COSTS:

    • Hosting & software $ - $$$$ (depending on features)
    • In-house programmer

Face-to-Face Interviews

  • Face-to-Face Interviews

    General Description:

    Conduct interviews in person with a written script and/or prompts

    Example Uses Pros Cons Considerations

    Determine the qualifications and interest level of an individual to serve on the State/Territory EMSC Advisory Committee

    • High rate of survey completion
    • Can explore answers with respondents
    • Can assist respondents with unfamiliar words or questions
    • Able to get more qualitative data
    • Difficult to reach certain populations
    • Expensive and time consuming (lots of resources)
    • Some may feel reluctant to share personal information or beliefs
    • Need trained interviewers
    • Must find a suitable place to conduct interview
    • Interviewer bias (verbal or non-verbal)
    • Longer more in-depth questions
    • If on-site need space and privacy
    • May be difficult or dangerous to go to person's home
    • Visual cue cards

    COSTS:

    • Training
    • Space & travel
    • AV & transcription costs
    • Incentives
  • Focus Groups

    General Description:

    A qualitative study; small group from target sample brought together to discuss the survey topic

    Example Uses Pros Cons Considerations

    Early on in the survey project, a group of 6-12 EMT-Ps are brought in to discuss their current comfort level in treating children; from this discussion initial survey questions can be developed and tested.

    • Great for pre-testing your initial ideas
    • Generate, explore, and identify key ideas/concepts
    • Understand your population
    • Read non-verbal feedback
    • Helps with formal survey development
    • Can assist respondents with unfamiliar words or questions
    • Questions can be asked as they arise
    • Explore related and unanticipated topics as they arise
    • Bias / Small Group
    • Quality of data influenced by the moderator
    • Used to identify themes and capture ideas.
    • Some may feel reluctant to share personal information or beliefs.
    • Needed trained interviewers or moderators
    • Unable to make major decisions based on the information
    • Can't necessarily generalize the findings to the population
    • Must find a suitable place to conduct interview
    • Avoid jumping to conclusions
    • Very broad questions, often with open-ended answers and limited prompting
    • Avoid Yes, No
    • Not a form of group interviewing
    • 6 to 12 people ideal

    COSTS:

    • Training
    • Travel for participants or moderator
    • Space
    • Incentives
    • AV & transcription costs

Telephone Interviews

  • Telephone Interviews

    General Description:

    Conduct interviews over the telephone with scripted questions and prompts or computer assisted

    Example Uses Pros Cons Considerations

    Calling different organizations to assess the inter-facility communication between various care sites

    • Able to use computer-assisted interview software (CATI) to help with prompts and survey navigation
    • Can explore options with respondents
    • Can assist respondents with unfamiliar words or questions
    • Fast method
    • Good for sensitive information
    • Non-response may be high if there is no obvious benefit to participation
    • Sales calls often masquerade as "research" calls (caller-id)
    • Need trained interviewers
    • Is the respondent reachable by phone?
    • If using computer-assisted interviews, will need technical knowledge
    • Up-to-date phone numbers
    • Schedule for reaching respondents

    COSTS:

    • Training
    • Telephone charges
    • Computer and technical expertise
    • Incentives
Back to Articles