CSR can provide all services needed to prepare, moderate, and analyze focus groups for qualitative research or as part of the development of a survey questionnaire. We also implement them as a post-survey debriefing technique to provide a richer context for survey responses.
- CSR can print, assemble, and mail survey mailings for small to large scale survey projects. Highly-trained mail assembly staff follow rigorous mail assembly procedures that involve multiple quality control checks to ensure that all materials are assembled correctly.
- CSR implements thorough cleaning of sample files prior to mail survey administration to produce efficient, high-quality mailings and to ensure that address information meets USPS standards.
- CSR prepares letters and postcards according to best available survey methods and formats "respondent-friendly" mail survey questionnaires. Principles of respondent-friendly design include an interesting title page, easy to read fonts, and use of simple headings, clear instructions, and ample white space to create a visually appealing, easy to complete instrument.
We work with each collaborator to develop a mail survey data collection protocol that minimizes error and costs. This may include personalization, development of letters and postcards that encourage participation, use of multiple contacts, use of special delivery services (e.g., FedEx), use of additional modes of data collection (e.g., Web, telephone), and respondent incentives.
We track large complex survey processes using a unique secure Web interface that allows our collaborators and CSR staff to exchange information, run progress reports, and download and upload documents, samples, and data files. For smaller projects, we have a case management system that records information on all contact attempts. All returned questionnaires are carefully reviewed and logged. Reports are generated that provide details on every attempt made (e.g., advance letter, first mailing of questionnaire), including the date, time, and current or final outcome of sample cases. These reports are used to monitor data collection progress.
The Research Laboratory within the CSR is our telephone survey laboratory unit, located in Room 123 of the CSR 120 suite of the Smith Research Center building. We normally operate 8 recording-enabled calling stations and two reserved supervisor stations, with the ability to expand the number of calling and supervisor stations. We also conduct in-person interviews.
Our Research Lab manager and director work with collaborators and staff to develop an interviewing data collection protocol that maximizes response and minimizes errors and costs. This may include use of a mailed advance letter, development of a tailored survey introduction, interviewer training on respondent concerns specific to the study, strategic scheduling and determination of number of callbacks to maximize contact, and respondent and interviewer incentives.
We also provide stand-alone calling support either as a quick supplement to an external project or as part of an overall nonresponse reduction strategy. Where telephone numbers are available or searchable, interviewers can follow up with nonparticipating subjects to address potential concerns and encourage participation in a study (regardless whether the main study itself is administered by the lab or by telephone), identify problems with contact information, or inquire about specific item missing data.
We use the CASES 5.6.1 CATI (computer-assisted telephone interviewing) system developed by the Computer-Assisted Survey Methods Program of the University of California at Berkeley and were one of the first academic research facilities to implement this sophisticated system. CASES allows multiple versions of questionnaires in the same instrument so that an interviewer may easily switch between different language or respondent subpopulation versions. We also have the capacity to use the Web version of CASES to allow for interviewing at multiple sites. In addition to these extensive interviewing capabilities, CASES also provides a complete case management system. The results of every telephone call and all case management processes are recorded in paradata files. Details on every call made, including date, time, outcome, interviewer, and length of the call are generated into reports and are used to monitor telephone data collection progress.
Interviewer Hiring, Training, and Quality Control
- The Research Lab’s setting in an academic community provides access to a highly-skilled and motivated workforce. We apply rigorous hiring standards in the selection of our interviewing staff.
- The Research Lab has established an intensive interviewer training program that exceeds standards required by state and federal contracts. Pedagogically, the training program is designed to accommodate a range of learning styles, with individual self-guided PowerPoint presentations, written workbook exercises, quizzes, group work, one-on-one coaching from supervisors, and peer mentoring. New interviewing staff receive at least 9 hours of training in standardized interviewing, case management, oral presentation, and refusal avoidance techniques, 2 to 4 hours of study-specific training, and up to 4 hours of mock-interviewing practice, prior to making production calls on a project.
- Quality control monitoring of real-time audio and on-screen visuals is performed without the knowledge of the interviewer, by senior lab staff at a supervisor-to-interviewer ratio of 1:7. Interviewers are regularly monitored throughout their tenure in the Research Lab for quality control of interview data and continual improvement of interviewing skills: new interviewers are monitored at least twice per a work shift of 4 hours, while more experienced interviewers are typically monitored once per work shift. A single monitoring session consists of at least 20 substantive questionnaire items and up to 10 study introduction and case management events. Interviewers are provided immediate feedback by supervisors on performance and each monitoring session is entered into a database for tracking interviewer performance indicators. Overall, about 20% of all interviews are monitored.
CSR has been programming and designing Internet surveys and related forms and interfaces since 1998. We use ColdFusion, ASP.NET, SQL, Qualtrics, and REDCap to administer and capture these data. We work with each collaborator to develop a Web survey environment appropriate for each project and our methodologists are available to help guide sampling, questionnaire design, and population-tailored recruitment and nonresponse strategies to maximize response and minimize errors and costs.
- Interfaces: For large-scale and complex projects, we utilize internally-developed interactive SQL-based interfaces to manage survey materials and processing. These interfaces are used both internally and by our partners and have many interconnected areas and functionalities. For example, colleges and universities in the National Survey of Student Engagement (NSSE), a random sample survey of college undergraduates from 500-700 participating institutions across the US and Canada, which we annually administer, may directly upload recruitment letters and sample files onto our interfaces which then feed into several internal workflow databases. Interfaces also allow school administrators to set survey administration dates, report school break dates, and make special requests. Similar interfaces have been developed for other projects.
- Sample Management System: We have developed a sophisticated sample management system for online and multimodal survey projects that records contact attempt information while safeguarding the security of sensitive or identifying information. Reports are generated, providing paradata details such as the date, time, and outcome of a recruitment attempt, and are used internally to monitor data collection progress.
- Survey Questionnaires: We can program both simple questionnaires and highly-sophisticated instruments alike. For complex and longitudinal designs, we often use our own custom survey-programming tool. Some examples of its functionality include complex logic branching such as conjoint analysis logic, conditional wording based on auxiliary data in the sample or antecedent survey questions, and real-time data checks and edits. We also have extensive experience with Qualtrics, a commonly used survey software.
