Social Science Computer Review

Volume 28, No. 3

Fall 2010


Special Issue on Mobile Surveys

Guest editor: Michael Bosnjak


‘N the Network'? Using Internet Resources for Predicting Cell Phone Number Status / Trent D. Buskirk, Mario Callegaro, Kumar Rao

Abstract: Despite higher hit rates for cell phone samples, inefficiencies in processing calls to these numbers relative to landline numbers continue to be documented in the U.S. literature.  In this study we propose one method for using cell phone provider information and Internet resources for validating number status. Specifically, we describe how we used “in network” options available from three major providers’ Websites to determine the validity of cell phone numbers. We tested differences in working number rates among valid and non valid numbers against a normal processing control group and determined that the working number rate among valid numbers was approximately 14 percentage points higher than the working number rate of the comparison group. This process also shows promise in reducing the effort required to determine working status and may provide a basis for developing screening tools for cell phones that capitalize on resources that are unique to this technology.


Coverage Bias in Variances, Associations, and Total Error from Exclusion of the Cell Phone-Only Population in the U.S. / Andy Peytchev, Lisa R. Carley-Baxter, Michele C. Black

Abstract: While landline telephone household surveys often draw inference about the general population, a proportion with only cell phones is excluded.  In the U.S. like in much of the world, this proportion is substantial and increasing, providing potential for coverage bias.  Studies have looked at bias in means, but undercoverage can affect other essential statistics.  The precision of point estimates can be biased, leading to erroneous conclusions.  Research examining multivariate relationships will be further affected by bias in associations. A national landline telephone survey was conducted, followed by a survey of adults with only cell phones.  In addition to estimates of means and proportions, differences were also found for variances and associations.  Bias in some point estimates was reduced through poststratification, but became larger and in opposite direction for others.  Different uses of survey data can be affected by omitting the cell-only population, while reliance on postsurvey adjustments can be misleading.


Mobile phones in an environment of competing survey modes: Applying metric for evaluation of costs and errors / Vasja Vehovar, Nejc Berzelak, Katja Lozar Manfreda

Abstract: In recent years, mobile phones are becoming an increasingly important component in survey data collection. This holds true for self-administered questionnaires and particularly for interview surveys, where mobile phones enforce the combination with other survey modes. However, whether to include mobile phones in a particular survey design depends on complex cost-error relationships. To address this issue we elaborate a metric - based on a product of costs and estimates of survey errors – that is then used for post-survey comparison of design alternatives. We illustrate this approach with a simulation study using parameters from empirical research. The results show that such evaluation can potentially change the selection of the preferred design option compared to situations where only some of the components (e.g. response rate, non-response bias etc.) are used for evaluation. More specifically, the decision about inclusion of mobile phones predominantly depends on their bias-removing potential, while it is much less sensitive to changes in costs and other error parameters.



Experiments in Mobile Web Survey Design: Similarities to Other Modes and Unique Considerations / Andy Peytchev, Craig A. Hill

Abstract: Self-administered surveys can be conducted on mobile web-capable devices, yet these devices have unique features that can affect response processes.  Ninety-two adults were randomly selected and provided with mobile devices to complete weekly web surveys.  Experiments were designed to address three main objectives.  First, we test fundamental findings found robust across other modes, manipulating question order and scale frequencies, but whose impact may be diminished in mobile web surveys largely related to the device.  Second, we test findings from experiments in (computer-administered) web surveys, altering the presentation of images and the number of questions per page.  Third, we experiment with the unique display, navigation, and input methods, through the need to scroll, the vertical vs. horizontal orientation of scales, and the willingness to provide open-ended responses.  While most findings from other modes are upheld, the small screen and keyboard introduce undesirable differences in responses.


Reaching the mobile respondent: Determinants of high-level mobile phone use among a high-coverage group / Christoph Burger, Valentin Riemer, Jürgen Grafeneder, Bianca Woisetschläger, Dragana Vidovic, & Andreas Hergovich

Abstract: The aim of this study is to identify the key determinants of high-level mobile phone use in a high-coverage target group by replicating an Australian study conducted by Walsh and White (2007). Factors predicting high-level mobile phone use and relations between self- and prototypical identity are investigated by using an extended version of the theory of planned behavior (TPB). A total of 215 Austrian university students participated by completing 2 questionnaires, 1 week apart. The first questionnaire assessed attitude, subjective norm, perceived behavioral control, and intention as well as self- and prototypical identity variables. The second questionnaire assessed mobile phone use in the previous week. It was found that all 3 TPB variables significantly predicted intention. The addition of identity variables improved the model with self-identity becoming the second strongest overall predictor. In terms of predicting high-level mobile phone behavior, intention and subjective norm, but not PBC, were significant predictors.


Understanding the Willingness to Participate in Mobile Surveys: Exploring the Role of Utilitarian, Affective, Hedonic, Social, Self-Expressive, and Trust-Related Factors / Michael Bosnjak, Gottfried Metzger, & Lorenz Gräf

Abstract: Mobile technology offers a promising means to collect survey data, though the factors that influence people’s willingness to participate in mobile surveys and their actual participation remain unknown. To identify these factors, this study considers six conceptually distinct influences that may relate to the propensity to participate in mobile surveys. Some of them affect technology acceptance and usage of (mobile) technology in general; another set comes from studies of participation in computer-assisted surveys. The proposed unified framework encompasses utilitarian, affective, hedonic, social, self-expressive, and trust-related factors. An empirical study suggests that this framework explains the intention to participate and actual participation well, though of the six factors, hedonic, affective, self-expressive, and trust-related ones are most influential. Utilitarian aspects and beliefs about perceived social pressure to participate do not play significant roles. The authors discuss the practical implications of these results and outline some further research avenues.


Non-Symposium Article


The Mode Effect in Mixed-Mode Surveys: Mail and Web Surveys / Bengü Börkan

Abstract: Web surveys can suffer from their non-random nature (coverage error) and low response rate (non-response error). Therefore, web surveys should be supported by mail survey to eliminate these problems. However, using different survey methods together may introduce another problem: the mode effect.  This experimental study investigated the mode effect between two survey modes. A randomly selected group of one thousand five hundred teachers were assigned to two experimental groups, one of which received mail surveys, while the other received web surveys. Non-respondents in both groups were followed up with the opposite mode.  Overall, results show that there is no mode effect between mail surveys and web surveys on psychometric quality of the rating scales and data quality (item non-response rate) of the survey except regarding respondent’ age and unit-response rate. Our findings indicate that web surveys had a substantially lower unit-response rate than mail surveys, and that web survey respondents are significantly younger than mail survey respondents.


Reports and Communications


New technology in developing countries: A critique of the one-laptop-per-child programme / Jeffrey James

Abtract: The OLPC project offers no rationale for its view that there should be no sharing in schools. It is certainly not because this view requires no defense. On the contrary I show that the programme causes so much to be invested in computers that other educational inputs are entirely neglected and in some cases this is also true of sectors other than education. The OLPC requires poor countries to use fewer students per computer than is recommended even in the developed countries. I argue, by contrast, in favour of what is defined as a balanced pattern of sharing that reflects the level of per capita income in poor relative to rich countries. The higher is per capita income the less is the need to rely on sharing arrangements and conversely.


Ag09. A computer program for interrater agreement for judgments / Roel Popping

Abstract: This text describes a computer program that allows computing the interrater agreement index Scott’s pi for at least two ratings per object. The program allows using weights; therefore the user is not restricted to data on a nominal level of measurement. If wanted, it is possible to compute the agreement per category.


Book Review


The Web’s Awake: An Introduction to the Field of Web Science and the Concept of Web Life, by Philip Tetlow / Reviewed by Teresa Torres-Coronas