June 18, 2015

Self-Report: We Can Do Better (And Are!)

The Religious Studies Project interview with Dr. Luke Galen conducted by Tommy Coleman was an excellent cross-section of some of the struggles which are occurring within the social sciences corner of religious studies, in particular the psychology of religion. As a field, we still have a long way to go in figuring out ways to both incorporate nonbelievers into our work as well as to signal when our findings only apply to a particular belief group, instead of all humans (ie. “Increased religiosity helps prevent recurring depression for religious believers” instead of “Religion prevents depression”). The idea that we need to explicitly include nonbelievers in our samples has begun to find solid ground, both in Dr. Galen’s work and others’ (e.g. Galen, 2012; Streib & Klein, 2013), but there have been some issues with developing this idea further. Dr. Galen alluded to one of the major issues in continuing to include nonbelievers, namely the increasing usage of the word “spirituality.” Does it include well-being and having a sense of meaning in life? Feelings of Awe and transcendence? Believing in Ghosts, angels, and demons? Yes, and this lack of clarity is a major problem for studies which try and link “spirituality” with mental health and well-being.

I strongly agree with Dr. Galen’s assertion that the amount of criterion contamination found in most discussions and measures of “spirituality” is problematic, and this point was well-highlighted in Tommy’s point about, “long walks on the beach.” A question that covers so much territory may not even be an accurate reflection of general well-being for people who prefer walking in the woods, let alone serve as a marker of the supernatural component implicit in “spirituality.” While I don’t think Dr. Galen presented a hard-experimentalist view completely dismissing self-report, the criterion contamination introduced by our fuzzy definition of “spirituality” and poorly-constructed self-report measures seem to be bundled up into a problem that exists for self-report measures in general. Just as in the study of moral reasoning, experimental designs which attempt to tap implicit beliefs risk ignoring the fact that humans also seem to be able to exert some conscious control over their beliefs and thus can’t be treated as simply heuristic machines (Cunningham et al., 2004; Turiel, 2008).

Instead, it seems best to attempt to fix the problem of poor self-report measures more directly. We can do this by making measures which don’t use double-barreled questions which nonbelievers can’t straightforwardly answer, explicitly addressing the issue of “supernatural spirituality,” and ensuring that aspects of the measure which tap more general well-being concepts are sufficiently differentiated from supernatural concepts. Additionally, to construct better measures we’ll need to include large enough samples of nonbelievers during all stages of scale development to ensure that the resulting measures are valid for both believers and nonbelievers.

I bring all of this up because there are already measures which have been (or are being) published which meet these criteria, so I can flagrantly advertise them. Cragun, Hammer, and Nielsen’s Nonreligious-Nonspirituality Scale (in press) addresses the problem of fuzzy-spirituality by clearly specifying that respondents should only respond in regards to their beliefs regarding the supernatural aspects of spirituality and not the more general well-being aspects. In addition, their scale was developed for use with believers as well and seems to validly measure the extent of their nonreligious and nonspiritual beliefs, allowing for comparisons between believers and nonbelievers which might not be feasible with “beach walking” measures of spirituality.

While Dr. Galen’s assertion that the well-being of nonbelievers has been underestimated due to incorrectly grouping them with believers who might be experiencing religious and/or spiritual struggles seems to be an accurate depiction of the literature at the moment, this also seems likely to be a problem of improperly interpreted self-report measures rather than with self-report in general. There is initial evidence pointing to a U-shaped curve of well-being related to the strength of a person’s (non)belief (Streib & Klein, 2013). Investigating this idea using the level of control afforded by in-lab experimental studies will be important, but it will also be important to leverage the generalizability of broad self-report studies. We just need a measure of “spiritual” struggles which actually works with the kinds of struggles which might point to lower levels of belief for both believers and nonbelievers.

At the risk of continuing to over-toot the horns of projects that I’m involved with, the Religious and Spiritual Struggles Scale (Exline, Pargament, Grubbs, & Yali, 2014) seems like it will work in that regard. While two of the sub-scales explicitly contain supernatural items, the scale is modular and our early analyses indicated that atheists experience less spiritual struggles than agnostics, when excluding the explicitly supernatural scales (Uzdavines, Bradley, & Exline, 2014). We are currently working on confirming that the scale is measurement invariant with fine-grained belief identification groups (ie. Atheists, Agnostics, Theists, etc) before investigating the link between non-supernatural “spiritual” struggles and well-being, but our early analyses show that it is invariant when considering nonbelievers and believers as two broad groups.

Which is all to say; those of us within psychology of religion who study secularity are privileged to be working in a time where secular beliefs and nonbelievers are starting to be taken seriously within the field as a whole. Maintaining a high level of rigor in the methodology we employ, while important in and of itself, is even more crucial because of the history of criterion contamination within the field that Dr. Galen discussed in this interview and in his own work. “Spirituality” is an overly broad term and, when interpreted incorrectly, can lead to conclusions that more religion leads to more well-being without considering that more nonreligion might also lead to more well-being. It will take much more work to shift the field towards accepting religious nonbelief as a discreet and important category, separate from religious belief even if we still need to clarify our terminology.

But rigorous does not only mean experimental. Self-report can provide interesting avenues of investigation, but more care needs to be taken in building self-report measures which minimize criterion contamination and allow nonbelievers to indicate their level of nonbelief or well-being without having to dance around double-barreled questions. Fortunately, the rapidly expanding breadth of research communities dedicated to investigating secularity should allow the field of secular studies to continue pooling ideas and methodology to illuminate the nature of nonbelief and nonbelievers.


Cragun, R. T., Hammer, J. H., & Nielsen, M. (in press). The Nonreligious-Nonspiritual Scale (NRNSS): Measuring Everyone from Atheists to Zionists. Science, Religion, and Culture.

Cunningham, W. A., Johnson, M. K., Raye, C. L., Gatenby, J. C., Gore, J. C., & Banaji, M. R. (2004). Separable neural components in the processing of black and white faces. Psychological Science, 15(12), 806–813.

Exline, J. J., Pargament, K. I., Grubbs, J. B., & Yali, A. M. (2014). The Religious and Spiritual Struggles Scale: Development and initial validation. Psychology of Religion and Spirituality, 6(3), 208–222. http://doi.org/10.1037/a0036465

Galen, L. W. (2012). Does religious belief promote prosociality? A critical examination. Psychological Bulletin, 138(5), 876–906. http://doi.org/10.1037/a0028251

Streib, H., & Klein, C. (2013). Atheists, Agnostics, and Apostates. In K. I. Pargament, J. J. Exline, & J. W. Jones (Eds.), APA handbook of psychology, peligion, and spirituality (Vol 1): Context, theory, and research (Vol. 1, pp. 713–728). Washington, DC, US: American Psychological Association.

Turiel, E. (2008). The Development of Children’s Orientations toward Moral, Social, and Personal Orders: More than a Sequence in Development. Human Development, 51(1), 21–39. http://doi.org/10.1159/000113154

Uzdavines, A., Bradley, D. F., & Exline, J. J. (2014). Struggle and the nonreligious: Do weaker forms of nonbelief increase susceptibility to spiritual struggle? In Religious and spiritual struggles: New research frontiers. La Mirada, CA.



Join the discussion

Your email address will not be published. Required fields are marked *