Journal of Research in Science, Mathematics and Technology Education

Measuring Opportunity Cost in Statistics Using Evaluative Space Grid Items: Results from a Pilot Study

Douglas Whitaker 1 * ,
Joseph Barss 1,
Bailey Drew 2
1 Mount Saint Vincent University2 Dalhousie University* Corresponding Author
Download PDF
Journal of Research in Science, Mathematics and Technology Education, Volume 5, Issue 1, January 2022, pp. 11-46
OPEN ACCESS VIEWS: 824 DOWNLOADS: 571 Publication date: 15 Jan 2022
ABSTRACT
Challenges to measuring students’ attitudes toward statistics remain despite decades of focused research. Measuring the expectancy-value theory (EVT) Cost construct has been especially challenging owing in part to the historical lack of research about it. To measure the EVT Cost construct better, this study asked university students to respond to items using both a Likert-type response and an Evaluative Space Grid (ESG)- type response. ESG items enable bivariate responses in a single item and permit distinguishing among two different types of neutral attitudes: indifferent and ambivalent. This pilot study evaluates the appropriateness of ESG-type items for measuring the EVT Cost construct by analyzing student response patterns to ESG-type items and comparing them with Likert-type items. Validity evidence is documented using descriptive statistics and graphs, correlations among items, and a trinomial hypothesis test. Internal consistency reliability indices are also reported. Friedman’s Test is used to compare the average response times for items of different types. Results indicate that students can meaningfully respond to ESG-type items in ways that are similar to their Likert-type responses, that students respond to ESG-type items quicker with more practice, and that distinguishing among indifferent and ambivalent attitudes seems appropriate for the EVT Cost construct. These findings suggest that ESG-type items may provide new insights not possible with Likert-type items but also that more research should be conducted to better understand their advantages and disadvantages within statistics education.
KEYWORDS
Evaluative Space Grid, Expectancy-value Theory, Statistics Attitudes, Statistics Education, Validity Evidence.
CITATION (APA)
Whitaker, D., Barss, J., & Drew, B. (2022). Measuring Opportunity Cost in Statistics Using Evaluative Space Grid Items: Results from a Pilot Study. Journal of Research in Science, Mathematics and Technology Education, 5(1), 11-46. https://doi.org/10.31756/jrsmte.511
REFERENCES
  1. American Educational Research Association, American Psychological Association, & National Council on
  2. Measurement in Education. (2014). Standards for educational and psychological testing. American
  3. Educational Research Association. Retrieved from https://www.testingstandards.net/open-access-files.html
  4. Audrezet, A. (2014). L’ambivalence des consommateurs: Proposition d’un nouvel outil de mesure (Thèse de
  5. doctorat, Université Paris-Dauphine). Université Paris-Dauphine, Paris. Retrieved from https://tel.archivesouvertes.fr/tel-01084068/document
  6. Audrezet, A., Olsen, S. O., & Tudoran, A. A. (2016). The GRID scale: A new tool for measuring service mixed
  7. satisfaction. Journal of Services Marketing, 30(1), 29–47. https://doi.org/10.1108/JSM-01-2015-0060
  8. Audrezet, A., & Parguel, B. (2018). Using the evaluative space grid to better capture manifest ambivalence in
  9. customer satisfaction surveys. Journal of Retailing and Consumer Services, 43, 285–295.
  10. https://doi.org/10.1016/j.jretconser.2018.04.008
  11. Bandalos, D. L. (2018). Measurement theory and applications for the social sciences. Guilford.
  12. Bian, G., McAleer, M., & Wong, W.-K. (2011). A trinomial test for paired data when there are many ties.
  13. Mathematics and Computers in Simulation, 81(6), 1153–1160.
  14. https://doi.org/10.1016/j.matcom.2010.11.002
  15. Borriello, A. (2017). The role of attitudes in determining individual behavior in transportation—From psychology to
  16. discrete choice modeling (Dissertation, Università della Svizzera Italiana). Università della Svizzera
  17. Italiana, Lugano, Switzerland. Retrieved from http://doc.rero.ch/record/306643/files/2018ECO001.pdf
  18. Cacioppo, J. T., & Berntson, G. G. (1994). Relationship between attitudes and evaluative space: A critical review,
  19. with emphasis on the separability of positive and negative substrates. Psychological Bulletin, 115(3), 401–
  20. 423. https://doi.org/10.1037/0033-2909.115.3.401
  21. Cacioppo, J. T., Berntson, G. G., Norris, C. J., & Gollan, J. K. (2012). The evaluative space model. In P. A. M. V. Lange, A. W. Kruglanski, & E. T. Higgins (Eds.), Handbook of theories of social psychology: Volume One (pp. 50–72). SAGE.
  22. Cacioppo, J. T., Gardner, W. L., & Berntson, G. G. (1997). Beyond bipolar conceptualizations and measures: The
  23. case of attitudes and evaluative space. Personality and Social Psychology Review, 1(1), 3–25.
  24. https://doi.org/10.1207/s15327957pspr0101_2
  25. Chew, P. K. H., Dillon, D. B., & Swinbourne, A. L. (2018). An examination of the internal consistency and structure
  26. of the Statistical Anxiety Rating Scale (STARS). PLOS ONE, 13(3), e0194195.
  27. https://doi.org/10.1371/journal.pone.0194195
  28. Conover, W. J. (1999). Practical nonparametric statistics (3rd ed). Wiley.
  29. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
  30. Retrieved from http://link.springer.com/article/10.1007/BF02310555
  31. Cruise, R. J., Cash, R. W., & Bolton, D. L. (1985). Development and validation of an instrument to measure
  32. statistical anxiety. Proceedings of the Section on Statistical Education, American Statistical Association,
  33. 92–98.
  34. DeVellis, R. F. (2017). Scale development: Theory and applications (Fourth edition). SAGE.
  35. Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–
  36. 132. Retrieved from http://outreach.mines.edu/cont_ed/Eng-Edu/eccles.pdf
  37. Eccles, J. S., & Wigfield, A. (2020). From expectancy-value theory to situated expectancy-value theory: A
  38. developmental, social cognitive, and sociocultural perspective on motivation. Contemporary Educational
  39. Psychology, 61, 101859. https://doi.org/10.1016/j.cedpsych.2020.101859
  40. Eccles (Parsons), J., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, C. (1983). Expectancies, values, and academic behaviors. In J. T. Spence (Ed.), Achievement and achievement motives: Psychological and sociological approaches. W.H. Freeman. Retrieved from http://web.archive.org/web/20170701031033/http://www.rcgd.isr.umich.edu/garp/articles/ecclesparsons83b
  41. .pdf
  42. Flake, J. K. (2012). Measuring cost: The forgotten component of expectancy-value theory (Doctoral Dissertation,
  43. James Madison University). Retrieved from http://commons.lib.jmu.edu/cgi/viewcontent.cgi?article=1221&context=master201019
  44. Flake, J. K., Barron, K. E., Hulleman, C., McCoach, B. D., & Welsh, M. E. (2015). Measuring cost: The forgotten component of expectancy-value theory. Contemporary Educational Psychology, 41, 232–244.
  45. https://doi.org/10.1016/j.cedpsych.2015.03.002
  46. Gal, I., & Ginsburg, L. (1994). The role of beliefs and attitudes in learning statistics: Towards an assessment
  47. framework. Journal of Statistics Education, 2(2), 1–15. Retrieved from https://ww2.amstat.org/publications/jse/v2n2/gal.html
  48. Ganesalingam, S. (1994). A nonparametric test procedure for paired data sampling, in the presence of considerable
  49. number of ties. Proceedings of the 30th Annual Meeting of the Operational Research Society of New Zealand and 45th Annual Conference of the New Zealand Statistical Association, 384–389. Retrieved from http://www.thebookshelf.auckland.ac.nz/docs/NZOperationalResearch/conferenceproceedings/1994proceedings/ORSNZ-proceedings-1994-69.pdf
  50. Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255–282.
  51. https://doi.org/10.1007/BF02288892
  52. Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient
  53. alpha. Measurement and Evaluation in Counseling and Development, 34(3), 177–189.
  54. Irwing, P., & Hughes, D. J. (2018). Test development. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley
  55. handbook of psychometric testing: A multidisciplinary reference on survey, scale, and test development
  56. (First Edition, pp. 3–48). Wiley.
  57. Jarek, S. (2012). mvnormtest: Normality test for multivariate variables (Version R package version 0.1-9). Retrieved from https://CRAN.R-project.org/package=mvnormtest
  58. Jiang, Y. (2015). The role of perceived cost in students’ academic motivation and achievement (Korea University).
  59. Retrieved from http://dcollection.korea.ac.kr/public_resource/pdf/000000060090_20191030101628.pdf
  60. Jiang, Y., Rosenzweig, E. Q., & Gaspard, H. (2018). An expectancy-value-cost approach in predicting adolescent
  61. students’ academic motivation and achievement. Contemporary Educational Psychology, 54, 139–152. https://doi.org/10.1016/j.cedpsych.2018.06.005
  62. Kolde, R. (2019). pheatmap: Pretty Heatmaps (Version R package version 1.0.12). Retrieved from https://CRAN.Rproject.org/package=pheatmap
  63. Larsen, J. T., Norris, C. J., McGraw, A. P., Hawkley, L. C., & Cacioppo, J. T. (2009). The evaluative space grid: A
  64. single-item measure of positivity and negativity. Cognition & Emotion, 23(3), 453–480.
  65. https://doi.org/10.1080/02699930801994054
  66. Likert, R. (1933/1967). The method of constructing an attitude scale. In M. Fishbein (Ed.), Readings in attitude
  67. theory and measurement (pp. 90–95). Wiley. (Original work published 1932)
  68. Nolan, M. M., Beran, T., & Hecker, K. G. (2012). Surveys assessing students’ attitudes toward statistics: A
  69. systematic review of validity and reliability. Statistics Education Research Journal, 11(2), 103–123.
  70. Pearl, D. K., Garfield, J. B., delMas, R., Groth, R. E., Kaplan, J. J., McGowan, H., & Lee, H. S. (2012). Connecting
  71. research to practice in a culture of assessment for introductory college-level statistics. Retrieved from https://www.causeweb.org/cause/archive/research/guidelines/ResearchReport_2012.pdf
  72. Pedersen, T. L. (2021). ggforce: Accelerating “ggplot2” (Version R package version 0.3.3). Retrieved from https://CRAN.R-project.org/package=ggforce
  73. Pohlert, T. (2021). PMCMRplus: Calculate pairwise multiple comparisons of mean rank sums extended (Version R package version 1.9.0). Retrieved from https://CRAN.R-project.org/package=PMCMRplus
  74. R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical
  75. Computing. Retrieved from http://www.R-project.org/
  76. Ramirez, C., Schau, C., & Emmioğlu, E. (2012). The importance of attitudes in statistics education. Statistics
  77. Education Research Journal, 11(2), 57–71. Retrieved from https://iaseweb.org/documents/SERJ/SERJ11(2)_Ramirez.pdf
  78. Revelle, W. (2021). psych: Procedures for personality and psychological Research (Version 2.1.3). Northwestern
  79. University. Retrieved from https://cran.r-project.org/web/packages/psych/index.html
  80. Roberts, D. M., & Bilderback, E. W. (1980). Reliability and validity of a statistics attitude survey. Educational and
  81. Psychological Measurement, 40(1), 235–238. https://doi.org/10.1177/001316448004000138
  82. Schau, C. (1992). Survey of attitudes toward statistics (SATS-28). Retrieved from http://evaluationandstatistics.com/
  83. Schau, C. (2003). Survey of attitudes toward statistics (SATS-36). Retrieved from http://evaluationandstatistics.com/
  84. Schmitt, N. (1996). Uses and Abuses of Coefficient Alpha. Psychological Assessment, 8(4), 350–353. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.615.4053&rep=rep1&type=pdf
  85. Thibaut, J. W., & Kelley, H. H. (1959). The social psychology of groups. Wiley. Retrieved from
  86. https://archive.org/details/socialpsychology00thib
  87. Uebersax, J. S. (2006). Likert scales: Dispelling the confusion. Retrieved from Statistical Methods for Rater
  88. Agreement website: https://john-uebersax.com/stat/likert.htm
  89. Unfried, A., Kerby, A., & Coffin, S. (2018). Developing a student survey of motivational attitudes toward statistics. 2018 JSM Proceedings. Presented at the 2018 Joint Statistical Meetings, Vancouver, Canada. Retrieved from http://sdsattitudes.com/wp/wp-content/uploads/2021/08/Unfried-et-al_2018_Developing-a-Student-
  90. Survey-of-Motivational-Attitudes-Toward-Statistics.pdf
  91. Unfried, A., Posner, M., Bond, M., Kerby-Helm, A., Bolon, W., Whitaker, D., & Batakci, L. (2021, August). Why
  92. do we need yet aNOTHER instrument measuring student attitudes? Presented at the 2021 Joint Statistics
  93. Meetings, Virtual. Retrieved from http://sdsattitudes.com/wp/jsm2021/
  94. Wei, T., & Simko, V. (2017). R package “corrplot”: Visualization of a correlation matrix (Version 0.84). Retrieved
  95. from https://github.com/taiyun/corrplot
  96. Whitaker, D. (2021, June). Developing and revising the student survey of motivational attitudes toward statistics:
  97. Results from a pilot study. Presented at the Statistics Society of Canada 2021 Annual Meeting, Virtual.
  98. Whitaker, D., Drew, B., & Barss, J. (2021) GridItemTools: Grid item tools. R package version 0.0.12.
  99. https://github.com/douglaswhitaker/GridItemTools
  100. Whitaker, D. & Hebert, N. (2021). MVQuickGraphs: Quick multivariate graphs. R package version 0.1.6. https://github.com/douglaswhitaker/MVQuickGraphs
  101. Whitaker, D., Unfried, A., & Bond, M. (2019). Design and validation arguments for the Student Survey of
  102. Motivational Attitudes toward Statistics (S-SOMAS) instrument. In J. D. Bostic, E. E. Krupa, & J. C. Shih (Eds.), Assessment in mathematics education contexts: Theoretical frameworks and new directions (1st ed., pp. 120–146). Routledge.
  103. Whitaker, D., Unfried, A., & Bond, M.E. (in press). Challenges associated with measuring attitudes using the SATS
  104. family of instruments. Statistics Education Research Journal.
  105. Whitaker, D., & White, A. (2020, May). Measuring statistics attitudes and anxieties. Poster presented at The Fifth
  106. Biennial Electronic Conference on Teaching Statistics (eCOTS). Retrieved from https://www.causeweb.org/cause/ecots/ecots20/posters/2-05
  107. Wickham, H. (2009). Ggplot2: Elegant graphics for data analysis. Springer.
  108. Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemporary
  109. Educational Psychology, 25(1), 68–81. https://doi.org/10.1006/ceps.1999.1015
  110. Wigfield, A., & Eccles, J. S. (2020). 35 years of research on students’ subjective task values and motivation: A look
  111. back and a look forward. In Advances in Motivation Science (Vol. 7, pp. 161–198). Elsevier.
  112. https://doi.org/10.1016/bs.adms.2019.05.002
  113. Wigfield, A., Rosenzweig, E. Q., & Eccles, J. S. (2017). Achievement values: Interactions, interventions, and future
  114. directions. In A. J. Elliot, C. S. Dweck, & D. S. Yeager (Eds.), Handbook of competence and motivation:
  115. Theory and application (Second edition, pp. 116–134). Guilford Press.
  116. Wise, S. L. (1985). The development and validation of a scale measuring attitudes toward statistics. Educational and
  117. Psychological Measurement, 45(2), 401–405. Retrieved from http://epm.sagepub.com/content/45/2/401.short
LICENSE
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.