Visual discrete format: an alternative to Likert-type formats of survey items sensitive enough to measure small changes in stable constructs such as self-concept in science

Visual discrete format: an alternative to Likert-type formats of survey items sensitive enough to measure small changes in stable constructs such as self-concept in science

Автор: Daniel Solis, David Hutchinson, Nancy Longnecker

Журнал: International Journal of Cognitive Research in Science, Engineering and Education @ijcrsee

Рубрика: Original research

Статья в выпуске: 2 vol.10, 2022 года.

Бесплатный доступ

A visual discrete format was developed for use in surveys as an alternative to a Likert-type format to measure changes in a relatively stable construct before and after an intervention. Visitors to a science centre ranging in age from 8 years old upwards were asked to complete surveys that included a seven-item questionnaire scale on self-concept in science using either the Likert-type format (n=446) or the visual discrete format (n=375), before and after their visit. A new set of statements to assess self-concept in science were developed and validated so they could be conducted on either format. Matched responses were used to calculate internal consistency, standard deviation, confidence interval and percentage of missing values; these were all similar for both formats. In the visual discrete format, text labels were replaced by visual labels with a single image of different size for each response option. While a Likert-type format describes a level of agreement/disagreement with a specific item, the visual discrete format relates more to choosing the best reflection of the self in relation to that item. The Likert-type format included a set of emoji in its descriptions to appeal to younger participants. The visual discrete questionnaire scale detected a small increase with medium effect size in self-concept in science after the visit to the science centre while the Likert-type questionnaire scale did not detect any change. This suggests the proposed new format can not only be a viable and useful alternative, but potentially more sensitive under certain conditions.

Еще

Likert-type scale, visual discrete scale, visual discrete format, self-concept in science, scientific literacy, questionnaire

Короткий адрес: https://sciup.org/170198656

IDR: 170198656   |   DOI: 10.23947/2334-8496-2022-10-2-01-16

Список литературы Visual discrete format: an alternative to Likert-type formats of survey items sensitive enough to measure small changes in stable constructs such as self-concept in science

  • Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational psychology review, 15(1), 1-40. https://doi.org/10.1023/A:1021302408382
  • Borgers, N., Hox, J., & Sikkel, D. (2003). Response quality in survey research with children and adolescents: the effect of labeled response options and vague quantifiers. International Journal of Public Opinion Research, 15(1), 83-94. https:// doi.org/10.1093/ijpor/15.1.83
  • Brennan, R. L., & Lee, W.-C. (2018). True Score. In B. B. Frey (Ed.), The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation (pp. 1724-1728). Thousand Oaks, CA: SAGE Publications. https://dx.doi. org/10.4135/9781506326139
  • Creswell, J. W. (2009). Research Design: Qualitative, Quantitative, and Mixed Methods Ap-proaches. Thousand Oaks, CA: SAGE Publications.
  • De Cruz, H. (2006). Why are some numerical concepts more successful than others? An evolu-tionary perspective on the history of number concepts. Evolution and Human Behavior, 27(4), 306-323. https://doi.org/10.1016/j. evolhumbehav.2006.02.001
  • Diamond, J., Horn, M., & Uttal, D. H. (2016). Practical evaluation guide: Tools for museums and other informal educational settings. Lanham, MD: Rowman & Littlefield.
  • Etikan, I., & Bala, K. (2017). Sampling and sampling methods. Biometrics & Biostatistics International Journal, 5(6), 00149. https://doi.org/10.15406/bbij.2017.05.00149
  • Fenichel, M., & Schweingruber, H. (2010). Surrounded by Science: Learning science in informal environments. Washington, DC: The National Academies Press. https://doi.org/10.17226/12614
  • Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics. Los Angeles, CA: SAGE Publications.
  • Fowler, F. J., Jr. (2013). Survey research methods (5th ed.) Los Angeles, CA: SAGE Publications.
  • Friborg, O., Martinussen, M., & Rosenvinge, J. H. (2006). Likert-based vs. semantic differen-tial-based scorings of positive psychological constructs: A psychometric comparison of two versions of a scale measuring resilience. Personality and Individual Differences, 40(5), 873-884. https://doi.org/10.1016/j.paid.2005.08.015
  • Friedman, A. J. (Ed.) (2008). Framework for Evaluating Impacts of Informal Science Education Projects. National Science Foundation. Retrieved from https://www.informalscience.org/framework-evaluating-impacts-informal-science-education-projects
  • Friedman, H. H., & Amoo, T. (1999). Rating the rating scales. Journal of Marketing Management, 9(3), 114-123. Retrieved from https://ssrn.com/abstract=2333648
  • Hall, L., Hume, C., & Tazzyman, S. (2016). Five degrees of happiness: Effective smiley face likert scales for evaluating with children. Paper presented at the Proceedings of the The 15th International Conference on Interaction Design and Children, New York, NY. https://doi.org/10.1145/2930674.2930719
  • Hernández, S., Fernández, C., & Baptista, P. (2014). Metodología de la Investigación [Investigation Methodology] (6th ed.). Mexico City, Mexico: McGraw-Hill.
  • Hirschman, E. C. (1986). The effect of verbal and pictorial advertising stimuli on aesthetic, utilitarian and familiarity perceptions. Journal of Advertising, 15(2), 27-34. https://doi.org/10.1080/00913367.1986.10673002
  • Illeris, K. (2018). Contemporary theories of learning: learning theorists... in their own words. London, UK: Routledge. https:// doi.org/10.4324/9781315147277
  • International Association for the Evaluation of Educational Achievement. (1998). IEA Third International Mathematics and Science Study - Repeat: Student questionnaire main survey. Retrieved from https://timssandpirls.bc.edu/timss1999i/ pdf/BM2_StudentG.pdf
  • International Association for the Evaluation of Educational Achievement. (2018). IEA Trends in International Mathematics and Science Study 2003: Main survey student questionnaire grade 4. Retrieved from https://nces.ed.gov/timss/pdf/T19_ GR4_StudentQ_USA_Questionnaire.pdf
  • Jansen, M., Scherer, R., & Schroeders, U. (2015). Students’ self-concept and self-efficacy in the sciences: Differential relations to antecedents and educational outcomes. Contemporary Educational Psychology, 41, 13-24. https://doi.org/10.1016/j. cedpsych.2014.11.002
  • Jansen, M., Schroeders, U., & Lüdtke, O. (2014). Academic self-concept in science: Multidimen-sionality, relations to achievement measures, and gender differences. Learning and Individual Differences, 30, 11-21. https://doi. org/10.1016/j.lindif.2013.12.003
  • Kelley, C. M., & Lindsay, D. S. (1993). Remembering mistaken for knowing: Ease of retrieval as a basis for confidence in answers to general knowledge questions. Journal of Memory and Language, 32(1), 1-24. https://doi.org/10.1006/ jmla.1993.1001
  • Kind, P., Jones, K., & Barmby, P. (2007). Developing attitudes towards science measures. International Journal of Science Education, 29(7), 871-893. https://doi.org/10.1080/09500690600909091
  • Krishnamurthi, A., & Rennie, L. J. (2012). Informal science learning and education: definition and goals. Retrieved from https:// resources.informalscience.org/informal-science-learning-and-education-definition-and-goals
  • Kromann, C., Bohnstedt, C., Jensen, M., & Ringsted, C. (2010). The testing effect on skills learning might last 6 months. Advances in Health Sciences Education, 15(3), 395-401. https://doi.org/10.1007/s10459-009-9207-x
  • Lantz, B. (2013). Equidistance of Likert-type scales and validation of inferential methods using experiments and simulations. The Electronic Journal of Business Research Methods, 11(1), 16-28. Retrieved from https://academic-publishing.org/ index.php/ejbrm/article/view/1299/1262
  • Lee, J. D. (1998). Which kids can “become” scientists? Effects of gender, self-concepts, and perceptions of scientists. Social psychology quarterly, 61(3), 199-219. https://doi.org/10.2307/2787108
  • Likert, R. (1932). A technique for the measurement of attitudes. Archives de Psychologie [Archives of Psychology](140), 1-55.
  • Longnecker, N. (2016). An integrated model of science communication. Journal of Science Communication, 15, 1-12. https:// doi.org/10.22323/2.15050401
  • Longnecker, N., Elliot, J., & Gondwe, M. (2014). Inspiring Australia: An evaluation tool for science engagement activities. https://doi.org/10.13140/RG.2.1.3477.9683
  • MacKenzie, S. B., & Podsakoff, P. M. (2012). Common method bias in marketing: causes, mechanisms, and procedural remedies. Journal of retailing, 88(4), 542-555. https://doi.org/10.1016/j.jretai.2012.08.001
  • Martin, A. J., Durksen, T. L., Williamson, D., Kiss, J., & Ginns, P. (2016). The role of a muse-umbased science education program in promoting content knowledge and science motivation. Journal of Research in Science Teaching, 53(9), 1364-1384. https://doi.org/10.1002/tea.21332
  • McLeod, A., Pippin, S., & Wong, J. A. (2011). Revisiting the Likert scale: can the fast form ap-proach improve survey research? International Journal of Behavioural Accounting and Finance, 2(3-4), 310-327. https://doi.org/10.1504/ IJBAF.2011.045019
  • Mellor, D., & Moore, K. A. (2013). The use of Likert scales with children. Journal of pediatric psychology, 39(3), 369-379. https:// doi.org/10.1093/jpepsy/jst079
  • Miles, R. S., Alt, M., Gosling, D., Lewis, B., & Tout, A. (Eds.). (1988). The Design of Educational Exhibits (2nd ed.). New York, NY: Routledge. https://doi.org/10.1111/j.2151-6952.1985.tb01634.x
  • Munshi, J. (2014). A method for constructing Likert scales. SSRN Electronic Journal, 1-12. http://dx.doi.org/10.2139/ ssrn.2419366
  • Nagengast, B., Marsh, H. W., Scalas, L. F., Xu, M. K., Hau, K.-T., & Trautwein, U. (2011). Who took the “×” out of expectancy-value theory? A psychological mystery, a substantive-methodological synergy, and a cross-national generalization. Psychological science, 22(8), 1058-1066. https://doi.org/10.1177/0956797611415540
  • National Research Council. (2009). Learning science in informal environments: People, places, and pursuits (Vol. 32). Washington, DC: National Academies Press. https://doi.org/10.17226/12190
  • Nenkov, G. Y., & Scott, M. L. (2014). “So cute I could eat it up”: Priming effects of cute products on indulgent consumption. Journal of Consumer Research, 41(2), 326–341. https://doi.org/10.1086/676581
  • Organisation for Economic Cooperation and Development. (2009). PISA 2006 Technical Report. https://doi. org/10.1787/9789264048096-en
  • Osborne, J. W., & Blanchard, M. R. (2011). Random responding from participants is a threat to the validity of social science research results. Frontiers in Psychology, 1, 220. https://doi.org/10.3389/fpsyg.2010.00220
  • Otago Museum. (2019). Annual report 2018-2019. Retrieved from https://otagomuseum.nz/assets/Otago-Museum-Annual- Report-2018-19-low-res-25.11.19.pdf
  • Paulhus, D. L., & Vazire, S. (2007). The self-report method. In R. W. Robins, R. C. Fraley, & R. A. Krueger (Eds.), Handbook of research methods in personality psychology (pp. 224-239). New York, NY: Guilford Press.
  • Phan, N. Q., Blome, C., Fritz, F., Gerss, J., Reich, A., Ebata, T., . . . Ständer, S. (2012). As-sessment of pruritus intensity: prospective study on validity and reliability of the visual analogue scale, numerical rating scale and verbal rating scale in 471 patients with chronic pruritus. Acta dermato-venereologica, 92(5), 502-507. https://doi.org/10.2340/00015555- 1246
  • Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method bi-ases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879-903. https://doi. org/10.1037/0021-9010.88.5.879
  • Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539-569. https://doi.org/10.1146/annurev-psych-120710-100452
  • Reynolds-Keefer, L., Johnson, R., Dickenson, T., & McFadden, L. (2009). Validity issues in the use of pictorial Likert scales. Studies in Learning, Evaluation, Innovation and Development, 6(3), 15-25.
  • Sasson, I. (2014). The role of informal science centers in science education: attitudes, skills, and self-efficacy. JOTSE, 4(3), 167-180. https://dx.doi.org/10.3926/jotse.123
  • Schwarz, N. (1999). Self-reports: how the questions shape the answers. American Psychologist, 54(2), 93-105. https://doi. org/10.1037/0003-066X.54.2.93
  • Scott, J., Brynin, M., & Smith, R. (1995). Interviewing children in the British household panel survey. In I. Stewart & R. Vaitilingam (Eds.), Advances in family research (pp. 259-266). Essex, UK: University of Essex.
  • Şentürk, E., & Özdemir, Ö. F. (2014). The effect of science centres on students’ attitudes towards science. International Journal of Science Education, Part B, 4(1), 1-24. https://doi.org/10.1080/21548455.2012.726754
  • Siersdorfer, S., Minack, E., Deng, F., & Hare, J. (2010). Analyzing and predicting sentiment of images on the social web. Paper presented at the 18th ACM international conference on multimedia, Firenze, Italy. https://doi. org/10.1145/1873951.1874060
  • Solis, D. H. (2020). Exploration and discovery: learning at a science centre. (Doctor of Philosophy), University of Otago, New Zealand. Retrieved from https://ourarchive.otago.ac.nz/handle/10523/10160
  • Solis, D. H., Hutchinson, D., & Longnecker, N. (2021). Formal Learning in Informal Settings—Increased Physics Content Knowledge After a Science Centre Visit. Frontiers in Education, 6(322). https://doi.org/10.3389/feduc.2021.698691
  • Stange, M., Barry, A., Smyth, J., & Olson, K. (2018). Effects of smiley face scales on visual processing of satisfaction questions in web surveys. Social Science Computer Review, 36(6), 756-766. https://doi.org/10.1177/0894439316674166
  • Stocklmayer, S. M., & Bryant, C. (2012). Science and the public—What should people know? International Journal of Science Education, Part B, 2(1), 81-101. https://doi.org/10.1080/09500693.2010.543186
  • Sturm, H., & Bogner, F. X. (2010). Learning at workstations in two different environments: A museum and a classroom. Studies in Educational Evaluation, 36(1-2), 14-19. https://doi.org/10.1016/j.stueduc.2010.09.002
  • van Laerhoven, H., van der Zaag-Loonen, H., & Derkx, B. H. (2004). A comparison of Likert scale and visual analogue scales as response options in children’s questionnaires. Acta paediatrica, 93(6), 830-835. https://doi. org/10.1111/j.1651-2227.2004.tb03026.x
  • Venville, G., Rennie, L. J., Hanbury, C., & Longnecker, N. (2013). Scientists reflect on why they chose to study science. Research in Science Education, 43(6), 2207-2233. https://doi.org/10.1007/s11165-013-9352-3
  • Weijters, B., Geuens, M., & Baumgartner, H. (2013). The effect of familiarity with the response category labels on item response to Likert scales. Journal of Consumer Research, 40(2), 368-381. https://doi.org/10.1086/670394
  • Wilkins, J. L. (2004). Mathematics and science self-concept: An international investigation. The Journal of Experimental Education, 72(4), 331-346. https://doi.org/10.3200/JEXE.72.4.331-346
  • Wittkower, D. E. (2012). On the origins of the cute as a dominant aesthetic category in digital culture. In T. W. Luke & J. Huninger (Eds.), Putting Knowledge to Work and Letting Information Play (pp. 167-175). Rotterdam, The Netherlands: Sense Publishers. https://doi.org/10.1007/978-94-6091-728-8_13
  • Worcester, R. M., & Burns, T. R. (1975). Statistical examination of relative precision of verbal scales. Journal of the Market Research Society, 17(3), 181-197.
Еще
Статья научная
QR-код для установки приложения SciUp Наведите камеру и скачайте бесплатное приложение SciUp