Students’ evaluation of teaching at a large Italian university: measurement scale validation


Abstract


The paper aims to verify the measurement capacity of the tool of teaching assessment at the University of Padua, Italy. The study is part of a project of improvement of the academic education innovation and quality of academic teaching: an evaluative research approach allows, indeed, reflection on teaching practice useful to share problems and find common solutions. The focus of this work is on contents and characteristics of statistical validty and reliability of the instrument used at Padua University, in the online survey to measure students' opinions on didactic activities (first-cycle, second-cycle and single-cycle degree courses).

DOI Code: 10.1285/i20705948v10n1p93

Keywords: validity, reliability, dimensionality, didactic activity evaluation, higher education

References


Aleamoni, L. M. (1999). Student rating myths versus research facts from 1924 to 1998. Journal of Personnel Evaluation in Education, 13, 153–166.

Apodaca, P., & Grad, H. (2005). The dimensionality of student ratings of teaching: Integration of uni- and multidimensional models. Studies in Higher Education, 30, 723–748.

Barnes, D., Engelland, B., Matherne, C., Martin, W., Orgeron, C., & Ring, J. (2008). Developing a psychometrically sound measure of collegiate teaching proficiency. College Student Journal, 42, 199-213.

Bassi, F. (2010). Experiential goods and customer satisfaction: an application to movies. Quality Technology & Quantitative Management, 7, 51-67.

Benton, S.L., & Cashin, W.E. (2012). Student ratings of teaching: A summary of research and literature. IDEA Paper n.50. Manhattan KS: IDEA Center.

Beran, T., & Violato, C. (2005). Ratings of university teacher instruction: How much do student and course characteristics really matter? Assessment and Evaluation in Higher Education, 30, 593–601.

Burdsal, C. A., & Harrison, P. D. (2008). Further evidence supporting the validity of both a multidimensional profile and an overall evaluation of teaching effectiveness. Assessment & Evaluation in Higher Education, 33, 567–576.

Cashin, W. E., & Perrin, P. B. (1978). Description of IDEA Standard Form Data Base IDEA. Technical Report No. 4. Manhattan, KS: Center for Faculty Evaluation and Development in Higher Education.

Centra, J. A. (1998). Development of The Student Instructional Report II. Princeton, NJ: Educational Testing Service.

Centra, J. A., & Gaubatz, N. B. (2000). Is there gender bias in student evaluations of teaching? The Journal of Higher Education, 71, 17–33.

Chalmers, D. (2007). An agenda for teaching and learning in universities. Sidney: The Carrick Institute for Learning and Teaching in Higher Education.

Cheung, D. (2000). Evidence of a single second-order factor in student ratings of teaching. Structural Equation Modeling, 7, 442–460.

Churchill, G.A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16, 64-73.

Clark, L.A., & Watson, D. (1995). Constructing validity: basic issues in objective scale development. Psychological Assessment, 7, 309-319.

Clayson, D. E. (2009). Student evaluations of teaching: Are they related to what stu¬dents learn? A meta-analysis and review of the literature. Journal of Marketing Education, 31, 16–30.

Cronbach, L.J. (1951). Coefficient Alpha and the internal structure of tests. Psychometrika, 16, 297-334.

Dalla Zuanna, G., Bassi F., Clerici, R., Paccagnella, O., Paggiaro, A., Aquario D., Mazzuco C., Martinoia, S., Stocco, C., & Pierobon, S. (2015). Tools for teaching assessment at Padua University: role, development and validation. PRODID Project (Teacher professional development and academic educational innovation) (Report of Research Unit n.3), Padua: Department of Statistical Sciences, University of Padua.

De Vellis, R. F. (1991). Scale Development. Theory and Applications. London: Sage.

Domenech, F., & Descals, A. (2003). Evaluation of the University teaching / learning process for the improvement of quality in higher education. Assessment and evaluation in higher education, 28, 165-178.

European Students’ Union (2015). Bologna with Student Eyes. Time to meet the expectations from 1999. Brussels: ESU.

European University Association (2006). Quality Culture in European Universities: A bottom-up approach. Brussels: EUA.

Ginns, P., Prosser, M., & Barrie, S. (2007). Students’ perceptions of teaching quality in higher education: The perspective of currently enrolled students. Studies in Higher Education, 32, 603-615.

Goldstein, G. S., & Benassi, V. A. (2006). Students’ and instructors’ beliefs about excellent lecturers and discussion leaders. Research in Higher Education, 47, 685–707.

Hair, J., Black, W.C., Anderson, R.E, & Tatham R.L. (2006) Multivariate Data Analysis. New Jersey: Prentice Hall.

Harrison, P., Douglas, D., & Burdsal, C. (2004). The relative merits of different types of overall evaluations of teaching effectiveness. Research in Higher Education, 45, 311–323.

Harvey, L., & Green, D. (1993). Defining quality. Assessment and Evaluation in Higher Education, 18, 8-35.

Harvey, L. (2006). Understanding quality. In L. Purser (Ed.), EUA Bologna Handbook: Making Bologna work. Brussels: European University Association and Berlin: Raabe, http://www.bologna-handbook.com/

Kember, D., Jenkins, W., & Kwok, C.N. (2004). Adult students’ perceptions of good teaching as a function of their conceptions of learning—Part 2. Implications for the evaluation of teaching. Studies in Continuing Education, 26, 81–97.

Kember, D., & Leung, D. (2008). Establishing the validity and reliability of course evaluation questionnaires. Assessment & Evaluation in Higher Education, 33, 341–353.

Litwin, M.S. (1995). How to Measure Survey Reliability and Validity. New York: Sage.

Malhotra, N.K. (1999). Marketing Research. London: Prentice Hall.

Marsh, H. W. (1982). SEEQ: A reliable, valid and useful instrument for collecting students’ evaluations of university teaching. British Journal of Educational Psychology, 52, 77–95.

Marsh, H. W. (1987). Student’s evaluations of university teaching: Research findings, methodological issues, and directions for further research. International Journal of Educational Research, 11, 253–388.

Marsh, H. W. (2007). Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 319–383). New York: Springer.

Marsh, H. W., Muthèn, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16, 439–476.

Marsh, H. W., & Roche, L A. (1997). Making students’ evaluations of teaching effec¬tiveness effective: The critical issues of validity, bias and utility. American Psychologist, 52, 1187–1197.

Mortelmans, D., & Spooren, P. (2009). A revalidation of the SET37-questionnaire for student evaluations of teaching. Educational Studies, 35, 547–552.

Newton J. (2007), What is quality?, in European University Association (a cura di), Embedding quality culture in higher education. A selection of papers from the 1st European Forum for Quality Assurance, Brussels.

Nunnally, J.C. (1978), Psychometric Theory. New York: McGraw-Hill.

Onwuegbuzie, A. J., Witcher, A. E., Collins, K. M. T., Filer, J. D., Wiedmaier, C. D., & Moore, C. W. (2007). Students’ perceptions of characteristics of effective college teachers: A validity study of a teaching evaluation form using a mixed-methods analysis. American Educational Research Journal, 44, 113–160.

Pozo-Munoz, C., Rebolloso-Pacheco, E., & Fernandez-Ramirez, B. (2000). The “Ideal Teacher”. Implications for student evaluations of teaching effectiveness. Assessment & Evaluation in Higher Education, 25, 253–263.

Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16, 129-150.

Rindermann, H. & Schofield, N. (2001). Generalizability of multidimensional student ratings of university instruction across courses and teachers. Research in Higher Education, 42, 377-399.

Roche, L., & Marsh, W. H. (2000). Multiple dimensions of University teacher self-concept: construct validation and the influence of students’ evaluations of teaching. Instructional Science, 28, 439-468.

Saroyan, A., Amundsen, C. (2001). Evaluating University teaching: time to take stock. Assessment and Evaluation in Higher Education, 26, 341-353.

Semeraro, R. (2006a). Paradigmi scientifici, rivisitazioni metodologiche, approcci multidimensionali. Milano: FrancoAngeli.

Semeraro, R. (2006b) (Ed.). Valutazione e qualità della didattica universitaria. Le prospettive nazionali e internazionali. Milano: FrancoAngeli.

Semeraro, R. (2006c) (Ed.). La valutazione della didattica universitaria. Docenti e studenti protagonisti in un percorso di ricerca. Milano: FrancoAngeli.

Smith, S. W., Yoo, J. H., Farr, A. C., Salmon, C. T., & Miller, V. D. (2007). The influ¬ence of student sex and instructor sex on student ratings of instructors: Results from a college of communication. Women’s Studies in Communication, 30, 64–77.

Spooren, P. (2010). On the credibility of the judge. A cross-classified multilevel anal¬ysis on student evaluations of teaching. Studies in Educational Evaluation, 36, 121–131.

Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83 (4), 598-642.

Svinicki, M., & McKeachie, W. J. (2011). McKeachie’s teaching tips: Strategies, research, and theory for college and university teachers (13th ed.). Belmont, CA: Wadsworth.

Theall, M. & Franklin, J. (2007) (Eds.). Student ratings of instruction: Issues for improving practice: New Directions for Teaching and Learning. San Francisco: Jossey-Bass.

Toland, M. & de Ayala, R.J. (2005). A multilevel factor analysis of students'evaluations of teaching. Educational and Psychological Measurement, 65, 272-296.

Zabaleta, F. (2007). The use and misuse of student evaluations of teaching. Teaching in Higher Education, 12 (1), 55–76.


Full Text: pdf
کاغذ a4

Creative Commons License
This work is licensed under a Creative Commons Attribuzione - Non commerciale - Non opere derivate 3.0 Italia License.