We tested the effectiveness of Faculty Institutes for Reforming Science Teaching IV (FIRST), a professional development program for postdoctoral scholars, by conducting a study of program alumni. Faculty professional development programs are critical components of efforts to improve teaching and learning in the STEM (Science, Technology, Engineering, and Mathematics) disciplines, but reliable evidence of the sustained impacts of these programs is lacking. We used a paired design in which we matched a FIRST alumnus employed in a tenure-track position with a non-FIRST faculty member at the same institution. The members of a pair taught courses that were of similar size and level. To determine whether teaching practices of FIRST participants were more learner-centered than those of non-FIRST faculty, we compared faculty perceptions of their teaching strategies, perceptions of environmental factors that influence teaching, and actual teaching practice. Non-FIRST and FIRST faculty reported similar perceptions of their teaching strategies and teaching environment. FIRST faculty reported using active learning and interactive engagement in lecture sessions more frequently compared with non-FIRST faculty. Ratings from external reviewers also documented that FIRST faculty taught class sessions that were learner-centered, contrasting with the teacher-centered class sessions of most non-FIRST faculty. Despite marked differences in teaching practice, FIRST and non-FIRST participants used assessments that targeted lower-level cognitive skills. Our study demonstrated the effectiveness of the FIRST program and the empirical utility of comparison groups, where groups are well matched and controlled for contextual variables (for example, departments), for evaluating the effectiveness of professional development for subsequent teaching practices.
FIRST Paired Teaching Practice Survey Data
The associated data file contains results from the Teaching Practice Survey which we designed for use in the Paired Study to assess the frequency with which faculty reported using active, learner-centered practices in their courses. This data set consists of data from 11 of the pairs. For complete survey questions and scales see Supplementary Materials S2 in the associated publication (Derting et al. 2016).
FIRST_paired_teaching_practice_survey_data.csv
FIRST Paired ATI Survey 22 Questions
The associated data file contains responses to 22 items of the Approaches to Teaching Inventory. See more about the 16-item and 22-item versions of the instrument in K. Trigwell, M. Prosser, Development and use of the approaches to teaching inventory. Educ Psychol Rev 16, 409-424 (2004).and in K. Trigwell, M. Prosser, P. Ginns, Phenomenographic pedagogy and a revised Approaches to teaching inventory. Higher Educ Res Dev 24, 349-360 (2005).This data file contains results from 20 complete faculty pairs using the 22-item ATI.
FIRST_Paired_ATI_22Q.csv
FIRST ETQ and SE Paired Study Surveys
The associated data file contains results from Paired Study Participant responses to the Experience of Teaching Questionnaire (ETQ). The ETQ is described in M. Prosser, K. Trigwell, Relations between perceptions of the teaching environment and approaches to teaching. Br. J. Educ. Psychol. 67, 25-35 (1997). Data set also includes results of Self Efficacy survey questions as described in S. Lindblom-Ylanne, K. Trigwell, A. Nevgi, P. Ashwin, How approaches to teaching are affected by discipline and teaching context. Stud Higher Educ 31, 285-298 (2006). This data set consists of data from 20 of the study pairs.
ETQ_and_SE_Paired_Study_Comparison.csv
LSQ Student Data Paired Study
The associated data file contains results from the Learning and Studying Questionnaire (LSQ) survey. See information for published questionnaire in: N. Entwistle, V. McCune, J. Hounsell, "Approaches to Studying and Perceptions of University Teaching-Learning Environments: Concepts, Measures and Preliminary Findings. Enhancing Teaching-Learning Environments in Undergraduate Courses Project," (2002). Rows are survey responses by individual students for the 10 scales of the LSQ survey. Data file contains student data from 13 instructor pairs.
LSQ_Student_Data_Paired_Study.csv
ETLQ Student Data Paired Study
The associated data file contains results from the Experiences of Teaching and Learning (ETLQ) survey. See information for both published questionnaires in: N. Entwistle, V. McCune, J. Hounsell, "Approaches to Studying and Perceptions of University Teaching-Learning Environments: Concepts, Measures and Preliminary Findings. Enhancing Teaching-Learning Environments in Undergraduate Courses Project," (2002). Rows are survey responses by individual students for the 19 scales of the ETLQ survey. Data file contains student responses from 11 instructor pairs.
ETLQ_Student_Data_Paired_Study.csv
FIRST Paired RTOP data
The associated data file contains results from the Reformed Teaching Observation Protocol by D. Sawada, "Reformed teacher education in science and mathematics: An evaluation of the Arizona Collaborative for Excellence in the Preparation of Teachers," (Arizona State University Document Production Services, Tempe, AZ, 2003). This data set consists of data from 17 complete pairs and two incomplete pairs from our paired study.
FIRST_Paired_RTOP_data.csv
Percent Average Bloom's Taxonomy Goals
The associated data file contains the average of the percent Bloom score (Bloom’s Taxonomy categories 1-6) for course goals as stated in syllabi. We scored stated course goals into cognitive skill categories and calculated the proportion of course goals assigned to each Bloom’s category (see Derting et al. 2016 for details). This data set consists of data from 14 FIRST Faculty and 12 non-FIRST Faculty.
PercentAverageBloomsGoals.csv
Percent Average Blooms Taxonomy for Assessments
The associated data file contains the average of the percent Bloom score (Bloom’s Taxonomy categories 1-6) for assessment questions. We scored individual question into cognitive skill categories and calculated the percent of points in each assessment assigned in each Bloom’s category (see Derting et al. 2016 for details). This data set consists of data from 17 FIRST Faculty and 15 non-FIRST Faculty.
PercentAverageBloomsAssess.csv
FIRST Paired Background Survey Data
The associated data file contains results from the paired Teaching Background Survey which we designed to characterize the prior biology teaching experiences for the faculty participants. This data set consists of data from 18 of the pairs. For complete survey questions and scales see Supplementary Materials S1 in the associated publication (Derting et al. 2016).
FIRST_Paired_Background_Survey.data.csv