Session 3213

Evaluation of Instructional Technology Tools in the Classroom

Sara Soderstrom and Chris Lorenz

University of Michigan, Ann Arbor, MI 48109

Introduction

The purpose of this study is to evaluate the effectiveness of the variety of IT tools available to students.  With increased time, effort, and funds directed towards the development of instructional technology (IT) tools, it is imperative to evaluate which methods have the most effect with respect to the students' learning styles.  The results of this study could provide a guide for faculty as to which IT methods should be developed in order to best improve the learning of each student.  Through the development of questionnaires and evaluations we have correlated the learning style of students to the effectiveness of specific teaching tools.  The questionnaires and the evaluations will be developed in conjunction with the University of Michigan Center of Research on Learning and Teaching.  A variety of measures have been studied including the relationship between the learning styles to the final course grades of individual students.  The course grade of each student were compared to their average chemical engineering grades.  The grade comparison results in a direct measure of the influence of educational tools on students' performance.

The large class size and the variety of learning methods available for the Reaction Engineering course at the University of Michigan Chemical Engineering Department make it an ideal test case for this study.   This course is taught to approximately 100 juniors.   Multiple teaching tools have been developed and incorporated into this course.   The students can use the IT tools on the course website and CD-ROM as supplements to the two 2-hour lectures per week and the course text.  The IT tools have been developed so that they address a number of different learning styles as described by Solomon.  For example, active learners can vary experimental conditions and evaluate the system’s response using the computer-based modules.  Interactive course notes, which are also on the web for students to use as resource, have been developed in visual and audio formats to appeal to the verbal learners and also include diagrams and flow charts which the visual learner might find more helpful.  The reflective learners in the class can evaluate their understanding of concepts using the self-tests, which are available in each section of the web notes. Derivations of equations, which are first introduced in class, can also be found on the web so the deductive learners have the opportunity to track the complete theoretical development.  Frequently asked questions (FAQs) have been collected and responded to over many terms, which have been compiled and listed for students to reference.  Suggestions from previous classes are also available as guidance for current students.  Using this specific course as a test case, we will be able to evaluate the relationship between learning styles and the effectiveness of IT tools enabling faculty to best develop IT methods to impact the learning of each student in a course.

IT Usage and Learning Style Surveys

A survey was included in the midterm course evaluation to determine the frequency of IT usage by students in preparing for the first exam.  This survey is included in Appendix X.  Students were also assigned a homework in which they had to describe the IT tools that they found most beneficial based on their learning styles.

The first exam occurred after 6 weeks of courses.  Figure X shows the average time spent by students using various media in preparation for the exam.  Non-technological tools such as office hours and old exams were used the most; however, Polymath, web lecture notes, ICMs, and self-tests were used for an average of 2-3 hours per student above time required for assignments.

Figure X. Average time spent by each student using various tools in preparation for the first exam

The results from the homework assignment regarding effective IT tools based on learning styles supported the initial development of the various tools.  Active learners found self-tests, web modules, and group use of technology to be beneficial while reflective learners found living learning problems and lecture and chapter summaries to be more helpful.  Sensing learners enjoyed the links of computer problems and modules to real world topics and intuitive learners liked being able to use trial and error aspect of modules and variations in Polymath programs to understand the effects of system parameters.  Visual learners commented that they found the color-coding within derivations helpful.  On the other hand, verbal learners liked the summary notes and audios on the web.  Global learner benefited from general overviews on lecture notes, real-world models, and concept summaries that were included in the modules.  Sequential learners prefered the in-depth derivations included in the web notes and the derivations included in the modules.  Thus, students were able to determine which tools they found most beneficial based on their learning styles and develop study strategies from this.  It is therefore important to teach students about learning styles so that they study more effectively.  Also, we found that students were less overwhelmed by the multiple IT tools after evaluating their learning styles; instead of trying to do everything, they focused on those tools they found most helpful.

Grade Improvement

A background survey was completed by the students in order to evaluate the students’ academic, work, and computer usage history.  This is included in Appendix X.  We found that the computer usage was comparable for most of the students in the class.  The students’ grades were then used as a comparative metric.  The course grades were compared to the students’ overall grade point average.  We found that nearly three of every four students received a better grade in this course.  This is summarized in Figure X. 

Figure X.  Comparison of number of students who did better in this course than previously to the total number of students in each category

Table X shows an improvement breakdown for students’ improvement within each learning style category.  With the exception of reflective learners, students with all types of learning styles were shown to improve in approximately the same ratios as the overall course.  We hypothesize that the improvement for reflective learners is less because of the heavy course workload; it is possible that the time commitment necessary for group homework and projects made it more difficult to spend significant time to review the material and notes.

Table X.  Comparison of grade improvement for students of various learning styles 

Conclusions and Recommendations 

We found grade improvement for students of all learning styles.  Without a control cohort it is impossible to conclude that this is due only to the use of IT tools, but we believe that it is influenced by the continued use of the IT tools throughout the course.  Students with physical and learning disabilities found that the IT tools provided more learning opportunities that they could use within their own schedule.  They commented that this positively influenced their course performance.  The learning style evaluations showed that understanding learning styles helps students not only in improving their individual study habits but also led to improvements in group dynamics and operation.