A Direct Method for Teaching and Measuring Engineering Professional Skills: A Validity Study for the National Science Foundation’s Research in Evaluation of Engineering and Science Education (REESE)

Abstract

Abstract A Direct Method for Teaching and Measuring Engineering Professional Skills: A Validity Study for the National Science Foundation’s Research in Evaluation of Engineering and Science Education (REESE)TRACK: Student DevelopmentIntroductionThis paper describes an ongoing research project in establishing the validity of a direct methodfor teaching and measuring undergraduate engineering students’ professional skills. Proficiencyin engineering professional skills (Table 1) is critical for success in the multidisciplinary,intercultural team interactions that characterize global 21st century engineering careers. Yet,programs across the nation have struggled to define, teach and measure professional skills sincetheir introduction as ABET criteria for engineering programs in 2000. In fall 2006, the XXXUniversity College of Engineering developed an innovative, direct method to teach and measurethe ABET professional skills simultaneously. The Engineering Professional Skills Assessment(EPSA) is the only direct method for teaching and measuring these skills simultaneously in theliterature. Table 1.ABET Criterion 3 Professional Skills Student Learning Outcomes 3d Ability to Function on Multidisciplinary Teams 3f Understanding of Professional and Ethical Responsibility 3g Ability to Communicate Effectively 3h Understanding of the Impact of Engineering Solutions in Global, Economic, Environmental, and Cultural/Societal Contexts 3i Recognition of and Ability to Engage in Life-Long Learning 3j Knowledge of Contemporary IssuesThe major accomplishments of the six years of on-going research conducted college wide sincespring 2007 using the EPSA method at the program level for evaluating the efficacy of theundergraduate engineering curriculum are: (1) an authentic performance task in the form of ascenario and prompts to elicit the ABET professional skills; (2) establishment of initial reliabilityand validity of the measurement instrument – the Engineering Professional Skills Rubric; and (3)a dedicated community of 40+ engineering faculty using direct assessment to evaluate theefficacy of their own programs, and to plan and implement improvement at both course andprogram levels. We are now in year 3 of a 3 year validity study of the method and tool acrossthree universities.The EPSA method is a discussion-based performance task designed to elicit students’ knowledgeand application of the ABET professional skills. In a 45-minute session, small groups of studentsare presented with a complex, real-world scenario that includes multi-faceted, multidisciplinaryengineering issues. They are then asked to determine the most important problem/s and todiscuss stakeholders, impacts, unknowns, and possible solutions. The EPS Rubric, an analyticrubric, was developed to measure the extent to which student performance in response to a givenscenario achieved the six learning outcomes associated with the ABET professional skills. Thismethod is flexible, easy to implement, and can be used at the course level for teaching andmeasuring engineering professional skills and the program level at the end of a curricularsequence for evaluating a program’s efficacy.