Using WIDS to Document How Colleges Assess Learning Outcomes

Using WIDS to Document How Colleges Assess Learning Outcomes

OverviewWhat is assessment?What are the drivers?How does curriculum fit in?How does WIDS help?

Assessment of student learning is a participatory, iterative process that:Provides data/information you need on your students learningEngages you and others in analyzing and using this data/information to confirm and improve teaching and learningProduces evidence that students are learning the outcomes you intendedGuides you in making educational and institutional improvementsEvaluates whether changes made improve/impact student learning, and documents the learning and your efforts.

NCA Higher Learning CommissionFrom Student Learning, Assessment and Accreditation: Criteria and Contexts, presented at Making a Difference in Student Learning: Assessment as a Core Strategy, a workshop from the Higher Learning Commission, July 26-28, 2006. Assessment of Learning

Informed judgments about achievement of intended learning outcomesBased on evidence of what students can do and their capacity to apply what they know (data) at the completion of the learning experience (course, certificate, program, or other credential)Valid, reliable, and fairBuilt into the plan for teachingData-driven evaluation that is actively used for continual improvement of teaching and learningAssessment of Learning

Sound AssessmentValidOutcomes based on standards (industry)Measures intended outcomesMeasures application and critical thinkingReliablePerformance assessment based on consistent rubrics, scoring guides, and rating scalesConsistent process - each learner is assessed in same way as other learners FairValidReliableLearners informed of expectations up front!Feedback to learners

Summative/Formative Assessment

What are the drivers?Learning & continuous improvement of teaching and learningAccreditation (NCA, AQIP, Industry)Carl Perkins IV

HLC/AQIP asks 5 fundamental questions:How are your stated learning outcomes appropriate to your mission, programs, students and degrees?How do you ensure shared responsibility for student learning & assessment of student learning?What evidence do you have that students achieve your stated learning outcomes?In what ways do you analyze and use evidence of student learning?How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?

NCA Higher Learning CommissionFrom Student Learning, Assessment and Accreditation: Criteria and Contexts, presented at Making a Difference in Student Learning: Assessment as a Core Strategy, a workshop from the Higher Learning Commission, July 26-28, 2006.

The What: Both secondary and postsecondary must develop a process for measuring technical skills that lead to an industry-recognized credential, or state-recognized credential, or locally-developed credential

The Why: Both secondary and postsecondary education systems must provide, as a separate measure of accountability, the extent of skill development in Career and Technical Education (CTE) programs

The How: Both secondary and postsecondary education must develop valid and reliable assessmentsCarl Perkins IV The New Law

Postsecondary IndicatorsTechnical Skill Attainment Credential, Certificate, or Diploma Student Retention or Transfer Student Placement Nontraditional Participation Nontraditional Completion

Carl Perkins IV The New Law

Carl Perkins IV The New LawWTCS will move toward a balance of silver and gold assessment of learning outcomes.

Gold Standard (ex. NCLEX, ASE, Barber/Cosmetology)External, third-party assessment that objectively measures student attainment of industry recognized skills upon graduation.Silver StandardWTCS approved assessment that objectively measures student attainment of industry recognized skills upon graduation. May be developed and/or implemented by the state system.May be developed and/or implemented locally according to system guidelines and with industry (advisory committee) approval.Must be valid and reliable.Bronze Non-assessment related indicators such as indirect measures.GPA and/or Course CompletionProgram Completion GraduationTeacher-developed exams that do not meet silver standard (not externally approved).

Carl Perkins IV The New LawSummary of Standards for Assessment of Technical Skill Attainment

PlanEstablish outcomesBuild assessmentsDesign learningDODeliverAssessDocumentCHECKAnalyze assessment dataPropose improvementsACTAdjust designAdjust deliveryAdjust assessmentAssessment Process and Curriculum

The Learning Assessment Design Technology Tool Kit

WIDS Learning Design System populates assessment, electronic portfolio, enterprise, and assessment systems with learning outcomes and performance standards.Assessment Management Software provides feedback loop for continual improvement of learning design using WIDS.Web-based Electronic Portfolio System presents evidence of individuals learning and professional experience.Learning Outcomes Learning Outcomes Learning OutcomesCurriculum Documents Learning OutcomesRubrics Student Learning DocumentationContinuous Improvement Feedback Student Data

What do they need to be able to achieve?Who are the learners?How will we know and show when theyve achieved it? How will they get there?

WIDS Model

Linked Competencies(performance assessments)Exit Learning OutcomesProgram Outcomes(summative assessment)Core Abilities(summative assessment)Gen Ed Outcomes(summative assessment)Job Task Analysis(DACUM)

External Standards(Research-based)

Integrating the OutcomesLinked Competencies(performance assessments)Linked Competencies(performance assessments)ExternalProgram-LevelCourse-Level

ExternalProgram-LevelCourse-LevelDental Hygiene ExampleGraduates are competent in applying ethical, legal and regulatory concepts to the provision and/or support of oral health care services.Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies Respond to a request to perform a task that is not legally permitted to be delegated to a dental hygienist.ADA Dental Hygiene Education StandardDental Hygiene Program OutcomeDental Hygiene Competency

ChallengesHow do we communicate WTCS long-standing commitment to industry input and standards?How do we continue WTCS commitment to performance assessment whenever possible (versus testing?)How do we ensure that we use quality assessments? (That we dont regress to testing regurgitation of information?)How do we ensure that we are doing what our model and assessment plans imply?

Create Program Design files for all programsDevelop Exit Learning OutcomesCore Abilities (mission level)Program Outcomes (discipline level)General Education Outcomes (Optional)Determine summative assessmentsDefine criteria for summative assessmentsLink courses to exit learning outcomesUsing WIDS Program DesignDesign. Document. Align

Create Performance Assessment Tasks (with scoring guides and rubrics) to support summative assessmentEmbed in program courses, using the program file as the master fileCreate a separate Summative Assessment course file for programUsing WIDS Learning DesignDesign. Document. Align

Generate Analyzer reportsLearning Outcomes MatrixProgram Course List MatrixAssessment Task Outcomes MatrixUsing WIDS AnalyzerDesign. Document. Align

Learning Outcomes Matrix

Assessment Task Outcomes Matrix

Maximize the potential of WIDS to support assessment planning:Create/maintain Program Outcome Summaries for all new and existing programs. Develop/maintain Course Outcome Summaries for all courses.Design Performance Assessment Tasks for course level (formative) and beyond-the-course (summative) assessments.Use WIDS Analyzer to build Learning Outcome Matrices that document how outcomes are linked with programs and courses.Present WIDS documentation as a central feature in accreditation.

* **JudyJudy*Judy*Judy