WED 463 - Assessment of Learning Performance
ADDIE - Instructional systems design model
Anaylyze, Design, Develop, Inplement, Evaluate
Formative evaluation - to collect data througout instruction to see if instruction is doing what it is designed to do, is student performing as intended
Donald Kirkpatrick - 4 levels of evaluation
1. Reaction - Non performance, feelings, attitude measurement, opininon, not considered best
data
2. Learning - Pre-post tests, assessment, perform in class (controlled environment)
3. Transfer - Performance in field, application, self appraisal
4. Business Impact - 40% of US worker time is rework - errors are caught 'down the line',
in the Japanese TPS employees can stop the line when an error is detected
Jack Phillips - 5th level of evaluation
Return on Investment - every complaint can be converted to a dollar sign, measures time and money, 5% of US companies perform an ROI assessment, measure for every dollar spent how much is returned
Train personnel to proficiency performance as soon as possible to increase productivity
Performance Gap Analysis
What is - current state Need - discrepancies What should be - desired state
Determine intervention - training or non-training, informaton, or motivation
Most effective way to fill the gap - cost, time, etc.
If the workers can perform under duress, skills are present - motivation is lacking
When pressure does not work, training may be the problem
Use a run chart to calculate data over a long period to identify anomalies
ADDIE - Instructional systems design model
Anaylyze, Design, Develop, Inplement, Evaluate
Formative evaluation - to collect data througout instruction to see if instruction is doing what it is designed to do, is student performing as intended
Donald Kirkpatrick - 4 levels of evaluation
1. Reaction - Non performance, feelings, attitude measurement, opininon, not considered best
data
2. Learning - Pre-post tests, assessment, perform in class (controlled environment)
3. Transfer - Performance in field, application, self appraisal
4. Business Impact - 40% of US worker time is rework - errors are caught 'down the line',
in the Japanese TPS employees can stop the line when an error is detected
Jack Phillips - 5th level of evaluation
Return on Investment - every complaint can be converted to a dollar sign, measures time and money, 5% of US companies perform an ROI assessment, measure for every dollar spent how much is returned
Train personnel to proficiency performance as soon as possible to increase productivity
Performance Gap Analysis
What is - current state Need - discrepancies What should be - desired state
Determine intervention - training or non-training, informaton, or motivation
Most effective way to fill the gap - cost, time, etc.
If the workers can perform under duress, skills are present - motivation is lacking
When pressure does not work, training may be the problem
Use a run chart to calculate data over a long period to identify anomalies
Test Making
Be unbiased when collecting data, be careful about variables that effect people's behavior
Hawthorne Effect: Aptitude does not predict performance
Pygmalion Effect: Once an expectation is set, we act in ways consistant with the expectation
Rapid Prototyping: Solid free-form fabrication of 3 dimensional objects w/CAD
Evaluation Myths
What is not measured, does not get done
Formative evaluation - measuring progress throughout the training process
Training - Instruction to improve skills for job performance (must include practice, feedback, and application) (Need to know information)
Education - General, academic (Nice to know information)
Presentation - Stating information (Spray and pray)
Organization infrastructure can also affect motivation to perform - policies, equipment, time lines, etc
Gap analysis - Collect data to fill the gap (observation, asking, surveys, interviews to get information (an intervention must be targeted at the right fix or problem to be effective)
Four Types of Assessments
1. Pre-test - Beginning of course measurement
2. Pre-requisites - Required before block of instruction
3. Post Test - After course measurement
4. Embedded - Measuring practice during the course, feedback, quizzes
Halo Effect - Positive or negative assumptions based on past performance
Be unbiased when collecting data, be careful about variables that effect people's behavior
Hawthorne Effect: Aptitude does not predict performance
Pygmalion Effect: Once an expectation is set, we act in ways consistant with the expectation
Rapid Prototyping: Solid free-form fabrication of 3 dimensional objects w/CAD
Evaluation Myths
What is not measured, does not get done
Formative evaluation - measuring progress throughout the training process
Training - Instruction to improve skills for job performance (must include practice, feedback, and application) (Need to know information)
Education - General, academic (Nice to know information)
Presentation - Stating information (Spray and pray)
Organization infrastructure can also affect motivation to perform - policies, equipment, time lines, etc
Gap analysis - Collect data to fill the gap (observation, asking, surveys, interviews to get information (an intervention must be targeted at the right fix or problem to be effective)
Four Types of Assessments
1. Pre-test - Beginning of course measurement
2. Pre-requisites - Required before block of instruction
3. Post Test - After course measurement
4. Embedded - Measuring practice during the course, feedback, quizzes
Halo Effect - Positive or negative assumptions based on past performance
No comments:
Post a Comment