Dynamic Learning Maps
Acknowledgements
1
Introduction
1.1
Impact of COVID-19 on the Administration of DLM Assessments
1.2
Background
1.3
Assessment
1.4
Technical Manual Overview
2
Map Development
3
Item and Test Development
3.1
Items and Testlets
3.1.1
Assessment Structure
3.1.2
Item Writing
3.2
External Reviews
3.2.1
Review Recruitment, Assignments, and Training
3.2.2
Results of Reviews
3.2.3
Test Development Decisions
3.3
Operational Assessment Items for 2019–2020
3.4
Field Testing
3.4.1
Description of Field Tests
3.4.2
Field Test Results
3.5
Conclusion
4
Test Administration
4.1
Overview of Key Administration Features
4.1.1
Change to Administration Model
4.1.2
Test Windows
4.1.3
The Instruction and Assessment Planner
4.2
Administration Evidence
4.2.1
Instructionally Embedded Administration
4.2.2
Administration Incidents
4.3
Implementation Evidence
4.3.1
Data Forensics Monitoring
4.3.2
Released Testlets
4.4
Conclusion
5
Modeling
5.1
Overview of the Psychometric Model
5.2
Calibrated Parameters
5.2.1
Probability of Masters Providing Correct Response
5.2.2
Probability of Non-Masters Providing Correct Response
5.2.3
Item Discrimination
5.2.4
Base Rate Probability of Mastery
5.3
Mastery Assignment
5.4
Model Fit
5.5
Conclusion
6
Standard Setting
7
Assessment Results
7.1
Data Files
7.2
Score Reports
7.2.1
Individual Student Score Reports
7.3
Quality Control Procedures for Data Files and Score Reports
7.4
Conclusion
8
Reliability
9
Validity Studies
9.1
Evidence Based on Test Content
9.1.1
Evaluation of Blueprint Coverage
9.2
Evidence Based on Response Processes
9.2.1
Test Administration Observations
9.2.2
Interrater Agreement of Writing Sample Scoring
9.3
Evidence Based on Internal Structure
9.3.1
Evaluation of Item-Level Bias
9.3.2
Internal Structure Within Linkage Levels
9.4
Evidence Based on Relation to Other Variables
9.4.1
Postsecondary Opportunities
9.5
Conclusion
10
Training and Professional Development
10.1
Required Training for Test Administrators
10.1.1
First-Time Test Administrator Training Content
10.1.2
Returning Test Administrator Training Content
10.2
Instructional Professional Development
10.2.1
Professional Development Participation and Evaluation
10.3
Conclusion
11
Conclusion and Discussion
11.1
Operational Assessment
11.1.1
Future Research
12
References
Appendix
A
Distribution of Essential Elements Tested
B
Differential Item Functioning Plots
B.1
Uniform Model
B.2
Combined Model
2019–2020 Technical Manual Update
2019–2020 Technical Manual Update
Instructionally Embedded Model
December 2020
Placeholder Title
Copyright © 2020 Accessible Teaching, Learning, and Assessment Systems (ATLAS)