Scoring Methods for Progress Monitoring Using CBM in Early Writing
DOI:
https://doi.org/10.18666/LDMJ-2023-V28-I2-12058Keywords:
curriculum-based measurement, early writing, assessment, trait-based rubricAbstract
This initial exploratory study assessed the evidence of reliability, criterion validity, and growth of scores obtained from total correct word sequences (CWS), correct word sequences per response (CWSR), and a trait-based rubric used with a curriculum-based measure of writing (CBM-W) for progress monitoring. Participants were 26 writers with or at risk for a learning disability (LD) in grades 2-3. Participants were given the CBM-W task twice weekly over 6 weeks and were given the Spelling and Writing Samples subtests of the Woodcock-Johnson Tests of Achievement-III at pre- and posttest. Results indicated large inter-rater reliability coefficients but mixed alternate form reliability and validity evidence. CWS was the only metric to demonstrate significant growth (r = .60). CWS has the strongest evidence for use as a progress monitoring scoring metric for students at risk for LD, although information from CWSR and rubric scores could be instructionally useful.
References
References
Allen, A. A., & Lembke, E. S. (2022). The effect of a morphological awareness intervention on
early writing outcomes. Learning Disability Quarterly, 45(2), 72-84. https://doi.org/10.1177%2F0731948720912414
Allen, A. A., Poch, A. L., Lembke, E. S. (2018). An exploration of alternative scoring indices
using curriculum-based measurement in early writing. Learning Disability Quarterly, 41(2), 85-99. https://doi.org/10.1177/0731948717725490
Coker, D. L., & Ritchey, K. D. (2010). Curriculum-based measurement of writing in
kindergarten and first grade: An investigation of production and qualitative scores. Exceptional Children, 76(2), 175–193. https://doi.org/10.1177/001440291007600203
Deno, S. L. (2003). Developments in curriculum-based measurement. The Journal of Special
Education, 37(3), 184-192. https://doi.org/10.1177%2F00224669030370030801
Deno, S., Marston, D., & Mirkin, P. (1982). Valid measurement procedures for
continuous evaluation of written expression. Exceptional Children, 48(4), 368–371. https://doi.org/10.1177/001440298204800417
Gansle, K. A., VanDerHeyden, A. M., Noell, G. H., Resetar, J. L., & Williams, K. L.
(2006). The technical adequacy of curriculum-based and rating-based measures of written expression for elementary school students. School Psychology Review, 35(3), 435–450. https://doi.org/10.1080/02796015.2006.12087977
Graham, S., Harris, K. R., & McKeown, D. (2013). The writing of students with learning disabilities, meta-analysis of self-regulated strategy development writing intervention studies, and future directions: Redux. In L. Swanson, K. R. Harris, & S. Graham (Eds.), Handbook of learning disabilities (pp. 405–438). Guilford Press.
Graham, S., Harris, K., and Hebert, M.A. (2011). Informing writing: The benefits of formative assessment. A Carnegie Corporation Time to Act Report. Alliance for Excellent Education. https://media.carnegie.org/filer_public/37/b8/37b87202-7138-4ff9-90c0-cd6c6f2335bf/ccny_report_2011_informing.pdf
Hampton, D. D., & Lembke, E. S. (2016). Examining the technical adequacy of progress
monitoring using early writing curriculum-based measures. Reading & Writing Quarterly, 32(4), 336-352. https://psycnet.apa.org/doi/10.1080/10573569.2014.973984
Hier, B. O., January, S. A. A., & Van Norman, E. R. (2020). A comparison of CBM-WE scoring
metrics and progress monitoring frequency among second-grade students. School Psychology Review, 49(3), 306-320. https://doi.org/10.1080/2372966X.2020.1763758
January, S. A. A., Van Norman, E. R., Christ, T. J., Ardoin, S. P., Eckert, T. L., & White, M. J.
(2019). Evaluation of schedule frequency and density when monitoring progress with curriculum-based measurement. School Psychology, 34(1), 119. https://doi.org/10.1037/spq0000274
Jung, P. G., McMaster, K. L., & delMas, R. C. (2017). Effects of early writing intervention
delivered within a data-based instruction framework. Exceptional Children, 83(3), 281-297. https://doi.org/10.1177/0014402916667586
Lembke, E., Deno, S. L., & Hall, K. (2003). Identifying an indicator of growth in early writing
proficiency for elementary school students. Assessment for Effective Intervention, 28(3–4), 23–35. https://psycnet.apa.org/doi/10.1177/073724770302800304
Marston, D. B. (1989). A curriculum-based measurement approach to assessing academic
performance: What it is and why do it. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp. 18–78). Guilford Press. https://psycnet.apa.org/record/1989-97717-002
McMaster, K. L., Ritchey, K. D., & Lembke, E. (2011a). Curriculum-based measurement for
beginning writers: Recent developments and future directions. In T. E. Scrugg, & M. A. Mastropieri (Eds.), Advances in learning and behavioral disabilities: Assessment and intervention (Vol. 24, pp. 111-148). Emerald. https://doi.org/10.1108/S0735-004X(2011)0000024008
McMaster, K. L., Du, X., Yeo, S., Deno, S. L., Parker, D., & Ellis, T. (2011b).
Curriculum-based measures of beginning writing: Technical features of the slope. Exceptional Children, 77, 185–206. https://doi.org/10.1177%2F001440291107700203
Parker, D. C., McMaster, K. L., Medhanie, A., & Silberglitt, B. (2011). Modeling early
writing growth with curriculum-based measures. School Psychology Quarterly, 26(4), 290–304. https://doi.org/10.1037/a0026833
Romig, J. E., Therrien, W. J., & Lloyd, J. W. (2017). Meta-analysis of criterion validity for
curriculum-based measurement in written language. The Journal of Special Education, 51(2), 72-82. https://doi.org/10.1177/0022466916670637
Schrank, F. A., McGrew, K. S., & Woodcock, R. W. (2001). Technical abstract
(Woodcock-Johnson III Assessment Service Bulletin No. 2). Riverside Publishing.
Smith, R. A., & Lembke, E. S. (2021). Aspects of technical adequacy of an early-writing
measure for English language learners in grades 1 to 3. Assessment for Effective Intervention, 47(1), 59-63. https://doi.org/10.1177/1534508420947157
Stecker, P. M., Fuchs, L. S., & Fuchs, D. (2005). Using curriculum‐based measurement to
improve student achievement: Review of research. Psychology in the Schools, 42(8), 795-819. https://doi.org/10.1002/pits.20113
Videen, J., Deno, S., & Marston, D. (1982). Correct word sequences: A valid indicator of
proficiency in written expression. (No. IRLDRR-84). University of Minnesota, Institute for Research on Learning Disabilities. https://files.eric.ed.gov/fulltext/ED225112.pdf
Wagner, K., Smith, A., Allen, A., McMaster, K., Poch, A., & Lembke, E. (2019). Exploration of
new complexity metrics for curriculum-based measures of writing. Assessment for Effective Intervention, 44(4), 256-266. https://doi.org/10.1177%2F1534508418773448
Woodcock, R. W., McGrew, F. A., Mather, N. & Schrank, F. A. (2007). Woodcock-
Johnson tests of achievement (3rd ed.). Riverside Publishing.
Downloads
Published
Issue
Section
License
Sagamore Publishing LLC (hereinafter the “Copyright Owner”)
Journal Publishing Copyright Agreement for Authors
PLEASE REVIEW OUR POLICIES AND THE PUBLISHING AGREEMENT, AND INDICATE YOUR ACCEPTANCE OF THE TERMS BY CHECKING THE ‘AGREE TO THE TERMS OF THIS COPYRIGHT NOTICE’ CHECKBOX BELOW.
I understand that by submitting an article to Learning Disabilities: A Multidisciplinary Journal, I am granting the copyright to the article submitted for consideration for publication in Learning Disabilities: A Multidisciplinary Journal to the Copyright Owner. If after consideration of the Editor of the Learning Disabilities: A Multidisciplinary Journal, the article is not accepted for publication, all copyright covered under this agreement will be automatically returned to the Author(s).
THE PUBLISHING AGREEMENT
Assignment of Copyright
I hereby assign to the Copyright Owner the copyright in the manuscript I am submitting in this online procedure and any tables, illustrations or other material submitted for publication as part of the manuscript in all forms and media (whether now known or later developed), throughout the world, in all languages, for the full term of copyright, effective when the article is accepted for publication.
Reversion of Rights
Articles may sometimes be accepted for publication but later be rejected in the publication process, even in some cases after public posting in “Articles in Press” form, in which case all rights will revert to the Author.
Retention of Rights for Scholarly Purposes
I understand that I retain or am hereby granted the Retained Rights. The Retained Rights include the right to use the Preprint, Accepted Manuscript, and the Published Journal Article for Personal Use and Internal Institutional Use.
All journal material is under a 12 month embargo. Authors who would like to have their articles available as open access should contact Sagamore-Venture for further information.
In the case of the Accepted Manuscript and the Published Journal Article, the Retained Rights exclude Commercial Use, other than use by the author in a subsequent compilation of the author’s works or to extend the Article to book length form or re-use by the author of portions or excerpts in other works.
Published Journal Article: the author may share a link to the formal publication through the relevant DOI.
Author Representations
- The Article I have submitted to the journal for review is original, has been written by the stated author(s) and has not been published elsewhere.
- The Article was not submitted for review to another journal while under review by this journal and will not be submitted to any other journal.
- The Article contains no libelous or other unlawful statements and does not contain any materials that violate any personal or proprietary rights of any other person or entity.
- I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in the Article.
- If the Article was prepared jointly with other authors, I have informed the co-author(s) of the terms of this Journal Publishing Agreement and that I am signing on their behalf as their agent, and I am authorized to do so.