Skip to Content

Buyer Beware: Lessons Learned from edTPA Implementation in New York State

by Deborah Greenblatt and Kate E. O’Hara

As states across the country continue their implementation of the edTPA, a complex and high-stakes certification requirement for teacher certification, there are important lessons for educators and education advocates to learn from New York State’s implementation. As Linda Darling-Hammond, developer and promoter of the edTPA, cautioned at the 2014 American Educational Research Association meeting: “New York is a prototype of how not [original emphasis] to implement teacher performance assessment.”1

edTPA stands for the Teacher Performance Assessment Portfolio, an assessment of teacher readiness developed by The Stanford Center for Assessment, Learning and Equity (SCALE) and the American Association of Colleges for Teacher Education (AACTE) but nationally distributed and scored by Pearson Education, Inc. It differs from previous assessments in that it purports to measure “performance” by requiring student teachers to compile a portfolio, including lesson plans, student work samples, a short classroom video (15 to 20 minutes), and a lengthy “instructional commentary” of 40 to 60 pages.

Currently, there are 622 educator preparation programs in 35 states and the District of Columbia participating in edTPA. Some states are still exploring its use while others require edTPA as part of program completion or for state licensure.2 Among them, New York’s story is unique: Although the New York State Education Department had begun working with Pearson in 2009 on its own teacher performance assessment, it switched to the edTPA when it became available in February 2012. The handbooks and rubrics were made available to faculty and students in New York’s schools of education that same spring.3 New York only conducted one year of field testing before fully implementing the edTPA as a high-stakes assessment.4

As a result of the rapid rollout, faculty at colleges of education had little time to reflect on their data and prepare their students for success: “We have basically set up a cohort of our students to fail,” warned Jamie Dangler, vice president of the United University Professions (UUP), the union of State University of New York educators, to New York State Education Department officials in January 2014, “and the consequences will be disastrous for students and teaching programs.’”

With the federal push to standardize a national evaluation requirement for pre-service teachers, all states and their educators must also consider and contend with the impact of profit-oriented corporations in the teacher preparation process. The certification of teachers has been taken out of the hands of the states and now turned over to a for-profit company that has much to gain from a national adoption of the edTPA: Pearson Education, Inc.

Concerns over the corporatization of teacher certification are fueled by Pearson’s lack of transparency. When participating in Pearson workshops, trainings, or test scoring, faculty have to sign non-disclosure agreements. Faculty are not allowed to share materials with their colleagues or their students.5 Furthermore, although the edTPA Myths and Facts document asserts the criteria for selecting and training scorers is “rigorous,” the teacher candidates’ score reports do not include the qualifications of their scorer nor is specific data about edTPA current scorers readily available online.

It is our hope that educators, activists, and policy makers will benefit from the lessons we have learned in New York and join our effort for a certification process that does not standardize teacher education programs but rather draws upon an effectively designed certification process and represents what is important to the profession, not politicians and corporations.

Lesson One: edTPA is Called a Teacher Performance Assessment—But that Doesn’t Mean it is One

While many stakeholders would agree that a performance assessment is a more effective way to measure teacher readiness than a pencil-and-paper test, the edTPA cannot fill that role. The edTPA relies greatly on teacher candidates’ reading, writing, and technological skills. Candidates are allowed to include only up 15 to 20 minutes of video to “feature the teaching and learning emphasis” for their subject area.6

For first-year teachers to be “effective,” the edTPA weighs heavily on data analysis skills, while de-emphasizing skills such as adaptability, relating to students’ interests, and fostering a cooperative environment. Additionally, the lengthy and tightly structured edTPA requirements have changed the focus of the student teaching experience and seminar from preparing for the first year of teaching to preparing to pass a test and create lessons under constraints that make the test an unauthentic assessment.7

Student teachers are faced with the challenge of manipulating both the structure and the content of their lessons to meet the demands of the edTPA. While student teachers are typically stressed about their coursework or teaching lessons, they now have the additional anxiety of making sure their teaching practices are labeled and organized according to the limitations of the edTPA questions, often struggling to manipulate what has been genuinely successful, into something that instead meets the unrealistic and unfamiliar demands of the test. A professor colleague elaborates, “We used to share successes and challenges in seminar, working together, digging deep. Now all we seem to do is go through the handbook page by page to make sure we understand what is expected of our students.”

Lesson Two: The edTPA Privileges Certain Student Teaching Placements

The challenges of the edTPA are exacerbated in schools in low-income communities where our K–12 students often are not scoring well on standardized tests. Not only are these schools more likely to have scripted curricula, but they also have students with a variety of special needs.8 Teacher candidates will need special permission to deviate from the mandated curriculum to showcase their best work in their videos, which can be a challenge depending on several factors such as how much principal surveillance the cooperating teachers are under or how much cooperating teachers are willing to “break the rules.” These conflicts often cause cooperating teachers to decide not to host a student teacher in future semesters.9

Although the edTPA purports to assess how general education teacher candidates address the needs of special education students as per their Individualized Education Program goals and the needs of English language learners, these prompts are not relevant in certain settings. Candidates placed in more homogenous settings will not face these additional demands, while those in more diverse classrooms must clearly address the needs of these students to score well.10 Although these teacher candidates may be better prepared for their careers when they student teach in such settings, schools of education may be motivated to sacrifice that preparation to improve edTPA passing rates.

Lesson Three: The edTPA Scoring is Inconsistent

Although the edTPA is based on National Board for Professional Teaching Standards, there is no evidence the edTPA has predictive validity, the ability to forecast future “success,” for any measure.11 SCALE, the originators of the edTPA, made a statement on the validity and reliability of the edTPA as follows:

A set of validation studies was conducted to confirm the content validity, job relevance, and construct validity of the assessments. In combination, these studies documented that the assessment is well-aligned to the professional standards it seeks to measure, reflects the actual work of teaching, and that the score measures a primary characteristic of effective teaching. Inter-rater reliability was evaluated using several different statistical tests. edTPA reliabilities reported here range from .83 to .92 (indicating the percentage of scorer agreement).12

This statement, however, addresses reliability or validity when it comes to inter-rater reliability only. In the same document, SCALE states the edTPA gives states the “ability to use a nationally available common measure that is valid and reliable to evaluate pre-service teachers’ readiness to teach.”13 It is certainly arguable if this common measure is truly valid and reliable. The National Center for Teacher Quality agrees saying:

What may be a very good culminating exercise for any program to administer is not necessarily a sufficiently valid and reliable measure of either an individual teacher or the quality of a program. For example, the edTPA allows candidates to choose the lessons they will deliver, rehearse as many times as they wish, and edit the videotape of their teaching. If a prospective elementary teacher chooses to teach a lesson on parallel lines rather than on equivalent fractions (because she really dislikes fractions), and even then edits out an instructional faux pas, is the resulting lesson a valid assessment of her overall teaching skills?14

Additionally, many teacher candidates and teacher educators question the consistency in the scoring of the test. A colleague from a State University of New York campus shares her experience: “My students did pass, but ironically the ones that weren’t as strong — you know, in pedagogy, or in classroom management— they passed, and passed with higher scores than my stronger candidates.” Experiences such as this call into question the reliability of the evaluations and the training of the Pearson evaluators.

According to a former Pearson evaluator, originally when portfolios were reviewed at scoring centers, two people graded each of the portfolios. However, scoring is now done remotely and scorers are recruited from across the country, even if their state does not use the edTPA. Also, now only one person scores each edTPA portfolio.15 Although there are “quality control” measures put in place by Pearson, with random portfolios being “back read” by a supervisor, it is not clear how often back reading is done.16 Given that scorers are paid per portfolio and are not held accountable until one of their portfolios is randomly selected for additional scoring, there is no existing measurement of inter-rater reliability.

According to Nancy A. De Korp, coordinator of Education Programs, Office of Higher Education for the NYSED, there is also a protocol if the total score is at, or around, a passing score. In this case, a second scorer will review the portfolio.

A third scorer (a scoring supervisor) will evaluate the portfolio if either 1) Scorer 1 and Scorer 2 are discrepant (more than one score point apart) on any rubric, or 2) Scorer 1 and Scorer 2 are on opposite sides of the recommended professional performance standard (for decision consistency). A .5 score occurs when two scorers have evaluated a portfolio and their score on the rubric is averaged. Scorer 1 and Scorer 2 may have exact agreement or adjacent agreement on each rubric. Any discrepant rubrics (more than one score point apart) are sent to a supervisor for resolution.17

However, we have seen instances where teacher educators, those who have assessed teacher candidates’ performance multiple times across the semester, have a different evaluation than the Pearson scorers. One teacher candidate shared with us that, although she was the only person in her student teaching seminar to get an A, she failed the edTPA. Her professor evaluated the portfolio herself, using the Pearson rubrics, and disagreed significantly with the scorer’s findings. As a result she is considering an appeal; however she would have to pay $200 to do so.18

Allowing per diem scorers of the edTPA to be the gatekeepers to the profession depersonalizes the relationship between teacher candidates and their students, cooperating teachers, field supervisors, and professors in a very troubling way.19

Lesson Four: The edTPA Shifts the Focus of the Student Teaching experience to Test Preparation

Teacher candidates spend hours reading, rereading, examining, and understanding their edTPA content specific handbook, the Making Good Choices guidebook, and other associated edTPA documents. In addition to the time devoted to typical planning, there are rubrics and prompts that must be dissected, discussed, and understood in order to earn a passing edTPA score. Candidates spend many additional hours analyzing student data, compressing and uploading videos, and writing pages of commentary, leaving little time for planning of high quality lessons. Jen Boerner, a graduate student at SUNY Brockport, said that the biggest drawback to the edTPA was the lack of attention she was able to pay to all of her special education students: “‘I feel I lost out on a lot of student teaching. I really couldn’t do as much as I wanted. I couldn’t go over all the lesson plans I wanted to try out because I was teaching to the test. That was unfortunate.’”20 In some cases, instead of teaching follow-up lessons in the days after their edTPA “learning segment,” candidates spent their nights answering the writing prompts for Pearson. It’s also important to note that in addition to the time committed to edTPA and student teaching, many teacher candidates are taking additional courses to maintain their financial aid eligibility. With so much at stake, the reality for many teacher candidates is that they often need to leave their coursework unfinished, miss classes and deadlines, or simply hand in acceptable, rather than exemplary work so they can focus on constructing their edTPA portfolio.21

Lesson Five: The edTPA Privileges Candidates and Institutes of Certain Financial Status

One must consider the repercussions of the increase in the cost of the exams package, which has now doubled to over $600. The exams package includes the edTPA plus the Educating All Students exam, Content Specialty Tests, and the Academic Literacy Skills test. Although Pell grant recipients are eligible to get a voucher for the $300 edTPA portion of the exam, they are not guaranteed one. Only 600 edTPA vouchers were distributed across all of New York State for all eligible teacher candidates.22

For example, when Hobart and William Smith Colleges applied for vouchers for its 24 students who received federal Pell grants, they received an e-mail from ESTestVoucher @Pearson.com saying that vouchers were allocated proportionally to institutions based on the number of undergraduate Pell recipients reported by each institution. For Hobart and William Smith’s 24 eligible students, they received only one voucher.23 At New York University, dozens of undergraduates and graduate students with specific financial aid and GPA requirements were eligible to enter a lottery to try to “win” one of only eight vouchers.24 Unfortunately, not all schools advertise these vouchers either. In some cases, students don’t know that such a voucher exists unless an individual faculty member makes an effort to inform their students.

Teacher candidates who can afford to attend well-funded teacher credentialing programs often enjoy additional support services, including the services of a full-time edTPA resource person.25 Schools with funding for that kind of position can provide edTPA workshops, seminars, and one-on-one consultation. Because of the benefits of these funded positions, teacher candidates’ course work or seminars do not have to be dedicated to edTPA preparation. Some colleges have subject specific edTPA coordinators who conduct workshops for students on the edTPA throughout their teacher education program which then allows teacher candidates to prepare for portions of their portfolio prior to student teaching experience. On the other hand, in colleges where funding is limited, online modules are often used or a website hosting edTPA resources is created to reach a broader audience of teacher candidates across disciplines. However, this approach leaves candidates with more general information and no human interaction. Additionally, while a single edTPA coordinator at one college might be in charge of all edTPA content portfolios, their counterparts at a wealthier school might have coordinators for each certification area.

Lesson Six: The edTPA Privileges Candidates from Certain Linguistic and Cultural Backgrounds

A teaching candidate may have carefully planned and successfully taught an effective “learning segment” but unless the candidate has also learned the language of the edTPA exam, followed particular directives, understood rubric objectives, and crafted their commentary by thoroughly reflecting the terminology designed and used by SCALE, she or he risks a failing score. This goal of learning the language of a teacher “bar exam” specifically disadvantages teacher candidates of color, as noted by Christine D. Clayton, Department chair of the Pace University School of Education:

Recent data from other states (where the assessment is not a certification exam) indicate that some groups, including teaching candidates of color and those from linguistic minority groups, were failing edTPA at disproportionate rates. Other reports showed evidence of candidates who performed well on edTPA, but did not earn fair supervision reports when actually teaching students.26

Wayne Au, associate professor in the education program at the University of Washington, expresses a legitimate concern, “Given the severe lack of teachers of color and teachers from working-class backgrounds, I wonder if the edTPA will systematically reproduce race and class inequalities, like every other high-stakes standardized test.”27

Lesson Seven: The edTPA Technology Requirements Privilege Certain Candidates and Institutions

The edTPA requirements assume that institutions have the technology that candidates need to complete their portfolio; it is also assumed that candidates possess the technology skills needed to create and share their edTPA portfolio. Purchasing or renting the required recording device for the taping of learning segments creates a financial burden on many candidates and institutions of higher education. Furthermore, the use of technology can prove challenging even to those with basic familiarity with the equipment. Teacher candidates must have the digital literacy skills to not only know how to use the device appropriately but they must also know how to “compress” and upload their videos according the required Pearson specifications. There are many issues related to the video component of the edTPA portfolio, from the quality of the recording to technical problems during the videotaping process.28 One City University of New York teacher candidate contacted her supervisor about her learning segment. She explained that even though the lesson she taught went well and she pleased with the work her students had produced, she was not able use a video clip from that particular lesson because her camera ran out of battery and her backup recording device ran out of hard drive space.

Other candidates reported technical issues in the uploading of their portfolios to the edTPA platform.29 Students have reported it taking them approximately two hours to review all the prompts, submit all the sections, and to upload their materials to meet the specific criteria, which include everything from the size and style of the font to the specifications for compressing and uploading videos.

Buyer Beware

The practical and ethical implications for implementing the edTPA are complex and significant. From our experiences in New York State, it is arguable whether or not the edTPA adequately assesses teacher performance. However, what we can say with certainty is that the edTPA privileges student teacher placements; shifts student teaching of candidates to test prep by candidates; has inherent inconsistencies in the scoring by Pearson; privileges certain candidates and higher education institutions; and makes assumptions about candidates’ technology access and skills.

As teacher educators, we have learned significant lessons, and so have our teacher candidates. “The moral of this story is to predict what the raters might want, and give it to them, no matter how relentlessly repetitive and monotonous the rubrics may be.”30 A follow-up lesson is that the teacher candidates do truly “perform” on this test, determined to create a show that their audience will like. With edTPA portfolios being outsourced nationally, teacher candidates can only hope that their performance earns them applause from the lone worker being paid $75 per portfolio. Although currently the edTPA is being used by more than 70 percent of teacher certification programs in the country, the flaws are evident. A word of advice from New York: Buyer beware.

Deborah Greenblatt is pursuing her Ph.D. in urban education at The Graduate Center at the City University of New York with a concentration in educational policy and leadership. Her dissertation is a mixed-methods study focused on how different variables affect elementary education teacher candidates’ experiences taking the edTPA. She is also an adjunct lecturer at Hunter College of The City University of New York in the Department of Curriculum and Teaching, and a student teaching supervisor for Teachers College, Columbia University.

Kate E. O’Hara, Ph.D., is an assistant professor in the School of Education at New York Institute of Technology. Her research, which employs the use of narrative and autoethnographic studies couched within a sociocultural framework, focuses on the effective use of technology to empower users to become agents of social change and also on teacher education within contexts of power, oppression, and social justice.

Endnotes

1. Nast, “Darling-Hammond: NY Is Messing up Our Teacher Prep Model,” Co-Opt-Ed.

2. American Association of College Teacher Education, “edTPA: Participation Map.”

3. D’Agati, “Teacher Performance Assessment for Initial Certification”; King, “NY TPA Letter from John King.”

4. United University Professions, “Dangler Brings edTPA Issue to SUNY Faculty Senate.”

5. Hogness, “Ed Faculty Protest Makes Gains.”

6. Stanford Center for Assessment, Learning and Equity, “Making Good Choices: A Support Guide for edTPA Candidates.”

7. Au, “What’s a Nice Test like You Doing in a Place like This?,” pp. 22-27.

8. Stillman et al., “To Follow, Reject, or Flip the Script: Managing Instructional Tension in an Era of High-Stakes Accountability,” pp. 22–37.

9. Au, op cit; Lanham, “Resisting the Privatization of American Education,” pp. 111–19.

10. Members of the Elementary Education Masters Program at The University of Illinois at Chicago, “edTPA Boycott: UIC Grad Students Follow Parents and Teachers’ Lead.”

11. New York State United Teachers, “edTPA and the New Teacher Certification Process: Frequently Asked Questions.”

12. Pecheone et al., “2013 edTPA Field Test: Summary Report,” p. 2.

13. Ibid., p. 8.

14. Greenberg and Walsh, “edTPA: Slow This Train down.”

15. DeKorp, email message to author. June 8, 2015.

16. New York State Education Department, Office of Higher Education, “edTPA Myths v. Facts.”

17. DeKorp, email message to author. June 8, 2015.

18. edTPA, “Frequently Asked Questions by Candidates.”

19. Chiu, “edTPA: An Assessment That Reduces the Quality of Teacher Education,” pp. 28–30; Lanham, op cit.; Madeloni and Gorlewski, “Wrong Answer to the Wrong Question: Why We Need Critical Teacher Education, Not Standardization,” pp. 16–21; Madeloni and Hoogstraten, “The Other Side of Fear,” pp. 7–19.

20. United University Professions, “Panel Discusses edTPA at SUNY Brockport.”

21. Okhremtchouk et al., “Voices of Pre-Service Teachers: Perspectives on the Performance Assessment for California Teachers (PACT),” pp. 39–62.

22. D’Agati, “Announcement: Information on Exam Fee Vouchers.”

23. Singer, “Problems with Pearson’s Student Teacher Evaluation System—It’s like Déjà vu All Over Again.”

24. New York University, “NYS Teacher Certification Exams Voucher Lottery | Student Matters.”

25. Au, op cit.

26. Clayton, “Test Teachers, but First Train Them.”

27. Au, op cit. p. 26.

28. Au, op cit.; Darryl McGrath, “NYSUT Wins Delay on edTPA,” p. 13.

29. Lanham, “Resisting the Privatization of American Education.”

30. McConville, “edTPA: You, Too, Shall Pass,” p. 34.

Works cited

American Association of Colleges for Teacher Education. 2015. “edTPA: Participation Map,” (January 13). Retrieved from: http://edtpa.aacte.org/state-policy.

Au, Wayne. 2013. “What’s a Nice Test like You Doing in a Place like This?” Rethinking Schools, (Summer).

Chiu, Stephanie. 2013. “edTPA: An Assessment That Reduces the Quality of Teacher Education.” Teachers College, Columbia University Working Papers in TESOL & Applied Linguistics 14, no. 1 (March 6).

Clayton, Christine D. 2014. “Test Teachers, but First Train Them.” Lohud.com, (April 28). Retrieved from: www.lohud.com/story/opinion/contributors/2014/04/28/test-teachers-first-train/8428219/.

D’Agati, John L. 2014. “Announcement: Information on Exam Fee Vouchers.” NYSED.gov Office of Higher Education, (June 6). Retrieved from: www.highered.nysed.gov/examfeevoucher.html.

edTPA. 2014. “Frequently Asked Questions by Candidates,” edTPA. Retrieved from:
www.edtpa.com/PageView.aspx?f=HTML_FRAG/GENRB_FAQ_Candidates.html

Greenberg, Julie and Kate Walsh. 2012. “edTPA: Slow This Train Down.” PDQ: Pretty Darn Quick Blog, (August 27).

Hogness, Peter. 2014. “Ed Faculty Protest Makes Gains.” Clarion, (May).

Lanham, Amy. 2012. “Resisting the Privatization of American Education.” Transformations: The Journal of Inclusive Scholarship & Pedagogy. (Spring/Summer), Vol. 23 Issue 1.

Madeloni, Barbara and Julie Gorlewski. 2013. “Wrong Answer to the Wrong Question: Why We Need Critical Teacher Education, Not Standardization.” Rethinking Schools, (Summer).

Madeloni, Barbara and Rachel Hoogstraten. 2013. “The Other Side of Fear.” Schools: Studies in Education 10, no. 1 (March).

McConville, Justine. 2014. “edTPA: You, Too, Shall Pass.” Teachers College, Columbia University Working Papers in TESOL & Applied Linguistics 14, no. 1 (March 6).

McGrath, Darryl. 2014. “NYSUT Wins Delay on edTPA.” NYSUT United (May/June).

Members of the Elementary Education Masters Program at The University of Illinois at Chicago. 2014. “edTPA Boycott: UIC Grad Students Follow Parents and Teachers’ Lead.” RADICAL FAGGOT, (March 2). Retrieved from: http://radfag.wordpress.com/2014/03/02/edtpa-boycott-uic-grad-students-follow-parents-and-teachers-lead/.

New York State Education Department, Office of Higher Education. 2014. “edTPA Myths v. Facts.” NYSED (March). Retrieved from: www.highered.nysed.gov/Mythsvs.FactsFinalVersion.pdf.

New York State United Teachers. “edTPA and the New Teacher Certification Process: Frequently Asked Questions.” Last modified April 20, 2014, www.nysut.org/resources/special-resources-sites/edtpa/what-is-edtpa.

New York University. 2015. “NYS Teacher Certification Exams Voucher Lottery | Student Matters.” Retrieved from: http://steinhardt.nyu.edu/site/studentmatters/2013/11/nys-teacher-certification-exams-voucher-lottery/.

Okhremtchouk, Irina, Sumer Seiki, Betsy Gilliland, Comfort Ateh, Matt Wallace, and Anna Kato. 2009. “Voices of Pre-Service Teachers: Perspectives on the Performance Assessment for California Teachers (PACT).” Part of Special Issue on the Use of Teaching Performance Assessment in Teacher Credentialing Programs 18, no. 1 (Spring).

Pecheone, Raymond L., Benjamin Shear, Andrea Whittaker, and Linda Darling-Hammond. 2013. “2013 edTPA Field Test: Summary Report.” Stanford Center for Assessment, Learning and Equity, (November).

Singer, Alan. 2013. “Problems with Pearson’s Student Teacher Evaluation System—It’s like Déjà vu All Over Again.” Huffington Post, (October 14). Retrieved from: www.huffingtonpost.com/alan-singer/problems-with-pearsons-te_b_4093772.html.

Stanford Center for Assessment, Learning and Equity. 2014. “Making Good Choices: A Support Guide for edTPA Candidates.” Stanford University. Retrieved from: www.edtpa.com/Content/Docs/edTPAMGC.pdf.

Stillman, Jamy, and Lauren Anderson. 2011. “To Follow, Reject, or Flip the Script: Managing Instructional Tension in an Era of High-Stakes Accountability.” Language Arts 89, no. 1 (September).

United University Professions. 2014. “Panel Discusses edTPA at SUNY Brockport,” UUPinfo, (May 11). Retrieved from: http://uupinfo.org/committees/teached/articles/140311.php.


download the article

Buyer Beware: Lessons Learned from edTPA Implementation in New York State (PDF)