Oral Reading API FAQ

How does the Oral Reading (OR) API work? What calls can be made to the OR API and what results can be returned?

An Application Programming Interface (API) is essentially a messenger that takes requests, translates, and returns responses (This process is termed “a call.”). MetaMetrics offers several API solutions to support partners and their individual needs in oral reading fluency assessment:

API integration with a partner’s teacher administered assessment. Partners send test passages, Words Correct Per Minute (WCPM) and Accuracy scores and the API returns student Lexile oral reading measure(s).

API integration with a partner’s own automated speech recognition provider. Partners who already assess oral reading fluency using a third party speech provider, MetaMetrics will validate their provider and the API returns student Lexile oral reading measure(s).

API integration as an end to end solution. Partners who are developing a fully automated oral reading fluency solution but do not have a relationship with a third party speech recognition provider can work with MetaMetrics to provide an end-to-end solution. This involves MetaMetrics providing oral reading passages and a fully automated scoring solution using one of our approved third party speech recognition providers.

There are 3 possible OR API calls:
  • Get oral readability measures for texts.
  • Get oral reading measures for students.
  • Get aggregate oral reading measures for students.
Possible returned OR API results include performance metrics, Lexile oral reading measure, and feedback:
  • Performance metrics: WCPM and Accuracy.
  • Student Oral Reading Ability: The student Lexile oral reading measure is appended with the letter “L” for Lexile and is returned with an uncertainty value that describes the degree of uncertainty of the measure expressed in Lexiles.
  • Feedback: Expressed in glow and grow statements, the feedback highlights at least one positive aspect of the student’s performance and identifies at least one area for practice or improvement.

How can the OR API be integrated into a partner’s assessment?

Integrating MetaMetrics’ Oral Reading API is fast and easy. It usually takes partners no more than 24 hours to complete. The integration process includes a scheduled implementation consultation supported by developer documentation to ensure a fully supported experience. Third-party speech recognition companies pre-integrate their voice technology into our oral reading assessment enabling our partners to provide a near-turnkey solution to their clients.

Why does a student audio file receive an “Unscorable” response code?

There are conditions for when unscorable (i.e. unreportable score) results occur and when they do, they receive either a 400 or 422 code response.

Code 400 responses occur when the request included an invalid input, misspelled parameter, or a missing required parameter. Examples include files that did not have a .txt extension or were not UTF-8 encoded.

When a 400 code is produced, double-check the input and parameters to ensure that all requirements are met to receive a valid score.

Code 422 responses occur for two reasons. The first is when there is a problem with the quality of the audio due to too much background noise, a buzzy microphone, an improperly connected microphone, or other similar barriers to clear audio. The second is when the student’s reading performance is too incomprehensible for the speech recognizer to process or has too many unrecognized words. This may occur due to poor articulation, speech that is too quiet, too many unintelligible sounds or too many words that are inconsistent with the passage text. These both produce low accuracy scores.
Note: Oral Reading audio files sent to MetaMetrics ASR that produce a 50% or lower accuracy score will receive a Code 422.

When a 422 code is produced, check to make sure that the student is taking the test in a quiet environment and that they speak clearly. Also, check to make sure the student has the appropriate skill level to take the oral reading test. If the student cannot accurately read the majority of the words in the oral reading passage, they may need more word or sentence-level practice. Ensuring that the student is ready to take an oral reading assessment and speaks clearly will help increase the number of scorable responses.

Why does a file receive a “503: Service is Unavailable” response code?

When a 503 code is produced, it usually means that there has been a temporary server disruption or other service issue that is being addressed. These are usually resolved very quickly, but in some cases it may be necessary to wait up to 15 minutes for the issue to be resolved and then try the call again.

How do measures from multiple test administrations (i.e. passages) get combined into a single Lexile oral reading measure?

When results from multiple test administrations/passages are combined, a weighted average of the previous measures using the uncertainty of the measures (higher uncertainty = lower weight) produces the final Lexile oral reading measure. MetaMetrics recommends that three passages are read within a single assessment event or across administrations that are no more than two weeks apart to receive a reliable Lexile oral reading measure.

How are WCPM and accuracy scores calculated in the OR API?

For partners who currently utilize a third-party automatic speech recognition (ASR) service or are looking for an end-to-end solution from MetaMetrics, the ASR is used to automatically calculate the WCPM and Accuracy. MetaMetrics calculates WCPM by dividing the number of words a student reads correctly in a minute by the total number of words read. Accuracy is calculated by dividing the number of words read correctly by the total number of words attempted. These scores are combined with the oral readability of the test passage and used to return a student’s Lexile oral reading measure.

Unfortunately, different assessments have different interpretations for “word read correctly” (see the italics in the preceding paragraph). For example, if you miss the “-s plural” from the end of the word, have you read it correctly? If you mispronounce a vowel, have you read the word correctly? MetaMetrics uses the following rules, which are based on industry best practices:
  • Words read correctly
    • Correctly pronounced words
    • Words self-corrected within 3 seconds
  • Words read incorrectly (errors)
    • Mispronounced words
    • Omissions
    • Substitutions
    • Hesitations of more than 3 seconds
    • Words read out of order
  • Words that do not count as errors
    • Insertions
    • Repetitions
    • Dialectal differences
  • Unscorable
    • If a response is total silence or completely unintelligible, mark it as “unscorable”

How does Automated Speech Recognition software (ASR) deal with accents?

Many ASR providers gather speech data from students worldwide, including students for whom English is a second language. This data collection makes the ASR software capable of handling accents, dialects, and speech of all kinds. The ASR software used by MetaMetrics and its partners all has robust international datasets and the capability to recognize accents of English spoken around the world.

How does MetaMetrics ensure data security?

MetaMetrics provides a Lexile Oral Reading service implemented as a stateless REST API. Each request contains all the information necessary for the service to calculate and return a result. In this design, audio performances are processed, and results are returned in real-time. MetaMetrics does not collect any additional student identifiers or personally identifiable information. A performance ID is created, and the measurement is associated with the performance ID. In this way, it is possible for metaMetrics to retrieve past performance measures without requiring the original audio and without any student identification. For this reason, it will be impossible for MetaMetrics to retrieve a specific student’s previously sent performance or response data, and it will be impossible for anyone else to do so. MetaMetrics' design decision to not store any customer data is an important security feature of the Oral Reading service. The only area in which data must be stored is authentication and authorization. MetaMetrics uses a number of industry best practice technologies to safely verify customer identity and authorize incoming API requests. These include use of Auth0, JWT access tokens, and TLS.