The Modified Angoff procedure is used to determine the initial passing score for an examination used to certify or license practitioners. It is one of the most popular procedures for setting a criterion-referenced passing point. It is not a norm-referenced method, when a candidate’s pass or fail status is determined by his or her performance in relation to other candidates. Using the Angoff procedure, once the passing point is determined, a candidate’s pass/fail performance is established independently of the group who sat for the exam. Candidates are judged by comparing their performance to an absolute standard, not to other candidates. Theoretically, all candidates can pass or all can fail.
The process begins by selecting a group of job experts to serve as judges. This is typically the client’s examination committee. The group develops a definition of a hypothetical “minimally competent practitioner” (MCP). Then the judges are shown an item on the exam and asked to consider how many of a group of 100 MCPs are likely to answer that question correctly. Initial ratings are discussed, and the judges are allowed to change their ratings, based on the discussion. This process is repeated for each item on the exam. The ratings are averaged across all items and all judges to determine the passing point. Additionally, a 95% confidence interval (CI) is calculated using the standard error of the ratings and the inter-rater reliability of the judges. After the first administration of the initial test form, a “reality check” is performed, i.e., the score distribution is reviewed with the client, and a passing point is selected from within the 95% CI. Once the initial passing point is set the standard is maintained through
statistical equating.
No comments:
Post a Comment