WEBVTT 00:00:06.480 --> 00:00:08.603 position:50% align:middle Hello. I'm Dr. Hong. 00:00:08.700 --> 00:00:11.219 position:50% align:middle I will talk about standard-setting. 00:00:11.453 --> 00:00:14.935 position:50% align:middle How are the passing standard for REx-PN set? 00:00:16.016 --> 00:00:22.430 position:50% align:middle After this session, you will know why conducting standard-setting, how important it is, 00:00:22.430 --> 00:00:26.328 position:50% align:middle and how the passing standard for the REx-PN was set. 00:00:27.240 --> 00:00:30.540 position:50% align:middle But first, what is standard-setting? 00:00:31.433 --> 00:00:36.923 position:50% align:middle It is a process of determining one or more cut scores on a test. 00:00:37.850 --> 00:00:43.950 position:50% align:middle If you are familiar with educational tests, they sometime classify students into three levels, 00:00:43.950 --> 00:00:47.007 position:50% align:middle basic, proficient, and advanced. 00:00:47.560 --> 00:00:53.726 position:50% align:middle In this case, you need two cuts scores to classify students into three levels. 00:00:54.264 --> 00:00:59.371 position:50% align:middle But for licensure exam, we only need one cut to classify students 00:00:59.371 --> 00:01:02.100 position:50% align:middle into pass or fail. 00:01:02.438 --> 00:01:12.389 position:50% align:middle In July 2021, the registrar and the CEO of the British Columbia College of Nurses and Midwives, BCCNM, 00:01:12.389 --> 00:01:19.600 position:50% align:middle and the executive director and CEO of the College of Nurses of Ontario, CNO, 00:01:19.600 --> 00:01:23.590 position:50% align:middle approved the passing standard for the REx-PN examination. 00:01:24.580 --> 00:01:29.912 position:50% align:middle The passing standard is -0.32 logits. 00:01:30.400 --> 00:01:40.123 position:50% align:middle The passing standard will take effect January 4th, 2022 and will remain in effect through December 2026. 00:01:41.198 --> 00:01:53.098 position:50% align:middle Currently, NCLEX-RN passing standard is 0.00 logits and NCLEX-PN passing standard is -0.18 logits. 00:01:53.620 --> 00:01:57.971 position:50% align:middle Does it mean REx-PN standard is lower than NCLEX? 00:01:58.250 --> 00:01:59.746 position:50% align:middle Not necessary. 00:02:00.140 --> 00:02:06.357 position:50% align:middle REx-PN and NCLEX are different exams, and they are not on the same skill. 00:02:07.384 --> 00:02:13.690 position:50% align:middle You may wonder, "The passing standard is -0.32 logits. 00:02:13.690 --> 00:02:15.351 position:50% align:middle What does it mean?" 00:02:16.156 --> 00:02:20.970 position:50% align:middle In fixed-form exams, students take same questions. 00:02:20.970 --> 00:02:26.597 position:50% align:middle If you get, say, 75% of questions correct, you pass the exam. 00:02:26.720 --> 00:02:30.078 position:50% align:middle The passing standard in the percentage correct. 00:02:30.764 --> 00:02:40.300 position:50% align:middle But REx-PN uses computerized adaptive testing, CAT, which means students take different questions based 00:02:40.300 --> 00:02:41.615 position:50% align:middle on their abilities. 00:02:41.970 --> 00:02:45.719 position:50% align:middle For low-ability students, you work at easy questions. 00:02:45.970 --> 00:02:49.741 position:50% align:middle For high-ability students, you will get harder questions. 00:02:50.187 --> 00:02:56.528 position:50% align:middle Therefore, percentage cannot be used as a passing standard in CAT exams. 00:02:57.426 --> 00:03:04.710 position:50% align:middle Instead, CAT uses current computer technology and measurement of theory to place students 00:03:04.710 --> 00:03:07.246 position:50% align:middle on an ability continuum. 00:03:08.041 --> 00:03:14.506 position:50% align:middle Passing standard is a cut point along the ability continuum that marks the minimum 00:03:14.506 --> 00:03:16.680 position:50% align:middle ability level requirement. 00:03:17.270 --> 00:03:21.622 position:50% align:middle If your ability is above the passing standard, you pass. 00:03:21.754 --> 00:03:27.504 position:50% align:middle If your ability is below or at the passing standard, you fail. 00:03:29.290 --> 00:03:32.670 position:50% align:middle I will use some terminologies in my presentation. 00:03:32.670 --> 00:03:38.505 position:50% align:middle I will say cut score, cut point, passing standard, or pass mark. 00:03:38.505 --> 00:03:40.726 position:50% align:middle They all mean the same thing. 00:03:41.430 --> 00:03:46.980 position:50% align:middle For people who set standards, I will call them standard-setting panelists, 00:03:46.980 --> 00:03:51.629 position:50% align:middle standard-setting judge, or standard-setting subject matter expert. 00:03:53.880 --> 00:03:56.611 position:50% align:middle Why is standard-setting important? 00:03:57.157 --> 00:04:01.048 position:50% align:middle If the standard is set too low, what will happen? 00:04:01.790 --> 00:04:05.822 position:50% align:middle Some candidates who are not qualified will be licensed. 00:04:06.190 --> 00:04:09.659 position:50% align:middle Do you want to be taken care of by those candidates? 00:04:09.659 --> 00:04:11.193 position:50% align:middle Probably not. 00:04:11.941 --> 00:04:18.619 position:50% align:middle As an educator, I understand that sometimes you wish the passing standard is lower 00:04:18.619 --> 00:04:21.185 position:50% align:middle so all your students can pass. 00:04:21.343 --> 00:04:25.694 position:50% align:middle But we must keep public safety in our mind. 00:04:26.520 --> 00:04:29.170 position:50% align:middle What if we set the standard too high? 00:04:29.170 --> 00:04:30.774 position:50% align:middle High standard is good. 00:04:31.430 --> 00:04:36.806 position:50% align:middle If the standard is set too high, some candidates who are qualified will be denied 00:04:36.806 --> 00:04:40.728 position:50% align:middle a license, and we would not have enough nurses. 00:04:41.420 --> 00:04:43.920 position:50% align:middle We can't set the standard too low. 00:04:43.920 --> 00:04:45.910 position:50% align:middle We can't set the standard too high. 00:04:46.160 --> 00:04:53.390 position:50% align:middle Our goal is to set an appropriate standard which accurately reflects the amount of nursing ability 00:04:53.390 --> 00:04:57.743 position:50% align:middle required to practice competently at entry level. 00:04:58.588 --> 00:05:02.447 position:50% align:middle Then, how are the passing standard for REx-PN set? 00:05:02.640 --> 00:05:08.399 position:50% align:middle Where does REx-PN cut score, -0.32, come from? 00:05:08.816 --> 00:05:12.159 position:50% align:middle Here are the steps for setting the passing standard. 00:05:12.285 --> 00:05:14.920 position:50% align:middle One, select the panel of judges. 00:05:14.920 --> 00:05:17.072 position:50% align:middle We need people to do it. 00:05:17.540 --> 00:05:24.795 position:50% align:middle Two, define minimally competent candidate, which is a key concept when you set a passing standard. 00:05:25.620 --> 00:05:29.145 position:50% align:middle Three, train judges on the standard-setting method. 00:05:29.145 --> 00:05:35.081 position:50% align:middle It's not like you just think about a number, there is a systematic way to do it. 00:05:35.320 --> 00:05:38.789 position:50% align:middle We need to train them so they understand what they are doing. 00:05:39.484 --> 00:05:42.990 position:50% align:middle Fourth, conduct a standard-setting workshop. 00:05:43.197 --> 00:05:47.487 position:50% align:middle This is a two-day workshop to set a passing standard. 00:05:48.240 --> 00:05:52.323 position:50% align:middle The final step is to establish the passing standard. 00:05:52.323 --> 00:05:55.272 position:50% align:middle I will describe these steps one by one. 00:05:56.491 --> 00:05:59.455 position:50% align:middle First step, select the panel of judges. 00:05:59.870 --> 00:06:03.237 position:50% align:middle You know it is high-stake decision. 00:06:03.237 --> 00:06:06.471 position:50% align:middle Can you guess how many judges are needed? 00:06:06.831 --> 00:06:09.722 position:50% align:middle The answer is nine. 00:06:09.959 --> 00:06:17.656 position:50% align:middle Minimum of 9 judges are needed, but we had 12 in June 2021 standard-setting workshop. 00:06:18.368 --> 00:06:21.833 position:50% align:middle Panel includes at least one faculty member. 00:06:22.180 --> 00:06:29.180 position:50% align:middle Panel includes at least one entry-level nurse with less than 12 months of clinical experience. 00:06:30.320 --> 00:06:35.272 position:50% align:middle Another important question is who are qualified as judges? 00:06:35.402 --> 00:06:41.687 position:50% align:middle We follow the criteria approved by Practical Nurse Exam Committee, PNEC. 00:06:43.520 --> 00:06:47.331 position:50% align:middle Step two is to define minimally competent candidate. 00:06:47.790 --> 00:06:57.060 position:50% align:middle Think about your 20 nursing graduates here from left to right, order them from lowest ability 00:06:57.060 --> 00:06:58.784 position:50% align:middle to the highest ability. 00:06:59.010 --> 00:07:07.440 position:50% align:middle Then, I ask you to draw a dividing line from where you're willing to hire them as an entry-level nurse. 00:07:08.070 --> 00:07:15.020 position:50% align:middle If you draw a line here, on the left side of the line, their ability is not so much. 00:07:15.130 --> 00:07:20.261 position:50% align:middle You wish they had never enrolled in the nursing program at the first place. 00:07:21.000 --> 00:07:24.555 position:50% align:middle For the people on the right side, they are good enough. 00:07:24.800 --> 00:07:28.786 position:50% align:middle The most-right one is your top one student. 00:07:28.786 --> 00:07:36.620 position:50% align:middle But our focus here is not the ones on the left, not the ones on the far right. 00:07:36.620 --> 00:07:39.785 position:50% align:middle Our focus is only one person. 00:07:40.090 --> 00:07:46.780 position:50% align:middle Who is the luckiest person that you are willing to hire that just have enough knowledge? 00:07:47.380 --> 00:07:50.272 position:50% align:middle The one on the borderline, right? 00:07:50.710 --> 00:07:56.016 position:50% align:middle This is why you'll think a lot, and then you'll say, "Okay, let's draw the line here. 00:07:56.016 --> 00:08:02.971 position:50% align:middle I think I can bear with you, train you, and you are safe enough to be an entry-level nurse." 00:08:03.340 --> 00:08:08.342 position:50% align:middle This one on the fence is the minimally competent candidate. 00:08:10.080 --> 00:08:15.961 position:50% align:middle To establish a passing standard, we focus on the candidates that possess just enough 00:08:15.961 --> 00:08:20.940 position:50% align:middle knowledge to practice entry-level nursing safely and effectively. 00:08:21.045 --> 00:08:26.511 position:50% align:middle We don't focus on students whose ability is very high or very low. 00:08:27.040 --> 00:08:31.027 position:50% align:middle We only concentrate on those who are on the fence. 00:08:31.690 --> 00:08:37.744 position:50% align:middle In the workshop, we would spend five hours to define minimally competent candidate. 00:08:38.160 --> 00:08:43.488 position:50% align:middle Our nurse will go through all the activity statements in eight content areas and look 00:08:43.488 --> 00:08:48.337 position:50% align:middle at how a minimally competent candidate would do for each activity. 00:08:49.020 --> 00:08:52.567 position:50% align:middle We even give a minimally competent candidate a name. 00:08:52.730 --> 00:08:55.210 position:50% align:middle Who is that student on the fence? 00:08:55.210 --> 00:08:57.149 position:50% align:middle Oh, that's Patty. 00:08:57.570 --> 00:09:01.118 position:50% align:middle How would Patty do in these activities statements? 00:09:01.893 --> 00:09:08.980 position:50% align:middle We spend a lot of time to discuss this concept, so every judge will be on the same page. 00:09:10.680 --> 00:09:16.805 position:50% align:middle After minimally competent candidate concept has been built, the third step is to train judges 00:09:16.805 --> 00:09:18.594 position:50% align:middle on standard-setting method. 00:09:19.480 --> 00:09:23.706 position:50% align:middle The method is called Modified Angoff Method. 00:09:24.278 --> 00:09:31.140 position:50% align:middle In this method, we ask judges, "Imagine, out of 100 minimally competent candidates, 00:09:31.140 --> 00:09:34.411 position:50% align:middle how many would answer the item correctly?" 00:09:34.860 --> 00:09:39.222 position:50% align:middle Instead of 1 Patty, now, I give you 100 Pattys. 00:09:39.320 --> 00:09:45.128 position:50% align:middle They are all minimally competent candidates standing right here and answering the same question. 00:09:45.550 --> 00:09:49.107 position:50% align:middle How many would answer the item correctly? 00:09:49.690 --> 00:09:53.109 position:50% align:middle The ratings will be between 1 to 99. 00:09:53.456 --> 00:10:03.066 position:50% align:middle If you think this item is so difficult, 100 Pattys, only 1 can answer it correctly, then the rating is 1. 00:10:03.226 --> 00:10:09.240 position:50% align:middle If you think this item is so easy, 99 Pattys can answer it correctly, 00:10:09.240 --> 00:10:11.532 position:50% align:middle then the rating is 99. 00:10:11.792 --> 00:10:14.536 position:50% align:middle But these are two extremes. 00:10:14.536 --> 00:10:15.831 position:50% align:middle Think about that. 00:10:16.057 --> 00:10:23.770 position:50% align:middle If you have 4-option multiple-choice item, even by guessing, 25 out of 100 Pattys would get 00:10:23.770 --> 00:10:25.606 position:50% align:middle it correct, right? 00:10:25.990 --> 00:10:30.264 position:50% align:middle So, most time, the rating will not below 25. 00:10:30.770 --> 00:10:38.940 position:50% align:middle On the other hand, the rating will not be, like, 90, 95, because for minimally competent candidates, 00:10:38.940 --> 00:10:41.052 position:50% align:middle they're not that competent, right? 00:10:42.050 --> 00:10:49.689 position:50% align:middle If you think an item is really easy, probably, 70 or 80 Pattys would get it correct. 00:10:50.070 --> 00:10:55.754 position:50% align:middle If you think item is middle-range difficulty, probably 50, 60. 00:10:56.416 --> 00:11:01.177 position:50% align:middle For difficult one, maybe 30, 40 out of 100. 00:11:01.747 --> 00:11:06.225 position:50% align:middle Therefore, lower ratings means more difficult item. 00:11:07.186 --> 00:11:12.151 position:50% align:middle Another thing you need to pay attention is would versus should. 00:11:12.400 --> 00:11:17.341 position:50% align:middle If you think about should, everybody should answer this item correctly. 00:11:17.670 --> 00:11:19.810 position:50% align:middle Too high expectations. 00:11:19.810 --> 00:11:21.576 position:50% align:middle Let's face reality. 00:11:22.336 --> 00:11:28.822 position:50% align:middle Give you 100 Pattys, how many of them would answer this item correctly? 00:11:28.822 --> 00:11:31.446 position:50% align:middle No should. Use would. 00:11:32.682 --> 00:11:35.289 position:50% align:middle Now, treat yourself as a judge. 00:11:35.398 --> 00:11:38.839 position:50% align:middle I will show you an item, and you give me a rating. 00:11:39.580 --> 00:11:45.206 position:50% align:middle Think about 100 minimally competent candidates, Patty or somebody else. 00:11:45.322 --> 00:11:49.216 position:50% align:middle In your mind, you know somebody right on the fence. 00:11:50.040 --> 00:11:56.575 position:50% align:middle Give you 100 of that person, how many would answer this item correctly? 00:11:58.490 --> 00:12:00.924 position:50% align:middle Read the item, and give me a rating. 00:12:01.176 --> 00:12:05.372 position:50% align:middle If you ask me which one is the key, I will not tell you. 00:12:06.204 --> 00:12:09.919 position:50% align:middle If I tell you the key, you will think this item is easy. 00:12:10.190 --> 00:12:13.360 position:50% align:middle And we do the same thing in the real workshop. 00:12:13.360 --> 00:12:17.280 position:50% align:middle We do not tell judges the key in the first round. 00:12:18.510 --> 00:12:20.605 position:50% align:middle Okay, what is your rating? 00:12:20.669 --> 00:12:24.054 position:50% align:middle Sixty? Seventy? Thank you. 00:12:24.220 --> 00:12:27.455 position:50% align:middle Now, I tell you, the key is A. 00:12:29.270 --> 00:12:32.265 position:50% align:middle That's what we do in our standard-setting workshop. 00:12:32.720 --> 00:12:34.610 position:50% align:middle It's a two-day workshop. 00:12:34.610 --> 00:12:42.180 position:50% align:middle Judges will review and rate 120 operational items across all test plan area. 00:12:42.180 --> 00:12:43.640 position:50% align:middle You just did one. 00:12:43.640 --> 00:12:48.901 position:50% align:middle And our judges will do 120, one by one. 00:12:49.350 --> 00:12:55.687 position:50% align:middle These items across all difficulty level and is a representative mini pool. 00:12:55.995 --> 00:12:59.700 position:50% align:middle After they give us a rating, we will ask them to discuss, 00:12:59.700 --> 00:13:06.330 position:50% align:middle why you think this item is easy for Pat or why do you think this item is difficult to Pat? 00:13:07.080 --> 00:13:11.201 position:50% align:middle After discussion, they would do second-round rating. 00:13:11.350 --> 00:13:15.564 position:50% align:middle And we will use second-round rating to calculate cut score. 00:13:16.418 --> 00:13:23.623 position:50% align:middle But you may wonder how to connect these ratings to -0.32 logits. 00:13:24.030 --> 00:13:29.475 position:50% align:middle What you get is 60 out of 100, 70 out of 100. 00:13:29.475 --> 00:13:34.304 position:50% align:middle It has nothing to do with -0.32 logits. 00:13:34.729 --> 00:13:38.559 position:50% align:middle How do you transform these ratings to logit scale? 00:13:39.262 --> 00:13:45.450 position:50% align:middle In REx-PN, it's not like how many percentages you get right, then you pass, 00:13:45.450 --> 00:13:48.074 position:50% align:middle because everyone gets different questions. 00:13:48.480 --> 00:13:53.943 position:50% align:middle You cannot use a percentage as a passing standard in REx-PN exam. 00:13:54.713 --> 00:13:58.399 position:50% align:middle Let me show you how to calculate a cut point. 00:14:00.415 --> 00:14:06.560 position:50% align:middle Here, instead of nine judges, I only listed two judges, Judge 1 and Judge 2. 00:14:06.724 --> 00:14:10.647 position:50% align:middle Instead of 120 items, I listed 3 items. 00:14:10.647 --> 00:14:12.246 position:50% align:middle But you'll get the idea. 00:14:13.120 --> 00:14:21.113 position:50% align:middle For Item 1, Judge 1 thinks 50 out of 100 minimally competent candidates would get this item correct. 00:14:21.113 --> 00:14:23.814 position:50% align:middle So, the rating is 0.5. 00:14:24.151 --> 00:14:31.451 position:50% align:middle But Judge 2 thinks it is very difficult item, only 33 out of 100 Pattys will get it correct. 00:14:31.451 --> 00:14:34.086 position:50% align:middle So, the rating is 0.33. 00:14:34.687 --> 00:14:40.258 position:50% align:middle In our workshop, we will have nine judges and nine ratings for Item 1. 00:14:40.590 --> 00:14:48.736 position:50% align:middle For Item 2, Judge 1 thinks it is difficult, and only 25 out of 100 Pattys would get it correct. 00:14:49.546 --> 00:14:55.050 position:50% align:middle Judge 2 thinks it is similar to Item 1, and the rating is 0.33. 00:14:55.112 --> 00:14:57.142 position:50% align:middle And so on for Item 3. 00:14:57.855 --> 00:15:03.001 position:50% align:middle If you add 3 items together for Judge 1, you get 1.75. 00:15:03.017 --> 00:15:06.313 position:50% align:middle For Judge 2, it is 1.16. 00:15:06.448 --> 00:15:08.037 position:50% align:middle Then you take average. 00:15:08.160 --> 00:15:11.350 position:50% align:middle Add the two numbers together and divide by two. 00:15:11.350 --> 00:15:14.987 position:50% align:middle You have cut score 1.46. 00:15:15.628 --> 00:15:23.238 position:50% align:middle For the 3-item test, based on 2 judges, if you get 1.46 out of 3, you pass. 00:15:23.238 --> 00:15:24.563 position:50% align:middle It makes sense. 00:15:24.720 --> 00:15:28.721 position:50% align:middle For three items, if you get half correct, you pass. 00:15:28.980 --> 00:15:33.179 position:50% align:middle But still, how does it transform to logit scale? 00:15:33.710 --> 00:15:37.941 position:50% align:middle I know, as a nurse, you are very familiar with the growth chart. 00:15:38.170 --> 00:15:40.340 position:50% align:middle Horizontal axis is age. 00:15:40.340 --> 00:15:42.270 position:50% align:middle Vertical axis is height. 00:15:42.862 --> 00:15:46.317 position:50% align:middle If I tell you the age, you can tell me the height. 00:15:47.280 --> 00:15:51.722 position:50% align:middle On the other hand, if I say the height, you can tell me the age. 00:15:52.577 --> 00:15:55.750 position:50% align:middle Now, let us look at unit of measurement. 00:15:55.750 --> 00:15:59.024 position:50% align:middle What is the unit of measurement for height? 00:15:59.310 --> 00:16:01.408 position:50% align:middle Inches or centimeters. 00:16:02.440 --> 00:16:05.578 position:50% align:middle What is the unit of measurement for age? 00:16:05.578 --> 00:16:06.711 position:50% align:middle Years. 00:16:06.817 --> 00:16:08.392 position:50% align:middle Now, you have the idea. 00:16:08.723 --> 00:16:14.513 position:50% align:middle These are two different things, but you can draw the line and transform to each other. 00:16:15.480 --> 00:16:18.614 position:50% align:middle It's similar in the psychometric field. 00:16:19.770 --> 00:16:22.599 position:50% align:middle Now, horizontal axis is ability. 00:16:22.599 --> 00:16:27.517 position:50% align:middle Vertical axis is the probability you get item correct. 00:16:28.205 --> 00:16:34.249 position:50% align:middle In previous growth chart, you have a line that shows for girls from 2 to 20. 00:16:34.249 --> 00:16:37.003 position:50% align:middle The older you are, the taller you are. 00:16:37.665 --> 00:16:45.050 position:50% align:middle Here, the line shows if your ability is higher, you have more chance to get the item correctly. 00:16:45.050 --> 00:16:46.546 position:50% align:middle Makes sense, right? 00:16:47.040 --> 00:16:52.380 position:50% align:middle If your ability is zero, what is the probability you'll get the item correct? 00:16:52.460 --> 00:16:58.499 position:50% align:middle Find the zero on horizontal axis, go up to reach the line, and then go left. 00:16:58.590 --> 00:17:00.934 position:50% align:middle It's about 0.4. 00:17:01.822 --> 00:17:06.017 position:50% align:middle If your probability is 0.6, what is your ability? 00:17:06.309 --> 00:17:07.647 position:50% align:middle About 1. 00:17:07.854 --> 00:17:10.599 position:50% align:middle What is the unit of measurement for ability? 00:17:11.050 --> 00:17:14.034 position:50% align:middle Unit of measurement is logit. 00:17:15.052 --> 00:17:16.739 position:50% align:middle Now, let's look at this. 00:17:16.739 --> 00:17:20.760 position:50% align:middle Horizontal axis is ability, the unit is logit. 00:17:20.930 --> 00:17:26.599 position:50% align:middle Vertical axis that goes from zero to three for our three-item test. 00:17:26.910 --> 00:17:31.435 position:50% align:middle The minimum you can get is zero, the maximum you can get is three. 00:17:31.990 --> 00:17:36.793 position:50% align:middle The line shows the high ability you are, more likely, you can get three. 00:17:37.410 --> 00:17:42.757 position:50% align:middle Do you remember our cut score for the three-item test based on two judges? 00:17:43.197 --> 00:17:45.421 position:50% align:middle 1.46. 00:17:45.751 --> 00:17:49.041 position:50% align:middle Then, what is the largest skill for this cut? 00:17:49.140 --> 00:17:56.525 position:50% align:middle You'll find 1.46 on the vertical axis, go right, reach the line, and then going down. 00:17:57.070 --> 00:17:59.992 position:50% align:middle It's about -1 logit. 00:18:00.840 --> 00:18:09.400 position:50% align:middle For REx-PN, we have ratings from 120 items, so the vertical axis goes from 0 to 120. 00:18:09.621 --> 00:18:19.980 position:50% align:middle Base on the ratings from 9 judges or more, we draw the line down, it is -0.32 logits for REx-PN. 00:18:19.980 --> 00:18:24.932 position:50% align:middle This is the passing standard recommended by the panel of judge. 00:18:27.610 --> 00:18:31.511 position:50% align:middle Then, the final step is to establish the passing standard. 00:18:31.976 --> 00:18:41.803 position:50% align:middle PNEC recommended the passing standard to the registrar and the CEO of BCCNM and the executive director and CEO 00:18:41.803 --> 00:18:45.450 position:50% align:middle of CNO considering two things. 00:18:45.950 --> 00:18:52.766 position:50% align:middle Of course, the first thing considered is the passing standard recommended by the standard-setting panel. 00:18:53.684 --> 00:19:01.188 position:50% align:middle PNEC also considered candidate performance from field testing in 2020 and 2021. 00:19:01.653 --> 00:19:09.759 position:50% align:middle PNEC understands if you recommended this standard, how many would pass based on field testing data. 00:19:10.430 --> 00:19:15.229 position:50% align:middle We want PNEC to understand the impact of their recommendation. 00:19:16.529 --> 00:19:25.023 position:50% align:middle Then, the registrar and the CEO of BCCNM and the executive director and the CEO of CNO approved 00:19:25.023 --> 00:19:30.144 position:50% align:middle the passing standard of -0.32 logits. 00:19:30.636 --> 00:19:32.131 position:50% align:middle Take-home message. 00:19:32.287 --> 00:19:37.171 position:50% align:middle Standard-setting is a policy decision, and no true value exists. 00:19:37.194 --> 00:19:40.340 position:50% align:middle As a psychometrician, I like true value. 00:19:40.340 --> 00:19:42.791 position:50% align:middle Like, 1 plus 1 must be 2. 00:19:43.335 --> 00:19:48.330 position:50% align:middle Unfortunately, in standard-setting, there is no true value. 00:19:48.510 --> 00:19:49.999 position:50% align:middle Think about that. 00:19:50.058 --> 00:19:59.090 position:50% align:middle Are there true differences between -0.32 logits to -0.3199 logits? 00:19:59.100 --> 00:20:00.407 position:50% align:middle Not really. 00:20:01.080 --> 00:20:07.566 position:50% align:middle Even though we do standard-setting in very rigorous way, there is no true value. 00:20:08.082 --> 00:20:12.912 position:50% align:middle Passing standard is related to mission of the organization and exam. 00:20:13.210 --> 00:20:17.846 position:50% align:middle For REx-PN, the mission is public protection. 00:20:18.550 --> 00:20:23.140 position:50% align:middle When we set the standard, we need to keep the mission in our mind. 00:20:23.586 --> 00:20:31.639 position:50% align:middle Standards should be revisited when test specifications, scope of practice, education programs, 00:20:31.639 --> 00:20:34.005 position:50% align:middle and/or other factors change. 00:20:34.210 --> 00:20:38.971 position:50% align:middle When we change a test plan, we need to reevaluate the standards. 00:20:39.570 --> 00:20:40.840 position:50% align:middle Thank you very much. 00:20:40.840 --> 00:20:43.732 position:50% align:middle I hope the presentation is helpful to you. 00:20:44.000 --> 00:20:46.817 position:50% align:middle We will have a live Q&A coming up shortly. 00:20:46.914 --> 00:20:52.888 position:50% align:middle You can enter any questions you might have in the Q&A box, and we will answer them later.