Predictive PANCE Analytics: How PA Programs See Risk Coming

Most PA programs treat PANCE results as a report card: the scores arrive, the program reacts. The strongest programs run it the other way — they use the assessment data they already collect to model PANCE risk before students test, while intervention can still change the outcome. This article walks through how that works, in plain language: correlation, statistical significance, regression, and the risk-scoring case studies that overturned scores programs assumed were "safe." It is part of our Assessment Systems, Data & Continuous Readiness pillar.

The statistical methods here are edition-neutral. The worked numbers are illustrative case studies from Scott Massey's consulting analyses of individual programs' cohort data — your program's thresholds must be derived from your own cohorts. Last reviewed August 2026.

What does correlation actually tell a PA program?

Correlation measures whether two variables move together — and how strongly. The correlation coefficient (R) runs from −1 to +1; the closer to either end, the stronger the negative or positive relationship. The working conventions: an R around 0.1 is small, around 0.3 is medium, and around 0.5 is large. In PA program outcome data, large correlations (R > 0.5) between program variables and PANCE scores are common — which is exactly why this analysis pays off.

The variables worth testing are the ones the ARC-PA already expects you to collect: admissions criteria and GPA, course outcomes, remediation practices and results, summative evaluation results, end-of-rotation exams (EORE), PACKRAT I and II, and end-of-curriculum (EOC) exams.

What makes a correlation trustworthy?

A correlation is only useful if it's unlikely to be random chance — that's what the p-value measures. The conventional cutoff is p < .05: less than a five percent probability that the pattern you're seeing is noise. Reading R and p together gives four scenarios, and only one of them is actionable: a high R with a low p is a usable predictor. A high R with a high p may be pure chance — refine the variables or grow the sample. A low R with a low p is real but explains too little to act on. A low R with a high p is no signal at all.

One real cohort analysis from our consulting work makes the point. Admissions GPA showed R = −.03 — essentially zero predictive value. Clinical Medicine showed a medium correlation (R = .29) that failed significance (p = .12). Emergency Medicine passed (R = .37, p = .04). EORE Surgery was strong and highly significant (R = .62, p < .01). And PACKRAT carried the highest R and lowest p of the set — the most significant predictor of all. Two lessons: the variable everyone screens on (admissions GPA) told this program nothing about PANCE performance, and the variables that did predict were the ones inside the program's own curriculum.

How does regression turn correlation into prediction?

Correlation says two variables move together; regression puts numbers on the relationship so you can predict one from the other. A regression model reports R (the correlation), R² (the share of outcome variance the predictor explains), and a coefficient for each variable. In one of our analyses, PACKRAT-to-PANCE yielded R² = 0.4586 — meaning roughly 54% of PANCE score variation was driven by something other than PACKRAT. That's the honest frame for all of this: prediction within limits, not prophecy.

The output is a working equation. That same analysis produced: expected PANCE score = (PACKRAT × 2.66) + 66.29 — a student scoring one point higher on PACKRAT than a peer is expected to score 2.66 points higher on the PANCE. Stepwise regression, which tests predictors in combination and keeps only those that independently clear significance, goes further. In one 2020-cohort case, EORE plus number of remediations were the significant predictors (multiple R = 0.76, 58.3% of variance explained). In another, six variables together — EORE, History & Physical I, Clinical Medicine I, Clinical Problem Solving I, the second summative, and undergraduate GPA — reached multiple R = 0.95, explaining 89.8% of PANCE variance. You don't need to run these models yourself; you need to understand what they mean when your statistician hands them over.

What did risk modeling reveal about "safe" scores?

This is where the analysis gets uncomfortable — and valuable. Binary logistic regression with ROC-curve analysis, run on a dataset of 416 individuals, replaced round-number cutoffs with probability-based ones. The findings from that case: a PACKRAT score of 140, widely assumed "not too bad," actually sat in a high-risk zone for PANCE failure. An EOC score at or below 1446 carried roughly a 50% probability of failure — and even 1470 still carried about a 20% chance. At the time of that 2022 analysis, PAEA's published benchmark treated 1400 as the minimum "safe" EOC score; the program-level data said otherwise. (Published benchmarks may have been updated since — the durable lesson is that national round numbers are not a substitute for your own cohort's probabilities.)

Programs act on this. One program in our work raised its first-time-taker EORE requirement from 70% to 75% based on exactly this kind of analysis — a defensible, data-driven policy change rather than a guess.

How do programs put prediction to work?

Three practices turn the statistics into a system. First, risk-tier your cohorts: frameworks like STAR (Students At Risk) classify students into risk levels using GPA, EORE, and PACKRAT scores, with individualized remediation plans for the higher tiers — framed as mentorship, not stigma, because reframing support changes whether students engage with it. Second, intervene on the predictors, not the outcome: if EORE and remediation count carry the signal, those are the dashboards to watch in real time. Third, feed the results back into the self-assessment process: predictive findings are precisely the kind of critical, triangulated analysis the Sixth Edition's C1.01 analysis questions reward — and if your first-time pass rate ever falls to 85% or below, the ARC-PA Pass Rate Report will demand this statistical groundwork on a deadline, retrospectively.

The honest constraint is capacity. Scott ran these analyses alongside a statistician for years, days at a time per cycle — it's the analytical foundation he later built Edulytic Solutions on, so the same early-warning modeling runs continuously instead of episodically. Platform or no platform, the discipline stands: derive your thresholds from your own data, watch them in-cycle, and act while there's still time to change the outcome.

For the companion piece on what to do when the scores have already arrived — segmenting cohorts and tracing outcomes back through the curriculum — read Beyond Pass Rates: Making PANCE Data Actionable.

Frequently asked questions

Can PACKRAT scores predict PANCE performance?
Yes — in program-level analyses PACKRAT is consistently among the strongest PANCE predictors, strong enough to support regression equations for expected scores. But the relationship is program-specific: in one worked case PACKRAT explained under half of PANCE score variance, so it should anchor a risk model, not act as a lone cutoff.

What R value counts as a strong correlation in program data?
By convention, roughly 0.1 is small, 0.3 is medium, and 0.5 is large. PA program outcome data commonly shows large correlations (R above 0.5) between internal assessments and PANCE scores — provided the relationship also clears statistical significance, conventionally p < .05.

Is admissions GPA a good predictor of PANCE success?
Often weaker than assumed. In one cohort analysis from our consulting work, admissions GPA correlated at R = −.03 with PANCE scores — essentially no predictive value — while in-program measures like end-of-rotation exams and PACKRAT carried the real signal. Every program should test its own admissions variables rather than assume.

What is a safe EOC score before the PANCE?
There is no universal safe score — that's the point of risk modeling. In one ROC-curve analysis of program data, an EOC score at or below 1446 carried roughly a 50% probability of PANCE failure, and 1470 still carried about 20%, despite published benchmarks at the time treating 1400 as safe. Derive cutoffs from your own cohorts.

Do PA programs need a statistician to do predictive analytics?
You need the analysis, not necessarily the job title. The program director's job is to understand what a regression model means when it's handed over — the modeling itself can come from a statistician, an institutional partner, or an analytics platform, provided the methods and parameters stay transparent and traceable.

This article is part of the MACS Insights pillar on Assessment Systems, Data & Continuous Readiness — start there for the full picture of building the assessment system these analyses run on.