# New research has shown how a single flawed AI model can reject the same person across the entire job market. **By:** Sam Illingworth **Published:** 2026-07-02T08:15:08.494Z **Source:** [LinkedIn](https://www.linkedin.com/posts/samillingworth_aibias-algorithmicfairness-hiring-activity-7478360601132511232-h3pe) --- # Sam Illingworth’s Post [Sam Illingworth](https://uk.linkedin.com/in/samillingworth?trk=public_post_feed-actor-name) New research has shown how a single flawed AI model can reject the same person across the entire job market. Researchers at [Stanford University](https://www.linkedin.com/school/stanford-university/?trk=public_post-text), [Northeastern University](https://www.linkedin.com/school/northeastern-university/?trk=public_post-text), & [Chapman University](https://www.linkedin.com/school/chapman-university/?trk=public_post-text) examined around 4 million applications from 3.4 million applicants across 156 employers, all screened by one vendor’s AI system. They tested each job position separately, which is how discrimination law actually defines the harm. Just under 26% of Black applicants’ and almost 15% of Asian applicants’ applications went to roles where the algorithm produced adverse impact against their group. Had they advanced at the same rate as White applicants, around 40,000 more applications would have moved to the next stage. Because so many employers use the same vendor, that rejection follows a person from one employer to the next. Here is the part that should worry every HR team. The bias is invisible to the standard audit. Pool all the applications together, as most fairness checks do, and the disparity washes out. It shows up only when you test each job as the law requires. A company can pass its fairness audit and still filter out Black and Asian applicants at scale. When everyone runs the same model, one bad decision becomes the whole market’s decision. The full [arXiv](https://www.linkedin.com/company/arxiv?trk=public_post-text) and data are open here on [GitHub](https://www.linkedin.com/company/github?trk=public_post-text): [https://lnkd.in/eP3pTc-F](https://www.linkedin.com/redir/redirect?url=https%3A%2F%2Flnkd%2Ein%2FeP3pTc-F&urlhash=bJU8&trk=public_post-text) [#AIbias](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Faibias&trk=public_post-text) [#AlgorithmicFairness](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Falgorithmicfairness&trk=public_post-text) [#Hiring](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Fhiring&trk=public_post-text) [#AIethics](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Faiethics&trk=public_post-text) [#ResponsibleAI](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Fresponsibleai&trk=public_post-text) [#FutureOfWork](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Ffutureofwork&trk=public_post-text) [#CriticalAILiteracy](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Fcriticalailiteracy&trk=public_post-text) [#HRtech](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Fhrtech&trk=public_post-text) [#Recruitment](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Frecruitment&trk=public_post-text) [#ArtificialIntelligence](https://www.linkedin.com/signup/cold-join?session_redirect=https%3A%2F%2Fwww.linkedin.com%2Ffeed%2Fhashtag%2Fartificialintelligence&trk=public_post-text) I’m pretty sure it does this for age and neurodiversity too. I see these studies often, yet the reality is that the recruting and hiring system is broken and it has nothing to do with AI. Traditional hiring processes are cumbersome, time-consuming, and ineffective at applicant screening and selection. Employers are going to keep using biased AI systems because the alternative isn't working either. It's ethics versus exhaustion and I believe we have to address both. Not to be an annoying european here, but we don't have to accept this 🫣 under the EU AI Act, AI systems within hiring, candidate management, people management (HR) etc are a priori defined as "high risk", exactly because of the risks to fundamental rights (the right to work, the right to non-discrimination). One obligation for providers is identifying, measuring, and then mitigating biases. [Rania Wazir](https://at.linkedin.com/in/raniawazir?trk=public_post_comment-text) and Michael Van Hartskamp are writing the European standard to help providers do this 💪🏾 Not surprised, like most parts of society, you need careful thought to implement changes that actually benefit society. In this case, there's a tension between the HR professional's demands to produce quickly and the society we say we want to become. At the end of the day, we're all just trying to find a good match. You need thinkers, not more models, to imagine something different. This is why I believe the providers that are auditing AI driven / assisted HR tech providers have a significant problem. 1. The data is small in their set. 2. They do not have access to the data inside an end employer using the assured HR tech so effectiveness is hard to quantity. I think we have a real problem. Pymetrics was audited by BABL AI and Northeastern University while this case counters those findings 100%. So how to we trust the audit community at large? This is exactly where AI governance becomes a transition question, not just a model question. If the same vendor logic is used across large parts of the labour market, one flawed screening system can become a repeated barrier to access, recognition and participation. What matters then is not only whether a model “passes” an audit in aggregate, but whether people are filtered out job by job, before human judgement ever begins. We need to move from model fairness to transition fairness: visibility of adverse impact per role, meaningful human review, contestability, and accountability across the full hiring pipeline. Paging [Mark McKenna](https://www.linkedin.com/in/mark-mckenna-1099335?trk=public_post_comment-text). The system punishes anybody who has taken a break from their career for any reason. Cognitive bias as a service at hyperscale. Lovely. 🍷 This is very interesting and considering the impact it may have too.