How AI Creates New Credentialing Gatekeepers

In July 2024, a federal judge in San Francisco declined to throw out a lawsuit against a company that sells hiring software. The plaintiff, Derek Mobley, said he had applied to more than a hundred jobs at employers using Workday's screening tools and was turned down every time. The court's order did not decide whether he was right. It said something quieter and, in my view, more important: a vendor that screens people on an employer's behalf might be answerable for what it screens out.

That case is a small window onto a large change. Credentials are not disappearing under pressure from AI, as the headlines about dead degrees and dead resumes suggest. The gate is moving, and the people and products standing at the new gate are different from the ones at the old one.

What Does a Credential Actually Do?

A credential is a shortcut for trust. An employer cannot watch you work for four years before deciding to hire you, so a degree stands in for that observation. A patient cannot test a surgeon, so a license stands in for the test. The shortcut works because someone with a reputation to protect vouched for you, and because producing the evidence was costly enough that faking it was hard.

The second half of that sentence is where AI bites. A well-written cover letter, a clean piece of code, or a thoughtful take-home essay used to be informative because it took effort. Now a competent version of almost any of those can be generated in seconds. When the cost of producing evidence collapses, the evidence stops telling you much. The old signals were priced by effort, and effort has been repriced.

I think this is the real story underneath the cheating panic. Most applicants and students have not suddenly become dishonest. The thing they hand over simply no longer proves what it used to prove.

Why Do Old Signals Fail When AI Is Cheap?

Take the hiring funnel. Applying is nearly free, and AI tools will tailor a resume to any listing, so postings draw far more applications than a person can read. Recruiters add software to sort the pile. Applicants use more AI to get past the software. Each side escalates and the signal in the middle gets thinner.

Education runs the same loop. Instructors assign essays, students use chatbots, instructors run detectors, and students run rewriters. Almost nobody in that loop learns what the student can do.

When a signal degrades, institutions rarely give up on signaling. They look for somewhere else to put their trust, and that somewhere is usually a third party that claims it can tell the real from the synthetic. That is the opening for the new gatekeeper.

Who Are the New Gatekeepers?

I count five kinds taking shape. They are different enough from each other that it helps to look at them one at a time.

Screening and ranking software

Applicant tracking systems and matching tools increasingly decide which resumes a human ever reads. A candidate who never reaches a person has, in practice, been rejected by a product, and whoever wrote the scoring rules has quietly become a credentialing body.

The procedural history of Mobley v. Workday, Inc. (N.D. Cal., No. 3:23-cv-00770) is worth stating carefully. In an order dated 12 July 2024, the court denied in part Workday's motion to dismiss and allowed the plaintiff's theory that Workday could be liable as an agent of the employers using its tools. It did not accept the alternative theory that Workday was itself an employment agency. In May 2025 the court conditionally certified a collective action on the Age Discrimination in Employment Act claim, covering applicants aged 40 and over. Neither ruling found that the software discriminated; both addressed whether the case could go forward and on what theory. I have not confirmed developments after my most recent sources, so anyone relying on the case should read the current docket rather than this summary.

Detection and proctoring tools

AI-text detectors and remote proctoring services sit between a test-taker and a result. Their output can decide whether someone passes or is referred for misconduct.

Their reliability is contested in a way that is documented. In a 2023 paper in the journal Patterns (volume 4, issue 7), Liang and colleagues tested several GPT detectors on essays written by non-native English speakers for the TOEFL exam and found that the detectors misclassified more than half of them as AI-generated, while performing much better on essays by US-born eighth graders. That is one study of particular detectors at a particular time, and the tools have changed since. It still shows how a detector can penalize the writers least able to argue with it.

The EU AI Act names this territory directly. Annex III, point 3 of Regulation (EU) 2024/1689 lists education and vocational training systems as high-risk, including those used to determine access or admission, to evaluate learning outcomes, and, at point 3(d), for "monitoring and detecting prohibited behaviour of students during tests." Separately, Article 5(1)(f) prohibits AI systems that infer emotions of people in workplaces and educational institutions, except for medical or safety reasons, which matters for proctoring products that claim to read stress or attention. That prohibition has applied since 2 February 2025.

Vendor-issued AI skill certificates

When every worker is told to become "AI literate," someone sells the proof. Model makers, cloud providers, and training companies have launched certificates and badges that say you know how to use their tools. Some will prove useful to employers. All of them define competence in the terms of the company that sells the tools being tested, and a certificate holder can rarely tell how much the market will come to value the badge.

Regulation adds demand. Article 4 of the EU AI Act requires providers and deployers to take measures to ensure, to their best extent, a sufficient level of AI literacy among their staff and others operating AI systems on their behalf. It has applied since 2 February 2025. It does not require any particular certificate, but a legal duty to show literacy creates a market for something to show. The Commission's November 2025 "Digital Omnibus" proposal would reshape this duty, shifting emphasis from an obligation on companies toward encouragement by the Commission and Member States. Whether that proposal has been adopted in final form is something to confirm in the Official Journal before relying on either version.

Verification and identity infrastructure

If anyone can generate a convincing diploma, transcript, or portfolio, the question becomes who can verify it. The W3C Verifiable Credentials Data Model 2.0 reached W3C Recommendation status on 15 May 2025, and the 1EdTech Open Badges 3.0 specification builds on the Verifiable Credentials model. Together they let a school or employer issue a cryptographically signed record that the holder carries and a third party can check. That is a real improvement over scanned PDFs.

A signed record is only as meaningful as its issuer, though, and a verification network is only as open as its rules about who may issue and who may check. The specifications themselves do not say who gets on a trust list. Whoever operates the registry, the wallet, or the list ends up with influence over who counts, and that governance is where the next argument will happen.

Independent bias auditors

The fifth gatekeeper is the least obvious, because it guards the other gates. New York City's rule requires an independent auditor to review certain hiring tools, which means a new professional class now certifies the tools that certify people. Auditors are paid by the entities they assess, a familiar arrangement from financial audit, with familiar questions about independence. That does not make the audits worthless. It makes the choice of auditor and the scope of the audit worth reading closely.

What Has the Law Said So Far?

New York City Local Law 144 of 2021

Local Law 144 covers "automated employment decision tools" used to screen candidates for employment or employees for promotion in New York City. It took effect on 1 January 2023, and the Department of Consumer and Worker Protection began enforcing it on 5 July 2023. The main requirements are these:

• An independent bias audit of the tool, completed within one year before it is used.
• A publicly posted summary of the most recent audit and the date the tool was first used.
• Notice to candidates and employees at least ten business days before the tool is used, including the job qualifications and characteristics it will assess, and information about requesting an alternative selection process or accommodation.
• Penalties of up to $500 for a first violation and for each additional violation on the same day, and $500 to $1,500 for each subsequent violation, with each day of unlawful use counted as a separate violation.

Compliance appears to be thin. A 2024 paper presented at the ACM FAccT conference, "Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability," examined 391 employers and found that only 18 had posted audit reports and 13 had posted the required transparency notices. The New York State Comptroller's office also published an audit of the Department's enforcement of the law in December 2025 that was critical of how complaints were handled. A rule that depends on employers to disclose and on a city agency to chase non-disclosure leaves a lot of room.

The EU AI Act and its timeline

Annex III of Regulation (EU) 2024/1689 lists the high-risk uses that matter here. Point 4(a) covers AI systems used for recruitment or selection, in particular to place targeted job advertisements, to analyse and filter applications, and to evaluate candidates. Point 4(b) covers decisions on promotion and termination, task allocation, and performance monitoring. Point 3, discussed above, covers education.

The dates are where practitioners get confused, so here is the sequence:

The delay was a proposal when I last confirmed its status. Because the final text is set by the Parliament and the Council, I would not tell a compliance team that Annex III duties are either enforceable or deferred without checking the Official Journal for the adopted version. When they apply, high-risk obligations include risk management, data governance, logging, transparency to deployers, and human oversight (Articles 9 to 15), and deployers that are employers must inform workers and their representatives before putting such a system into use (Article 26(7)).

Where US and EU privacy law reach

GDPR Article 22 gives people the right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects. In SCHUFA Holding (Case C-634/21, 7 December 2023), the Court of Justice held that producing a credit score can itself be such a decision when a third party draws strongly on it. A screening score that employers treat as decisive could face similar reasoning, though the case concerned credit and has not been tested on hiring tools in the same way.

In the United States, the main exposure has been ordinary anti-discrimination law. The EEOC issued technical assistance in 2022 (on the ADA) and in May 2023 (on adverse impact under Title VII) about algorithmic selection tools, and the agency removed those documents from its website in January 2025. The statutes did not change. Title VII, the ADEA, the ADA, and the Uniform Guidelines on Employee Selection Procedures, including the four-fifths rule of thumb at 29 CFR 1607.4(D), remain the framework courts apply, and Mobley is a live example.

What strikes me about all of these rules is what they leave alone. They regulate tools. They say much less about which credentials gain power because the tools prefer them. If a screening product learns that applicants with a certain badge get hired more often, the badge gains authority without anyone deciding it should.

Why Do New Gatekeepers Tend to Concentrate Power?

Three forces push in the same direction.

Verification rewards scale. A detection model improves as it sees more submissions, a screening product as it sees more hires, and a registry as more issuers join. Each of those favors a few large players.

The buyer is not the person being judged. The employer picks the screener, the school picks the proctoring service, and the applicant has no vote. A tool that wrongly rejects a good candidate costs the company nothing it can see, while a tool that lets one bad hire through costs a lot, so products drift toward serving the buyer's fear of liability.

Measures turn into targets. Once a score decides outcomes, people tune themselves to the score. Candidates learn which keywords the filter likes and students learn which text passes the detector, and the score drifts from what it was meant to capture. That has always been true of metrics and it happens faster when the tuning is done with AI.

For a wider look at how rules and the entities they govern end up entangled, see my piece on regulatory capture in AI governance.

Who Wins and Who Loses?

Vendors that become the default screening or verification layer win, along with large employers that can afford custom assessments. Candidates with strong networks also win. When written artifacts are suspect, a warm introduction from someone who has watched you work becomes more valuable, and it is also about as unevenly distributed as anything in the labor market.

The people most exposed are those whose paths do not match the training data: self-taught workers, career changers, people with gaps in their history, and non-native English writers. The Liang study is the one piece of evidence I would point to here, and it covers only detectors and only writing. The broader worry that pattern-matching systems reproduce the past is a reasonable inference and an open empirical question, and Mobley is partly an attempt to test it in court.

None of this needs bad faith. It is what accumulates when many reasonable-looking purchasing decisions pile up.

What Should Employers and Institutions Ask Before Buying?

For a compliance reader, "ask the vendor hard questions" is not enough. Here is a procurement checklist that maps to the instruments above.

Before purchase

1. Classification. Does the vendor state whether the tool is an automated employment decision tool under Local Law 144, a high-risk system under Annex III, or neither, and on what reasoning?
2. Audit evidence. Obtain the most recent independent bias audit, the auditor's identity, the data used, and the date. Check that the auditor is not the vendor's affiliate.
3. Error rates. Ask for false positive and false negative rates by relevant group, and the sample they came from. For detectors, ask specifically about non-native English writers.
4. Adverse impact method. Ask how selection rates are compared, and whether the vendor's analysis would support a four-fifths check under 29 CFR 1607.4(D).
5. Solely automated decisions. If GDPR applies, document whether the tool makes decisions with legal or similarly significant effects under Article 22, and what human review exists.

Contract clauses

• Audit cooperation: the vendor supplies data and technical support for annual bias audits and for any regulator or court request.
• Notice and transparency: the vendor provides the information the buyer needs for candidate notices (ten business days under Local Law 144) and for deployer duties under the AI Act.
• Change control: material model changes require notice, and trigger re-audit.
• Liability and indemnity: allocation of responsibility when the tool is alleged to discriminate, with the Mobley agent theory in mind.
• Logs and retention: the vendor keeps decision logs long enough to answer a complaint.
• Human override: a named person with authority to overrule a score, and a defined appeal path for the person scored.
• Data use: restrictions on using candidate data to train the vendor's other products.

After deployment

• Run your own selection-rate comparison on your own applicant pool at least annually.
• Keep records of overrides and appeals, since they show whether human oversight is real.
• Where a short live conversation could replace a stack of automated checks, use the conversation.

For candidates and students, I would keep a record of real work: drafts, decisions, and the reasoning behind them. The point is not to prove you avoided AI. It is to show a human reader that you understood what you were doing. Where a tool rejects you automatically, ask for the alternative process Local Law 144 requires in New York City, and ask whether a human reviewed the result.

Is There a Better Way to Credential in an AI World?

I do not think we can go back to unaided essays and unobserved take-home tests, and I do not think a perfect detector is coming. The more promising direction, in my view, is to move evidence away from artifacts and toward observed performance.

Oral exams, live problem-solving, supervised practicums, apprenticeships, and work trials all put a real person in front of another real person. They cost more and scale poorly. They also test what we care about, which is whether this person can do the work with the tools that now exist. If AI is part of the job, using it well is part of the skill, and an assessment that bans the tool measures a world that has gone.

Two other habits seem worth protecting. Portable, signed records that people hold themselves keep any single registry from becoming a toll booth. And a certificate sold by the company that sells the tool tells you that company's definition of competence, which may or may not be yours.

I wrote about why working with AI is becoming a baseline expectation in an essay on professional literacy. If that is right, a credential for "using AI" will age quickly and the durable thing will be the judgment underneath it.

Where Does This Leave Us?

Credentialing was never about paper. It was about who gets to say what counts, and AI has reopened that question just as the old answers weaken. The danger is that a handful of systems quietly fill the vacuum while we get used to them.

I do not know whether the next generation of gatekeepers will be more open than the last. My guess is that it depends on whether the people being judged, and the people doing the judging, insist on seeing the gate.

Last updated: 2026-11-04

Frequently Asked Questions

How does AI create new credentialing gatekeepers?

AI makes polished work products cheap to produce, so the artifacts that used to prove ability carry less information. Institutions respond by trusting third parties that claim to separate real from synthetic work: screening software, AI-text detectors and proctoring tools, vendor AI certificates, verification registries, and independent bias auditors. Each of these now influences who is seen, passed, or trusted.

What does NYC Local Law 144 require of employers?

Employers using automated employment decision tools on candidates or employees in New York City must have the tool independently bias-audited within one year before use, post a summary of the audit, and give notice at least ten business days before use. Penalties run up to $500 for a first violation and $500 to $1,500 for each subsequent violation, counted per day. Enforcement by the city's consumer and worker protection department began on 5 July 2023.

Are AI hiring and education tools high-risk under the EU AI Act?

Yes, by classification. Annex III of Regulation (EU) 2024/1689 lists recruitment and selection tools (point 4) and education tools, including those that monitor and detect prohibited student behavior during tests (point 3(d)), as high-risk. The date those obligations bite was subject to the Commission's November 2025 Digital Omnibus proposal for delay, so confirm the adopted text in the Official Journal before relying on any date.

What is the status of Mobley v. Workday?

In July 2024 the Northern District of California (No. 3:23-cv-00770) denied in part Workday's motion to dismiss and allowed an agent-liability theory to proceed, and in May 2025 it conditionally certified an ADEA collective action. Neither ruling found that the software discriminated. Check the current docket for later developments.

Do AI text detectors treat all writers fairly?

Not reliably. A 2023 study in the journal Patterns (volume 4, issue 7) found that several GPT detectors misclassified more than half of TOEFL essays by non-native English speakers as AI-generated. The tools have changed since, but the result is a reason to avoid using a detector score as the sole basis for a misconduct finding.