AI Interview Red Flags: A Candidate’s Go/No-Go Audit

By InterviewOS Lab· · 9 min read
AI Interview Red Flags: A Candidate’s Go/No-Go Audit
Original AI-generated editorial image created for this guide.

AI interview red flags are signs that an automated hiring step is opaque, weakly connected to the job, inaccessible, or collecting more personal information than necessary. Before continuing, identify what the tool does, what it evaluates, who makes the decision, how data is handled, and whether you can request an alternative. Continue when the process is job-related and explained; pause when answers are vague, pressure is high, or no human or accommodation route is available.

Key takeaways

  • Separate scheduling and information-collection tools from systems that score, classify, rank, or screen candidates.
  • Ask what qualifications or characteristics are assessed, whether a human reviews the result, how long data is retained, and how to request an accommodation or alternative.
  • Treat vague measures such as “enthusiasm,” “fit,” voice, facial expression, or posture as questions requiring evidence, not as proof of fairness or job relevance.
  • For covered New York City hiring processes, look for a recent public bias-audit summary and required information about the tool’s use.
  • Document unanswered questions and use a simple continue, pause, escalate, or withdraw decision instead of accepting an opaque process automatically.

An automated employment decision tool is software that substantially assists or replaces human judgment in an employment decision by generating a prediction, classification, or recommendation about a candidate. That is different from a tool that only schedules an interview, stores an application, or collects answers for later human review. The distinction matters because the more influence software has over screening or selection, the more important notice, job relevance, oversight, accessibility, and data handling become.

Start by classifying what the system actually does

Do not rely on labels such as “AI-powered interview,” “smart assessment,” or “bias-free.” Ask what happens after you submit an answer. A chatbot may simply collect availability or basic qualifications. A video platform may record responses for a recruiter. An assessment may score work samples. Another system may rank candidates or reject them before a person reviews the application. These are materially different processes.

The practical first question is: “Does this system merely support administration and information collection, or does it substantially influence who advances?” New York City guidance explicitly treats screening as part of employment decisions and discusses tools according to their role in generating predictions or classifications. You can read the guidance from the New York City Department of Consumer and Worker Protection.

Look for clues in the invitation, privacy notice, assessment instructions, and employer careers page. Words such as “ranked,” “scored,” “recommended,” “matched,” “shortlisted,” or “automatically reviewed” suggest that the system may influence progression. They do not prove exactly how it works, so treat them as prompts for specific questions rather than conclusions.

You can send a short message before starting:

Could you clarify whether this assessment is used only to collect information or whether an automated system scores, ranks, or screens candidates? If it contributes to a decision, which parts of my response are evaluated, and is there human review before an application is rejected?

A refusal to explain every technical detail is not automatically a red flag. Employers may not disclose proprietary implementation details. But they should be able to explain the assessment’s purpose, the general factors considered, the role of human reviewers, and how to request help or an alternative.

Run a four-question transparency audit

A useful audit focuses on four answers: what is assessed, who decides, how long information is retained, and what alternative or accommodation path exists. These questions are simple enough to ask in one email and specific enough to expose important gaps.

1. What qualifications or characteristics are assessed?

The employer should be able to connect the assessment to the work. For a writing role, that might mean evaluating a written response against stated requirements. For a coding role, it might involve a work sample or technical problem. You should be cautious when the process relies on broad, unexplained concepts such as “executive presence,” “energy,” “culture fit,” or “enthusiasm.” Ask what observable evidence supports the score and why that evidence is relevant to the role.

The U.S. Equal Employment Opportunity Commission testimony on automated systems describes risks involving rigid credential rules, employment-gap filters, voice analysis, facial or posture analysis, and systems that may reject applicants without human review. Those examples do not establish that every employer uses such methods, but they show why the evaluation mechanism matters. Read the EEOC testimony on AI and automated systems before treating a polished interface as evidence of a sound process.

2. Does a human review the result?

Ask whether a recruiter or hiring manager reviews the complete application and response, or whether the automated result can determine rejection by itself. “A human is involved somewhere” is not the same as meaningful review. A person who only sees a final score may have little opportunity to correct a misleading transcription, missing context, or unsuitable criterion.

A useful follow-up is: “At what stage can a human override or investigate the automated result?” Also ask whether reviewers see the underlying response, the score, or both. If the employer cannot identify who owns the decision, treat that uncertainty as significant because you cannot tell where to direct a correction or complaint.

3. How long is interview data retained, and who receives it?

An automated interview may involve recordings, transcripts, written answers, identity information, assessment results, or technical metadata. Before submitting sensitive information, find the applicable privacy notice and ask how long the data is kept, who can access it, whether a vendor processes it, and whether it is used for purposes beyond this vacancy.

A vague statement that data is retained “as needed” is less useful than a defined retention period or a clear explanation of the decision process. If the employer cannot answer, record that fact. You do not need to accuse the organization of wrongdoing to decide that the information provided is insufficient for you to continue.

4. How do you request an accommodation or alternative?

The route should be usable before the deadline, not hidden after a rejection. Ask for a named contact or functional inbox, the types of alternative assessment available, and whether requesting help will affect your candidacy. Explain only what is necessary to request support; you do not need to disclose unrelated medical details to a chatbot.

A practical message is:

I would like to understand the available accommodation or alternative assessment route before I begin. Who should I contact, what information is required, and can the same job-related competencies be evaluated without relying on the automated feature?

A clear process, responsive contact, and reasonable explanation are positive signals. A broken form, inaccessible interface, last-minute deadline, or instruction to disclose sensitive information to an automated system deserves a pause and possibly escalation.

Check job relevance, fairness, and accessibility

Job relevance is the central test. A tool should assess capabilities the role actually requires, using a method that gives candidates a fair opportunity to demonstrate them. That does not mean every candidate must receive the same format in every circumstance. It means the employer should be able to explain why the format and criteria relate to the work.

For example, a timed coding exercise may be relevant when the role requires coding, but you can still ask what exactly is scored: correctness, approach, documentation, or speed. A recorded video response may test communication for a presentation-heavy role, but that does not explain why facial movement, background, eye direction, or vocal style should determine suitability. If the employer cannot distinguish the job skill from a behavioral proxy, ask for a work sample or structured interview instead. It also helps to read our guide to preparing for an AI-scored video interview, which covers mapping competencies, checking the scoring rules and testing your setup.

Accessibility questions should be concrete. Can the platform be used with assistive technology? Is there a non-video option? Can you receive questions in writing? Can technical problems be reported without losing the opportunity? Is extra time available where appropriate? You are looking for a real path to demonstrate the same relevant skills, not merely a generic statement that the employer values inclusion. On a related note, our guide to AI interview skills assessments shows how to find out what is being measured and which tools are allowed.

Privacy is also part of fairness. A camera or microphone requirement may reveal information unrelated to your ability to do the job. Ask whether audio, video, or identity checks are essential, what alternatives exist, and whether the employer can evaluate the same competency without collecting unnecessary material. Avoid sending identity documents, financial information, or unrelated personal records unless you understand why they are required and who receives them.

For practical preparation around the meeting technology itself, use this video interview checklist for Zoom, Teams, and Meet. It can help you separate ordinary connection problems from concerns about how the employer evaluates or stores your responses.

Use location-specific checks without overgeneralizing

Rules and disclosure requirements vary by location, employer, and role. Do not assume that a notice designed for one jurisdiction applies everywhere. However, local guidance can give you a strong checklist even when it is not directly controlling your situation.

For candidates in New York City and covered hiring processes, the city guidance describes public notice and bias-audit requirements for covered automated employment decision tools. The audit summary should identify the assessed characteristics and provide selection or scoring information, and the guidance says the audit must be current within one year. Search the employer’s careers, privacy, and equal-opportunity pages for the tool name, audit summary, distribution date, and any instructions for candidates.

Do not treat the existence of an audit as proof that the process is suitable for you. An audit may answer particular questions about selection outcomes while leaving other issues unresolved, such as accessibility, data retention, the quality of human review, or whether the assessed characteristic is genuinely related to the job. Use it as evidence to examine, not as a substitute for judgment.

If you cannot find the disclosure, ask directly where it is published. Save the page, email, or invitation and note the date. Documentation is useful because automated hiring processes can involve several vendors and contacts, and a later recruiter may not know what the original invitation promised. A related read: our guide to AI recruiter phone screens, which explains how they work and how to give answers that transcribe well.

Make a documented go/no-go decision

After your audit, choose among four actions: continue, continue after clarification, pause and escalate, or withdraw. The decision should reflect the seriousness of the unanswered question, not just your general feelings about AI.