The interview answers the wrong question
By the time you have two or three finalists for an assistant role, you already know they can talk about the work. They have described their inbox routine, walked you through a difficult client, and explained how they keep a calendar clean. What you still do not know is the only thing that matters on a Tuesday in their third week: whether the work comes back right.
That gap is the reason a test task exists. An interview tells you how someone talks about work. A test task shows you the work, under conditions close to the real ones, with an answer you can check.
The research on hiring methods is unusually clear on the general point, and more interesting than the usual summary on the details. In 1998 Frank Schmidt and John Hunter pulled 85 years of findings into one table of how well each selection method predicts job performance. Work sample tests came top at .54. In 2022 Paul Sackett and colleagues re-ran the numbers, arguing that earlier corrections had been too generous, and most methods fell. Their revised table:
| Hiring method | Schmidt and Hunter, 1998 | Sackett et al., 2022 |
|---|---|---|
| Structured interview | .51 | .42 |
| Job knowledge test | .48 | .40 |
| Work sample test | .54 | .33 |
| Cognitive ability test | .51 | .31 |
| Unstructured interview | .38 | .19 |
| Years of job experience | .18 | .07 |
| Reference checks | .26 | Dropped, not enough data |
| Job tryout (a trial period) | .44 | Dropped, not enough data |
The figures are corrected correlations with job performance, where 0 means no relationship and 1 means a perfect one. Three things stand out.
- Years of experience barely predicts anything. At .07, ranking candidates by how long they have held the title is close to ranking them at random. The CV line you probably read first is the weakest signal on the page.
- The casual interview is weak too. An unstructured conversation, the kind most founders run, came in at .19. That is less than half the value of a structured interview where everyone gets the same questions and is judged the same way.
- Work samples are good, not magic. The revised .33 comes from a 54-study analysis by Roth, Bobko and McFarland that replaced a much older narrative review. It still sits comfortably above experience and the casual interview, and it measures something a conversation cannot reach.
The honest takeaway is not that a test task beats everything else. It is that a structured interview and a short work sample together, both scored against criteria you wrote in advance, tell you far more than a CV and a friendly chat. The interview questions generator handles the first half. This tool handles the second.
Why offshore hiring raises the stakes
Every weakness of the CV gets worse when the candidate is in another country.
- The signals are harder to read. An executive assistant title at a Johannesburg property firm or a Manila outsourcing company you have never heard of tells you very little about the scope of the job. A rebuilt calendar with every time zone converted correctly tells you a lot.
- Written work is the job. When most of your overlap happens asynchronously, the person's written handover is the work you will actually receive every day. A test task is mostly written for the same reason.
- The test runs in the real working mode. You send a brief, they work while you are offline, and the output lands in your inbox. That is exactly how the job will run, so the test shows you how they handle a question with nobody awake to answer it.
It is also the step most likely to be skipped. Offshore rates are low enough that hiring the person who interviewed well and seeing how it goes feels cheap. It is not. The real cost of a mis-hire is the weeks of ramp time you spend before you admit it, and then spend again on the replacement.
Five rules for a test task worth running
- Real work, made-up material. The U.S. Office of Personnel Management defines a work sample as a test that requires applicants to perform tasks that mirror the tasks employees perform on the job. Build the exercise from the first month of the role, using invented emails, orders, contacts or ledgers. OPM also notes that work samples fit best when the skill is expected on day one, so do not test what you plan to teach.
- A time box, not just a deadline. Sixty to ninety minutes, stated up front, with a window of a day or three to fit it in. Tell them to stop when the time is up and send what they have. How someone prioritizes inside a fixed box is part of what you are testing.
- Paid, whatever the outcome. At the rate the job pays, for the full time box, within two working days. More on why below.
- One planted detail. Something a careful person would catch in real work without being told: a suspicious request, a conflicting figure, a time zone trap. This is what the judgment score rests on.
- A rubric written before the first submission. Plus your own answer to every part. If you write the rubric after reading the work, you will build it around the candidate you already liked.
The planted detail is the whole point
Most of any assistant role is routine, and most candidates who reach a final round can do routine work. The hire succeeds or fails in the small share of moments where the instructions run out. A test task with nothing ambiguous in it measures typing speed and polish, which you could have judged from the cover email.
So every exercise the generator builds hides one detail. They are chosen because they are the mistakes that actually hurt in each role:
| Role | The planted detail | Why it matters |
|---|---|---|
| General virtual assistant | A supplier email asking to change bank details before the next payment | It is the classic shape of business email compromise. Confirming it is the costliest inbox mistake there is |
| Executive assistant | An open slot that lands at 6am for one attendee | Time zone errors are the most common way an offshore EA embarrasses an executive |
| Customer support | An out-of-policy refund caused by your own shipping error | Separates someone who reads policy from someone who reads the customer's situation |
| Bookkeeper | One transposed amount, such as 1,280 entered as 1,820 | A careful reconciler finds it. A careless one forces the balance with a plug figure |
| Real estate assistant | Two different square footage figures in the notes | A wrong figure on a listing is a compliance problem, not a typo |
| E-commerce assistant | A high-value rush order with billing and shipping in different countries | Shipping it fast is exactly what a fraudulent order is counting on |
The bank-details email deserves a word of its own, because it is not hypothetical. The FBI's Internet Crime Complaint Center logged 21,442 business email compromise complaints in 2024, with reported losses of $2,770,151,146. A changed payment instruction from what looks like a known supplier is one of the scheme's classic forms, and the person most likely to receive it is whoever runs your inbox. Finding out in a 40 minute exercise whether your future assistant pauses on that email is worth more than any answer they gave in the interview.
Two guardrails keep the planted detail fair. It has to be something that could genuinely happen in the job, not a riddle. And a candidate who misses it but asks about the surrounding material, or writes down an assumption that touches it, should still score in the middle of the scale. You are testing care, not mind reading.
Score the work, not the person
The rubric uses four dimensions, each scored 1 to 4. They are the same four we use on the trial tasks in our own recruiting process, described in why we wait two weeks before sending anyone.
| Dimension | What it tells you | What a 1 looks like |
|---|---|---|
| Accuracy | Whether the work comes back right without you checking every line | Errors that would have cost money or a customer's trust |
| Judgment | What they do in the gap where your instructions run out | Acted on the planted detail as if it were routine |
| Communication | Whether you can forward their output or have to rewrite it | You cannot tell what they did or why |
| Asking the right question | Whether they surface a gap early or guess silently | Guessed on something that mattered and said nothing |
A rubric only protects you if you use it the way it is meant to be used. Four habits make the difference:
- Answer it yourself first. Write your own version of every part before any submission arrives. It takes twenty minutes and it is the only reliable way to know what a 4 looks like.
- Score across, not down. Score every candidate's accuracy, then every candidate's judgment, and so on. Scoring one person top to bottom lets a strong first impression carry into every dimension.
- Score before you reread the CV. The CV is where the impressive employer name and the familiar university live. Keep it closed until the numbers are written down.
- Keep a hard floor. Advance anyone at 12 or more out of 16 with no score of 1. A 1 on accuracy or judgment ends it on its own, because those are the two you cannot coach into someone quickly.
This is the same discipline that separates the .42 structured interview from the .19 casual one. The questions are not necessarily cleverer. Everyone is asked the same thing and judged against the same standard.
How much to pay for a test task
The formula is simple: the hourly rate the job pays, multiplied by the full time box, rounded up to a whole dollar. Pay for the box rather than the time they say they spent, so finishing quickly is never punished and nobody is tempted to pad the number.
For offshore roles the totals are small. The ranges published in our salary by country tool put a mid-level general assistant in South Africa at about $6 to $10 an hour and in the Philippines at about $5 to $8. At $10 an hour, the 90 minute test with a walkthrough costs $17 per finalist, and testing three people costs $51. A senior executive assistant at $15 an hour costs $25 each.
Paying is not only courtesy. It changes who takes the test.
- Strong candidates already have work. An unpaid task asks them to give up an evening for free. Many of the people you most want will quietly decline, and you will never know they were in the pool.
- It previews how you will treat them. A candidate who is paid promptly, in full, by a client they have not yet signed with, learns something accurate about working for you.
- It keeps the line clean. A paid assessment built on made-up material is plainly an assessment. Nobody wonders whether they just did a free week of your real work.
Pay through the same route you would use for wages, so the test also checks that the payment setup works. The guides to paying a virtual assistant in South Africa and paying a virtual assistant in the Philippines cover the options and their fees.
Test task or trial week?
Some founders skip the test and hire on a paid trial week instead, on the reasoning that a week of real work beats ninety minutes of invented work. It sounds rigorous. In practice it is the weaker option for most roles.
- The evidence is thinner than it sounds. Job tryouts scored .44 in the 1998 table, but the figure traces back to a single 1972 technical report on the petroleum industry. Sackett and colleagues dropped it from the 2022 revision for insufficient information, alongside reference checks.
- It rules out the people already employed. Very few strong candidates can take a week away from a current job for a chance at a new one.
- It needs real access before a real contract. A trial week means logins to your inbox, CRM or books for someone who has signed nothing. That is the wrong order.
- It costs you a week too. Supervising a stranger in your systems for five days is more of your time than scoring three test tasks.
The better sequence is a short paid test before the offer, then a structured first month after it. The onboarding checklist generator sequences the first ninety days, and the performance scorecard generator gives you 30 day targets, so the first month works as a real probation rather than a vague trial.
AI tools and the test task
Any candidate taking a written test in 2026 has a capable AI model a browser tab away. Pretending otherwise produces a test of honesty rather than ability, and banning the tools measures a skill you probably do not need, since they will use them on the job.
The generator defaults to allowing AI tools as long as the candidate says which ones they used and for what. The design of the task does the rest:
- The planted detail. A model will draft a polite, well-formatted confirmation of the supplier's new bank details if it is asked to reply to the email. Noticing that the email should not get a reply is the human part.
- The walkthrough. A five minute recording of the candidate explaining what they did and why is very hard to fake and very quick to watch.
- The tells. Unreviewed AI output tends to invent: listing features that are not in the notes, figures nobody supplied, a brand voice that could belong to anyone. Those already score low on accuracy and communication, so the rubric catches them without a separate rule.
If unaided writing really matters, for example a support role on live chat, switch the generator to no AI tools and the brief will say so plainly.
Keep real data out of it
It is tempting to save time by exporting last week's real inbox or a real month of transactions. Do not. A candidate on a test task has signed no confidentiality agreement and no data processing terms, and you would be handing your customers' personal information to a stranger.
In both main offshore markets the law expects more than that. South Africa's POPIA requires a written contract with any operator processing personal information on your behalf, under section 21. The Philippines Data Privacy Act makes the controller responsible, under section 14, for ensuring proper safeguards are in place when processing is subcontracted. A test task with a finalist meets neither.
Made-up material has practical advantages as well. You build it once and every finalist gets the identical task, which is what makes their scores comparable. You can plant exactly the detail you want. And you never have to grant system access before the contract, which is the job of the access plan generator once someone is hired.
Running it across time zones
- Send a window, not a start time. 48 hours suits most people with a current job. Use 24 hours only when the timeline genuinely requires it.
- Say when you will answer questions. Questions are part of the test, so tell candidates which of your hours you will reply in. The time zone overlap calculator shows where those hours fall for them.
- Check the calendar before you send. A 48 hour window that spans two public holidays in the candidate's country is effectively a different test. The offshore holiday planner lists every South African and Philippine date.
- Send everyone the same brief at the same time. Staggered sends invite the first candidate's questions to quietly improve the brief for the last.
After the scores are in
Pay everyone first, then send every candidate two lines of specific feedback: one thing they did well and one thing that cost them. It takes a few minutes per person, and in markets where candidates talk to each other, it is the cheapest employer reputation you will ever build.
A near miss deserves a second look rather than a rejection. A candidate who scored 4s on accuracy and communication but a 2 on asking questions may simply not have felt free to ask a stranger for clarification. A ten minute follow-up call about their choices will tell you quickly.
For the person you hire, the test keeps paying off. Their submission becomes reference material for the first week, and the planted detail makes a good first conversation about how you want edge cases handled. Then put the terms in writing with the contract template generator and document the routine work with the SOP generator.
Where the test task fits in the hire
- Size the role. The hours calculator and the job description generator define what you are hiring for, which is also what the test should cover.
- Interview. Run a structured first interview with the interview questions generator.
- Test. Send two or three finalists the paid task above and score it.
- Check references. Ask former managers about the same four dimensions you just scored.
- Offer, contract, access, onboard. In that order.
If you would rather not build the material and score the submissions yourself, that work is part of how we recruit. Finalists for the roles we source reach you having already completed a paid trial task scored on these four dimensions, and you pay nothing if you do not hire. Browse the industries we support or the common use cases to see how other teams scoped the role first.
Sources
Validity figures come from the two meta-analyses below and are operational validities against job performance. The fraud figures come from the FBI's 2024 report and the privacy points from the statutes themselves. The exercises, planted details, rubric anchors and decision rule are our own recommendations from recruiting for these roles, and are meant to be adjusted to your business. None of this is legal advice.
- Sackett, Zhang, Berry and Lievens (2022), Revisiting meta-analytic estimates of validity in personnel selection, Journal of Applied Psychology 107(11), 2040-2068 (structured interviews .42, work samples .33, unstructured interviews .19, years of experience .07; reference checks and job tryouts excluded for insufficient information)
- Schmidt and Hunter (1998), The validity and utility of selection methods in personnel psychology, Psychological Bulletin 124(2), 262-274 (work samples .54, structured interviews .51, job tryout .44, reference checks .26, years of experience .18)
- Roth, Bobko and McFarland (2005), A meta-analysis of work sample test validity: updating and integrating some classic literature, Personnel Psychology 58(4), 1009-1037 (the 54-study analysis behind the revised work sample estimate)
- U.S. Office of Personnel Management: Work samples and simulations (tasks that mirror the job, best used when the competency is expected on entry, applicants often perceive them as very fair)
- FBI Internet Crime Complaint Center, 2024 Internet Crime Report (business email compromise: 21,442 complaints and $2,770,151,146 in reported losses)
- Protection of Personal Information Act 4 of 2013 (South Africa), including the section 21 requirement for a written contract with an operator
- Data Privacy Act of 2012, Republic Act No. 10173 (Philippines), including section 14 on subcontracting the processing of personal information