Key takeaways
- Nearly half of REALTORS already use AI daily (23%) or weekly (25%)[1], so a pilot is about structure, not introduction.
- Only 22% of US employees say their organization has a clear AI plan[2]; a written pilot plan fills that gap.
- Choose one problem, record a baseline before day one, and agree success measures in advance.
- Build guardrails for fair housing, disclosure and calling consent before any client sees AI output.
- End with an explicit decision: scale, adjust or stop.
Your agents are already experimenting
Most brokerages are not deciding whether agents will use AI. They are deciding whether that use will be organized. NAR's 2026 technology survey found 23% of REALTORS use AI daily and 25% weekly, another 31% experiment occasionally, and only 12% say they are not using it and have no plans to[1]. Among those who use it, the most common task is writing listing descriptions, cited by 75%[1].
- Daily 23%
- Weekly 25%
- Experimenting occasionally 31%
- Curious, not tried 9%
- Not using, no plans 12%
Source: National Association of REALTORS®, 2026[1]
Show the data
| Item | Value |
|---|---|
| Daily | 23% |
| Weekly | 25% |
| Experimenting occasionally | 31% |
| Curious, not tried | 9% |
| Not using, no plans | 12% |
Agents report results, too: 55% of REALTORS say AI has had a positive effect on their business[1]. The risk is that each agent invents their own rules. Gallup found 44% of US employees say their organization has begun integrating AI, but only 22% say it has communicated a clear plan[2]. A short, well-run pilot gives your brokerage that plan, grounded in what worked in your own office rather than in a vendor's case study.
Pick one problem worth solving
A pilot that tries to test AI in general proves nothing. Pick one workflow where the pain is visible and measurable, where mistakes are easy to catch before they reach a client, and where you can compare before and after within a month.
- Good candidates: drafting first versions of listing descriptions for agent review, summarizing inspection reports for the agent, drafting follow-up emails after open houses, answering inbound calls outside office hours with message capture.
- Poor candidates: anything that gives pricing advice to consumers, writes contract language, or sends messages to clients without an agent seeing them first.
- Test of fit: can you name the person who reviews every AI output during the pilot? If not, the scope is too wide.
If the problem is phone coverage, for example, a pilot of Ava answering evening and weekend calls for a handful of listings is narrow enough to measure and easy to switch off. If the problem is content, keep the pilot to drafts that an agent edits and approves.
Measure before you start
The most common pilot failure is discovering on day 30 that nobody knows what things looked like on day zero. Spend the week before launch recording a baseline for each measure, and write the target next to it so the decision at the end is not a matter of mood.
| Measure | Baseline method | Target set in advance | Source of truth |
|---|---|---|---|
| Time per task | Agents log minutes for five instances of the task | A specific reduction in minutes | Agent time log |
| Quality | Broker or team lead scores five recent human outputs on a 1–5 rubric | AI-assisted output scores at least as high | Reviewer scorecard |
| Error rate | Count corrections needed on human drafts | No increase in factual corrections | Review log |
| Adoption | Not applicable | Most pilot agents still using it in week four | Usage report |
| Client impact | Response time or booked appointments for the workflow | Improvement or no change | CRM report |
Keep the rubric simple. For listing descriptions, for instance, score accuracy against the listing sheet, compliance with fair housing language, and whether the agent would publish it with minor edits.
Give three people clear roles
Pilots stall when everyone is involved and nobody is responsible. Name three roles before week zero, even in a small office where one person wears two hats.
- Sponsor: usually the broker or managing broker. Approves the scope, signs off on guardrails and makes the final decision in week four.
- Pilot lead: an operations manager, team lead or tech-minded agent who configures the tool, runs the weekly survey, keeps the scorecard and raises problems the same day.
- Reviewers: one or two experienced agents who read AI output before it reaches clients and score it against the rubric. They need protected time, so reduce their other admin during the month.
Write the roles, the scope paragraph and the guardrails on a single page and share it with the pilot group. That page becomes the first draft of your brokerage AI policy if the pilot succeeds, and agents who join later can read it in five minutes.
The 30-day plan
Week 0: set up
Choose four to six agents, write the scope in one paragraph, configure the tool, record baselines and brief the group on guardrails and how to report problems.
Week 1: shadow mode
The AI produces output alongside the normal process, but nothing reaches clients. Reviewers compare AI drafts with what agents actually did.
Week 2: limited live use
Agents use AI output for real work after review. Each output is logged with a quick quality score and any correction made.
Week 3: full scope
The workflow runs as planned for every pilot agent. Collect the weekly feedback survey and fix configuration issues quickly.
Week 4: review and decide
Compare results with the baseline, read a sample of outputs together, and make the scale, adjust or stop decision in a single meeting.
Guardrails before any client sees output
- Fair housing: listing copy and messages must describe the property, not the people the agent imagines living there. The Fair Housing Act bars discrimination based on race, color, national origin, religion, sex, familial status and disability[3], and AI tools can echo biased phrasing from their training material.
- Calling consent: in 2024 the FCC ruled that AI-generated voices are artificial under the TCPA[4]. AI-voiced marketing calls to consumers need prior express written consent, and other automated calls have their own consent rules, so keep voice pilots to inbound calls unless your records support more.
- Client data: decide which data the tool may see. Keep financial details, Social Security numbers and wiring information out of any tool that is not covered by your data agreements.
- Human review: during the pilot, nothing AI-generated goes to a client without an agent reading it first.
- Kill switch: name the person who can turn the tool off immediately if something goes wrong.
Get honest feedback from agents
Agents are polite in meetings and blunt in the break room. Use a short anonymous form every Friday instead of asking how it is going. The same five questions each week let you see the trend.
- How many times did you use the tool this week?
- On the last task, did it save you time, cost you time or make no difference?
- What did you have to fix in its output?
- Did anything it produced make you uncomfortable sending to a client?
- If the pilot ended today, would you ask to keep it?
Watch question four closely. A single uncomfortable output is a configuration problem to fix now, not a statistic to average away.
Be open with clients
Clients are noticing. Zillow found that among sellers who hired an agent, 21% knew and another 36% suspected their agent was using AI tools[5]. Public attitudes are mixed: Pew reports 50% of US adults are more concerned than excited about the growing use of AI in daily life[6]. Being upfront costs little. A line such as “We use AI tools to help draft some materials; every word is reviewed by your agent” answers the question before it becomes a worry.
Example: a follow-up email pilot
Example: a fictional 30-agent brokerage pilots AI-drafted follow-up emails after open houses with five agents. The baseline shows agents spend about 25 minutes per open house writing follow-ups, and many visitors get nothing for two days. During the pilot, drafts are ready the same evening, agents spend about 8 minutes editing each batch, and reviewers score quality at 4.1 out of 5 against a baseline of 3.8.
Example findings: two drafts in week two described a home as ideal for a particular kind of family, which the reviewer flagged as steering language. The team added a rule to the prompt and a fair housing checklist to the review step, and no further flags appeared. Four of five agents asked to keep the tool.
Decide: scale, adjust or stop
- Scale when targets are met, the error rate did not rise and most pilot agents want to keep it. Roll out to the next group with the same guardrails and training.
- Adjust when results are promising but one measure missed. Fix the cause and run two more weeks with the same group.
- Stop when time savings did not appear or quality fell. Write a short note on why, so the next pilot starts smarter.
Whatever you decide, publish a one-page summary for all agents. It is the clear plan most workplaces are missing. Before you scale, check the costs against your technology budget and the adoption tests in choosing tools agents will actually use.
Frequently asked questions
How long should an AI pilot at a brokerage run?
Thirty days is long enough to move from shadow mode to full use and short enough to keep attention. Extend by two weeks only if one specific measure needs more data.
How many agents should be in a pilot?
Four to six is usually enough to see patterns without losing control of review. Include a skeptic and a newer agent, not only enthusiasts.
Do we have to tell clients we use AI?
Check your state rules and your MLS policies. Many clients already suspect it[5], and a short, plain statement that an agent reviews all AI-assisted material builds trust.
Can an AI voice assistant call leads back automatically?
The FCC treats AI-generated voices as artificial under the TCPA[4], so AI-voiced marketing calls need prior express written consent. Keep pilots to inbound calls unless your consent records clearly allow it.
References
National Association of REALTORS® (2026). REALTORS® Adopt Technology to Save Time and Improve the Client Experience, NAR Report Finds (2026 REALTORS® Technology Report) (opens in a new tab) ↑
Gallup (2025). AI Use at Work Has Nearly Doubled in Two Years (opens in a new tab) ↑
U.S. Department of Housing and Urban Development (2026). Housing Discrimination Under the Fair Housing Act (opens in a new tab) ↑
Federal Communications Commission (2024). FCC Makes AI-Generated Voices in Robocalls Illegal (opens in a new tab) ↑
Zillow Research (2026). Sellers: Results from the Zillow Consumer Housing Trends Report 2026 (opens in a new tab) ↑
Pew Research Center (2025). How Americans View AI and Its Impact on Human Abilities, Society (opens in a new tab) ↑