How to roll out AI at work: 100 tests, a handful of wins

AI is the main focus of conversation these days, and a big decision for companies. So I got the AI leaders of Wonder Women who are actually deploying it into one meeting. One tested roughly 100 ways to use it and kept a handful. The other is rolling it out with the FDA looking over her shoulder. Here's what happened at the table.

Why I started AI Leaders

People in the Wonder Women network wanted to learn more about AI, and they fell into two camps: beginners learning how to use it, and women building products and implementing AI inside companies, where it's hard, and some of it isn't working.

AI Leaders is a monthly gathering focused on learning from each other on challenges, roadblocks, what's working, what's not, and how to get it done.

Test 100 things, keep the handful that work

Jennifer Vianello, former CMO of Cars.com, took the job shortly before ChatGPT launched. By early 2024, people across the company were playing with AI. Her worry was that the good ideas would stay "in some secret pocket of a manager somewhere in the organization."

So she gave her team an assignment. Write down every idea. One sentence each. Which terrible task does everybody hate? What have we never been able to do, because we could never hire enough people to do it?

They started with maybe 20 or 30 ideas. Every one went through the same quarterly process: find three to five vendors, try them, send the use case to legal, and write down whether it worked. If it failed, back to vendor search. If it worked, it went into next year's budget. Jennifer's phrase is "test, learn, fail or scale." The whole thing lived in a PowerPoint deck.

It started cheap, too. Free pilots, or about a hundred bucks to let five people try a tool for a month.

Her rough math at our meeting: 20 projects times five vendors is about 100 tests. Maybe three to five were really scalable. She said it reminded her of the early days of mobile apps, when many failed or became features inside other apps.

Test, learn, fail or scale.Jennifer Vianello, former CMO, Cars.com

What worked, and what backfired

• The chore. Cars.com runs a regular social media series comparing a car then and now, with side-by-side photos. It took hours of a designer's time for every post. It was the very first AI use they identified, and it scaled.
• The impossible job. Every used car is one of a kind, so video for each individual car could never be done by hand. A European provider made it possible, with brand rules built in. Jennifer's point: that one didn't eliminate jobs. It created something that could never have existed before.
• The rewrite. Before she paid for a single writing tool, she looked at the scope of work and how the team was built. Then came the tools. In 2024, nothing was good enough. Later versions were.
• The money. About half the savings went back to the company. The rest funded the tools and more senior people. "I didn't really need another graphic designer," she told us. "What I needed was another creative director."

My favorite is the failure. Before you shoot a TV ad, you test it with animatics, deliberately clunky illustrated storyboards with a rough voiceover. Her team used AI to make them faster and cheaper. The AI version looked much better than a normal animatic. "It was so good that consumers thought that it was a terrible final product," she said. They had to delay production and redo all the research the old way.

When the FDA is looking over your shoulder

Delphine Imbert is Chief Technical Officer of Pliant Therapeutics, a small biotech in the San Francisco area. She rolled out AI only a few months ago, so she doesn't have Jennifer's years of trial and error. She has a harder problem.

Try it first and write the rules later doesn't work in biotech. The lawyers came in up front. The FDA is an early adopter of AI itself, so Delphine wants her team using the same tools. But the FDA has also dinged companies for leaning on AI too heavily.

You'll hear two phrases constantly in her world: "human in the loop" and "human in the lead." So the work goes into two bins. Work the FDA audits is high risk, and AI on those activities needs approval from a governance committee. Everything else has more room. And because her company has intellectual property that isn't filed yet, she needed AI that stays contained and doesn't train a bigger system.

Then she did the opposite of Jennifer. Instead of proving use cases first, she ran a small beta, got the policy written, and gave AI to all employees.

My favorite part was her homework. She told her team to go talk to colleagues at other companies, sharing nothing confidential, and ask how they use AI for the exact work they do. Go ask your friends. That's networking.

How she gets people to try it

A consultant comes in every Tuesday morning and sits down with anyone who wants help. Super users volunteered to run AI labs, like how to make a slide deck or get a daily digest of your inbox. And some people are tech phobic, so she treats it like dinner with a small child.

I want you to try the food and you can say you don't like it. That's okay. But I want you to try.Delphine Imbert, Chief Technical Officer, Pliant Therapeutics

Next year, everyone on her team will have a small AI goal, no exceptions. She's honest about what she hasn't solved yet: how to measure the gain, and where the line is. If AI does half the review, does the human still count as in the loop? She's been asking for tactical best practices for months.

What the rest of the room said

The open discussion was where it got good. A few things I heard, without names:

• Marketing and tech are colliding. Marketers now show up with working prototypes. One woman's job is keeping that from becoming Lord of the Flies. Another described the tech team's attitude as "don't put your chocolate in my peanut butter." You need translators who speak both languages.
• Copy what someone else built. A friend of mine is chief of staff at a fintech that gave everyone AI and ran a leaderboard of the skills people built. Colleagues could grab one with a click. The winner? Expense reports and employee reviews. People just don't want to do the busy work.
• There are three levels. Make existing processes better. Redesign the process completely. Change how people behave. One leader pointed out that the first happens bottom up, but the other two need leadership pushing from the top.
• Fix the process first. A leader in a high-risk industry said you can get a lot of benefit from leaning out a process and using regular automation before you add AI. Another said to skip mapping the old process and work backwards from the outcome you want.
• The slop is real. More emails. More decks. One woman's fix: you can't send a 13-page deck when you used to send five.

How to roll out AI at work

1. Write the list. One sentence per idea. Start with the task everybody hates and the thing you could never afford to do.
2. Set the rules early. If you're regulated, they come first. If you're not, don't let them stop people from trying things.
3. Test cheap, on a schedule. Try several vendors for each idea, and write down what happened every quarter.
4. Make people taste it. Start with something they already do. Train them, and give them someone to sit down with.
5. Share what works. Delphine is pushing for this now, because how finance uses AI might give another team an idea.
6. Pick a few. You won't do everything on the list.
7. Measure it. Jennifer's team tracked hours saved and put a fully loaded cost on them. When the goal was growth, they tied it directly to revenue.
8. Ask other women who are doing it. You won't get this from a vendor's demo.

That last one is why we meet every month. If you're the one deploying AI where you work, put your name in.

Questions women ask me

How do you roll out AI in a company?

Start with a list of real problems, test tools cheaply, and record what works. Set your governance early, especially in a regulated industry. Then train people, give them support, and measure the results in hours saved or revenue.

How many AI pilots actually work?

Expect most to fail. Jennifer Vianello's rough count at Cars.com was about 100 tests and three to five that were really scalable. That's one marketing team's experience, but it's a good way to set expectations.

How do you get employees to use AI?

Have them try it on something they already do. Offer training and one-on-one help, let super users teach, and share what's working. Delphine Imbert is also adding a small AI goal for everyone on her team next year.

What does human in the loop mean?

A person stays responsible for the work AI touches. In regulated fields like biotech, that matters a great deal, and teams are still working out how to document it.

How do you measure the value of AI?

It depends on the process. Track hours saved and multiply by fully loaded cost for efficiency goals. For growth goals, tie it directly to revenue.

Dr. Yao Huang founded Wonder Women more than twenty years ago with a table of four friends in New York. Today it's 20,000 women and 5,000 dinners in 30+ cities. She is also the founder of The Hatchery and a three-time TEDx speaker.