Evolv might be new to the game but it’s already made a name for itself. And not a good one.
It was an integral part of New York City Mayor Eric Adams’ ongoing run of public failures. The mayor announced Evolv would be placing its “gun detection” tech in the city’s subways, despite the public admission of Evolv CEO Peter George (during a call with investors) that the tech wouldn’t work all that well in subways.
“Subways, in particular, are not a place that we think is a good use case for us,” George said, due to the “interference with the railways.”
He probably meant interference from the railways, but the end result of Evolv’s trial run could probably be described as “interference with the railways” just as accurately.
A pilot program testing AI-powered weapons scanners inside some New York City subway stations this summer did not detect any passengers with firearms — but falsely alerted more than 100 times, according to newly released police data.
Through nearly 3,000 searches, the scanners turned up more than 118 false positives as well as 12 knives, police said, though they declined to say whether the positive hits referred to illegal blades or tools, such as pocket knives, that are allowed in the transit system.
On one hand, CEO Peter George definitely didn’t oversell the tech’s effectiveness when he expressed his reluctance to deploy it in city subways. On the other hand, it would appear Evolv’s sales force has overstated the tech’s effectiveness so often, the Federal Trade Commission has been forced to step in. Here’s more from Matthew Guariglia and Cooper Quintin of the EFF:
The Federal Trade Commission has entered a settlement with self-styled “weapon detection” company Evolv, to resolve the FTC’s claim that the company “knowingly” and repeatedly” engaged in “unlawful” acts of misleading claims about their technology. Essentially, Evolv’s technology, which is in schools, subways, and stadiums, does far less than they’ve been claiming.
The FTC alleged in their complaint that despite the lofty claims made by Evolv, the technology is fundamentally no different from a metal detector: “The company has insisted publicly and repeatedly that Express is a ‘weapons detection’ system and not a ‘metal detector.’ This representation is solely a marketing distinction, in that the only things that Express scanners detect are metallic and its alarms can be set off by metallic objects that are not weapons.”
Evolv is selling metal detectors with some unproven AI stapled to them. Because there’s AI involved, the company has no qualms about selling its metal detectors for up to five times the going rate of regular, non-AI-tainted metal detectors. If customers balk at the markup, that’s where the salespeople step in to, apparently, overstate the accuracy of Evolv’s tech and its presumed effectiveness in reducing violent crime by detecting weapons.
Here’s what the settlement [PDF] prevents Evolv from making representations about in its marketing materials, advertising, or anything connected with pitching its products to potential customers:
A. the ability to detect weapons;
B. the ability to ignore harmless personal items;
C. the ability to detect weapons while ignoring harmless personal items;
D. the ability to ignore harmless personal items without requiring visitors to remove any such items from pockets or bags;
E. weapons detection accuracy, including in comparison to the use of metal detectors;
F. false alarm rates, including comparisons to the use of metal detectors;
G. the speed at which visitors can be screened, as compared to the use of metal detectors;
H. labor costs, including comparisons to the use of metal detectors;
I. testing, or the results of any testing; or
J. any material aspect of its performance, efficacy, nature, or central characteristics, including, but not limited to, the use of algorithms, artificial intelligence, or other automated systems or tools
Wow. That’s pretty much everything the company could possibly say about its products, which pretty much means it can only provide customers with makes/model numbers and the price of those offerings. There’s no “pitch” left to be made, which means the company is certainly already working towards having this settlement heavily altered or rescinded on appeal.
It also instructs the company to inform all of its educational facility customers that they can cancel their contracts immediately and pay only what’s owed through the point the contract is cancelled.
The only upside for Evolv is that this settlement only applies to its Evolv Express product and only to its marketing to customers in the educational field. It’s still open season elsewhere, but this settlement contains admissions by the company that it misled these particular customers, which should make other potential customers in other areas (hospitals, subways, etc.) far more wary of trusting Evolv’s effectiveness assertions.
