\n<\/p>\n
A US teenager was handcuffed by armed police after an artificial intelligence (AI) system mistakenly said he was carrying a gun – when really he was holding a packet of crisps. <\/p>\n
“Police showed up, like eight cop cars, and then they all came out with guns pointed at me talking about getting on the ground,” 16-year-old Baltimore pupil Taki Allen told local outlet WMAR-2 News<\/a>. <\/p>\n Baltimore County Police Department said their officers “responded appropriately and proportionally based on the information provided at the time”.<\/p>\n It said the AI alert was sent to human reviewers who found no threat – but the principal missed this and contacted the school’s safety team, who ultimately called the police.<\/p>\n<\/div>\n But the incident has prompted calls by some for the schools’ procedures around the use of such technology to be reviewed.<\/p>\n Mr Allen told local news he had finished a bag of Doritos after football practice, and put the empty packet in his pocket.<\/p>\n He said 20 minutes later, armed police arrived. <\/p>\n “He told me to get on my knees, arrested me and put me in cuffs,” he said.<\/p>\n Baltimore County Police Department told BBC News Mr Allen was handcuffed but not arrested.<\/p>\n “The incident was safely resolved after it was determined there was no threat,” they said in a statement.<\/p>\n Mr Allen said he now waits inside after football practice, as he does not think it is “safe enough to go outside, especially eating a bag of chips or drinking something”.<\/p>\n<\/div>\n In a letter to parents, school principal Kate Smith said the school’s safety team “quickly reviewed and cancelled the initial alert after confirming there was no weapon”.<\/p>\n “I contacted our school resource officer (SRO) and reported the matter to him, and he contacted the local precinct for additional support,” she said.<\/p>\n “Police officers responded to the school, searched the individual and quickly confirmed that they were not in possession of any weapons.”<\/p>\n However, local politicians have called for further investigation into the incident.<\/p>\n “I am calling on Baltimore County Public Schools to review procedures around its AI-powered weapon detection system,” Baltimore County local councilman Izzy Pakota wrote on Facebook<\/a>.<\/p>\n Omnilert, the provider of the AI tool, told BBC News: “We regret this incident occurred and wish to convey our concern to the student and the wider community affected by the events that followed.”<\/p>\n It said its system initially detected what appeared to be a firearm and an image of it was subsequently verified by its review team.<\/p>\n This, Omnilert said, was then passed to the Baltimore County Public Schools (BCPS) safety team along with further information “within seconds” for their assessment.<\/p>\n The security firm said its involvement with the incident ended once it was marked as resolved in its system – adding it had “operated as designed” on the whole.<\/p>\n “While the object was later determined not to be a firearm, the process functioned as intended: to prioritise safety and awareness through rapid human verification,” it said.<\/p>\n Omnilert says it is a “leading provider” of AI gun detection – citing a number of US schools among its case studies on its website<\/a>.<\/p>\n “Real-world gun detection is messy,” it states.<\/p>\n But Mr Allen said: “I don’t think no chip bag should be mistaken for a gun at all.”<\/p>\n The adequacy of AI to accurately identify weapons has been subject to scrutiny.<\/p>\n Last year, a US weapons scanning company Evolv Technology was banned from making unsupported claims about its products<\/a> after saying its AI scanner, used in thousands of US schools, hospitals and stadiums entrances, could detect all weapons.<\/p>\n BBC News investigations showed these claims to be false<\/a>.<\/p>\n<\/div>\n