There was a senate hearing a few days ago where this woman testified. Also Benn Jordan and Chad Marlow from the EFF were htere. I was happy hearing them put into words the problems on the national stage.
Interestingly, conservative Republican sheriff Ross Teeple also spoke against the idea of networked/data-storing ALPRs and Flock in particular at that hearing.
Flock did not put the woman in jail. The police and DA put her in Jail. This is not a failure of the technology (do they have confidence levels?). This is a failure of how police are outsourcing their reasoning and decision making to an AI camera service that seems to fail at the tail.
But Flock gives police overconfidence to commit actions instead of having due diligence. It is like the best student in your classroom tells you 2 + 2 = 5, and you naturally feel compelled to believe him, even there is clear evidence against the claim.
This is also comparable to self-driving car: Waymo's system has achieve the level of "less error than human on average" long time ago, but they are still only doing slow expansion. Why? Because when AI is involved, human naturally expect them to be almost 100% correct, not just barely better than human, otherwise why bother use AI (and all the troubles to legalize for AI) in the first place? So in this case, I guess the police just blindly trust the AI being 99.99% correct.
> It is like the best student in your classroom tells you 2 + 2 = 5, and you naturally feel compelled to believe him, even there is clear evidence against the claim.
Yes, but it is still your fault. “I am dumb and I was deceived” is no valid defense, least of all for a police officer.
It is always both. A product that can be abused to this extent is a bad product.
I have never released a single product that has seen people die through common user misuse.
The standards for solutions shouldn't get _lower_ as the stakes get _higher_.
If a private company can puts hundreds of tons of payload into orbit without ever causing someone to accidentally be imprisoned a company that sells cameras on a stick shouldn't either.
Indeed. It’s insane to see these arguments implying the contrary. The institution committing the violence is the one responsible for the outcome.
If a random person walks into a police station and accuses a random stranger of murder, and the police put that stranger in jail based on that alone, the main problem is not the accuser.
But in this case, it wasn't a random person, it was a police-vetted system. Of course the authorities are to blame, but Flock hold a lot of responsibility, too.
Had the 'evidence' been a random dashcam rather than Flock cam, this wouldn't even be a story beyond severe police incompetence.
Flock didn't fail to inspect the car for damage and not find any, flock didn't take 13 days to conduct a cursory review of evidence, flock didn't fail to request cell tower triangulation data and rule out her being in the location of the accident...
I get that people feel their privacy's invaded by being filmed, but this story doesn't appear to have any more to do with Flock than what the officers had for breakfast or the brand of air freshener she had in her car at the time.
Except of course the entire reason the police picked her in specific is because of information flock gave them.
Could they have, you know, double checked it? Yes, obviously. But they didn't.
I agree that this story is more about the incredible incompetence of the police involved than flock attempting to record everyone and everything, but flock giving the police an excuse to stop looking and arrest an innocent is by no means irrelevant.
> Regardless, the true takeaway of Isaacs’ experience is that ALPR technology, whether it’s from Flock or any of the other competitors in this space providing the same functionality (Axon, etc), is dangerous not only because it is so easily abused by police (or even by hackers) to invade the privacy and liberty of citizens, but also because it enables a lazy police department to outsource basic critical thinking to machines and then take action based on a single data point to ruin a person’s life.
The accusation that some machine made any decision here goes beyond the reporting in the article. It seems concocted to raise AI alarm. All we have is that it supplied accurate data that police then abused. That isn't in the vicinity of "outsource[d] basic critical thinking to machines".
I want such general government surveillance banned, but the machines in this story had no more volition than a breathalyzer.
The machines matter, because we know how human behavior works. We know that if you give humans an automated tool that works most of the time, they get lazy, to the point where they make more mistakes than if they had no automation at all.
Flock isn’t an automated tool though; you don’t give it a crime, and it gives back an automated package saying “this is who the criminal is”.
As far as I know, it lets law enforcement query for things against the data. Can be specific (like a license plate) or more general (like a dark colored mid-sized SUV around <location> at <time>).
To get where this story is at, cops here must have done the latter, and taken the first name off the top of the pile and patted themselves on the back, and then waited 7 months, and went and arrested her. Unless there’s an explanation for a 7 month delay, it seems designed to allow defensive evidence to spoil.
There isn’t an uncanny-valley of automation here. There is just lazy, grossly-incompetent, bad-faith, and malicious cops/investigators.
Volition has nothing to do with it. A breathalyser doesn't point to a person and say this is the likely drunk, so that comparison is fatuous. These identification systems do point the finger.
the problem here appears to be that police officers are generally of average or below average intelligence. combined with the fact that they face no consequences for incompetence, you have the perfect mixture for nonsense like this happening.
I expect the solution they will settle on is to form a department for interacting with technology like this, who understand its outputs are not edicts.
you can't really expect any meaningful police reforms (such as liability and insurance) because anyone who is in any position to enact them is perfectly fine with dumb muscle that holds a monopoly on violence and is deferential to them. in fact the public disliking the police is an ideal outcome for those decision makers as it gives them more leverage over the police.
I had no idea that cameras were walking the streets and arresting people in America. Only the police should be able to do that. They are clearly far more competent. Oh wait-I just read the article...
> Police didn't care that Lindsey Isaacs' car was the wrong color, and wasn't damaged. They still arrested her because of Flock data.
This is literally the first sentence of the article.
You can dislike public surveillance, but statements like "Flock camera data put this woman in jail for 13 days" blatantly ignores the the fact that Flock didn't put anyone in jail, a bunch of cops who very clearly knew that it wasn't her.
> The Flock camera, meanwhile, had recorded Isaacs’ car and its license plate several miles away from the site of the accident, sometime around the time of the incident.
Blaming a tool, when the cops in question are clearly willing to grasp to whatever piece of "evidence" makes their job "easier" is a giant cop-out and excuse for the people who are abusing it.
Any piece of evidence requires some form of critical examination; that is what law enforcement's job is. If you're just going to go with the first theory/piece-of-evidence they find.... they're not doing their job (and violating peoples constitutional rights through sheer laziness).
That’s all true. But we’ve known police and prosecutors are like this forever. No need to give them tools to be even lazier and that make it easier for them to abuse us.
Thats what always been portrayed in films and stories for so long.Police never solved cases. There were detectives involved specialist who solves crime. While only job of police is to arrest any suspicious person, take them to station, then to court, then to jail. Essentially they are just drives carrying criminals. The very first loose evidence and suspect, that it arrest them question them, torture, if officer is lucky, that person is the criminal or one who just accepts blame. All their effort is solving as fast as they can to maintain record and stars on their shoulder
Right. It's not even clear to me what the alternative critical position is here (for gp)? At the end of the day, the company is not putting you in jail, the cops who do are simply their customers.
"Flock is bad" metonymizes the situation, but doesn't let anyone off the hook.
Multiple things can be bad at the same time and cops have a demonstrated record of failures such that add a system like flock is most likely going to lead to more injustice, not less.
Absolutely not all cops and prosecutors, or even the majority of them, are this lazy. Every tool is a tool that lets them "be event lazier" and make it "easier for them to abuse us". Removing any tool that let bad-actors be lazy is not an alternative for removing the bad-actor themselves.
This situation the equivalent of an eye-witness saying "Ya, I think it might have been this person I know, Tom!" and a cop/prosecutor taking that at face value and arresting Tom (and the system that allowed that to happen).
The people who make decisions and take action so (grossly) negligently need to be held accountable and face consequences. Blaming the tool (whatever tool it is) is giving the people who have the actual responsibility an escape hatch to avoid accountability.
As the child of a public defender who has heard some STORIES: they absolutely are not doing the work. Not because of some inherent trait, but because they are all (prosecutors and defenders) assigned a workload that forces shortcuts and allows negligence with no chance of consequence.
All public defenders and most prosecutors are happy to admit anonymously that the justice system is 90% gears grinding people. They ALL know an innocent person that pled guilty to end the grinding. Judges too. You can find plenty of court videos on YouTube of people pleading guilty while saying they didn’t do it but need to get on with their life. The judge will explain to them that they can’t accept a guilty plea from someone who claims innocence, and the defendant then folds. Everyone in the room knows that there is more to the story, but no one is willing to do the work.
You'd think with tech being a force multiplier and all it'd be real easy for agencies to shed the supposed few and far between 'bad apples'. They should be able to do more with less bodies, with flock and other tech helping them, right? It seems to me the entire system failed this woman and waiving it away as a single lazy cop gives the system a pass.
There is no real mechanism for holding them accountable and the consequences of these tools include death (it only takes one perceived wrong move for officers thinking they've bagged a felon to ventilate you. Hell not even that, see that officer who mag dumped his own car after he mistook an acorn for a gunshot.) This lady is quite lucky that it was only 13 days, if she had a career that's probably hosed but it might not be totally life ruining if she has a strong support system.
I think there's a combination of selection bias and survivorship bias with bad cops.
When it is made pretty clear that as a cop you won't face consequences for your malicious and/or incompetent actions, you're going to select for people who find that very appealing. Similarly, the people who don't like that kind of system might leave, allowing for a reinforcement the malicious and/or incompetent behavior.
But whats the difference on the ground here? Are we meant to take the transitive property here if you will and say there is actually nothing wrong with the Flock company itself and what it enables?
There will always be cops willing to misuse whatever they have access to.
Part of it is holding cops accountable in these situations. Part of it is limiting or stopping the use of these systems so cops don't have as many opportunities to abuse them.
From the same article : “what else would you call the account of a woman who was arrested by incompetent police based on nothing more than a single Flock camera piece of data, and held in jail for 13 days”
When the headline of the article is explicitly blaming Flock, and not the cop/DA/judge who never even looked at the evidence available, it is 100%, whether on purpose or not, letting/helping the cops point the finger at another party.
The headline is what 99% of people read, and remember.
I prefer to discuss the actual material. If you want discuss what you believe other people believe about just a headline, let’s do it somewhere other than a thread discussing content specifically from the article.
The point is that if the flock surveillance data hadn't existed in the first place, we wouldn't be in this situation. Everybody knows police will abuse this kind of surveillance technology, regardless of how accurate it is.
> critical examination; that is what law enforcement's job is
Critical examination is the judiciary's job, not the police's. Jordan v. City of New London established that police departments can and do deny employment to prospective officers who possess excessive cognitive ability.
https://www.youtube.com/watch?v=klE498xFjOY
This is also comparable to self-driving car: Waymo's system has achieve the level of "less error than human on average" long time ago, but they are still only doing slow expansion. Why? Because when AI is involved, human naturally expect them to be almost 100% correct, not just barely better than human, otherwise why bother use AI (and all the troubles to legalize for AI) in the first place? So in this case, I guess the police just blindly trust the AI being 99.99% correct.
Yes, but it is still your fault. “I am dumb and I was deceived” is no valid defense, least of all for a police officer.
I have never released a single product that has seen people die through common user misuse.
The standards for solutions shouldn't get _lower_ as the stakes get _higher_.
If a private company can puts hundreds of tons of payload into orbit without ever causing someone to accidentally be imprisoned a company that sells cameras on a stick shouldn't either.
If a random person walks into a police station and accuses a random stranger of murder, and the police put that stranger in jail based on that alone, the main problem is not the accuser.
Whistle-blower found dead 1 month after exposing "AI" firms systematic copyright violation.
Looks like a bigger problem than just constitutional rights erosion for citizens. =3
Flock didn't fail to inspect the car for damage and not find any, flock didn't take 13 days to conduct a cursory review of evidence, flock didn't fail to request cell tower triangulation data and rule out her being in the location of the accident...
I get that people feel their privacy's invaded by being filmed, but this story doesn't appear to have any more to do with Flock than what the officers had for breakfast or the brand of air freshener she had in her car at the time.
Could they have, you know, double checked it? Yes, obviously. But they didn't.
I agree that this story is more about the incredible incompetence of the police involved than flock attempting to record everyone and everything, but flock giving the police an excuse to stop looking and arrest an innocent is by no means irrelevant.
- Does anything connect this vehicle to the accident?
- Did it look damaged; as though it had been in a crash?
- Did it look like it was fleeing or driving erratically?
- Was there anything suspicious about the vehicle or the driver?
Police would have deemed the photo irrelevant and moved on. But that didn't happen in this case.
Their failure to do so doesn't have to do with source of the image (Flock, shop CCTV, dashcam), but with their inexplicable lack of rigour.
Can you give an account of the lack of rigor, or is it some kind of holy mystery?
The accusation that some machine made any decision here goes beyond the reporting in the article. It seems concocted to raise AI alarm. All we have is that it supplied accurate data that police then abused. That isn't in the vicinity of "outsource[d] basic critical thinking to machines".
I want such general government surveillance banned, but the machines in this story had no more volition than a breathalyzer.
As far as I know, it lets law enforcement query for things against the data. Can be specific (like a license plate) or more general (like a dark colored mid-sized SUV around <location> at <time>).
To get where this story is at, cops here must have done the latter, and taken the first name off the top of the pile and patted themselves on the back, and then waited 7 months, and went and arrested her. Unless there’s an explanation for a 7 month delay, it seems designed to allow defensive evidence to spoil.
There isn’t an uncanny-valley of automation here. There is just lazy, grossly-incompetent, bad-faith, and malicious cops/investigators.
Guns can be used for all sorts of things. A handgun in a house is likely to be used for murder/suicide.
I expect the solution they will settle on is to form a department for interacting with technology like this, who understand its outputs are not edicts.
you can't really expect any meaningful police reforms (such as liability and insurance) because anyone who is in any position to enact them is perfectly fine with dumb muscle that holds a monopoly on violence and is deferential to them. in fact the public disliking the police is an ideal outcome for those decision makers as it gives them more leverage over the police.
https://www.youtube.com/watch?v=CkUJbJCFjN8
The problem is the police-camera system, of which the camera is an instrumental component
This is literally the first sentence of the article.
You can dislike public surveillance, but statements like "Flock camera data put this woman in jail for 13 days" blatantly ignores the the fact that Flock didn't put anyone in jail, a bunch of cops who very clearly knew that it wasn't her.
> The Flock camera, meanwhile, had recorded Isaacs’ car and its license plate several miles away from the site of the accident, sometime around the time of the incident.
Blaming a tool, when the cops in question are clearly willing to grasp to whatever piece of "evidence" makes their job "easier" is a giant cop-out and excuse for the people who are abusing it.
Any piece of evidence requires some form of critical examination; that is what law enforcement's job is. If you're just going to go with the first theory/piece-of-evidence they find.... they're not doing their job (and violating peoples constitutional rights through sheer laziness).
"Flock is bad" metonymizes the situation, but doesn't let anyone off the hook.
This situation the equivalent of an eye-witness saying "Ya, I think it might have been this person I know, Tom!" and a cop/prosecutor taking that at face value and arresting Tom (and the system that allowed that to happen).
The people who make decisions and take action so (grossly) negligently need to be held accountable and face consequences. Blaming the tool (whatever tool it is) is giving the people who have the actual responsibility an escape hatch to avoid accountability.
All public defenders and most prosecutors are happy to admit anonymously that the justice system is 90% gears grinding people. They ALL know an innocent person that pled guilty to end the grinding. Judges too. You can find plenty of court videos on YouTube of people pleading guilty while saying they didn’t do it but need to get on with their life. The judge will explain to them that they can’t accept a guilty plea from someone who claims innocence, and the defendant then folds. Everyone in the room knows that there is more to the story, but no one is willing to do the work.
There is no real mechanism for holding them accountable and the consequences of these tools include death (it only takes one perceived wrong move for officers thinking they've bagged a felon to ventilate you. Hell not even that, see that officer who mag dumped his own car after he mistook an acorn for a gunshot.) This lady is quite lucky that it was only 13 days, if she had a career that's probably hosed but it might not be totally life ruining if she has a strong support system.
When it is made pretty clear that as a cop you won't face consequences for your malicious and/or incompetent actions, you're going to select for people who find that very appealing. Similarly, the people who don't like that kind of system might leave, allowing for a reinforcement the malicious and/or incompetent behavior.
If punishing bad actors alone was enough we wouldn't have a ton of 4th amendment case law.
Corrupt, arrogant, bigoted, or with a superiority complex? Way more than half.
There will always be cops willing to misuse whatever they have access to.
Part of it is holding cops accountable in these situations. Part of it is limiting or stopping the use of these systems so cops don't have as many opportunities to abuse them.
No one is letting cops off the hook.
The headline is what 99% of people read, and remember.
Critical examination is the judiciary's job, not the police's. Jordan v. City of New London established that police departments can and do deny employment to prospective officers who possess excessive cognitive ability.