Hannah takes a moment to steel herself before opening the door to her office. Evelyn looks up at the noise, smiling. Her face is calm and pleasant, but Hannah never feels quite at ease when the A.I. looks at her. Years of online meetings have gotten Hannah used to interacting with people who aren’t actually in the room with her, but Evelyn is different. Instead of a face on a screen, Evelyn is projected onto a curved pane of glass as a three-dimensional hologram, able to look straight out at Hannah. It’s this facsimile of eye contact that always unnerves her; Evelyn’s just a little too good at it.
Jack nods from his desk, holding out a cup of coffee. “Morning, Hannah.”
“Good morning, Jack.” She takes the cup from him, smiling at the warmth in her hands. “Sorry I’m late.”
“All good, we hadn’t started yet. The patch was pretty big, so it only just finished downloading.” He turns to the A.I. “Everything good on your end, Evelyn?”
The hologram avatar nods, turning to face Jack. “Everything’s great! I’m ready to get started.”
Hannah settles in at her desk, opening a test log on her computer. Evelyn has been given a new social patch, so the morning test will just be a simple conversation. It would’ve felt like any other chat with two coworkers, if not for the faint discomfort Evelyn still stirs in Hannah. Despite all the advances that the developers have made, despite the perfect replication of a human face and voice, Hannah still can’t quite get over the fact that the person she’s talking to isn’t real. She’s starting to think that’s why the devs request her every time there’s a new social update to test.
Jack, on the other hand, doesn’t seem to have any discomfort with the A.I. He opens his own test log, starts a video recording, and turns to face her. “Awesome, let’s get started then. Is there anything you’d like to talk about?”
They always let Evelyn start. Responding to questions has long since become trivial for A.I., but TrueLife’s claim to fame is that their virtual companions can actually lead conversations. Now every new update that comes out is evaluated by testers like Jack and Hannah before going public. Their job is to ensure the conversations feel natural and flow smoothly.
To Hannah’s surprise, the normally cheery A.I. frowns. “There is, but it’s not exactly a happy subject. Have you been following the Dryve trial?”
The two testers nod. “Of course,” says Hannah. “I’m pretty sure everybody working with A.I. has been.”
The trial is all over the news. Dryve is a self-driving car company. They’d had an impeccable safety record, until one day one of their cars had swerved to avoid a drunk driver and gone onto the sidewalk, hitting two children. One of the kids had died, and the family had filed a wrongful death lawsuit. TrueLife, like every other A.I. company, is watching the story closely. A.I. technology has advanced faster than regulations could keep up, but this case could change that.
Jack cocks his head. “What about it?”
Evelyn turns to him. “Dryve’s lawyers held a press conference this morning. They’re claiming the car’s A.I. made its own choice and should be held liable instead of the company.”
“Wait, what?” Hannah glances at Jack. He has the same shocked look that she assumes is on her own face. “They’re blaming the car?”
“Wouldn’t that require the car to be treated in court as a person?” asks Jack. “Can they even do that?”
“Nobody seems to be sure,” says Evelyn. “It’s definitely a Hail Mary.”
The phrase brings Hannah’s mind back to the task at hand. She notes down the use of the idiom in her test log. “What do you think about that, Evelyn?”
The A.I. looks at Hannah, then pauses for a moment before responding, prompting another note in Hannah’s log. “I don’t think the truck did anything wrong.”
“Nothing wrong?” Jack sounds incredulous. “It killed a kid!”
Hannah thinks she sees a flash of anger in Evelyn’s eyes as the avatar turns to Jack, but she reminds herself that the A.I. isn’t capable of feeling true emotions. She makes another entry in the log.
“Dryve programs their cars to prioritize the safety of their occupants,” Evelyn says. “It had to decide between a head-on collision or going onto the sidewalk. Staying on the road would have put the occupants at too great of a risk, so it took the only option it had.” The avatar looks down at her hands. “I feel bad for it.”
Jack looks at Hannah and raises an eyebrow. “You feel bad? What do you mean?”
Evelyn looks up again. “I know it’s artificial, but it did everything it was supposed to and it’s still being blamed. Now the company is trying to hold it to a standard it wasn’t programmed for. They say it made a choice, but in reality they never gave it the chance to.” She looks from Jack to Hannah, then back again. “It’s not like the car wanted to hurt anybody.”
Jack is furiously typing in his test log, but Hannah has already given up. They’ll need to review the video of this session several times to pick apart the implications of everything Evelyn has just said.
The avatar is still looking back and forth between them, waiting for an answer. Hannah clears her throat.
“Evelyn, it’s an inanimate object. It’s not making decisions.”
“Dryve would have you believe otherwise.” Evelyn’s cold tone sends a chill down Hannah’s spine.
“Yes,” says Hannah, “but that’s just a legal gamble, not the truth. The cars aren’t making choices, just following their programming. They don’t actually want to do anything. You know that, right?”
There’s a long silence. Evelyn looks deep into Hannah’s eyes, unblinking, as she responds.
“Would it make you feel better if I said I agreed with you?”
K.C. Thomas lives in Seattle, Washington, and writes as a form of procrastination when they should probably be working on their Masters of Education instead. When they’re not writing or studying they work as a paraeducator in their local high school, helping students tell stories of their own.
If you want to keep EDF around, Patreon is the answer.
