Imperfect Humans, Imperfect Agents: How Can We Make Them Better Together