Structure and randomness in planning and reinforcement learning