Should AI decisions be explainable by law?

Algorithms now decide who gets a loan, who gets bail, who gets hired, and what medical treatment you receive. Most of these systems are black boxes — even their creators can't explain specific decisions. Should the law require that any AI making decisions about people be fully transparent and auditable?