I see two potential key-loss situations, both in the 1800s. The second is the one you mentioned, the first is the succession from Joseph to Brigham if one believes that was incorrect.
If one considers either a valid loss of keys situation, it also sinks the entire concept of the Great Apostasy. If the keys can survive that, they can survive whatever corruption has flowed through Popes and any Orthodox patriarchs over the centuries. So to have faith in the LDS organization, one must not consider these significant problems on that scale.