Organolead halide perovskites are a new class of photovoltaic materials for potential use in thin film solar cells, but suffer from rapid degradation under light and humidity, limiting their viability. Recently developed quasi-2D Ruddlesden-Popper phase perovskites show improved stability but, at present, the mechanism behind the stability is poorly understood. Here, we have used time-of-flight secondary ion mass spectrometry (ToF-SIMS), photoluminescence spectroscopy (PL), electrochemistry, and other techniques to study the chemistry and performance of 2D and 3D phases of methylammonium lead triiodide (MAPI) solar cells as a function of humidity exposure. ToF-SIMS depth profiles of both 2D and 3D MAPI planar solar devices (n-i-p, C60/perovskite/PEDOT:PSS) show the formation of a significant hydrolysis-based degradation layer at the surface of only 3D MAPI devices after humidity exposure. Isotopic D2O studies confirm the layer is caused by ambient humidity, while electrochemistry confirms it is directly related to loss of cell performance. The growth of this hydrolysis layer is found to be inhibited in 2D MAPI devices by the formation of a thin protective layer composed of more thermodynamically-stable 2D phases, clearly observed in PL spectra, which protects the less stable 2D bulk. To confirm that this newly formed 2D layer improves stability, we created solar cells consisting of 3D MAPI films coated with a thin layer of the 2D perovskite, and show that these cells are more stable that un-treated 3D MAPI devices. As a result, we conclude that using 2D perovskite as a protective interface may be a simple way to improve device stability.