The 2011 Dodd-Frank Act requires the Federal Reserve Board to conduct supervisory stress tests to assess the capital adequacy of the largest U.S. banking institutions and yet the accuracy of stress-test capital projections is undocumented. I compare pre-crisis stress test forecasts to the performance of 14 large US banks during the financial crisis. Each bank calibrates its stress test models using the bank’s own historical data while the regulator compares bank projections with estimates from a representative bank model. I consider three modelling methodologies: a multi-equation CLASS-style approach, a model based on Bayesian averaging, and a parsimonious Lasso model. Representative bank model forecasts differ dramatically from the forecasts from bank-specific models and from actual outcomes. The Lasso methodology is most accurate approach but there is no guarantee it will be in other samples, and it is impossible to identify the superiority of the Lasso model ex ante. The results highlight the policy uncertainty inherent in using stress tests, both to set minimum bank capital requirements and to assess the capital adequacy needed to maintain banking system stability.