A simple gray box model has been developed for residential buildings to predict the future HVAC energy use given the weather forecast. The model is based on thermostat data (temperature, setpoint, and HVAC on/off information) and the outside weather conditions (outdoor temperature, solar irradiation and wind speed). The accuracy of the future HVAC energy use for over 1000 houses across the United States predicted by the models was compared with the actual data. Predicted energy use from the model is within +/- 10% of the actual HVAC used. The temperature prediction error was within +/- 0.3C.