Saturday, December 19, 2015

The Battle over the Purpose and Accuracy of Zillow

In a series of articles in the Washington Post in 2014, David Howell, CIO at McEnearney Associates, a Northern Virginia Realty firm, and Stan Humphries, Chief Economist at Zillow and architect of the Zestimate, slugged out their views on the accuracy and purpose of Zillow Zestimates.  To read their original articles just click on their respective names above.  I have not checked out the accuracy of Howell's claims about the accuracy, or, rather, inaccuracy of Zestimates for the northern Virginia market, but I would venture to say that his numbers are not inconsistent with the observations that I have seen elsewhere.  Humphries, on the other hand, while suggesting that the numbers put forward by Howell are slightly inaccurate because he has not performed the analysis on a large national basis, points out that the Zestimates over time are getting more accurate, which may, in fact, be true based upon the way that both he and Howell measure their results.  However, I believe that in the arguments put forward by Humphries there are some very troubling and subtle slights of hand being proffered:

First, all the studies of accuracy are based upon the Zestimate at the time of sale vs. actual sale price of a given home that has sold in the marketplace during the period being measured for accuracy.  This sounds like a simple enough measurement.  However,  I have shown in an earlier writing, a phenomena that I have called the Zillow Bump.  For those of you unfamiliar with this phenomena I suggest that you read the entry.  In that analysis I show a very curious change in Zestimates that occurs once a property gets listed.  Namely, if there is a significant difference between the Zestimate and the listing price, then the Zestimate will start on a very rapid journey to get much closer to the listing price. I have speculated as to the reason for this Zillow Bump and the reason why I feel this is a disingenuous attempt by Zillow to "improve" Zestimate results over time.  In essence, there is a small window of time for Zillow to more closely align the Zestimate to the sale price, so that at the time of sale (ie., when the data is collected) the Zestimate is within a closer range of the actual sale price.  That window is between the time that a home goes up for sale and the time it actually sells. 

Before I get into this point any deeper I want to cover another issue raised by Humphries -- that the original listing price vs. actual sale price is no more accurate than the Zestimate. The implication here is that the two numbers (ie., original listing price and Zestimate) are to be considered identical prognostications of the actual sale price. Nothing could be further from the truth.  A listing price by an owner in consultation with a broker is usually intended to be close to the intended actual sale price, but usually with a bias to the higher side of the actual sale price. This is not always the case, particularly in very hot markets where bidding wars are expected.  I do not have accurate statistics, but I would venture to say that in normal markets the actual sale price of a home is somewhere between 5% to 10% below the listing price (except, as I noted, in very hot markets).  The listing price is never intended to be the market value of a home.  It is intended to be approximately 5% to 10% above the actual sale price.  So, to do an apples to apples comparison between listing price and Zestimate the upside bias of the listing price should be removed.  Once this bias is incorporated into an analysis it will be evident that this "adjusted listing price" is a much better estimate of actual selling price than the Zestimate is.


Now, getting back to the first point, we can now more clearly see what the Zillow strategy is -- to "cook the books" on the analysis of Zestimate vs. actual sale price by altering the Zestimate and hopefully bring the Zestimate to about 8% below the listing price before the house sells and then claim that they are within a tight range.  This is a shell game.  It is an implicit acknowledgement by Zillow that the broker/owner listing price is a much better basis for actual sale price (adjusted to account for the broker bias) than the original Zestimate and that the original Zestimate before the house was listed is literally worthless.


The final point to be made would be that Zillow is talking out of both sides of their mouth when they claim, on the one hand, that the Zestimate is not an appraisal price, but, on the other hand, that because it is as accurate as the listing price, it can be used as a measure of value.  This is a very dangerous argument.  Howell has aptly pointed out that the homebuyer is greatly influenced by the internet and because of the blurring of what a Zestimate really is and how inaccurate it is, the prospective  buyer is unfairly influenced by it.  This means that the seller is unfairly influenced by it too.  

Clearly, Zillow is free to compute whatever it wants to, but at the same time, owners should be able to opt out of this charade of the Zestimate accuracy.  In my piece Zestimates from Zillow are Worthless I argue that the Zestimate is as important as a person's FICO score and that the abusive implementation of Zestimates by Zillow should be regulated with legislation similar to the Fair Credit Reporting Act.
Bill Gasset, a ReMax realtor in Hopkinton, Massachusetts, wrote a very interesting piece shortly after the Washington Post articles appeared on the Zestimate inaccuracy issue.  In Bill's article he very aptly sums up many of the problems with the Zillow Zestimate, and actually had some positive things to say about the Zillow platform.
In 2015, The Los Angeles Times also ran an article on the inaccuracy of the Zillow Zestimates.  They reported on an interview on the TV show "CBS This Morning" where co-host Norah O'Donnell asked Zillow's Chief Executive, Spencer Rascoff, about the inaccuracy of their company's Zestimates. Rascoff's position is that the Zestimate is a "starting point" for determining the value of your home. In my humble opinion, it is a "starting point" as good as one you might hit when throwing a dart while blindfolded.  It is a marketing gimmick to attract attention to the Zillow website, no more, no less.

The proper numbers to measure are pre listing Zestimate vs. Actual Sales price.  You may want to factor in some sort of small deviation from this difference to take into account changes in the areas during the listing time.  For example, if immediately before the time that a home was listed it had a Zestimate of, let's Say X, and it took six months for the home to sell, then it would make sense to alter that number X by the amount that homes in the immediate area (say zip code, or something like this) have changed during that period.  For the case in point, let's say that it is six months and that the zip code has increased in value by 1.3%, then it would be fair to argue that the proper Zestimate value to use for comparison to the sale price is 1.013X, not X.