Wednesday, September 10, 2014

Zestimates from Zillow are Worthless

Zestimates are what  Zillow calls its market estimate of a real estate parcel's value.  And Zestimates aren't worth as much as the electrons it takes to get them sent to you over the Internet.
In other words -- they are worthless in most cases.  For those instances that they are accurate, you have no way to judge that to be the case without some independent method being applied to the property at hand, so it is worthless in all cases.

According to Zillow, "Nationally,...Zestimate has a median error rate of 8.3%."  To put that in more concrete terms, "half of the Zestimates in an area are closer than the error percentage and half are farther off."  But note that a Zestimate may be above or below by that 8.3% to qualify for being within 8.3%. Note also that the maximum error off the median on the closer side cannot be more than 8.3%, but the error on the farther side could be considerably greater.  In other words, the price differencial between the Zestimate and your true market value (appraisal) may be very significantly more than 8.3%.

In my case, I looked at the value of my house on Zillow and I think I have figured out what they think they are doing.  My house was put on the market for a rental and the real estate broker inadvertently listed the monthly rental price as the sale price. Two days later, realizing the mistake, the realtor removed the listing.  Now, you can well imagine that the monthly rental price is only an infinitesimal fraction of the selling price.  But, then the Zillow Zestimate took a nose dive in value.

I have reported the error to Zillow on a number of occasions and, try as I might, the folks at Zillow have never corrected the error, nor the Zestimate, nor have they tried to contact me, nor go to the public records to see if the house was ever listed at that price for sale.

Look at this Zestimate chart below.  The line on top is the Zestimate from mid 2008 through 2013 for my home.  You can see that it has fluctuated from a low of around $800k to a high of almost $1.4m.  What is most notable is the precipitous drop in 2012.  The coincident lines below are the Zestimate values for the town and Zip Code for this home.  Nothing dramatic there, and there was nothing dramatically different at this home other than the slip up on the part of the realtor who put the rental price in as a sale price.  How do you explain this kind of variation? By the way, the large increase at the end of 2009 was caused by a change in the property taxes on the property (due to an error by the assessor).  So, the comment that only a trained professional can appraise a house is very apt.  That appraisal can weed through the erroneous data and come to a reasonable number for the market value.  Obviously a Zestimate is fraught with problems in this regard.

Insert update.  This insert was placed here on 4/25/15
I have had so many communications with Zillow on this matter that I am not even counting anymore.
They have altered the erroneous data and the Zestimate has now changed.  Unfortunately, the system still appears to have some other kind of flaw.  Here is the same chart as above except with today's date and the history:

What I can't figure out is that the Zestimate, when corrected, showed just over $1.1m, and then, mysteriously, and precipitously it dropped to just under $1.0m.  I also find it curious that the Zestimate for the period of time before the original error was introduced into Zillow is different.
I have to conclude that they have little elves in their Zestimate factory making up these numbers.


Insert Update.  This insert was placed here on 5/3/15 (about one week after the previous insert)

I happened to take another look at the Zestimate chart.  It has changed again!

This is absurd.
Insert Update.  This insert was placed here on 5/3/15 (about 5 hours after the previous insert).
It seems that the Zestimates are changing more than once a day now.

Absurd isn't even the correct way to describe this!

I suppose, in a way, a Zestimate is sort of like a FICO score.  For those of you who are unfamiliar with what a FICO score is you can look it up, but it is the in reality the report card score of your financial health.  Every loan, morgage, car loan, credit card interest rate, is based upon your FICO score.

You could argue, no, it's not like the FICO score because the FICO score is based upon accurate experience information on the consumers' use of credit. The FICO score can have a significant impact upon a consumers' life, every financial or credit decision involving that consumer usually boils down to what his or her FICO score is. It's like that pesky SAT score that was on your forehead when you applied to college.

If your home is your largest financial investment and the Zestimate is the alleged "score" that you have achieved with that investment, it could materially affect the decisions that are made that involve your financial condition. Zillow is the first to point out in their terms and conditions that financial institutions cannot use the Zestimate as a basis for a loan. This makes me think of the porn sites that say that if you are under 18 years of age that you are not allowed to use the site. When was the last time you thought a teenager would heed such a notice?

 If the Zestimate is lower than it should be it could influence a potential buyer for your home should you want to sell it. If you are contemplating the purchase of a home it could impact your decision about whether to buy that home. Perhaps, you have applied for a business loan and as part of your Net Worth statement you list the equity in your home. No loan officer in the world is going to perform a CMA (Certified Market Analysis) on your home under those circumstances. It is too costly and takes too much time. A quick and easy Zestimate is available to check the number on that Net Worth statement.

Do we really believe that this computed value has any accuracy? This is easily measurable. Take a community and find every sale during a given period of time and compare the sale price to the Zestimate that was in effect at the time of the sale. Just how accurate can such a crude Zestimate actually be?

According to Carol Augustus, a Realtor with Pacific Union in Marin County California, Zillow itself cites the following figures as interpreted for Marin County:

1. 88% of Marin homes are in Zillow
2. 80% have Zestimates
3. 20% are within 5% of a recent 3 month sales price (80% are NOT)
4. 38% are within 10% of a recent sales price (68% are NOT)
5. 64% are within 20% of a recent sales price (36% are NOT)

I, quite frankly, don't know what to make of these numbers. Does this mean that a given home has a 20% chance that another home in the county sold within 5% of the price within the last 3 months? If so, what does that mean? It sounds almost meaningless.

Alternatively, if it means that 20% of the homes are within 5% of a hypothetical sales price, this would be dreaming of a dream. I don't have any other interpretations that could make sense of these cited statistics.

So, now the big question is, what if products like Zillow and their Zestimates become more pervasive and can meaningfully affect your daily life (either positively or negatively)?
Will you have the ability to redress your grievances if you feel you have been maligned? Will there be the equivalent of the Fair Credit Reporting Act to protect consumers from erroneous data being used negatively against them in these new financial marketplaces?

What other products are there out there now, or can reasonably be expected to appear on the web near you in the not too distant future? I'd like to hear from you. Please let me know your thoughts on this issue.

And shame on you Zillow.

(Note: I have subsequent to the writing of this post, written a few other posts on this subject, most notably one called The Zillow Bump, which I think you might enjoy reading.)