Monday, November 9, 2015

The Zillow Bump

For readers of this blog, my fascination with the audacity of Zillow is not new.  If you will recall, I wrote in September, 2014, that Zestimates from Zillow are Worthless by showing that there is no rhyme or reason to the numbers that they publish as the "market value" to a home. I posted many updates to that post.  I have now observed a variation on this that further supports this hypothesis.

My first example is a home that had a Zestimate of $1,700,000.  Then, on March 4, 2015, this home was listed for sale for $3,780,000.   You can see from the chart below what has happened to the Zestimate for this home primarily based upon the home being placed on the market:

Five months later, in August, 2015, the Zestimate had zoomed to $3,358,700.  This is almost a 100% rise in the Zestimate.  Subsequently, on September 11, 2015, the listing for the home was shown as a pending sale, On October 28, 2015 the home closed at $3,300,000. As of November 9, 2015,  only two weeks later the Zestimate dropped by $385,600  to $2,973,100.  What could possibly be the explanation of this bump? My hypothesis was that it would be too obvious if Zillow increased the Zestimate to match slightly less than the listing price on an immediate basis, but rather gradually increase the Zestimate, so that by the time of the sale a number of months later the Zestimate would rise enough in the interim to more closely correlate to the sale price.

However, here is another example that seems to demonstrate that they have no shame. This home had a Zestimate of $1,358,500 on October 9, 2015.  Then, on November 4, 2015, this home was listed for sale for $2,495,000.   You can see from the chart below what has happened to the Zestimate for this home solely based upon the home being placed on the market:
As of November 11, 2015, the Zestimate for the home was $1,914,300, an increase of $555,800 over 30 days previously.  That is an increase approaching 50% in one month.

Zillow prides itself on the fact that the statistical correlation between the Zestimate and a home's sale price is high.  Could it be that Zillow is "cooking the books," so to speak, by trying make the Zestimates "catch up" to the expected sales price by BUMPING them up when the home gets listed for sale?

As a corollary, consider this: realtors pride themselves in pointing out that listing prices are strongly statistically correlated to sales price.  I believe this to be true.  Realtors like to have realistic prices on homes going up for sale and strive to get the owner to select a price approximately 5% to 10% above the projected sales price.  That is good for them because it means that they have a better chance of actually selling a more realistically priced home.

So, if the correlation between listing price and sale price is stronger than the Zestimate price and sale price, then Zillow is correct to dramatically alter the Zestimate by moving it much closer to the broker listed price upon listing.  But, the other corollary is that this is an open admission by Zillow that the Zestimate published by them prior to listing are truly worthless!

If the powers that be at Zillow would like to try to refute this conclusion I would be more than happy to publish it here (so long as they remain factual).


**************update as of 11/30/15 with additions afterwards*******************

Here is a spreadsheet for these two properties above with data to make them current.  The theory of the interpretation of the Zestimate algorithm function changes seems to be gaining strength. I think what remains is to find out the weighting factor for increases and decreases on the spread of the Zestimate from the listing price.

                                Property #1                                    Property #2

Date                                    Total change                                     Total change
                    Zestimate        in  Zestimate            Zestimate         in  Zestimate

3/4/15          1,700,000     (Listed $3.78M)                                                 
8/1/15          3,358,700        +$1,658,700                                                    
9/11/15        3,358,700   (Listing Removed)                                               
10/9/15                                                              1,358,500  
10/28/15      3,358,700       (Sold $3.3M)                    
11/4/15                                                              1,358,500     (Listed $2.495M)
11/9/15        2,973,100           -$385,600           1,914,300          +$555,800
11/17/15                                                            1,917,400          +$558,100
11/18/15                                                            1,922,400          +$563,100
11/24/15      2,895,800           -$462,900           1,926,200          +$566,999 
11/26/15      2,880,799           -$478,000           1,925,700          +$566,400
11/27/15      2,857,900           -$500,800           1,928,500          +$569,200
11/30/15      2,846,200           -$512,500           1,931,200          +$571,900       
12/2/15        2,842,200           -$516,500           1,931,100          +$571,800
12/3/15        2,842,000           -$516,500           1,931,100      (List Reduce to $2.395M)
12/4/15        2,838,400           -$520,100           1,931,800           +$572,500
12/7/15        2,834,700           -$523,800           1,926,700           +$567,400
12/11/15      2,821,600           -$536,900           1,924,000           +$564,700
12/14/15      2,819,800           -$538,700           1,925,800           +$566,500
12/18/15      2,816,500           -$542,000           1,927,200           +$567,900
12/24/15      2,810,600           -$548,400           1,929,200           +$569,900
12/28/15      2,805,400           -$553,600           1,930,200           +$570,900
12/30/15      2,805,300           -$553,700           1,931,900           +$572,600
1/2/16          2,805,100           -$553,900           1,934,500           +$575,200
1/4/16          2,804,100           -$554,900           1,937,100           +$577,800
1/6/16          2,805,800           -$553,200           1,940,800           +$581,500
1/8/16          2,810,500           -$548,500           1,944,400           +$585,100
1/9/16          2,815,900           -$543,100           1,946,600           +$587,300
1/10/16        2,814,600           -$544,400           1,949,400           +$590,100
1/11/16        2,813,200           -$545,900           1,949,900           +$590,600
1/12/16        2,812,100           -$547,000           1,950,700           +$591,400
1/13/16        2,809,400           -$549,700           1,950,900           +$591,600
1/14/16        2,809,000           -$550,100           1,951,600           +$592,300
1/15/16        2,815,000           -$544,100           1,948,700           +$589,400
1/16/16        2,820.800           -$538,300           1,948,900           +$589.600
1/17/16        2,826,500           -$532,600           1,949,400           +$590,100
1/19/16        2,837,400           -$521,700           1,949,900           +$590,600
1/21/16        2,846,700           -$512,400           1,952,200           +$592,900
1/22/16        2,849,900           -$509,200           1,952,300           +$593,000
1/23/16        2,853,400           -$505,700           1,951,800           +$592,500
1/24/16        2,857,100           -$502,000           1,951,800           +$592,500
1/26/16        2,861,500           -$497,600           1,952,400           +$593,100
1/27/16        2,866,100           -$493,000           1,951,200       (List Reduce to $2.249M)
1/29/16        2,865,300           -$493,800           1,950,200           +$592,100 (sale pending)
1/31/16        2,869,200           -$489,900           1,951,000           +$592,900
2/2/16          2,876,100           -$483,000           1,948,900           +$590,800
2/7/16          2,894,700           -$464,400           1,945,100           +$587,000
2/8/16          2,895,300           -$463,800           1,944,800           +$586,700
2/14/16        2,917,500           -$441,600           1,944,700           +$586,600
2/16/16        2,926,000           -$433,100           1,942,200           +$584,100
3/3/16          2,972,800           -$386,300           1,921,400           +$563,300 (sold at $2.225M)

On Property #2 the Zestimate increased to within $564,800 (22.6%) of the listing price, but since the listing price was subsequently reduced the Zestimate of 1,952,400 was within $442,600 (18.5%) of the reduced listing price ($2.395M). Then, the listing price was reduced again ($2.249M) so that the Zestimate is within $297.8M of this new price, so the Zestimate is within 13.2% of the listed price. Finally, the house is listed as pending sold. Once we have the actual sale price we can compute the accuracy of the pre listing Zestimate and the final Zestimate at the time of sale with the sale price.  What is the target that Zillow is aiming for?
Now the results are in. The house sold for $2,225,000 based upon going into contract on 1/27/16.
On that date the Zestimate was 1,951,200. The Zestimate at the time of contract was 12.3% below the sale price.  The Zestimate was $1,358,100 on the day the house went up for sale. So the unadulterated Zestimate was 39% below the presumed "market price". Wow!
We see that for Property #1 above the Zestimate increased to within $420,000 (11.1%) of the Listing price on or around 9/11/15. After the listing was removed for this home (it had gone into contract) the Zestimate has dropped to a low of $554,900 (16.4%).  While the Zestimate of $3,358,700 at the time of the listing removal was within $58,700 (1.7%) of its actual sale price, it is clear that the bumping algorithm worked well in this instance with the objective of Zillow being achieved in getting the sale price close to the Zestimate at the time of the sale.  But it is also clear that the Zestimate prior to the one that did not have the benefit of a professional estimating the listing price was worthless. It was a full 48.5 % below the actual sale price.  And what do we make of the new Zestimate low on 1/4/16 of $2,804,100 which is 16.5% below the actual sale price which was only about two months prior to this date?  This makes no sense. And now again between 1/4/16 and 2/14/16 the Zestimate has increased by $113,400.  That is an increase of 4% in just over 40 days.  And by 3/7/16 it had risen from 1/4/16 the Zestimate had increased by $168,700. That's 6% in two months!                                                                                            


******************update as of 1/2/16*********************************

I have now found some additional examples of the Zillow Bump that should add more fuel to the fire.
First, let me put up the data for consideration.  This is the asking pricing information for this home
It was listed at $995,000 in late June 2015.  In December, after the fourth price drop it was listed at $795,000


Now, let's look at the Zestimate. I do not have complete information on the history here.  It shows that sometime around the time the house was listed, the Zestimate was around $960,000.  There is no record of a Zestimate price for this home shown before that date.  However, this is a subdivision and the other 40 homes in this subdivision all have Zestimates between $580,000 to $708,000. This home does not stand out, so it would be fair to assume that before the listing for sale the Zestimate was around $644,000.  Interestingly, Zestimates for other homes in this subdivision have increased in value consistently about 5.4% during this period of time, in contrast with this home where the Zestimate has diminished in value by about 15.1%, making the variance a whopping 20.5%.

 
Now, to make this Zillow entry even more bizarre is the fact that the house appears to be listed twice, but only one of them is listed for sale.  So, we have a duplication of the same data, once for the home that is listed, and once for the very same home that is not listed.  The listed home Zestimate is $720,800 and the identical home not listed has a Zestimate of $646,100, about a few hundred dollars off the median for the other 40 homes in the subdivision.  I think this is as conclusive as it is going to get evidencing the true slight of hand being used by Zillow to fudge the Zestimates of listed homes just before they sell to bring them closer to the anticipated sale price to improve their "claim" of Zestimate accuracy. Also, note that as of 1/2/16 the current Zestimate is $65,000 (8.2%) below the current asking price, which appears to be Zillow's best guess at where the sale price will be at this time.  Very interesting, indeed!

Below I am tracking the information on changes to Zestimates for the two variants of this one home:

Property #3
                                 Zestimate           Zestimate            Asking       Listed Zestimate %
Date                       Listed Home     Unlisted Home         Price        Below Asking Price
1/2/16                       $730,000           $644,800            $795,000             8.2%
1/7/16                       $728,100           $644,700            $795,000             8.4%
1/8/16                       $727,100           $644,400            $795,000             8.5%
1/9/16                       $726,600           $644,400            $795,000             8.6%
1/10/16                     $726,700           $644,900            $795,000             8.6%
1/11/16                     $726,100           $644,900            $795,000             8.7%
1/12/16                     $725,400           $644,800            $795,000             8.8%
1/13/16                     $724,900           $645,200            $795,000             8.8%        
1/14/16                     $724,600           $645,300            $795,000             8.9%
1/15/16                     $723,900           $644,700            $795,000             8.9%
1/16/16                     $723,600           $644,800            $795,000             9.0%
1/17/16                     $723,600           $644,700            $795,000             9.0%
1/19/16                     $722,500           $645,300            $795,000             9.1%
1/21/16                     $721,800           $645,000            $795,000             9.2%
1/22/16                     $721,500           $645,100            $795,000             9.3%
1/23/16                     $721,200           $645,300            $795,000             9.3%
1/24/16                     $720,900           $645,000            $795,000             9.3%
1/26/16                     $720,800           $645,100            $795,000             9.3%
1/27/16                     $720,400           $645,500            $795,000             9.4%(sale pending)
1/29/16                     $719,900           $646,200            $795,000             9.5%
1/31/16                     $718,700           $647,400            $795,000             9.6%
2/2/16                       $717,900           $647,000            $795,000             9.7%
2/7/16                       $716,800           $645,900            $795,000             9.8%
2/8/16                       $716,200           $645,300            $795,000             9.9%
2/14/16                     $716,700           $652,200            $795,000             9.8%
2/16/16                        ------               $653,900            $750,000             4.4% (sale recorded)

Currently, there is only one home listed above (Property #2) that is currently for sale (pending) and it has a Zestimate that is 13.6% below the listing price.   Right now I am approximating that the home will sell for approximately 8.9% below the listing price, so that would bring the Zestimates to be about 4.7% below the asking price. Recall that the Zestimate for property #1 above ultimately sold for 1.4% below the sale price.

Property #3 is a bit more difficult to analyze because Zillow had two distinct listings for the same home. I think the only fair way to do this is to look at the two algorithms that they used.
In the first case the Zestimate was was essentially $646,100 before the house was listed, and it sold for $750,000.  This means that the Zestimate was really understating the market price by 13.6%.  But, the artificial Zestimate created after the house was listed was $720,400 at the time that the house became a pending sale. This is an understatement of $29,600 for the market price that it was eventually sold for, or just 3.9% below, meaning that they fudged the accuracy of the Zestimate with this technique by a full 9.7%. Astounding!

 So, it looks like their cooking the data books technique is working quite well for them, pity the poor person that relies on their claims of accuracy.  It may be that the Zestimate target for listed homes is variable by region, or possibly even by price or some other metric that we don't know about.

A final note on Property #3 now that the property has sold at $750,000 please note that the Zestimate is currently $653,900, a full 12.8% below the current market price.  You would think that they might adjust the Zestimate with the new data point.  We will continue to follow this bizarre behavior of this algorithm a bit further.


last modified 3/7/2016

2 comments:

  1. Isn't there some Federal Agency that might hold them accountable?

    ReplyDelete
  2. Paul,
    There is no agency of the government that has regulatory power over Zillow on this matter. However, there are two other ways that this could be changed.
    1. A consumer could file a suit against Zillow, either individually, or in a class action suit, claiming that they were damaged in some fashion. I personally think that this will be difficult to prove at the individual level.
    2. Legislation could be passed, either at the State or Federal level requiring a certain performance level by Zillow and other information providers, or an outright ban on the practice of providing unverified and inaccurate information that could be used by others to make financial decisions. I think this would be the more effective approach.

    ReplyDelete