Sourav Goswami, Managing Director, Buckingham Companies, and cofounder of Haystacks.AI.
“The sun’ll come out tomorrow,” proclaims the musical’s eponymous Annie—even going so far as to suggest the listener can “bet their bottom dollar” on the outcome. She is basically making a prediction about a future event: sunrise. But what happens if the model on which she’s based her observations is flawed? What about when the sun does NOT, in fact, come out as expected, such as during a rare solar eclipse? What if the probability the sun rises as expected is conditional on some or all of her own observations?
This example is meant to illustrate the heart of an important debate between frequentist (linear regression) and Bayesian (probabilistic) statistics: Is a large enough data set sufficient to predict a future outcome? Or is the nature of the data (and how to incorporate deviations, or “outliers,” into the forecast) more crucial to evolving the predictive model?
In real estate, how this question is resolved could provide a paradigm-changing view on how to forecast rents and even cap rates. Leading real estate funds such as BentallGreenOak, Blackstone (subscription required), Buckingham and Starwood Capital have been investing in developing more cutting-edge tools, even going so far as building out internal data science teams. This is important since innovation in real estate often requires consensus views.
Real estate technology is about evolution, not revolution. Changes need to be driven from within the industry in order to best capture what is useful for investors. BentallGreenOak has maintained that being able to build robust models and create tools to obtain fast and actionable insights from big data is crucial in generating real alpha for investors.
Understanding Linear Regression And Frequentism
Today, the state-of-the-art modeling rubric used in residential real estate for rent forecasting is linear regression. The biggest innovation has been the aggregation of an ever-increasing quantum of data. Companies such as HouseCanary and Corelogic have been providing forecasts using this methodology and have been leading service providers in this approach.
As processing power has increased and various other industries have started to develop AI and machine-learning models with a dynamic approach, an alternative approach has emerged. Let’s think about an easy-to-understand traditional example of probability: the coin flip. A linear regression would suggest that flipping a coin 100 times should result in 50 heads and 50 tails. If the outcome yields 60 heads, the assumption is that the sample set is not sufficiently large, and so the statistician would flip it another hundred times … and another hundred … and another, until the distribution is closer to the expected 50/50.
Such a calculation is entirely dependent on the hidden assumption of independence and identical distribution (IID), which may not be true. In popular parlance, this is called frequentism, which relies on simple counting and division. As such, it is not a probability but a frequency.
Now, let’s explore the Bayesian approach, which my company’s solutions leverage. This probabilistic modeling technique would deviate from the prior example in a key way: What if getting 60 heads was not an aberration? What if the model was excluding key variables that were unknowable? The distribution could be impacted by things such as the coin’s weight distribution, the concavity/convexity of the sides of the coin, how the airflow in the room was impacting the rotation of the coin, or any number of other “alternative” factors that cause the IID expectation to break down.
In finance, this can be tested by assuming time dependency, in the sense that a period of high (or low) market volatility tends to persist until some shock, such as geopolitical upheaval, occurs. The idea of such persistence was deeply explored by Andrei Markov, giving rise to Markov chains, which have been extremely important to modern financial modeling.
We can further delve into this with the idea of inverse probability—that an effect is proportionately dependent on its cause. This idea was posed long ago by the Reverend Thomas Bayes, with his theory first being used by his friend, Richard Price, to test the probability that God exists given all we see around us. This question was made more mathematically tractable by Simon Laplace. In popular parlance, this is called Bayesian (or probabilistic) analysis, and Bayesians typically think that it is the only logical way to consider probabilities, especially when data is sparse, as they are in real estate. This approach allows models to better predict the future as more data is factored in.
It wasn’t until the advent of high-speed computing that using Bayesian/probabilistic approaches was feasible for forecasting. For real estate, they can be a critical component to forecast rent growth, future micro-market cap rates, and valuation drivers. To tie back to the prior example of a coin flip, this would entail “alternative” variables, such as sentiment measures, safety scores, willingness to commute and “ancillary” data, such as walkability, local job growth and educational attainment, being incorporated into forecasting models. The model could then be refined and rebalanced in an iterative manner until the past observations were more closely predictive of future occurrences.
As the market for institutional investments into the SFR and BTR sectors continue to grow, it is important that sponsors take into account all models available to them. There continues to be value in the traditional approach of linear regressions, especially for tools such as revenue management and evaluating in-fill locations, especially as the Bayesian models continue to evolve and improve. Over time, it is likely that the data present in information-heavy frequentist models will be ingested into probabilistic SaaS models to generate more precise forecasts. Alpha in real estate investing boils down to basis points, so any incremental improvement can have a significant impact on efficiency-seeking flow of capital.
Forbes Business Council is the foremost growth and networking organization for business owners and leaders. Do I qualify?
Read the full article here