You might also like
A deep dive on the framework that lets us establish essentially the most priceless listings for our company.
By: Carlos Sanchez-Martinez, Sean O’Donnell, Lo-Hua Yuan, Yunshan Zhu
At Airbnb, we at all times attempt to offer our group with the very best expertise. To take action, it’s necessary to grasp what sorts of lodging listings are priceless to our company. We obtain this by calculating and utilizing estimates of itemizing lifetime worth. These estimates not solely permit us to establish which kinds of listings resonate finest with company, but in addition assist us develop assets and suggestions for hosts to extend the worth pushed by their listings.
Many of the current literature on lifetime worth focuses on conventional gross sales channels through which a single vendor transacts with many consumers (e.g. a retailer promoting clothes to a buyer). In distinction, this weblog put up explains how we mannequin lifetime worth in a platform like Airbnb, with a number of sellers and consumers. Within the first part, we describe our basic listings lifetime worth framework. Within the second part, we talk about related challenges when placing this framework into observe.
Our itemizing lifetime worth (LTV) framework estimates three totally different portions of curiosity: baseline LTV, incremental LTV, and marketing-induced incremental LTV.
(1) Baseline LTV
To measure LTV, we have to outline what we imply by “worth” and what time horizon constitutes a “lifetime.” Simplifying barely for the needs of this weblog put up, we outline and estimate our baseline itemizing LTV as the entire variety of bookings {that a} itemizing will make on Airbnb over the subsequent one year.
We depend on machine studying and the wealthy info we’ve about our listings to estimate this amount for every particular person itemizing. In observe, we additionally comply with monetary steerage to reach at current worth by projecting outcomes into the longer term and making use of a related low cost charge to future worth.
Desk 1 exhibits some hypothetical baseline LTV estimates. As you possibly can see from the examples, LTV shouldn’t be static, and might evolve as we enhance the accuracy of our estimates, observe modifications in our market, and even develop a list (e.g., by offering steerage that helps hosts enhance the itemizing to get extra bookings).
We use baseline LTV estimates to section our listings and establish which kinds of listings resonate finest with our company. This informs our provide enlargement technique. We additionally use baseline LTV to establish listings that aren’t anticipated to succeed in their full reserving potential and should profit from extra steerage.
(2) Incremental LTV
When estimating lifetime worth, we face a problem that’s frequent throughout multi-sided marketplaces: the transactions made by one itemizing may come on the expense of one other itemizing’s transactions. For instance, when a brand new itemizing joins our market, this itemizing will get some bookings from company who had been beforehand reserving different listings. We have to account for this dynamic if we wish to precisely measure how a lot worth is added by every itemizing.
We deal with this problem by creating “incremental LTV” estimates. We check with the extra transactions that might not have occurred with out the itemizing’s participation as “incremental worth,” and the transactions that might have occurred even with out the itemizing’s participation as “cannibalized worth.” We estimate the incremental LTV for a list by subtracting cannibalized worth estimates from the baseline LTV. We clarify this adjustment in additional element when discussing measurement challenges.
(3) Advertising and marketing-induced incremental LTV
Lifetime worth shouldn’t be static, and our LTV mannequin wants to inform us how our inside initiatives deliver extra itemizing worth. For instance, suppose we run a advertising and marketing marketing campaign that gives hosts with recommendations on the best way to efficiently enhance their listings. To grasp the return from the marketing campaign, we have to measure how a lot worth is accrued because of the marketing campaign, and the way a lot worth would have been organically accrued with out our advertising and marketing intervention. We calculate “marketing-induced incremental LTV” to measure how a lot extra itemizing LTV is created by our inside initiatives.
Having outlined our measurement framework (summarized in Determine 2), we now cowl a few of the technical challenges we confronted when placing this framework into observe.
Problem (1): Precisely measuring baseline LTV
A very powerful requirement for our framework is correct estimation of baseline LTV. Determine 3 illustrates our estimation setup. First, we leverage itemizing options snapshotted at estimation time t. This information contains wealthy data we’ve about every itemizing and host (availability, worth, location, host tenure, and many others). We then use these options to coach our machine studying mannequin. As a price label, we use the variety of bookings made inside the subsequent 365-day interval, which is noticed on date t + 365.
This setup has two necessary implications that affect accuracy and analysis:
- We’ve to attend one year to completely consider the accuracy of a prediction.
- Our preliminary coaching information won’t permit us to make correct predictions if we observe shocks between the time when the coaching information was captured, and the time once we rating the mannequin.
In observe, we felt the complete penalties of those implications through the COVID-19 pandemic, when journey got here to a halt and market dynamics modified drastically. Our mannequin’s coaching information from earlier than the pandemic had dramatically totally different traits relative to the scoring information we collected after the pandemic. When coping with this shock, we carried out numerous methods that helped us enhance mannequin accuracy:
- Decreasing coaching home windows, permitting us to cut back mannequin drift.
- Feeding the mannequin with granular geographic information and human-provided details about exterior components as borders closed and reopened because of the pandemic.
- Adopting LightGBM, which handles excessive cardinality options just like the geographic variables talked about beforehand.
Problem (2): Measuring incrementality
Accounting for incrementality is difficult as a result of we by no means observe the bottom fact. Whereas we observe what number of bookings are made per itemizing, we can not inform which bookings are incremental and which bookings are cannibalized from different listings.
Since we don’t have an incrementality label to estimate this consequence instantly, we as a substitute estimate a manufacturing operate. Intuitively, incrementality is closely depending on our potential to attach either side of our market. Manufacturing features permit us to establish when our provide of listings and demand from company join and supply incremental worth. Incrementality estimates might be excessive when a section has excessive visitor demand and comparatively low itemizing provide. In distinction, incrementality might be low when segments have a big quantity of itemizing provide and comparatively low demand, which means company have a straightforward time discovering a spot to remain and a brand new itemizing is extra more likely to cannibalize bookings from different listings.
Particularly, we mannequin how our whole provide of listings (S) and whole demand from company (D) impacts our goal consequence bookings (O), as in equation (1):
We estimate this mannequin with historic provide, demand, and consequence information aggregated throughout internally-defined segments which have little overlapping demand. Having estimated mannequin (1), we calculate how further provide of listings ends in extra bookings within the given section: that is our estimate of incrementality.
Problem (3): Dealing with uncertainty
To deal with the uncertainty we skilled through the pandemic, we started updating our LTV estimates as listings obtained higher or fewer numbers of bookings than initially anticipated. This strategy has helped us seize any shocks that happen after making our preliminary predictions.
To point out how this may be helpful, let’s return to our advertising and marketing marketing campaign instance. Assume that we run this marketing campaign for six months, and that we measure the success of this marketing campaign by evaluating marketing-induced incremental LTV towards our whole advertising and marketing funding within the marketing campaign. As a primary strategy, we might use the preliminary baseline LTV figures (which feed into marketing-induced LTV) estimated on the time when the itemizing was first focused by our initiative. Nevertheless, listings focused on day 1 of the advertising and marketing marketing campaign can have six months of reserving historical past by the point the marketing campaign ends and we consider success. A extra correct strategy makes use of realized bookings after the preliminary prediction to begin correcting for mannequin error.
Desk 2 illustrates how this works. Suppose that on 2024–01–01, we anticipate that Itemizing A will get a complete of 16 bookings by the top of the 12 months. If six months into the 365 day interval, Itemizing A has obtained 16 bookings, we should always regulate its anticipated worth upward to, say, 21 bookings. Actually, every single day for one year after 2024–01–01, we will have a look at the bookings that Itemizing A has accrued and regulate the anticipated bookings accordingly. By building, the anticipated and accrued bookings converge to the ultimate bookings one year after the preliminary reserving date. Going again to our advertising and marketing instance, if Itemizing A in the end receives 20 bookings, updating the preliminary estimate means we went from 20% underprediction on day 0 to a extra affordable 5% overprediction as of month 6.
In observe, we make each day changes to a list’s anticipated worth based mostly on the itemizing’s accrued worth, up to date itemizing options, and worth arrival patterns for related listings estimated utilizing historic information.
On this weblog put up, we defined how we strategy itemizing lifetime worth at Airbnb. We coated our measurement framework, together with baseline LTV, incremental LTV, and marketing-induced incremental LTV. We additionally zoomed into measurement challenges, like when journey patterns modified drastically through the COVID pandemic and precisely estimating LTV turned tougher.
Estimating the lifetime worth for every itemizing is necessary as a result of it helps us serve our group extra successfully. Use instances embody:
- Figuring out distinctive itemizing segments by way of which new hosts can showcase their hospitality to a big visitor viewers.
- Pinpointing places the place listings have a possibility to get extra bookings, and may profit from extra demand.
- Figuring out which inside advertising and marketing initiatives deliver essentially the most worth to our group.
It’s additionally price noting that our measurement framework could prolong to different purposes, such because the lifetime worth for Airbnb Experiences listings, the place the worth of an expertise itemizing will closely rely on journey developments and on company’ potential to find these experiences.
We proceed to resolve attention-grabbing issues round LTV every single day (and as extra insights come up, we’ll preserve sharing them on our weblog). Are you able to see your self making an affect right here? In that case, we encourage you to discover the open roles on our staff.
