Broker vs Bot

Robot Outperforms Realtors Finding Homes Buyers Prefer

 

QValue Find More Genius Nominated for Technology Innovation Award

The QValue Home Search Bot, Find More Genius, is nominated for Inman News Most Innovative Technology Award in the real estate industry for 2016.

 

 

Inman News Broker vs Bot Challenge asks: can a machine compete with experienced real estate brokers at suggesting homes a buyer likes?

This is a more in-depth explanation of the Inman News Broker vs Bot Challenge conducted the last week of April 2016.

Our goals were simple but to attain the abilities to meet the challenge required some sophisticated algorithms and more importantly understanding why someone actually buys a home.  This is the story of how we did this.

 

 

Some of the News Sources Covering QValue and our Find More Genius Robot

 

Will home buyers finally have an automatic way to find homes they actually love?

Denver Colorado May 10, 2016.  Over three days a robot (computer algorithm) competed head-to-head with three highly experienced Denver real estate brokers to see who recommended homes a buyer most preferred.  The buyer consistently ranked the homes suggested by the robot as their first-choice favorites all three days.

The public’s interest in home search and technology is exploding as demonstrated by more than 123 million unique visitors per month to the top fifteen US real estate web portals.  The home search algorithm tested here may offer them an automatic means for finding homes they love.

Inman News conducted the contest April 27, 28, and 29, presenting the results May 10. QValue® provided the “robot” algorithm called “Find More Genius” (AKA QValueBOT).  Participating brokers’ information is available at Inman News.  Local Denver real estate expert John Rebchook acted as the buyer and judged the results of the experiment.

Each day started with the buyer identifying a sample home they truly liked.  It was the job of the broker and robot to suggest homes available for sale they hoped would be as emotionally stimulating to the buyer as the sample home. Daily the broker and the robot each returned three homes to the buyer, who then rank ordered, for very personal reasons which they did not have to explain, the homes they most preferred.

 

What the Broker vs Bot Challenge was all About

This was the test: can a machine compete with a human (experienced broker) at suggesting homes which are similar to what a buyer subjectively likes?  If so then computerized Home Search is stepping outside of matching physical (quantitative) criteria and entering into the realm of interpreting the kinds of homes and subjective qualities a home buyer really desires (qualitative criteria).

 

Goal #1: Robot to Offer Homes a Buyer Cannot Differentiate From Those Suggested by a Seasoned Professional Broker

Each day one of three different brokers and the robot received information on a home a buyer likes a lot, a sample home demonstrating the qualities the buyer prefers.  Then the broker and bot each suggested three homes they hoped would be as emotionally stimulating for the buyer as the sample home.  This same process occurred April 27, 28, and 29, 2016.

The buyer received six homes back each day: three from the broker and three from the robot.  The homes were randomly presented without the buyer knowing from whom they came.  It was the buyer’s challenge to determine which three homes came from a robot.  If the buyer could consistently differentiate the homes suggested by the robot then obviously the algorithm was too mechanical–not seeking out and providing the emotionally stimulating esoteric qualities the buyer most desired.  If the buyer could not differentiate the homes suggested by the robot from the homes suggested by the broker, this portion of the test is considered a success.

(At the bottom of this post is information about the Turing Test, where a person attempts to identify whether they are communicating with a person or a robot via computer terminals.  Borrowing concepts from the original Turing Test fueled the idea for this Broker vs Bot Challenge–but with a harsh twist.  In our test the home buyer “interrogator” not only had to determine who provided the homes (broker or robot), but presented the robot with a massive challenge of suggesting homes emotionally stimulating for the buyer.  To our knowledge this has never been done in the real estate industry.)

 

The Results of Goal #1:

Day 1

  1. Favorite home came from the robot; buyer mistakenly thought it came from the broker.
  2. Second most favorite home came from the robot; buyer mistakenly thought it came from the broker.
  3. Third most favorite home came from the broker; buyer mistakenly thought it came from the robot.

Day one exceeded the goal of the robot suggesting homes which cannot be differentiated from a seasoned professional broker.

Day 2

  1. Favorite home came from the robot; buyer mistakenly thought it came from the broker.
  2. Second most favorite home came from the broker; buyer mistakenly thought it came from the robot.
  3. Third most favorite home came from the robot; buyer thought it came from the robot.

Day two met the goal of the buyer not consistently identifying which homes came from which source.

Day 3

  1. Favorite home came from the robot; buyer mistakenly thought it came from the broker.
  2. Second most favorite home came from both the robot and the broker (both suggested the same home).
  3. Third most favorite home came from the broker; buyer mistakenly thought it came from the robot.

Day three exceeded the goal of the robot suggesting homes which cannot be differentiated from a seasoned professional broker.

 

Goal #1 overall exceeded the initial Turing Test parameters: the “interrogator” (our home buyer) could not differentiate which homes came from the robot.  Goal #2 was the grander challenge: can the robot suggest homes as emotionally appealing for the buyer as highly experienced real estate brokers can?

 

Goal #2: Determine if the Robot Can Suggest Homes with Emotionally Triggering Qualities the Buyer Most Desired

Each day the buyer rank ordered the six homes presented from best to worst based on their subjective preference of the homes.  They simply rank ordered, for very personal reasons which they did not have to explain, the homes they most liked.  Obviously the broker has human empathy and can visualize what the buyer most desires, can see and understand pictures of each home and how the form, flow, and finish of each compared to the home the buyer defined as one they like each day,  and can try to interpret what and why the buyer most likes about homes.  A computer algorithm cannot do any of these things.

So the monster challenge is whether the robot can suggest homes attaining ranking within the first three of the buyer’s favorites each day–a daunting challenge for a computer algorithm.

If the computer consistently fails to attain any of the top three rankings then obviously it cannot suggest homes based on the emotionally triggering qualities a home buyer most desires.  But if the algorithm can attain consistent results in the top three buyer favorites then automatic home search has moved from basic quantitative matching functions such as price range and bedroom count to more esoteric searches based on emotionally triggering qualities of homes–the real reasons someone falls in love with and buys a home.

 

The Results of Goal #2:

Day 1

  • Favorite home came from the robot.
  • Second most favorite home came from the robot.

Day 2

  • Favorite home came from the robot.
  • Third most favorite home came from the robot.

Day 3

  • Favorite home came from the robot.
  • Second most favorite home came from both the robot and the broker.

Because the robot’s suggested homes were deemed the buyer’s favorites all three days, and because two of the robot’s choices each day held two of the top three buyer choices, goal #2 is deemed a success.

 

The algorithm’s recommendations consistently placed two homes within the top three buyer choices each day, and placed as the buyer’s #1 favorite home every time all three days.

This tells me the algorithm can indeed suggest homes based on emotionally triggering qualities buyers desire (qualitative criteria), and has moved far away from primitive current strategies used by all other websites, portals, and including all MLSs, which search by quantitative measures such as price range and bedroom counts.

We are moving toward some really exciting stuff–Creed Smith, QValue

 

How the Automatic Home Search Algorithm Works

Through intuition and from watching the video above you may believe simple text matching produces the output from the algorithm.  It does not.

Simple text matching is available in some MLS systems and in very limited function at some web search portals and sites.  This rudimentary search function can sometimes match exact words but these words are without context.  For example the word “remodeled” may be searchable in an MLS or on a website, but without the inferred understanding of what the buyer really wants it is of no use.  Are you capturing the text from a sentence that states “the home is completely remodeled” or from a sentence stating “the home needs completely remodeled?”

While text analysis is used by the home search algorithm to designate potential key terms, the algorithm additionally attempts to define purpose or context; another daunting task for an algorithm based on the incomplete descriptions on most homes in the MLS and then propagated throughout the internet.  Simply watch the video above and you will notice that 75% of what brokers place within a home’s description are meaningless “fluff” terms, all the while leaving out very important qualities each home has which would trigger an emotional response from a home buyer.

Additionally the algorithm uses a lexicon of nearly 1,000 words to match and assign context to home descriptions.  This occurs nearly simultaneously instead of the single-word or incremental search pattern of MLS systems or internet portals.  It’s the interrelation of the context and implied meaning of all key terms within the algorithm that allows it to “interpret” what a home buyer desires and then suggest appropriate homes.  This is nearly impossible to accomplish using standard text matching, and is exhausting for a human to attempt.

The Find More Genius™ home search tool allows a buyer to simply point to any home they really like, click one button and immediately see every home available much like it.  It’s an instant custom home search based on what a buyer most loves about any home.

Allowing the buyer to simply see a home they love and click one button is vital to suggesting more homes they will love.  Prior home search tools attempted forcing a buyer to denote a series of key words for creating search parameters.  Other search engines additionally forced the buyer to weight the importance of each term.  Aside from the aforementioned text matching shortcomings mentioned above, several additional problems and inaccuracies are created with these tactics.

1) A major problem of asking a buyer questions about a home is generically referred to as Survey Bias.  Falling in love with a home is an emotional experience.  By forcing a buyer to list or weight criteria you shift their focus from a subconscious emotional process to a fully cognitive process.  You will receive erroneous guidance and provide poor results.

2) Often much of the decision process occurs at the subconscious level.  A buyer falls in love with a home when particular qualities within the home cause them to envision future life events.  Most buyers do not realize this is happening so it is impossible for them to define ahead of time what causes them to like particular homes.  It’s better just to let them see a home they love, click a single button with no questions asked, and present them homes of similar qualities triggering them to envision these future life events.

3) Extremely sophisticated multi-layered tracking algorithms and recommendation engines attempt to note every move a buyer makes on both the primary search site and perhaps throughout the internet to determine what they desire.  And while this can be instructive it requires very large-scale, complex, and probably cost-prohibitive programming and data.  It may be easier to just let the buyer see something them love and instantly offer them every home for sale much like it.

Along with using the meaning of key qualities to search out and suggest homes QValue® also controls for a range of physical characteristics such as square feet and bedroom count and price range, location–but not so much a straight line or diameter search like most methods, but instead “shopping” neighborhood to neighborhood, and controlling for similar qualities of the areas suggested homes are in.  This is all done nearly simultaneously and within tolerances.

So we nearly simultaneously use a potential 1,000 data points to compare and contrast homes based on which qualities of homes and their locations trigger a buyer to envision future life events and fall in love with a home.  As soon as they spot a great home, whether it’s, an “industrial urban loft with twelve foot ceilings, walls of glass, concrete floors, and steel beams”, or a “turn-of-the-century Victorian home with original character, ten inch hand-carved moldings, oak built-ins, and restored but not remodeled condition”, our algorithm “understands” the emotions our buyer feels and presents them a list of homes, rank ordered with the best matches at the top of the list, all in about two seconds.

Home search should be fun, instant, and effective–not some time consuming and exhausting chore.  Find More Genius™ is an instant custom home search based on what a buyer most loves about a home.

 

 

 

 

 

Real Estate’s Turing Test

(This is an overview of how the broker vs bot challenge was designed.)

Have computer algorithms advanced to the point of allowing home buyers instant property matches based on qualitative desires such as sunny and bright homes and open flowing designed rooms, as opposed to strictly quantitative ones like bedroom counts and price range?

Are the algorithms good enough that the home buyer cannot differentiate which suggested homes come from a human (broker) or from a robot (computer algorithm)?

If so this marks a profound change in the real estate industry.

This is the test: can a machine compete with a human (experienced broker) at suggesting homes which are similar to what a buyer subjectively likes.  If so computerized Home Search is stepping outside of matching physical (quantitative) criteria and entering into the realm of interpreting the kinds of homes and subjective qualities a home buyer really desires (qualitative criteria).

The concept of a Turing Test is where a human “interrogator” and two other participants (one human and the other a computer) interact from separate locations to see if the interrogator can distinguish the human from the computer.

I propose a similar experiment specifically from the real estate industry perspective.  And instead of the interrogator simply determining which of the other parties the computer or the human is through questions, the interrogator judges the quality of veiled homes suggested to them from each source.

With the public’s interest in home search and technology absolutely exploding as demonstrated by more than 123 million unique visitors per month to the top fifteen US real estate web portals, the execution of this experiment over a period of one week should garner substantial readership and reactions.

The end results could set the entire real estate industry back on its heels if a robot no longer just produces matching of defined quantitative search criteria such as price range and bedroom count, but instead offers home buyers properties with qualities they emotionally desire such as airy, flowing, sunlit rooms.  This is a true scientific experiment which could greatly benefit home buyers and sellers, and which carries a considerable potential economic impact in the marketplace.

 

Step 1:

Each of five days our interrogator goes to the real estate portal of their pleasing (perhaps to a different one each day), completes a home search, and chooses a home they truly like.  Our interrogator does not explain why they like the home.  They simply identify a home they truly like.  (Update: this was reduced to three days, only because two of five broker volunteers backed out just before we took the test live.)

It is imperative that they honestly like the home on a personal level based on various subjective qualities such as “bright, open, flowing”, or very personal tastes such as “Victorian style, or industrial inner-city chic style”, or other criteria such as “fully remodeled home or gourmet kitchen” qualities.  The criteria do not matter as much as that the interrogator has a personal affinity for the qualities just as any home buyer has.

Our experiment then entails having a broker and our robot suggesting three similar homes based on the one our interrogator has chosen.  The goal of the broker and the robot are to offer homes as emotionally stimulating back to the buyer as the sample home the buyer presents each day.  From the properties presented back to our interrogator they not only determine which homes they think came from human or machine, but rank orders the homes based on their affinity toward each.

The first decision is binary: the interrogator determines whether the result came from a human or came from a machine, then quickly explains why they believe this—just a sentence or two for each home.

The second decision is purely subjective: “I like this home more than the next on the list” thereby creating a ranked order of the suggested properties each day.  This second choice requires no explanation—just rank order them; it’s a purely emotional response.  But the interrogator must use a gut-level personal feeling in defining their favorite homes.  An honest emotional response is needed.  Since the interrogator is not asked to explain their emotionally based “which home I like more” decisions, the experiment should allow the opportunity for great honesty in ordering which homes they truly prefer.

Again, this is the real test: can a machine compete with a human at suggesting properties which are similar to a home a buyer likes including any subjective peculiarities they prefer.

Each day a different kind or style of home should be chosen.  Our interrogator must still truly like each home chosen, but can choose substantially different homes and/or locations and/or price ranges.

The homes can be anything from $100K fixer uppers to $3 million mansions.  But the interrogator must substantially like characteristics of each home, as they are judging in two areas: 1) which of the suggested properties were from a human or a machine—and very briefly why, and 2) purely which homes they personally like more and rank order them from 1 to 6 (#1 being their favorite).

 

Step 2:

Five brokers are chosen ahead of time with their email addresses at the ready.  Five different brokers are used (one each day) to ensure different search styles and skills, and that the results cannot be due to a single broker’s biases.

At the appointed time each day the MLS identification number of the home of our interrogator’s choosing is emailed to the broker.  Their job is to suggest three similar properties—just as a broker might be requested to do from someone new to town.  “Hey I found home MLS# 1234567 that I really like.  I am in town just for a couple of meetings and have to fly out tomorrow.  But will you arrange for me to see this home and a few others that you suggest are like it before I leave.”

It’s the broker’s and machine’s job to not simply match physical or quantitative criteria of the subject home; it’s their challenge to suggest homes our buyer will be emotionally excited about based on the sample home they have shown us.

Matching homes predicated on qualities a buyer finds emotionally stimulating is a tremendous challenge for a machine, and even a broker—just ask them.  Buyers are hard to please, especially when there’s extremely limited housing inventory such as we have right now In Denver.

The goal of our human and machine is to impress their new client by suggesting homes with the subjective qualities that are as emotionally stimulating as the home our interrogator told us they like.  (Ideally our interrogator should choose homes with a least a little pizazz.  If they choose a particularly bland home, well that’s probably what they are going to get in return.)

The brokers searching via the MLS represent the skilled “manual style” of home search, while the instant machine results represent the more “no human skills involved digital style” home search.

Additionally, if you wish to compare the above results to any other internet home search you might also record our interrogator’s search at Zillow.com or Realtor.com portals while they discover their favored home of the day

The interesting part would occur when our interrogator suffers through trying to discover other homes available with qualities similar to their sample property using the industry’s archaic quantitative-based home search tools currently offered.  This frustrating activity is what our 123 million home searchers mentioned above go through every month.

 

The rules

Rule 1: fully blind testing.  The interrogator can have zero communication with any of the brokers about this study for the entire week.  The suggested properties are of course veiled from which source they came and presented in random order to the interrogator.

Rule 2: brokers involved with the study cannot know the interrogator personally; this would bias their choices.  They must suggest three homes based solely on the sample home the interrogator likes without any prior knowledge of the interrogator’s personal preferences and without any interaction with the interrogator.

Rule 3: no one knows who the brokers are providing results or “who” the robot is until the final report/article is presented.  The brokers cannot talk to anyone about the study.

Note: I can only offer to complete this experiment in Metro Denver.  If you can find another portal or franchise or brokerage offering a similar automated home search system you could simultaneously test theirs also.  But you will need to manage quite a bit of additional complexity in keeping the results uncontaminated and meaningful.

As a final note, this test will be completed under very trying market conditions.  The city of Denver has only about 1,200 properties actively for sale (detached and attached combined).   It’s a fairly extreme “seller’s market” operating at about only 25% of a more normal “healthy” inventory, which makes finding homes matching a buyer’s desires very difficult for both human and machine.  This will be a hardcore test of human against machine as there’s no “fat” in the market.

 

 

 

Inman News Broker vs Bot Challenge

Copyright © creed smith 2017

*Subject to QValue™ and your website provider finalizing the terms of service and reaching an agreement.  But in essence we pull data from the MLS/RETS data on their server via an API we supply, run our multi-step bot algorithms, and provide the data back to them for presentation on your website via an API they provide to us.  We do require a minimum number of agent site subscriptions within each metropolitan area.

Envision™, Envision Instant Home Search™, and No BOT! Then what do you got?™, QValue™, QValueBOT™, Find More Genius™, Today’s New Technology Standard™, Tech for the Non-Technical™, and Instant Access™ are trademarks of QValue™ and Creed Smith.  Patents pending on the QValue™ AVM, Envision™ Instant Home Search™, and QValue™ Find More bot.  All rights reserved by QValue and Creed Smith.