One of the things I’ve always wanted from technology is the ability to point my phone at something or take a screenshot, and have it say, here’s what that thing is and here’s where you can buy it. Well, my lavish consumerist dream is starting to come true, and GrokStyle is one company working to make it so — in this case, automatically identifying furniture and home decor from just about any picture or angle.
The basic idea is this: you open an app or web interface, and upload or take a picture of, say, a chair or lamp you like. Any angle, any style. The GrokStyle service immediately returns the closest matches, either including the object itself or ones very like it.
That would of course be very handy for any interior decorator, but ordinary shoppers can make it work, too. See a cool lamp at Restoration Hardware but can’t see yourself spending $500 on it? Maybe CB2 has something similar — GrokStyle would have hooks into individual stores’ inventories so you could check that sort of thing, or sort by price range.
As a business, it would offer its service as an API that retail apps or web interfaces could call. And having the deep knowledge of what different designers, rooms and regions have put together, it could even recommend, say, an end table to go with the couch you just looked up.
At heart it’s all about computer vision and machine learning, of course: figuring out what the picture is of, then the contours of the piece, the dimensions, colors and so on, and matching that info to a library of millions the company has collected.
“The company started from research at Cornell,” said co-founder and CEO Sean Bell. “We were focused on the question ‘what is it?’ And we developed the best tech for that problem.”
It isn’t easy: lighting, occlusion, weird angles — not to mention the intentions of the user. Did they mean to shoot the table or the chair next to it? And is that walnut or cherry?
Easy or not, they made it work, and the resulting system is so fast it almost looks fake (it isn’t). Watch a prototype in action here:
Co-founder Kavita Bala led the lab that Bell joined as a grad student, and they presented the foundation for GrokStyle’s tech at SIGGRAPH in 2015. As it showed so much promise, they decided to spin the project off as its own company, something Cornell clearly approved of.
“Entrepreneurship at Cornell has become a big deal over the last few years,” Bala said. “We’re very plugged in, they want us to succeed.”
Part of that was, of course, getting hooked up with the people who have the money. GrokStyle started with a grant from the National Science Foundation for $225,000, and have parlayed that into a total of $2 million from a variety of firms and angels. I don’t want to leave anyone out, so I’m just going to quote Sean’s summary:
Canaan Partners, Amino Capital, Neuron.VC, Krishna Bharat (ex-head of Google News), Amit Singhal (ex-head of Google Search), Luc Vincent (VP of Engineering at Lyft), Red Bear Angels (a Cornell angel network), and several other angel investors.
CB Insights liked them enough to include them in the AI 100 list — no small feat, considering the number of AI startups these days. The plan right now is to push hard in the furniture space, but that’s by no means the end of the story.
“We think the algorithm is generalizable,” said Bala. “Multiple people have talked about TV as a use case — if you had the right partners for that…”
Not to mention how this type of tech can be applied in fashion, tourism and any other highly visual market. Furniture, while there’s a great variety of it, is at least stationary and relatively rigid. You don’t have to worry about an ottoman flapping in the wind, or whether a bar stool’s chairs will be in the same place second by second. Shoes and handbags, however, might act as a stepping-off point for the company in that space.
For now the company is working on getting partnerships up and running (what they were, I couldn’t get them to tell me), and further expanding the algorithm’s accuracy and flexibility by hiring “a huge pile of engineers.”
You can learn more about GrokStyle’s systems and what they’re up to (or see about getting API access) over at the company’s website.