Why Bixby will be the star of the Samsung Galaxy S8

Samsung has a new voice. And it has world-changing ambitions.

Pressing it will activate Bixby, Samsung's new voice assistant. Once activated, Bixby will help you navigate what's arguably the most sophisticated piece of technology you own — the smartphone in your hand.

If Samsung gets its wish, though, Bixby will eventually do much more than just help you order Lyfts or set up complex calendar appointments. The long-term vision is for Bixby to act as a kind of uber-interface for all of Samsung's products: TVs, wearables, washing machines, even remote controls.

Samsung designed Bixby with a specific goal in mind, one that veers away from its fellow voice assistants — Apple's Siri, Amazon's Alexa, Microsoft's Cortana and the Google Assistant. Those platforms were generally built to help users quickly perform common tasks ("Remind me to buy milk") and perform search queries ("What's the capital of Brazil?"). Bixby, on the other hand, is all about making the phone itself easier to use, replicating the functions of many apps with voice commands.

Yes, Siri et al. already do that to a certain extent — you can easily set a reminder with your voice, for example — but the voice integration typically only handles the basics. The goal of Bixby is to voice-enable every single action in an app that you'd normally do via touch, starting with Samsung's apps. So, not just "set a reminder to buy pickles at 6 p.m., but "Set a reminder on my Shopping List to buy pickles at 6 p.m. and make it repeat every week, then share the list with my wife."

Bixby speaks

Injong Rhee, CTO of Samsung Mobile and the architect behind Bixby, says the voice assistant is nothing short of an "interface revolution," freeing users from hunting down hidden functionality within menus and hard-to-find screens.

"Bixby is an intelligent user interface, emphasis... on 'interface,'" Rhee says. "A lot of agents are looking at being knowledgeable, meaning that you can ask questions like, 'Who's president of the U.S.?' A lot of these are glorified extensions of search. What we are doing with Bixby, and what Bixby is capable of doing, is developing a new interface to our devices."Although it makes its debut on the Galaxy S8, it will soon spread. Rhee sees the Bixby button eventually spreading to all kinds of smart-home devices, from TVs to refrigerators to air conditioners.

"Anywhere there is an internet connection and a microphone, Bixby can be used," he says. "There is some technology in the device, but a lot of it lives in the cloud. That's why the range of devices goes beyond just a smartphone. It means it can be in any device we produce."

Samsung began work on Bixby about 18 months ago, Rhee says. It grew out of the company's S Voice tool, which has been on Samsung phones since 2012. (The timing might explain why Samsung's smart fridge — announced right around then — failed to deliver on its planned integration with Alexa.) S Voice hadn't progressed much over the years, but then last year Samsung acquired the much-hyped Viv Labs and its sophisticated assistant, a strong indicator of the company's renewed interest in voice control. However, Rhee says Viv's technology is planned for future updates to Bixby and doesn't have a role in the initial release.

The name Bixby came out of Samsung's focus groups, but it was actually their third choice overall. It was the top pick among millennials — a demographic the company is specifically targeting with the Galaxy S8 — so it won out. (Rhee declined to say what the other names were.) It's also distinctive enough, with hard consonants, for it to work well as an activation word. Bixby, which will initially speak just English and Korean, is intended to be a user's "bright sidekick," helping them navigate their devices in a more natural way.

"[What came before], it's been people trying to learn how the machine interacts with the world, but... it should be the machine learns how the human interacts with the world," Rhee says. "The learning curve shouldn't be steep."

All talk, all action

For an app to be considered Bixby-supported, every possible touch action needs to be mapped to a voice command. Rhee explains that, for a typical app, there are about 300 different actions the user can perform. That doesn't sound too bad until you consider there are around 15,000 different ways to perform them. And the ways to verbalize those actions number in the millions. That's a lot of stuff to map out.

Still, Samsung says it's up for the challenge, at least as far as its built-in apps are concerned. But what about third-party apps? Considering the amount of development work, will Snapchat or Facebook ever work as well with Bixby as Samsung's apps?

Rhee says Samsung has a plan to get third-party apps talking to Bixby, and an SDK to be released at a later date will introduce tools that make the mapping much easier. He also suggests Viv's technology can help here, too.

"Viv Labs is coming in by way expanding our vision into third-party ecosystems. It doesn't necessarily have to be all of the touch commands that they can perform. At a minimum, [Bixby will perform] the basic functionalities: like the settings, or changing the language from English to French."

On the Galaxy S8, a total of 10 apps will be Bixby-supported, Rhee says, with a second "wave" coming a few weeks later. Out of the gate, users will be able to use Bixby with Contacts, Gallery, Settings, Camera, Reminders and a few others.

Another way Bixby is different from its peers: it will be aware of what you're doing on the phone and suggest different actions depending on what's on screen. So if you press the button while, say, looking at a single photo in the Gallery, editing and sharing controls are probably more relevant to you than searching. And if Bixby doesn't understand every aspect of a complex command, it will take you as far as it can rather than just hitting you with a "Sorry, I didn't catch that."

All this "awareness" brings up an important question: How much data is Samsung collecting about you? Rhee says most user-specific data is kept on the device, but, as a cloud service, Bixby needs to store some information in the cloud. It's not yet clear what the exact breakdown is.

The button

Having a dedicated button for Bixby brings a number of advantages. For starters, it means Samsung won't have any need for Clippy-style pop-ups directing users to the assistant — people will inevitably find it on their own. It also ensures there will be far fewer accidental activations than if Bixby were mixed into a home button — something users of Siri are all too familiar with.

"We actually have done a lot of research to have the Bixby button as part of the home button like our friends in Cupertino," Rhee says. "A lot of people find it a little awkward to use it in public. The home button is a very overloaded place — there's a lot of functionality into it. Having a dedicated button really removes a lot of friction."

And since the idea is to press and hold, lifting your finger when you're done, Bixby will know definitively when you're done speaking. Still, there will also be a wake-up phrase — you can just say "Hi Bixby," to activate the assistant at any time.

It's the dedicated button that really epitomizes Samsung's approach, and if it indeed ends up on all Samsung products, Bixby will become much more than just a smartphone assistant — it'll become the gateway for Samsung to finally, truly become a major player in the internet of things.

Sure, Samsung has had its "Smart" devices for a long time, and its low-power Tizen OS is ideal for powering the many products with connections to the internet. It also acquired SmartThings in 2014 to strengthen its IoT brand.

But until now, Samsung has lacked a gateway for its customers to really take advantage of that interconnectivity. For most, it's hard work hunting down the right settings on your phone to connect a smart TV to an air conditioner, but what if you could just tell Bixby to do it? And if you can talk to it from all those devices — asking any question or even making phone calls — then you're really onto something.

"It's actually omnipresent in a sense," Rhee says. "Even if I speak to Bixby in, say, a washing machine, you can still do a lot of things that you do on your phone. For instance, you can say, 'Bixby, send a text to my friend Michael,' or 'Make a phone call.' That's the vision."

The more capable assistant

Amazon and Google already know this, and the success of Alexa and buzz around Home are a testament to the unquestionable efficiency of adding voice control to devices. But Samsung, with its high standard of controlling all functions of a device via Bixby, might end up with the advantage. Alexa, for all of its "skills," often falls short of full control (you can turn on or dim LED lights, for example, but might not be able to select specific colors), so the market has room for a more capable competitor. Of course, how and when Bixby will mix with third-party products and services remains an open question.

"Philosophically, what we are looking at is revolutionizing phone interfaces," Rhee says. "We understand our applications better than anybody else out there — that's why we started with our own technology, but going forward we have plans to work with our partners."

Eventually, Rhee says a Bixby app might come to non-Samsung Android phones and even iOS, possibly partnering with Google Assistant for search-related queries (though he cautions Google and Samsung haven't "gotten to the specifics" on how that would work).

At the same time, Bixby control could extend to all kinds of smart products, not just Samsung ones. That would probably take a level of cooperation with competitors that Samsung hasn't really shown before, but if Bixby becomes ubiquitous in the long term, whatever OS this or that device is running will become less relevant.

That's a future Samsung is clearly hoping for, since software has traditionally been its weakness. Samsung may be a chief Android partner, but it's struggled to differentiate its many services from Google's, and the company lacks an OS of its own (Tizen notwithstanding). Samsung's browser, Samsung Pay, S Health — they're all duplicates of Google products, and are widely regarded as inferior.

That's why Bixby may be the best thing to happen to Samsung software in a long time. If customers respond, Bixby could, in the long term, finally get Samsung users to think of its phones as Samsung phones rather than just the best-performing Android phones on the market. All Android vendors try to differentiate to some extent, but Bixby's app-simplifying skills and potential IoT capabilities are a compelling sell.

Bixby represents an important step for Samsung when it comes to services: finally a good answer to "Why should I use your software?" Effortless voice control of everything — not just your phone — is a tantalizing promise, and if Samsung can pull it off in the long term, its "bright sidekick" might end up being the only assistant we actually want to talk to.

Writer: PETE PACHAL