North (Acquired by Google)

Contextual Computing

A phone spends most of the day in a pocket and learns about a situation only when someone takes it out and provides input. Glasses are different: they are always worn, oriented with the wearer, and exposed to what is happening around them.

The opportunity was to make that context available across Focals. Instead of every experience beginning with a menu or explicit command, a shared engine could interpret device signals, infer the wearer’s situation, and surface what was most relevant in that moment.

The engine sat between the device’s sensors and its applications, translating signals such as time, movement, and ambient sound into reusable context. Each application became a design probe: it tested whether an inference was useful and helped define the sensors, models, and logging capabilities the platform needed next.

QuickCompose Case Study

QuickCompose was one of the first applications built on the engine. Messaging on Focals was receive-only: users could read a message, but had to take out their phone to respond—undermining the promise of glasses designed to keep them present.

Composing text with a ring was impractical, so we reframed replying as selection. QuickCompose presented a short list of phrases that users could choose with a flick and a click. If the right response appeared among the first few options, the interaction worked; if it did not, the user returned to their phone. Context turned the experience into a ranking problem.

I analyzed messaging logs and facilitated concept workshops to understand the routine messages people sent throughout a typical day. The most common intents grouped into six categories: commute, general greetings, meetings, everyday routines, holidays, and location-based messages.

The categories defined what to suggest; context determined when it mattered. A commute response might be useful around 8 a.m. or 5:30 p.m., but irrelevant at 2 p.m. That pattern could be learned rather than encoded as a collection of fixed rules.

The first model scored each phrase against time of day using a Gaussian distribution. Every response had a relevance that rose and fell throughout the day, naturally moving evening commute and routine messages to the top around 5:30 p.m.

I built the model into a minimum viable prototype and ran an internal study. It validated the interaction but exposed a content problem: users quickly exhausted the narrow message set and saw the same suggestions repeatedly.

The fix was structural. We broadened the categories and added category relevance as a second ranking signal:

Final score = phrase relevance × category relevance

If “On my way home” scored 1.0 for the current time and the commute category scored 0.5, its final ranking became 0.5.

To reduce repetition, we created three to four variations for each intent. I used an Amazon Mechanical Turk survey to learn how people naturally expressed those messages instead of writing the language ourselves and assuming it would sound authentic.

Outcome & Impact

QuickCompose proved the experience, but the broader outcome was the contextual-computing platform it helped create:

• QuickCompose reached production. A second internal study and beta feedback refined the experience before launch. Focals was no longer receive-only for messaging: users could respond without returning to their phones.
• The engine became reusable infrastructure. It gained on-device ML through TensorFlow Lite, always-on motion detection, a step counter, an analytics logger and dashboard, and Bayesian rules that could adapt to an individual rather than only a population.
• AutoVolume reused the same foundation. It adjusted playback volume using ambient-noise context instead of requiring manual control.
• AutoDND extended the model further. It inferred when to enter do-not-disturb mode from the wearer’s situation rather than relying on a toggle.

QuickCompose proved one interaction. The lasting outcome was an engine that made subsequent context-aware experiences possible without building a new sensing stack each time.