Natter

Brainstorming with your hands and your voice, demoed at AI Tinkerers London

Transcribed from my talk at AI Tinkerers London on 19 November 2026.

A dream I’ve had for a while is reimagining how we interact with our devices. I built a prototype to show a spark of what’s possible, as a way to brainstorm and ideate.

We’ve had little screens in our pocket for most of my adult life. Hunched over typing into a small box, like you see people on the train, is an outdated mode of interacting. I’m very excited for a new future where we interact with our whole bodies, like we do in real life.

Right now I’m talking to you, I’m moving around. A lot of that is lost when we’re just typing. Voice agents are great, I’m a big fan of those, but I think we can go much further.

What inspired this was one of my early startup experiences with my manager. We would spend half the day on a physical whiteboard, drawing ideas, and then go and build something and come back. It was so generative and fun, and we were in the room together rather than sitting alone at our desks. Most people I know still write handwritten notes, because you can’t capture that experience fully with a computer, but then you miss out on all the powerful things technology and LLMs can do.

So I’m trying to merge those two.

How it works

Fist on empty space, say an idea
Point at a bubble and pull away to branch
Delete mode: point at a bubble to delete it
Draw zones in the air

It’s me on the webcam, running in my browser. It detects my hands and my gestures, a palm and a bunch of different things it can tell I’m doing, and I’ve connected those with different actions. The hand tracking is MediaPipe, a Google model. It can do full body poses as well, and it’s running in the browser, so it’s really fast. Speech to text runs on my Mac with a small Parakeet model, and a cloud LLM cleans up the labels so it looks nice.

What I tried to do with this is be really low latency, because even waiting 10 seconds, my mind wanders and I lose the flow. So it uses small quick models, on device where I can. I think that is part of the user experience that makes it delightful to use rather than clunky.

On stage

Doing this kind of demo live is definitely a little bit risky. I have a mode where you give it an arbitrary instruction and it organises the nodes any way you want, but it didn’t fire on stage. It’s not fully working yet, but it’s super fun.

I also have a mode where two people can go on. Turns out detecting four hands at once and two microphones gets kind of complicated, and I didn’t have time to get that working.

What’s next

This is just the beginning. My next goal is to make it multiplayer. Maybe this is a virtual thing, and we’re all in our rooms ideating and throwing ideas, moving around. Maybe we have multiple cameras. I’m very excited to put some stuff in my own glasses, including bone conducting sound.
But I want to avoid becoming a meme like we saw when the Apple Vision Pro came out - people wearing it on the train looking ridiculous and isolated from their environment. So there’s some balance we’ve got to keep.

I would love to have my work day where I’m just walking around in space, free associating, and have really rich media generated on the fly.

The code is on GitHub: github.com/simonwisdom/natter.