5. Touch and tablets#
In this chapter you pair a tablet or a phone with your session host. Then you work with the pie menu and the mpmux keyboard.
Pair a device#
The tablet connects to the session host from chapter 3. Pairing gives the tablet its own key, so you never type a password or an address on glass.
On the computer that runs the session host, keep mpmux open with the admin session from the quickstart (section 3), and create a one-time QR code:
mpmux-cli pair qr-code --device-label "Android tablet" \
--output /tmp/mpmux-pair.png
In the app, open the session host setup, choose QR pairing, and scan the image. The code works once and then expires.
No camera at hand? Use a short code instead:
mpmux-cli pair short-code --device-label "Android tablet" \
--code-output-file /tmp/mpmux-pair.code
Enter the code in the app's short-code pairing, then delete the code file.
Pairing over the network needs the host set up for it once. The quickstart has the steps (sections 3 and 4).
Gestures#
| Action | Gesture |
|---|---|
| Open the pie menu | Touch and hold a pane |
| Open the command palette | Swipe with two fingers |
| Open the overview | Swipe up with three fingers |
| Close the overview | Swipe down with three fingers |
The pie menu#
The pie menu puts the common actions in a ring under your thumb. Hold, then slide toward an entry and lift. To close the menu without a choice, lift in the middle.
| Direction | Entry | What it does |
|---|---|---|
| Up | Panes | Split right, split down, maximize (Layout), new tab, Overview (down right) |
| Up right | Todos | Show the agent's task list |
| Up left | Terminal | Interrupt (Ctrl+C), Clear, and Reset (see below) |
| Right | Connect | Manage hosts and sessions |
| Down right | Voice | Dictate into the active pane |
| Down | Keyboard | Select, copy, paste, and keyboard settings |
| Down left | Prompts | Send a ready-made prompt to an agent (chapter 6) |
| Left | More | The command palette with every action |
On a desktop, press Ctrl+Alt+X (macOS: Cmd+Alt+X) to open the same menu. There, the entry at the bottom is Layout (maximize and restore).
When the terminal is in a bad state#
A program that crashed, or cat of a binary file, can leave the terminal
garbled: odd characters, no colors, a mouse that selects nothing. Under
Terminal in the pie menu, as in Ghostty:
- Clear clears the screen and the scrollback. The shell then draws its prompt again, with what you were typing.
- Reset resets the terminal itself: modes, colors, character sets, the mouse and keyboard protocols. Use it when Clear is not enough.
Both are also in the command palette (Clear Terminal, Reset Terminal). In a hosted session they reach the session host, so the next device that opens the session sees the clean terminal too.
The mpmux keyboard#
A shell needs keys that phone keyboards hide. The mpmux keyboard has Esc, Tab, Ctrl, Alt, and the arrow keys. Ctrl and Alt stay pressed for the next key, so Ctrl then C sends Ctrl+C.
Under Keyboard in the pie menu you can:
- choose mpmux for the terminal keyboard, or Native for the system keyboard (for example to use its word suggestions),
- choose Hide to see the whole screen,
- change the keyboard height under Height.
A hardware keyboard works as on a desktop. Ctrl+C, Ctrl+D, and the other control keys go to the shell.
Voice input#
Tap the microphone key on the mpmux keyboard, pick Voice in the pie menu, or run Voice Input from the command palette, and speak. The waveform shows that the microphone hears you and the words appear as you talk. Tap the button again, or press Space, and the text goes to the active pane, so you can dictate a prompt to an agent. Esc or Cancel throws the recording away.
Speech is transcribed on the device by Whistle, a 17 MB model for English, German, French, Spanish, Italian, Dutch and Polish. Nothing leaves the device. Whistle runs on Android, Linux (not in the .deb, whose Ubuntu 22.04 base is too old for it), and macOS on Apple silicon. On Android without Whistle, the system speech recognizer takes over. You can add Parakeet for the finished text, on Linux and on Android: run Download Parakeet Voice Model from the command palette (633 MB, once; use Wi-Fi on a phone). It is much more accurate than Whistle, in 25 European languages. Whistle still shows the words while you speak; Parakeet writes the text you insert. It loads when you start recording and leaves memory (about 1 GB) after ten minutes without use. On the desktop, a Parakeet model that the Handy app already downloaded is used as is.
Whistle favours the words on the active pane, such as file names and
commands, so a prompt about what you see comes out right. Pauses before and
after you speak are cut before the final pass. Set the language, engine and
words to favour in [voice].
Next#
Chapter 6 shows how to run coding agents and steer them from any device.