dimamitya

← writing

from clubdeck to my own clubhouse client

claudima listens and speaks in live clubhouse rooms. an early version relied on clubdeck for its connection. moving to a client of my own meant taking ownership of everything around the model: joining a room, tracking participants, receiving audio, publishing speech, and leaving cleanly.

the starting point was a shipped desktop application. the result was a rust backend with its own session, event subscription, and native audio process. the useful test came when clubdeck was closed and i could still talk to claudima from a second account on my phone.

reconstruct the workflow

i inspected selected javascript bundles from clubdeck 2.6.5, using syntax trees and constant string transformations to recover the relevant behavior. the bundle and its decoder were not executed as part of that inspection. i kept hashes and source locations so each observation could be traced back to the version i had examined.

the interesting findings were sequences of operations. accepting an invitation involved a server request, updated connection credentials, a change of audio role, and a muted microphone. muting my own audio happened through the native sdk. treating every visible button as an independent http request would have missed those relationships.

the old client’s behavior gave me hypotheses to implement and test. server responses and live room state then determined which parts still worked. the new backend runs my implementation and does not load clubdeck’s files at runtime.

three connections share one room

the reconstructed client has three different jobs to coordinate:

connection responsibility
clubhouse http api account and room state, membership, and control requests
pubnub subscription room events and presence
agora rtc incoming participant audio and outgoing speech

the rust application owns the room lifecycle. a separate native helper runs the agora sdk. commands and audio are associated with the current room and connection generation, so a late callback from a retired connection cannot silently become work for its replacement.

those connections can disagree. an http request can succeed while the audio transport is unavailable. an audio connection can recover before the participant list is current. readiness has to account for the pieces an operation actually depends on.

keep the account attached to the audio

the earlier loopback route supplied a mixed room recording. for conversations, i needed the source account to travel with each utterance.

agora exposes participant audio before mixing. the input path associates each stream with its transport identity and the current room’s account mapping, then keeps separate buffers for speech detection and recognition. the resulting turn carries the account alongside the transcript.

that also gives interruption handling an earlier signal: another account can start speaking before a recognizer has produced any words. the application can cancel synthesis and playback while continuing to listen.

the attribution is to an account. someone can play a recording or share a microphone through that account. the live checks established attribution for separate participants; simultaneous overlapping speech still needs its own evaluation.

the sdk selected the wrong microphone

one live test exposed a problem that a successful join response concealed. after joining, the sdk selected the system microphone instead of the virtual device carrying claudima’s speech. the application’s audio-route check refused to unmute.

i changed the helper to restore the selected device before reporting readiness. a later check caught another device reset after joining. that required handling the change during the connection too: stop publication, restore the pinned device, and leave the microphone muted until an explicit unmute passes a fresh route check.

device selection became part of the connection contract. a room membership response alone could never establish that the intended audio was being sent.

close the loop with another device

the end-to-end checks used an existing account session and a second account on an iphone. incoming speech appeared under the expected account in claudima’s panel. an outgoing response passed through the model, local speech synthesis, the selected virtual device, and agora; i confirmed that response was audible on the phone with clubdeck closed. music playback was checked separately.

that established a working two-way conversation on the tested path. fresh phone sign-in and some invitation flows remained outside that result. connection recovery needed a separate set of experiments, which later found bugs of its own.

owning the client made those failures visible at useful boundaries: which room the application believed it was in, which account supplied an utterance, which device was publishing, and which connection still owed a cleanup operation. those became things i could inspect and test alongside the agent’s decisions.

continue with teaching a voice agent to stay quiet and testing recovery by breaking the connection.