← ProjectsSound Polaroid
A camera that listens: turning the sound of a place into a printed souvenir.
Open the listening prototype- Role
- HCI designer & creative technologist — concept, firmware, backend, enclosure
- Timeline
- Final Major Project · 2026
- Team
- Independent project, UCA
A camera that listens
A place is not only remembered through images. It is remembered through voices, birds, traffic, footsteps, wind, water, echo and silence.
Sound Polaroid is a handheld device that photographs a place by listening to it. Press one button: it records ten seconds of ambient sound, works out what it thinks it heard, turns those detections into a readable prompt, generates an image from that prompt, and prints the result on thermal paper.
It is the physical form of Soundscene, my final major project — which asks what happens when a machine listens to a place, simplifies it, interprets it, and makes something visual from that listening.
The problem with the default souvenir
The usual way to keep a moment is to raise a phone, frame it, retake it and check the screen — and the result looks like everyone else's photograph of the same spot.
I wanted a capture device that records atmosphere rather than appearance. And I wanted to do it without quietly shipping people's private ambient audio to a black box that hands back a polished picture with no explanation.
The translation chain
Place → ambient sound → detected labels → authored visual mapping → prompt → generated image → print.
Every stage changes the original moment. Most systems hide that; this one shows it. A dense soundscape collapses into three words — traffic, speech, birds — and that collapse is the interesting part of the project, not the embarrassing part.
What the device says while it works
Nineteen illustrated 128×128 cards, drawn in Figma and compiled into the firmware, narrate the whole capture on the little panel: welcome, tell us where you are, I heard you are at…, please keep quiet, listening, taking picture, creating souvenir, printing, collect your souvenir.
Only the two things the artwork cannot know — the place that was heard, and how far along a timed step is — are drawn over it in code. A visitor who has never seen the object understands what it is doing without being told.
Ten seconds, already recorded
A XIAO ESP32-S3 holds a rolling ten-second buffer in PSRAM, so pressing the button captures the sound that just happened rather than starting a countdown. An external INMP441 microphone feeds it over I²S; a 1.44 inch ST7735 panel carries the flow.
The classifier models are far too large for the board, so the clip goes to a Vercel route backed by Modal containers running Google Perch 2.0 — trained on close to 15,000 species — with AST and PANNs behind it. Where a general classifier answers "Bird", Perch returns a name. Perch also warns its species scores are uncalibrated, so the print treats a name as a likely match, never as proof.
Designing the paper, not just the payload
The receipt is authored, not dumped. Header, place, date, a large generated image, a thin waveform strip beneath it, the honest detected labels, the duration, and the closing line: A PLACE, HEARD ONCE.
The generated scene is dithered to one-bit black and white for the 58 mm M02X and sent over BLE. If a value is missing — no coordinates, nothing confidently heard — the row is left off the paper rather than filled with something plausible.
Westminster Bridge, at night
The first end-to-end field test ran on Westminster Bridge: press, listen, upload, classify, generate, print — outdoors, on battery, with real traffic, real crowds and real river noise underneath it.
It printed. The paper came out with Big Ben on it, drawn from what the device heard rather than from anything it saw.
Decisions worth defending
Send the prompt, not the recording. The image model never receives microphone audio. It receives a sentence assembled from the detected labels, an authored visual mapping and the place — the same sentence the user can read.
Show every stage. Labels, mapping and prompt stay visible, because a system that hides its interpretation asks to be trusted rather than understood.
One button, and a spoken place. The trigger is a plain GPIO rather than the boot pin, so it can be held at power-on without changing how the board boots. The location is spoken aloud at the start of a capture and kept in flash as a fallback — there is no keyboard on a device like this.
Print only what was heard. No invented labels, no placeholder coordinates.
An honest prototype. Visible seams, screws, the paper path and the panel stay on show. A believable student prototype communicates more than a fake glossy render.
What I learned
The interesting design space was not accuracy but interpretation. A beautiful souvenir does not prove the model was right, and a correct label does not prove the souvenir meant anything.
Generative models drift towards the generic and the touristic. Prompt design is therefore a representational decision, not a technical one — it needs examining for bias rather than tuning for prettiness.
Joining fragile things punishes optimism: I²S capture, PSRAM, Wi-Fi, remote inference, BLE printing, battery and enclosure. Bringing each layer up alone, and writing down why, was the only way through.
Claims need evidence boundaries. A compiling sketch, a valid bitmap and a generated image each prove something narrow. The Westminster test is what proves the chain end to end — and it is one night, not a study.
Next: labelled-recording evaluation of the detection route, a participant study on whether the souvenir feels like the place, and user-editable sound-to-image mappings so the interpretation becomes a collaboration rather than a verdict.
A place, heard once.