OpenAI-compatible server

TranscribeCppSharp.Server exposes one loaded model over HTTP, with the routes OpenAI’s audio API uses, so a client that already speaks that API can point at a machine running transcribe.cpp.

It is not a re-implementation of OpenAI’s models and it is not a gateway: one process serves one model, the one given at startup.

dotnet run --project src/TranscribeCppSharp.Server -- --model whisper-tiny --allow-download

curl -s http://127.0.0.1:8080/v1/audio/transcriptions \
  -F file=@interview.mp3 \
  -F model=whisper-tiny \
  -F response_format=verbose_json

The first run with --allow-download fetches the alias from HuggingFace, verifies its sha256 and caches it — the same cache the transcribe CLI uses, so a model either front end fetched is available to the other.

Endpoints

Route What it does
POST /v1/audio/transcriptions multipart: file, model, language, response_format, timestamp_granularities[], diarize
POST /v1/audio/translations the same fields; runs the native translate task
GET /v1/models the single model this process serves
GET /health liveness, and whether the model is in memory

response_format accepts json (default), text, verbose_json, srt and vtt.

verbose_json returns task, language, duration, text, segments and words; a segment carries speaker only when attribution actually ran. The fields OpenAI returns that transcribe.cpp does not produce — avg_logprob, compression_ratio, no_speech_prob, token arrays — are absent, not zeroed. A client that requires them will see them missing, which is the truthful outcome.

srt and vtt return the merged segments as cues, from the same renderer the CLI uses (src/Shared/SubtitleWriter.cs); vtt is served as text/vtt and srt as application/x-subrip. Speaker labels follow the request’s diarize setting, and cues are the model’s own segment boundaries — nothing is re-timed to look shorter.

Word timestamps are returned only when the model reports it can produce them (Capabilities.MaxTimestampKind); otherwise words is an empty list rather than an error, because the transcript itself is still valid.

What it deliberately refuses

A request that asks for something this server does not do gets a 400 with a message naming the field, instead of a 200 that quietly ignored it:

  • prompt and a non-zero temperature — not wired to transcribe.cpp by this server;
  • any model other than the one this process loaded (404 model_not_found).

Speaker attribution: the model’s own, or a second pass

--diarize asks for speakers. How they are produced depends on --diarizer:

  • Without it, the ASR model is asked (DiarizeModeOn). A model that cannot attribute speakers reports every segment as speaker 0 and the native layer warns; nothing is invented on top.
  • With --diarizer <model>, the ASR run does not ask the model, and a DIARIZE-role model (Sortformer, for example) runs over the whole file after the transcript. Its turns are joined to the segments by SpeakerAssignment — the same code the desktop app uses: at word level when the transcript has word timestamps (each word takes the turn it overlaps most, and the segment takes the majority speaker), with the largest-overlap rule as the fallback. Explicit configuration wins over the model’s own capability, which is what makes the result predictable.

Two costs to know before turning it on:

  • Both models stay loaded. The server caches them, so the second request does not pay the load again, but the process now holds two models’ weights.
  • The diarizer reads the whole file into memory. Upstream has no streaming diarization and speaker ids are per run, so the file cannot be windowed without renumbering speakers at every seam. An hour of 16 kHz mono audio is 219 MiB in that pass, on top of the models.

It is off by default, and a request with diarize=false never runs the second pass. GET /v1/models still lists only the ASR model: the diarizer is a pipeline stage, not a second model a client can choose.

Measured while wiring it: on the 11 s test-audio/jfk.wav with ggml-tiny.bin and Sortformer, the diarizer returned 4 turns and the single segment was assigned speaker 1. That clip has one speaker and is far too short to say anything about diarization quality — what the run verifies is the plumbing, not the attribution.

Concurrency, and why requests queue

transcribe.cpp allows one run per model at a time, across all of that model’s sessions. The server therefore serializes every request behind one semaphore: a second request waits for the first instead of racing it. A long file blocks the queue for as long as it runs. Raising that ceiling means one process per model on different ports — not a second thread in the same process.

Because every waiting request is an upload staged on disk, the queue is bounded: --max-pending (default 4) is how many requests may be in flight at once, and the next one gets 429 too_many_requests with Retry-After: 5 before its body is read. The check happens first on purpose: a client that ignores the 429 must not be able to fill the temp directory one gigabyte at a time.

Long audio, memory, and the window

Above --chunk seconds (default 300, and never above the model’s own limit) the audio is split with a 1 s overlap, one window in memory at a time. The seam is deduplicated with a 500 ms tolerance: a segment that starts slightly before the previous one ended is kept rather than dropped, because dropping it would lose the audio after the seam. The cost is that up to 500 ms of audio can be transcribed twice. Measured on test-audio/jfk.wav at --chunk 5, the merged text read ... ask not! not what ... — the same result the CLI produces, because it is the same TranscriptMerger. The tolerance is a tradeoff, not a measurement.

Two things to know before running a long file on a small GPU:

  • If a window does not fit in memory, the native layer returns its out-of-memory status and the server halves the window and restarts the file, down to 5 s, for at most ten values. Each retry is logged to stderr. The failed attempt’s partial transcript is discarded, so the result cannot contain duplicates; the cost is re-transcribing the windows that had already succeeded. The policy is WindowAdjustment, shared with the desktop app.
  • Only that out-of-memory status is retried. Any other native failure is returned to the client as a 500; the server never falls back to another backend or device.

Access and safety

  • It binds http://127.0.0.1:8080 by default. --urls changes that; binding a public interface is your decision.
  • Without --api-key (or TRANSCRIBE_SERVER_API_KEY), anyone who can reach the port can transcribe. With a key, every route but /health requires Authorization: Bearer <key>. The comparison is fixed-time; this is not a full auth system, and --urls does not add TLS. /health stays open so a supervisor can tell “not up yet” from “up but refusing”; it echoes the model argument, which is a path when --model was given one.
  • Cross-site requests are refused (403 cross_site_request). A multipart POST is a CORS “simple request”, so any page in the user’s browser could send one to the loopback address without a preflight and have the server transcribe the uploaded blob; the response is unreadable to that page, but the work is done. A request carrying an Origin that is not this server, or Sec-Fetch-Site: cross-site, is refused before the body is read. Direct clients send neither header and are unaffected.
  • DNS rebinding is only closed with --allowed-host. After a name resolves to 127.0.0.1, the browser sends Origin and Host as that same name, so the origin check above cannot see it. Passing --allowed-host <name> (repeatable, or ;-separated) makes any other Host get 403 host_not_allowed. It is off by default because a reverse proxy legitimately forwards its own name; behind one, either pass the public name here or do the check in the proxy.
  • Downloads are off by default: a request cannot make the server fetch something from the network unless the process was started with --allow-download. A path to an existing GGUF always works.
  • Uploads are capped at --max-upload-mb (default 1024), and an upload over the cap is refused with 413 payload_too_large.
  • Nothing audio-related is kept after the request. The upload is copied to a temporary file created 0600 on Unix (so no other local user can read it), and that file is deleted in a finally. Audio that is not a 16 kHz mono 16-bit WAV is decoded by ffmpeg into a second temporary file, also 0600, opened and unlinked on Unix: once decoding is done the name is gone even if the process is killed. Neither file survives the request.
  • Nothing is logged but request lines and errors; no transcript is written to disk by the server. Native and I/O failures answer a generic 500 and put the detail (paths included) on stderr, not in the response.

What it is not

  • Not a streaming endpoint. There is no SSE and no chunked transcript. The native streaming API exists but the library documents its limits; the server does not expose it.
  • Not a model router. One process, one ASR model (plus an optional diarizer). GET /v1/models lists that one.
  • Not packaged. There is no NuGet package and no release archive for it yet; run it from a clone. It needs the native runtime, which tools/FetchNative places in the output directory, and ffmpeg on PATH for any input that is not a 16 kHz mono 16-bit WAV.
This site uses Just the Docs, a documentation theme for Jekyll.