Speech client v2, my, working OK, example

Hey guys, I am not sure where to put it, but I just want to share my implementation of the speech client v2 and some thoughts about migrating from v1 to v2.
Official docs are unfortunately provide only python code, and there is not much info except this repo and the example I used to migrate: transcribeStreaming.v2.js

My sdk verison is:

"@google-cloud/speech": "^6.0.1",

In my code I first initialize the service as:

const service = createGoogleService({ language, send })

and then use service.transcribeAudio(data) whenever there is a new audio coming from the frontend, which uses

const mediaRecorder = new MediaRecorder(audioStream, { mimeType: 'audio/webm;codecs=opus' }) // its a default param;
mediaRecorder.ondataavailable = (event: BlobEvent) => {
... send the event.data to the backend
}

thus an audio chunk is just a browser Blob object.

My service:

import { logger } from '../..//logger';
import { getText, transformGoogleResponse } from './utils';
import { v2 as speech } from '@google-cloud/speech';
import { StreamingRecognizeResponse } from './google.types';
import { TranscriptionService } from '../transcription.types';
import { MachineEvent } from '../../websocket/websocket.types';
import { Sender } from 'xstate';
import { parseErrorMessage } from '../../../utils';
import { findRecognizerByLanguageCode } from './recognizers';

export const createGoogleService = ({
  language,
  send,
}: {
  language: string;
  send: Sender<MachineEvent>;
}): Promise<TranscriptionService> => {
  return new Promise((resolve, reject) => {
    try {
      const client = new speech.SpeechClient({
        keyFilename: 'assistant-demo.json',
      });

      const recognizer = findRecognizerByLanguageCode(language).name;

      const configRequest = {
        recognizer,
        streamingConfig: {
          config: {
            autoDecodingConfig: {},
          },
          streamingFeatures: {
            enableVoiceActivityEvents: true,
            interimResults: false,
          },
        },
      };

      logger.info('Creating Google service with recogniser:', recognizer);

      const recognizeStream = client
        ._streamingRecognize()
        .on('error', error => {
          logger.error('Error on "error" in recognizeStream', error);
          send({ type: 'ERROR', data: parseErrorMessage(error) });
        })
        .on('data', (data: StreamingRecognizeResponse) => {
          if (data.speechEventType === 'SPEECH_ACTIVITY_END') {
            send({ type: 'SPEECH_END', data: 'SPEECH_END' });
          }
          if (data.results.length > 0) {
            const transcription = transformGoogleResponse(data);
            if (transcription) {
              const transcriptionText = getText(transcription);
              if (!transcriptionText?.length) {
                // if the transcription is empty, do nothing
                return;
              }
              send({ type: 'NEW_TRANSCRIPTION', data: transcriptionText });
            }
          }
        })
        .on('end', () => {
          logger.warn('Google recognizeStream ended');
        });

      let configSent = false;

      const transcribeAudio = (audioData: Buffer) => {
        if (!configSent) {
          recognizeStream.write(configRequest);
          configSent = true;
        }
        recognizeStream.write({ audio: audioData });
      };

      const stop = () => {
        if (recognizeStream) {
          recognizeStream.end();
        }
      };

      resolve({ stop, transcribeAudio });
    } catch (error) {
      logger.error('Error creating Google service:', error);
      reject(error);
    }
  });
};

Migration considerations

to use v2 you need to create a recognizer, I did it with this function:

/**
 * Creates a new recognizer.
 *
 * @param {string} projectId - The ID of the Google Cloud project.
 * @param {string} location - The location for the recognizer.
 * @param {string} recognizerId - The ID for the new recognizer.
 * @param {string} languageCode - The language code for the recognizer.
 * @returns {Promise<object>} The created recognizer.
 * @throws Will throw an error if the recognizer creation fails.
 */
export const createRecognizer = async (
  projectId: string,
  location: string,
  recognizerId: string,
  languageCode: string
) => {
  const client = new v2.SpeechClient({
    keyFilename: 'assistant-demo.json',
  });

  const request = {
    parent: `projects/${projectId}/locations/${location}`,
    recognizer: {
      languageCodes: [languageCode],
      model: 'latest_long',
      // Add any additional configuration here
    },
    recognizerId,
  };

  try {
    console.log('Creating recognizer...', request);
    const [operation] = await client.createRecognizer(request);
    const [recognizer] = await operation.promise();
    return recognizer;
  } catch (error) {
    console.error('Failed to create recognizer:', error);
    throw error;
  }
};

The config object now should be sent as first data to the stream object, immediately before the audio, so if you did recognizingClient.write(audioData) before, now you should do (but only once!)recognizingClient.write(newConfigWithRecognizer) and then recognizingClient.write({ audio: audioData }) <<< notice the object notation
The config object itself has been changed to:

public streamingConfig?: (google.cloud.speech.v2.IStreamingRecognitionConfig|null);

/** Properties of a StreamingRecognitionConfig. */
interface IStreamingRecognitionConfig {

** StreamingRecognitionConfig config */
config?: (google.cloud.speech.v2.IRecognitionConfig|null);

/** StreamingRecognitionConfig configMask */
configMask?: (google.protobuf.IFieldMask|null);

/** StreamingRecognitionConfig streamingFeatures */
streamingFeatures?: (google.cloud.speech.v2.IStreamingRecognitionFeatures|null);
}

When instantiating streamingClient use _streamingRecognize() (this probably is likely to be changed)

@sorokinvj

Does this still work for you?

I created my recognizer in cloud console and am just specifying it as a string. Im however getting the error "Error: 3 INVALID_ARGUMENT: Malordered Data Received. Expected audio but none was set. Send exactly one config, followed by audio data.".

I really cant figure out what im doing wrong...

Edit: @sorokinvj I fixed it so the audio data can send but get back no response from STT. The stream ends up timing out. I have interimResults to true.

@adambeer

Yeah, all of this is because you are somehow sending wrong audio data, if you share your code I might get more insight. @adambeer sorry, I saw it just now. My code runs production server, no problem so far

I get code: 7,
details: 'The caller does not have permission',
metadata: Metadata { internalRepr: Map(0) {}, options: {} }

when I execute recognizeStream.write(streamingRecognizeRequest);

This my code:

const config: IRecognitionConfig = {
languageCodes: ['de-DE'],
model: 'chirp',
autoDecodingConfig: {},
};

const streamingRecognitionConfig: google.cloud.speech.v2.IStreamingRecognitionConfig =
{
config: config,
streamingFeatures: {
interimResults: true,
},
};

const streamingRecognizeRequest: google.cloud.speech.v2.IStreamingRecognizeRequest =
{
recognizer: projects/${process.env.FIREBASE_PROJECT_ID}/locations/asia-southeast1/recognizers/_,
streamingConfig: streamingRecognitionConfig,
};

While a non-streaming request works well:
const recognitionOutputConfig: IRecognitionOutputConfig = { gcsOutputConfig: { uri:${STAGING_BUCKET_URL}/transcriptions/`,
},
};

const request: RecognizeBatchRequest = {
processingStrategy: 'DYNAMIC_BATCHING',
toJSON(): { [p: string]: any } {
return {};
},
recognitionOutputConfig,
recognizer: projects/${process.env.FIREBASE_PROJECT_ID}/locations/asia-southeast1/recognizers/_,
config: config,
// content: buffer,
files: [
{
uri: ${STAGING_BUCKET_URL}/audio_recordings/${audioRecordingId},
},
],
};
`

Any idea why? the service account i use has cloud speech admin permissions.
Is it maybe because of my location asia-southeast1?
When I change the model to "short" or "long" I get 'The language "auto" is not supported by the model "short" in the location named "asia-southeast1".',
Happens for any other language.

PS: sorry for the non-code formatting.

I am getting Error: 3 INVALID_ARGUMENT: Invalid resource field value in the request.
I cant track it down. Can you please help. Here are relevant parts of code:

import { v2 as speech } from '@google-cloud/speech';

const speechClient = new speech.SpeechClient();

const request = {
recognizer:'projects/redacted/locations/us-central1/recognizers/redacted',
streamingConfig: {
config: {
languageCode: 'en-US',
},
streamingFeatures: {
enableVoiceActivityEvents: true,
interimResults: false,
},
},
};

const recognizeStream = speechClient.streamingRecognize(request)

NOTE: I am on node 18.

I am getting Error: 3 INVALID_ARGUMENT: Invalid resource field value in the request. I cant track it down. Can you please help. Here are relevant parts of code:

import { v2 as speech } from '@google-cloud/speech';

const speechClient = new speech.SpeechClient();

const request = { recognizer:'projects/redacted/locations/us-central1/recognizers/redacted', streamingConfig: { config: { languageCode: 'en-US', }, streamingFeatures: { enableVoiceActivityEvents: true, interimResults: false, }, }, };

const recognizeStream = speechClient.streamingRecognize(request)

NOTE: I am on node 18.

I don't know, man, sorry. Your code is doing something else than mine, but to understand your usecase I would need to see more than what you've shared.

You are using .streamingRecognize vs. _streamingRecognize in my code. I am not sure I understand what is your function doing.
Moreover, your setup looks like you want to instantiate a stream calling streamingRecognize once? For streaming you want to write continously some chunks of data, which you are missing.

const config: IRecognitionConfig = { languageCodes: ['de-DE'], model: 'chirp', autoDecodingConfig: {}, };

Can you try to not use any model and leave this field undefined?

const streamingRecognizeRequest: google.cloud.speech.v2.IStreamingRecognizeRequest = { recognizer: projects/${process.env.FIREBASE_PROJECT_ID}/locations/asia-southeast1/recognizers/_, streamingConfig: streamingRecognitionConfig, };

Ensure FIREBASE_PROJECT_ID is set

recognizeStream.write(streamingRecognizeRequest)

It might be useful to have a basic check in the code, that you send this request only once in the first packet, though I doubt it has anything to do with permission issue. Sorry, have no idea why you might stuck with this.

Thank you for your time. I got the config set correctly. I was also able to get a stream from a local file to transcribe correctly. However, I am getting no response on this.
const parsedMessage = JSON.parse(decodedMessage);

  if (parsedMessage.event === "media" && parsedMessage.media) {
    
    const decodedPayload = Buffer.from(parsedMessage.media.payload, 'base64');
    recognizeStream.write({audio:decodedPayload});

This is a base64 encoded payload coming from a websocket message. The config sends properly but there is clear some sort of problem with the buffer and audio data. I verified the incoming data and that the data is sent to _streamingRecognize. These lines worked in v1 by the way.

No error codes. Just no response then timeout after cancelling the project.

UPDATE: I just realized the other is only giving a response on stream end. I think this is my problem. this needs to transcribe voice data via a websocket.

@sorokinvj

@sorokinvj Im calling it from a google function onCall with the code below and Im receiving Unhandled error Error: 3 INVALID_ARGUMENT: Invalid resource field value in the request.

I tried without the model, but still the same problem

const client = new speech.v2.SpeechClient();

    const config: speech.protos.google.cloud.speech.v2.IRecognitionConfig = {
      languageCodes: ['en-US'],
      features: {
        "profanityFilter": true,
        "enableAutomaticPunctuation": true,
        "enableSpokenEmojis": true,
      },
      autoDecodingConfig: {},
    };
    const speechRequest: speech.protos.google.cloud.speech.v2.IRecognizeRequest = {
      content: request.data.audio,
      config: config,
    };
  
    const [response] = await client.recognize(speechRequest);'

UPDATE
I could make it work with:

const client = new speech.v2.SpeechClient({
        apiEndpoint: `us-central1-speech.googleapis.com`,
    });

    const config: speech.protos.google.cloud.speech.v2.IRecognitionConfig = {
        languageCodes: ["en-US"],
        model: "long",
        features: {
            profanityFilter: true,
            enableAutomaticPunctuation: true,
            enableSpokenEmojis: true,
        },
        autoDecodingConfig: {},
    };

const speechRequest: speech.protos.google.cloud.speech.v2.IRecognizeRequest = {
        recognizer: "projects/{{FIREBASE_PROJECT_ID}}locations/us-central1/recognizers/_",
        content: request.data.audio,
        config: config,
    };

I can get transcriptions but the problem I am having is that transcriptions only come back after the stream has ended

I know the models offer mulaw encoding but everytime i set mulaw with a recognizer in console it flips back to linear16. Anyone able to help?

// Use 'import' instead of 'require' for ES Modules
import { v2 } from '@google-cloud/speech';
import { Storage } from '@google-cloud/storage';
import path from 'path';

// --- Configuration ---
// Your recognizer's location and ID
const location = 'europe-west4';
const recognizerId = 'recognizer';

// The local audio file you want to transcribe
const localAudioFile = '/Users/tobiasanhalt/Desktop/test.ogg';
// --- End of Configuration ---

async function main() {
  const clientOptions = {
    apiEndpoint: `${location}-speech.googleapis.com`,
  };

  const speechClient = new v2.SpeechClient(clientOptions);
  const storage = new Storage();

  const projectId = await storage.getProjectId();
  console.log(`✅ Using project: ${projectId}`);

  const bucketName = `${projectId}-speech-transcripts`;
  const bucket = storage.bucket(bucketName);

  const [bucketExists] = await bucket.exists();
  if (!bucketExists) {
    console.log(`Bucket '${bucketName}' not found. Creating it now in '${location}'...`);
    await storage.createBucket(bucketName, {
      location: location,
    });
    console.log(`✅ Bucket created.`);
  } else {
    console.log(`✅ Bucket '${bucketName}' already exists.`);
  }

  const audioFilename = path.basename(localAudioFile);
  const gcsInputUri = `gs://${bucketName}/${audioFilename}`;
  const gcsOutputUri = `gs://${bucketName}/transcripts/`;

  console.log(`Uploading '${localAudioFile}' to '${gcsInputUri}'...`);
  await bucket.upload(localAudioFile, {
    destination: audioFilename,
  });
  console.log(`✅ File uploaded.`);

  console.log(`Starting synchronous batch transcription job...`);
  console.log(`(This may take a moment...)`);

  const recognizerName = speechClient.recognizerPath(
    projectId,
    location,
    recognizerId
  );

  const request = {
    recognizer: recognizerName,
    files: [{
      uri: gcsInputUri
    }],
    recognitionConfig: {
      autoDecodingConfig: {},
    },
    recognitionOutputConfig: {
      gcsOutputConfig: {
        uri: gcsOutputUri,
      },
    },
  };

  const [operation] = await speechClient.batchRecognize(request);
  const [response] = await operation.promise();

  console.log(`✅ Job completed.`);

  // --- NEW CODE STARTS HERE ---
  // The response contains the GCS URI of the transcript. Let's download it.
  
  // 1. Get the output URI from the response metadata.
  // We'll iterate through the results in case there were multiple files.
  for (const inputFileUri in response.results) {
    const result = response.results[inputFileUri];
    if (result.error) {
      console.error(`Error processing ${inputFileUri}:`, result.error);
      continue;
    }
    
    const transcriptUri = result.uri;
    console.log(`\n📄 Transcript for ${inputFileUri} saved to: ${transcriptUri}`);

    // 2. Parse the GCS URI to get the bucket name and file name.
    const transcriptBucketName = transcriptUri.split('/')[2];
    const transcriptFileName = transcriptUri.substring(transcriptUri.indexOf('/', 5) + 1);

    // 3. Download the result file from GCS.
    console.log(`Downloading transcript from GCS...`);
    const transcriptBucket = storage.bucket(transcriptBucketName);
    const [transcriptContentBuffer] = await transcriptBucket.file(transcriptFileName).download();

    // 4. Parse and print the actual transcript.
    const transcriptJson = JSON.parse(transcriptContentBuffer.toString());

    console.log("---");
    console.log("TRANSCRIPT CONTENT:");
    console.log(JSON.stringify(transcriptJson, null, 2));
    console.log("---");
  }
  // --- NEW CODE ENDS HERE ---
}

main().catch(console.error);

I got this to work. It might help you

It took me one day to set up google voice transcription.

What should be a 15 min job turned out to be a day of effort.

Can anyone point me to official documentation for doing streaming with the Google V2 API for Node.js? And if not, explain why we still have to use this workaround?

Navigation Menu

Repository navigation

Speech client v2, my, working OK, example #3578

My service:

Migration considerations

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Speech client v2, my, working OK, example #3578

Description

My service:

Migration considerations

Activity

adambeer commented on Jan 15, 2024

sorokinvj commented on Mar 23, 2024

MilanHofmann commented on Mar 29, 2024

carstarai commented on Apr 30, 2024

sorokinvj commented on May 1, 2024

sorokinvj commented on May 1, 2024

carstarai commented on May 2, 2024

rodrifmed commented on May 2, 2024

rodrifmed commented on May 3, 2024

carstarai commented on May 3, 2024

carstarai commented on May 6, 2024

Mythli commented on Sep 30, 2025

jkelvie commented on Mar 27, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions