does not work: matches = transcript.word_search(["price", "product"])

[Faizan-Saifullah]

[marcusolsson]

Hi @Faizan-Saifullah! 👋 Could you describe the problem you're seeing? Are you getting an exception, or is the output not what you expected?

[Faizan-Saifullah]

Actually, I am trying to run a script where i do Speech2Txt using AssemblyAI and then i send the data to gpt which processes the answer. from there, i want to out back speech (TTS) using elevenlabs api. But the issue is that the transcriber does not stop after a pause in speech. therefore, I am trying to track a key word which when comes, the transcriber can stop. i need help in that . below is the code:

import assemblyai as aai
import elevenlabs
import openai as openai
from elevenlabs import ElevenLabs
from queue import Queue

client = openai.Client(api_key="")

###api keys slot:
aai.settings.api_key =""

OpenAI.api_key = ""

client = ElevenLabs(api_key = "")

transcript_queue = Queue()

def on_open(session_opened: aai.RealtimeSessionOpened):
print("Session ID:", session_opened.session_id)

def on_data(transcript: aai.RealtimeTranscript):
if not transcript.text:
return
if isinstance(transcript, aai.RealtimeFinalTranscript):
transcript_queue.put(transcript.text + " ")
print("User: ", transcript.text, end="\r\n")
else:
print(transcript.text, end="\r")

def on_error(error: aai.RealtimeError):
print("Error: ", error)

def on_close():
print("Closing Session")

def handle_conversation():
while True:
transcriber = aai.RealtimeTranscriber(on_data=on_data, on_error=on_error, sample_rate=44_100, on_open=on_open, on_close=on_close)
###starting connection
transcriber.connect()
###opening microphone stream
microphone_stream = aai.extras.MicrophoneStream()
###streaming audio from microphone
transcriber.stream(microphone_stream)
###close current transcription session with CTRL+C
transcriber.close()

    ###retrieving data from queue
    transcript_result = transcript_queue.get()
    print("Transcript: ", transcript_result)
    transcripted = transcript_result

    # completion = client.chat.completions.create(model="gpt-4o",messages=[{"role": "developer", "content": "You are a helpful assistant."},{"role": "user","content": "What is 2+2?"}])
    # print(completion.choices[0].message.content)

    ###sending transcript to openai to generate a response
    response = client.chat.completions.create(model = 'gpt-4o', messages = [{"role": "developer", "content": "You are a highly skilled AI, answer the questions given within a maximum of 1000 characters."}, {"role": "user", "content": transcripted}])
    text = response.choices[0].message.content
    print("response: ", text)
    


    # ###convert the response to audio and play it
    # audio = client.text_to_speech.convert_as_stream(text = text, voice_id="JBFqnCBsd6RMkjVDRZzb", output_format="mp3_44100_128",model_id="eleven_multilingual_v2")
    # # audio = voice_generation(text = text, voice = "en-US-Wavenet-D", speed = 1.0, pitch = 0.0, volume = 1.0)
    # print("\nAI: ", text, end="\r\n")
    # elevenlabs.play(audio)

handle_conversation()

[ccampbell-aai]

Hi @Faizan-Saifullah, thanks for the follow-up and sorry this thread went quiet. Looks like we're dealing with two separate things here.

On the title word_search is a prerecorded feature. It runs against a completed transcript and calls the word-search endpoint for the individual words or phrases. It has no role in a live session, so it can't act as a trigger for streaming audio.

On what you're trying to do in your code snippet You want the live transcriber to stop after the user finishes speaking so you can hand the text to GPT and then to TTS. The current streaming client gives you that directly, no keyword needed. Every Turn event carries end_of_turn. When it's True, the model has decided the speaker paused, which is the boundary you're looking for.

If you're looking to update the above script, please refer to the README.md's streaming section and MIGRATION.md to cover how to upgrade to the latest 1.1.0.

Closing this issue, if you continue to experience issues, please open a new issue with the SDK version and what you observed.