1- option top_logprobs allows you not just to get the most likely token, but the top most likely tokens.
You can branch, by just chosing any point in your generated string and feed it back to the LLM, for example:
{
"user":"what is the colour of love?",
"assistant":"the colour of love is"}
It's true that it will add an "assistant" tag, wand old completions was better for this.
1- option top_logprobs allows you not just to get the most likely token, but the top most likely tokens.
You can branch, by just chosing any point in your generated string and feed it back to the LLM, for example: { "user":"what is the colour of love?", "assistant":"the colour of love is"}
It's true that it will add an "assistant" tag, wand old completions was better for this.