loc translations, integrity test, token approx#147
Conversation
| @@ -3437,6 +3437,42 @@ | |||
| "event_type": "user_utterance", | |||
| "utterance": "Yes, please submit it." | |||
| }, | |||
| { | |||
There was a problem hiding this comment.
It might not matter a lot, but these tool calls are not in the right place in the expected trace right?
There was a problem hiding this comment.
yea those were hand filled, the trace is really only used to extract and apply write tools
There was a problem hiding this comment.
I'm just nervous people might use them for other purposes, so ideally, the tool calls would still be correctly placed.
| resolved["starting_utterance"] = _replace_in( | ||
| utt, first, last, first_rom, last_rom, phone, comp_first, comp_first_rom | ||
| ) | ||
| resolved_utt = _replace_in(utt, first, last, first_rom, last_rom, phone, comp_first, comp_first_rom) |
There was a problem hiding this comment.
This is not new - but it would be better to add * in the args of _replace_in, and to add keywords arguments here, since it would be easy to mess up the order here.
gabegma
left a comment
gabegma
left a comment
There was a problem hiding this comment.
Excellent work Raghav - thanks for taking the time to make this solution more robust and fail proof with a test!
2 participants
Force user goals to see translated location names so they don't say things in English
Database integrity test: constructs expected DB from initial DB and expected trace for every language. Everything from names, phone numbers, and location names are all translated per language during the replay
Threw in a small fallback for LLM and alm_vllm reasoning token count as well, and added compatibility for latest vllm versions