OpenHermes 2.5 Mistral 7B
teknium/openhermes-2.5-mistral-7b
Created Nov 204,096 context
$0.17/M input tokens$0.17/M output tokens
A continuation of OpenHermes 2 model, trained on additional code datasets. Potentially the most interesting finding from training on a good ratio (est. of around 7-14% of the total dataset) of code instruction was that it has boosted several non-code benchmarks, including TruthfulQA, AGIEval, and GPT4All suite. It did however reduce BigBench benchmark score, but the net gain overall is significant.