OpenHermes 2.5 Mistral 7B

teknium/openhermes-2.5-mistral-7b

Updated Nov 204,096 context
$0.17/M input tkns$0.17/M output tkns

A continuation of OpenHermes 2 model, trained on additional code datasets. Potentially the most interesting finding from training on a good ratio (est. of around 7-14% of the total dataset) of code instruction was that it has boosted several non-code benchmarks, including TruthfulQA, AGIEval, and GPT4All suite. It did however reduce BigBench benchmark score, but the net gain overall is significant.