Mixtral 8x7B FP8 on H100 with Friendli Engine #shorts #mixtral #vllm

Mixtral 8x7B FP8 is action on Friendli Engine! Friendli Engine runs blazingly fast compared to vLLM. * Under the same load condition, we send the same generation request to each engine. #shorts #vllm #mixtral