Hacker News — vinext + Cloudflare Workers

50 points by Maverick617 5 hours ago | 6 comments

PcChip 5 hours ago [-]

I didn't see any benchmarks against vllm, sglang, exllama, etc

rancor 4 hours ago [-]

Since this is basically a wrapper around libllama.so, I would assume that the performance is roughly the same as llama.cpp upstream.

dlcarrier 3 hours ago [-]

From what I've seen, Vulkan adds a lot of overhead on Intel hardware.

peddling-brink 3 hours ago [-]

> llama.cpp via Vulkan (AMD / Intel / NVIDIA) or CPU fallback

I got excited about someone paying attention to intel. Oh well.

wronglebowski 22 minutes ago [-]

What hardware do you have? I’ve been playing with a 258V and OpenVINO has come a longggggg way.

kamranjon 2 hours ago [-]

llama.cpp sycl and vllm xmx work is pretty incredible right now - you just gotta build it with some extra flags

shayanjavadi 2 hours ago [-]

[dead]

Rendered at 01:56:52 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.