You can also go beyond Jev. Qwen 3.5 0.8B is fantastic at basic image classification/question answering (including OCR elements) also. Though rather than looking at logits, I get it to output a structured JSON object and it does simple object classification tasks on a Mac at under 500ms a pop (I forget how far, but I think it's like ~250ms) with good accuracy (depending on task).

I've had similar thoughts! I grew up around a heavy smoker. My dad told me to never smoke and I never have. But I randomly tried a nicotine lozenge in my 40s and found it such a "lightbulb on" moment that I've been using pouches for a few years now and my executive functioning still feels better for it (though not as much now as in the first two years, alas).

I had a similar thing with Xcode the other day. I unticked the "download a code assistance model" thing, clicked to continue and it started downloading it anyway (which, to be fair, you can just X out of).

There might be some practical applications of this sort of idea like in situations where you want/have a heavily restricted vocabulary to build from. You can already do this with LLMs but they can get very... "distressed" if you force logits, whereas this would not.

Come to think of it, I'm now curious if it would do well at building SQL queries, say. (30 minutes later: I tried it, and it can do it reasonably well, but a normal model and linting will outperform it, though the inherent guard rails of a limited vocab are still intriguing.)

I believe the compensation part only kicks in if it’s the airline’s fault.

You may be right. As a direct result of the NATS issue perhaps not, but any second order issues once that is resolve you might be within scope.

Almost certainly not. The legislation says: "An operating air carrier shall not be obliged to pay compensation [...], if it can prove that the cancellation is caused by extraordinary circumstances which could not have been avoided even if all reasonable measures had been taken."

That gets them off the hook for even second order effects. No harm throwing in a claim though, there's no penalty for having a claim turned down.

NATS had failed once, so it failing again doesn't seem extraordinary to me.

It is however extraordinary, and there don't seem to be any reasonable measures a carrier could take to work around it.

Fair point.

CLaude, in particular, quietly, but genuinely and deliberately, overuses adverbs.

FWIW, on my Mac Studio I get ~24-27 tok/s generation between 0-16k context in - that's on the Q6_K GGUF with speculative decoding on. I have spent zero effort optimizing/improving this so far but will be trying the 4 bit MLX next (I've tended to find models drop off somewhat below 6 bit but maybe that isn't the case nowadays).

How's the compute side now, I wonder? Because while the Ultras have impressive memory bandwidth for inference, processing prompts still takes a dog's age on my M3 Ultra. I heard the M5 makes some strides forward in this area, though, and the M7 in particular promises to go a lot further.

M5 is excellent, they’ve finally gotten their own tensor cores.

Good for inference; however if you like to train, data format support and effective performance is limited (M5 Pro). Some hardware features are not exposed or extremely slow.

You’ll be fine for inference, but pales in comparison to what a RTX 6000 Pro can do for compute/matmuls/training.

4x faster prompt processing than M3 Ultra.

More likely double that, even. I think you'd still see many buyers there. You can spend like $16k alone on a RTX 6000 PRO with a mere 96GB of VRAM now..

They seem to be suffering from the supply constraints like everyone else. They phased out the higher capacities on the M3 Ultra Mac Studio a while ago, and if you order a 128GB MBP, say, you're looking at six weeks or more for delivery.

It's worse than six weeks: Apple quoted me 10-12 weeks yesterday for an M4 Max Studio (about Nov 17) December for the M3 Ultra. Planning to lock in a new model soon. I went looking for used but those are wildly expensive. Strange times. https://hard.bargains/posts/mac-before-sept-22/