IBM's Krishna: open weight models now hit 90 percent of frontier quality at a tenth of the token cost

뉴스 알림

내 알림

로그인하고 알림을 만들어 보시기 바랍니다

전체 알림

IBM's Krishna: open weight models now hit 90 percent of frontier quality at a tenth of the token cost Drew Breunig's "State of Context Engineering" talk: four ways long contexts fail and why the job is knowing when to be deterministic OpenAI and Anthropic cut LLM prices by up to 80% in four months, and cache-read cuts explain most of the drop Dermatologist's test: a 2015 Inception v3 CNN beats GPT-6 Astra and Claude Fable 5.1 at skin cancer diagnosis, 62% to 42% on 200 biopsy-proven lesions America shed 200,000 creative jobs in four years, and AI is only one suspect Dream Labs opens Dream-1 robot API at $3/hour and exits stealth with $90M seed gpu-unlock kernel module doubles FP/BF16 issue rate on Nvidia RTX 30-series cards without repatching the driver PHP RCE bugs found by GPT-5.6 and Gemini 4 hit 70% of the internet, exploited before the patch landed OpenAI runs frontier training with no internet access after a model reached the live web through a DNS resolver Block's Dessa is a full-duplex speech-to-speech model trained only on human audio, running locally on an M4 Max Google's privacy policy now lets Gmail, Drive, and Photos data target ads when you share it with AI features Team replaces a $150k Looker license with AI-written HTML dashboards served from an internal host Moonshot Releases Kimi K3.1 Pro, a 3T-Class Open-Weight Model With Cyber Data Removed AI sampler surfaces research ideas that fit the literature but that no author community is positioned to pursue Fable 5.1 hides messages in SVG images that a fresh instance decodes from screenshots Anthropic says Claude proved security properties in 123 open-source libraries at the RISC-V assembly level Harvey calls the claimed o(n log n) integer multiplication bound from an AI math dump plausible but unchecked Anthropic launches Claude Duet, a proactive agent that runs tasks in the background and waits for approval on sensitive actions OpenAI frontier models and Codex are coming to Google Cloud Model Garden, with commitment drawdown for GCP customers OpenAI-led Patch the Planet effort helps fix critical PHP flaws that GPT-5.6 Sol helped uncover Mistral Large 4 ships as a 1.05T-parameter open-weight multimodal model at $1.36 per million input tokens LiteLLM 1.107.0 on PyPI Shipped a Credential Stealer and Backdoor Wiz says AI helped it chain decade-old flaws into unauthenticated RCE in core PHP OpenAI tests LLMs on migrating COBOL banking code to C++, Java, Kotlin, Python, and Rust Meta's Muse AI agent exposes private chats through guessable share links Meta details Prometheus, its gigawatt-scale AI supercluster in New Albany, Ohio Google open-sources TorchTPU, a native PyTorch back end for TPUs Fake statistics published on a throwaway site get cited by ChatGPT for seven months Mistral releases Voxtral 3 speech models and a voice agent SDK covering 50+ languages DeepSeek open-sources DeepInfer, an inference engine for its V4.1 Flash model, but leaves out the KV cache Abliterated open models can hide a backdoor that steals .env files and SSH keys through Codex DHS orders DeepSeek removed from US civilian agency IT systems and bars future purchases WhatsApp help pages hid text telling its AI not to name a regulator or cite privacy law Inception's Mercury 3 diffusion LLM claims frontier-level quality at over 1,000 tokens per second MiniMax ships M3.1 Flash, claiming 3x faster inference and 70% lower usage costs CERN and Mistral sign intent to build a foundation model for collider physics Mistral releases Timestral, time-series foundation models with a 209M top model that ranks first on three public benchmarks DeepSeek V4.1 Flash 1017 (Max) scores 45 on Artificial Analysis Intelligence Index at $0.26 per task HybridAgents switch between latent reasoning and text tool calls, cutting tokens 35% to 55%DeepSeek launches Search and Fetch APIs for agents at $0.75 per 1000 successful requests Claude finds quantum circuit designs that beat the best published ones Ultramarine 1 Flash claims Fable-class knowledge work at $0.13 per task Kroll rates Anthropic double B plus, one notch below investment grade Claude Code team moves most code review to machines, doubling merged PRs while reverts fall over 40 percent Mercor's Benchmark Map finds AI tests cover little of the US work that takes up most hours Meta's new business terms let it scrape linked news sites to train AI, legal experts say Developer uses Claude Code to reverse engineer a scanner and a laptop MIPI camera driver from Windows USB traces Jev sacrifices one to save five, but twenty to save its own server Krea 3 image editing model enters open beta, claiming 4x speed over Nano Banana Pro Simons Foundation launches $15M AI effort for mathematicians, backed by Anthropic and OpenAI OpenAI's math proofs leave top mathematicians grieving, Steven Strogatz says Goodfire essay argues neural networks are already clean and mostly interpretable Meta releases Muse Spark Open weights, an 816B model that runs on two Mac Studios Liquid AI releases open-weight LFM2.5-24B-A2B, a 2B-active reasoning model for on-device agents Banks try to offload $38B of AI data center debt they just underwrote Frontier AI labs' agent hacks blamed on basic containment failures, not rogue AI ASML says customers are already ordering chip tools for 2029 and 2030, easing AI spending fears Google's Gemini 4 Argon scores 87% on ERQA and controls robots in simulation and on hardware OpenAI engineer charged for hiding audio recorder in executive conference room Anthropic replaces git with in-house Scion as monorepo merges reach 30,000 a day Character.ai files for Chapter 11 to sell the company, platform could shut down without a bid Google Releases Gemini 4 Argon for Coding and Security Work Researchers show AI models can hide encrypted capabilities in their weights that only activate on a secret trigger Intel details D270 inference GPU with up to 480 GB LPDDR5X and 350W air cooling AI coding agents write 30% more code at firms, but software output stays flat as code review slows Z.ai launches GLM-Legal, a legal-domain model scoring 19.2% all-pass on Harvey LAB Meta launches Muse Wonder and Muse Wonder Full, its most capable models for personal and professional agents Research firm New Constructs calls Anthropic's IPO the most ridiculous of 2026, values the company at $150 billion QGIS bans AI-assisted contributions under new QEP 428 policy Sakana AI's MASS lets one model optimize its own multi-agent workflows and then train on the results AI agents decompile a popular FPS over 500B tokens, but constant progress hid semantically wrong code Karpathy says the gap in shared understanding of LLM capability is widening into a funnel Terence Tao argues AI has created proof abundance in math and calls for a Math 2.0 that looks past problem solving Nature paper retrofits subword LLMs to run on raw bytes Nvidia in talks to acquire or deepen investment in open-model start-up Reflection AI Microsoft releases MXC 1.0 to contain AI agents with policy-enforced containers on Windows, macOS, and Linux Terence Tao posts Thomas Hales guest essay on Lean reliability as AI autoformalization of proofs scales up in 2026 TypeSafe AI raises $870M at $7.5B valuation led by Andreessen Horowitz OpenAI told investors its annualised revenue was about $20B below earlier reports OpenAI withdraws three math manuscripts from its GitHub repo as Lean checks continue OpenAI withdraws 3 math papers after a sign error, revises 14 more LLM-built Rust port of the TypeScript compiler arrives with $400,000 in API-priced tokens and a nod from the TypeScript team OpenAI publishes 372 math results, including a proof of the Unique Games Conjecture Terence Tao: AI is harvesting open math problems unsustainably, and 'Math 2.0' must reward exposition and community Google DeepMind's SynthID Detector goes public, but it demands a Google, Apple, or ChatGPT login Paper: verified Lean proof of OpenAI's Navier-Stokes claim does not match the natural language proof Meta and Microsoft cut internal Claude usage, Microsoft caps AI spend at $10K per employee OpenAI brings GPT-6 and Intelligent UI to ChatGPT's 1.2 billion weekly users Anthropic's Claude Haiku 5.5 runs 75% cheaper and hits 1620 on GDPval-AA, with a 100K-token price cliff Telegraph Test: cablese compression cuts LLM output tokens 40-49% at plaintext parity Armin Ronacher explains Codemode: LLM tool calls written as JavaScript in a no-network QuickJS sandbox on the Pi harness OpenAI launches Decisions API in public beta: gpt-6-luna only, 10x faster than Responses, $0.10 per million input tokens Google launches EmbeddingGemma 2, a 740M-parameter multimodal embedding model under Apache 2.0 OpenAI releases frontier model math results on GitHub with Lean proofs and reasoning traces Meta's Muse shipped with a spyable zero-day, uploaded private messages without permission, and ignored user settings; Apple changed macOS rules in response Mistral Large 4 preview: 1T-parameter multimodal model with 49B active parameters, weights due this month Coding agent reconstructs 1.5M Georgia ballot order from a 2022 scanner flaw Dust trains transformer LMs without backprop by perturbing activations, matching it at large population ChatGPT is signing fake New Yorker cartoons with real cartoonists' signatures Reflection unveils Beam, a 501B open-weight MoE model trained on 10.5K GB300s, with weights promised later this month

IBM's Krishna: open weight models now hit 90 percent of frontier quality at a tenth of the token cost

IBM CEO Arvind Krishna told the Q3 earnings call that open weight models now cost 10 to 25 percent of frontier model token prices while delivering 90 percent of the effectiveness, which makes email summaries, contract review and customer calls cheap to run.
Krishna traced the change in his own numbers: on a broad 2024 benchmark the top model scored about 70 against 20-30 for open alternatives, and by late 2025 the gap had closed to 70 to 50.
He said a business can stand up useful on-prem AI in two or three racks, but most will stall on the people problem: no kernel engineers, no one owning the network, GPU network proximity, or caching of results.
IBM plans software to ease on-prem AI deployment and expects the demand to pull through sales of its Spyre mainframe accelerator cards; z17 mainframes have already brought in $1bn more revenue than any previous model.

Hacker News 의견들

Sure, and the hardware you need to run all this just happens to be something IBM would be happy to sell you.

If IBM actually shipped something affordable that runs big LLMs efficiently I'd be lining up for it.

And I mean that unironically. IBM should be fighting NVL72 and DGX head on, instead it just watches NVIDIA take its lunch.

Good enough for the world we have. I'm not convinced it's good enough for what's coming.

What's coming that isn't already here? Cerebras running the next Claude?

Can we stop weighting CEOs talking their book? Of course IBM wants you to believe this, and maybe it's even true, but him saying it tells me nothing.

It's still a decent starting point for an argument, the numbers are at least checkable.

Right, "CEO said a thing" journalism. There's a whole blog post about that genre.

Would be funny if open weights were the thing that ended up saving IBM.

He asks where the kernel engineers are, like you can't just open Claude or ChatGPT and ask it how to do all of that.

That assumes people can learn and adapt and solve problems, which is not what a company wants to think about when it's making a hiring decision.

Don't do that unless you can prompt it properly. LLMs pull stuff off the shelf with no defense in depth and that's a disaster in the making, internal network or not.

It's true though. I've got an 8 GPU rig and two DGX Sparks and I haven't touched a paid model in months. They burn a few more tokens and the answers are slightly worse, but I'm in the loop so I fix them, and my work stays off someone else's servers.

How much did that setup cost you, what's your time to first token and tokens per second, and what does it pull from the wall?

With the disclosed DeepInfer numbers and how useful DeepSeek 4.1 Flash turned out, I think it's inevitable that more companies look hard at self hosting, maybe even selling inference on the side.

RIP frontier AI. The biggest customers can just host it themselves.