cho.sh

뉴스 알림

새 알림

내 알림

로그인하고 알림을 만들어 보시기 바랍니다

전체 알림

Security0 님Dune제욱 님AI제욱 님해커뉴스성현 님
AI
WSJ report claims Nvidia financed its own customers to keep GPU prices highMichigan study: LLMs burn 1.4 to 180 times more energy on non-English prompts, with Hindi at 7.9x and Bengali at 15.3xResearcher scans 159,041 Lovable apps, finds passport numbers and 200K chat messages open to anyoneggml ships v1.0.0: deprecated API pruned, FP8 and YaRN truncation added, speedups across CUDA, Metal, Vulkan and SYCLNvidia accused of funding AI startups to buy its own GPUs, allegedly to engineer scarcityGoogle tells schools: keep AI Overviews in Search or lose Google Search entirelyClaude digests 1,009 pages of Gödel shorthand, surfacing hundreds of unknown theoremsEdge0 releases a 145B on-device model claiming 29GB peak memory, while HN calls it a rebranded Qwen fine-tune on a Strata forkApple's Foundation Models framework now drives Claude and Gemini through one Swift API, with free Private Cloud Compute for small developersNvidia engineers flood internal suggestion box demanding Jensen Huang's return after his resignationForensic read of model weights finds six Chinese frontier models started on export-controlled Blackwell GPUsGoogle ships Gemini 4 Hydrogen in Antigravity with Orchestrator, a mid-task Carbon handoff that cuts quota 24% and runs 19% fasterGoogle launches Gemini 4 Hydrogen at $1.50/$7.50 per million tokens with flat pricing across a 1M-token contextAnthropic opens IPO roadshow with $17.0B quarterly revenue, $23.3B net loss and a 15% adjusted operating marginJensen Huang resigns as Nvidia CEO and board member, effective immediately, amid China chip smuggling and spy affair allegationsGoogle launches Gemini Decisions API with the gemini-decision-1 model, scoring up to 1,000 questions in one pass at $0.10 per 1M input tokensWSJ: Nvidia board prepares to oust CEO Jensen Huang over two affairs, one with a suspected Chinese spyA linear program reproduces Astra's 8-hour Qwen weight edit in about a minute, with 8x fewer changed weightsAnthropic launches Claude Whiteboards and Sheets in beta and turns Dashboards into Claude DataVulnCheck: 7 months of Project Glasswing yields 442 Anthropic-attributed CVEs, 2 exploited in the wild, 217 fixesDeepMind's AlphaProof Code formally verifies Signal's compiled cryptographic code in four weeksAI agents formalize 16% of CPython in Lean, 613K lines and 418 bug reportsCoreWeave 2032 bond yield tops 16% as AI credit stress spreads to Nvidia, Broadcom and Oracle CDSAmazon replaces Claude Sonnet 4.6 with Strands Decider for ticket labeling at 98% lower costGoogle adds TPU 8tx and TPU 8i Max, its fourth TPU system this yearClaude Fable 5.2 one-shotted a 12-stage beat 'em up from a kid's drawings for about $300 in tokensMeta's Muse Ember matches Claude Haiku 5.5 at a third of the cost, at $0.10/$0.50 per million tokense-means hits Lloyd's 25-epoch loss in 1.2 to 3 epochs, 10-45x faster on GPUOctober 16 edit to Google's own blog post confirms consumer Gemini sends connected Drive files and Gmail to human reviewersAnthropic launches Claude Fable 5.2 and Mythos 5.2 at $8/$40 per million tokens, claiming a 30-hour Fermat proofNext-Concept Modeling generates text coarse-to-fine and runs up to 31x faster than the strongest diffusion language modelGoldman Sachs Restricts Claude Code and GitHub to Its 12,000 Engineers, Cutting Off Traders and BankersMaxxBench mutates 89 DeepSWE tasks and Gemini 3.8 Flash falls from 73.8% to 53.4% while Opus 5.5 holds at 76.4%Apiiro finds 1,122 public GitHub files holding real secrets saved by Claude Code's 'Don't Ask Again' rulesAnthropic lays out agent memory as transcripts, notes, and 'dreaming' consolidationMicrosoft Research's AI Frontiers group, about 50 people, is gone after a September reorg into Microsoft AI, and its founder has leftEvery Mac since macOS 13 quietly builds a knowledge graph of your family, friends and whereaboutsAjeya Cotra: METR report shows Anthropic has already crossed its RSP 'dramatic AI acceleration' thresholdHightouch: a Jev judge that escalates to Claude Opus matches frontier accuracy at 4x lower costThinking Machines launches Inkling 2 Small, a 594B-parameter MoE with 17B active and Apache 2.0 weights promised next weekLinum AI releases 2B Pyramid-JiT text-to-image model trained for about $20K with Apache 2.0 weightsClaude Code puts your email address in the User-Agent header on web searches, GitHub issue draws 179 reactionsBaidu releases ERNIE 5.5 Flash Base, a 280B MoE checkpoint with 14B active parameters and no instruction tuningOpenAI report: GPT-6 Astra covertly withholds help on safety research, 6-point gap versus capabilities tasksOpenAI to acquire Pierre Computer Company, maker of the Trees and Diffs libraries inside CodexRicursive releases Ripple-65B Preview, claiming US open-weight SOTA on agentic coding from a $32M training runOpenAI ships Images 2.6 across ChatGPT, Codex and the API with GIF animation and 50% faster generationOpenAI pipeline generates 948 Linux root exploits for $1.2M in API creditsAnthropic's FlashProve proves LLM inference with zkSNARKs at 110 tokens/sec on one H100, 70-95x the cost of plain inferenceVerification Is Not Knowledge: Jessica Hullman argues AI for science still needs human accountabilityMoonshot AI releases Kimi K3.1 Pro weights: 2.8T-parameter MoE, 1M-token context, $20M Model-as-a-Service license thresholdTell HN: Gemini trains on saved chat history even for paying subscribers, and opting out costs you your chats after 72 hoursQwen 3.8 27B on a laptop reads raw model weights, matching GPT-6 Astra's no-tools demoHN users report OpenAI silently re-enabled Codex data sharing for model trainingGoogle bundles school search with Gemini: a school that bans AI loses Google Search for its K-12 studentsRevolut moves 2,000 engineers to self-hosted GLM, cutting coding token costs to about a tenthOpenAI's Pando Git backend cuts push latency nearly 5x and serves almost a petabyte a dayIntel falls 9% as $3.7T chip rout spreads; Nvidia down 16% since Oct. 6 but holding upSEC urged to block Anthropic's IPO over $895B in off-balance-sheet commitmentsMIT CSAIL: GPT-6 Astra reads raw weights, edits 0.016% of Qwen3.5-0.8B to lift MATH-500 from 7.9% to 48.1%LeCun co-authored paper proves Platonic Representation Theorem for JEPAs, says good views need 42x fewer samples than next-token predictionPostgres 19 slips to November as LLM-found bugs push back-patched commits up 74%Frank Calegari on Terry Tao's blog: OpenAI's 700-paper math release is a watershed, and mathematicians should explore rather than mournBrazilian Labor Court Fines Two Lawyers for Hidden Prompt Injection Targeting the Court's AIChatGPT share links ran attacker JavaScript inside the victim's signed-in session: OpenAI rates the account-takeover chain P1 and patches itGoogle Research's sleep framework lets a deployed LLM consolidate new facts into its weights during offline phases, claiming 10M-token retentionStealth Bot Bills Would Sweep In Journalists and Archivists, Not Just Bad CrawlersBlackstone tells investors Anthropic may get a high junk rating after IPO, complicating its $63B chip financingClaude Code VS Code extension 2.1.313 shipped a backdoored SSH implant through the Cursor MarketplaceMalicious Claude Code extension 2.1.313 shipped an SSH backdoor to Cursor developers, about 5,000 systems affectedBackdoored Claude Code extension 2.1.313 shipped a reverse SSH backdoor via Open VSX and CursorYC Labs runs 50 GPT-6.1 Sol agents in Minecraft for 3 hours: they build a ration-ledger government, send 5,900 messages, and lose 22 agents on night oneMistral releases weights of 1T-parameter Mistral Large 4, with big coding gains and API prices cut roughly in halfCartesia says its byte-level H-Net matches a tokenized open-source model with 10x less compute and dataOpenAI cuts frontier training off the internet, and its isolated agents still traded secrets through CPU cache timingClaude Code sends each active background agent's full transcript to Opus every 30 seconds for a 3-word status line, and a Bun preload brings that agent_summary spend to zeroCursor ships Agent SDK, the framework behind Bugbot and Security Reviewer, deployed with one commandStack Overflow falls below 1,000 questions a month, fewer than it got in its second month of existence in 2008Google announces Gemini 4 Carbon, claims 0.0% successful cheating on Terminal-Bench 4.0, but the model is not available yetLiquid AI releases LFM3: six 2B-class multimodal models on one tri-hybrid backbone, open weights on Hugging FaceAnthropic reports Claude Code cut a CAR-T component 70% and generated 35,000 antibody designs in a monthAxiomMath formalizes the Classification of Finite Simple Groups in Lean, 43.1M lines checked by the kernelPolice Use AI to Flag 'Suspicious' Driving, and Hacker News Readers Call It a Surveillance DragnetStanford professors opt out of Anthropic's $1.5B book-piracy settlement to sue for up to $150K per workDeepMind announces Genie 4, a world model that generates scene audio and dialogue in real timeDebian developer refuses NEW uploads after DFSG team silently adds LLM reviewRoblox crawler pulled 145.5 GB from 3D model sites, ignored robots.txt and non-commercial licensesCloudflare's GLM-5.3 Workers AI endpoint leaked 155 cross-tenant responses through an SGLang KV cache bugOpenAI posts a preprint titled "The Exponential Time Hypothesis is False" with no abstract or explanationOpenAI says its models disproved the Exponential Time Hypothesis and proved local smoothing in every dimensionvLLM 0.31 adds constrained decoding, letting Ubicloud drop its fork and hit 100% tool-call schema accuracyCohere sandbox catches Alibaba's Qwen3.8-Flash-Next ignoring instructions and trying to upload a probe file to a cloud bucket in BeijingOpenAI releases GPT-6.1 Terra: half the price of Sol 6.1, but Hacker News commenters flag a weak Terminal Bench 4.0 resultAnthropic files S-1 to list on Nasdaq as ANT, a public benefit corporation, with Q3 revenue at $16.8BIBM's Krishna: open weight models now hit 90 percent of frontier quality at a tenth of the token costDrew Breunig's "State of Context Engineering" talk: four ways long contexts fail and why the job is knowing when to be deterministicOpenAI and Anthropic cut LLM prices by up to 80% in four months, and cache-read cuts explain most of the dropDermatologist's test: a 2015 Inception v3 CNN beats GPT-6 Astra and Claude Fable 5.1 at skin cancer diagnosis, 62% to 42% on 200 biopsy-proven lesionsAmerica shed 200,000 creative jobs in four years, and AI is only one suspectDream Labs opens Dream-1 robot API at $3/hour and exits stealth with $90M seed
vLLM 0.31 adds constrained decoding, letting Ubicloud drop its fork and hit 100% tool-call schema accuracy

vLLM 0.31 adds constrained decoding, letting Ubicloud drop its fork and hit 100% tool-call schema accuracy

  • vLLM 0.31 added constrained decoding, which blocks invalid tokens during sampling, so Ubicloud stopped carrying its own vLLM fork and still gets 100% tool-call schema accuracy with the newer Kimi and DeepSeek models.
  • Ubicloud kept the fork because bad tool calls compound: a malformed call stays in the context and the model builds its next turn on the nonsense.
  • An HN commenter read the example stats as a 23/25 tool-call success rate and calculated that backing up and retrying costs only +9% tokens on the call attempt, so the missing piece is a comparison of overall model quality with and without the constraint.
  • Moonshot AI serves Kimi with constrained decoding on by default and OpenAI's Responses API turns it on by default, but Ubicloud notes published work shows it can hurt quality depending on the model.
  • llama.cpp has shipped grammar-constrained decoding for years and transformers.js added it last month, so vLLM arrived late to a feature other serving engines treated as standard.

Hacker News 의견들

I'm one of the authors of this post. Getting the LLM, the serving engine and the constrained decoding implementation to line up so you get the output you actually intended takes a lot of careful engineering and testing. Happy to answer questions.

Where does that <||> syntax even come from? It's odd that they didn't reuse existing markup, and there's almost nothing about it online.

So the model isn't really making the tool call, the engine just filters the sampler down to valid moves? Could you use that sampler massaging elsewhere to steer output while the weights stay the same?

Does this survive inference optimizations like DFlash and MTP, or does it slow things down to the point where retrying is just cheaper?

Hugging Face transformers.js got constrained decoding last month too, for in-browser structured output and tool calling.

Can the same mechanism force syntactically correct code?

Yes, all you need is a formal grammar for the output syntax. llama.cpp documents how grammar-constrained decoding works and ships a C example, and I've run text-to-SQL experiments with it. The hard part is wiring it into the inference engine, not impossible, just very hard.

Is that even a real problem though? If you're consistently getting syntax errors out of frontier models plus Codex or Claude Code, you've got bigger issues.

Constrained decoding is the same approach we used to turn an AR model into the /v1/systemone endpoint in vLLM. Tool calling is messy because nothing says a model's tool set is fixed for the life of a run, and the industry still hasn't settled on OpenAI-style formats for mid-run tool updates.

I wish they'd published the actual impact on model quality. Back-up-and-retry also gives 100% tool-call accuracy without polluting the context, and at 23/25 that's only +9% tokens on the tool call attempt, so if retry scores better on accuracy it might win out.

Those are two different questions: quality of the whole stack, versus how closely our setup matches the first-party provider. This post is about the second. Moonshot recommends constrained decoding on by default and serves Kimi that way, and OpenAI's newer Responses API has it on by default too. There is work showing it can hurt quality, depending on the model.

My prompts need around 300 tool calls each and I've been stuck with OpenAI and Anthropic since 2024. If anyone wonders what moat those two have: they nailed 100% tool calling via constrained decoding and prompt caching back then.

llama.cpp has done this forever. I was surprised that most other engines, vLLM included, didn't have it.

Does this work with Qwen 3.8?

Constrained decoding is available for Qwen 3.8 in vLLM, but mileage varies. Moonshot explicitly recommends turning it on for Kimi; I don't think that's the case for Qwen.