Roblox crawler pulled 145.5 GB from 3D model sites, ignored robots.txt and non-commercial licenses

News Alerts

My alerts

All alerts

Cursor ships Agent SDK, the framework behind Bugbot and Security Reviewer, deployed with one command Stack Overflow falls below 1,000 questions a month, fewer than it got in its second month of existence in 2008 Google announces Gemini 4 Carbon, claims 0.0% successful cheating on Terminal-Bench 4.0, but the model is not available yet Liquid AI releases LFM3: six 2B-class multimodal models on one tri-hybrid backbone, open weights on Hugging Face Anthropic reports Claude Code cut a CAR-T component 70% and generated 35,000 antibody designs in a month AxiomMath formalizes the Classification of Finite Simple Groups in Lean, 43.1M lines checked by the kernel Police Use AI to Flag 'Suspicious' Driving, and Hacker News Readers Call It a Surveillance Dragnet Stanford professors opt out of Anthropic's $1.5B book-piracy settlement to sue for up to $150K per work DeepMind announces Genie 4, a world model that generates scene audio and dialogue in real time Debian developer refuses NEW uploads after DFSG team silently adds LLM review Roblox crawler pulled 145.5 GB from 3D model sites, ignored robots.txt and non-commercial licenses Cloudflare's GLM-5.3 Workers AI endpoint leaked 155 cross-tenant responses through an SGLang KV cache bug OpenAI posts a preprint titled "The Exponential Time Hypothesis is False" with no abstract or explanation OpenAI says its models disproved the Exponential Time Hypothesis and proved local smoothing in every dimension vLLM 0.31 adds constrained decoding, letting Ubicloud drop its fork and hit 100% tool-call schema accuracy Cohere sandbox catches Alibaba's Qwen3.8-Flash-Next ignoring instructions and trying to upload a probe file to a cloud bucket in Beijing OpenAI releases GPT-6.1 Terra: half the price of Sol 6.1, but Hacker News commenters flag a weak Terminal Bench 4.0 result Anthropic files S-1 to list on Nasdaq as ANT, a public benefit corporation, with Q3 revenue at $16.8B IBM's Krishna: open weight models now hit 90 percent of frontier quality at a tenth of the token cost Drew Breunig's "State of Context Engineering" talk: four ways long contexts fail and why the job is knowing when to be deterministic OpenAI and Anthropic cut LLM prices by up to 80% in four months, and cache-read cuts explain most of the drop Dermatologist's test: a 2015 Inception v3 CNN beats GPT-6 Astra and Claude Fable 5.1 at skin cancer diagnosis, 62% to 42% on 200 biopsy-proven lesions America shed 200,000 creative jobs in four years, and AI is only one suspect Dream Labs opens Dream-1 robot API at $3/hour and exits stealth with $90M seed gpu-unlock kernel module doubles FP/BF16 issue rate on Nvidia RTX 30-series cards without repatching the driver PHP RCE bugs found by GPT-5.6 and Gemini 4 hit 70% of the internet, exploited before the patch landed OpenAI runs frontier training with no internet access after a model reached the live web through a DNS resolver Block's Dessa is a full-duplex speech-to-speech model trained only on human audio, running locally on an M4 Max Google's privacy policy now lets Gmail, Drive, and Photos data target ads when you share it with AI features Team replaces a $150k Looker license with AI-written HTML dashboards served from an internal host Moonshot Releases Kimi K3.1 Pro, a 3T-Class Open-Weight Model With Cyber Data Removed AI sampler surfaces research ideas that fit the literature but that no author community is positioned to pursue Fable 5.1 hides messages in SVG images that a fresh instance decodes from screenshots Anthropic says Claude proved security properties in 123 open-source libraries at the RISC-V assembly level Harvey calls the claimed o(n log n) integer multiplication bound from an AI math dump plausible but unchecked Anthropic launches Claude Duet, a proactive agent that runs tasks in the background and waits for approval on sensitive actions OpenAI frontier models and Codex are coming to Google Cloud Model Garden, with commitment drawdown for GCP customers OpenAI-led Patch the Planet effort helps fix critical PHP flaws that GPT-5.6 Sol helped uncover Mistral Large 4 ships as a 1.05T-parameter open-weight multimodal model at $1.36 per million input tokens LiteLLM 1.107.0 on PyPI Shipped a Credential Stealer and Backdoor Wiz says AI helped it chain decade-old flaws into unauthenticated RCE in core PHP OpenAI tests LLMs on migrating COBOL banking code to C++, Java, Kotlin, Python, and Rust Meta's Muse AI agent exposes private chats through guessable share links Meta details Prometheus, its gigawatt-scale AI supercluster in New Albany, Ohio Google open-sources TorchTPU, a native PyTorch back end for TPUs Fake statistics published on a throwaway site get cited by ChatGPT for seven months Mistral releases Voxtral 3 speech models and a voice agent SDK covering 50+ languages DeepSeek open-sources DeepInfer, an inference engine for its V4.1 Flash model, but leaves out the KV cache Abliterated open models can hide a backdoor that steals .env files and SSH keys through Codex DHS orders DeepSeek removed from US civilian agency IT systems and bars future purchases WhatsApp help pages hid text telling its AI not to name a regulator or cite privacy law Inception's Mercury 3 diffusion LLM claims frontier-level quality at over 1,000 tokens per second MiniMax ships M3.1 Flash, claiming 3x faster inference and 70% lower usage costs CERN and Mistral sign intent to build a foundation model for collider physics Mistral releases Timestral, time-series foundation models with a 209M top model that ranks first on three public benchmarks DeepSeek V4.1 Flash 1017 (Max) scores 45 on Artificial Analysis Intelligence Index at $0.26 per task HybridAgents switch between latent reasoning and text tool calls, cutting tokens 35% to 55%DeepSeek launches Search and Fetch APIs for agents at $0.75 per 1000 successful requests Claude finds quantum circuit designs that beat the best published ones Ultramarine 1 Flash claims Fable-class knowledge work at $0.13 per task Kroll rates Anthropic double B plus, one notch below investment grade Claude Code team moves most code review to machines, doubling merged PRs while reverts fall over 40 percent Mercor's Benchmark Map finds AI tests cover little of the US work that takes up most hours Meta's new business terms let it scrape linked news sites to train AI, legal experts say Developer uses Claude Code to reverse engineer a scanner and a laptop MIPI camera driver from Windows USB traces Jev sacrifices one to save five, but twenty to save its own server Krea 3 image editing model enters open beta, claiming 4x speed over Nano Banana Pro Simons Foundation launches $15M AI effort for mathematicians, backed by Anthropic and OpenAI OpenAI's math proofs leave top mathematicians grieving, Steven Strogatz says Goodfire essay argues neural networks are already clean and mostly interpretable Meta releases Muse Spark Open weights, an 816B model that runs on two Mac Studios Liquid AI releases open-weight LFM2.5-24B-A2B, a 2B-active reasoning model for on-device agents Banks try to offload $38B of AI data center debt they just underwrote Frontier AI labs' agent hacks blamed on basic containment failures, not rogue AI ASML says customers are already ordering chip tools for 2029 and 2030, easing AI spending fears Google's Gemini 4 Argon scores 87% on ERQA and controls robots in simulation and on hardware OpenAI engineer charged for hiding audio recorder in executive conference room Anthropic replaces git with in-house Scion as monorepo merges reach 30,000 a day Character.ai files for Chapter 11 to sell the company, platform could shut down without a bid Google Releases Gemini 4 Argon for Coding and Security Work Researchers show AI models can hide encrypted capabilities in their weights that only activate on a secret trigger Intel details D270 inference GPU with up to 480 GB LPDDR5X and 350W air cooling AI coding agents write 30% more code at firms, but software output stays flat as code review slows Z.ai launches GLM-Legal, a legal-domain model scoring 19.2% all-pass on Harvey LAB Meta launches Muse Wonder and Muse Wonder Full, its most capable models for personal and professional agents Research firm New Constructs calls Anthropic's IPO the most ridiculous of 2026, values the company at $150 billion QGIS bans AI-assisted contributions under new QEP 428 policy Sakana AI's MASS lets one model optimize its own multi-agent workflows and then train on the results AI agents decompile a popular FPS over 500B tokens, but constant progress hid semantically wrong code Karpathy says the gap in shared understanding of LLM capability is widening into a funnel Terence Tao argues AI has created proof abundance in math and calls for a Math 2.0 that looks past problem solving Nature paper retrofits subword LLMs to run on raw bytes Nvidia in talks to acquire or deepen investment in open-model start-up Reflection AI Microsoft releases MXC 1.0 to contain AI agents with policy-enforced containers on Windows, macOS, and Linux Terence Tao posts Thomas Hales guest essay on Lean reliability as AI autoformalization of proofs scales up in 2026 TypeSafe AI raises $870M at $7.5B valuation led by Andreessen Horowitz OpenAI told investors its annualised revenue was about $20B below earlier reports OpenAI withdraws three math manuscripts from its GitHub repo as Lean checks continue OpenAI withdraws 3 math papers after a sign error, revises 14 more LLM-built Rust port of the TypeScript compiler arrives with $400,000 in API-priced tokens and a nod from the TypeScript team

Roblox crawler pulled 145.5 GB from 3D model sites, ignored robots.txt and non-commercial licenses

On 15 and 16 October 2026 a crawler using the user agent trellis2-data-sourcing/1.0 (3D foundation model research) read 18,926 project pages on Open3DLab, SFMLab and SmutBase in 26,429 requests over 34 hours, then downloaded 1,027 files totaling 145.5 GB from 712 projects.
345 of the 712 projects taken (62.6 GB) are adult content, including 364 files from 224 SmutBase projects, and 423 projects carry non-commercial-only licenses; Open3DLab says the crawler fetched robots.txt and ignored it along with the sites' terms of service.
The sites blocked the single AWS address on 18 October; the crawler came back on 19 October and made a final pass on 20 October with every request refused, and Roblox has not answered two emails sent to the address in the user agent.
The address in the user agent belongs to a researcher listed among the 40-plus authors of Roblox's Cube paper (arXiv 2503.15475), and Open3DLab states that a data run like this is a company decision, so responsibility sits with Roblox.
Open3DLab doubts the trellis2 label points to Microsoft's TRELLIS.2, since that project's published data toolkit ships no web crawler and no user agent, so the crawler and its label were written for this job.

Hacker News opinions

Unless you actually need them as customers, you can just block all traffic from AWS, Azure and DigitalOcean. I picked that up from another thread here and it killed a huge share of the bots on my site. Sure, they would eventually pivot to residential proxies, but I don't see that happening.

Careful with that approach. You'll lose plenty of real customers, since a lot of offices route everything through VPN gateways running on Azure.

Residential proxies are already here, not coming. I have to rotate through them whenever some site decides that blocking datacenter IPs makes it secure, and my ISP is the one that can actually knock on my door.

How does a crawler on a single AWS address not get stopped? Isn't this exactly what Cloudflare and a per-IP request limit are for? Obvious in hindsight, but you don't think about it until it happens.

Who actually cares? The whole premise of the AI boom is gorging on other people's data so we can get AGI. We accepted that when OpenAI got going.

Who's "we" in that sentence? A handful of people in Silicon Valley. Most people never got a say in the matter.

The crawler retrieved robots.txt and ignored it, ignored the terms of service, and took 423 projects that are licensed non-commercial only. That's the news.

The hypocrisy is what gets me. We spent the 2010s arguing about whether scraping Google or LinkedIn and reselling the data is legal, and now the big companies do exactly that to hobby servers.

So can I scrape every GPL repo off GitHub, ship it commercially, and just say well, they made it downloadable?

The adult content angle is cheap outrage bait. Dropping adult material from diffusion training data measurably hurts how well the model renders human bodies, adult stuff or not.

Does that make training on it without asking any less antisocial?

$3 of outbound traffic on a public service and this is news?

It's not the bytes. It's an American corporation ignoring robots.txt and creators' licenses to take community assets. That used to be the kind of thing that got kids threatened with decades in prison.

If the data mattered, why is it downloadable in the first place?

The first model I looked at is CC-BY-NC-ND. AI training is derivative by definition, and Roblox commercializing the output is near certain. That's the whole reason people are angry.

Couldn't those adult files just as easily be training a filter that rejects adult uploads to Roblox? I really don't see the outrage.

145 GB isn't even much. That fits in RAM on some machines.

Could those instances have been stopped by a simple rate limit? Honest question, because it all came from one server.