cho.sh
2,238 notes
261021261021Toward Autopoiesis26F6DF오토포이에시스를 향해84A5EE차세대 디지털 브레인 만들기C24273261019261019261018261018디지털 시대의 언어 다양성을 위한 선언문507C2CManifesto for Linguistic Diversity in the Digital Age6B5B52261012261012코리지빌리티1CAB16261005261005261004261004Mesh EngineeringF27D1B그물망 엔지니어링881E3BThe devil is in the details94F308260923260923260919260919260917260917In the Agentic Era, should we become a dumb pipe?D815AC260916260916Coscientist6975BA260915260915Coscientific Swarm5F70E8협력적 연구 군체B9E26A
Imagining WebNPU API

Imagining WebNPU API

Warning

Over a year old. May be outdated

WebGPU is a fairly new web API for modern GPU rendering and compute, over backends like Direct3D 12, Metal, and Vulkan. Same goal as WebGL, but exposes more advanced GPU features. What if we could do this with NPUs? Here's my pitch for a hypothetical WebNPU API.

  • unlocks the bare-metal power of CPUs.
  • WebGPU unlocks the bare-metal power of GPUs.
  • WebNPU unlocks the bare-metal power of NPUs.

On-device ML inference is already blooming with Apple's Neural Engine. It runs everything from simple Siri commands to full-blown Stable Diffusion models. These chips are way weaker than server-scale inference, but they have far better latency and privacy, and some jobs need exactly that. Other companies are catching up, like Google with Tensor. Sadly, so far, you can only reach these chips from native apps. On the web, we have high-level libraries running on the CPU, like TensorFlow.js.

At some point, when being (just like us Cloud-Natives) becomes the norm, we'll need an on-device API for NPUs more and more, especially on the web. And it's good for everyone involved.

  • Customers. Everything stays local, fast, and secure. No need to install an app to get an AI tailored to your preferences and behavior. Just visit a website.
  • Developers. No inference server to spin up. And they can Write Once Run Everywhere.
  • Companies. Collecting data will only get harder as regulations and restrictions pile up. WebNPU removes the need to collect it in the first place. And since AI tasks run on user devices, servers carry less load, so companies can scale their services more cheaply.
  • Hardware vendors. New business. They can build specialized hardware tuned for AI tasks and sell it to device makers and other customers.
  • Regulators. AI tasks run locally on user devices, so less data travels over the network, and the risk of data breaches and other security issues may drop too. That makes it easier for regulators to make sure data is handled responsibly and securely.

That said, we're still in the early days of AI. There's no standard way to build NPUs, and every hardware vendor has its own approach and requirements. For example, Core ML uses the Core ML model format, while TensorFlow Lite uses FlatBuffers. We'll see Apple and Google fight again, just like or USDZ vs glTF.

Backlinks3