Leading Next-Gen MLOps FE for Top MedAI

Recently I have poured most of my effort into MLOps at Lunit. At my first job here I led the modern redevelopment of the entire front-end for our MLOps platform. It ended up half success, half failure. Here's the story.

AutoML

Our team is AutoML. We run INCL, Lunit's MLOps platform.

Lunit is a medical AI company, and hundreds of ML experiments run there every day. We started with on-premise servers to handle the heavy compute affordably, but it required frequent manual intervention and was hard to scale. As the team grew, demand grew. Near deadlines it spiked. New hardware was expensive and slow to get, and guessing the capacity we needed risked buying too much or too little. Old servers also meant frequent GPU failures, worse performance, downtime, and rising maintenance costs.

So we considered migrating to the cloud, but the hassle of using it was the biggest problem. Typically the flow was create a VM, set up training, train, save to cloud storage, delete the VM, and scientists had to manage all of it tediously. Beyond that there was even more overhead for result management, log monitoring, error handling, and so on.

Lunit built INCL around 2021 to automate that. Think Weights & Biases, Vessl, or SkyPilot, but tuned for medical AI. Give it experiment code and the metric names to track and it automatically runs the experiment in the cloud and shows clear graphs. Experiment speed shot up, and in about four years since launch approximately 8 million experiments have run on it. More in Lunit's official blog post or my summary.

But it was an internal app, so tech debt piled up. If it just worked, that was enough. Cleanup and optimization always lost to new features, and the service got big and heavy.

Since joining Lunit in May 2024, I have advocated for improving this. Every action taking several seconds to load adds up to dozens of hours stolen from researchers each month. I also hoped a revamped INCL could work as an open service. The team agreed this was a good direction toward becoming an open service. They had various features they wanted to add, but the bloated codebase made changes extremely difficult and user-requested features were always in limbo. This was seen as a great opportunity to break through.

It was a shared dream with different visions. Everyone still agreed on making INCL the most attractive MLOps service.

Netherlands and Data Fetching Paradigms

INCL had a critical engineering debt that is not intuitively obvious. Every cloud resource lives in the Netherlands, because when the locations were picked, the Netherlands GCP region had the most idle resources. The backend has to record and track vast amounts of data from containers, and putting it elsewhere would mean egress costs, so it sits in the Netherlands too. All backend resources were stuck in the Netherlands. It was physically bound to take a long time. If you can't picture it, click the button below to gauge it yourself. Some pages chained 4 network requests, which meant 8 round trips over that distance!

But as they say, desperate times call for desperate measures. Heroes emerge in troubled times. Starting from pnpm init gave us a chance to redo everything. It was the perfect chance to overhaul from the start with new Next.js and React tech like App Router and React Server Components, a potential technical breakthrough that could solve the Netherlands problem in one stroke. Early on I researched various ways to use Server Components, SWR, Streaming SSR, and Partial Prerendering.

sequenceDiagram
    title Client Fetching
    autonumber
    actor Client
    participant Server
    participant API
    participant API2
    Client->>Server: Request INCL
    Server->>Client: HTML
    Note over Client: Execute JS (spinner)
    Client->>API: Fetch
    API->>Client: New data
    Client->>API2: Fetch
    API2->>Client: New data
    Client->>Client: Render UI

sequenceDiagram
    title Partial Prerendering
    autonumber
    actor Client
    participant Edge
    participant Server
    participant API
    participant API2 as API 2
    loop ISR
        Server->>API: Fetch
        API->>Server: Data
        Server->>API2: Fetch
        API2->>Server: Data
        Server->>Edge: Static HTML
    end
    %% request-time
    Client->>Edge: Request the INCL domain
    Edge->>Client: Static HTML
    Server->>Client: Progressive streaming render

Hybrid

sequenceDiagram
    title Hybrid
    autonumber
    actor Client
    participant Edge
    participant Server as Next Server Components
    participant API
    loop Refresh
        Server->>API: Fetch
        API->>Server: Data
        Server->>Edge: Update cache
    end
    %% request-time flow
    Client->>Edge: Request the INCL domain
    Edge->>Client: Stream HTML + cache initial data
    Note over Client: Set SWR fallback cache
    %% Client-side live updates
    Client->>API: SWR refresh fetch
    API->>Client: New data
    Client->>Client: Update UI

I landed on a workaround. Inject seed data from the server into SWR as fallback data, and turn off SWR's initial isLoading on purpose. Next.js Server Components stream the data in through Next's cache layer, and Streaming SSR hands it straight to SWR as fallback. That way we could achieve both the fast first load of Server Components and SWR's live data, at the cost of bundle size.

• 👍 Improves data fetching itself (initially displayed data is cached)
• 👍 Updates to latest info after a few seconds via client-side SWR
• 👍 Screen auto-updates without refresh via client-side SWR
• 👍 Suitable for apps with lots of personalized data
• 👎 Relatively larger bundle size

Video

The screen updates itself with fresh data, no refresh. This was the feature that most improved quality of life for researchers constantly monitoring experiment results!

I turned off isLoading because I wanted one place to control loading state. SWR's isLoading is always true on first load even when fallback data exists, so the loading screen would show even with server seed data in hand. Now there are more advanced patterns, most notably passing a prefetch Promise started on the server down to the client and reading it with the use hook. If server-side data fetching takes long, you still need to consider caching in various ways, though.

I think PPR and Server Components are unsuitable for interaction-heavy apps. We used Server Components less for the benefits Vercel pitches and more for the Next.js Server Cache Directive. Move the DB and API closer and SWR alone would probably do. It was also disappointing that experiment data can run to tens of MBs while the Next.js Cache caps at 2MB, which forced more workarounds.

Toward a Modern Web

Next App Router

I fully adopted Next App Router this time. Most of it is just a different way to do what Pages Router did, without overwhelming convenience, but Nested Layout was overwhelmingly convenient (except when layout and children get hard to draw as anything but rectangles). By actively utilizing that pattern, I could also compose layouts like this...

Miller Columns

Miller columns are columns that stretch sideways through a directory with a mixed hierarchy.

For example, this screen in Mac Finder is also a type of Miller column.

There are many users, projects, and jobs, and users need to navigate between many jobs across different projects. The old UI tied it all into a tree with pagination inside the tree, making it very difficult to find jobs.

I wanted a structure like this.

This was mistaken for typical nested layouts, but then making the list view visible instead of the main content on mobile would be difficult.

It looks simple at first glance, but without App Router's Nested Layout it's really painful. A message from Brandon Bayer of Blitz.js and FlightControl.dev helped a lot. Put the content in layout.tsx and leave page.tsx empty on purpose.

Video

I'm proud of this one. Just a few hundred lines handles the complex hierarchy, through deep UX consideration and technical understanding of nested layout.tsx.

The final implementation is shown above. Multiple columns overlap, and the selected one folds away with its text running vertically, like a book on a shelf. Every column has infinite scroll, context menus (the three-dot menus), sorting, and filtering.

It's still early. Some people haven't gotten used to the new UI and improvement feedback keeps coming in, but we have confirmed that users generally like the considerate interface developed across all aspects.

Here are some components I made. Try them (keyboard shortcuts need a click on the iframe above first so it has focus).

Mobile Support

Work studio apps tend to focus only on desktop. Most work is done on desktop anyway. INCL was the same. But AI training typically takes hours, so there was steady demand to check progress on mobile while on the go. Mobile hadn't been considered at all before, making users very uncomfortable. With Tailwind's responsive utilities the new UI looks good on desktop and mobile while sharing most of the code.

Toward an Open Ecosystem

I actively adopted and improved big libraries like TanStack/table and shadcn-ui/ui and newer ones like toss/es-toolkit and 47ng/nuqs, using them where appropriate, and I also contributed back to help shape the roadmap. Contributing upstream when stuck became routine. Here are some examples.

• toss/es-toolkit: (feature) Add Custom Delimiter Support to flattenObject Utility #933
• 47ng/nuqs: Server-side `clearOnDefault` and `urlKeys`
• juliencrn/usehooks-ts: useLocalStorage Hydration Mismatch -- Need to setIsClient everywhere
• vercel/next.js: Using Async Tags with Revalidate on Next.js Server Fetch Contaminates JSON Response
• vercel/swr: `revalidateOnMount` With Fallback triggers `isLoading` instead of `isValidating`
• vercel/swr: Visiting 404 Not Found clears all SWR Cache

Some disappointments with open source too. TanStack Table doesn't seem built for row data that changes in real time. For example, when row values change you need to write custom logic for table pages, selected rows, and which columns show.

The shadcn/ui dialog renders through a React Portal, which made it cumbersome to calculate mouse offset on screen when implementing drag and drop. I ended up building my own drag-and-drop dialog that looks identical to the Shadcn one.

State Management

I've consistently advocated that SWR removes the need for separate state management. All information should be managed by the server anyway. Update it there and SWR pushes the fresh state to the client in real time.

Often when I say this, the discussion gets dismissed as lacking experience with complex apps, and when I ask why it doesn't work in complex apps, there's no convincing rationale. So I tried it myself and learned what they meant by lacking complex app experience. All front-end complexity problems stem from lack of API control. SWR is great at keeping one endpoint fresh. But a graph built from several sources needed one giant SWR hook conducting a pile of smaller SWR hooks, like a game of Simon Says. Custom fetch functions significantly simplified it later, but the conclusion was that SWR alone is insufficient for managing state when combining several APIs.

SWR-only state only works in the ideal case, one unified API per feature. In that sense it's regrettable that management points increased as combination hooks proliferated. The real problem was not thoroughly specifying the API specifications, and I keenly felt that this team lacks an explicit technical PM.

Also, some info never lives on the server. For example, graph values and table view settings users configure. I thought those should be stored in URLs from the start and users should bookmark them. Then large data hard to fit in a URL showed up, and I ended up writing code to sync local storage data with URL state. For keeping the URL as the source of truth for local state, that seemed like the best approach. For data like table information, though, I had to exclusively use useLocalStorage.

Looking Ahead

So why rebuild at all, and what are we actually chasing?

Cursor for ML

Lunit wants to build Cursor for ML. Rebuilding INCL's front-end was part of that. Now Lunit is integrating various agentic AI to create a platform where users can input this.

and the experiment just runs, automatically and intelligently. The vision is to accelerate medical AI development through this. Lunit wants to transform AutoML's meaning from Automated ML to Autonomous ML.

The AWS of AI

On the way there, I believe the intermediate byproduct products, our MLOps tools included, can be externalized to grow into a platform company. These intermediate products could become cash cows for medical AI development.

For example, why did Amazon release AWS? Amazon hadn't solidified its position as the #1 internet store yet, and releasing AWS was a risky decision that could help offline competitors transition to cloud and threaten Amazon. Yet Amazon released AWS because it was a way to diversify their portfolio by growing AWS into a product line independent of Amazon, increasing financial stability while stabilizing the platform through external AWS customers. Now AWS's overwhelming profit margins effectively back Amazon's high-volume, low-margin strategy, serving as the catalyst for Amazon's growth into the undisputed #1 e-commerce company. None of that happens if AWS stays internal.

I believe Lunit's ML product line could play a similar role. Numerous MLOps products exist, but few companies are as committed to end-to-end cost optimization as Lunit. In short, most MLOps platforms focus on making it difficult for customers to easily optimize costs. They do minimal cost optimization and bill customers for the rest. Lunit is both an ML training customer and an ML training platform provider, and knowing both sides' information enables optimizing all costs across the whole pipeline, and INCL specializes in this. Using INCL actively saves cloud idle resources, reducing costs by 60-91%. Isn't that appealing from a customer's perspective?

For these reasons, I believe Lunit's MLOps product line can be externalized to grow into a platform company.

Our Team Is Hiring

• Senior
• Junior

So, we're hiring someone to work directly with me to advance the best MLOps platform and integrate agents to build Cursor for ML. That requires understanding across backend, front-end, infrastructure, and the AI ecosystem. As one example, to solve the problem of finding idle cloud resources for LLMs being difficult, I'm working on integrating tracking for model training experiments on personal GPU environments. It combines cloud training's convenience with the power of users' personal GPUs. Think GitHub Actions self-hosted runners.

If you're an engineer with broad platform understanding and want in on the dream of building Cursor for ML, just apply to the job posting. I can say this because I also applied to Lunit on a whim, and that knock and the door shall be opened mindset brought me here.