How to use Node's fs in the browser for custom playgrounds

I was building a custom web playground for Typeconf, which uses TypeSpec compiler to generate code from schemas. The compiler is a Node.js package that expects filesystem access for reading files and dynamically loading libraries. Here's how I've solved this.

Initial Approaches

memfs

First attempt was with memfs. If you don't have real fs, use memory, pretty simple:

import { fs } from "memfs";

fs.writeFileSync("/file.txt", "content");

But it still doesn't help with module resolution which is required for TypeSpec.

WebContainers

WebContainers provide a full Node.js environment in the browser, including a working fs implementation. The drawbacks:

~4MB initial download
Significant startup time (~2-3s on fast connections)
Memory overhead from running a virtualized environment

const webcontainer = await WebContainer.boot();
await webcontainer.fs.writeFile("/file.txt", "content");
await webcontainer.spawn("node", ["compiler/dist/index.js"]);
// Too heavy for a simple playground

Custom Virtual FS

Then I've started implementing a minimal virtual filesystem myself, first to better understand how it all works and to be able to mock everything that TypeSpec needs. I've created a simple interface:

interface VirtualFS {
  readFile(path: string): Promise<string>;
  writeFile(path: string, content: string): Promise<void>;
  exists(path: string): Promise<boolean>;
  readDir(path: string): Promise<string[]>;
  // libraries? how to load them?
}

I was getting confused with what to implement to hook to the loading libraries. But then it hit me. TypeSpec's has their own playground!

The TypeSpec Solution

Looking at TypeSpec's playground implementation revealed a better approach. Instead of virtualizing the entire filesystem, they abstract only the operations their compiler needs.

CompilerHost Interface

The is TypeSpec's CompilerHost interface which provides the abstraction:

interface CompilerHost {
  // fs operations
  readUrl(url: string): Promise<SourceFile>;
  readFile(path: string): Promise<SourceFile>;
  writeFile(path: string, content: string): Promise<void>;
  readDir(path: string): Promise<string[]>;
  rm(path: string, options?: RmOptions): Promise<void>;
  mkdirp(path: string): Promise<string | undefined>;
  stat(path: string): Promise<{ isDirectory(): boolean; isFile(): boolean }>;
  realpath(path: string): Promise<string>;
  getExecutionRoot(): string;

  // url module
  fileURLToPath(url: string): string;
  pathToFileURL(path: string): string;

  // loading libraries
  getLibDirs(): string[];
  getJsImport(path: string): Promise<Record<string, any>>;
}

Browser Implementation

Here's how TypeSpec implements the CompilerHost interface in browser:

class BrowserCompilerHost implements CompilerHost {
  private files = new Map<string, string>();
  private readonly libraryFiles: Record<string, string>;

  async readFile(path: string): Promise<string> {
    if (this.files.has(path)) {
      return this.files.get(path)!;
    }

    throw new Error(`File not found: ${path}`);
  }
}

export async function createBrowserHost(
  libsToLoad: readonly string[],
  importOptions: LibraryImportOptions = {},
): Promise<BrowserHost> {
  const libraries: Record<string, PlaygroundTspLibrary & { _TypeSpecLibrary_: any }> = {};
  for (const libName of libsToLoad) {
    const { _TypeSpecLibrary_, $lib, $linter } = (await importLibrary(
      libName,
      importOptions,
    )) as any;
    libraries[libName] = {
      name: libName,
      isEmitter: $lib?.emitter,
      definition: $lib,
      packageJson: JSON.parse(_TypeSpecLibrary_.typespecSourceFiles["package.json"]),
      linter: $linter,
      _TypeSpecLibrary_,
    };
  }
  return createBrowserHostInternal({
    compiler: await importTypeSpecCompiler(importOptions),
    libraries,
  });

Here's a couple notable things:

They still implement file system in memory.
They bundle all the dependencies using a custom bundler which adds the index of files of the library.
To be able to read library files they bundle them using importmaps.

I think JS ecosystem would benefit from abstractions like system languages have for different operating systems. If there would be a way to provide a platform which will implement all the necessary standard methods this would make Node code way mode portable.

This experience has taught me to avoid implementing a full filesystem. Instead you should take it from the one that already implemented it. Kudos to TypeSpec team!