Polkassembly - Referendum #1477

Building upon #812, #948 & #1334. This proposal requests funding for the continuation of ReactiveDOT, alongside its related projects (DOTConsole, DOTConnect).

Reactive DOT is a library designed to:

Simplify development: Provide a set of intuitive front-end functions and utilities to facilitate Substrate chain interactions, making development accessible for developers of all skill levels.
Enhance developer experience: Reduce boilerplate code and complexity involved in integrating DApps with Substrate chains, allowing developers to focus on building robust applications.
Promote adoption: Lower the entry barrier for developers through improved tooling and familiar patterns, encouraging the adoption of Polkadot.

Previous goals

Stabilise API

Various updates were made to the public-facing API, moving many components to internal APIs, except for those identified as highly stable:

Achieve greater than 65% test coverage

Current code coverage details are available here: https://app.codecov.io/gh/tien/reactive-dot/

Provide detailed API documentation and sample usage

Continued documentation updates and maintenance for new features include:

Create improved tutorials

Created PR to enhance the "Substrate Kitties" tutorial, showcasing the significant reduction in code by using ReactiveDOT.
Assisted Parity in developing create-polkadot-app, a CLI tool designed to quickly bootstrap Polkadot DApps using PAPI, ReactiveDOT, and DOTConnect. My role was primarily advisory, with Parity employees handling the core development.

Cut release for version 1.0.0

Although the API is currently highly stable and ready for a v1.0.0 release, I've opted to remain in the pre-1.0 phase to retain flexibility for additional features work (will be outlined in later sections).

Previous stretch goals achieved

Context-based controllable subscription state

Useful for building screens with a large number of subscriptions, requiring targeted optimization, i.e. infinite lazy loadable list.

PR: https://github.com/tien/reactive-dot/pull/525

Documentation: https://reactivedot.dev/react/guides/performance#controlling-subscriptions

Demonstration: https://x.com/TienNguyenK/status/1895581356908065277

Ergonomic light-client config & Substrate Connect integration

PR: https://github.com/tien/reactive-dot/pull/368

Demonstration: https://x.com/TienNguyenK/status/1896888190499443132

Mimir wallet support

As per how plug-n-play wallet was designed, integration is optional, and dependencies must be explicitly added with:

yarn add @reactive-dot/wallet-mimir

import { defineConfig } from "@reactive-dot/core";
import { MimirWalletProvider } from "@reactive-dot/mimir";

export const config = defineConfig({
  // ...
  wallets: [new MimirWalletProvider()],
});

PR: https://github.com/tien/reactive-dot/pull/517

Announcement by Mimir: https://x.com/Mimir_global/status/1894576922228916584

Stable React 19 suspense behavior

React 19 changed how suspense works, resulting in micro-suspense render even for resolved promises. For users, this means occasionally seeing sub-second flickers of loading state.

From countless hours researching & debugging (including on how React works internally), a solution was found and fixed via:

New features on DOTConsole

Mainly used as a testing ground for ReactiveDOT, demonstrating how little code is needed for building highly performant, light-client first DApp.

Assets: https://dotconsole.app/assets
Referenda: https://dotconsole.app/referenda
Collectives: https://dotconsole.app/collectives
Controlled validators view: https://dotconsole.app/accounts/validators

Miscellaneous external contributions

Besides working on my own projects, I also help out other projects within the ecosystem whenever possible, you can check my activities here.

Future goals

Ink contract support

Whereas chain queries are default-reactive. Contract queries are chain-agnostic and default non-reactive. Integration goals, besides basic Ink support, will be to aid developers in dealing with the contract's default-non-reactive nature. In short, beating the dev experience of the current gold standard of Contract FE DApp development: Ethereum's WAGMI.

Incremental loading

Currently, if you need to derive values based on the results of multiple queries, the UI will remain suspended until all data has fully loaded. This approach isn't ideal when you'd prefer to display partial UI content as soon as some data becomes available.

Here's a current scenario:

function Component() {
  const items = useLazyLoadQuery((builder) =>
    builder.readStorageEntries("Pallet", "Items", []),
  );

  // This causes significant UI suspension time
  const itemAmounts = useLazyLoadQuery((builder) =>
    builder.readStorages(
      "Pallet",
      "ItemAmount",
      items.map(([key]) => key),
    ),
  );

  // Ideally, we'd derive values progressively from loaded data
  const sortedItems = useMemo(
    () =>
      items
        .map((item, index) => ({ ...item, amount: itemAmounts.at(index)! }))
        .toSorted((a, b) => a.amount - b.amount),
    [items, itemAmounts],
  );

  return (
    <ol>
      {sortedItems.map((item, index) => (
        <li key={index}>{/* ... */}</li>
      ))}
    </ol>
  );
}

A better approach might involve introducing an API like this:

import { pending } from "@reactive-dot/core";

function Component() {
  const items = useLazyLoadQuery((builder) =>
    builder.readStorageEntries("Pallet", "Items", []),
  );

  const itemAmounts = useLazyLoadQuery((builder) =>
    builder.readStorages(
      "Pallet",
      "ItemAmount",
      items.map(([key]) => key),
      {
        concurrency: "wait-for-none", // Immediately returns available data without suspending; alternative strategies like `wait-for-one` could also be implemented
      },
    ),
  );

  const sortedItems = useMemo(
    () =>
      items
        .map((item, index) => {
          const itemAmount = itemAmounts.at(index)!;
          return { ...item, amount: itemAmount === pending ? 0 : itemAmount };
        })
        .toSorted((a, b) => a.amount - b.amount),
    [items, itemAmounts],
  );

  return (
    <ol>
      {sortedItems.map((item, index) => (
        <li key={index}>{/* ... */}</li>
      ))}
    </ol>
  );
}

This proposed API enables partial UI rendering, making your components more responsive and improving overall user experience.

Query composition

Current state

ReactiveDOT suspense-powered query ideal usage is as follows:

function ChildComponent({ id }) {
  const [item1, item2] = useLazyLoadQuery((builder) =>
    builder
      .readStorage("Pallet1", "Storage1", [id])
      .readStorage("Pallet2", "Storage2", [id]),
  );

  return (
    <article>
      <header>{item1}</header>
      <p>{item2}</p>
    </article>
  );
}

function ParentComponent() {
  const overviewInfo = useLazyLoadQuery((builder) =>
    builder.readStorage("Pallet3", "Storage3", []),
  );

  const ids = [
    /* ... */
  ];

  return (
    <section>
      <header>{overviewInfo}</header>

      <ul>
        {ids.map((id) => (
          <li key={id}>
            <Suspense fallback={<p>Loading item {id}</p>}>
              <ChildComponent id={id} />
            </Suspense>
          </li>
        ))}
      </ul>
    </section>
  );
}

export function App() {
  return (
    <Suspense fallback={<p>Loading</p>}>
      <ParentComponent />
    </Suspense>
  );
}

Where a component requests exactly and only what it needs, and directly map this data to UI elements. This bottom-up approach ensures components remain lean, load quickly, render fast, and are easy to understand and maintain.

However, this ideal scenario isn't always achievable because values often need to be derived or reduced from complex storage structures. Current approaches include:

Custom hooks using PAPI directly: Offers maximum flexibility and is currently the recommended approach.
Custom hooks leveraging ReactiveDOT's useLazyLoadQuery: Easy to implement and effective for simpler cases but susceptible to suspense waterfalls.
Hooks/functions derived from values provided by useLazyLoadQuery: Technically optimal but may negatively impact developer experience due to increased complexity and additional code requirements.

Planned R&D aims to provide developers with enhanced hooks and utilities, simplifying the composition of queries that are both suspendable and resistant to suspense waterfalls. This could occur either:

During render: Simpler but less performant.
Outside render: More performant but complex, requiring effective caching strategies.

Note: These challenges could potentially be resolved entirely through WASM view functions, making ReactiveDOT's ideal usage consistently effective.

Achieve greater than 90% test coverage

Currently, the test coverage is approximately ~70%, with more than 90% coverage already achieved for core logic packages (@reactive-dot/core, @reactive-dot/utils, @reactive-dot/wallet-*). The remaining gaps are primarily within the render layer (@reactive-dot/react, @reactive-dot/vue).

End-to-end (E2E) tests will likely be necessary to reach the targeted coverage. These tests may be implemented using tools like chopsticks or zombienet.

Continued improvements to DOTConnect & DOTConsole

Further enhancements will be guided by community feedback or requirements identified to support the ongoing testing and development of ReactiveDOT.

Requesting

Period: March 2025 to June 2025.
Duration: 4 months / 86 workdays / 688 hours.
Requested amount: 89,440 USDC @ 130 USDC an hour.