Skip to content

Commit 22d2382

Browse files
authored
feat: Delta-based streaming (#6)
* feat: implement delta-based streaming with O(n) bandwidth efficiency ## Major Changes ### Delta-Based Streaming Architecture - Replace O(n²) accumulated content streaming with O(n) delta-based streaming - Add `streamingMessages` table for stream lifecycle tracking - Add `streamDeltas` table for storing incremental text chunks - Implement `DeltaStreamer` class for batched, throttled delta writes ### New Stream API - `stream.create` - Create a new stream for a conversation - `stream.addDelta` - Add incremental content (returns false if aborted) - `stream.finish` - Mark stream complete, clean up deltas - `stream.abort` / `stream.abortByConversation` - Cancel generation - `stream.getStream` - Get stream state (streaming/finished/aborted) - `stream.listDeltas` - Fetch deltas from cursor position ### Abort Support - Users can stop generation mid-stream with proper cleanup - Stream abort propagates to LLM via early return from addDelta - React hooks handle abort state to prevent error display - Added `wasAbortedRef` pattern to ignore aborted request results ### Text Smoothing - Add `useSmoothText` hook for typewriter effect on streaming text - Add `SmoothText` component for easy integration - Adaptive speed that matches text arrival rate - Configurable chars/second, min/max delay ### Removed Backwards Compatibility - Remove deprecated `stream.init`, `stream.update`, `stream.clear` - Remove deprecated `stream.getContent` query - Remove `getStreamingContent` from client wrapper - Remove `StreamingState` type (use `StreamState` instead) - Remove legacy tests ### Example App Improvements - Implement Stop button with immediate feedback - Add `isStopping` state for better UX - Localhost detection to bypass rate limits in dev - Filter abort-related errors from display ### Documentation - Comprehensive README update with delta streaming examples - Document new streaming API and abort functionality - Add text smoothing usage examples - Update API reference table ## Files Changed - convex/component/stream.ts - New delta-based streaming mutations/queries - convex/component/deltaStreamer.ts - DeltaStreamer class (new file) - convex/component/schema.ts - Add streamingMessages, streamDeltas tables - convex/component/client.ts - Remove deprecated methods, update docs - src/react.tsx - Delta accumulation, abort handling, useSmoothText - src/index.ts - Export new hooks and components - example/ - Updated to use delta streaming with abort support - README.md - Complete documentation overhaul * chore: add convex generated files for type safety Include convex/_generated/ files to enable TypeScript type checking and editor autocomplete without requiring npx convex dev to be run.
1 parent 019d29b commit 22d2382

29 files changed

Lines changed: 3085 additions & 525 deletions

.cursor/rules/convex_rules.mdc

Lines changed: 34 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,7 @@ export const exampleQuery = query({
198198
handler: async (ctx, args) => {
199199
const idToUsername: Record<Id<"users">, string> = {};
200200
for (const userId of args.userIds) {
201-
const user = await ctx.db.get(userId);
201+
const user = await ctx.db.get("users", userId);
202202
if (user) {
203203
idToUsername[user._id] = user.username;
204204
}
@@ -236,8 +236,8 @@ const messages = await ctx.db
236236

237237

238238
## Mutation guidelines
239-
- Use `ctx.db.replace` to fully replace an existing document. This method will throw an error if the document does not exist.
240-
- Use `ctx.db.patch` to shallow merge updates into an existing document. This method will throw an error if the document does not exist.
239+
- Use `ctx.db.replace` to fully replace an existing document. This method will throw an error if the document does not exist. Syntax: `await ctx.db.replace('tasks', taskId, { name: 'Buy milk', completed: false })`
240+
- Use `ctx.db.patch` to shallow merge updates into an existing document. This method will throw an error if the document does not exist. Syntax: `await ctx.db.patch('tasks', taskId, { completed: true })`
241241

242242
## Action guidelines
243243
- Always add `"use node";` to the top of files containing actions that use Node.js built-in modules.
@@ -307,7 +307,7 @@ export const exampleQuery = query({
307307
args: { fileId: v.id("_storage") },
308308
returns: v.null(),
309309
handler: async (ctx, args) => {
310-
const metadata: FileMetadata | null = await ctx.db.system.get(args.fileId);
310+
const metadata: FileMetadata | null = await ctx.db.system.get("_storage", args.fileId);
311311
console.log(metadata);
312312
return null;
313313
},
@@ -434,7 +434,7 @@ Internal Functions:
434434
"description": "This example shows how to build a chat app without authentication.",
435435
"version": "1.0.0",
436436
"dependencies": {
437-
"convex": "^1.17.4",
437+
"convex": "^1.31.2",
438438
"openai": "^4.79.0"
439439
},
440440
"devDependencies": {
@@ -667,6 +667,35 @@ export default defineSchema({
667667
});
668668
```
669669

670+
#### convex/tsconfig.json
671+
```typescript
672+
{
673+
/* This TypeScript project config describes the environment that
674+
* Convex functions run in and is used to typecheck them.
675+
* You can modify it, but some settings required to use Convex.
676+
*/
677+
"compilerOptions": {
678+
/* These settings are not required by Convex and can be modified. */
679+
"allowJs": true,
680+
"strict": true,
681+
"moduleResolution": "Bundler",
682+
"jsx": "react-jsx",
683+
"skipLibCheck": true,
684+
"allowSyntheticDefaultImports": true,
685+
686+
/* These compiler options are required by Convex */
687+
"target": "ESNext",
688+
"lib": ["ES2021", "dom"],
689+
"forceConsistentCasingInFileNames": true,
690+
"module": "ESNext",
691+
"isolatedModules": true,
692+
"noEmit": true
693+
},
694+
"include": ["./**/*"],
695+
"exclude": ["./_generated"]
696+
}
697+
```
698+
670699
#### src/App.tsx
671700
```typescript
672701
export default function App() {

0 commit comments

Comments
 (0)