Commit 22d2382
authored
feat: Delta-based streaming (#6)
* feat: implement delta-based streaming with O(n) bandwidth efficiency
## Major Changes
### Delta-Based Streaming Architecture
- Replace O(n²) accumulated content streaming with O(n) delta-based streaming
- Add `streamingMessages` table for stream lifecycle tracking
- Add `streamDeltas` table for storing incremental text chunks
- Implement `DeltaStreamer` class for batched, throttled delta writes
### New Stream API
- `stream.create` - Create a new stream for a conversation
- `stream.addDelta` - Add incremental content (returns false if aborted)
- `stream.finish` - Mark stream complete, clean up deltas
- `stream.abort` / `stream.abortByConversation` - Cancel generation
- `stream.getStream` - Get stream state (streaming/finished/aborted)
- `stream.listDeltas` - Fetch deltas from cursor position
### Abort Support
- Users can stop generation mid-stream with proper cleanup
- Stream abort propagates to LLM via early return from addDelta
- React hooks handle abort state to prevent error display
- Added `wasAbortedRef` pattern to ignore aborted request results
### Text Smoothing
- Add `useSmoothText` hook for typewriter effect on streaming text
- Add `SmoothText` component for easy integration
- Adaptive speed that matches text arrival rate
- Configurable chars/second, min/max delay
### Removed Backwards Compatibility
- Remove deprecated `stream.init`, `stream.update`, `stream.clear`
- Remove deprecated `stream.getContent` query
- Remove `getStreamingContent` from client wrapper
- Remove `StreamingState` type (use `StreamState` instead)
- Remove legacy tests
### Example App Improvements
- Implement Stop button with immediate feedback
- Add `isStopping` state for better UX
- Localhost detection to bypass rate limits in dev
- Filter abort-related errors from display
### Documentation
- Comprehensive README update with delta streaming examples
- Document new streaming API and abort functionality
- Add text smoothing usage examples
- Update API reference table
## Files Changed
- convex/component/stream.ts - New delta-based streaming mutations/queries
- convex/component/deltaStreamer.ts - DeltaStreamer class (new file)
- convex/component/schema.ts - Add streamingMessages, streamDeltas tables
- convex/component/client.ts - Remove deprecated methods, update docs
- src/react.tsx - Delta accumulation, abort handling, useSmoothText
- src/index.ts - Export new hooks and components
- example/ - Updated to use delta streaming with abort support
- README.md - Complete documentation overhaul
* chore: add convex generated files for type safety
Include convex/_generated/ files to enable TypeScript type checking
and editor autocomplete without requiring npx convex dev to be run.1 parent 019d29b commit 22d2382
29 files changed
Lines changed: 3085 additions & 525 deletions
File tree
- .cursor/rules
- convex
- _generated
- component
- example
- convex
- _generated
- src
- components
- hooks
- src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
198 | 198 | | |
199 | 199 | | |
200 | 200 | | |
201 | | - | |
| 201 | + | |
202 | 202 | | |
203 | 203 | | |
204 | 204 | | |
| |||
236 | 236 | | |
237 | 237 | | |
238 | 238 | | |
239 | | - | |
240 | | - | |
| 239 | + | |
| 240 | + | |
241 | 241 | | |
242 | 242 | | |
243 | 243 | | |
| |||
307 | 307 | | |
308 | 308 | | |
309 | 309 | | |
310 | | - | |
| 310 | + | |
311 | 311 | | |
312 | 312 | | |
313 | 313 | | |
| |||
434 | 434 | | |
435 | 435 | | |
436 | 436 | | |
437 | | - | |
| 437 | + | |
438 | 438 | | |
439 | 439 | | |
440 | 440 | | |
| |||
667 | 667 | | |
668 | 668 | | |
669 | 669 | | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
670 | 699 | | |
671 | 700 | | |
672 | 701 | | |
| |||
0 commit comments