Skip to content

Commit dc19203

Browse files
babblebeybabblebey
authored
doc: add doc for vector store ping to dev/script README (#195)
## Description <!-- Please add PR description (don't leave blank) - example: This PR [adds/removes/fixes/replaces] the [feature/bug/etc] --> This pull request updates the `dev/README.md` documentation to clarify environment variable requirements and adds comprehensive instructions for a new Qdrant cluster health check script. The changes focus on improving developer onboarding, usage clarity, and maintaining the Qdrant vector store for ✨jAI. **Documentation improvements:** * Updated environment variable requirements for running scripts, specifying `OPENAI_API_KEY`, `QDRANT_URL`, and `QDRANT_API_KEY` for clarity and correctness. * Added explicit documentation for required environment variables under the configuration section, detailing each variable's purpose and usage. **Qdrant cluster health check script:** * Introduced a new section describing the Vector Store Cluster Ping Script, including its purpose, when to use it, prerequisites, and usage instructions for both local development and CI/CD pipelines. * Provided detailed technical implementation notes, error handling scenarios, automated scheduling via GitHub Actions, and example output to help developers debug and maintain cluster connectivity. ## Related Issue <!-- Please prefix the issue number with Fixes/Resolves - example: Fixes #123 or Resolves #123 --> - #183 - #181 - #192 - #193 - #194 ## Screenshots/Screencasts <!-- Please provide screenshots or video recording that demos your changes (especially if it's a visual change) --> NA ## Notes to Reviewer <!-- Please state here if you added a new npm packages, or any extra information that can help reviewer better review you changes --> NA
1 parent d42cca3 commit dc19203

1 file changed

Lines changed: 105 additions & 1 deletion

File tree

‎dev/README.md‎

Lines changed: 105 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ Run this script when you need to:
3838

3939
Before running this script, ensure you have:
4040
- All dependencies installed (`npm ci`)
41-
- OPENAI and QDRANT environment variables properly configured in your `.env` file
41+
- `OPENAI_API_KEY`, `QDRANT_URL` and `QDRANT_API_KEY` environment variables properly configured in your `.env` file
4242
- Network access to fetch from jargons.dev API
4343
- Sufficient disk space for temporary dictionary file
4444

@@ -69,6 +69,12 @@ The script leverages several key technologies:
6969

7070
### Configuration Options
7171

72+
Required environment variables:
73+
74+
- **QDRANT_URL**: Your Qdrant cluster endpoint (e.g., `https://your-cluster.gcp.cloud.qdrant.io`)
75+
- **QDRANT_API_KEY**: Your Qdrant cluster API key for authentication
76+
- **OPENAI_API_KEY**: Your OpenAI API Key with appopriate chat and embedding models (value specified in `OPENAI_CHAT_MODEL` and `OPENAI_EMBEDDINGS_MODEL`) allowed
77+
7278
Key parameters that can be adjusted:
7379

7480
- **Chunk Size**: Currently 1000 characters (optimal for most search queries)
@@ -98,6 +104,104 @@ Cleaned up the dictionary file at /path/to/dev/dictionary.json
98104

99105
Once completed, ✨jAI will have access to the processed dictionary content and can provide intelligent responses about software engineering terms.
100106

107+
## Vector Store Cluster Ping Script
108+
109+
This script performs a lightweight health check on the Vector Store (Qdrant) cluster to keep it active and prevent automatic deletion due to inactivity. It's designed to be run both locally for testing and automatically via GitHub Actions.
110+
111+
### When to Use
112+
113+
Run this script when you need to:
114+
- Test Qdrant cluster connectivity and collection status
115+
- Keep the cluster active to prevent auto-deletion from inactivity
116+
- Verify that the 'dictionary' collection exists and is accessible
117+
- Debug vector store connection issues during development
118+
119+
### Prerequisites
120+
121+
Before running this script, ensure you have:
122+
- `QDRANT_URL` and `QDRANT_API_KEY` environment variables configured
123+
- Network access to your Qdrant cluster
124+
- The 'dictionary' collection exists in your Qdrant instance
125+
126+
### Usage
127+
128+
**Local Development:**
129+
```bash
130+
npm run ping:qdrant
131+
```
132+
133+
**CI/CD (without .env file):**
134+
```bash
135+
npm run ping:qdrant:ci
136+
```
137+
138+
### How It Works
139+
140+
The script performs these health checks:
141+
142+
1. **Environment Validation**: Verifies required environment variables are set
143+
2. **Basic Connectivity**: Tests the cluster endpoint with a health check request
144+
3. **Collections Check**: Retrieves and validates the existence of the 'dictionary' collection
145+
4. **Status Reporting**: Provides detailed success/failure feedback with helpful error hints
146+
147+
### Technical Implementation
148+
149+
The script uses several key approaches:
150+
151+
- **Direct REST API Calls**: Uses native fetch() for lightweight, dependency-free requests
152+
- **Comprehensive Error Handling**: Provides specific error messages and troubleshooting hints
153+
- **Environment Variable Validation**: Checks configuration before attempting connections
154+
- **Collection Verification**: Ensures the required 'dictionary' collection exists
155+
156+
### Configuration
157+
158+
Required environment variables:
159+
160+
- **QDRANT_URL**: Your Qdrant cluster endpoint (e.g., `https://your-cluster.gcp.cloud.qdrant.io`)
161+
- **QDRANT_API_KEY**: Your Qdrant cluster API key for authentication
162+
- **COLLECTION_NAME**: Hardcoded to "dictionary" (the collection ✨jAI uses)
163+
164+
### Automated Scheduling
165+
166+
The script is automatically run via GitHub Actions:
167+
- **Schedule**: Every Sunday at 2 AM UTC
168+
- **Manual Trigger**: Can be run manually from GitHub Actions tab
169+
- **Purpose**: Prevents cluster deletion due to inactivity
170+
171+
### Error Handling
172+
173+
The script includes detailed error handling for:
174+
- Missing or invalid environment variables
175+
- Network connectivity issues
176+
- Authentication failures (401 Unauthorized)
177+
- Missing collections
178+
- API response parsing errors
179+
180+
### Example Output
181+
182+
**Successful Run:**
183+
```
184+
🚀 Starting Vector Store (Qdrant) cluster ping...
185+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
186+
🔍 Checking Vector Store (Qdrant) cluster connectivity...
187+
✅ Cluster is accessible
188+
✅ SUCCESS: 'dictionary' collection exists
189+
📊 Total collections: 1
190+
🎯 Cluster ping completed successfully at 2025-09-16T10:30:00.000Z
191+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
192+
🎉 Cluster ping completed successfully!
193+
```
194+
195+
**Error Example:**
196+
```
197+
🚀 Starting Vector Store (Qdrant) cluster ping...
198+
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
199+
❌ ERROR: QDRANT_URL environment variable is not set
200+
💡 Make sure QDRANT_URL is configured in your environment or GitHub secrets
201+
```
202+
203+
This script ensures your Qdrant cluster remains active and accessible for ✨jAI's vector search functionality.
204+
101205
## Format-Staged Script
102206

103207
This script provides a cross-platform solution for formatting only the files that are staged in Git, making it perfect for pre-commit workflows without requiring external dependencies like Husky or lint-staged.

0 commit comments

Comments
 (0)