v1

Author: nedimf

README.md

@@ -6,12 +6,15 @@ A fast Go CLI tool that parses Apple's DocC JSON documentation output and conver
 
 - **Parses DocC JSON output** from `.doccarchive` or processed documentation folders
 - **Generates custom SDK schema** with full type information, method signatures, and documentation
+- **Comprehensive documentation extraction** - extracts abstracts, discussions, examples, and code listings from DocC comments
 - **Filters by access level** with optional `--public-only` flag
 - **Fast parallel processing** of JSON files
 - **Rich metadata extraction** including parameters, return types, async/throws modifiers
 - **Cross-reference resolution** for type relationships and protocol conformances
 - **Automatic protocol unmangling** - converts mangled Swift protocol names (e.g., `s8SendableP`, `Se`) to clean names (`Sendable`)
 - **Smart conformance filtering** - filters out stdlib protocols while keeping custom protocols
+- **Flexible grouping options** - group types by prefix or custom configuration
+- **Articles and tutorials support** - optionally include documentation articles
 - **Navigation structure generation** for easy website integration
 
 ## Installation
@@ -50,6 +53,26 @@ docc2json -i ./docs -o output.json --compact
 
 # Filter out specific protocols from conformances
 docc2json -i ./docs -o output.json --filter-protocols "Se,ScA,objc"
+
+# Group types by name prefix (e.g., Wish, WishStatus, WishFilters -> Wish group)
+docc2json -i ./docs -o output.json --group-by-prefix
+
+# Use custom grouping configuration
+docc2json -i ./docs -o output.json --group-by-prefix --group-config ./group-config.yaml
+
+# Include articles and tutorials
+docc2json -i ./docs -o output.json --include-articles
+
+# Complete example with all features
+docc2json \
+  -i ../iOS/AppGramSDK/docs/data \
+  -o sdk.json \
+  --include-articles \
+  --group-by-prefix \
+  --group-config ./group-config.yaml \
+  --filter-protocols "Se,ScA,objc,SE,SQ,SH,SY,SL" \
+  --keep-all-conformances \
+  --verbose
 ```
 
 ### Command-Line Flags
@@ -64,6 +87,9 @@ docc2json -i ./docs -o output.json --filter-protocols "Se,ScA,objc"
 | `--compact` | - | Output compact JSON (no indentation) | `false` |
 | `--filter-protocols` | - | Comma-separated list of protocol names to exclude from conformances | - |
 | `--keep-all-conformances` | - | Keep all conformances without filtering (for debugging) | `false` |
+| `--group-by-prefix` | - | Group types by name prefix (e.g., Wish, WishFilters → Wish group) | `false` |
+| `--group-config` | - | Path to YAML file with custom grouping configuration | - |
+| `--include-articles` | - | Include documentation articles and tutorials | `false` |
 | `--version` | - | Print version information | - |
 
 ## Protocol Conformances
@@ -114,6 +140,139 @@ To see all conformances without any filtering (useful for debugging):
 docc2json -i ./docs -o output.json --keep-all-conformances --verbose
 ```
 
+## Type Grouping
+
+The tool supports flexible grouping of types, protocols, and enums for better organization in documentation sites.
+
+### Automatic Prefix-Based Grouping
+
+Use `--group-by-prefix` to automatically group types by their name prefixes:
+
+```bash
+docc2json -i ./docs -o output.json --group-by-prefix
+```
+
+This will group types like:
+- `Wish`, `WishStatus`, `WishFilters` → **Wish** group
+- `Survey`, `SurveyMap`, `SurveyData` → **Survey** group
+- `AppGramSDK`, `AppGramTheme`, `AppGramConfig` → **AppGram** group
+
+### Custom Grouping with YAML Configuration
+
+For more control, create a `group-config.yaml` file:
+
+```yaml
+groups:
+  # Feedback and Wishes
+  Feedback:
+    description: "User feedback and wish management"
+    types:
+      - Wish
+      - WishFilters
+      - Vote
+      - Comment
+    enums:
+      - WishStatus
+      - WishSortOption
+    patterns:
+      - "Wish*"
+      - "*Wish"
+
+  # Surveys
+  Survey:
+    description: "Survey and mapping functionality"
+    patterns:
+      - "Survey*"
+    types:
+      - SurveyMap
+      - SurveyData
+
+  # UI Components
+  UI:
+    description: "User interface components"
+    patterns:
+      - "*View"
+      - "*Theme"
+    protocols:
+      - ViewConfigurable
+```
+
+Then use it with:
+
+```bash
+docc2json -i ./docs -o output.json --group-by-prefix --group-config ./group-config.yaml
+```
+
+**Configuration options:**
+- `types`: Explicit list of type names
+- `protocols`: Explicit list of protocol names
+- `enums`: Explicit list of enum names
+- `patterns`: Wildcard patterns (`"Prefix*"`, `"*Suffix"`, `"*Infix*"`)
+- `description`: Group description for documentation
+
+Types not matching any group will be placed in an "Other" group.
+
+## Documentation Extraction
+
+The tool fully extracts DocC documentation including:
+
+- **Abstract**: Brief one-line description
+- **Discussion**: Detailed explanation from `## Discussion` sections
+- **Examples**: Code examples from `## Example` sections with full code listings
+- **Parameters**: Parameter descriptions for methods
+- **Return values**: Return value documentation
+- **Throws**: Exception documentation
+
+### Example from Swift Source:
+
+```swift
+/// Represents a feedback item (wish) submitted by users.
+///
+/// ## Discussion
+/// Wishes are user-submitted feedback items that can be voted on and commented on.
+/// They represent feature requests, bug reports, or other feedback.
+///
+/// ## Example
+/// ```swift
+/// let wish = Wish(
+///     id: "wish123",
+///     title: "Dark mode support",
+///     description: "Please add dark mode"
+/// )
+/// ```
+public struct Wish { ... }
+```
+
+### Extracted JSON:
+
+```json
+{
+  "name": "Wish",
+  "description": {
+    "abstract": "Represents a feedback item (wish) submitted by users.",
+    "discussion": "Wishes are user-submitted feedback items that can be voted on and commented on. They represent feature requests, bug reports, or other feedback.",
+    "examples": [{
+      "title": "Example",
+      "code": "let wish = Wish(\n    id: \"wish123\",\n    title: \"Dark mode support\",\n    description: \"Please add dark mode\"\n)"
+    }]
+  }
+}
+```
+
+## Articles and Tutorials
+
+Include documentation articles and tutorials with `--include-articles`:
+
+```bash
+docc2json -i ./docs -o output.json --include-articles
+```
+
+This will include:
+- Getting started guides
+- Tutorials
+- Conceptual documentation
+- Best practices articles
+
 ## Output Schema
 
 The tool generates a JSON file with the following structure:
@@ -157,7 +316,12 @@ Each type includes:
 - **properties**: Array of property definitions
 - **methods**: Array of method definitions
 - **initializers**: Array of initializers
-- **description**: Documentation with abstract and discussion
+- **description**: Documentation object containing:
+  - **abstract**: Brief description
+  - **discussion**: Detailed explanation
+  - **examples**: Code examples with titles
+  - **returnValue**: Return value documentation (for methods)
+  - **throws**: Exception documentation (for methods)
 - **topicSections**: Grouped navigation structure
 
 ### Method Representation
@@ -244,7 +408,9 @@ Tested on AppGramSDK documentation (38,618 files):
 - **Parsing**: ~0.4 seconds (parallel)
 - **Transformation**: ~0.03 seconds
 - **Total**: < 1 second
-- **Output size**: 756KB for 171 symbols (143 types, 8 protocols, 20 enums)
+- **Output size**:
+  - Types only: ~756KB for 129 symbols (103 types, 8 protocols, 18 enums)
+  - With articles: ~1.2MB for 129 symbols + 92 articles
 
 ## Architecture
 
@@ -277,9 +443,10 @@ docc2json/
 
 ## Tested With
 
-- **AppGramSDK** - 38,618 documentation files, 171 public symbols
+- **AppGramSDK** - 38,618 documentation files, 129 symbols + 92 articles
 - **Swift 5.9+** DocC output
 - **DocC schema version** 0.3.0
+- **iOS SDK documentation** with full discussion and example extraction
 
 ## Requirements
 
@@ -298,6 +465,19 @@ docc2json/
 - **SDK Analysis**: Programmatically analyze SDK structure
 - **Cross-Platform Docs**: Generate documentation for other platforms
 
+## Recent Improvements
+
+### v1.1.0 (2025-12-28)
+
+- **Fixed Discussion extraction**: Corrected heading text extraction from DocC JSON to properly capture Discussion, Example, and other documentation sections
+- **Added flexible grouping**: Support for automatic prefix-based grouping and custom YAML configuration
+- **Articles support**: Optional inclusion of documentation articles and tutorials
+- **Enhanced documentation**: Full extraction of abstracts, discussions, and code examples from DocC comments
+
+### Key Bug Fixes
+
+- **Discussion sections**: Fixed issue where DocC heading text was incorrectly extracted from `InlineContent` instead of `Text` field, causing Discussion and Example sections to be lost
+
 ## License
 
 MIT License
@@ -308,4 +488,4 @@ Built by [AppGram](https://github.com/AppGram) for converting iOS SDK documentat
 
 ## Version
 
-1.0.0
+1.1.0 - Enhanced documentation extraction and flexible grouping (2025-12-28)