docs: 更新 add-trending 命令文档并启用代理配置
- 更新 add-trending 命令描述为英文，强调使用 Agent 工作流 - 文档中明确所有数据处理由专用 Agent 完成，主窗口仅负责协调 - 启用 Claude Code 代理设置（HTTP_PROXY/HTTPS_PROXY） - 添加 .trending-workspace/ 到 .gitignore - 完善 CLAUDE.md 文档：数据库约束、级联删除、tag 处理逻辑、新增组件等 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude
@@ -1,5 +1,5 @@
|
|||||||
---
|
---
|
||||||
description: 从多个数据源（GitHub Trending、Hugging Face 等）自动获取 AI 相关项目并入库
|
description: Use the specialized agents workflow to automatically fetch AI projects from multiple data sources (GitHub Trending, Hugging Face, etc.) and ingest them into the database
|
||||||
---
|
---
|
||||||
|
|
||||||
## 用户输入
|
## 用户输入
|
||||||
@@ -12,6 +12,11 @@ $ARGUMENTS
|
|||||||
|
|
||||||
## 概述
|
## 概述
|
||||||
|
|
||||||
|
本命令使用 **专用 Agent 工作流** 来完成多源数据获取和入库任务。核心原则：
|
||||||
|
- **所有数据处理逻辑由 Agent 执行**，不在主窗口执行
|
||||||
|
- **使用 Task 工具调用各个 Agent**，让 Agent 自主完成其职责
|
||||||
|
- **主窗口仅负责协调 Agent 调用**，不直接处理业务逻辑
|
||||||
|
|
||||||
本命令通过任务派发器架构，从多个数据源并行爬取 AI 相关项目，经过去重、分析、质量评分后批量入库。
|
本命令通过任务派发器架构，从多个数据源并行爬取 AI 相关项目，经过去重、分析、质量评分后批量入库。
|
||||||
|
|
||||||
**命令格式**: `/add-trending [source] [period] [limit]`
|
**命令格式**: `/add-trending [source] [period] [limit]`
|
||||||
@@ -29,7 +34,7 @@ $ARGUMENTS
|
|||||||
|
|
||||||
## 执行流程
|
## 执行流程
|
||||||
|
|
||||||
**重要**: 每个 Stage 都要**读取并执行对应 Agent 文件中定义的逻辑**。
|
**重要**: 本命令使用专用的 Agent 架构处理工作流，由 Agent 之间相互调用完成整个流程。
|
||||||
|
|
||||||
### Stage 1: 初始化与任务派发
|
### Stage 1: 初始化与任务派发
|
||||||
|
|
||||||
@@ -42,8 +47,7 @@ $ARGUMENTS
|
|||||||
- 初始化 `progress.json` 文件
|
- 初始化 `progress.json` 文件
|
||||||
|
|
||||||
3. **执行任务派发器**:
|
3. **执行任务派发器**:
|
||||||
- **读取** `.claude/agents/task-dispatcher.md`
|
- **使用 Task 工具调用** `task-dispatcher` agent
|
||||||
- **按照该文件中定义的步骤**执行任务派发逻辑
|
|
||||||
- 将 source/period/limit/workspace 参数传递给任务派发器
|
- 将 source/period/limit/workspace 参数传递给任务派发器
|
||||||
- 任务派发器并行调度对应的爬虫
|
- 任务派发器并行调度对应的爬虫
|
||||||
- 输出 `raw-projects.json`
|
- 输出 `raw-projects.json`
|
||||||
@@ -51,18 +55,18 @@ $ARGUMENTS
|
|||||||
### Stage 2: 统一去重
|
### Stage 2: 统一去重
|
||||||
|
|
||||||
1. **执行去重器**:
|
1. **执行去重器**:
|
||||||
- **读取** `.claude/agents/deduplicator.md`
|
- **使用 Task 工具调用** `deduplicator` agent
|
||||||
- **按照该文件中定义的步骤**执行去重逻辑
|
- 将 workspace 参数传递给去重器
|
||||||
- 读取 `raw-projects.json`
|
- 去重器读取 `raw-projects.json`
|
||||||
- 使用 **dbhub PostgreSQL MCP** 执行去重查询
|
- 使用 **dbhub PostgreSQL MCP** 执行去重查询
|
||||||
- 输出 `new-projects.json` 和 `task-queue.json`
|
- 输出 `new-projects.json` 和 `task-queue.json`
|
||||||
|
|
||||||
### Stage 3: 项目分析
|
### Stage 3: 项目分析
|
||||||
|
|
||||||
1. **执行项目分析器**:
|
1. **执行项目分析器**:
|
||||||
- **读取** `.claude/agents/project-analyzer.md`
|
- **使用 Task 工具调用** `project-analyzer` agent
|
||||||
- **按照该文件中定义的步骤**执行分析逻辑
|
- 将 workspace 参数传递给分析器
|
||||||
- 处理任务队列中的项目
|
- 分析器处理任务队列中的项目
|
||||||
- 使用 **chrome-devtools-mcp** 访问项目页面
|
- 使用 **chrome-devtools-mcp** 访问项目页面
|
||||||
- 生成中英双语内容
|
- 生成中英双语内容
|
||||||
- 计算质量评分
|
- 计算质量评分
|
||||||
@@ -71,8 +75,8 @@ $ARGUMENTS
|
|||||||
### Stage 4: 批量入库
|
### Stage 4: 批量入库
|
||||||
|
|
||||||
1. **执行入库器**:
|
1. **执行入库器**:
|
||||||
- **读取** `.claude/agents/database-ingestor.md`
|
- **使用 Task 工具调用** `database-ingestor` agent
|
||||||
- **按照该文件中定义的步骤**执行入库逻辑
|
- 将 workspace 参数传递给入库器
|
||||||
- 调用 `POST /api/webhook/projects`
|
- 调用 `POST /api/webhook/projects`
|
||||||
- 输出 `ingestion-result.json`
|
- 输出 `ingestion-result.json`
|
||||||
|
|
||||||
@@ -103,7 +107,8 @@ $ARGUMENTS
|
|||||||
|
|
||||||
## 关键规则
|
## 关键规则
|
||||||
|
|
||||||
- **必须**使用 chrome-devtools-mcp 访问数据源网站
|
- **必须使用 Agent 工作流**：所有数据处理由专用 Agent 完成，不在主窗口执行
|
||||||
|
- **必须使用 Task 工具**调用 Agent：让 Agent 自主完成其职责
|
||||||
- **必须**先去重再分析，避免处理已存在的项目
|
- **必须**先去重再分析，避免处理已存在的项目
|
||||||
- **必须**进行质量评分，仅入库 >= 40 分的项目
|
- **必须**进行质量评分，仅入库 >= 40 分的项目
|
||||||
- **必须**保留工作区 7 天用于调试和审计
|
- **必须**保留工作区 7 天用于调试和审计
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
// {
|
{
|
||||||
// "env": {
|
"env": {
|
||||||
// "HTTP_PROXY": "http://proxy3.bj.petrochina:8080",
|
"HTTP_PROXY": "http://proxy3.bj.petrochina:8080",
|
||||||
// "HTTPS_PROXY": "http://proxy3.bj.petrochina:8080"
|
"HTTPS_PROXY": "http://proxy3.bj.petrochina:8080"
|
||||||
// }
|
}
|
||||||
// }
|
}
|
||||||
|
|||||||
@@ -60,6 +60,7 @@ logs/
|
|||||||
tmp/
|
tmp/
|
||||||
temp/
|
temp/
|
||||||
.cache/
|
.cache/
|
||||||
|
.trending-workspace/
|
||||||
|
|
||||||
# Package manager lock files (optional - uncomment if needed)
|
# Package manager lock files (optional - uncomment if needed)
|
||||||
# package-lock.json
|
# package-lock.json
|
||||||
|
|||||||
@@ -52,28 +52,40 @@ src/app/
|
|||||||
```
|
```
|
||||||
|
|
||||||
### Database (Prisma + PostgreSQL)
|
### Database (Prisma + PostgreSQL)
|
||||||
- **Schema**: `prisma/schema.prisma` defines models: `Project`, `Tag`, `ExternalLink`
|
- **Schema**: `prisma/schema.prisma` defines models: `Project`, `Tag`, `ExternalLink`, `ProjectTag`
|
||||||
- **Enums**: `ProjectStatus` (ACTIVE/ARCHIVED), `LinkType` (WEBSITE/GITHUB/HUGGINGFACE/PAPER)
|
- **Enums**: `ProjectStatus` (ACTIVE/ARCHIVED), `LinkType` (WEBSITE/GITHUB/HUGGINGFACE/PAPER)
|
||||||
- **Client singleton**: `src/lib/prisma.ts` exports Prisma client instance
|
- **Client singleton**: `src/lib/prisma.ts` exports Prisma client instance
|
||||||
- **Multilingual fields**: Most models have `name`/`nameEn`, `description`/`descriptionEn`, `content`/`contentEn` pairs
|
- **Multilingual fields**: Most models have `name`/`nameEn`, `description`/`descriptionEn`, `content`/`contentEn` pairs
|
||||||
- **Indexes**: `idx_project_status_createdAt`, `idx_project_slug`, `idx_tag_slug`, `idx_link_projectId`, `idx_link_type`
|
- **Cascade deletions**: `ExternalLink` and `ProjectTag` use `onDelete: Cascade` - deleting a project automatically cleans up its links and tag connections
|
||||||
|
- **Key constraints**:
|
||||||
|
- `Project.slug`: Unique
|
||||||
|
- `Tag.name`: Unique (tag names are globally unique)
|
||||||
|
- `Tag.slug`: Unique
|
||||||
|
- `ExternalLink`: `@@unique([projectId, url])` (each project can't have duplicate URLs)
|
||||||
|
- **Indexes**: `idx_project_status_createdAt`, `idx_project_slug`, `idx_tag_slug`, `idx_link_projectId`, `idx_link_type`, `idx_link_type_url` (composite for efficient URL-based deduplication)
|
||||||
|
|
||||||
### Webhook Deduplication Strategy
|
### Webhook Deduplication Strategy
|
||||||
The webhook at `src/app/api/webhook/projects/route.ts` implements a **multi-level deduplication** strategy to prevent duplicate projects:
|
The webhook at `src/app/api/webhook/projects/route.ts` implements a **multi-level deduplication** strategy to prevent duplicate projects:
|
||||||
1. **GitHub URL exact match** (highest priority) - via `ExternalLink` table
|
1. **GitHub URL exact match** (highest priority) - via `ExternalLink` table using `idx_link_type_url` index
|
||||||
2. **Website URL exact match** - via `ExternalLink` table
|
2. **Website URL exact match** - via `ExternalLink` table using `idx_link_type_url` index
|
||||||
3. **slug match** (fallback) - via `Project.slug` field
|
3. **slug match** (fallback) - via `Project.slug` field
|
||||||
|
|
||||||
When updating an existing project, the webhook:
|
When updating an existing project, the webhook:
|
||||||
- Updates all project fields (name, description, content, status, source)
|
- Updates all project fields (name, description, content, status, source)
|
||||||
- Replaces all tags (deletes old connections, creates new ones)
|
- Replaces all tags (deletes old `ProjectTag` connections via `ProjectTag` table, creates new ones)
|
||||||
- Replaces all links (deletes old links, creates new ones)
|
- Replaces all links (deletes old `ExternalLink` entries, creates new ones)
|
||||||
|
|
||||||
|
**Tag handling special case**: Due to `Tag.name` unique constraint, tag upsert follows:
|
||||||
|
1. First try to find existing tag by name
|
||||||
|
2. If not found, try upsert by slug
|
||||||
|
3. If slug conflicts, use the existing tag with that slug
|
||||||
|
|
||||||
### Data Fetching (Server-Side)
|
### Data Fetching (Server-Side)
|
||||||
- **Location**: `src/hooks/useProjects.ts` (server functions, not React hooks)
|
- **Location**: `src/hooks/useProjects.ts` (server functions, not React hooks)
|
||||||
- **Functions**: `getProjects()`, `getProjectBySlug()`, `getAllTags()`, `getTagsWithProjectCounts()`
|
- **Functions**: `getProjects()`, `getProjectBySlug()`, `getAllTags()`, `getTagsWithProjectCounts()`
|
||||||
- **Usage**: Directly called in Server Components and route handlers
|
- **Usage**: Directly called in Server Components and route handlers
|
||||||
- **ISR**: Project detail pages use `export const revalidate = 300` (5 minutes)
|
- **Query transformation**: `getProjects()` and `getProjectBySlug()` flatten the `ProjectTag` junction table structure to return tags directly
|
||||||
|
- **ISR**: Project detail pages use `export const revalidate = 300` (5 minutes) at `src/app/[locale]/projects/[id]/page.tsx`
|
||||||
|
|
||||||
### Validation (Zod)
|
### Validation (Zod)
|
||||||
- **Schemas**: `src/lib/validations.ts` defines all Zod schemas
|
- **Schemas**: `src/lib/validations.ts` defines all Zod schemas
|
||||||
@@ -101,9 +113,9 @@ When updating an existing project, the webhook:
|
|||||||
- **Radix UI primitives**: `@radix-ui/react-slot`, `@radix-ui/react-navigation-menu`, `@radix-ui/react-dropdown-menu`, `@radix-ui/react-separator`
|
- **Radix UI primitives**: `@radix-ui/react-slot`, `@radix-ui/react-navigation-menu`, `@radix-ui/react-dropdown-menu`, `@radix-ui/react-separator`
|
||||||
- **Lucide React icons**: Used throughout the app (package imports optimized via `experimental.optimizePackageImports`)
|
- **Lucide React icons**: Used throughout the app (package imports optimized via `experimental.optimizePackageImports`)
|
||||||
- **Custom components**: `src/components/` organized by domain
|
- **Custom components**: `src/components/` organized by domain
|
||||||
- `layout/`: Header, Footer
|
- `layout/`: Header, Footer, AnnouncementBar
|
||||||
- `locale/`: LocaleSwitcher
|
- `locale/`: LocaleSwitcher
|
||||||
- `project/`: ProjectCard, ProjectList, ProjectDetail, ProjectSidebar, RelatedProjects, TagCloud, ExternalLinkCard
|
- `project/`: ProjectCard, ProjectList, ProjectDetail, ProjectSidebar, RelatedProjects, TagCloud, ExternalLinkCard, ShareButtons, MarkdownContent, GitHubBadges, GitHubTextStatsCard
|
||||||
- `search/`: SearchBar
|
- `search/`: SearchBar
|
||||||
- `ui/`: Base UI components (buttons, cards, etc.)
|
- `ui/`: Base UI components (buttons, cards, etc.)
|
||||||
|
|
||||||
@@ -113,18 +125,23 @@ When updating an existing project, the webhook:
|
|||||||
- Image domains: localhost, *.anthropic.com
|
- Image domains: localhost, *.anthropic.com
|
||||||
- Lucide-react package import optimization
|
- Lucide-react package import optimization
|
||||||
- **tsconfig.json**: ES2022 target, strict mode enabled
|
- **tsconfig.json**: ES2022 target, strict mode enabled
|
||||||
|
- **Testing**: Vitest for unit tests, Playwright for E2E tests (configured but not extensively used yet)
|
||||||
|
|
||||||
## MCP Servers Usage (按需使用)
|
## MCP Servers Usage (按需使用)
|
||||||
|
|
||||||
1. **context7**: 不确定 API 用法时查阅最新文档
|
1. **context7**: 不确定 API 用法时查阅最新文档
|
||||||
2. **chrome-devtools-mcp**: 查看页面效果、调试 UI 修复 BUG
|
2. **chrome-devtools-mcp**: 查看页面效果、调试 UI 修复 BUG
|
||||||
3. **dbhub mcp**: 查询数据库数据（PostgreSQL）
|
3. **web-search-prime**: 默认联网搜索工具
|
||||||
4. **shadcn mcp**: 确认 shadcn/ui 组件用法
|
4. **vision-mcp-server**: 图片/视频理解
|
||||||
5. **web-search-prime**: 默认联网搜索工具
|
|
||||||
6. **vision-mcp-server**: 图片/视频理解
|
|
||||||
|
|
||||||
## Git Commits
|
## Git Commits
|
||||||
|
|
||||||
提交信息主要使用中文，使用描述性的提交格式。
|
提交信息主要使用中文，使用描述性的提交格式。
|
||||||
|
|
||||||
如果你需要查看页面 请不要直接启动项目 而是应该先检查是否已经有启动好的项目在localhost:3000运行。如果有的话请直接访问，如果没有的话再执行相关命令启动项目。
|
## Development Workflow
|
||||||
|
|
||||||
|
- Before viewing the page, check if there's already a project running at localhost:3000. If yes, access it directly; if no, then run `pnpm dev`
|
||||||
|
- The dev server runs with hot-reload enabled for fast iteration
|
||||||
|
- The middleware handles locale detection and routing automatically - no manual locale configuration needed
|
||||||
|
- When adding new translations, update both `src/messages/zh.json` and `src/messages/en.json`
|
||||||
|
- Database changes require running `pnpm prisma migrate dev` to update the schema
|
||||||
|
|||||||
Reference in New Issue
Block a user