217 lines
20 KiB
Markdown
217 lines
20 KiB
Markdown
# Codebase Concerns
|
|
|
|
**Analysis Date:** 2026-04-20
|
|
|
|
## Tech Debt
|
|
|
|
**Oversized mixed-responsibility modules:**
|
|
- Issue: Database access, retry policy, cache fallback, query normalization, and response shaping are combined in the same modules instead of being split into smaller server-side layers.
|
|
- Files: `src/hooks/useProjects.ts`, `src/hooks/useHome.ts`, `src/app/[locale]/projects/ProjectsResultsClient.tsx`, `src/components/project/TagFilterPanel.tsx`, `src/components/signals/SignalFeedClient.tsx`, `src/app/[locale]/layout.tsx`
|
|
- Impact: Safe changes require understanding unrelated concerns at once, so regressions in caching, pagination, or rendering are easy to introduce.
|
|
- Fix approach: Split server data access into `src/lib/` or `src/server/`, keep client state in client components only, and extract URL/query helpers plus cache wrappers into dedicated modules.
|
|
|
|
**Server-only data modules are mislabeled as hooks:**
|
|
- Issue: `src/hooks/useProjects.ts` and `src/hooks/useHome.ts` are not React hooks. They contain Prisma queries, `unstable_cache`, retry logic, and server-only fallback behavior.
|
|
- Files: `src/hooks/useProjects.ts`, `src/hooks/useHome.ts`
|
|
- Impact: The naming invites accidental client imports and hides the real server/client boundary during refactors.
|
|
- Fix approach: Move these modules to a server-oriented location and reserve `src/hooks/` for actual React hooks.
|
|
|
|
**n8n workflow state is documented, but executable exports are still missing:**
|
|
- Issue: The repository now carries rich n8n metadata and generated context, but `docs/integrations/n8n/exports/` contains only `.gitkeep` and every workflow entry currently records `Export file: not recorded`.
|
|
- Files: `docs/integrations/n8n/README.md`, `docs/integrations/n8n/registry.json`, `docs/integrations/n8n/CONTEXT.generated.md`, `docs/integrations/n8n/exports/.gitkeep`, `scripts/generate-n8n-context.mjs`
|
|
- Impact: The repo captures contracts and touchpoints, but not the actual workflow logic. Incident response, review, and reproducibility still depend on external n8n access.
|
|
- Fix approach: Commit workflow exports alongside `registry.json`, treat them as versioned artifacts, and keep `pnpm n8n:context` as a verification step instead of the primary source of truth.
|
|
|
|
**Repository instructions drift from the actual toolchain:**
|
|
- Issue: `AGENTS.md` documents `pnpm test:e2e` and an `e2e/` suite, but `package.json` has no `test:e2e` script and the repo contains no `playwright.config.*` or `e2e/` directory.
|
|
- Files: `AGENTS.md`, `package.json`
|
|
- Impact: Contributors can assume browser-level regression coverage exists when it does not.
|
|
- Fix approach: Either add the documented Playwright harness or remove the claim from `AGENTS.md` so verification expectations match reality.
|
|
|
|
## Known Bugs
|
|
|
|
**Related projects are picked from the newest three projects, not the actual related set:**
|
|
- Symptoms: The project detail page fetches `getProjects({ limit: 3 })` and then filters that tiny subset for shared tags.
|
|
- Files: `src/app/[locale]/projects/[id]/page.tsx`, `src/hooks/useProjects.ts`
|
|
- Trigger: Open a project whose genuinely related items are not among the newest three active projects.
|
|
- Workaround: None in code. The page simply shows fewer or zero related projects.
|
|
|
|
**AI search route crashes at module initialization when its env var is missing:**
|
|
- Symptoms: Importing the route throws before handling a request because `process.env.N8N_AI_SEARCH_WEBHOOK` is read at module scope and hard-fails.
|
|
- Files: `src/app/api/search/ai/route.ts`
|
|
- Trigger: Start the app, build, or run tests without `N8N_AI_SEARCH_WEBHOOK` configured.
|
|
- Workaround: Provide the env var in every environment that loads the route.
|
|
|
|
**Visible navigation and conversion entry points are placeholders:**
|
|
- Symptoms: Submit-project, footer resource/legal links, social links, and the newsletter form do not connect to real destinations or handlers. The mobile menu button also has no behavior.
|
|
- Files: `src/app/[locale]/layout.tsx`, `src/components/project/ProjectCard.tsx`
|
|
- Trigger: Click the submit CTA, footer links, social buttons, or submit the newsletter form on any localized page.
|
|
- Workaround: None in code. Users stay on the same page or submit to a no-op form target.
|
|
|
|
## Security Considerations
|
|
|
|
**Admin and ingestion endpoints depend on a single shared API key passed in request bodies:**
|
|
- Risk: The code checks only equality against `WEBHOOK_API_KEY`. There is no request signature, timestamp, nonce, replay protection, or application-side rate limiting.
|
|
- Files: `src/lib/auth.ts`, `src/app/api/webhook/signals/route.ts`, `src/app/api/tags/maintenance/route.ts`, `src/app/api/tags/reset-projects/route.ts`, `docs/integrations/n8n/CONTEXT.generated.md`, `docs/integrations/n8n/workflows/07-signals-aggregation.md`, `docs/integrations/n8n/workflows/08-project-tag-reset.md`
|
|
- Current mitigation: `src/lib/auth.ts` uses `crypto.timingSafeEqual`, and the n8n docs call out the shared-secret problem explicitly.
|
|
- Recommendations: Move to HMAC-signed requests or provider-native webhook verification, reject stale timestamps, and apply rate limiting at the app or edge layer.
|
|
|
|
**AI search forwards user queries to n8n in a GET query string:**
|
|
- Risk: Search text and filters are serialized into the webhook URL, which is more likely to be logged by reverse proxies, platforms, and third-party tooling.
|
|
- Files: `src/app/api/search/ai/route.ts`, `docs/integrations/n8n/workflows/06-rag-project-search.md`
|
|
- Current mitigation: Not detected in code.
|
|
- Recommendations: Switch to `POST`, move request data into the body, and document the retention/logging expectations for the n8n side.
|
|
|
|
**Runtime secret handling spreads certificate material and credentials outside the repo boundary:**
|
|
- Risk: Database TLS assets are reconstructed from env vars onto disk at runtime and the client identity password is appended to the Prisma datasource URL. The n8n docs also confirm workflows that need app secrets or raw database access outside this repository.
|
|
- Files: `src/lib/prisma-url.ts`, `src/lib/prisma.ts`, `docs/integrations/n8n/CONTEXT.generated.md`, `docs/integrations/n8n/workflows/04-github-star-refresh.md`, `docs/integrations/n8n/workflows/05-project-description-vectorization.md`
|
|
- Current mitigation: `src/lib/prisma-url.ts` writes files with restrictive permissions and uses an environment-configurable certificate directory.
|
|
- Recommendations: Prefer mounted secrets over reconstructing certs in temp storage, keep passwords out of DSN strings where possible, scope database users per workflow, and document secret ownership/rotation outside the app.
|
|
|
|
**Raw markdown content can load remote images from arbitrary hosts:**
|
|
- Risk: Markdown rendering sanitizes HTML, but it still renders remote `<img>` URLs directly. If project content is not fully trusted, remote hosts can observe client IPs and referrers.
|
|
- Files: `src/components/project/MarkdownContent.tsx`
|
|
- Current mitigation: `rehype-sanitize` removes unsafe HTML and external links use `rel="noopener noreferrer"`.
|
|
- Recommendations: Proxy images, restrict allowed hosts, or disable markdown image rendering for untrusted content.
|
|
|
|
## Performance Bottlenecks
|
|
|
|
**Signal ingestion performs per-item existence checks and upserts sequentially:**
|
|
- Problem: Each signal performs validation, `findUnique`, and `upsert` inside a loop, with optional retry into a second upsert path when hotness columns are unavailable.
|
|
- Files: `src/app/api/webhook/signals/route.ts`
|
|
- Cause: The route is optimized for straightforward per-item error reporting, not for throughput.
|
|
- Improvement path: Bulk-load existing keys, batch inserts/updates, and move high-volume ingestion to a queue or transactional batch writer.
|
|
|
|
**Project and signal search rely on case-insensitive `contains` scans instead of search-specific indexes:**
|
|
- Problem: Search touches multiple text fields with `contains`, while `prisma/schema.prisma` indexes sorting/filter columns but no text-search structures.
|
|
- Files: `src/hooks/useProjects.ts`, `src/app/api/signals/route.ts`, `prisma/schema.prisma`
|
|
- Cause: Search is implemented as application-level substring matching over Prisma filters.
|
|
- Improvement path: Add PostgreSQL full-text or trigram indexes, or move search to a dedicated retrieval service.
|
|
|
|
**AI search does extra in-memory filtering and O(n²) result hydration after the webhook call:**
|
|
- Problem: `src/app/api/search/ai/route.ts` fetches candidate IDs from n8n, loads projects from the database, repeatedly calls `projects.find(...)` to restore ranking order, and then re-applies tag/domain/product-form filtering in memory.
|
|
- Files: `src/app/api/search/ai/route.ts`, `src/hooks/useProjects.ts`
|
|
- Cause: Responsibility is split awkwardly between n8n ranking and route-side post-processing.
|
|
- Improvement path: Use an ID-to-project map, keep filtering ownership on one side of the contract, and avoid loading more rows than the final page needs.
|
|
|
|
**Graceful-degradation fallbacks can return empty or zero-shaped data under DB failure:**
|
|
- Problem: Homepage and project metadata helpers catch database errors and degrade to empty arrays or zero counts.
|
|
- Files: `src/hooks/useProjects.ts`, `src/hooks/useHome.ts`
|
|
- Cause: The code prefers availability over surfacing failures and uses the same modules for cached and uncached access.
|
|
- Improvement path: Separate failure-aware fetchers from UI fallback shaping, emit explicit telemetry, and avoid treating outage-shaped responses as normal product data.
|
|
|
|
## Fragile Areas
|
|
|
|
**Projects results UI maintains two overlapping state machines in one client component:**
|
|
- Files: `src/app/[locale]/projects/ProjectsResultsClient.tsx`, `src/app/api/projects/route.ts`, `src/app/api/search/ai/route.ts`
|
|
- Why fragile: Traditional search and AI search keep separate pagination, sort, loading, error, and URL-sync state inside the same file. The component also mutates the browser URL with `window.history.replaceState`, which is easy to desynchronize from server-rendered state.
|
|
- Safe modification: Change only one search mode at a time, verify deep-linking after every edit, and extract shared query-state helpers before adding more filters or sorts.
|
|
- Test coverage: No tests detected for `src/app/[locale]/projects/ProjectsResultsClient.tsx`, `src/app/api/projects/route.ts`, or `src/app/api/search/ai/route.ts`
|
|
|
|
**Direct database-writing n8n jobs bypass repository API routes and cache invalidation:**
|
|
- Files: `docs/integrations/n8n/CONTEXT.generated.md`, `docs/integrations/n8n/workflows/04-github-star-refresh.md`, `docs/integrations/n8n/workflows/05-project-description-vectorization.md`, `prisma/schema.prisma`, `src/hooks/useProjects.ts`, `src/app/api/search/ai/route.ts`
|
|
- Why fragile: The documented star-refresh and vectorization workflows write straight to Postgres instead of going through repository routes. That bypasses app-level validation, audit points, and any future route-based revalidation logic.
|
|
- Safe modification: Treat the Prisma schema and n8n registry as one shared contract, and change database columns/indexes only with coordinated workflow updates plus runtime verification.
|
|
- Test coverage: No contract or integration tests detected for these cross-system write paths
|
|
|
|
**The project ingestion loop is not self-contained in this repository:**
|
|
- Files: `docs/integrations/n8n/DATAFLOW.md`, `docs/integrations/n8n/CONTEXT.generated.md`, `docs/integrations/n8n/workflows/01-topic-discovery.md`, `docs/integrations/n8n/workflows/02-github-trending-discovery.md`, `docs/integrations/n8n/workflows/03-project-ingestion-multi-source.md`
|
|
- Why fragile: The docs explicitly depend on `/api/discovery/check-duplicates` and `/api/discovery/tasks*`, but there is no `src/app/api/discovery/` implementation in this repo. The repo depends on an upstream discovery service to stay operational.
|
|
- Safe modification: Treat discovery endpoints as an external contract, version their request/response shapes, and avoid assuming local end-to-end reproducibility for ingestion work.
|
|
- Test coverage: No in-repo tests can cover the full discovery-to-ingestion flow because the required service is absent here
|
|
|
|
**Bulk tag mutation endpoints couple validation, writes, and cache invalidation at the route layer:**
|
|
- Files: `src/app/api/tags/maintenance/route.ts`, `src/app/api/tags/maintenance/service.ts`, `src/app/api/tags/reset-projects/route.ts`
|
|
- Why fragile: A single request can mutate many tags or projects and immediately trigger localized page revalidation. Business rules, transactional writes, and cache invalidation are tightly coupled.
|
|
- Safe modification: Preserve transaction boundaries, keep mutation rules covered with focused tests, and review every `revalidatePath` target before changing route semantics.
|
|
- Test coverage: Unit-style route and service tests exist, but there are no broader integration tests across Prisma writes, cache invalidation, and localized page rendering
|
|
|
|
## Scaling Limits
|
|
|
|
**Offset pagination on projects degrades with table size:**
|
|
- Current capacity: `src/hooks/useProjects.ts` uses `skip` and `take`, and `src/app/api/projects/route.ts` allows `limit` up to `100`.
|
|
- Limit: Deep pages require larger offset scans in PostgreSQL, especially when combined with tag joins and text filters.
|
|
- Scaling path: Move project listings to cursor pagination keyed by indexed sort fields or restrict deep paging.
|
|
|
|
**Signals pagination is ordered by mutable ranking fields:**
|
|
- Current capacity: `src/app/api/signals/route.ts` sorts hot feeds by `isHot`, `hotScore`, `engagement`, `publishedAt`, and `id`, but cursors by `id` only.
|
|
- Limit: As new signals arrive or hotness changes, clients can observe duplicates or skips between pages because the rank can move independently of the cursor key.
|
|
- Scaling path: Use a stable compound cursor that includes the sort fields or snapshot the ranking inputs for pagination windows.
|
|
|
|
**Bulk tag reset scales linearly in both writes and cache invalidations:**
|
|
- Current capacity: `src/app/api/tags/reset-projects/route.ts` can process many projects in one request and revalidate both locale list pages plus every updated detail page.
|
|
- Limit: Large batch operations increase request time and invalidation fan-out.
|
|
- Scaling path: Batch revalidation, shift large maintenance jobs to background execution, or use broader cache-tag invalidation when available.
|
|
|
|
## Dependencies at Risk
|
|
|
|
**External discovery services are mandatory but not versioned here:**
|
|
- Risk: Project discovery, dedupe, and task lifecycle all depend on endpoints that are documented but not implemented in this repo.
|
|
- Impact: Local development, replay, debugging, and disaster recovery are incomplete without a second system.
|
|
- Migration plan: Either bring `src/app/api/discovery/*` into the repo or maintain a separately versioned API contract with tests and operational ownership.
|
|
|
|
**AI search depends on an external n8n webhook contract with no local fallback:**
|
|
- Risk: `/api/search/ai` assumes the remote workflow exists, responds quickly, and preserves the `results[].id` plus `similarity` schema.
|
|
- Impact: AI search becomes a single external point of failure and contract drift breaks the feature immediately.
|
|
- Migration plan: Version the webhook contract in `docs/integrations/n8n/registry.json`, add contract tests around `src/app/api/search/ai/route.ts`, and return controlled degraded responses when the webhook is unavailable.
|
|
|
|
**Workflow metadata is committed, but workflow behavior still lives outside git:**
|
|
- Risk: The repo carries registry entries and generated context, but not the executable n8n exports.
|
|
- Impact: Reviewers can understand intent, but they still cannot reproduce or diff actual workflow logic from the repository alone.
|
|
- Migration plan: Start committing workflow exports to `docs/integrations/n8n/exports/` and reference them from `docs/integrations/n8n/registry.json`.
|
|
|
|
**`unstable_cache` behavior still depends on runtime capabilities:**
|
|
- Risk: The app uses `runWithCacheFallback` to catch environments where `unstable_cache` is unavailable.
|
|
- Impact: Cache behavior differs across local development, tests, and deployed runtimes, which complicates debugging and performance expectations.
|
|
- Migration plan: Centralize cache policy, document supported runtimes, and replace `unstable_cache` with stable APIs when the stack allows it.
|
|
|
|
## Missing Critical Features
|
|
|
|
**The repository does not contain a self-contained discovery ingestion loop:**
|
|
- Problem: The n8n docs depend on discovery task endpoints that do not exist under `src/app/api/`, so the repo cannot run its own project-intake workflow end to end.
|
|
- Blocks: End-to-end ingestion tests, local replay of failed discovery tasks, and full incident debugging from this repo alone.
|
|
|
|
**Browser-level regression coverage is missing:**
|
|
- Problem: The repo ships no `playwright.config.*`, no `e2e/` directory, and no `test:e2e` script even though repository guidance claims they exist.
|
|
- Blocks: Localized routing, multi-step filtering, AI search UX, and layout-level interaction regressions are not protected at the browser layer.
|
|
|
|
**User-facing submission and newsletter flows are still not implemented:**
|
|
- Problem: The visible submit-project and newsletter UI does not connect to backend handlers or third-party providers.
|
|
- Blocks: Users cannot submit projects, subscribe for updates, or rely on footer resource/legal/social destinations.
|
|
|
|
## Test Coverage Gaps
|
|
|
|
**Search and listing APIs remain untested:**
|
|
- What's not tested: `GET /api/projects`, `GET /api/projects/[slug]`, `POST /api/search/ai`, and `GET /api/signals`
|
|
- Files: `src/app/api/projects/route.ts`, `src/app/api/projects/[slug]/route.ts`, `src/app/api/search/ai/route.ts`, `src/app/api/signals/route.ts`
|
|
- Risk: Pagination, sorting, env-missing behavior, cursor correctness, and external-service error handling can break unnoticed.
|
|
- Priority: High
|
|
|
|
**Signal ingestion still has no route-level tests:**
|
|
- What's not tested: Validation, per-item error accounting, hotness fallback, and write behavior in `POST /api/webhook/signals`
|
|
- Files: `src/app/api/webhook/signals/route.ts`, `src/lib/signal-hotness.ts`
|
|
- Risk: Ingestion regressions can silently drop, mis-rank, or partially write signals.
|
|
- Priority: High
|
|
|
|
**n8n and discovery contracts have no automated verification in this repo:**
|
|
- What's not tested: That `docs/integrations/n8n/registry.json`, `docs/integrations/n8n/CONTEXT.generated.md`, and route expectations stay aligned with live n8n workflows and the external discovery service.
|
|
- Files: `docs/integrations/n8n/registry.json`, `docs/integrations/n8n/CONTEXT.generated.md`, `scripts/generate-n8n-context.mjs`, `src/app/api/search/ai/route.ts`, `src/app/api/tags/reset-projects/route.ts`, `src/app/api/webhook/signals/route.ts`
|
|
- Risk: Cross-system contract drift is detected late, usually only after production failures.
|
|
- Priority: High
|
|
|
|
**Large client components and shared layout have no regression tests:**
|
|
- What's not tested: Filter toggling, URL synchronization, AI/traditional search mode switching, signal feed paging, and layout-level placeholder interactions
|
|
- Files: `src/app/[locale]/projects/ProjectsResultsClient.tsx`, `src/components/project/TagFilterPanel.tsx`, `src/components/signals/SignalFeedClient.tsx`, `src/app/[locale]/layout.tsx`
|
|
- Risk: UI regressions are likely because these files are state-heavy and branchy.
|
|
- Priority: High
|
|
|
|
**Current tests focus on tag admin flows and low-level helpers, not main user journeys:**
|
|
- What's not tested: Most database-backed pages and operational boundaries outside tag maintenance, auth comparison, validation schemas, and Prisma URL construction
|
|
- Files: `src/app/api/tags/route.test.ts`, `src/app/api/tags/reset-projects/route.test.ts`, `src/app/api/tags/maintenance/route.test.ts`, `src/app/api/tags/maintenance/service.test.ts`, `src/lib/auth.test.ts`, `src/lib/validations.test.ts`, `src/lib/validations.tag-maintenance.test.ts`, `src/lib/prisma-url.test.ts`
|
|
- Risk: The suite provides confidence for admin mutation helpers but not for the main product surfaces or external integration edges.
|
|
- Priority: Medium
|
|
|
|
---
|
|
|
|
*Concerns audit: 2026-04-20*
|