Compare commits

20 Commits
v1.0.0 ... main

Author SHA1 Message Date
16ee9e3beb sync: malware page, security alerts, review diagnostics, skill detail improvements
- Add malware security advisory page and SecurityAlertBanner component
- Add review diagnostics API and reviewed stats page
- Improve skill detail page with better scoring display and metadata
- Update review API endpoints with enhanced filtering and stats
- Add skill file serving improvements and cache enhancements
- Remove legacy curation scripts (moved to internal tooling)
- Update CLI search with score filtering support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 16:08:12 +02:00
airano
3e22d2e238 sync: AI review scores, reviewed page, search improvements, score filter
- Surface AI review scores across skill pages, browse, and search
- Add dedicated /reviewed page with score filtering
- Add recommended sort using Meilisearch relevance + AI scores
- Fix search sort defaulting to stars instead of recommended
- Fix CLI TypeScript null check for aiScore in search command
- Adjust cache TTL for reviewed content

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 05:05:37 +03:30
airano
4212b5ba47 sync: stale skill detection, sentry filtering, claim removal, review improvements
Stale Detection:
- Detect skills removed/moved from GitHub (3 consecutive 404s threshold)
- Hide stale skills from browse/search, show warning banner on detail page
- Serve cached files with isStale flag when GitHub returns 404
- Add stale-check crawler command for batch verification
- CLI shows warning when installing stale skills from cache

Sentry & Error Handling:
- Filter browser extension errors and add denyUrls
- Anti-inflation measures and sentinel recalibration for curation

Claim & Removal:
- Enhanced ClaimForm with repo-level removal support
- Add repo-removal-request API endpoint with tests
- Improved owner page with bilingual content

Review Pipeline:
- Review version and reviewer tracking in submit API
- Source format filter for pending reviews
- Updated review tests

Other:
- Updated i18n strings (en/fa)
- BrowseFilters improvements
- Dockerfile updates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 05:59:26 +03:30
airano
447f9eff59 sync: add AI review pipeline, parallel skill-files fetch, Redis cache improvements
- feat(review): add skill_reviews table, review API endpoints (pending/submit/stats), admin auth
- perf: parallel file fetching in skill-files API (was sequential → timeout)
- fix: handle Date serialization from Redis cache
- fix: align curation batch scripts with current browseReadyFilter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 19:51:43 +03:30
airano
328520bb70 sync: Redis caching for browse/skill API, progress bar on search, default sort by stars
- Add Redis caching to browse search results and skill detail API
- Show progress bar on homepage search
- Default sort to stars

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 16:32:07 +03:30
airano
d19113795e sync: Redis caching, progress bar rewrite, sentry filtering, UI improvements
- Add Redis caching to all pages and uncached API routes
- Rewrite progress bar with direct DOM manipulation for search/filters/navigation
- Filter generic network errors from Sentry client reporting
- Fix non-null assertion for repo.owner in popular-repos strategy
- Remove next-nprogress-bar dependency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 12:58:16 +03:30
airano
0713607693 sync: claim removal fix, auto-crawl email, progress bar, search optimization
- Fix claim removal request flow
- Add email notification on auto-crawl skill discovery
- Add progress bar component
- Optimize Meilisearch sync and search indexing
- Improve skill parser and queries

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 04:58:01 +03:30
airano
b05cb21a65 sync: multi-branch scanning, discovery strategies, UTF-8 fix, curation improvements
- Add multi-branch scanning to discover skills on non-default branches
- Add popular repos and commits search discovery strategies
- Fix UTF-8 encoding sanitization in skill parser
- Add repo_created_at for threshold-based duplicate tie-breaking
- Add indexer tests and vitest config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 04:16:35 +03:30
airano
3f599a23fb sync: SEO canonical URLs, curation improvements, batch scripts, stats fix
- Dynamic canonical URLs and alternates for all pages (SEO fix)
- Shared seo.ts utility for canonical path generation
- Improved duplicate detection: standalone > aggregator priority, forks/created_at tie-breakers
- browseReadyFilter now includes unique aggregator skills
- Stats aligned with browseReadyFilter (was showing 16k instead of 62k+)
- Re-review mechanism: content_hash changes trigger needs-re-review
- review_status field added to skills schema
- Batch security scan and quality score scripts
- Indexer Dockerfile updated for curation deps
- Removed roadmap from README

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 23:40:03 +03:30
airano
77917bf91f fix(indexer): persist quality score during skill upsert
Quality was calculated by SkillAnalyzer during crawl but never saved to
database — same class of bug as the securityStatus fix in Phase 2.
Added qualityScore and qualityDetails to both the indexSkill upsert call
and the onConflictDoUpdate set clause.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 18:56:14 +03:30
airano
9b2a741179 fix(stats): show all downloads without browse-ready filter
Downloads represent real user actions and should not be filtered by
browse-ready criteria. Skills count and contributors remain filtered.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 16:48:27 +03:30
airano
c531db31db fix(browse): format filter + sentry AbortError filtering
- Preserve 'all' value in format filter URL param so "All Formats"
  selection persists correctly
- Filter AbortError from client-side Sentry error reporting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 15:52:45 +03:30
airano
c8216dfcd2 feat(curation): auto-curation after crawl + quality messaging
Automate curation:
- Add runPostCrawlCuration() to DB queries — runs after every
  full and incremental crawl (classify, dedup, category counts)
- No more need for manual curate.mjs runs after each crawl

Quality messaging:
- Homepage stats label: "Skills" → "Curated Skills" (en/fa)
- Add curation note: "deduplicated and quality-filtered from X+ repos"
- Update fallback counts from 172K to 16K across all touchpoints
  (BetaBanner, getting-started prompts, email templates)
- getting-started skill count query now uses browse-ready filter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 15:06:18 +03:30
airano
caca09fbe7 feat(curation): data quality pipeline — Phases 1-3
Add comprehensive data curation system to clean up the 197K skill
dataset and show only quality browse-ready skills to users.

Phase 1 — Database exploration:
- Explore scripts (explore.ts, explore.mjs, explore.sql) for analysis
- Discovered: 69% duplicates, 77% aggregator/fork noise

Phase 2 — Data cleanup and classification:
- Schema: 6 new curation columns + 4 indexes
- curate.mjs: 8-step pipeline (classify, dedup, fork detection, etc.)
- Result: 197K → 60K unique → 16K browse-ready skills
- Bug fix: securityStatus was computed but never stored during crawl

Phase 3 — UI browse-ready filters:
- browseReadyFilter applied to 17+ query functions
- Homepage stats show accurate browse-ready counts
- Stats API filtered (previously had no WHERE clause)
- Category counts recalculated (e.g. 45K → 3.1K)
- Featured skills exclude duplicates and aggregators

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 14:31:25 +03:30
airano
31f5df0900 fix(db): add stable tiebreaker to search pagination
Without a deterministic tiebreaker on skill ID, offset-based pagination
through large groups of skills with identical star counts produced
non-deterministic ordering. This caused ~1,516 skills to be missed
during Meilisearch sync (192,311 sent but only 190,795 unique indexed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 04:16:26 +03:30
airano
2d37a5c16e fix(seo): generate correct sitemap URLs for default locale
Sitemap was generating /en/* URLs which redirect to unprefixed paths
due to localePrefix: 'as-needed', causing Google Search Console
"Page with redirect" warnings and wasting crawl budget.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:27:16 +03:30
airano
444a6668d0 fix(indexer): prevent token exhaustion and reserve 20% budget for website users
- Lower exhaustion threshold from <10 to <2 (stop wasting remaining tokens)
- Refresh ALL tokens after wait instead of only one
- Fix token/Octokit mismatch by returning token from getBestInstance()
- Add budget checking (20% reserve) to deep-scan, discover-repos, awesome-lists,
  and full-enhanced commands
- Add rate limit error handling with retry in DeepScanCrawler
- Sync translation and query changes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 18:29:39 +03:30
airano
e8bf89d311 fix(indexer): update skillPath/branch on upsert to fix stale paths
The upsert query was missing skillPath and branch in onConflictDoUpdate,
so when a repo was restructured (e.g., skills moved to category
subdirectories), the DB retained the old path forever. Also updated
skill-indexer to not skip re-indexing when only the path changed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 04:56:40 +03:30
airano
908576a307 chore(cli): bump version to 0.2.5
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 04:49:00 +03:30
airano
394830280f fix(cli): resolve SKILL.md not found errors from duplicate path prefixes
When skillPath from the registry had a prefix (e.g., "skills/foo"), fallback
paths were incorrectly generated with duplicate prefixes like
".claude/skills/skills/foo/SKILL.md". Now extracts the leaf skill name to
build correct fallback paths. Also adds raw.githubusercontent.com as a
universal fallback for 404s, improves error messages with actual paths tried,
and skips API results that lack the main instruction file.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 04:46:22 +03:30
103 changed files with 7471 additions and 721 deletions

5
.gitignore vendored
View File

@@ -43,6 +43,10 @@ drizzle/*.sql
*.dump *.dump
data/backups/ data/backups/
# Curation session data (review batches/results contain raw_content)
scripts/curation/data/*
!scripts/curation/data/.gitkeep
# Claude Code # Claude Code
.claude/skills/ .claude/skills/
.claude/settings.local.json .claude/settings.local.json
@@ -50,3 +54,4 @@ data/backups/
# !.claude/agents/ # !.claude/agents/
# !.claude/council_reasoning.md # !.claude/council_reasoning.md
.claude/settings.json .claude/settings.json
.mcp.json

View File

@@ -94,6 +94,17 @@ docker compose exec indexer node dist/crawl.js full # Full crawl
docker compose exec indexer node dist/crawl.js incremental # Last 24h docker compose exec indexer node dist/crawl.js incremental # Last 24h
docker compose exec indexer node dist/crawl.js sync-meili # Sync to Meilisearch docker compose exec indexer node dist/crawl.js sync-meili # Sync to Meilisearch
docker compose exec indexer node dist/crawl.js deep-scan # Scan discovered repos docker compose exec indexer node dist/crawl.js deep-scan # Scan discovered repos
docker compose exec indexer node dist/crawl.js add-repo <owner/repo> # Manually add a repo to DB
docker compose exec indexer node dist/crawl.js process-add-requests # Process user-submitted add requests
```
### Database Migration (container)
```bash
# Add new columns (run when schema changes are deployed)
docker compose exec db psql -U skillhub -d skillhub -c "ALTER TABLE skills ADD COLUMN IF NOT EXISTS is_owner_claimed boolean NOT NULL DEFAULT false;"
docker compose exec db psql -U skillhub -d skillhub -c "ALTER TABLE discovered_repos ADD COLUMN IF NOT EXISTS is_blocked boolean NOT NULL DEFAULT false;"
docker compose exec db psql -U skillhub -d skillhub -c "CREATE INDEX IF NOT EXISTS idx_skills_owner_claimed ON skills(is_owner_claimed) WHERE is_owner_claimed = true;"
docker compose exec db psql -U skillhub -d skillhub -c "CREATE INDEX IF NOT EXISTS idx_discovered_repos_blocked ON discovered_repos(is_blocked) WHERE is_blocked = true;"
``` ```
## Security ## Security

View File

@@ -184,17 +184,6 @@ Always review skill source code before installing.
--- ---
## Roadmap
- [x] Web marketplace with growing skill catalog
- [x] CLI tool
- [x] Multi-platform support
- [x] Security scanning
- [ ] Claude Code MCP plugin
- [ ] AI-powered recommendations
- [ ] Enterprise features
---
## License ## License

View File

@@ -1,6 +1,6 @@
{ {
"name": "skillhub", "name": "skillhub",
"version": "0.2.4", "version": "0.2.11",
"description": "CLI tool for managing AI Agent skills - search, install, and update skills for Claude, Codex, Copilot, and more", "description": "CLI tool for managing AI Agent skills - search, install, and update skills for Claude, Codex, Copilot, and more",
"author": "SkillHub Contributors", "author": "SkillHub Contributors",
"license": "MIT", "license": "MIT",

View File

@@ -39,7 +39,20 @@ export async function install(skillId: string, options: InstallOptions): Promise
try { try {
skillInfo = await getSkill(skillId); skillInfo = await getSkill(skillId);
if (skillInfo) { if (skillInfo) {
spinner.succeed(`Found in registry: ${skillInfo.name}`); // Block installation of malicious skills
if (skillInfo.isMalicious) {
spinner.fail(chalk.red('This skill has been flagged as malicious (contains malware).'));
console.log(chalk.red('Installation blocked for your safety.'));
console.log(chalk.dim(`See: https://skills.palebluedot.live/en/skill/${skillId}`));
process.exit(1);
}
const reviewBadge = skillInfo.aiScore && skillInfo.reviewStatus === 'ai-reviewed'
? chalk.dim(` | AI: ${skillInfo.aiScore}/100`)
: skillInfo.reviewStatus === 'verified'
? chalk.green(` | AI: ${skillInfo.aiScore}/100 ✓`)
: '';
spinner.succeed(`Found in registry: ${skillInfo.name}${reviewBadge}`);
spinner.start('Preparing installation...'); spinner.start('Preparing installation...');
} }
} catch (error) { } catch (error) {
@@ -98,15 +111,29 @@ export async function install(skillId: string, options: InstallOptions): Promise
const cachedFiles = await getSkillFiles(skillInfo.id); const cachedFiles = await getSkillFiles(skillInfo.id);
if (cachedFiles && cachedFiles.files.length > 0) { if (cachedFiles && cachedFiles.files.length > 0) {
// Warn if skill is stale (served from cache because GitHub returned 404)
if (cachedFiles.isStale) {
spinner.warn(chalk.yellow(
'This skill may have been removed from its GitHub repository.\n' +
' Files were served from the SkillHub cache and may be outdated.'
));
spinner.start('Installing from cached files...');
}
// Use sourceFormat from API response if available // Use sourceFormat from API response if available
if (cachedFiles.sourceFormat) { if (cachedFiles.sourceFormat) {
sourceFormat = cachedFiles.sourceFormat as SourceFormat; sourceFormat = cachedFiles.sourceFormat as SourceFormat;
} }
// Convert API response to SkillContent format // Convert API response to SkillContent format
content = convertCachedFilesToSkillContent(cachedFiles, sourceFormat); const converted = convertCachedFilesToSkillContent(cachedFiles, sourceFormat);
// Only use API result if the main instruction file was found
if (converted.skillMd) {
content = converted;
spinner.text = cachedFiles.fromCache spinner.text = cachedFiles.fromCache
? `Using cached files (${cachedFiles.files.length} files)` ? `Using cached files (${cachedFiles.files.length} files)`
: `Downloaded ${cachedFiles.files.length} files via API`; : `Downloaded ${cachedFiles.files.length} files via API`;
} else {
spinner.text = 'API returned files but main instruction file missing, falling back...';
}
} }
} catch { } catch {
// API was reachable but file fetch failed (timeout, server error, etc.) // API was reachable but file fetch failed (timeout, server error, etc.)

View File

@@ -18,7 +18,7 @@ export async function search(query: string, options: SearchOptions): Promise<voi
try { try {
const limit = parseInt(options.limit || '10'); const limit = parseInt(options.limit || '10');
const page = parseInt(options.page || '1'); const page = parseInt(options.page || '1');
const sort = options.sort || 'downloads'; const sort = options.sort || 'recommended';
const result = await searchSkills(query, { const result = await searchSkills(query, {
platform: options.platform, platform: options.platform,
limit, limit,
@@ -34,7 +34,7 @@ export async function search(query: string, options: SearchOptions): Promise<voi
return; return;
} }
const sortLabel = { downloads: 'downloads', stars: 'GitHub stars', rating: 'rating', recent: 'recently updated' }[sort] || sort; const sortLabel = { recommended: 'recommended', downloads: 'downloads', stars: 'GitHub stars', rating: 'rating', recent: 'recently updated', aiScore: 'AI score' }[sort] || sort;
console.log(chalk.bold(`Found ${result.pagination.total} skills (sorted by ${sortLabel}):\n`)); console.log(chalk.bold(`Found ${result.pagination.total} skills (sorted by ${sortLabel}):\n`));
// Print results as a table // Print results as a table
@@ -57,9 +57,15 @@ export async function search(query: string, options: SearchOptions): Promise<voi
`${num} ${verified} ${chalk.cyan(skill.id.padEnd(38))} ${security}` `${num} ${verified} ${chalk.cyan(skill.id.padEnd(38))} ${security}`
); );
// Second line: downloads, stars, description // Second line: AI score + downloads + stars + description
const aiScore = skill.aiScore;
const isAiReviewed = aiScore != null && aiScore > 0 && skill.reviewStatus && skill.reviewStatus !== 'unreviewed' && skill.reviewStatus !== 'auto-scored';
const showAiScore = isAiReviewed;
const aiPrefix = showAiScore
? `${chalk.magenta('AI')} ${(aiScore >= 75 ? chalk.green : aiScore >= 50 ? chalk.yellow : chalk.dim)(String(aiScore).padStart(2))} `
: '';
console.log( console.log(
`${formatNumber(skill.downloadCount).padStart(6)}${formatNumber(skill.githubStars).padStart(6)} ${chalk.dim(skill.description.slice(0, 55))}${skill.description.length > 55 ? '...' : ''}` ` ${aiPrefix}${formatNumber(skill.downloadCount).padStart(6)}${formatNumber(skill.githubStars).padStart(6)} ${chalk.dim(skill.description.slice(0, showAiScore ? 45 : 55))}${skill.description.length > (showAiScore ? 45 : 55) ? '...' : ''}`
); );
// Third line: Rating (only if ratingCount >= 3) // Third line: Rating (only if ratingCount >= 3)
@@ -83,8 +89,8 @@ export async function search(query: string, options: SearchOptions): Promise<voi
); );
} }
if (sort === 'downloads') { if (sort === 'recommended') {
console.log(chalk.dim(`Sort options: ${chalk.white('--sort stars|rating|recent')}`)); console.log(chalk.dim(`Sort options: ${chalk.white('--sort aiScore|downloads|stars|rating|recent')}`));
} }
} catch (error) { } catch (error) {
spinner.fail('Search failed'); spinner.fail('Search failed');

View File

@@ -46,7 +46,7 @@ program
.command('search <query>') .command('search <query>')
.description('Search for skills in the registry') .description('Search for skills in the registry')
.option('-p, --platform <platform>', 'Filter by platform') .option('-p, --platform <platform>', 'Filter by platform')
.option('-s, --sort <sort>', 'Sort by: downloads, stars, rating, recent', 'downloads') .option('-s, --sort <sort>', 'Sort by: recommended, aiScore, downloads, stars, rating, recent', 'recommended')
.option('-l, --limit <number>', 'Number of results', '10') .option('-l, --limit <number>', 'Number of results', '10')
.option('--page <number>', 'Page number', '1') .option('--page <number>', 'Page number', '1')
.action(async (query: string, options) => { .action(async (query: string, options) => {

View File

@@ -126,6 +126,9 @@ export interface SkillInfo {
sourceFormat?: string; sourceFormat?: string;
rating?: number | null; rating?: number | null;
ratingCount?: number | null; ratingCount?: number | null;
reviewStatus?: string | null;
aiScore?: number | null;
isMalicious?: boolean;
isVerified: boolean; isVerified: boolean;
compatibility: { compatibility: {
platforms: string[]; platforms: string[];
@@ -231,6 +234,8 @@ export interface SkillFilesResponse {
files: SkillFile[]; files: SkillFile[];
fromCache: boolean; fromCache: boolean;
cachedAt?: string; cachedAt?: string;
isStale?: boolean;
staleWarning?: string;
} }
/** /**

View File

@@ -100,12 +100,26 @@ export async function fetchSkillContent(
} }
} else { } else {
// SKILL.md - try multiple common paths // SKILL.md - try multiple common paths
pathsToTry = [ // Extract the leaf skill name to avoid duplicate prefixes
basePath, // e.g., "skills/claude-md-architect" → "claude-md-architect"
...(skillPath && !skillPath.startsWith('skills/') ? [`skills/${skillPath}/SKILL.md`] : []), const skillName = skillPath
...(skillPath && !skillPath.startsWith('.claude/') ? [`.claude/skills/${skillPath}/SKILL.md`] : []), ? skillPath.replace(/^(skills|\.claude\/skills|\.github\/skills)\//, '')
...(skillPath && !skillPath.startsWith('.github/') ? [`.github/skills/${skillPath}/SKILL.md`] : []), : '';
pathsToTry = [basePath];
if (skillName) {
// Add all common locations, deduplicating against basePath
const candidates = [
`skills/${skillName}/${filename}`,
`.claude/skills/${skillName}/${filename}`,
`.github/skills/${skillName}/${filename}`,
]; ];
for (const candidate of candidates) {
if (candidate !== basePath) {
pathsToTry.push(candidate);
}
}
}
} }
for (const pathToTry of pathsToTry) { for (const pathToTry of pathsToTry) {
@@ -123,7 +137,6 @@ export async function fetchSkillContent(
if (error.message?.includes('timeout') || error.message?.includes('network')) { if (error.message?.includes('timeout') || error.message?.includes('network')) {
try { try {
const rawContent = await fetchRawFile(owner, repo, pathToTry, branch); const rawContent = await fetchRawFile(owner, repo, pathToTry, branch);
// Create a mock response compatible with Octokit
skillMdResponse = { skillMdResponse = {
data: { data: {
content: Buffer.from(rawContent).toString('base64'), content: Buffer.from(rawContent).toString('base64'),
@@ -131,22 +144,51 @@ export async function fetchSkillContent(
} }
} as any; } as any;
break; break;
} catch (rawError) { } catch {
// If raw fetch also fails, throw original error // Raw fetch also failed, continue to next path
throw new Error(`GitHub API timeout. Try using --no-api flag or check your network connection.`); continue;
} }
} }
// If 404, try next path // If 404, try next path
if (error.status === 404) { if (error.status === 404) {
continue; continue;
} }
// Rate limiting - provide a clear error
if (error.status === 403) {
throw new Error(
`GitHub API rate limit exceeded. Set GITHUB_TOKEN environment variable for higher rate limits.`
);
}
// Other errors, throw immediately // Other errors, throw immediately
throw new Error(`Failed to fetch from GitHub: ${error.message}`); throw new Error(`Failed to fetch from GitHub: ${error.message}`);
} }
} }
// If Octokit failed on all paths, try raw.githubusercontent.com as last resort
// (handles cases where GitHub API returns 404 but raw CDN has the file,
// e.g., during API caching delays or edge cases)
if (!skillMdResponse) { if (!skillMdResponse) {
throw new Error(`${filename} not found at ${owner}/${repo} (tried ${pathsToTry.length} paths)`); for (const pathToTry of pathsToTry) {
try {
const rawContent = await fetchRawFile(owner, repo, pathToTry, branch);
skillMdResponse = {
data: {
content: Buffer.from(rawContent).toString('base64'),
encoding: 'base64' as const,
}
} as any;
break;
} catch {
continue;
}
}
}
if (!skillMdResponse) {
const triedPaths = pathsToTry.map(p => ` - ${owner}/${repo}/${p}`).join('\n');
throw new Error(
`${filename} not found at ${owner}/${repo} (tried ${pathsToTry.length} paths):\n${triedPaths}\n\nThe skill may have been moved or deleted from the repository.`
);
} }
if (!('content' in skillMdResponse.data)) { if (!('content' in skillMdResponse.data)) {

View File

@@ -45,6 +45,7 @@ ARG NEXT_PUBLIC_OPENPANEL_API_URL
ARG NEXT_PUBLIC_OPENPANEL_SCRIPT_URL ARG NEXT_PUBLIC_OPENPANEL_SCRIPT_URL
ARG NEXT_PUBLIC_UMAMI_WEBSITE_ID ARG NEXT_PUBLIC_UMAMI_WEBSITE_ID
ARG NEXT_PUBLIC_UMAMI_URL ARG NEXT_PUBLIC_UMAMI_URL
ARG NEXT_PUBLIC_IS_PRIMARY
# Convert ARGs to ENVs so they're available during build # Convert ARGs to ENVs so they're available during build
ENV NEXT_PUBLIC_APP_URL=$NEXT_PUBLIC_APP_URL ENV NEXT_PUBLIC_APP_URL=$NEXT_PUBLIC_APP_URL
@@ -56,6 +57,7 @@ ENV NEXT_PUBLIC_OPENPANEL_API_URL=$NEXT_PUBLIC_OPENPANEL_API_URL
ENV NEXT_PUBLIC_OPENPANEL_SCRIPT_URL=$NEXT_PUBLIC_OPENPANEL_SCRIPT_URL ENV NEXT_PUBLIC_OPENPANEL_SCRIPT_URL=$NEXT_PUBLIC_OPENPANEL_SCRIPT_URL
ENV NEXT_PUBLIC_UMAMI_WEBSITE_ID=$NEXT_PUBLIC_UMAMI_WEBSITE_ID ENV NEXT_PUBLIC_UMAMI_WEBSITE_ID=$NEXT_PUBLIC_UMAMI_WEBSITE_ID
ENV NEXT_PUBLIC_UMAMI_URL=$NEXT_PUBLIC_UMAMI_URL ENV NEXT_PUBLIC_UMAMI_URL=$NEXT_PUBLIC_UMAMI_URL
ENV NEXT_PUBLIC_IS_PRIMARY=$NEXT_PUBLIC_IS_PRIMARY
# Build the application (turbo will build dependencies first) # Build the application (turbo will build dependencies first)
RUN pnpm turbo run build --filter=@skillhub/web RUN pnpm turbo run build --filter=@skillhub/web

View File

@@ -2,6 +2,20 @@ import { getTranslations, setRequestLocale } from 'next-intl/server';
import { Header } from '@/components/Header'; import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import { Code2, Globe, Shield } from 'lucide-react'; import { Code2, Globe, Shield } from 'lucide-react';
import { getPageAlternates } from '@/lib/seo';
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/about'),
};
}
export default async function AboutPage({ export default async function AboutPage({
params, params,

View File

@@ -3,6 +3,8 @@ import Link from 'next/link';
import { Header } from '@/components/Header'; import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import { Github, Heart, Users, Code, GitFork, ExternalLink, Database, Clock } from 'lucide-react'; import { Github, Heart, Users, Code, GitFork, ExternalLink, Database, Clock } from 'lucide-react';
import { getPageAlternates } from '@/lib/seo';
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -65,6 +67,18 @@ function formatLicenseName(license: string, locale: string): string {
return names ? names[locale as 'en' | 'fa'] || names.en : license; return names ? names[locale as 'en' | 'fa'] || names.en : license;
} }
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/attribution'),
};
}
export default async function AttributionPage({ export default async function AttributionPage({
params, params,
}: { }: {

View File

@@ -5,6 +5,9 @@ import { createDb, skillQueries, categoryQueries } from '@skillhub/db';
import { BrowseFilters, SearchBar, Pagination, ActiveFilters, EmptyState } from '@/components/BrowseFilters'; import { BrowseFilters, SearchBar, Pagination, ActiveFilters, EmptyState } from '@/components/BrowseFilters';
import { SkillCard } from '@/components/SkillCard'; import { SkillCard } from '@/components/SkillCard';
import { toPersianNumber } from '@/lib/format-number'; import { toPersianNumber } from '@/lib/format-number';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL, hashSearchParams } from '@/lib/cache';
// Force dynamic rendering to fetch fresh data from database // Force dynamic rendering to fetch fresh data from database
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -22,6 +25,7 @@ interface BrowsePageProps {
} }
// Get skills directly from database with filters - all filtering at database level // Get skills directly from database with filters - all filtering at database level
// Results are cached in Redis for 30 minutes, keyed by a hash of all filter parameters
async function getSkills(params: { async function getSkills(params: {
q?: string; q?: string;
platform?: string; platform?: string;
@@ -31,41 +35,59 @@ async function getSkills(params: {
category?: string; category?: string;
}) { }) {
try { try {
const db = createDb();
const limit = 20; const limit = 20;
const page = parseInt(params.page || '1'); const page = parseInt(params.page || '1');
// Default sort: lastDownloaded for browsing, recommended when searching
const defaultSort = params.q ? 'recommended' : 'lastDownloaded';
const effectiveSort = params.sort || defaultSort;
const hash = hashSearchParams({
q: params.q,
category: params.category,
platform: params.platform && params.platform !== 'all' ? params.platform : undefined,
format: params.format || 'skill.md',
sort: effectiveSort,
page,
});
return await getOrSetCache(
cacheKeys.searchSkills(hash),
cacheTTL.search,
async () => {
const db = createDb();
const offset = (page - 1) * limit; const offset = (page - 1) * limit;
const sortMap: Record<string, 'stars' | 'downloads' | 'rating' | 'updated' | 'lastDownloaded'> = { const sortMap: Record<string, 'stars' | 'downloads' | 'rating' | 'updated' | 'lastDownloaded' | 'aiScore' | 'recommended'> = {
'stars': 'stars', 'stars': 'stars',
'downloads': 'downloads', 'downloads': 'downloads',
'recent': 'updated', 'recent': 'updated',
'rating': 'rating', 'rating': 'rating',
'lastDownloaded': 'lastDownloaded', 'lastDownloaded': 'lastDownloaded',
'aiScore': 'aiScore',
'recommended': 'recommended',
}; };
// Build filter options - push ALL filters to database level
const filterOptions = { const filterOptions = {
query: params.q, query: params.q,
category: params.category, category: params.category,
platform: params.platform && params.platform !== 'all' ? params.platform : undefined, platform: params.platform && params.platform !== 'all' ? params.platform : undefined,
sourceFormat: params.format || 'skill.md', sourceFormat: params.format || 'skill.md',
sortBy: sortMap[params.sort || 'lastDownloaded'] || 'lastDownloaded', sortBy: sortMap[effectiveSort] || defaultSort,
sortOrder: 'desc' as const, sortOrder: 'desc' as const,
limit, limit,
offset, offset,
}; };
// Fetch paginated results directly from database const [skills, total] = await Promise.all([
const skills = await skillQueries.search(db, filterOptions); skillQueries.search(db, filterOptions),
skillQueries.count(db, {
// Get accurate total count for pagination
const total = await skillQueries.count(db, {
query: params.q, query: params.q,
category: params.category, category: params.category,
platform: params.platform && params.platform !== 'all' ? params.platform : undefined, platform: params.platform && params.platform !== 'all' ? params.platform : undefined,
sourceFormat: params.format || 'skill.md', sourceFormat: params.format || 'skill.md',
}); }),
]);
const totalPages = Math.ceil(total / limit); const totalPages = Math.ceil(total / limit);
@@ -73,12 +95,26 @@ async function getSkills(params: {
skills, skills,
pagination: { total, page, totalPages }, pagination: { total, page, totalPages },
}; };
}
);
} catch (error) { } catch (error) {
console.error('Error fetching skills:', error); console.error('Error fetching skills:', error);
return { skills: [], pagination: { total: 0, page: 1, totalPages: 1 } }; return { skills: [], pagination: { total: 0, page: 1, totalPages: 1 } };
} }
} }
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/browse'),
};
}
export default async function BrowsePage({ params, searchParams }: BrowsePageProps) { export default async function BrowsePage({ params, searchParams }: BrowsePageProps) {
const { locale } = await params; const { locale } = await params;
setRequestLocale(locale); setRequestLocale(locale);
@@ -87,14 +123,16 @@ export default async function BrowsePage({ params, searchParams }: BrowsePagePro
const tCommon = await getTranslations('common'); const tCommon = await getTranslations('common');
const sortOptions = [ const sortOptions = [
{ id: 'recommended', name: t('filters.sortOptions.recommended') },
{ id: 'lastDownloaded', name: t('filters.sortOptions.lastDownloaded') }, { id: 'lastDownloaded', name: t('filters.sortOptions.lastDownloaded') },
{ id: 'downloads', name: t('filters.sortOptions.downloads') }, { id: 'downloads', name: t('filters.sortOptions.downloads') },
{ id: 'stars', name: t('filters.sortOptions.stars') }, { id: 'aiScore', name: t('filters.sortOptions.aiScore') },
{ id: 'recent', name: t('filters.sortOptions.recent') }, { id: 'recent', name: t('filters.sortOptions.recent') },
{ id: 'rating', name: t('filters.sortOptions.rating') }, { id: 'rating', name: t('filters.sortOptions.rating') },
{ id: 'stars', name: t('filters.sortOptions.stars') },
]; ];
// Fetch categories hierarchically for filter dropdown with translations // Fetch categories hierarchically for filter dropdown with translations (cached 12h)
const tCategories = await getTranslations('categories'); const tCategories = await getTranslations('categories');
type HierarchicalCategory = { type HierarchicalCategory = {
id: string; id: string;
@@ -105,8 +143,14 @@ export default async function BrowsePage({ params, searchParams }: BrowsePagePro
}; };
let categories: HierarchicalCategory[] = []; let categories: HierarchicalCategory[] = [];
try { try {
const rawCategories = await getOrSetCache(
cacheKeys.categoriesHierarchical(),
cacheTTL.categories,
async () => {
const db = createDb(); const db = createDb();
const rawCategories = await categoryQueries.getHierarchical(db); return await categoryQueries.getHierarchical(db);
}
);
categories = rawCategories.map(parent => ({ categories = rawCategories.map(parent => ({
id: parent.id, id: parent.id,
name: tCategories(`parents.${parent.slug}`) || parent.name, name: tCategories(`parents.${parent.slug}`) || parent.name,
@@ -150,7 +194,7 @@ export default async function BrowsePage({ params, searchParams }: BrowsePagePro
const emptyStateTranslations = { const emptyStateTranslations = {
noResults: t('noResults') || 'No skills found', noResults: t('noResults') || 'No skills found',
noResultsWithQuery: t('noResultsWithQuery') || 'No results for "{query}"', noResultsWithQuery: t('noResultsWithQuery', { query: searchParamsResolved.q || '' }),
tryDifferent: t('emptyState.tryDifferent') || 'Try different search terms or adjust your filters', tryDifferent: t('emptyState.tryDifferent') || 'Try different search terms or adjust your filters',
clearFilters: t('emptyState.clearFilters') || 'Clear filters', clearFilters: t('emptyState.clearFilters') || 'Clear filters',
browseAll: t('emptyState.browseAll') || 'Browse Featured Skills', browseAll: t('emptyState.browseAll') || 'Browse Featured Skills',
@@ -166,9 +210,10 @@ export default async function BrowsePage({ params, searchParams }: BrowsePagePro
// Get category name for active filters display // Get category name for active filters display
const currentCategory = searchParamsResolved.category; const currentCategory = searchParamsResolved.category;
const currentSort = searchParamsResolved.sort || 'lastDownloaded'; const defaultSort = searchParamsResolved.q ? 'recommended' : 'lastDownloaded';
const currentSort = searchParamsResolved.sort || defaultSort;
const currentFormat = searchParamsResolved.format || ''; const currentFormat = searchParamsResolved.format || '';
const hasActiveFilters = !!(searchParamsResolved.q || currentCategory || (currentSort && currentSort !== 'lastDownloaded') || currentFormat); const hasActiveFilters = !!(searchParamsResolved.q || currentCategory || (currentSort && currentSort !== defaultSort) || currentFormat);
// Find category name from hierarchical categories // Find category name from hierarchical categories
let categoryName = ''; let categoryName = '';

View File

@@ -31,6 +31,9 @@ import {
} from 'lucide-react'; } from 'lucide-react';
import { createDb, categoryQueries } from '@skillhub/db'; import { createDb, categoryQueries } from '@skillhub/db';
import { formatNumber } from '@/lib/format-number'; import { formatNumber } from '@/lib/format-number';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
// Force dynamic rendering to fetch fresh data from database // Force dynamic rendering to fetch fresh data from database
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -85,17 +88,31 @@ const parentColorMap: Record<string, string> = {
'specialized': 'bg-gray-50 text-gray-600 dark:bg-gray-800 dark:text-gray-400', 'specialized': 'bg-gray-50 text-gray-600 dark:bg-gray-800 dark:text-gray-400',
}; };
// Get categories hierarchically from database // Get categories hierarchically with Redis caching (12 hour TTL)
async function getHierarchicalCategories() { async function getHierarchicalCategories() {
try { try {
return await getOrSetCache(cacheKeys.categoriesHierarchical(), cacheTTL.categories, async () => {
const db = createDb(); const db = createDb();
return await categoryQueries.getHierarchical(db); return await categoryQueries.getHierarchical(db);
});
} catch (error) { } catch (error) {
console.error('Error fetching categories:', error); console.error('Error fetching categories:', error);
return []; return [];
} }
} }
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/categories'),
};
}
export default async function CategoriesPage({ export default async function CategoriesPage({
params, params,
}: { }: {

View File

@@ -2,9 +2,23 @@ import { getTranslations, setRequestLocale } from 'next-intl/server';
import { Header } from '@/components/Header'; import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import { ClaimForm } from '@/components/ClaimForm'; import { ClaimForm } from '@/components/ClaimForm';
import { getPageAlternates } from '@/lib/seo';
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/claim'),
};
}
export default async function ClaimPage({ export default async function ClaimPage({
params, params,
}: { }: {
@@ -28,6 +42,7 @@ export default async function ClaimPage({
tabs: { tabs: {
remove: t('tabs.remove'), remove: t('tabs.remove'),
add: t('tabs.add'), add: t('tabs.add'),
removeRepo: t('tabs.removeRepo'),
}, },
form: { form: {
skillId: t('form.skillId'), skillId: t('form.skillId'),
@@ -50,6 +65,8 @@ export default async function ClaimPage({
success: { success: {
title: t('success.title'), title: t('success.title'),
description: t('success.description'), description: t('success.description'),
pendingTitle: t('success.pendingTitle'),
pendingDescription: t('success.pendingDescription'),
viewRequests: t('success.viewRequests'), viewRequests: t('success.viewRequests'),
}, },
addSuccess: { addSuccess: {
@@ -62,6 +79,27 @@ export default async function ClaimPage({
foundSkillsIn: t('addSuccess.foundSkillsIn'), foundSkillsIn: t('addSuccess.foundSkillsIn'),
root: t('addSuccess.root'), root: t('addSuccess.root'),
andMore: t.raw('addSuccess.andMore') as string, andMore: t.raw('addSuccess.andMore') as string,
reEnabledTitle: t('addSuccess.reEnabledTitle'),
reEnabledDescription: t('addSuccess.reEnabledDescription'),
},
removeRepoForm: {
repoUrl: t('removeRepoForm.repoUrl'),
repoUrlPlaceholder: t('removeRepoForm.repoUrlPlaceholder'),
repoUrlHelp: t('removeRepoForm.repoUrlHelp'),
reason: t('removeRepoForm.reason'),
reasonPlaceholder: t('removeRepoForm.reasonPlaceholder'),
submit: t('removeRepoForm.submit'),
submitting: t('removeRepoForm.submitting'),
},
removeRepoSuccess: {
title: t('removeRepoSuccess.title'),
description: t.raw('removeRepoSuccess.description') as string,
descriptionZero: t('removeRepoSuccess.descriptionZero'),
},
removeRepoError: {
notOwner: t('removeRepoError.notOwner'),
invalidRepo: t('removeRepoError.invalidRepo'),
parseError: t('removeRepoError.parseError'),
}, },
error: { error: {
notOwner: t('error.notOwner'), notOwner: t('error.notOwner'),
@@ -74,6 +112,7 @@ export default async function ClaimPage({
rateLimitExceeded: t('error.rateLimitExceeded'), rateLimitExceeded: t('error.rateLimitExceeded'),
networkTimeout: t('error.networkTimeout'), networkTimeout: t('error.networkTimeout'),
generic: t('error.generic'), generic: t('error.generic'),
repoBlockedByOwner: t('error.repoBlockedByOwner'),
}, },
myRequests: { myRequests: {
title: t('myRequests.title'), title: t('myRequests.title'),

View File

@@ -1,4 +1,5 @@
import { getTranslations, setRequestLocale } from 'next-intl/server'; import { getTranslations, setRequestLocale } from 'next-intl/server';
import { getPageAlternates } from '@/lib/seo';
import Link from 'next/link'; import Link from 'next/link';
import type { Metadata } from 'next'; import type { Metadata } from 'next';
import { import {
@@ -17,6 +18,7 @@ import { Footer } from '@/components/Footer';
import { EarlyAccessForm } from '@/components/EarlyAccessForm'; import { EarlyAccessForm } from '@/components/EarlyAccessForm';
import { createDb, skills, sql } from '@skillhub/db'; import { createDb, skills, sql } from '@skillhub/db';
import { formatCompactNumber } from '@/lib/format-number'; import { formatCompactNumber } from '@/lib/format-number';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -31,6 +33,7 @@ export async function generateMetadata({
return { return {
title: t('metadata.title'), title: t('metadata.title'),
description: t('metadata.description'), description: t('metadata.description'),
alternates: getPageAlternates(locale, '/claude-plugin'),
openGraph: { openGraph: {
title: t('metadata.title'), title: t('metadata.title'),
description: t('metadata.description'), description: t('metadata.description'),
@@ -40,11 +43,13 @@ export async function generateMetadata({
async function getStats() { async function getStats() {
try { try {
return await getOrSetCache(cacheKeys.pageCount('claude-plugin'), cacheTTL.pageCount, async () => {
const db = createDb(); const db = createDb();
const skillsResult = await db const skillsResult = await db
.select({ count: sql<number>`count(*)::int` }) .select({ count: sql<number>`count(*)::int` })
.from(skills); .from(skills);
return skillsResult[0]?.count ?? 0; return skillsResult[0]?.count ?? 0;
});
} catch { } catch {
return 119000; return 119000;
} }

View File

@@ -1,4 +1,18 @@
import { redirect } from 'next/navigation'; import { redirect } from 'next/navigation';
import { getPageAlternates } from '@/lib/seo';
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/contact'),
};
}
export default async function ContactPage({ export default async function ContactPage({
params, params,

View File

@@ -5,6 +5,20 @@ import { ApiEndpointSection } from '@/components/ApiEndpointSection';
import type { EndpointDef } from '@/components/ApiEndpointSection'; import type { EndpointDef } from '@/components/ApiEndpointSection';
import Link from 'next/link'; import Link from 'next/link';
import { ArrowLeft, ArrowRight, Search, FileCode, Users, Compass, Mail } from 'lucide-react'; import { ArrowLeft, ArrowRight, Search, FileCode, Users, Compass, Mail } from 'lucide-react';
import { getPageAlternates } from '@/lib/seo';
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/docs/api'),
};
}
export default async function ApiDocsPage({ export default async function ApiDocsPage({
params, params,
@@ -47,7 +61,7 @@ export default async function ApiDocsPage({
{ name: 'format', type: 'string', required: false, description: t('api.params.format'), default: 'skill.md' }, { name: 'format', type: 'string', required: false, description: t('api.params.format'), default: 'skill.md' },
{ name: 'verified', type: 'boolean', required: false, description: t('api.params.verified') }, { name: 'verified', type: 'boolean', required: false, description: t('api.params.verified') },
{ name: 'minStars', type: 'number', required: false, description: t('api.params.minStars') }, { name: 'minStars', type: 'number', required: false, description: t('api.params.minStars') },
{ name: 'sort', type: 'string', required: false, description: t('api.params.sort') + ' (stars, downloads, rating, recent)', default: 'downloads' }, { name: 'sort', type: 'string', required: false, description: t('api.params.sort') + ' (recommended, aiScore, downloads, stars, rating, recent)', default: 'recommended' },
{ name: 'page', type: 'number', required: false, description: t('api.params.page'), default: '1' }, { name: 'page', type: 'number', required: false, description: t('api.params.page'), default: '1' },
{ name: 'limit', type: 'number', required: false, description: t('api.params.limit'), default: '20' }, { name: 'limit', type: 'number', required: false, description: t('api.params.limit'), default: '20' },
], ],
@@ -65,7 +79,9 @@ export default async function ApiDocsPage({
"rating": 4.5, "rating": 4.5,
"ratingCount": 23, "ratingCount": 23,
"isVerified": true, "isVerified": true,
"compatibility": { "platforms": ["claude", "cursor"] } "compatibility": { "platforms": ["claude", "cursor"] },
"reviewStatus": "ai-reviewed",
"aiScore": 85
} }
], ],
"pagination": { "pagination": {
@@ -74,7 +90,7 @@ export default async function ApiDocsPage({
"total": 150, "total": 150,
"totalPages": 8 "totalPages": 8
}, },
"searchEngine": "meilisearch" "searchEngine": "postgresql"
}`, }`,
notes: t('api.notes.searchFallback'), notes: t('api.notes.searchFallback'),
}, },
@@ -101,7 +117,17 @@ export default async function ApiDocsPage({
"isVerified": true, "isVerified": true,
"compatibility": { "platforms": ["claude"] }, "compatibility": { "platforms": ["claude"] },
"rawContent": "# Code Review\\n...", "rawContent": "# Code Review\\n...",
"sourceFormat": "skill.md" "sourceFormat": "skill.md",
"reviewStatus": "ai-reviewed",
"aiScore": 85,
"review": {
"ai_score": 85,
"instruction_quality": 90,
"description_precision": 80,
"usefulness": 85,
"technical_soundness": 85,
"review_notes": "RATIONALE: ..."
}
}`, }`,
notes: t('api.notes.viewCount'), notes: t('api.notes.viewCount'),
}, },

View File

@@ -3,6 +3,20 @@ import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import Link from 'next/link'; import Link from 'next/link';
import { ArrowLeft, ArrowRight } from 'lucide-react'; import { ArrowLeft, ArrowRight } from 'lucide-react';
import { getPageAlternates } from '@/lib/seo';
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/docs/cli'),
};
}
export default async function CliDocsPage({ export default async function CliDocsPage({
params, params,
@@ -106,7 +120,10 @@ export default async function CliDocsPage({
<div> <div>
<p className="text-sm text-text-muted mb-1">{tContent('exSearchSort')}</p> <p className="text-sm text-text-muted mb-1">{tContent('exSearchSort')}</p>
<code className="block text-sm font-mono"> <code className="block text-sm font-mono">
<span className="text-text-muted">$</span> npx skillhub search &quot;code review&quot; --sort stars --limit 5 <span className="text-text-muted">$</span> npx skillhub search &quot;code review&quot; --sort aiScore --limit 5
</code>
<code className="block text-sm font-mono text-text-muted mt-1">
# Sort options: recommended (default), aiScore, downloads, stars, rating, recent
</code> </code>
</div> </div>
{/* Update all */} {/* Update all */}

View File

@@ -6,18 +6,24 @@ import Link from 'next/link';
import { ArrowLeft, ArrowRight } from 'lucide-react'; import { ArrowLeft, ArrowRight } from 'lucide-react';
import { createDb, skills, sql } from '@skillhub/db'; import { createDb, skills, sql } from '@skillhub/db';
import { formatPromptSkillCount } from '@/lib/format-number'; import { formatPromptSkillCount } from '@/lib/format-number';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
async function getSkillCount(): Promise<string> { async function getSkillCount(): Promise<string> {
try { try {
return await getOrSetCache(cacheKeys.pageCount('getting-started'), cacheTTL.pageCount, async () => {
const db = createDb(); const db = createDb();
const result = await db const result = await db
.select({ count: sql<number>`count(*)::int` }) .select({ count: sql<number>`count(*)::int` })
.from(skills); .from(skills)
return formatPromptSkillCount(result[0]?.count ?? 170000); .where(sql`${skills.isDuplicate} = false`);
return formatPromptSkillCount(result[0]?.count ?? 16000);
});
} catch { } catch {
return '170,000+'; return '16,000+';
} }
} }
@@ -38,6 +44,7 @@ npx skillhub uninstall <skill-name> # when done
Rules: Rules:
- Only install skills with security status "Pass" - Only install skills with security status "Pass"
- Prefer skills with AI review scores (shown as "AI 75+" in results)
- After installing, read the SKILL.md and follow its instructions - After installing, read the SKILL.md and follow its instructions
- Search during planning phase for best results`, - Search during planning phase for best results`,
@@ -49,9 +56,21 @@ npx skillhub install <skill-id> --project
npx skillhub uninstall <skill-name> npx skillhub uninstall <skill-name>
\`\`\` \`\`\`
Search for unfamiliar tech or complex tasks. Only install "Pass" security status. Read SKILL.md after install.`, Search for unfamiliar tech or complex tasks. Only install "Pass" security. Prefer "AI 75+" reviewed skills. Read SKILL.md after install.`,
oneshot: `You have access to SkillHub (${count} AI agent skills, ${siteUrl}). For unfamiliar or complex tasks: \`npx skillhub search "<query>" --limit 5\` then \`npx skillhub install <id> --project\`. Only "Pass" security. Read SKILL.md after install. Uninstall when done.`, oneshot: `You have access to SkillHub (${count} AI agent skills, ${siteUrl}). For unfamiliar or complex tasks: \`npx skillhub search "<query>" --limit 5\` then \`npx skillhub install <id> --project\`. Only "Pass" security, prefer "AI 75+" reviewed. Read SKILL.md after install. Uninstall when done.`,
};
}
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/docs/getting-started'),
}; };
} }

View File

@@ -3,6 +3,20 @@ import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import { BookOpen, Terminal, Code } from 'lucide-react'; import { BookOpen, Terminal, Code } from 'lucide-react';
import Link from 'next/link'; import Link from 'next/link';
import { getPageAlternates } from '@/lib/seo';
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/docs'),
};
}
export default async function DocsPage({ export default async function DocsPage({
params, params,

View File

@@ -6,6 +6,8 @@ import { Footer } from '@/components/Footer';
import { FavoritesList } from '@/components/FavoritesList'; import { FavoritesList } from '@/components/FavoritesList';
import { FavoritesSignIn } from '@/components/FavoritesSignIn'; import { FavoritesSignIn } from '@/components/FavoritesSignIn';
import { createDb, userQueries } from '@skillhub/db'; import { createDb, userQueries } from '@skillhub/db';
import { getPageAlternates } from '@/lib/seo';
// Force dynamic rendering // Force dynamic rendering
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -14,6 +16,18 @@ interface FavoritesPageProps {
params: Promise<{ locale: string }>; params: Promise<{ locale: string }>;
} }
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/favorites'),
};
}
export default async function FavoritesPage({ params }: FavoritesPageProps) { export default async function FavoritesPage({ params }: FavoritesPageProps) {
const { locale } = await params; const { locale } = await params;
setRequestLocale(locale); setRequestLocale(locale);

View File

@@ -5,6 +5,9 @@ import { createDb, skillQueries } from '@skillhub/db';
import { toPersianNumber } from '@/lib/format-number'; import { toPersianNumber } from '@/lib/format-number';
import { Pagination } from '@/components/BrowseFilters'; import { Pagination } from '@/components/BrowseFilters';
import { SkillCard } from '@/components/SkillCard'; import { SkillCard } from '@/components/SkillCard';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
// Force dynamic rendering to fetch fresh data from database // Force dynamic rendering to fetch fresh data from database
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -14,9 +17,10 @@ interface FeaturedPageProps {
searchParams: Promise<{ page?: string }>; searchParams: Promise<{ page?: string }>;
} }
// Get featured skills with pagination // Get featured skills with pagination and Redis caching (2 hour TTL)
async function getFeaturedSkills(page: number, limit: number) { async function getFeaturedSkills(page: number, limit: number) {
try { try {
return await getOrSetCache(cacheKeys.featuredPage(page), cacheTTL.featured, async () => {
const db = createDb(); const db = createDb();
const offset = (page - 1) * limit; const offset = (page - 1) * limit;
@@ -31,12 +35,25 @@ async function getFeaturedSkills(page: number, limit: number) {
} }
return { skills: featuredSkills, total }; return { skills: featuredSkills, total };
});
} catch (error) { } catch (error) {
console.error('Error fetching featured skills:', error); console.error('Error fetching featured skills:', error);
return { skills: [], total: 0 }; return { skills: [], total: 0 };
} }
} }
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/featured'),
};
}
export default async function FeaturedPage({ export default async function FeaturedPage({
params, params,
searchParams, searchParams,

View File

@@ -5,7 +5,9 @@ import { notFound } from 'next/navigation';
import { locales, localeDirection, type Locale } from '@/i18n'; import { locales, localeDirection, type Locale } from '@/i18n';
import { Providers } from '../providers'; import { Providers } from '../providers';
import { Suspense } from 'react'; import { Suspense } from 'react';
import { SecurityAlertBanner } from '@/components/SecurityAlertBanner';
import { QueryNotification } from '@/components/QueryNotification'; import { QueryNotification } from '@/components/QueryNotification';
import { ProgressBar } from '@/components/ProgressBar';
import '../globals.css'; import '../globals.css';
export async function generateMetadata({ export async function generateMetadata({
@@ -67,9 +69,11 @@ export default async function LocaleLayout({
<body className="min-h-screen bg-surface"> <body className="min-h-screen bg-surface">
<Providers> <Providers>
<NextIntlClientProvider messages={messages}> <NextIntlClientProvider messages={messages}>
<SecurityAlertBanner />
<Suspense fallback={null}> <Suspense fallback={null}>
<QueryNotification /> <QueryNotification />
</Suspense> </Suspense>
<ProgressBar />
{children} {children}
</NextIntlClientProvider> </NextIntlClientProvider>
</Providers> </Providers>

View File

@@ -0,0 +1,149 @@
import { getTranslations, setRequestLocale } from 'next-intl/server';
import Link from 'next/link';
import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer';
import { createDb, skillReviewQueries } from '@skillhub/db';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
import { ShieldAlert, Ban, ExternalLink } from 'lucide-react';
export const dynamic = 'force-dynamic';
async function getMalwareData(page: number, limit: number) {
try {
return await getOrSetCache(cacheKeys.malwarePage(page), cacheTTL.malware, async () => {
const db = createDb();
const offset = (page - 1) * limit;
const [skills, total] = await Promise.all([
skillReviewQueries.getMaliciousSkills(db, limit, offset),
skillReviewQueries.countMaliciousSkills(db),
]);
return { skills, total };
});
} catch (error) {
console.error('Error fetching malware skills:', error);
return { skills: [], total: 0 };
}
}
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/malware'),
};
}
export default async function MalwarePage({
params,
searchParams,
}: {
params: Promise<{ locale: string }>;
searchParams: Promise<{ page?: string }>;
}) {
const { locale } = await params;
const searchParamsResolved = await searchParams;
setRequestLocale(locale);
const t = await getTranslations('malwarePage');
const tMalicious = await getTranslations('malicious');
const limit = 20;
const page = parseInt(searchParamsResolved.page || '1');
const { skills: malwareSkills } = await getMalwareData(page, limit);
return (
<div className="min-h-screen flex flex-col">
<Header />
<main className="flex-1">
{/* Header */}
<section className="section-header bg-gradient-subtle">
<div className="container-main text-center">
<div className="inline-flex items-center justify-center w-14 h-14 rounded-2xl bg-error/10 text-error mb-4">
<ShieldAlert className="w-7 h-7" />
</div>
<h1 className="hero-title mb-4">{t('title')}</h1>
<p className="hero-subtitle max-w-2xl mx-auto">{t('subtitle')}</p>
</div>
</section>
<section className="section bg-surface">
<div className="container-main max-w-4xl">
{malwareSkills.length === 0 ? (
<p className="text-center text-text-muted py-12">{t('noResults')}</p>
) : (
<div className="space-y-4">
{malwareSkills.map((skill) => (
<div
key={skill.id}
className="card p-5 border-error/20 bg-error/[0.02]"
>
<div className="flex flex-col sm:flex-row sm:items-start gap-4">
<div className="flex-1 min-w-0">
<div className="flex items-center gap-2 mb-2 flex-wrap">
<Link
href={`/${locale}/skill/${skill.id}`}
className="text-lg font-semibold text-text-primary hover:text-primary-600 transition-colors truncate"
>
{skill.name || skill.id}
</Link>
<span className="flex items-center gap-1 px-1.5 py-0.5 text-xs font-bold rounded border text-error bg-error/10 border-error/20 shrink-0">
<ShieldAlert className="w-3 h-3" />
{tMalicious('badge')}
</span>
</div>
<p className="text-sm text-text-muted mb-2 font-mono">
{skill.githubOwner}/{skill.githubRepo}
</p>
{skill.description && (
<p className="text-sm text-text-secondary line-clamp-2 mb-3">
{skill.description}
</p>
)}
<div className="flex items-center gap-4 text-xs text-text-muted flex-wrap">
<span className="flex items-center gap-1">
<Ban className="w-3.5 h-3.5 text-error" />
{t('downloadBlocked')}
</span>
<span className="flex items-center gap-1">
<Ban className="w-3.5 h-3.5 text-error" />
{t('installBlocked')}
</span>
</div>
</div>
<a
href={`https://github.com/${skill.githubOwner}/${skill.githubRepo}`}
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-1 text-sm text-text-muted hover:text-text-primary transition-colors shrink-0"
>
GitHub <ExternalLink className="w-3.5 h-3.5" />
</a>
</div>
</div>
))}
</div>
)}
{/* Back to stats link */}
<div className="text-center mt-8">
<Link
href={`/${locale}/reviewed/stats`}
className="btn-secondary"
>
{locale === 'fa' ? 'بازگشت به آمار بررسی‌ها' : 'Back to Review Statistics'}
</Link>
</div>
</div>
</section>
</main>
<Footer />
</div>
);
}

View File

@@ -7,6 +7,9 @@ import { Clock, RefreshCw } from 'lucide-react';
import { createDb, skillQueries } from '@skillhub/db'; import { createDb, skillQueries } from '@skillhub/db';
import { toPersianNumber } from '@/lib/format-number'; import { toPersianNumber } from '@/lib/format-number';
import { Pagination } from '@/components/BrowseFilters'; import { Pagination } from '@/components/BrowseFilters';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
// Force dynamic rendering to fetch fresh data from database // Force dynamic rendering to fetch fresh data from database
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -17,10 +20,11 @@ interface NewSkillsPageProps {
} }
// Format date to "X hours/days ago" with locale support // Format date to "X hours/days ago" with locale support
function formatTimeAgo(date: Date | null, locale: string): string { function formatTimeAgo(date: Date | string | null, locale: string): string {
if (!date) return locale === 'fa' ? 'اخیراً' : 'Recently'; if (!date) return locale === 'fa' ? 'اخیراً' : 'Recently';
const d = date instanceof Date ? date : new Date(date);
const now = new Date(); const now = new Date();
const diffMs = now.getTime() - date.getTime(); const diffMs = now.getTime() - d.getTime();
const diffHours = Math.floor(diffMs / (1000 * 60 * 60)); const diffHours = Math.floor(diffMs / (1000 * 60 * 60));
const diffDays = Math.floor(diffHours / 24); const diffDays = Math.floor(diffHours / 24);
@@ -39,9 +43,10 @@ function formatTimeAgo(date: Date | null, locale: string): string {
return 'Just now'; return 'Just now';
} }
// Get skills based on tab with pagination // Get skills based on tab with pagination and Redis caching (30 min TTL)
async function getSkillsForTab(tab: 'new' | 'updated', page: number, limit: number) { async function getSkillsForTab(tab: 'new' | 'updated', page: number, limit: number) {
try { try {
return await getOrSetCache(cacheKeys.newSkills(tab, page), cacheTTL.newSkills, async () => {
const db = createDb(); const db = createDb();
const offset = (page - 1) * limit; const offset = (page - 1) * limit;
@@ -54,27 +59,42 @@ async function getSkillsForTab(tab: 'new' | 'updated', page: number, limit: numb
const total = await skillQueries.countUpdatedSkills(db); const total = await skillQueries.countUpdatedSkills(db);
return { skills, total }; return { skills, total };
} }
});
} catch (error) { } catch (error) {
console.error('Error fetching skills:', error); console.error('Error fetching skills:', error);
return { skills: [], total: 0 }; return { skills: [], total: 0 };
} }
} }
// Get counts for both tabs // Get counts for both tabs with Redis caching (30 min TTL)
async function getTabCounts() { async function getTabCounts() {
try { try {
return await getOrSetCache(cacheKeys.newSkillsCounts(), cacheTTL.newSkills, async () => {
const db = createDb(); const db = createDb();
const [newCount, updatedCount] = await Promise.all([ const [newCount, updatedCount] = await Promise.all([
skillQueries.countNewSkills(db), skillQueries.countNewSkills(db),
skillQueries.countUpdatedSkills(db), skillQueries.countUpdatedSkills(db),
]); ]);
return { newCount, updatedCount }; return { newCount, updatedCount };
});
} catch (error) { } catch (error) {
console.error('Error fetching counts:', error); console.error('Error fetching counts:', error);
return { newCount: 0, updatedCount: 0 }; return { newCount: 0, updatedCount: 0 };
} }
} }
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/new'),
};
}
export default async function NewSkillsPage({ export default async function NewSkillsPage({
params, params,
searchParams, searchParams,

View File

@@ -7,9 +7,25 @@ import { createDb, skillQueries } from '@skillhub/db';
import { formatCompactNumber, toPersianNumber } from '@/lib/format-number'; import { formatCompactNumber, toPersianNumber } from '@/lib/format-number';
import { ExternalLink, Download, Package, Eye, GitFork, ArrowUpDown, FolderGit2 } from 'lucide-react'; import { ExternalLink, Download, Package, Eye, GitFork, ArrowUpDown, FolderGit2 } from 'lucide-react';
import Link from 'next/link'; import Link from 'next/link';
import type { Metadata } from 'next';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
import { auth } from '@/lib/auth';
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string; username: string }>;
}): Promise<Metadata> {
const { locale, username } = await params;
return {
title: `${decodeURIComponent(username)} | SkillHub`,
alternates: getPageAlternates(locale, `/owner/${username}`),
};
}
const ITEMS_PER_PAGE = 24; const ITEMS_PER_PAGE = 24;
type SortOption = 'popularity' | 'downloads' | 'stars'; type SortOption = 'popularity' | 'downloads' | 'stars';
@@ -34,12 +50,18 @@ export default async function OwnerPage({ params, searchParams }: OwnerPageProps
const activeRepo = searchParamsResolved.repo || ''; const activeRepo = searchParamsResolved.repo || '';
const db = createDb(); const db = createDb();
const session = await auth();
const isOwner = session?.user?.username?.toLowerCase() === username.toLowerCase();
// Fetch stats, count, and repo list in parallel // Fetch stats and repo list with caching (30 min TTL), count is dynamic per filter
const [stats, totalSkills, ownerRepos] = await Promise.all([ const [stats, totalSkills, ownerRepos] = await Promise.all([
skillQueries.getOwnerStats(db, username), getOrSetCache(cacheKeys.ownerStats(username), cacheTTL.owner, () =>
skillQueries.getOwnerStats(db, username)
),
skillQueries.countByOwner(db, username, activeRepo || undefined), skillQueries.countByOwner(db, username, activeRepo || undefined),
skillQueries.getOwnerRepos(db, username), getOrSetCache(cacheKeys.ownerRepos(username), cacheTTL.owner, () =>
skillQueries.getOwnerRepos(db, username)
),
]); ]);
if (stats.totalSkills === 0) { if (stats.totalSkills === 0) {
@@ -183,8 +205,7 @@ export default async function OwnerPage({ params, searchParams }: OwnerPageProps
<span className="text-sm text-text-secondary">{t('repo.filter')}:</span> <span className="text-sm text-text-secondary">{t('repo.filter')}:</span>
<Link <Link
href={buildUrl({ repo: '', page: 1 })} href={buildUrl({ repo: '', page: 1 })}
className={`px-3 py-1 text-xs rounded-full transition-colors ${ className={`px-3 py-1 text-xs rounded-full transition-colors ${!activeRepo
!activeRepo
? 'bg-primary text-primary-foreground font-medium' ? 'bg-primary text-primary-foreground font-medium'
: 'bg-surface-elevated text-text-secondary hover:bg-surface-subtle border border-border' : 'bg-surface-elevated text-text-secondary hover:bg-surface-subtle border border-border'
}`} }`}
@@ -195,8 +216,7 @@ export default async function OwnerPage({ params, searchParams }: OwnerPageProps
<Link <Link
key={r.repo} key={r.repo}
href={buildUrl({ repo: r.repo, page: 1 })} href={buildUrl({ repo: r.repo, page: 1 })}
className={`px-3 py-1 text-xs rounded-full transition-colors ${ className={`px-3 py-1 text-xs rounded-full transition-colors ${activeRepo === r.repo
activeRepo === r.repo
? 'bg-primary text-primary-foreground font-medium' ? 'bg-primary text-primary-foreground font-medium'
: 'bg-surface-elevated text-text-secondary hover:bg-surface-subtle border border-border' : 'bg-surface-elevated text-text-secondary hover:bg-surface-subtle border border-border'
}`} }`}
@@ -227,8 +247,7 @@ export default async function OwnerPage({ params, searchParams }: OwnerPageProps
<Link <Link
key={opt.value} key={opt.value}
href={buildUrl({ sort: opt.value, page: 1 })} href={buildUrl({ sort: opt.value, page: 1 })}
className={`px-3 py-1.5 text-sm rounded-lg transition-colors ${ className={`px-3 py-1.5 text-sm rounded-lg transition-colors ${sort === opt.value
sort === opt.value
? 'bg-primary text-primary-foreground font-medium' ? 'bg-primary text-primary-foreground font-medium'
: 'bg-surface-elevated text-text-secondary hover:bg-surface-subtle border border-border' : 'bg-surface-elevated text-text-secondary hover:bg-surface-subtle border border-border'
}`} }`}
@@ -240,7 +259,20 @@ export default async function OwnerPage({ params, searchParams }: OwnerPageProps
</div> </div>
</div> </div>
{/* Claim CTA - subtle inline hint */} {/* Claim CTA */}
{isOwner ? (
<div className="flex items-center gap-3 mb-6 px-4 py-3 bg-primary/5 border border-primary/20 rounded-lg">
<div className="flex-1 text-sm text-text-secondary">
{t('ownerManageCta')}
</div>
<Link
href={`/${locale}/claim`}
className="text-sm text-primary hover:underline font-medium whitespace-nowrap"
>
{t('ownerManageButton')}
</Link>
</div>
) : (
<div className="flex items-center gap-2 mb-6 text-xs text-text-muted"> <div className="flex items-center gap-2 mb-6 text-xs text-text-muted">
<span>{t('claimCta')}</span> <span>{t('claimCta')}</span>
<Link <Link
@@ -250,6 +282,7 @@ export default async function OwnerPage({ params, searchParams }: OwnerPageProps
{t('claimButton')} {t('claimButton')}
</Link> </Link>
</div> </div>
)}
{/* Repos + Skills */} {/* Repos + Skills */}
{repos.map((repo) => ( {repos.map((repo) => (

View File

@@ -5,56 +5,66 @@ import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import { HeroSearch } from '@/components/HeroSearch'; import { HeroSearch } from '@/components/HeroSearch';
import { SkillCard } from '@/components/SkillCard'; import { SkillCard } from '@/components/SkillCard';
import { createDb, categoryQueries, skillQueries, skills, sql } from '@skillhub/db'; import { createDb, skillQueries, skills, sql } from '@skillhub/db';
import { formatCompactNumber } from '@/lib/format-number'; import { formatCompactNumber } from '@/lib/format-number';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
// Force dynamic rendering to fetch fresh data from database // Force dynamic rendering to fetch fresh data from database
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
// Get stats directly from database // Get stats with Redis caching (1 hour TTL)
async function getStats() { async function getStats() {
try { try {
return await getOrSetCache(cacheKeys.homeStats(), cacheTTL.stats, async () => {
const db = createDb(); const db = createDb();
// Get total skills count (SKILL.md only - real reusable skills) // Browse-ready filter: exclude duplicates (matches browseReadyFilter in queries.ts)
const skillsResult = await db const browseReady = sql`${skills.isDuplicate} = false AND ${skills.isStale} = false`;
.select({ count: sql<number>`count(*)::int` })
// Run all independent count queries in parallel
const [skillsResult, downloadsResult, contributorsResult, totalIndexedResult, reviewedResult] = await Promise.all([
// Get total skills count (browse-ready, SKILL.md only)
db.select({ count: sql<number>`count(*)::int` })
.from(skills) .from(skills)
.where(sql`${skills.sourceFormat} = 'skill.md' AND ${skills.isBlocked} = false`); .where(sql`${skills.sourceFormat} = 'skill.md' AND ${skills.isBlocked} = false AND ${browseReady}`),
const totalSkills = skillsResult[0]?.count ?? 0; // Get total downloads (ALL skills — downloads are real user actions)
db.select({ sum: sql<number>`coalesce(sum(${skills.downloadCount}), 0)::int` })
// Get total downloads .from(skills),
const downloadsResult = await db // Get unique contributors (browse-ready skills only)
.select({ sum: sql<number>`coalesce(sum(${skills.downloadCount}), 0)::int` }) db.select({ count: sql<number>`count(distinct ${skills.githubOwner})::int` })
.from(skills); .from(skills)
const totalDownloads = downloadsResult[0]?.sum ?? 0; .where(browseReady),
// Get total indexed skills (all, before curation) for curation note
// Get total categories db.select({ count: sql<number>`count(*)::int` })
const categories = await categoryQueries.getAll(db); .from(skills)
const totalCategories = categories.length; .where(sql`${skills.isBlocked} = false`),
// Get AI-reviewed skills count
// Get unique contributors (github owners) db.select({ count: sql<number>`count(*)::int` })
const contributorsResult = await db .from(skills)
.select({ count: sql<number>`count(distinct ${skills.githubOwner})::int` }) .where(sql`${skills.sourceFormat} = 'skill.md' AND ${skills.isBlocked} = false AND ${browseReady} AND ${skills.reviewStatus} IN ('ai-reviewed', 'verified')`),
.from(skills); ]);
const totalContributors = contributorsResult[0]?.count ?? 0;
return { return {
totalSkills, totalSkills: skillsResult[0]?.count ?? 0,
totalDownloads, totalDownloads: downloadsResult[0]?.sum ?? 0,
totalCategories, totalContributors: contributorsResult[0]?.count ?? 0,
totalContributors, totalIndexed: totalIndexedResult[0]?.count ?? 0,
totalReviewed: reviewedResult[0]?.count ?? 0,
platforms: 5, platforms: 5,
}; };
});
} catch (error) { } catch (error) {
console.error('Error fetching stats:', error); console.error('Error fetching stats:', error);
return null; return null;
} }
} }
// Get featured skills directly from database // Get featured skills with Redis caching (2 hour TTL)
async function getFeaturedSkills() { async function getFeaturedSkills() {
try { try {
return await getOrSetCache(cacheKeys.homeFeatured(), cacheTTL.featured, async () => {
const db = createDb(); const db = createDb();
// Get featured skills, or top skills by popularity if none are featured // Get featured skills, or top skills by popularity if none are featured
let featuredSkills = await skillQueries.getFeatured(db, 6); let featuredSkills = await skillQueries.getFeatured(db, 6);
@@ -63,6 +73,7 @@ async function getFeaturedSkills() {
featuredSkills = await skillQueries.getFeaturedWithDiversity(db, 6, 2, 3); featuredSkills = await skillQueries.getFeaturedWithDiversity(db, 6, 2, 3);
} }
return featuredSkills; return featuredSkills;
});
} catch (error) { } catch (error) {
console.error('Error fetching featured skills:', error); console.error('Error fetching featured skills:', error);
return []; return [];
@@ -70,6 +81,18 @@ async function getFeaturedSkills() {
} }
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/'),
};
}
export default async function HomePage({ export default async function HomePage({
params, params,
}: { }: {
@@ -89,10 +112,10 @@ export default async function HomePage({
]); ]);
const stats = [ const stats = [
{ value: statsData ? formatCompactNumber(statsData.totalSkills, locale) : '۰', label: t('stats.skills'), icon: Layers }, { value: statsData ? formatCompactNumber(statsData.totalSkills, locale) : '۰', label: t('stats.skills'), icon: Layers, href: `/${locale}/browse` },
{ value: statsData ? formatCompactNumber(statsData.totalDownloads, locale) : '۰', label: t('stats.downloads'), icon: Download }, { value: statsData ? formatCompactNumber(statsData.totalDownloads, locale) : '۰', label: t('stats.downloads'), icon: Download, href: `/${locale}/browse?sort=downloads` },
{ value: statsData ? formatCompactNumber(statsData.totalContributors, locale) : '۰', label: t('stats.contributors'), icon: Users }, { value: statsData ? formatCompactNumber(statsData.totalContributors, locale) : '۰', label: t('stats.contributors'), icon: Users, href: `/${locale}/attribution` },
{ value: statsData ? formatCompactNumber(statsData.totalCategories || 8, locale) : '۸', label: t('stats.categories'), icon: Sparkles }, { value: statsData ? formatCompactNumber(statsData.totalReviewed ?? 0, locale) : '۰', label: t('stats.reviewed'), icon: Sparkles, href: `/${locale}/reviewed` },
]; ];
const steps = [ const steps = [
@@ -168,19 +191,22 @@ export default async function HomePage({
<div className="container-main"> <div className="container-main">
<div className="grid grid-cols-2 md:grid-cols-4 gap-8"> <div className="grid grid-cols-2 md:grid-cols-4 gap-8">
{stats.map((stat, index) => ( {stats.map((stat, index) => (
<div key={index} className="text-center"> <Link key={index} href={stat.href} className="text-center group">
<div className="inline-flex items-center justify-center w-12 h-12 rounded-xl bg-primary-50 text-primary-600 mb-3"> <div className="inline-flex items-center justify-center w-12 h-12 rounded-xl bg-primary-50 text-primary-600 mb-3 group-hover:bg-primary-100 transition-colors">
<stat.icon className="w-6 h-6" /> <stat.icon className="w-6 h-6" />
</div> </div>
<div className="text-3xl font-bold text-text-primary ltr-nums mb-1"> <div className="text-3xl font-bold text-text-primary ltr-nums mb-1">
{stat.value} {stat.value}
</div> </div>
<div className="text-text-secondary"> <div className="text-text-secondary group-hover:text-primary-600 transition-colors">
{stat.label} {stat.label}
</div> </div>
</div> </Link>
))} ))}
</div> </div>
<p className="text-center text-text-muted text-sm mt-6">
{t('stats.curationNote', { totalIndexed: formatCompactNumber(statsData?.totalIndexed ?? 0, locale) })}
</p>
</div> </div>
</section> </section>

View File

@@ -3,6 +3,8 @@ import Link from 'next/link';
import { Header } from '@/components/Header'; import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import { Shield, Database, Cookie, Users, Lock, Mail, ArrowRight, ArrowLeft } from 'lucide-react'; import { Shield, Database, Cookie, Users, Lock, Mail, ArrowRight, ArrowLeft } from 'lucide-react';
import { getPageAlternates } from '@/lib/seo';
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -15,6 +17,18 @@ const sectionIcons = {
contact: Mail, contact: Mail,
}; };
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/privacy'),
};
}
export default async function PrivacyPage({ export default async function PrivacyPage({
params, params,
}: { }: {

View File

@@ -0,0 +1,160 @@
import { getTranslations, setRequestLocale } from 'next-intl/server';
import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer';
import { createDb, skillQueries } from '@skillhub/db';
import { toPersianNumber } from '@/lib/format-number';
import { Pagination } from '@/components/BrowseFilters';
import { SkillCard } from '@/components/SkillCard';
import { ReviewedSortSelector } from '@/components/ReviewedSortSelector';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
import { Sparkles, BarChart3 } from 'lucide-react';
import Link from 'next/link';
// Force dynamic rendering to fetch fresh data from database
export const dynamic = 'force-dynamic';
interface ReviewedPageProps {
params: Promise<{ locale: string }>;
searchParams: Promise<{ page?: string; sort?: string; score?: string }>;
}
async function getReviewedSkillsData(sort: string, page: number, limit: number, minScore: number) {
const validSort = (sort === 'aiScore' ? 'aiScore' : 'reviewDate') as 'reviewDate' | 'aiScore';
try {
return await getOrSetCache(cacheKeys.reviewedPage(validSort, page, minScore), cacheTTL.reviewed, async () => {
const db = createDb();
const offset = (page - 1) * limit;
const [skills, total] = await Promise.all([
skillQueries.getReviewedSkills(db, validSort, limit, offset, minScore),
skillQueries.countReviewedSkills(db, minScore),
]);
return { skills, total };
});
} catch (error) {
console.error('Error fetching reviewed skills:', error);
return { skills: [], total: 0 };
}
}
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/reviewed'),
};
}
export default async function ReviewedPage({
params,
searchParams,
}: ReviewedPageProps) {
const { locale } = await params;
const searchParamsResolved = await searchParams;
setRequestLocale(locale);
const t = await getTranslations('reviewed');
const tBrowse = await getTranslations('browse');
const limit = 12;
const page = parseInt(searchParamsResolved.page || '1');
const sort = searchParamsResolved.sort || 'reviewDate';
const minScore = searchParamsResolved.score === '0' ? 0 : 50;
const { skills: reviewedSkills, total } = await getReviewedSkillsData(sort, page, limit, minScore);
const totalPages = Math.ceil(total / limit);
const startItem = (page - 1) * limit + 1;
const endItem = Math.min(page * limit, total);
const paginationTranslations = {
previous: tBrowse('pagination.previous'),
next: tBrowse('pagination.next'),
page: tBrowse('pagination.page'),
of: tBrowse('pagination.of'),
};
const sortTranslations = {
sortBy: t('sortBy'),
reviewDate: t('sortOptions.reviewDate'),
aiScore: t('sortOptions.aiScore'),
scoreFilter: t('scoreFilter'),
scoreAbove50: t('scoreOptions.above50'),
scoreAll: t('scoreOptions.all'),
};
return (
<div className="min-h-screen flex flex-col">
<Header />
<main className="flex-1">
<section className="section-header bg-gradient-subtle">
<div className="container-main text-center">
<div className="inline-flex items-center justify-center w-14 h-14 rounded-2xl bg-primary-50 text-primary-600 mb-4">
<Sparkles className="w-7 h-7" />
</div>
<h1 className="hero-title mb-4">{t('title')}</h1>
<p className="hero-subtitle max-w-2xl mx-auto mb-6">{t('subtitle')}</p>
<p className="text-text-secondary text-sm max-w-3xl mx-auto leading-relaxed mb-4">
{t('explanation')}
</p>
<Link
href={`/${locale}/reviewed/stats`}
className="inline-flex items-center gap-1.5 text-sm text-primary-600 hover:text-primary-700 font-medium"
>
<BarChart3 className="w-4 h-4" />
{locale === 'fa' ? 'مشاهده آمار بررسی‌ها' : 'View Review Statistics'}
</Link>
</div>
</section>
<section className="section bg-surface">
<div className="container-main">
{/* Sort and results info */}
<div className="flex flex-col sm:flex-row items-start sm:items-center justify-between gap-4 mb-6">
{total > 0 && (
<p className="text-text-secondary text-sm">
{tBrowse('resultsRange', {
start: locale === 'fa' ? toPersianNumber(startItem) : startItem,
end: locale === 'fa' ? toPersianNumber(endItem) : endItem,
total: locale === 'fa' ? toPersianNumber(total) : total
})}
</p>
)}
<ReviewedSortSelector
locale={locale}
translations={sortTranslations}
/>
</div>
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-6">
{reviewedSkills.map((skill) => (
<SkillCard
key={skill.id}
skill={skill}
locale={locale}
/>
))}
</div>
{total === 0 && (
<p className="text-center text-text-muted py-12">
{locale === 'fa' ? 'هنوز مهارتی بررسی نشده است.' : 'No reviewed skills yet.'}
</p>
)}
{/* Pagination */}
<Pagination
currentPage={page}
totalPages={totalPages}
locale={locale}
translations={paginationTranslations}
/>
</div>
</section>
</main>
<Footer />
</div>
);
}

View File

@@ -0,0 +1,224 @@
import { getTranslations, setRequestLocale } from 'next-intl/server';
import Link from 'next/link';
import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer';
import { createDb, skillReviewQueries } from '@skillhub/db';
import { getPageAlternates } from '@/lib/seo';
import { getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
import { BarChart3, ShieldAlert, AlertTriangle, CheckCircle } from 'lucide-react';
export const dynamic = 'force-dynamic';
interface StatsData {
pipeline: Record<string, number>;
totalReviews: number;
scoreDistribution: Record<string, number>;
securityStats: Record<string, number>;
malwareCount: number;
}
async function getPublicReviewStats(): Promise<StatsData | null> {
try {
return await getOrSetCache(cacheKeys.reviewStatsPublic(), cacheTTL.reviewStatsPublic, async () => {
const db = createDb();
const [pipeline, totalReviews, scoreDistribution, securityStats, malwareCount] = await Promise.all([
skillReviewQueries.getPublicPipelineStats(db),
skillReviewQueries.countTotalReviews(db),
skillReviewQueries.getScoreDistribution(db),
skillReviewQueries.getSecurityStats(db),
skillReviewQueries.countMaliciousSkills(db),
]);
return { pipeline, totalReviews, scoreDistribution, securityStats, malwareCount };
});
} catch (error) {
console.error('Error fetching review stats:', error);
return null;
}
}
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/reviewed/stats'),
};
}
function StatCard({
icon: Icon,
label,
value,
color,
href,
}: {
icon: React.ElementType;
label: string;
value: number;
color: string;
href?: string;
}) {
const content = (
<div className={`flex items-center gap-3 p-4 rounded-xl border border-border bg-surface ${href ? 'hover:border-primary-300 transition-colors' : ''}`}>
<div className={`flex items-center justify-center w-10 h-10 rounded-lg ${color}`}>
<Icon className="w-5 h-5" />
</div>
<div>
<p className="text-2xl font-bold text-text-primary ltr-nums">{value.toLocaleString()}</p>
<p className="text-sm text-text-secondary">{label}</p>
</div>
</div>
);
if (href) {
return <Link href={href}>{content}</Link>;
}
return content;
}
function BarRow({
label,
value,
total,
color,
}: {
label: string;
value: number;
total: number;
color: string;
}) {
const pct = total > 0 ? (value / total) * 100 : 0;
return (
<div className="space-y-1">
<div className="flex items-center justify-between text-sm">
<span className="text-text-secondary">{label}</span>
<span className="font-medium text-text-primary ltr-nums">
{value.toLocaleString()} <span className="text-text-muted">({pct.toFixed(1)}%)</span>
</span>
</div>
<div className="h-3 rounded-full bg-surface-subtle overflow-hidden">
<div
className={`h-full rounded-full ${color} transition-all duration-500`}
style={{ width: `${Math.max(pct, 0.5)}%` }}
/>
</div>
</div>
);
}
export default async function ReviewStatsPage({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
setRequestLocale(locale);
const t = await getTranslations('reviewStats');
const data = await getPublicReviewStats();
if (!data) {
return (
<div className="min-h-screen flex flex-col">
<Header />
<main className="flex-1 flex items-center justify-center">
<p className="text-text-muted">Failed to load stats.</p>
</main>
<Footer />
</div>
);
}
// Total = all browse-ready SKILL.md skills (for meaningful percentages)
const pipelineTotal = Object.values(data.pipeline).reduce((sum, n) => sum + n, 0);
const scoreTotal =
data.scoreDistribution.high +
data.scoreDistribution.mid +
data.scoreDistribution.low;
return (
<div className="min-h-screen flex flex-col">
<Header />
<main className="flex-1">
{/* Header Section */}
<section className="section-header bg-gradient-subtle">
<div className="container-main text-center">
<div className="inline-flex items-center justify-center w-14 h-14 rounded-2xl bg-primary-50 text-primary-600 mb-4">
<BarChart3 className="w-7 h-7" />
</div>
<h1 className="hero-title mb-4">{t('title')}</h1>
<p className="hero-subtitle max-w-2xl mx-auto">{t('subtitle')}</p>
</div>
</section>
<section className="section bg-surface">
<div className="container-main max-w-4xl">
{/* Review Pipeline */}
<div className="card p-6 mb-8">
<h2 className="text-xl font-semibold text-text-primary mb-1">{t('pipelineTitle')}</h2>
<p className="text-sm text-text-muted mb-6">{t('pipelineDescription')}</p>
<div className="space-y-4">
<BarRow label={t('aiReviewed')} value={data.pipeline['ai-reviewed'] ?? 0} total={pipelineTotal} color="bg-primary-500" />
<BarRow label={t('needsReReview')} value={data.pipeline['needs-re-review'] ?? 0} total={pipelineTotal} color="bg-warning" />
</div>
<div className="mt-6 pt-4 border-t border-border flex items-center justify-between text-sm">
<span className="text-text-secondary">{t('totalReviews')}</span>
<span className="font-semibold text-text-primary ltr-nums">{data.totalReviews.toLocaleString()}</span>
</div>
</div>
{/* Score Distribution */}
<div className="card p-6 mb-8">
<h2 className="text-xl font-semibold text-text-primary mb-1">{t('scoreTitle')}</h2>
<p className="text-sm text-text-muted mb-6">{t('scoreDescription')}</p>
<div className="space-y-4">
<BarRow label={t('scoreHigh')} value={data.scoreDistribution.high} total={scoreTotal} color="bg-success" />
<BarRow label={t('scoreMid')} value={data.scoreDistribution.mid} total={scoreTotal} color="bg-gold" />
<BarRow label={t('scoreLow')} value={data.scoreDistribution.low} total={scoreTotal} color="bg-error" />
</div>
</div>
{/* Security & Malware */}
<div className="grid grid-cols-1 md:grid-cols-2 gap-8">
{/* Security Scan */}
<div className="card p-6">
<h2 className="text-xl font-semibold text-text-primary mb-1">{t('securityTitle')}</h2>
<p className="text-sm text-text-muted mb-6">{t('securityDescription')}</p>
<div className="space-y-3">
<StatCard icon={CheckCircle} label={t('securityPass')} value={data.securityStats.pass} color="bg-success/10 text-success" />
<StatCard icon={AlertTriangle} label={t('securityWarning')} value={data.securityStats.warning} color="bg-warning/10 text-warning" />
<StatCard icon={ShieldAlert} label={t('securityFail')} value={data.securityStats.fail} color="bg-error/10 text-error" />
</div>
</div>
{/* Malware Detection */}
<div className="card p-6">
<h2 className="text-xl font-semibold text-text-primary mb-1">{t('malwareTitle')}</h2>
<p className="text-sm text-text-muted mb-6">{t('malwareDescription')}</p>
<StatCard
icon={ShieldAlert}
label={t('malwareFlagged')}
value={data.malwareCount}
color="bg-error/10 text-error"
href={`/${locale}/malware`}
/>
{data.malwareCount > 0 && (
<Link
href={`/${locale}/malware`}
className="inline-flex items-center gap-1.5 mt-4 text-sm text-primary-600 hover:text-primary-700 font-medium"
>
{t('malwareViewAll')}
</Link>
)}
</div>
</div>
</div>
</section>
</main>
<Footer />
</div>
);
}

View File

@@ -4,7 +4,7 @@ import { notFound } from 'next/navigation';
import { headers } from 'next/headers'; import { headers } from 'next/headers';
import { import {
Star, Download, Shield, CheckCircle, Copy, Star, Download, Shield, CheckCircle, Copy,
ExternalLink, Github, Calendar, User, Tag, ChevronRight, Eye ExternalLink, Github, Calendar, User, Tag, ChevronRight, Eye, Sparkles, XCircle, ShieldAlert
} from 'lucide-react'; } from 'lucide-react';
import { Header } from '@/components/Header'; import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
@@ -12,29 +12,83 @@ import { FavoriteButton } from '@/components/FavoriteButton';
import { RatingStars } from '@/components/RatingStars'; import { RatingStars } from '@/components/RatingStars';
import { InstallSection } from '@/components/InstallSection'; import { InstallSection } from '@/components/InstallSection';
import { ShareButton } from '@/components/ShareButton'; import { ShareButton } from '@/components/ShareButton';
import { createDb, skillQueries } from '@skillhub/db'; import { createDb, skillQueries, skillReviewQueries } from '@skillhub/db';
import { FORMAT_LABELS } from 'skillhub-core'; import { FORMAT_LABELS, parseReviewNotes } from 'skillhub-core';
import { formatCompactNumber } from '@/lib/format-number'; import { formatCompactNumber } from '@/lib/format-number';
import { shouldCountView } from '@/lib/cache'; import { shouldCountView, getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
import type { Metadata } from 'next';
import { getPageAlternates } from '@/lib/seo';
// Force dynamic rendering to fetch fresh data from database // Force dynamic rendering to fetch fresh data from database
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string; id: string[] }>;
}): Promise<Metadata> {
const { locale, id } = await params;
const skillId = id.join('/');
// Optionally fetch skill name for title
const dbSkill = await getSkill(skillId);
const title = dbSkill ? `${dbSkill.name} | SkillHub` : 'SkillHub';
const description = dbSkill ? dbSkill.description : undefined;
return {
title,
description,
alternates: getPageAlternates(locale, `/skill/${skillId}`),
};
}
interface SkillPageProps { interface SkillPageProps {
params: Promise<{ locale: string; id: string[] }>; params: Promise<{ locale: string; id: string[] }>;
} }
// Get skill directly from database // Get skill with Redis caching (1 hour TTL)
async function getSkill(skillId: string) { async function getSkill(skillId: string) {
try { try {
return await getOrSetCache(cacheKeys.skillDetail(skillId), cacheTTL.skill, async () => {
const db = createDb(); const db = createDb();
return await skillQueries.getById(db, skillId); return await skillQueries.getById(db, skillId);
});
} catch (error) { } catch (error) {
console.error('Error fetching skill:', error); console.error('Error fetching skill:', error);
return null; return null;
} }
} }
// Get skill review with Redis caching (1 hour TTL)
async function getSkillReview(skillId: string) {
try {
return await getOrSetCache(cacheKeys.skillReview(skillId), cacheTTL.skill, async () => {
const db = createDb();
return await skillReviewQueries.getLatestBySkillId(db, skillId);
});
} catch {
return null;
}
}
// Score bar component for the review section
function ScoreBar({ label, score }: { label: string; score: number | null | undefined }) {
if (score === null || score === undefined) return null;
const percentage = Math.min(score, 100);
const color = score >= 75 ? 'bg-success' : score >= 50 ? 'bg-gold' : 'bg-text-muted';
return (
<div>
<div className="flex justify-between text-xs mb-1">
<span className="text-text-muted">{label}</span>
<span className="text-text-primary font-medium ltr-nums">{score}</span>
</div>
<div className="h-1.5 bg-surface-subtle rounded-full">
<div className={`h-full rounded-full ${color}`} style={{ width: `${percentage}%` }} />
</div>
</div>
);
}
export default async function SkillPage({ params }: SkillPageProps) { export default async function SkillPage({ params }: SkillPageProps) {
const { locale, id } = await params; const { locale, id } = await params;
setRequestLocale(locale); setRequestLocale(locale);
@@ -44,8 +98,11 @@ export default async function SkillPage({ params }: SkillPageProps) {
const skillId = id.join('/'); const skillId = id.join('/');
const isRTL = locale === 'fa'; const isRTL = locale === 'fa';
// Get skill from database // Get skill and review data from database (in parallel)
const dbSkill = await getSkill(skillId); const [dbSkill, review] = await Promise.all([
getSkill(skillId),
getSkillReview(skillId),
]);
if (!dbSkill) { if (!dbSkill) {
notFound(); notFound();
@@ -56,6 +113,8 @@ export default async function SkillPage({ params }: SkillPageProps) {
notFound(); notFound();
} }
const isMalicious = dbSkill.isMalicious ?? false;
// Track view count with IP-based rate limiting (1 hour cooldown per IP) // Track view count with IP-based rate limiting (1 hour cooldown per IP)
// Get client IP from headers (works with Cloudflare, nginx, etc.) // Get client IP from headers (works with Cloudflare, nginx, etc.)
const headersList = await headers(); const headersList = await headers();
@@ -71,9 +130,9 @@ export default async function SkillPage({ params }: SkillPageProps) {
const db = createDb(); const db = createDb();
shouldCountView(dbSkill.id, clientIp).then((shouldCount) => { shouldCountView(dbSkill.id, clientIp).then((shouldCount) => {
if (shouldCount) { if (shouldCount) {
skillQueries.incrementViews(db, dbSkill.id).catch(() => {}); skillQueries.incrementViews(db, dbSkill.id).catch(() => { });
} }
}).catch(() => {}); }).catch(() => { });
} }
// Map database response to expected format // Map database response to expected format
@@ -94,12 +153,22 @@ export default async function SkillPage({ params }: SkillPageProps) {
securityStatus: dbSkill.securityStatus || 'pass', securityStatus: dbSkill.securityStatus || 'pass',
isVerified: dbSkill.isVerified || false, isVerified: dbSkill.isVerified || false,
createdAt: dbSkill.createdAt, createdAt: dbSkill.createdAt,
updatedAt: dbSkill.updatedAt ? dbSkill.updatedAt.toLocaleDateString(locale === 'fa' ? 'fa-IR' : 'en-US') : 'N/A', updatedAt: dbSkill.updatedAt ? new Date(dbSkill.updatedAt).toLocaleDateString(locale === 'fa' ? 'fa-IR' : 'en-US') : 'N/A',
rating: dbSkill.rating || 0, rating: dbSkill.rating || 0,
ratingCount: dbSkill.ratingCount || 0, ratingCount: dbSkill.ratingCount || 0,
sourceFormat: dbSkill.sourceFormat || 'skill.md', sourceFormat: dbSkill.sourceFormat || 'skill.md',
}; };
// Parse review notes for structured display
const parsedNotes = review?.reviewNotes ? parseReviewNotes(review.reviewNotes) : null;
const hasReview = review && review.aiScore != null && (dbSkill.reviewStatus === 'ai-reviewed' || dbSkill.reviewStatus === 'verified');
const isRejected = hasReview && review.aiScore === 0;
// Review is outdated only if both hashes are non-empty and differ
const reviewOutdated = hasReview
&& review.contentHashAtReview && review.contentHashAtReview.length > 1
&& dbSkill.contentHash && dbSkill.contentHash.length > 1
&& dbSkill.contentHash !== review.contentHashAtReview;
// Content section title based on source format (uses FORMAT_LABELS from skillhub-core) // Content section title based on source format (uses FORMAT_LABELS from skillhub-core)
const getContentTitle = (format: string) => { const getContentTitle = (format: string) => {
const label = FORMAT_LABELS[format as keyof typeof FORMAT_LABELS] || FORMAT_LABELS['skill.md']; const label = FORMAT_LABELS[format as keyof typeof FORMAT_LABELS] || FORMAT_LABELS['skill.md'];
@@ -231,12 +300,13 @@ export default async function SkillPage({ params }: SkillPageProps) {
</p> </p>
{/* Author, Version, License & Last Update */} {/* Author, Version, License & Last Update */}
<div className="flex flex-wrap items-center gap-4 text-sm"> <div className="flex flex-wrap items-center gap-3 text-sm">
<a <a
href={`https://github.com/${skill.author}`} href={`https://github.com/${skill.author}`}
target="_blank" target="_blank"
rel="noopener noreferrer" rel="noopener noreferrer"
className="flex items-center gap-2 text-text-secondary hover:text-primary-600 transition-colors" className="flex items-center gap-2 text-text-secondary hover:text-primary-600 transition-colors"
aria-label={`GitHub profile: ${skill.author}`}
> >
<div className="w-6 h-6 rounded-full bg-surface-subtle flex items-center justify-center"> <div className="w-6 h-6 rounded-full bg-surface-subtle flex items-center justify-center">
<User className="w-4 h-4" /> <User className="w-4 h-4" />
@@ -244,29 +314,25 @@ export default async function SkillPage({ params }: SkillPageProps) {
<span className="font-medium">@{skill.author}</span> <span className="font-medium">@{skill.author}</span>
</a> </a>
{skill.version && ( {skill.version && (
<> <span className="flex items-center gap-1.5 text-text-muted bg-surface-subtle px-2 py-0.5 rounded">
<span className="text-text-muted"></span> <Tag className="w-3.5 h-3.5" />
<span className="flex items-center gap-1.5 text-text-muted">
<Tag className="w-4 h-4" />
<span className="ltr-nums">v{skill.version}</span> <span className="ltr-nums">v{skill.version}</span>
</span> </span>
</>
)} )}
<span className="text-text-muted"></span>
<span className="text-text-muted bg-surface-subtle px-2 py-0.5 rounded"> <span className="text-text-muted bg-surface-subtle px-2 py-0.5 rounded">
{skill.license} {skill.license}
</span> </span>
<span className="text-text-muted"></span> <span className="flex items-center gap-1.5 text-text-muted bg-surface-subtle px-2 py-0.5 rounded">
<span className="flex items-center gap-1.5 text-text-muted"> <Calendar className="w-3.5 h-3.5" />
<Calendar className="w-4 h-4" />
<span className="ltr-nums">{skill.updatedAt}</span> <span className="ltr-nums">{skill.updatedAt}</span>
</span> </span>
</div> </div>
</div> </div>
{/* Right: Actions */} {/* Right: Actions + AI Score Summary */}
<div className="flex items-center gap-3"> <div className="flex flex-col items-start lg:items-end gap-4">
<FavoriteButton skillId={skill.id} size="lg" showLabel={true} /> <div className="flex items-center gap-2">
<FavoriteButton skillId={skill.id} size="lg" showLabel={false} />
<ShareButton <ShareButton
title={skill.name} title={skill.name}
path={`/${locale}/skill/${skill.id}`} path={`/${locale}/skill/${skill.id}`}
@@ -281,10 +347,44 @@ export default async function SkillPage({ params }: SkillPageProps) {
target="_blank" target="_blank"
rel="noopener noreferrer" rel="noopener noreferrer"
className="p-3 rounded-xl bg-surface-elevated hover:bg-surface-subtle border border-border text-text-secondary hover:text-text-primary transition-colors" className="p-3 rounded-xl bg-surface-elevated hover:bg-surface-subtle border border-border text-text-secondary hover:text-text-primary transition-colors"
title="GitHub" aria-label="GitHub repository"
> >
<Github className="w-5 h-5" /> <Github className="w-5 h-5" />
</a> </a>
{skill.homepage && (
<a
href={skill.homepage}
target="_blank"
rel="noopener noreferrer"
className="p-3 rounded-xl bg-surface-elevated hover:bg-surface-subtle border border-border text-text-secondary hover:text-text-primary transition-colors"
aria-label={isRTL ? 'وبسایت' : 'Homepage'}
>
<ExternalLink className="w-5 h-5" />
</a>
)}
</div>
{/* Compact AI Review Score */}
{hasReview && (
<div className={`flex items-center gap-3 px-4 py-3 rounded-xl border ${isRejected ? 'bg-error/5 border-error/20' : 'bg-surface-elevated border-border'}`}>
{isRejected ? (
<>
<XCircle className="w-5 h-5 text-error" />
<span className="text-lg font-bold text-error">
{isRTL ? 'رد شده' : 'Rejected'}
</span>
</>
) : (
<>
<Sparkles className={`w-5 h-5 ${review.aiScore! >= 75 ? 'text-success' : review.aiScore! >= 50 ? 'text-gold' : 'text-text-muted'}`} />
<span className={`text-2xl font-bold ltr-nums ${review.aiScore! >= 75 ? 'text-success' : review.aiScore! >= 50 ? 'text-gold' : 'text-text-primary'}`}>
{review.aiScore}
</span>
<span className="text-sm text-text-muted">{t('review.outOf100')}</span>
</>
)}
</div>
)}
</div> </div>
</div> </div>
</div> </div>
@@ -315,11 +415,57 @@ export default async function SkillPage({ params }: SkillPageProps) {
</div> </div>
)} )}
{/* Malicious Skill Warning Banner */}
{isMalicious && (
<div className="bg-error/10 border-y-2 border-error px-4 py-6">
<div className="container-main flex items-start gap-4">
<ShieldAlert className="w-8 h-8 text-error flex-shrink-0 mt-0.5" />
<div>
<h2 className="text-lg font-bold text-error mb-1" dir="auto">
{isRTL ? 'بدافزار شناسایی شد' : 'Malware Detected'}
</h2>
<p className="text-text-secondary text-sm" dir="auto">
{isRTL
? 'این مهارت به عنوان مخرب شناسایی شده است. شامل کد مبهم‌سازی شده برای دانلود و اجرای بارهای مضر است. دانلود فایل و نصب مسدود شده است.'
: 'This skill has been flagged as malicious. It contains obfuscated code designed to download and execute harmful payloads. File downloads and installation are blocked.'}
</p>
</div>
</div>
</div>
)}
{/* Stale Skill Warning Banner */}
{dbSkill.isStale && (
<div className="bg-warning/10 border-b border-warning/20">
<div className="container-main py-3">
<div className="flex items-start gap-3 text-sm">
<span className="text-warning text-lg flex-shrink-0"></span>
<div>
<p className="text-warning-foreground dark:text-warning" dir="auto">
{isRTL
? 'این مهارت ممکن است از مخزن GitHub اصلی حذف یا جابجا شده باشد. فایل‌ها از حافظه پنهان SkillHub ارائه می‌شوند و ممکن است قدیمی باشند.'
: 'This skill may have been removed or moved from its GitHub repository. Files are served from the SkillHub cache and may be outdated.'}
</p>
<a
href={`https://github.com/${skill.author}/${skill.repo}`}
target="_blank"
rel="noopener noreferrer"
className="inline-flex items-center gap-1 mt-1 text-primary-600 hover:text-primary-700 dark:text-primary-400 dark:hover:text-primary-300 font-medium"
>
{isRTL ? 'بررسی مخزن GitHub' : 'Check GitHub repository'}
<ExternalLink className="w-3 h-3" />
</a>
</div>
</div>
</div>
</div>
)}
{/* Stats Bar */} {/* Stats Bar */}
<div className="bg-surface-elevated border-b border-border"> <div className="bg-surface-elevated border-b border-border">
<div className="container-main"> <div className="container-main">
<div className="flex items-center gap-6 lg:gap-10 py-4 overflow-x-auto"> <div className="flex flex-wrap items-center gap-4 lg:gap-8 py-4">
<div className="flex items-center gap-2 min-w-fit"> <div className="flex items-center gap-2">
<RatingStars <RatingStars
skillId={skill.id} skillId={skill.id}
averageRating={skill.rating} averageRating={skill.rating}
@@ -327,29 +473,29 @@ export default async function SkillPage({ params }: SkillPageProps) {
size="sm" size="sm"
/> />
</div> </div>
<div className="h-6 w-px bg-border" /> <div className="hidden sm:block h-5 w-px bg-border" />
<div className="flex items-center gap-2 min-w-fit"> <div className="flex items-center gap-2">
<Star className="w-5 h-5 text-gold" /> <Star className="w-4 h-4 lg:w-5 lg:h-5 text-gold" />
<span className="font-semibold text-text-primary ltr-nums"> <span className="font-semibold text-text-primary ltr-nums">
{formatCompactNumber(skill.stars, locale)} {formatCompactNumber(skill.stars, locale)}
</span> </span>
<span className="text-text-muted text-sm">{tCommon('stars')}</span> <span className="text-text-muted text-sm hidden sm:inline">{tCommon('stars')}</span>
</div> </div>
<div className="h-6 w-px bg-border" /> <div className="hidden sm:block h-5 w-px bg-border" />
<div className="flex items-center gap-2 min-w-fit"> <div className="flex items-center gap-2">
<Download className="w-5 h-5 text-primary-500" /> <Download className="w-4 h-4 lg:w-5 lg:h-5 text-primary-500" />
<span className="font-semibold text-text-primary ltr-nums"> <span className="font-semibold text-text-primary ltr-nums">
{formatCompactNumber(skill.downloads, locale)} {formatCompactNumber(skill.downloads, locale)}
</span> </span>
<span className="text-text-muted text-sm">{tCommon('downloads')}</span> <span className="text-text-muted text-sm hidden sm:inline">{tCommon('downloads')}</span>
</div> </div>
<div className="h-6 w-px bg-border" /> <div className="hidden sm:block h-5 w-px bg-border" />
<div className="flex items-center gap-2 min-w-fit"> <div className="flex items-center gap-2">
<Eye className="w-5 h-5 text-text-muted" /> <Eye className="w-4 h-4 lg:w-5 lg:h-5 text-text-muted" />
<span className="font-semibold text-text-primary ltr-nums"> <span className="font-semibold text-text-primary ltr-nums">
{formatCompactNumber(skill.views, locale)} {formatCompactNumber(skill.views, locale)}
</span> </span>
<span className="text-text-muted text-sm">{tCommon('views')}</span> <span className="text-text-muted text-sm hidden sm:inline">{tCommon('views')}</span>
</div> </div>
</div> </div>
</div> </div>
@@ -362,6 +508,14 @@ export default async function SkillPage({ params }: SkillPageProps) {
<div className="lg:col-span-2 space-y-6"> <div className="lg:col-span-2 space-y-6">
{/* Quick Install (Mobile) */} {/* Quick Install (Mobile) */}
<div className="lg:hidden"> <div className="lg:hidden">
{isMalicious ? (
<div className="bg-error/5 border border-error/20 rounded-lg p-4 text-center">
<ShieldAlert className="w-6 h-6 text-error mx-auto mb-2" />
<p className="text-sm text-error font-medium" dir="auto">
{isRTL ? 'نصب مسدود شده — این مهارت حاوی بدافزار است' : 'Installation blocked — this skill contains malware'}
</p>
</div>
) : (
<InstallSection <InstallSection
skillId={skill.id} skillId={skill.id}
skillName={skill.name} skillName={skill.name}
@@ -392,8 +546,52 @@ export default async function SkillPage({ params }: SkillPageProps) {
folderNoteSuffix: t('install.folderNoteSuffix') || '" will be created', folderNoteSuffix: t('install.folderNoteSuffix') || '" will be created',
}} }}
/> />
)}
</div> </div>
{/* AI Review Card (Mobile) */}
{hasReview && (
<div className={`lg:hidden rounded-2xl border p-6 ${isRejected ? 'bg-error/5 border-error/20' : 'bg-surface-elevated border-border'}`}>
<h3 className="font-semibold text-text-primary mb-4 flex items-center gap-2">
{isRejected ? <XCircle className="w-4 h-4 text-error" /> : <Sparkles className="w-4 h-4 text-primary-500" />}
{t('review.title')}
</h3>
{isRejected ? (
<div className="space-y-4">
<div className="text-center">
<div className="text-xl font-bold text-error">
{isRTL ? 'رد شده' : 'Rejected'}
</div>
<div className="text-sm text-text-muted">
{isRTL ? 'این مهارت معیارهای کیفیت را ندارد' : 'Does not meet quality standards'}
</div>
</div>
{parsedNotes?.rationale && (
<p className="text-sm text-text-secondary pt-3 border-t border-error/20" dir="auto">{parsedNotes.rationale}</p>
)}
</div>
) : (
<div className="space-y-4">
<div className="space-y-3">
<ScoreBar label={t('review.instructionQuality')} score={review.instructionQuality} />
<ScoreBar label={t('review.descriptionPrecision')} score={review.descriptionPrecision} />
<ScoreBar label={t('review.usefulness')} score={review.usefulness} />
<ScoreBar label={t('review.technicalSoundness')} score={review.technicalSoundness} />
</div>
{parsedNotes?.rationale && (
<p className="text-sm text-text-secondary pt-3 border-t border-border" dir="auto">{parsedNotes.rationale}</p>
)}
{reviewOutdated && (
<p className="text-xs text-warning flex items-center gap-1.5 pt-3 border-t border-border">
<span></span>
{isRTL ? 'بررسی بر اساس نسخه قبلی' : 'Review based on previous version'}
</p>
)}
</div>
)}
</div>
)}
{/* README Section */} {/* README Section */}
<div className="bg-surface-elevated rounded-2xl border border-border overflow-hidden"> <div className="bg-surface-elevated rounded-2xl border border-border overflow-hidden">
<div className="px-6 py-4 border-b border-border bg-surface-subtle/50"> <div className="px-6 py-4 border-b border-border bg-surface-subtle/50">
@@ -404,59 +602,129 @@ export default async function SkillPage({ params }: SkillPageProps) {
</div> </div>
<div className="p-6"> <div className="p-6">
<div className="prose prose-slate dark:prose-invert max-w-none" dir="auto"> <div className="prose prose-slate dark:prose-invert max-w-none" dir="auto">
<pre className="whitespace-pre-wrap text-sm leading-relaxed bg-surface-subtle rounded-xl p-4 overflow-x-auto text-start border border-border"> <pre className="whitespace-pre-wrap break-words text-sm leading-relaxed bg-surface-subtle rounded-xl p-4 overflow-x-auto text-start border border-border">
<code>{skill.longDescription}</code> <code>{skill.longDescription}</code>
</pre> </pre>
</div> </div>
</div> </div>
</div> </div>
{/* Source Links (Mobile) */}
<div className="lg:hidden bg-surface-elevated rounded-2xl border border-border p-6">
<h3 className="font-semibold text-text-primary mb-4">
{isRTL ? 'لینک‌ها' : 'Links'}
</h3>
<div className="space-y-3">
<a
href={skill.repository}
target="_blank"
rel="noopener noreferrer"
className="flex items-center justify-between p-3 rounded-xl bg-surface-subtle hover:bg-surface-muted transition-colors group"
>
<div className="flex items-center gap-3">
<Github className="w-5 h-5 text-text-muted" />
<div>
<div className="font-medium text-text-primary">{t('meta.repository')}</div>
<div className="text-sm text-text-muted">{skill.author}/{skill.repo}</div>
</div>
</div>
<ExternalLink className="w-4 h-4 text-text-muted group-hover:text-primary-600 transition-colors" />
</a>
{skill.homepage && (
<a
href={skill.homepage}
target="_blank"
rel="noopener noreferrer"
className="flex items-center justify-between p-3 rounded-xl bg-surface-subtle hover:bg-surface-muted transition-colors group"
>
<div className="flex items-center gap-3">
<ExternalLink className="w-5 h-5 text-text-muted" />
<div>
<div className="font-medium text-text-primary">{t('meta.homepage')}</div>
<div className="text-sm text-text-muted truncate max-w-[200px]">{skill.homepage}</div>
</div>
</div>
<ExternalLink className="w-4 h-4 text-text-muted group-hover:text-primary-600 transition-colors" />
</a>
)}
</div>
</div>
</div> </div>
{/* Right: Sidebar (Desktop) */} {/* Right: Sidebar (Desktop) */}
<div className="hidden lg:block space-y-6"> <div className="hidden lg:block space-y-6">
<div className="sticky top-24 max-h-[calc(100vh-7rem)] overflow-y-auto space-y-6 scrollbar-thin"> <div className="sticky top-24 max-h-[calc(100vh-7rem)] overflow-y-auto space-y-6 scrollbar-thin">
{/* AI Review Card (Desktop) — above Install for visibility */}
{hasReview && (
<div className={`rounded-2xl border p-6 ${isRejected ? 'bg-error/5 border-error/20' : 'bg-surface-elevated border-border'}`}>
<h3 className="font-semibold text-text-primary mb-4 flex items-center gap-2">
{isRejected ? <XCircle className="w-4 h-4 text-error" /> : <Sparkles className="w-4 h-4 text-primary-500" />}
{t('review.title')}
</h3>
{isRejected ? (
<>
{/* Rejected status */}
<div className="text-center mb-4">
<div className="text-2xl font-bold text-error">
{isRTL ? 'رد شده' : 'Rejected'}
</div>
<div className="text-sm text-text-muted">
{isRTL ? 'این مهارت معیارهای کیفیت را ندارد' : 'Does not meet quality standards'}
</div>
</div>
{/* Rationale for rejection */}
{parsedNotes?.rationale && (
<div className="pt-4 border-t border-error/20">
<p className="text-sm text-text-secondary" dir="auto">{parsedNotes.rationale}</p>
</div>
)}
</>
) : (
<>
{/* Overall Score */}
<div className="text-center mb-4">
<div className={`text-4xl font-bold ${review.aiScore! >= 75 ? 'text-success' : review.aiScore! >= 50 ? 'text-gold' : 'text-text-primary'}`}>
{review.aiScore}
</div>
<div className="text-sm text-text-muted">{t('review.outOf100')}</div>
</div>
{/* 4-axis score bars */}
<div className="space-y-3">
<ScoreBar label={t('review.instructionQuality')} score={review.instructionQuality} />
<ScoreBar label={t('review.descriptionPrecision')} score={review.descriptionPrecision} />
<ScoreBar label={t('review.usefulness')} score={review.usefulness} />
<ScoreBar label={t('review.technicalSoundness')} score={review.technicalSoundness} />
</div>
{/* Rationale */}
{parsedNotes?.rationale && (
<div className="mt-4 pt-4 border-t border-border">
<p className="text-sm text-text-secondary" dir="auto">{parsedNotes.rationale}</p>
</div>
)}
{/* Tags: Audience, Maturity, Complexity, Use Cases */}
{(parsedNotes?.maturity || parsedNotes?.complexity || (parsedNotes?.audience && parsedNotes.audience.length > 0) || (parsedNotes?.useCases && parsedNotes.useCases.length > 0)) && (
<div className="mt-4 pt-4 border-t border-border">
<div className="flex flex-wrap gap-1.5">
{parsedNotes?.maturity && (
<span className="px-2 py-0.5 text-xs rounded bg-surface-subtle text-text-muted border border-border">
{parsedNotes.maturity}
</span>
)}
{parsedNotes?.complexity && (
<span className="px-2 py-0.5 text-xs rounded bg-surface-subtle text-text-muted border border-border">
{parsedNotes.complexity}
</span>
)}
{parsedNotes?.audience?.map((a: string) => (
<span key={a} className="px-2 py-0.5 text-xs rounded bg-primary-50 text-primary-700 dark:bg-primary-900/20 dark:text-primary-400">
{a.trim()}
</span>
))}
{parsedNotes?.useCases?.map((uc: string) => (
<span key={uc} className="px-2 py-0.5 text-xs rounded bg-success/10 text-success">
{uc.trim()}
</span>
))}
</div>
</div>
)}
</>
)}
{/* Reviewer info */}
<div className="mt-3 text-xs text-text-muted">
{t('review.reviewedBy', { reviewer: review.reviewer || 'AI' })}
{review.reviewedAt && (
<> {t('review.reviewedOn', { date: new Date(review.reviewedAt).toLocaleDateString(locale === 'fa' ? 'fa-IR' : 'en-US') })}</>
)}
</div>
{/* Outdated review warning */}
{!isRejected && reviewOutdated && (
<div className="mt-3 pt-3 border-t border-border">
<p className="text-xs text-warning flex items-center gap-1.5">
<span></span>
{isRTL ? 'بررسی بر اساس نسخه قبلی' : 'Review based on previous version'}
</p>
</div>
)}
</div>
)}
{/* Install Section */} {/* Install Section */}
{isMalicious ? (
<div className="bg-error/5 border border-error/20 rounded-lg p-4 text-center">
<ShieldAlert className="w-6 h-6 text-error mx-auto mb-2" />
<p className="text-sm text-error font-medium" dir="auto">
{isRTL ? 'نصب مسدود شده — این مهارت حاوی بدافزار است' : 'Installation blocked — this skill contains malware'}
</p>
</div>
) : (
<InstallSection <InstallSection
skillId={skill.id} skillId={skill.id}
skillName={skill.name} skillName={skill.name}
@@ -487,24 +755,6 @@ export default async function SkillPage({ params }: SkillPageProps) {
folderNoteSuffix: t('install.folderNoteSuffix') || '" will be created', folderNoteSuffix: t('install.folderNoteSuffix') || '" will be created',
}} }}
/> />
{/* Links Card (Desktop) */}
{skill.homepage && (
<div className="bg-surface-elevated rounded-2xl border border-border p-6">
<h3 className="font-semibold text-text-primary mb-4">
{isRTL ? 'لینک‌ها' : 'Links'}
</h3>
<a
href={skill.homepage}
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-2 text-sm text-text-secondary hover:text-primary-600 transition-colors py-2"
>
<ExternalLink className="w-4 h-4" />
<span className="flex-1">{t('meta.homepage')}</span>
<ExternalLink className="w-3 h-3" />
</a>
</div>
)} )}
</div> </div>
</div> </div>

View File

@@ -2,6 +2,8 @@ import { getTranslations, setRequestLocale } from 'next-intl/server';
import { Header } from '@/components/Header'; import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import { Mail, Bitcoin, ExternalLink } from 'lucide-react'; import { Mail, Bitcoin, ExternalLink } from 'lucide-react';
import { getPageAlternates } from '@/lib/seo';
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -9,6 +11,18 @@ interface SupportPageProps {
params: Promise<{ locale: string }>; params: Promise<{ locale: string }>;
} }
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/support'),
};
}
export default async function SupportPage({ params }: SupportPageProps) { export default async function SupportPage({ params }: SupportPageProps) {
const { locale } = await params; const { locale } = await params;
setRequestLocale(locale); setRequestLocale(locale);

View File

@@ -3,6 +3,8 @@ import Link from 'next/link';
import { Header } from '@/components/Header'; import { Header } from '@/components/Header';
import { Footer } from '@/components/Footer'; import { Footer } from '@/components/Footer';
import { FileText, UserCheck, AlertTriangle, Scale, Trash2, RefreshCw, ArrowRight, ArrowLeft } from 'lucide-react'; import { FileText, UserCheck, AlertTriangle, Scale, Trash2, RefreshCw, ArrowRight, ArrowLeft } from 'lucide-react';
import { getPageAlternates } from '@/lib/seo';
export const dynamic = 'force-dynamic'; export const dynamic = 'force-dynamic';
@@ -15,6 +17,18 @@ const sectionIcons = {
changes: RefreshCw, changes: RefreshCw,
}; };
export async function generateMetadata({
params,
}: {
params: Promise<{ locale: string }>;
}) {
const { locale } = await params;
return {
alternates: getPageAlternates(locale, '/terms'),
};
}
export default async function TermsPage({ export default async function TermsPage({
params, params,
}: { }: {

View File

@@ -3,6 +3,7 @@ import { createDb, ratingQueries, skillQueries, userQueries } from '@skillhub/db
import { auth } from '@/lib/auth'; import { auth } from '@/lib/auth';
import { withRateLimit, createRateLimitResponse, createRateLimitHeaders } from '@/lib/rate-limit'; import { withRateLimit, createRateLimitResponse, createRateLimitHeaders } from '@/lib/rate-limit';
import { sanitizeReview } from '@/lib/sanitize'; import { sanitizeReview } from '@/lib/sanitize';
import { getOrSetCache, invalidateCache, cacheKeys, cacheTTL } from '@/lib/cache';
const db = createDb(); const db = createDb();
@@ -28,10 +29,13 @@ export async function GET(request: NextRequest) {
return NextResponse.json({ error: 'skillId is required' }, { status: 400 }); return NextResponse.json({ error: 'skillId is required' }, { status: 400 });
} }
const data = await getOrSetCache(
cacheKeys.skillRatings(skillId, limit, offset),
cacheTTL.ratings,
async () => {
const ratings = await ratingQueries.getForSkill(db, skillId, limit, offset); const ratings = await ratingQueries.getForSkill(db, skillId, limit, offset);
const skill = await skillQueries.getById(db, skillId); const skill = await skillQueries.getById(db, skillId);
return {
return NextResponse.json({
ratings: ratings.map((r) => ({ ratings: ratings.map((r) => ({
id: r.rating.id, id: r.rating.id,
rating: r.rating.rating, rating: r.rating.rating,
@@ -48,7 +52,11 @@ export async function GET(request: NextRequest) {
average: skill?.rating || 0, average: skill?.rating || 0,
count: skill?.ratingCount || 0, count: skill?.ratingCount || 0,
}, },
}, { };
}
);
return NextResponse.json(data, {
headers: createRateLimitHeaders(rateLimitResult), headers: createRateLimitHeaders(rateLimitResult),
}); });
} catch (error) { } catch (error) {
@@ -103,6 +111,12 @@ export async function POST(request: NextRequest) {
review: sanitizeReview(review) ?? undefined, review: sanitizeReview(review) ?? undefined,
}); });
// Invalidate ratings and skill detail caches
await Promise.all([
invalidateCache(cacheKeys.skillRatings(skillId, 10, 0)),
invalidateCache(cacheKeys.skillDetail(skillId)),
]);
// Get updated skill aggregates // Get updated skill aggregates
const updatedSkill = await skillQueries.getById(db, skillId); const updatedSkill = await skillQueries.getById(db, skillId);

View File

@@ -0,0 +1,106 @@
import { type NextRequest, NextResponse } from 'next/server';
import { createDb, skillQueries, sql } from '@skillhub/db';
import { requireAdmin } from '@/lib/admin-auth';
import { withRateLimit, createRateLimitResponse, createRateLimitHeaders } from '@/lib/rate-limit';
const db = createDb();
/**
* GET /api/review/diagnose?id=owner/repo/skill-name
* Returns which review pipeline filters a skill passes/fails.
* Calls the actual PostgreSQL raw_content_passes_prefilter function to detect
* discrepancies between JS approximation and SQL reality (e.g. invalid UTF-8).
* Admin-only endpoint for debugging why skills don't appear in pending list.
*/
export async function GET(request: NextRequest) {
const rateLimitResult = await withRateLimit(request, 'anonymous');
if (!rateLimitResult.allowed) {
return createRateLimitResponse(rateLimitResult);
}
const adminCheck = await requireAdmin(request);
if (!adminCheck.authorized) {
return adminCheck.response;
}
try {
const { searchParams } = new URL(request.url);
const skillId = searchParams.get('id');
if (!skillId) {
return NextResponse.json({ error: 'Missing id parameter' }, { status: 400 });
}
const skill = await skillQueries.getById(db, skillId);
if (!skill) {
return NextResponse.json({ error: 'Skill not found', id: skillId }, { status: 404 });
}
// Call the ACTUAL PostgreSQL function to check prefilter
// This catches UTF-8 issues that the JS approximation misses
let sqlPrefilterPass = false;
try {
const result = await db.execute(
sql`SELECT raw_content_passes_prefilter(raw_content) AS passes FROM skills WHERE id = ${skillId}`
);
const row = [...result][0] as { passes?: boolean } | undefined;
sqlPrefilterPass = row?.passes === true;
} catch {
sqlPrefilterPass = false;
}
// JS approximation for comparison
const rawContent = skill.rawContent ?? '';
const contentLength = Buffer.byteLength(rawContent, 'utf8');
const hasGeneratedComment = rawContent.includes('<!-- generated');
const hasUserPath = rawContent.substring(0, 1000).includes('/Users/') ||
rawContent.substring(0, 1000).includes('C:\\Users\\');
const jsPrefilterPass = contentLength >= 200 && !hasGeneratedComment && !hasUserPath;
const filters = {
// browseReadyFilter conditions
isDuplicate: { value: skill.isDuplicate, pass: !skill.isDuplicate || skill.isOwnerClaimed },
isStale: { value: skill.isStale, pass: !skill.isStale },
isMalicious: { value: skill.isMalicious, pass: !skill.isMalicious },
// Other conditions
isBlocked: { value: skill.isBlocked, pass: !skill.isBlocked },
sourceFormat: { value: skill.sourceFormat, pass: skill.sourceFormat === 'skill.md' },
isDeprecated: { value: skill.isDeprecated, pass: !skill.isDeprecated },
securityStatus: { value: skill.securityStatus, pass: skill.securityStatus === 'pass' },
qualityScore: { value: skill.qualityScore, pass: (skill.qualityScore ?? 0) >= 50 },
reviewStatus: { value: skill.reviewStatus, pass: skill.reviewStatus === 'auto-scored' },
// Prefilter: actual PostgreSQL function result
sqlPrefilter: { value: sqlPrefilterPass, pass: sqlPrefilterPass },
// JS approximation breakdown (for debugging discrepancies)
jsPrefilter: { value: jsPrefilterPass, pass: jsPrefilterPass },
contentLength: { value: contentLength, pass: contentLength >= 200 },
hasGeneratedComment: { value: hasGeneratedComment, pass: !hasGeneratedComment },
hasUserPath: { value: hasUserPath, pass: !hasUserPath },
};
// Use sqlPrefilter as the real filter (not JS approximation)
const failedFilters = Object.entries(filters)
.filter(([key, f]) => !f.pass && key !== 'jsPrefilter' && key !== 'contentLength' && key !== 'hasGeneratedComment' && key !== 'hasUserPath')
.map(([name]) => name);
return NextResponse.json(
{
id: skill.id,
name: skill.name,
downloadCount: skill.downloadCount,
wouldAppearInPending: failedFilters.length === 0,
failedFilters,
filters,
// Flag discrepancy between JS and SQL prefilter
...(jsPrefilterPass !== sqlPrefilterPass ? { prefilterDiscrepancy: true } : {}),
},
{ headers: createRateLimitHeaders(rateLimitResult) }
);
} catch (error) {
console.error('[Review] Diagnose error:', error);
return NextResponse.json({ error: 'Failed to diagnose skill' }, { status: 500 });
}
}

View File

@@ -0,0 +1,164 @@
import { type NextRequest, NextResponse } from 'next/server';
import { createDb, skillReviewQueries } from '@skillhub/db';
import { requireAdmin } from '@/lib/admin-auth';
import { withRateLimit, createRateLimitResponse, createRateLimitHeaders } from '@/lib/rate-limit';
export const maxDuration = 60; // seconds — queries 55K+ skills with owner-cap logic
const db = createDb();
/**
* GET /api/review/pending
* Returns a batch of skills ready for AI review.
* Supports owner-capped batches for diversity and hybrid re-review/new-review mixing.
*
* Query params:
* batch_size - number of skills to return (default 20, max 50)
* offset - number of skills to skip for pagination (default 0)
* min_quality - minimum quality_score (default 50)
* security - security_status filter (default "pass")
* priority - "re-review" to show needs-re-review first, "re-review-all" to include already ai-reviewed skills
* owner_limit - max skills per github_owner in batch (default 0=unlimited, max 10)
* sort_by - sort order: "quality" (default), "stars", "downloads"
* min_ai_score - minimum latestAiScore filter (for targeted re-review)
* max_ai_score - maximum latestAiScore filter (for targeted re-review)
* reviewed_before - ISO date string, only include skills reviewed before this date
*/
export async function GET(request: NextRequest) {
// Rate limiting
const rateLimitResult = await withRateLimit(request, 'anonymous');
if (!rateLimitResult.allowed) {
return createRateLimitResponse(rateLimitResult);
}
// Admin check (supports API key auth via Authorization header)
const adminCheck = await requireAdmin(request);
if (!adminCheck.authorized) {
return adminCheck.response;
}
try {
const { searchParams } = new URL(request.url);
const batchSize = Math.min(
Math.max(parseInt(searchParams.get('batch_size') ?? '20', 10) || 20, 1),
50
);
const offset = Math.max(parseInt(searchParams.get('offset') ?? '0', 10) || 0, 0);
const minQuality = parseInt(searchParams.get('min_quality') ?? '50', 10) || 50;
const securityPass = searchParams.get('security') !== 'any';
const priorityParam = searchParams.get('priority') ?? '';
const priorityReReview = priorityParam === 're-review';
const reReviewAll = priorityParam === 're-review-all';
const ownerLimit = Math.min(
Math.max(parseInt(searchParams.get('owner_limit') ?? '0', 10) || 0, 0),
10
);
const currentReviewVersion = Math.max(
parseInt(searchParams.get('review_version') ?? '0', 10) || 0, 0
);
const sortBy = (['quality', 'stars', 'downloads'] as const).includes(
searchParams.get('sort_by') as 'quality' | 'stars' | 'downloads'
) ? (searchParams.get('sort_by') as 'quality' | 'stars' | 'downloads') : 'quality';
const minAiScoreParam = searchParams.get('min_ai_score');
const minAiScore = minAiScoreParam ? parseInt(minAiScoreParam, 10) : undefined;
const maxAiScoreParam = searchParams.get('max_ai_score');
const maxAiScore = maxAiScoreParam ? parseInt(maxAiScoreParam, 10) : undefined;
const reviewedBefore = searchParams.get('reviewed_before') || undefined;
// Run counts in parallel
const [totalPending, reReviews] = await Promise.all([
skillReviewQueries.countPending(db, { minQuality, securityPass }),
skillReviewQueries.countReReviews(db),
]);
// Re-review-all mode: skip hybrid mixing, just fetch all reviewable skills
let batch: Array<{ id: string; githubOwner?: string; github_owner?: string; [key: string]: unknown }> = [];
if (reReviewAll) {
const extraSlots = ownerLimit > 0 ? Math.min(batchSize, 10) : 0;
const allBatch = await skillReviewQueries.getPending(db, {
batchSize: batchSize + extraSlots,
offset,
minQuality,
securityPass,
reReviewAll: true,
ownerLimit,
currentReviewVersion,
sortBy,
minAiScore,
maxAiScore,
reviewedBefore,
}) as typeof batch;
batch = [...allBatch].slice(0, batchSize);
} else {
// Normal mode: Hybrid batch — mix re-reviews (up to 5) with new reviews
const reReviewSlots = Math.min(5, reReviews, batchSize);
if (reReviewSlots > 0 && !priorityReReview) {
// Fetch re-reviews first (up to 5)
const reReviewBatch = await skillReviewQueries.getPending(db, {
batchSize: reReviewSlots,
minQuality: 0, // re-reviews regardless of quality
securityPass,
priorityReReview: true,
ownerLimit,
sortBy,
});
batch = [...reReviewBatch] as typeof batch;
}
// Fill remaining slots with new reviews (or all slots if priority=re-review)
const remainingSlots = batchSize - batch.length;
if (remainingSlots > 0) {
// Request extra to compensate for owner deduplication
const extraSlots = ownerLimit > 0 ? Math.min(remainingSlots, 10) : 0;
const newBatch = await skillReviewQueries.getPending(db, {
batchSize: priorityReReview ? batchSize : (remainingSlots + extraSlots),
offset,
minQuality,
securityPass,
priorityReReview,
ownerLimit,
sortBy,
}) as typeof batch;
if (priorityReReview) {
batch = [...newBatch];
} else if (ownerLimit > 0 && batch.length > 0) {
// Deduplicate owners: count per-owner across both batches
const ownerCounts: Record<string, number> = {};
for (const s of batch) {
const owner = (s.githubOwner ?? s.github_owner ?? 'unknown') as string;
ownerCounts[owner] = (ownerCounts[owner] || 0) + 1;
}
for (const s of newBatch) {
const owner = (s.githubOwner ?? s.github_owner ?? 'unknown') as string;
if ((ownerCounts[owner] || 0) >= ownerLimit) continue; // Skip — owner already at cap
ownerCounts[owner] = (ownerCounts[owner] || 0) + 1;
batch.push(s);
if (batch.length >= batchSize) break;
}
} else {
batch = [...batch, ...newBatch].slice(0, batchSize);
}
}
}
return NextResponse.json(
{
total_pending: totalPending,
re_reviews: reReviews,
batch,
},
{
headers: createRateLimitHeaders(rateLimitResult),
}
);
} catch (error) {
console.error('[Review] Error fetching pending:', error);
return NextResponse.json(
{ error: 'Failed to fetch pending reviews' },
{ status: 500 }
);
}
}

View File

@@ -0,0 +1,78 @@
import { type NextRequest, NextResponse } from 'next/server';
import { createDb, skillReviewQueries } from '@skillhub/db';
import { requireAdmin } from '@/lib/admin-auth';
import { withRateLimit, createRateLimitResponse, createRateLimitHeaders } from '@/lib/rate-limit';
import { getCached, setCache, cacheKeys, cacheTTL } from '@/lib/cache';
const db = createDb();
interface ReviewStatsData {
total_skills: number;
ai_reviewed: number;
needs_re_review: number;
total_reviews: number;
}
/**
* GET /api/review/stats
* Returns review pipeline status statistics.
*/
export async function GET(request: NextRequest) {
// Rate limiting
const rateLimitResult = await withRateLimit(request, 'anonymous');
if (!rateLimitResult.allowed) {
return createRateLimitResponse(rateLimitResult);
}
// Admin check (supports API key auth via Authorization header)
const adminCheck = await requireAdmin(request);
if (!adminCheck.authorized) {
return adminCheck.response;
}
try {
// Check cache
const cacheKey = cacheKeys.reviewStats();
const cached = await getCached<ReviewStatsData>(cacheKey);
if (cached) {
return NextResponse.json(cached, {
headers: {
'X-Cache': 'HIT',
...createRateLimitHeaders(rateLimitResult),
},
});
}
// Run pipeline stats and total reviews count in parallel
const [statusCounts, totalReviews] = await Promise.all([
skillReviewQueries.getPublicPipelineStats(db),
skillReviewQueries.countTotalReviews(db),
]);
// total_skills = sum of all statuses from pipeline query (browse-ready SKILL.md)
const totalSkills = Object.values(statusCounts).reduce((sum, n) => sum + n, 0);
const data: ReviewStatsData = {
total_skills: totalSkills,
ai_reviewed: statusCounts['ai-reviewed'] ?? 0,
needs_re_review: statusCounts['needs-re-review'] ?? 0,
total_reviews: totalReviews,
};
// Cache for 60 seconds
await setCache(cacheKey, data, cacheTTL.reviewStats);
return NextResponse.json(data, {
headers: {
'X-Cache': 'MISS',
...createRateLimitHeaders(rateLimitResult),
},
});
} catch (error) {
console.error('[Review] Error fetching stats:', error);
return NextResponse.json(
{ error: 'Failed to fetch review stats' },
{ status: 500 }
);
}
}

View File

@@ -0,0 +1,184 @@
import { type NextRequest, NextResponse } from 'next/server';
import { createDb, skillQueries, skillReviewQueries } from '@skillhub/db';
import { requireAdmin } from '@/lib/admin-auth';
import { withRateLimit, createRateLimitResponse, createRateLimitHeaders } from '@/lib/rate-limit';
export const maxDuration = 60; // seconds — batch DB writes can be slow
const db = createDb();
interface ReviewItem {
skill_id: string;
ai_score?: number;
instruction_quality?: number;
description_precision?: number;
usefulness?: number;
technical_soundness?: number;
review_notes?: string;
suggested_categories?: string[];
blog_worthy?: boolean;
collection_candidate?: string | null;
needs_improvement?: string | null;
i18n_priority?: number;
content_hash_at_review?: string;
set_verified?: boolean;
review_version?: number;
reviewer?: string;
recommendation?: 'flag-malicious' | null;
}
function validateReviews(body: unknown): { reviews: ReviewItem[] } | { error: string } {
if (!body || typeof body !== 'object' || !('reviews' in body)) {
return { error: 'Missing "reviews" array in request body' };
}
const { reviews } = body as { reviews: unknown };
if (!Array.isArray(reviews) || reviews.length === 0) {
return { error: '"reviews" must be a non-empty array' };
}
if (reviews.length > 50) {
return { error: '"reviews" array cannot exceed 50 items' };
}
for (let i = 0; i < reviews.length; i++) {
const r = reviews[i];
if (!r || typeof r !== 'object') {
return { error: `reviews[${i}] is not an object` };
}
const item = r as Record<string, unknown>;
if (typeof item.skill_id !== 'string' || item.skill_id.length === 0) {
return { error: `reviews[${i}].skill_id is required and must be a non-empty string` };
}
// Validate score fields (0-100 integers, optional)
for (const field of ['ai_score', 'instruction_quality', 'description_precision', 'usefulness', 'technical_soundness']) {
if (item[field] !== undefined && item[field] !== null) {
if (typeof item[field] !== 'number' || !Number.isInteger(item[field]) || (item[field] as number) < 0 || (item[field] as number) > 100) {
return { error: `reviews[${i}].${field} must be an integer 0-100` };
}
}
}
// Validate i18n_priority (0-2)
if (item.i18n_priority !== undefined && item.i18n_priority !== null) {
if (typeof item.i18n_priority !== 'number' || !Number.isInteger(item.i18n_priority) || item.i18n_priority < 0 || item.i18n_priority > 2) {
return { error: `reviews[${i}].i18n_priority must be an integer 0-2` };
}
}
// Validate review_version (positive integer)
if (item.review_version !== undefined && item.review_version !== null) {
if (typeof item.review_version !== 'number' || !Number.isInteger(item.review_version) || item.review_version < 1) {
return { error: `reviews[${i}].review_version must be a positive integer` };
}
}
// Validate reviewer (non-empty string, max 50 chars)
if (item.reviewer !== undefined && item.reviewer !== null) {
if (typeof item.reviewer !== 'string' || item.reviewer.length === 0 || item.reviewer.length > 50) {
return { error: `reviews[${i}].reviewer must be a non-empty string (max 50 chars)` };
}
}
// Validate recommendation
if (item.recommendation !== undefined && item.recommendation !== null) {
if (item.recommendation !== 'flag-malicious') {
return { error: `reviews[${i}].recommendation must be 'flag-malicious' or null` };
}
}
}
return { reviews: reviews as ReviewItem[] };
}
/**
* POST /api/review/submit
* Submit AI review results for a batch of skills.
*/
export async function POST(request: NextRequest) {
// Rate limiting
const rateLimitResult = await withRateLimit(request, 'anonymous');
if (!rateLimitResult.allowed) {
return createRateLimitResponse(rateLimitResult);
}
// Admin check (supports API key auth via Authorization header)
const adminCheck = await requireAdmin(request);
if (!adminCheck.authorized) {
return adminCheck.response;
}
// Primary server check
const isPrimary = process.env.IS_PRIMARY_SERVER !== 'false';
if (!isPrimary) {
return NextResponse.json(
{ error: 'Write operations only on primary server' },
{ status: 503 }
);
}
try {
const body = await request.json();
const result = validateReviews(body);
if ('error' in result) {
return NextResponse.json(
{ error: result.error },
{ status: 400 }
);
}
const { reviews } = result;
// Insert review rows
const dbReviews = reviews.map((r) => ({
skillId: r.skill_id,
reviewer: r.reviewer || 'claude-code',
aiScore: r.ai_score,
instructionQuality: r.instruction_quality,
descriptionPrecision: r.description_precision,
usefulness: r.usefulness,
technicalSoundness: r.technical_soundness,
reviewNotes: r.review_notes,
suggestedCategories: r.suggested_categories,
blogWorthy: r.blog_worthy,
collectionCandidate: r.collection_candidate ?? undefined,
needsImprovement: r.needs_improvement ?? undefined,
i18nPriority: r.i18n_priority,
contentHashAtReview: r.content_hash_at_review,
reviewVersion: r.review_version,
recommendation: r.recommendation ?? undefined,
}));
await skillReviewQueries.createBatch(db, dbReviews);
// Update review_status on each skill
let verifiedCount = 0;
for (const r of reviews) {
const newStatus = r.set_verified ? 'verified' : 'ai-reviewed';
if (r.set_verified) verifiedCount++;
await skillReviewQueries.updateSkillReviewStatus(db, r.skill_id, newStatus, r.ai_score, new Date());
}
// Flag malicious skills
let flaggedCount = 0;
for (const r of reviews) {
if (r.recommendation === 'flag-malicious') {
await skillQueries.flagMalicious(db, r.skill_id);
flaggedCount++;
}
}
return NextResponse.json(
{
submitted: reviews.length,
verified: verifiedCount,
flagged: flaggedCount,
},
{
headers: createRateLimitHeaders(rateLimitResult),
}
);
} catch (error) {
console.error('[Review] Error submitting reviews:', error);
return NextResponse.json(
{ error: 'Failed to submit reviews' },
{ status: 500 }
);
}
}

View File

@@ -9,8 +9,17 @@ export const maxDuration = 60; // 60 seconds
// Maximum recursion depth to prevent infinite loops // Maximum recursion depth to prevent infinite loops
const MAX_DEPTH = 5; const MAX_DEPTH = 5;
// Maximum number of files to fetch per skill
const MAX_FILES = 50;
// Maximum total content size (2MB) to prevent huge responses
const MAX_TOTAL_SIZE = 2 * 1024 * 1024;
// Concurrency limit for parallel file content fetches
const PARALLEL_FETCH_LIMIT = 5;
// Fetch timeout in milliseconds // Fetch timeout in milliseconds
const FETCH_TIMEOUT = 30000; // 30 seconds const FETCH_TIMEOUT = 15000; // 15 seconds per individual fetch
// Create database connection // Create database connection
const db = createDb(); const db = createDb();
@@ -30,6 +39,7 @@ interface SkillFile {
content: string; content: string;
size: number; size: number;
isBinary: boolean; isBinary: boolean;
fetchFailed?: boolean;
} }
interface CachedFiles { interface CachedFiles {
@@ -77,6 +87,14 @@ export async function GET(request: NextRequest) {
); );
} }
// Block file downloads for malicious skills
if (skill.isMalicious) {
return NextResponse.json(
{ error: 'This skill has been flagged as malicious. File downloads are blocked.', code: 'MALICIOUS' },
{ status: 403 }
);
}
const { githubOwner, githubRepo, skillPath, branch, commitSha, sourceFormat } = skill; const { githubOwner, githubRepo, skillPath, branch, commitSha, sourceFormat } = skill;
// === CACHE CHECK === // === CACHE CHECK ===
@@ -127,7 +145,7 @@ export async function GET(request: NextRequest) {
} }
// Fetch skill folder contents from GitHub // Fetch skill folder contents from GitHub
const files = await fetchSkillFiles( const { files, fetchFailureCount } = await fetchSkillFiles(
githubOwner, githubOwner,
githubRepo, githubRepo,
skillPath, skillPath,
@@ -137,19 +155,22 @@ export async function GET(request: NextRequest) {
token token
); );
// === SAVE TO CACHE === // === SAVE TO CACHE (only if all files fetched successfully) ===
// Prepare cached files structure if (fetchFailureCount > 0) {
console.warn(`[skill-files] Skipping cache for ${skillId}: ${fetchFailureCount} file(s) failed to fetch`);
} else {
const filesToCache: CachedFiles = { const filesToCache: CachedFiles = {
fetchedAt: new Date().toISOString(), fetchedAt: new Date().toISOString(),
commitSha: commitSha || 'unknown', commitSha: commitSha || 'unknown',
totalSize: files.reduce((sum, f) => sum + f.size, 0), totalSize: files.reduce((sum, f) => sum + f.size, 0),
items: files, items: files.map(({ fetchFailed: _, ...rest }) => rest),
}; };
// Save to database (async, don't block response) // Save to database (async, don't block response)
skillQueries.updateCachedFiles(db, skillId, filesToCache).catch((err) => { skillQueries.updateCachedFiles(db, skillId, filesToCache).catch((err) => {
console.error(`[skill-files] Failed to cache files for ${skillId}:`, err); console.error(`[skill-files] Failed to cache files for ${skillId}:`, err);
}); });
}
// Note: Download count is NOT incremented here. // Note: Download count is NOT incremented here.
// It should only be incremented in /api/skills/install after successful download. // It should only be incremented in /api/skills/install after successful download.
@@ -171,6 +192,7 @@ export async function GET(request: NextRequest) {
downloadUrl: item.isBinary ? `https://raw.githubusercontent.com/${githubOwner}/${githubRepo}/${branch || 'main'}/${skillPath}/${item.path}` : undefined, downloadUrl: item.isBinary ? `https://raw.githubusercontent.com/${githubOwner}/${githubRepo}/${branch || 'main'}/${skillPath}/${item.path}` : undefined,
})), })),
fromCache: false, fromCache: false,
...(fetchFailureCount > 0 ? { fetchFailures: fetchFailureCount } : {}),
}, { }, {
headers: createRateLimitHeaders(rateLimitResult), headers: createRateLimitHeaders(rateLimitResult),
}); });
@@ -195,6 +217,44 @@ export async function GET(request: NextRequest) {
} }
if (errorMessage.includes('404')) { if (errorMessage.includes('404')) {
// Increment stale check (non-blocking) — after 3 consecutive 404s, skill is marked stale
const skillId404 = request.nextUrl.searchParams.get('id');
if (skillId404) {
skillQueries.incrementStaleCheck(db, skillId404).catch((err) => {
console.error(`[skill-files] Failed to increment stale check for ${skillId404}:`, err);
});
}
// If we have cached files in DB, serve them with stale warning instead of 404
if (skillId404) {
const skill404 = await skillQueries.getById(db, skillId404);
if (skill404?.cachedFiles) {
const cached = skill404.cachedFiles as CachedFiles;
return NextResponse.json({
skillId: skillId404,
githubOwner: skill404.githubOwner,
githubRepo: skill404.githubRepo,
skillPath: skill404.skillPath,
branch: skill404.branch || 'main',
sourceFormat: skill404.sourceFormat || 'skill.md',
files: cached.items.map(item => ({
name: item.name,
path: item.path,
type: 'file' as const,
size: item.size,
content: item.isBinary ? undefined : item.content,
downloadUrl: item.isBinary ? `https://raw.githubusercontent.com/${skill404.githubOwner}/${skill404.githubRepo}/${skill404.branch || 'main'}/${skill404.skillPath}/${item.path}` : undefined,
})),
fromCache: true,
cachedAt: cached.fetchedAt,
isStale: true,
staleWarning: 'This skill may have been removed from GitHub. Files served from cache.',
}, {
headers: createRateLimitHeaders(rateLimitResult),
});
}
}
return NextResponse.json( return NextResponse.json(
{ error: 'Skill files not found on GitHub', code: 'GITHUB_NOT_FOUND' }, { error: 'Skill files not found on GitHub', code: 'GITHUB_NOT_FOUND' },
{ status: 404 } { status: 404 }
@@ -209,42 +269,56 @@ export async function GET(request: NextRequest) {
} }
/** /**
* Recursively fetch all files in a skill folder from GitHub * Helper: run async tasks with concurrency limit
* @param depth - Current recursion depth (max MAX_DEPTH levels)
* @param headers - GitHub API headers (with token rotation)
* @param token - Current token for stats tracking
*/ */
async function fetchSkillFiles( async function parallelLimit<T>(
tasks: (() => Promise<T>)[],
limit: number
): Promise<T[]> {
const results: T[] = [];
let idx = 0;
async function runNext(): Promise<void> {
while (idx < tasks.length) {
const currentIdx = idx++;
results[currentIdx] = await tasks[currentIdx]();
}
}
const workers = Array.from({ length: Math.min(limit, tasks.length) }, () => runNext());
await Promise.all(workers);
return results;
}
/**
* Recursively collect file metadata (directory listings only, no content fetch).
* Returns flat list of file entries to fetch content for.
*/
async function collectFileEntries(
owner: string, owner: string,
repo: string, repo: string,
path: string, path: string,
ref: string, ref: string,
depth: number = 0, depth: number,
headers: Record<string, string>, headers: Record<string, string>,
token: string | null token: string | null
): Promise<SkillFile[]> { ): Promise<Array<{ item: GitHubFile; relativePath: string }>> {
// Prevent infinite recursion
if (depth > MAX_DEPTH) { if (depth > MAX_DEPTH) {
console.warn(`Max depth (${MAX_DEPTH}) reached at: ${path}`); console.warn(`Max depth (${MAX_DEPTH}) reached at: ${path}`);
return []; return [];
} }
const files: SkillFile[] = [];
// Fetch directory contents with timeout
const apiUrl = `https://api.github.com/repos/${owner}/${repo}/contents/${path}?ref=${ref}`; const apiUrl = `https://api.github.com/repos/${owner}/${repo}/contents/${path}?ref=${ref}`;
const response = await fetch(apiUrl, { const response = await fetch(apiUrl, {
headers, headers,
signal: AbortSignal.timeout(FETCH_TIMEOUT), signal: AbortSignal.timeout(FETCH_TIMEOUT),
}); });
// Update token stats for rotation
if (token) { if (token) {
await updateTokenStats(token, response.headers); await updateTokenStats(token, response.headers);
} }
if (!response.ok) { if (!response.ok) {
// Check for rate limiting
if (response.status === 403) { if (response.status === 403) {
const rateLimitRemaining = response.headers.get('x-ratelimit-remaining'); const rateLimitRemaining = response.headers.get('x-ratelimit-remaining');
if (rateLimitRemaining === '0') { if (rateLimitRemaining === '0') {
@@ -255,60 +329,98 @@ async function fetchSkillFiles(
} }
const contents: GitHubFile[] = await response.json(); const contents: GitHubFile[] = await response.json();
const entries: Array<{ item: GitHubFile; relativePath: string }> = [];
// Process each item
for (const item of contents) { for (const item of contents) {
const relativePath = item.path.replace(`${path}/`, '').replace(path, '') || item.name;
if (item.type === 'file') { if (item.type === 'file') {
// Determine if file is binary entries.push({ item, relativePath });
} else if (item.type === 'dir') {
const subEntries = await collectFileEntries(
owner, repo, `${path}/${item.name}`, ref, depth + 1, headers, token
);
for (const sub of subEntries) {
entries.push({
item: sub.item,
relativePath: `${item.name}/${sub.relativePath}`,
});
}
}
}
return entries;
}
/**
* Fetch all files in a skill folder from GitHub.
* Phase 1: Collect directory tree (sequential, required for recursion)
* Phase 2: Fetch file contents in parallel (fast, with concurrency limit)
*/
async function fetchSkillFiles(
owner: string,
repo: string,
path: string,
ref: string,
depth: number = 0,
headers: Record<string, string>,
token: string | null
): Promise<{ files: SkillFile[]; fetchFailureCount: number }> {
// Phase 1: Collect all file entries from directory tree
const entries = await collectFileEntries(owner, repo, path, ref, depth, headers, token);
// Apply MAX_FILES limit
if (entries.length > MAX_FILES) {
console.warn(`[skill-files] Skill has ${entries.length} files, limiting to ${MAX_FILES}`);
entries.length = MAX_FILES;
}
// Phase 2: Fetch file contents in parallel
let totalSize = 0;
let fetchFailureCount = 0;
const tasks = entries.map((entry) => async (): Promise<SkillFile> => {
const { item, relativePath } = entry;
const isBinary = !isTextFile(item.name); const isBinary = !isTextFile(item.name);
if (!isBinary && item.size < 1024 * 1024) { if (!isBinary && item.size < 1024 * 1024 && totalSize + item.size <= MAX_TOTAL_SIZE) {
// For text files (< 1MB), fetch content
try { try {
const content = await fetchFileContent(owner, repo, item.path, ref, headers, token); const content = await fetchFileContent(owner, repo, item.path, ref, headers, token);
files.push({ totalSize += item.size;
return {
name: item.name, name: item.name,
path: item.path.replace(`${path}/`, '').replace(path, '') || item.name, path: relativePath,
content, content,
size: item.size, size: item.size,
isBinary: false, isBinary: false,
}); fetchFailed: false,
};
} catch { } catch {
// If content fetch fails, mark as binary (will use download URL) // Content fetch failed — mark as failed so we skip caching
files.push({ fetchFailureCount++;
return {
name: item.name, name: item.name,
path: item.path.replace(`${path}/`, '').replace(path, '') || item.name, path: relativePath,
content: '', content: '',
size: item.size, size: item.size,
isBinary: true, isBinary: true,
}); fetchFailed: true,
};
} }
} else { }
// For binary or large files, don't store content (use download URL)
files.push({ // Binary, too large, or total size exceeded
return {
name: item.name, name: item.name,
path: item.path.replace(`${path}/`, '').replace(path, '') || item.name, path: relativePath,
content: '', content: '',
size: item.size, size: item.size,
isBinary: true, isBinary: true,
fetchFailed: false,
};
}); });
}
} else if (item.type === 'dir') {
// Recursively fetch subdirectory (with depth limit)
const subPath = `${path}/${item.name}`;
const subFiles = await fetchSkillFiles(owner, repo, subPath, ref, depth + 1, headers, token);
// Add subdirectory files with proper paths const files = await parallelLimit(tasks, PARALLEL_FETCH_LIMIT);
for (const subFile of subFiles) { return { files, fetchFailureCount };
files.push({
...subFile,
path: `${item.name}/${subFile.path}`,
});
}
}
}
return files;
} }
/** /**

View File

@@ -1,6 +1,6 @@
import { NextResponse, type NextRequest } from 'next/server'; import { NextResponse, type NextRequest } from 'next/server';
import { createDb, skillQueries } from '@skillhub/db'; import { createDb, skillQueries, skillReviewQueries } from '@skillhub/db';
import { shouldCountView } from '@/lib/cache'; import { shouldCountView, getOrSetCache, cacheKeys, cacheTTL } from '@/lib/cache';
// Create database connection // Create database connection
const db = createDb(); const db = createDb();
@@ -35,8 +35,19 @@ export async function GET(
const { id } = await params; const { id } = await params;
const skillId = id.join('/'); const skillId = id.join('/');
// Get skill from database // Get skill and latest review from database (cached 1h)
const skill = await skillQueries.getById(db, skillId); const [skill, review] = await Promise.all([
getOrSetCache(
cacheKeys.skill(skillId),
cacheTTL.skill,
() => skillQueries.getById(db, skillId)
),
getOrSetCache(
cacheKeys.skillReview(skillId),
cacheTTL.skill,
() => skillReviewQueries.getLatestBySkillId(db, skillId)
),
]);
if (!skill) { if (!skill) {
return NextResponse.json( return NextResponse.json(
@@ -73,15 +84,34 @@ export async function GET(
downloadCount: skill.downloadCount, downloadCount: skill.downloadCount,
viewCount: skill.viewCount, viewCount: skill.viewCount,
securityScore: skill.securityScore, securityScore: skill.securityScore,
securityStatus: skill.securityStatus,
qualityScore: skill.qualityScore,
isVerified: skill.isVerified, isVerified: skill.isVerified,
isFeatured: skill.isFeatured, isFeatured: skill.isFeatured,
isMalicious: skill.isMalicious ?? false,
isDuplicate: skill.isDuplicate ?? false,
isStale: skill.isStale ?? false,
isDeprecated: skill.isDeprecated ?? false,
isBlocked: skill.isBlocked ?? false,
compatibility: skill.compatibility, compatibility: skill.compatibility,
triggers: skill.triggers, triggers: skill.triggers,
rawContent: skill.rawContent, rawContent: skill.rawContent,
sourceFormat: skill.sourceFormat || 'skill.md', sourceFormat: skill.sourceFormat || 'skill.md',
reviewStatus: skill.reviewStatus,
aiScore: skill.latestAiScore,
createdAt: skill.createdAt, createdAt: skill.createdAt,
updatedAt: skill.updatedAt, updatedAt: skill.updatedAt,
indexedAt: skill.indexedAt, indexedAt: skill.indexedAt,
review: review ? {
aiScore: review.aiScore,
instructionQuality: review.instructionQuality,
descriptionPrecision: review.descriptionPrecision,
usefulness: review.usefulness,
technicalSoundness: review.technicalSoundness,
reviewNotes: review.reviewNotes,
reviewer: review.reviewer,
reviewedAt: review.reviewedAt,
} : null,
}); });
} catch (error) { } catch (error) {
console.error('Error fetching skill:', error); console.error('Error fetching skill:', error);

View File

@@ -0,0 +1,229 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { NextRequest } from 'next/server';
// Mock auth
const mockAuth = vi.fn();
vi.mock('@/lib/auth', () => ({
auth: () => mockAuth(),
}));
// Mock sanitize
vi.mock('@/lib/sanitize', () => ({
sanitizeReason: (r: string) => r,
}));
// Mock email
vi.mock('@/lib/email', () => ({
sendClaimSubmittedEmail: vi.fn().mockResolvedValue(undefined),
}));
// Mock db
vi.mock('@skillhub/db', () => ({
createDb: vi.fn(() => ({})),
userQueries: {
getByGithubId: vi.fn(),
upsertFromGithub: vi.fn(),
},
skillQueries: {
unblockByRepo: vi.fn(),
},
addRequestQueries: {
hasPendingRequest: vi.fn(),
create: vi.fn(),
updateStatus: vi.fn(),
},
discoveredRepoQueries: {
getById: vi.fn(),
upsert: vi.fn(),
unblockRepo: vi.fn(),
},
}));
import { POST, GET } from './route';
import { userQueries, skillQueries, addRequestQueries, discoveredRepoQueries } from '@skillhub/db';
function createMockUser(overrides: Partial<{ id: string; githubId: string; email: string }> = {}) {
return {
id: 'user-123',
githubId: 'gh-12345',
username: 'testuser',
email: 'test@example.com',
preferredLocale: 'en',
...overrides,
};
}
function createRequest(body: unknown) {
return new NextRequest('http://localhost:3000/api/skills/add-request', {
method: 'POST',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' },
});
}
describe('/api/skills/add-request', () => {
beforeEach(() => {
vi.clearAllMocks();
// Disable real fetch by default
vi.stubGlobal('fetch', vi.fn());
});
describe('GET', () => {
it('should return 401 when not authenticated', async () => {
mockAuth.mockResolvedValue(null);
const response = await GET();
const data = await response.json();
expect(response.status).toBe(401);
expect(data.error).toBeDefined();
});
it('should return empty requests for unknown user', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345' } });
vi.mocked(userQueries.getByGithubId).mockResolvedValue(null as any);
const response = await GET();
const data = await response.json();
expect(response.status).toBe(200);
expect(data.requests).toEqual([]);
});
});
describe('POST', () => {
it('should return 401 when not authenticated', async () => {
mockAuth.mockResolvedValue(null);
const response = await POST(createRequest({ repositoryUrl: 'https://github.com/owner/repo' }));
const data = await response.json();
expect(response.status).toBe(401);
expect(data.code).toBe('AUTH_REQUIRED');
});
it('should return 400 for missing repositoryUrl', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'testuser' } });
const response = await POST(createRequest({}));
const data = await response.json();
expect(response.status).toBe(400);
expect(data.code).toBe('INVALID_INPUT');
});
it('should return 400 for invalid GitHub URL', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'testuser' } });
const mockUser = createMockUser();
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
// gitlab.com does not contain the substring 'github.com'
const response = await POST(createRequest({ repositoryUrl: 'https://gitlab.com/owner/repo' }));
const data = await response.json();
expect(response.status).toBe(400);
expect(data.code).toBe('INVALID_URL');
});
it('should return 403 REPO_BLOCKED_BY_OWNER when non-owner tries to add blocked repo', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'non-owner' } });
const mockUser = createMockUser({ githubId: 'gh-12345' });
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
vi.mocked(discoveredRepoQueries.getById).mockResolvedValue({
id: 'rawveg/skillsforge-marketplace',
isBlocked: true,
} as any);
// GitHub API says the owner is 'rawveg', not 'non-owner'
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
ok: true,
json: async () => ({ owner: { login: 'rawveg' }, default_branch: 'main', private: false }),
} as any));
const response = await POST(createRequest({ repositoryUrl: 'https://github.com/rawveg/skillsforge-marketplace' }));
const data = await response.json();
expect(response.status).toBe(403);
expect(data.code).toBe('REPO_BLOCKED_BY_OWNER');
});
it('should re-enable repo when owner re-adds their blocked repo', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'rawveg' } });
const mockUser = createMockUser({ githubId: 'gh-12345' });
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
vi.mocked(discoveredRepoQueries.getById).mockResolvedValue({
id: 'rawveg/skillsforge-marketplace',
isBlocked: true,
} as any);
vi.mocked(skillQueries.unblockByRepo).mockResolvedValue(undefined);
vi.mocked(discoveredRepoQueries.unblockRepo).mockResolvedValue(undefined);
vi.mocked(addRequestQueries.hasPendingRequest).mockResolvedValue(false as any);
vi.mocked(addRequestQueries.create).mockResolvedValue('req-123' as any);
vi.mocked(addRequestQueries.updateStatus).mockResolvedValue(undefined);
vi.mocked(discoveredRepoQueries.upsert).mockResolvedValue(undefined as any);
// First fetch: ownership check (inside blocked-repo logic)
// Second fetch: repo validation (validateGitHubRepo → repoResponse)
// Third fetch: tree scan (findSkillMdFiles)
vi.stubGlobal('fetch', vi.fn()
.mockResolvedValueOnce({
ok: true,
json: async () => ({ owner: { login: 'rawveg' }, default_branch: 'main', private: false }),
} as any)
.mockResolvedValueOnce({
ok: true,
json: async () => ({ owner: { login: 'rawveg' }, default_branch: 'main', private: false }),
} as any)
.mockResolvedValueOnce({
ok: true,
json: async () => ({
tree: [
{ path: 'SKILL.md', type: 'blob' },
],
truncated: false,
}),
} as any)
);
const response = await POST(createRequest({ repositoryUrl: 'https://github.com/rawveg/skillsforge-marketplace' }));
const data = await response.json();
expect(response.status).toBe(200);
expect(data.success).toBe(true);
expect(data.reEnabled).toBe(true);
expect(skillQueries.unblockByRepo).toHaveBeenCalledWith(expect.anything(), 'rawveg', 'skillsforge-marketplace');
expect(discoveredRepoQueries.unblockRepo).toHaveBeenCalledWith(expect.anything(), 'rawveg/skillsforge-marketplace');
});
it('should proceed normally when repo is not blocked (Case C)', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'newuser' } });
const mockUser = createMockUser();
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
vi.mocked(discoveredRepoQueries.getById).mockResolvedValue(null as any);
vi.mocked(addRequestQueries.hasPendingRequest).mockResolvedValue(false as any);
vi.mocked(addRequestQueries.create).mockResolvedValue('req-456' as any);
vi.mocked(addRequestQueries.updateStatus).mockResolvedValue(undefined);
vi.mocked(discoveredRepoQueries.upsert).mockResolvedValue(undefined as any);
vi.stubGlobal('fetch', vi.fn()
.mockResolvedValueOnce({
ok: true,
json: async () => ({ owner: { login: 'someowner' }, default_branch: 'main', private: false }),
} as any)
.mockResolvedValueOnce({
ok: true,
json: async () => ({
tree: [{ path: 'SKILL.md', type: 'blob' }],
truncated: false,
}),
} as any)
);
const response = await POST(createRequest({ repositoryUrl: 'https://github.com/someowner/newrepo' }));
const data = await response.json();
expect(response.status).toBe(200);
expect(data.success).toBe(true);
expect(data.reEnabled).toBeUndefined();
});
});
});

View File

@@ -1,7 +1,7 @@
import type { NextRequest } from 'next/server'; import type { NextRequest } from 'next/server';
import { NextResponse } from 'next/server'; import { NextResponse } from 'next/server';
import { auth } from '@/lib/auth'; import { auth } from '@/lib/auth';
import { createDb, addRequestQueries, userQueries } from '@skillhub/db'; import { createDb, addRequestQueries, discoveredRepoQueries, skillQueries, userQueries } from '@skillhub/db';
import { sanitizeReason } from '@/lib/sanitize'; import { sanitizeReason } from '@/lib/sanitize';
import { sendClaimSubmittedEmail } from '@/lib/email'; import { sendClaimSubmittedEmail } from '@/lib/email';
@@ -337,6 +337,52 @@ export async function POST(request: NextRequest) {
); );
} }
// Check if repo was previously blocked by its owner
let reEnabled = false;
const existingRepo = await discoveredRepoQueries.getById(db, `${parsed.owner}/${parsed.repo}`);
if (existingRepo?.isBlocked) {
// Verify whether the current requester is the repo owner
let requesterIsOwner = false;
try {
const ownerCheckRes = await fetch(
`https://api.github.com/repos/${parsed.owner}/${parsed.repo}`,
{
headers: {
Accept: 'application/vnd.github.v3+json',
'User-Agent': 'SkillHub',
...(process.env.GITHUB_TOKEN && {
Authorization: `Bearer ${process.env.GITHUB_TOKEN}`,
}),
},
signal: AbortSignal.timeout(10000),
}
);
if (ownerCheckRes.ok) {
const repoData = await ownerCheckRes.json() as { owner?: { login?: string } };
requesterIsOwner =
repoData.owner?.login?.toLowerCase() === session.user.username.toLowerCase();
}
} catch {
// If we can't reach GitHub, treat as non-owner (safe default)
}
if (!requesterIsOwner) {
// Case B: repo blocked by its owner, non-owner trying to re-add
return NextResponse.json(
{
error: 'This repository was removed by its owner and cannot be re-added.',
code: 'REPO_BLOCKED_BY_OWNER',
},
{ status: 403 }
);
}
// Case A: owner is re-enabling their previously blocked repo
await skillQueries.unblockByRepo(db, parsed.owner, parsed.repo);
await discoveredRepoQueries.unblockRepo(db, `${parsed.owner}/${parsed.repo}`);
reEnabled = true;
}
// Check if user already has a pending request for this repository + path combination // Check if user already has a pending request for this repository + path combination
const hasPending = await addRequestQueries.hasPendingRequest( const hasPending = await addRequestQueries.hasPendingRequest(
db, db,
@@ -389,6 +435,26 @@ export async function POST(request: NextRequest) {
hasSkillMd: validation.hasSkillMd, hasSkillMd: validation.hasSkillMd,
}); });
// Auto-approve and queue for crawling if SKILL.md found
if (validation.hasSkillMd && validation.skillPaths.length > 0) {
await addRequestQueries.updateStatus(db, requestId, {
status: 'approved',
});
// Queue repo for indexer crawling via discovered_repos
try {
await discoveredRepoQueries.upsert(db, {
id: `${parsed.owner}/${parsed.repo}`,
owner: parsed.owner,
repo: parsed.repo,
discoveredVia: 'add-request',
githubStars: 0,
});
} catch (err) {
console.warn('[Claim] Failed to queue repo for crawling:', err);
}
}
// Build appropriate response message // Build appropriate response message
let message: string; let message: string;
if (validation.skillPaths.length > 1) { if (validation.skillPaths.length > 1) {
@@ -418,6 +484,7 @@ export async function POST(request: NextRequest) {
skillCount: validation.skillPaths.length, skillCount: validation.skillPaths.length,
skillPaths: validation.skillPaths, skillPaths: validation.skillPaths,
message, message,
...(reEnabled && { reEnabled: true }),
}); });
} catch (error) { } catch (error) {
console.error('Error creating add request:', error); console.error('Error creating add request:', error);

View File

@@ -67,6 +67,8 @@ export async function GET(request: NextRequest) {
securityStatus: skill.securityStatus, securityStatus: skill.securityStatus,
isVerified: skill.isVerified, isVerified: skill.isVerified,
compatibility: skill.compatibility, compatibility: skill.compatibility,
reviewStatus: skill.reviewStatus,
aiScore: skill.latestAiScore,
})), })),
}; };

View File

@@ -1,6 +1,6 @@
import { NextResponse, type NextRequest } from 'next/server'; import { NextResponse, type NextRequest } from 'next/server';
import { createDb, skillQueries } from '@skillhub/db'; import { createDb, skillQueries } from '@skillhub/db';
import { invalidateCache, cacheKeys, shouldCountDownload } from '@/lib/cache'; import { invalidateCache, invalidateCachePattern, cacheKeys, shouldCountDownload } from '@/lib/cache';
// Create database connection // Create database connection
const db = createDb(); const db = createDb();
@@ -69,12 +69,13 @@ export async function POST(request: NextRequest) {
await skillQueries.incrementDownloads(db, skillId); await skillQueries.incrementDownloads(db, skillId);
} }
// Invalidate relevant caches so featured/recent lists reflect the new download // Invalidate relevant caches so featured/recent/browse lists reflect the new download
await Promise.all([ await Promise.all([
invalidateCache(cacheKeys.featuredSkills()), invalidateCache(cacheKeys.featuredSkills()),
invalidateCache(cacheKeys.recentSkills()), invalidateCache(cacheKeys.recentSkills()),
invalidateCache(cacheKeys.stats()), invalidateCache(cacheKeys.stats()),
invalidateCache(cacheKeys.skill(skillId)), invalidateCache(cacheKeys.skill(skillId)),
invalidateCachePattern('skills:search:*'),
]); ]);
return NextResponse.json({ return NextResponse.json({

View File

@@ -124,6 +124,7 @@ export async function POST(request: NextRequest) {
// Check repository ownership using public API (no token needed for public repos) // Check repository ownership using public API (no token needed for public repos)
let isOwner = false; let isOwner = false;
let repoDeleted = false;
let githubError: string | null = null; let githubError: string | null = null;
try { try {
@@ -144,7 +145,7 @@ export async function POST(request: NextRequest) {
isOwner = true; isOwner = true;
} }
} else if (repoResponse.status === 404) { } else if (repoResponse.status === 404) {
githubError = 'Repository not found on GitHub'; repoDeleted = true;
} else if (repoResponse.status === 403) { } else if (repoResponse.status === 403) {
githubError = 'GitHub API rate limit exceeded'; githubError = 'GitHub API rate limit exceeded';
} }
@@ -160,23 +161,34 @@ export async function POST(request: NextRequest) {
); );
} }
if (!isOwner) { if (!isOwner && !repoDeleted) {
return NextResponse.json( return NextResponse.json(
{ error: 'You are not the owner of this repository', code: 'NOT_OWNER' }, { error: 'You are not the owner of this repository', code: 'NOT_OWNER' },
{ status: 403 } { status: 403 }
); );
} }
// Create the removal request (auto-approved since owner is verified) // Create the removal request
const sanitizedReason = reason ? sanitizeReason(reason) : 'No reason provided'; const sanitizedReason = reason ? sanitizeReason(reason) : 'No reason provided';
const requestId = await removalRequestQueries.create(db, { const requestId = await removalRequestQueries.create(db, {
userId: dbUser.id, userId: dbUser.id,
skillId, skillId,
reason: sanitizedReason, reason: sanitizedReason,
verifiedOwner: true, verifiedOwner: !repoDeleted,
}); });
// Auto-approve: Block the skill from being re-indexed if (repoDeleted) {
// Repo not found — submit for admin review instead of auto-approving
return NextResponse.json({
success: true,
requestId,
pending: true,
blocked: false,
message: 'Repository not found on GitHub. Your removal request has been submitted for admin review.',
});
}
// Owner verified — auto-approve: Block the skill from being re-indexed
await skillQueries.block(db, skillId); await skillQueries.block(db, skillId);
// Update the request status to approved // Update the request status to approved

View File

@@ -0,0 +1,163 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { NextRequest } from 'next/server';
// Mock auth
const mockAuth = vi.fn();
vi.mock('@/lib/auth', () => ({
auth: () => mockAuth(),
}));
// Mock sanitize
vi.mock('@/lib/sanitize', () => ({
sanitizeReason: (r: string) => r,
}));
// Mock db
vi.mock('@skillhub/db', () => ({
createDb: vi.fn(() => ({})),
userQueries: {
getByGithubId: vi.fn(),
upsertFromGithub: vi.fn(),
},
skillQueries: {
countByRepo: vi.fn(),
blockByRepo: vi.fn(),
},
discoveredRepoQueries: {
blockRepo: vi.fn(),
},
}));
import { POST } from './route';
import { userQueries, skillQueries, discoveredRepoQueries } from '@skillhub/db';
function createMockUser(overrides: Partial<{ id: string; githubId: string }> = {}) {
return {
id: 'user-123',
githubId: 'gh-12345',
username: 'rawveg',
...overrides,
};
}
function createRequest(body: unknown) {
return new NextRequest('http://localhost:3000/api/skills/repo-removal-request', {
method: 'POST',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' },
});
}
describe('/api/skills/repo-removal-request', () => {
beforeEach(() => {
vi.clearAllMocks();
vi.stubGlobal('fetch', vi.fn());
});
describe('POST', () => {
it('should return 401 when not authenticated', async () => {
mockAuth.mockResolvedValue(null);
const response = await POST(createRequest({ repoUrl: 'rawveg/skillsforge-marketplace' }));
const data = await response.json();
expect(response.status).toBe(401);
expect(data.code).toBe('AUTH_REQUIRED');
});
it('should return 400 for missing repoUrl', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'rawveg' } });
const response = await POST(createRequest({}));
expect(response.status).toBe(400);
});
it('should return 400 PARSE_ERROR for invalid repo format', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'rawveg' } });
const mockUser = createMockUser();
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
const response = await POST(createRequest({ repoUrl: 'not-a-valid-format' }));
const data = await response.json();
expect(response.status).toBe(400);
expect(data.code).toBe('PARSE_ERROR');
});
it('should return 404 INVALID_REPO when repo not found on GitHub', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'rawveg' } });
const mockUser = createMockUser();
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
ok: false,
status: 404,
} as any));
const response = await POST(createRequest({ repoUrl: 'rawveg/skillsforge-marketplace' }));
const data = await response.json();
expect(response.status).toBe(404);
expect(data.code).toBe('INVALID_REPO');
});
it('should return 403 NOT_OWNER when requester is not repo owner', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'other-user' } });
const mockUser = createMockUser({ githubId: 'gh-12345' });
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
ok: true,
json: async () => ({ owner: { login: 'rawveg' } }),
} as any));
const response = await POST(createRequest({ repoUrl: 'rawveg/skillsforge-marketplace' }));
const data = await response.json();
expect(response.status).toBe(403);
expect(data.code).toBe('NOT_OWNER');
});
it('should block all skills and return success when owner removes repo', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'rawveg' } });
const mockUser = createMockUser();
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
vi.mocked(skillQueries.countByRepo).mockResolvedValue(16 as any);
vi.mocked(skillQueries.blockByRepo).mockResolvedValue(undefined);
vi.mocked(discoveredRepoQueries.blockRepo).mockResolvedValue(undefined);
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
ok: true,
json: async () => ({ owner: { login: 'rawveg' } }),
} as any));
const response = await POST(createRequest({ repoUrl: 'rawveg/skillsforge-marketplace', reason: 'I want my skills removed' }));
const data = await response.json();
expect(response.status).toBe(200);
expect(data.success).toBe(true);
expect(data.blockedCount).toBe(16);
expect(data.repo).toBe('rawveg/skillsforge-marketplace');
expect(skillQueries.blockByRepo).toHaveBeenCalledWith(expect.anything(), 'rawveg', 'skillsforge-marketplace');
expect(discoveredRepoQueries.blockRepo).toHaveBeenCalledWith(expect.anything(), 'rawveg/skillsforge-marketplace');
});
it('should accept full GitHub URL format', async () => {
mockAuth.mockResolvedValue({ user: { githubId: 'gh-12345', username: 'rawveg' } });
const mockUser = createMockUser();
vi.mocked(userQueries.getByGithubId).mockResolvedValue(mockUser as any);
vi.mocked(skillQueries.countByRepo).mockResolvedValue(0 as any);
vi.mocked(skillQueries.blockByRepo).mockResolvedValue(undefined);
vi.mocked(discoveredRepoQueries.blockRepo).mockResolvedValue(undefined);
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
ok: true,
json: async () => ({ owner: { login: 'rawveg' } }),
} as any));
const response = await POST(createRequest({ repoUrl: 'https://github.com/rawveg/skillsforge-marketplace' }));
const data = await response.json();
expect(response.status).toBe(200);
expect(data.success).toBe(true);
expect(data.repo).toBe('rawveg/skillsforge-marketplace');
});
});
});

View File

@@ -0,0 +1,158 @@
import type { NextRequest } from 'next/server';
import { NextResponse } from 'next/server';
import { auth } from '@/lib/auth';
import { createDb, skillQueries, discoveredRepoQueries, userQueries } from '@skillhub/db';
import { sanitizeReason } from '@/lib/sanitize';
export const dynamic = 'force-dynamic';
const db = createDb();
function parseOwnerRepo(input: string): { owner: string; repo: string } | null {
try {
if (input.startsWith('https://') || input.startsWith('http://')) {
const url = new URL(input);
if (!url.hostname.includes('github.com')) return null;
const parts = url.pathname.split('/').filter(Boolean);
if (parts.length < 2) return null;
return { owner: parts[0], repo: parts[1] };
}
const parts = input.split('/').filter(Boolean);
if (parts.length < 2) return null;
return { owner: parts[0], repo: parts[1] };
} catch {
return null;
}
}
/**
* POST /api/skills/repo-removal-request
* Block all skills from a GitHub repository at once.
* Verifies the authenticated user owns the repo, then blocks all skills and
* prevents the repo from being re-indexed.
*/
export async function POST(request: NextRequest) {
try {
const session = await auth();
if (!session?.user?.githubId || !session.user.username) {
return NextResponse.json(
{ error: 'Authentication required', code: 'AUTH_REQUIRED' },
{ status: 401 }
);
}
const body = await request.json();
const { repoUrl, reason } = body;
if (!repoUrl) {
return NextResponse.json(
{ error: 'repoUrl is required', code: 'INVALID_INPUT' },
{ status: 400 }
);
}
const parsed = parseOwnerRepo(String(repoUrl).trim());
if (!parsed) {
return NextResponse.json(
{ error: 'Invalid format. Use https://github.com/owner/repo or owner/repo', code: 'PARSE_ERROR' },
{ status: 400 }
);
}
const { owner, repo } = parsed;
const username = session.user.username;
// Get or create user
let dbUser = await userQueries.getByGithubId(db, session.user.githubId);
if (!dbUser) {
dbUser = await userQueries.upsertFromGithub(db, {
githubId: session.user.githubId,
username: session.user.username,
displayName: session.user.name || undefined,
email: session.user.email || undefined,
avatarUrl: session.user.image || undefined,
});
}
if (!dbUser) {
return NextResponse.json(
{ error: 'Failed to create user record', code: 'USER_CREATE_FAILED' },
{ status: 500 }
);
}
// Verify repo ownership via GitHub API
let isOwner = false;
let githubError: string | null = null;
try {
const repoResponse = await fetch(
`https://api.github.com/repos/${owner}/${repo}`,
{
headers: {
Accept: 'application/vnd.github.v3+json',
'User-Agent': 'SkillHub',
},
signal: AbortSignal.timeout(10000),
}
);
if (repoResponse.ok) {
const repoData = await repoResponse.json();
if (repoData.owner?.login?.toLowerCase() === username.toLowerCase()) {
isOwner = true;
}
} else if (repoResponse.status === 404) {
return NextResponse.json(
{ error: 'Repository not found on GitHub', code: 'INVALID_REPO' },
{ status: 404 }
);
} else if (repoResponse.status === 403) {
githubError = 'GitHub API rate limit exceeded. Please try again later.';
}
} catch (fetchError) {
console.error('GitHub API fetch error:', fetchError);
githubError = 'Failed to verify repository ownership';
}
if (githubError) {
return NextResponse.json(
{ error: githubError, code: 'GITHUB_ERROR' },
{ status: 502 }
);
}
if (!isOwner) {
return NextResponse.json(
{ error: 'You are not the owner of this repository', code: 'NOT_OWNER' },
{ status: 403 }
);
}
// Count skills before blocking (for response message)
const blockedCount = await skillQueries.countByRepo(db, owner, repo);
// Block all skills from the repo
await skillQueries.blockByRepo(db, owner, repo);
// Block the discovered_repo entry to prevent re-indexing
await discoveredRepoQueries.blockRepo(db, `${owner}/${repo}`);
const sanitizedReason = reason ? sanitizeReason(String(reason)) : undefined;
console.log(
`[RepoRemoval] ${username} removed ${owner}/${repo} (${blockedCount} skills blocked)${sanitizedReason ? `${sanitizedReason}` : ''}`
);
return NextResponse.json({
success: true,
blockedCount,
repo: `${owner}/${repo}`,
});
} catch (error) {
console.error('Error processing repo removal request:', error);
return NextResponse.json(
{ error: 'Internal server error', code: 'SERVER_ERROR' },
{ status: 500 }
);
}
}

View File

@@ -97,8 +97,14 @@ export async function GET(request: NextRequest) {
const useMeilisearch = await isMeilisearchHealthy(); const useMeilisearch = await isMeilisearchHealthy();
if (useMeilisearch) { if (useMeilisearch) {
// Use Meilisearch for full-text search with relevance ranking // For 'recommended' sort, fetch more results from Meilisearch (by relevance),
// Note: category filter not supported in Meilisearch, handled in PostgreSQL fallback // then post-process to boost ai-reviewed skills to the top
const isRecommended = sort === 'recommended';
const meiliSort = isRecommended ? undefined : sort as 'stars' | 'downloads' | 'rating' | 'recent';
// Fetch extra results for recommended so we have enough after re-ranking
const meiliLimit = isRecommended ? Math.min(limit * 3, 60) : limit;
const meiliOffset = isRecommended ? 0 : offset;
const meiliResult = await meilisearchSearch({ const meiliResult = await meilisearchSearch({
query, query,
filters: { filters: {
@@ -106,13 +112,13 @@ export async function GET(request: NextRequest) {
minStars: minStars > 0 ? minStars : undefined, minStars: minStars > 0 ? minStars : undefined,
verified: verified ? true : undefined, verified: verified ? true : undefined,
}, },
sort: sort as 'stars' | 'downloads' | 'rating' | 'recent', sort: meiliSort,
limit, limit: meiliLimit,
offset, offset: meiliOffset,
}); });
if (meiliResult) { if (meiliResult) {
const skills = meiliResult.hits.map((hit) => ({ let skills = meiliResult.hits.map((hit) => ({
id: restoreIdFromMeili(hit.id), id: restoreIdFromMeili(hit.id),
name: hit.name, name: hit.name,
description: hit.description, description: hit.description,
@@ -121,13 +127,31 @@ export async function GET(request: NextRequest) {
githubStars: hit.githubStars, githubStars: hit.githubStars,
downloadCount: hit.downloadCount, downloadCount: hit.downloadCount,
securityScore: hit.securityScore, securityScore: hit.securityScore,
securityStatus: null, // Not available in Meilisearch yet securityStatus: hit.securityStatus || null,
rating: hit.rating, rating: hit.rating,
ratingCount: null, // Not available in Meilisearch yet ratingCount: null, // Not available in Meilisearch yet
isVerified: hit.isVerified, isVerified: hit.isVerified,
compatibility: { platforms: hit.platforms }, compatibility: { platforms: hit.platforms },
reviewStatus: hit.reviewStatus || null,
aiScore: hit.aiScore || null,
})); }));
// For recommended sort: boost ai-reviewed skills with score >= 75 to top,
// preserving Meilisearch relevance order within each group
if (isRecommended) {
const reviewed = skills.filter(s =>
s.aiScore && s.aiScore >= 75 &&
(s.reviewStatus === 'ai-reviewed' || s.reviewStatus === 'verified')
);
const rest = skills.filter(s =>
!(s.aiScore && s.aiScore >= 75 &&
(s.reviewStatus === 'ai-reviewed' || s.reviewStatus === 'verified'))
);
// Sort reviewed by score desc (relevance already handled by position)
reviewed.sort((a, b) => (b.aiScore || 0) - (a.aiScore || 0));
skills = [...reviewed, ...rest].slice(offset, offset + limit);
}
// Cache the result (5 minutes TTL) // Cache the result (5 minutes TTL)
await setCache( await setCache(
cacheKey, cacheKey,
@@ -159,12 +183,14 @@ export async function GET(request: NextRequest) {
// Fall back to PostgreSQL search // Fall back to PostgreSQL search
// Map sort parameter to database column // Map sort parameter to database column
const sortByMap: Record<string, 'stars' | 'downloads' | 'rating' | 'updated' | 'lastDownloaded'> = { const sortByMap: Record<string, 'stars' | 'downloads' | 'rating' | 'updated' | 'lastDownloaded' | 'aiScore' | 'recommended'> = {
stars: 'stars', stars: 'stars',
downloads: 'downloads', downloads: 'downloads',
rating: 'rating', rating: 'rating',
recent: 'updated', recent: 'updated',
lastDownloaded: 'lastDownloaded', lastDownloaded: 'lastDownloaded',
aiScore: 'aiScore',
recommended: 'recommended',
security: 'stars', // Use stars as fallback security: 'stars', // Use stars as fallback
}; };
const sortBy = sortByMap[sort] || 'downloads'; const sortBy = sortByMap[sort] || 'downloads';
@@ -209,6 +235,8 @@ export async function GET(request: NextRequest) {
isVerified: skill.isVerified, isVerified: skill.isVerified,
compatibility: skill.compatibility, compatibility: skill.compatibility,
updatedAt: skill.updatedAt, updatedAt: skill.updatedAt,
reviewStatus: skill.reviewStatus,
aiScore: skill.latestAiScore,
})); }));
// Cache the result (5 minutes TTL) // Cache the result (5 minutes TTL)

View File

@@ -18,17 +18,29 @@ vi.mock('@/lib/cache', () => ({
// Mock the db module - must be before imports that use it // Mock the db module - must be before imports that use it
vi.mock('@skillhub/db', () => { vi.mock('@skillhub/db', () => {
const mockStatsRow = [{ totalSkills: 100, totalContributors: 50 }];
const mockDownloadsRow = [{ totalDownloads: 5000 }];
const mockCategoryRow = [{ count: 8 }];
return { return {
createDb: vi.fn(() => ({ createDb: vi.fn(() => ({
select: vi.fn().mockReturnValue({ select: vi.fn().mockReturnValue({
from: vi.fn().mockResolvedValue([ from: vi.fn((table: unknown) => {
{ totalSkills: 100, totalDownloads: 5000, totalContributors: 50 }, // categories table query returns count directly (no .where())
]), if (table === 'categories-table') {
return Promise.resolve(mockCategoryRow);
}
// skills table: thenable for downloads (no .where()) + .where() for filtered stats
return {
where: vi.fn().mockResolvedValue(mockStatsRow),
then: (resolve: (v: unknown) => void, reject: (e: unknown) => void) =>
Promise.resolve(mockDownloadsRow).then(resolve, reject),
};
}),
}), }),
})), })),
skills: { downloadCount: 'download_count', githubOwner: 'github_owner' }, skills: { downloadCount: 'download_count', githubOwner: 'github_owner', isDuplicate: 'is_duplicate', skillType: 'skill_type' },
categories: {}, categories: 'categories-table',
sql: vi.fn(() => 'mock-sql'), sql: vi.fn((..._args: unknown[]) => 'mock-sql'),
}; };
}); });

View File

@@ -34,13 +34,23 @@ export async function GET(request: NextRequest) {
}); });
} }
// Consolidate all skill stats into a single query for better performance // Browse-ready filter: exclude duplicates (matches browseReadyFilter in queries.ts)
const browseReady = sql`${skills.isDuplicate} = false AND ${skills.isStale} = false`;
// Browse-ready skill stats (skills count + contributors)
const statsResult = await db const statsResult = await db
.select({ .select({
totalSkills: sql<number>`count(*)::int`, totalSkills: sql<number>`count(*)::int`,
totalDownloads: sql<number>`coalesce(sum(${skills.downloadCount}), 0)::int`,
totalContributors: sql<number>`count(distinct ${skills.githubOwner})::int`, totalContributors: sql<number>`count(distinct ${skills.githubOwner})::int`,
}) })
.from(skills)
.where(browseReady);
// Total downloads: count ALL downloads (real user actions, not filtered)
const downloadsResult = await db
.select({
totalDownloads: sql<number>`coalesce(sum(${skills.downloadCount}), 0)::int`,
})
.from(skills); .from(skills);
// Get category count in separate query (different table) // Get category count in separate query (different table)
@@ -53,7 +63,7 @@ export async function GET(request: NextRequest) {
const data: StatsData = { const data: StatsData = {
totalSkills: stats?.totalSkills ?? 0, totalSkills: stats?.totalSkills ?? 0,
totalDownloads: stats?.totalDownloads ?? 0, totalDownloads: downloadsResult[0]?.totalDownloads ?? 0,
totalCategories, totalCategories,
totalContributors: stats?.totalContributors ?? 0, totalContributors: stats?.totalContributors ?? 0,
platforms: 5, // Claude, Codex, Copilot, Cursor, Windsurf platforms: 5, // Claude, Codex, Copilot, Cursor, Windsurf

View File

@@ -261,6 +261,22 @@
@apply bg-surface-subtle text-text-primary; @apply bg-surface-subtle text-text-primary;
} }
/* Thin scrollbar */
.scrollbar-thin {
scrollbar-width: thin;
scrollbar-color: var(--color-border) transparent;
}
.scrollbar-thin::-webkit-scrollbar {
width: 6px;
}
.scrollbar-thin::-webkit-scrollbar-track {
background: transparent;
}
.scrollbar-thin::-webkit-scrollbar-thumb {
background: var(--color-border);
border-radius: 3px;
}
/* Card */ /* Card */
.card { .card {
@apply bg-surface-elevated rounded-2xl; @apply bg-surface-elevated rounded-2xl;

View File

@@ -1,8 +1,6 @@
import type { MetadataRoute } from 'next'; import type { MetadataRoute } from 'next';
import { createDb, skillQueries, categoryQueries } from '@skillhub/db'; import { createDb, skillQueries, categoryQueries } from '@skillhub/db';
import { getCanonicalPath, primaryDomain as BASE_URL } from '@/lib/seo';
const BASE_URL =
process.env.NEXT_PUBLIC_APP_URL || 'https://skills.palebluedot.live';
const locales = ['en', 'fa'] as const; const locales = ['en', 'fa'] as const;
@@ -27,7 +25,8 @@ const staticRoutes = [
]; ];
function makeEntry( function makeEntry(
path: string, locale: string,
route: string,
options?: { options?: {
lastModified?: Date; lastModified?: Date;
changeFrequency?: MetadataRoute.Sitemap[number]['changeFrequency']; changeFrequency?: MetadataRoute.Sitemap[number]['changeFrequency'];
@@ -35,17 +34,13 @@ function makeEntry(
} }
): MetadataRoute.Sitemap[number] { ): MetadataRoute.Sitemap[number] {
return { return {
url: `${BASE_URL}${path}`, url: `${BASE_URL}${getCanonicalPath(locale, route)}`,
lastModified: options?.lastModified, lastModified: options?.lastModified,
changeFrequency: options?.changeFrequency, changeFrequency: options?.changeFrequency,
priority: options?.priority, priority: options?.priority,
alternates: { alternates: {
languages: Object.fromEntries( languages: Object.fromEntries(
locales.map((locale) => { locales.map((l) => [l, `${BASE_URL}${getCanonicalPath(l, route)}`])
// For root path, use /en or /fa; for others, prefix with locale
const localePath = path.replace(/^\/(en|fa)/, `/${locale}`);
return [locale, `${BASE_URL}${localePath}`];
})
), ),
}, },
}; };
@@ -64,10 +59,9 @@ export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
// Static pages (for each locale) // Static pages (for each locale)
for (const locale of locales) { for (const locale of locales) {
for (const route of staticRoutes) { for (const route of staticRoutes) {
const path = `/${locale}${route}`;
const isHome = route === ''; const isHome = route === '';
entries.push( entries.push(
makeEntry(path, { makeEntry(locale, route, {
changeFrequency: isHome ? 'daily' : 'weekly', changeFrequency: isHome ? 'daily' : 'weekly',
priority: isHome ? 1.0 : 0.5, priority: isHome ? 1.0 : 0.5,
}) })
@@ -82,7 +76,7 @@ export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
for (const skill of allSkills) { for (const skill of allSkills) {
for (const locale of locales) { for (const locale of locales) {
entries.push( entries.push(
makeEntry(`/${locale}/skill/${skill.id}`, { makeEntry(locale, `/skill/${skill.id}`, {
lastModified: skill.updatedAt, lastModified: skill.updatedAt,
changeFrequency: 'weekly', changeFrequency: 'weekly',
priority: 0.7, priority: 0.7,
@@ -103,7 +97,7 @@ export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
for (const [owner, lastModified] of uniqueOwners) { for (const [owner, lastModified] of uniqueOwners) {
for (const locale of locales) { for (const locale of locales) {
entries.push( entries.push(
makeEntry(`/${locale}/owner/${owner}`, { makeEntry(locale, `/owner/${owner}`, {
lastModified, lastModified,
changeFrequency: 'weekly', changeFrequency: 'weekly',
priority: 0.6, priority: 0.6,
@@ -121,7 +115,7 @@ export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
for (const category of allCategories) { for (const category of allCategories) {
for (const locale of locales) { for (const locale of locales) {
entries.push( entries.push(
makeEntry(`/${locale}/browse?category=${category.slug}`, { makeEntry(locale, `/browse?category=${category.slug}`, {
changeFrequency: 'weekly', changeFrequency: 'weekly',
priority: 0.5, priority: 0.5,
}) })

View File

@@ -35,7 +35,7 @@ export function BetaBanner() {
if (!isVisible) return null; if (!isVisible) return null;
const count = skillCount || (locale === 'fa' ? '۱۷۲k' : '172k'); const count = skillCount || (locale === 'fa' ? '۱۶k' : '16k');
return ( return (
<div className="relative bg-gradient-to-r from-primary-600 to-primary-500 text-white"> <div className="relative bg-gradient-to-r from-primary-600 to-primary-500 text-white">

View File

@@ -58,13 +58,15 @@ export function BrowseFilters({
// Get current values from URL // Get current values from URL
const currentCategory = searchParams.get('category') || ''; const currentCategory = searchParams.get('category') || '';
const currentSort = searchParams.get('sort') || 'lastDownloaded'; const hasQuery = !!searchParams.get('q');
const defaultSort = hasQuery ? 'recommended' : 'lastDownloaded';
const currentSort = searchParams.get('sort') || defaultSort;
const currentFormat = searchParams.get('format') || ''; const currentFormat = searchParams.get('format') || '';
// Count active filters for mobile badge // Count active filters for mobile badge
const activeFilterCount = [ const activeFilterCount = [
currentCategory ? 1 : 0, currentCategory ? 1 : 0,
currentSort !== 'lastDownloaded' ? 1 : 0, currentSort !== defaultSort ? 1 : 0,
currentFormat ? 1 : 0, currentFormat ? 1 : 0,
].reduce((a, b) => a + b, 0); ].reduce((a, b) => a + b, 0);
@@ -74,7 +76,7 @@ export function BrowseFilters({
const params = new URLSearchParams(searchParams.toString()); const params = new URLSearchParams(searchParams.toString());
Object.entries(updates).forEach(([key, value]) => { Object.entries(updates).forEach(([key, value]) => {
if (value === null || value === '' || value === 'all' || value === 'false') { if (value === null || value === '' || value === 'false') {
params.delete(key); params.delete(key);
} else { } else {
params.set(key, value); params.set(key, value);
@@ -98,7 +100,7 @@ export function BrowseFilters({
// Handle sort change // Handle sort change
const handleSortChange = (e: React.ChangeEvent<HTMLSelectElement>) => { const handleSortChange = (e: React.ChangeEvent<HTMLSelectElement>) => {
updateParams({ sort: e.target.value === 'lastDownloaded' ? null : e.target.value }); updateParams({ sort: e.target.value === defaultSort ? null : e.target.value });
}; };
// Handle format change // Handle format change
@@ -272,6 +274,10 @@ export function SearchBar({ placeholder, defaultValue = '' }: SearchBarProps) {
if (searchQuery) { if (searchQuery) {
params.set('q', searchQuery); params.set('q', searchQuery);
// Remove sort param so server-side defaults to 'recommended' for searches
if (!params.get('sort') || params.get('sort') === 'lastDownloaded') {
params.delete('sort');
}
} else { } else {
params.delete('q'); params.delete('q');
} }
@@ -357,7 +363,8 @@ export function ActiveFilters({
const searchParams = useSearchParams(); const searchParams = useSearchParams();
const [isPending, startTransition] = useTransition(); const [isPending, startTransition] = useTransition();
const hasFilters = query || categoryId || (sortBy && sortBy !== 'lastDownloaded'); const defaultSort = query ? 'recommended' : 'lastDownloaded';
const hasFilters = query || categoryId || (sortBy && sortBy !== defaultSort);
if (!hasFilters) return null; if (!hasFilters) return null;
@@ -406,7 +413,7 @@ export function ActiveFilters({
</span> </span>
)} )}
{sortBy && sortBy !== 'lastDownloaded' && sortName && ( {sortBy && sortBy !== defaultSort && sortName && (
<span className="inline-flex items-center gap-1.5 px-3 py-1.5 bg-surface-subtle text-text-secondary text-sm rounded-full"> <span className="inline-flex items-center gap-1.5 px-3 py-1.5 bg-surface-subtle text-text-secondary text-sm rounded-full">
<SlidersHorizontal className="w-3 h-3" /> <SlidersHorizontal className="w-3 h-3" />
<span>{sortName}</span> <span>{sortName}</span>
@@ -466,7 +473,7 @@ export function EmptyState({ query, hasFilters, locale = 'en', translations }: E
<Sparkles className="w-10 h-10 text-text-muted" /> <Sparkles className="w-10 h-10 text-text-muted" />
</div> </div>
<h3 className="text-xl font-semibold text-text-primary mb-2"> <h3 className="text-xl font-semibold text-text-primary mb-2">
{query ? translations.noResultsWithQuery.replace('{query}', query) : translations.noResults} {query ? translations.noResultsWithQuery : translations.noResults}
</h3> </h3>
<p className="text-text-secondary mb-6 max-w-md mx-auto"> <p className="text-text-secondary mb-6 max-w-md mx-auto">
{translations.tryDifferent} {translations.tryDifferent}

View File

@@ -2,7 +2,7 @@
import { useState, useEffect, useCallback } from 'react'; import { useState, useEffect, useCallback } from 'react';
import { useSession, signIn } from 'next-auth/react'; import { useSession, signIn } from 'next-auth/react';
import { Loader2, CheckCircle, AlertCircle, Github, Clock, Plus, Minus, ChevronDown, ChevronUp, ExternalLink } from 'lucide-react'; import { Loader2, CheckCircle, AlertCircle, Github, Clock, Plus, Minus, ChevronDown, ChevronUp, ExternalLink, Trash2 } from 'lucide-react';
import { fetchWithCsrf } from '@/lib/csrf-client'; import { fetchWithCsrf } from '@/lib/csrf-client';
const isMirror = process.env.NEXT_PUBLIC_IS_PRIMARY === 'false'; const isMirror = process.env.NEXT_PUBLIC_IS_PRIMARY === 'false';
@@ -33,6 +33,26 @@ interface ClaimFormProps {
tabs: { tabs: {
remove: string; remove: string;
add: string; add: string;
removeRepo: string;
};
removeRepoForm: {
repoUrl: string;
repoUrlPlaceholder: string;
repoUrlHelp: string;
reason: string;
reasonPlaceholder: string;
submit: string;
submitting: string;
};
removeRepoSuccess: {
title: string;
description: string;
descriptionZero: string;
};
removeRepoError: {
notOwner: string;
invalidRepo: string;
parseError: string;
}; };
form: { form: {
skillId: string; skillId: string;
@@ -55,6 +75,8 @@ interface ClaimFormProps {
success: { success: {
title: string; title: string;
description: string; description: string;
pendingTitle: string;
pendingDescription: string;
viewRequests: string; viewRequests: string;
}; };
addSuccess: { addSuccess: {
@@ -67,6 +89,8 @@ interface ClaimFormProps {
foundSkillsIn: string; foundSkillsIn: string;
root: string; root: string;
andMore: string; andMore: string;
reEnabledTitle: string;
reEnabledDescription: string;
}; };
error: { error: {
notOwner: string; notOwner: string;
@@ -79,6 +103,7 @@ interface ClaimFormProps {
rateLimitExceeded: string; rateLimitExceeded: string;
networkTimeout: string; networkTimeout: string;
generic: string; generic: string;
repoBlockedByOwner: string;
}; };
myRequests: { myRequests: {
title: string; title: string;
@@ -120,15 +145,15 @@ interface AddRequest {
export function ClaimForm({ translations }: ClaimFormProps) { export function ClaimForm({ translations }: ClaimFormProps) {
const { data: session, status } = useSession(); const { data: session, status } = useSession();
const [activeTab, setActiveTab] = useState<'remove' | 'add'>('add'); // Default to 'add' tab const [activeTab, setActiveTab] = useState<'remove' | 'add' | 'remove-repo'>('add'); // Default to 'add' tab
const [expandedRequests, setExpandedRequests] = useState<Set<string>>(new Set()); const [expandedRequests, setExpandedRequests] = useState<Set<string>>(new Set());
// Read tab from URL hash on mount and update on hash change // Read tab from URL hash on mount and update on hash change
useEffect(() => { useEffect(() => {
const readHashTab = () => { const readHashTab = () => {
const hash = window.location.hash.slice(1); const hash = window.location.hash.slice(1);
if (hash === 'remove' || hash === 'add') { if (hash === 'remove' || hash === 'add' || hash === 'remove-repo') {
setActiveTab(hash); setActiveTab(hash as 'remove' | 'add' | 'remove-repo');
} }
}; };
readHashTab(); readHashTab();
@@ -137,7 +162,7 @@ export function ClaimForm({ translations }: ClaimFormProps) {
}, []); }, []);
// Update URL hash when tab changes // Update URL hash when tab changes
const handleTabChange = useCallback((tab: 'remove' | 'add') => { const handleTabChange = useCallback((tab: 'remove' | 'add' | 'remove-repo') => {
setActiveTab(tab); setActiveTab(tab);
window.history.replaceState(null, '', `#${tab}`); window.history.replaceState(null, '', `#${tab}`);
}, []); }, []);
@@ -161,6 +186,7 @@ export function ClaimForm({ translations }: ClaimFormProps) {
const [isSubmittingRemove, setIsSubmittingRemove] = useState(false); const [isSubmittingRemove, setIsSubmittingRemove] = useState(false);
const [removeError, setRemoveError] = useState(''); const [removeError, setRemoveError] = useState('');
const [removeSuccess, setRemoveSuccess] = useState(false); const [removeSuccess, setRemoveSuccess] = useState(false);
const [removePending, setRemovePending] = useState(false);
const [removalRequests, setRemovalRequests] = useState<RemovalRequest[]>([]); const [removalRequests, setRemovalRequests] = useState<RemovalRequest[]>([]);
const [loadingRemovalRequests, setLoadingRemovalRequests] = useState(false); const [loadingRemovalRequests, setLoadingRemovalRequests] = useState(false);
@@ -174,9 +200,19 @@ export function ClaimForm({ translations }: ClaimFormProps) {
const [addHasSkillMd, setAddHasSkillMd] = useState(true); const [addHasSkillMd, setAddHasSkillMd] = useState(true);
const [addSkillCount, setAddSkillCount] = useState(0); const [addSkillCount, setAddSkillCount] = useState(0);
const [addSkillPaths, setAddSkillPaths] = useState<string[]>([]); const [addSkillPaths, setAddSkillPaths] = useState<string[]>([]);
const [addReEnabled, setAddReEnabled] = useState(false);
const [addRequests, setAddRequests] = useState<AddRequest[]>([]); const [addRequests, setAddRequests] = useState<AddRequest[]>([]);
const [loadingAddRequests, setLoadingAddRequests] = useState(false); const [loadingAddRequests, setLoadingAddRequests] = useState(false);
// Remove-repo form state
const [repoUrl, setRepoUrl] = useState('');
const [repoReason, setRepoReason] = useState('');
const [isSubmittingRepo, setIsSubmittingRepo] = useState(false);
const [repoError, setRepoError] = useState('');
const [repoSuccess, setRepoSuccess] = useState(false);
const [repoBlockedCount, setRepoBlockedCount] = useState(0);
const [repoName, setRepoName] = useState('');
// Fetch user's existing requests // Fetch user's existing requests
useEffect(() => { useEffect(() => {
if (session) { if (session) {
@@ -267,6 +303,7 @@ export function ClaimForm({ translations }: ClaimFormProps) {
} }
setRemoveSuccess(true); setRemoveSuccess(true);
setRemovePending(data.pending || false);
setSkillId(''); setSkillId('');
setRemoveReason(''); setRemoveReason('');
fetchRemovalRequests(); fetchRemovalRequests();
@@ -327,6 +364,9 @@ export function ClaimForm({ translations }: ClaimFormProps) {
case 'ALREADY_PENDING': case 'ALREADY_PENDING':
setAddError(translations.error.alreadyPending); setAddError(translations.error.alreadyPending);
break; break;
case 'REPO_BLOCKED_BY_OWNER':
setAddError(translations.error.repoBlockedByOwner);
break;
case 'AUTH_REQUIRED': case 'AUTH_REQUIRED':
signIn('github'); signIn('github');
return; return;
@@ -338,6 +378,7 @@ export function ClaimForm({ translations }: ClaimFormProps) {
} }
setAddSuccess(true); setAddSuccess(true);
setAddReEnabled(data.reEnabled ?? false);
setAddHasSkillMd(data.hasSkillMd ?? true); setAddHasSkillMd(data.hasSkillMd ?? true);
setAddSkillCount(data.skillCount ?? 0); setAddSkillCount(data.skillCount ?? 0);
setAddSkillPaths(data.skillPaths ?? []); setAddSkillPaths(data.skillPaths ?? []);
@@ -351,6 +392,61 @@ export function ClaimForm({ translations }: ClaimFormProps) {
} }
}; };
const handleRepoRemovalSubmit = async (e: React.FormEvent) => {
e.preventDefault();
if (!session) {
signIn('github');
return;
}
if (!repoUrl.trim()) return;
setIsSubmittingRepo(true);
setRepoError('');
setRepoSuccess(false);
try {
const res = await fetchWithCsrf('/api/skills/repo-removal-request', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ repoUrl: repoUrl.trim(), reason: repoReason.trim() }),
});
const data = await res.json();
if (!res.ok) {
switch (data.code) {
case 'NOT_OWNER':
setRepoError(translations.removeRepoError.notOwner);
break;
case 'INVALID_REPO':
setRepoError(translations.removeRepoError.invalidRepo);
break;
case 'PARSE_ERROR':
setRepoError(translations.removeRepoError.parseError);
break;
case 'AUTH_REQUIRED':
signIn('github');
return;
default:
console.error('Repo removal error:', data);
setRepoError(translations.error.generic);
}
return;
}
setRepoSuccess(true);
setRepoBlockedCount(data.blockedCount ?? 0);
setRepoName(data.repo ?? repoUrl.trim());
setRepoUrl('');
setRepoReason('');
} catch {
setRepoError(translations.error.generic);
} finally {
setIsSubmittingRepo(false);
}
};
const getStatusBadge = (requestStatus: string) => { const getStatusBadge = (requestStatus: string) => {
switch (requestStatus) { switch (requestStatus) {
case 'pending': case 'pending':
@@ -438,11 +534,38 @@ export function ClaimForm({ translations }: ClaimFormProps) {
<div className="card p-8 text-center"> <div className="card p-8 text-center">
<CheckCircle className="w-12 h-12 mx-auto mb-4 text-success" /> <CheckCircle className="w-12 h-12 mx-auto mb-4 text-success" />
<h2 className="text-xl font-semibold text-text-primary mb-2"> <h2 className="text-xl font-semibold text-text-primary mb-2">
{translations.success.title} {removePending ? translations.success.pendingTitle : translations.success.title}
</h2> </h2>
<p className="text-text-secondary mb-6">{translations.success.description}</p> <p className="text-text-secondary mb-6">
{removePending ? translations.success.pendingDescription : translations.success.description}
</p>
<button <button
onClick={() => setRemoveSuccess(false)} onClick={() => { setRemoveSuccess(false); setRemovePending(false); }}
className="btn-secondary"
>
{translations.success.viewRequests}
</button>
</div>
);
}
// Repo removal success state
if (repoSuccess) {
return (
<div className="card p-8 text-center">
<CheckCircle className="w-12 h-12 mx-auto mb-4 text-success" />
<h2 className="text-xl font-semibold text-text-primary mb-2">
{translations.removeRepoSuccess.title}
</h2>
<p className="text-text-secondary mb-6">
{repoBlockedCount > 0
? translations.removeRepoSuccess.description
.replace('{count}', String(repoBlockedCount))
.replace('{repo}', repoName)
: translations.removeRepoSuccess.descriptionZero}
</p>
<button
onClick={() => { setRepoSuccess(false); setRepoBlockedCount(0); setRepoName(''); }}
className="btn-secondary" className="btn-secondary"
> >
{translations.success.viewRequests} {translations.success.viewRequests}
@@ -454,8 +577,12 @@ export function ClaimForm({ translations }: ClaimFormProps) {
// Add success state // Add success state
if (addSuccess) { if (addSuccess) {
// Determine which message to show // Determine which message to show
let successTitle = translations.addSuccess.title;
let successDescription: string | React.ReactNode; let successDescription: string | React.ReactNode;
if (addSkillCount > 1) { if (addReEnabled) {
successTitle = translations.addSuccess.reEnabledTitle;
successDescription = translations.addSuccess.reEnabledDescription;
} else if (addSkillCount > 1) {
successDescription = ( successDescription = (
<> <>
{translations.addSuccess.descriptionMultiplePrefix} {translations.addSuccess.descriptionMultiplePrefix}
@@ -473,7 +600,7 @@ export function ClaimForm({ translations }: ClaimFormProps) {
<div className="card p-8 text-center"> <div className="card p-8 text-center">
<CheckCircle className="w-12 h-12 mx-auto mb-4 text-success" /> <CheckCircle className="w-12 h-12 mx-auto mb-4 text-success" />
<h2 className="text-xl font-semibold text-text-primary mb-2"> <h2 className="text-xl font-semibold text-text-primary mb-2">
{translations.addSuccess.title} {successTitle}
</h2> </h2>
<p className="text-text-secondary mb-4"> <p className="text-text-secondary mb-4">
{successDescription} {successDescription}
@@ -494,7 +621,7 @@ export function ClaimForm({ translations }: ClaimFormProps) {
</div> </div>
)} )}
<button <button
onClick={() => setAddSuccess(false)} onClick={() => { setAddSuccess(false); setAddReEnabled(false); }}
className="btn-secondary" className="btn-secondary"
> >
{translations.addSuccess.viewRequests} {translations.addSuccess.viewRequests}
@@ -529,6 +656,17 @@ export function ClaimForm({ translations }: ClaimFormProps) {
<Plus className="w-4 h-4" /> <Plus className="w-4 h-4" />
{translations.tabs.add} {translations.tabs.add}
</button> </button>
<button
onClick={() => handleTabChange('remove-repo')}
className={`flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors ${
activeTab === 'remove-repo'
? 'border-error text-error'
: 'border-transparent text-text-muted hover:text-text-primary'
}`}
>
<Trash2 className="w-4 h-4" />
{translations.tabs.removeRepo}
</button>
</div> </div>
{/* Remove Form */} {/* Remove Form */}
@@ -646,6 +784,79 @@ export function ClaimForm({ translations }: ClaimFormProps) {
</> </>
)} )}
{/* Remove Repository Form */}
{activeTab === 'remove-repo' && (
<div className="card p-6">
<form onSubmit={handleRepoRemovalSubmit} className="space-y-4">
<div>
<label
htmlFor="repoUrl"
className="block text-sm font-medium text-text-primary mb-2"
>
{translations.removeRepoForm.repoUrl}
</label>
<input
type="text"
id="repoUrl"
value={repoUrl}
onChange={(e) => setRepoUrl(e.target.value)}
placeholder={translations.removeRepoForm.repoUrlPlaceholder}
className="w-full px-4 py-2 border border-border rounded-lg bg-surface text-text-primary placeholder-text-muted focus:outline-none focus:ring-2 focus:ring-error"
dir="ltr"
required
/>
<p className="text-xs text-text-muted mt-1">
{translations.removeRepoForm.repoUrlHelp}
</p>
</div>
<div>
<label
htmlFor="repoReason"
className="block text-sm font-medium text-text-primary mb-2"
>
{translations.removeRepoForm.reason} <span className="text-text-muted">{translations.optional}</span>
</label>
<textarea
id="repoReason"
value={repoReason}
onChange={(e) => setRepoReason(e.target.value)}
placeholder={translations.removeRepoForm.reasonPlaceholder}
rows={3}
className="w-full px-4 py-2 border border-border rounded-lg bg-surface text-text-primary placeholder-text-muted focus:outline-none focus:ring-2 focus:ring-primary-500 resize-none"
/>
</div>
{repoError && (
<div className="p-3 bg-error-bg border border-error/30 rounded-lg">
<div className="flex items-center gap-2 text-error">
<AlertCircle className="w-4 h-4" />
<p className="text-sm">{repoError}</p>
</div>
</div>
)}
<button
type="submit"
disabled={isSubmittingRepo || !repoUrl.trim()}
className="w-full justify-center inline-flex items-center gap-2 px-4 py-2 bg-error text-white rounded-lg font-medium hover:bg-error/90 transition-colors disabled:opacity-50"
>
{isSubmittingRepo ? (
<>
<Loader2 className="w-4 h-4 animate-spin" />
{translations.removeRepoForm.submitting}
</>
) : (
<>
<Trash2 className="w-4 h-4" />
{translations.removeRepoForm.submit}
</>
)}
</button>
</form>
</div>
)}
{/* Add Form */} {/* Add Form */}
{activeTab === 'add' && ( {activeTab === 'add' && (
<> <>

View File

@@ -16,6 +16,7 @@ export function Footer() {
{ name: t('links.categories'), href: `/${locale}/categories` }, { name: t('links.categories'), href: `/${locale}/categories` },
{ name: t('links.featured'), href: `/${locale}/featured` }, { name: t('links.featured'), href: `/${locale}/featured` },
{ name: t('links.newSkills'), href: `/${locale}/new` }, { name: t('links.newSkills'), href: `/${locale}/new` },
{ name: t('links.reviewedSkills'), href: `/${locale}/reviewed` },
], ],
resources: [ resources: [
{ name: t('links.documentation'), href: `/${locale}/docs` }, { name: t('links.documentation'), href: `/${locale}/docs` },

View File

@@ -21,6 +21,7 @@ export function Header() {
{ name: t('home'), href: `/${locale}` }, { name: t('home'), href: `/${locale}` },
{ name: t('browse'), href: `/${locale}/browse` }, { name: t('browse'), href: `/${locale}/browse` },
{ name: t('categories'), href: `/${locale}/categories` }, { name: t('categories'), href: `/${locale}/categories` },
{ name: t('reviewed'), href: `/${locale}/reviewed` },
{ name: t('docs'), href: `/${locale}/docs` }, { name: t('docs'), href: `/${locale}/docs` },
]; ];

View File

@@ -15,6 +15,7 @@ export function HeroSearch({ placeholder, locale }: HeroSearchProps) {
const handleSubmit = (e: FormEvent) => { const handleSubmit = (e: FormEvent) => {
e.preventDefault(); e.preventDefault();
window.dispatchEvent(new Event('progressbar:start'));
if (query.trim()) { if (query.trim()) {
router.push(`/${locale}/browse?q=${encodeURIComponent(query.trim())}`); router.push(`/${locale}/browse?q=${encodeURIComponent(query.trim())}`);
} else { } else {

View File

@@ -0,0 +1,164 @@
'use client';
import { useEffect, useRef } from 'react';
export function ProgressBar() {
const barRef = useRef<HTMLDivElement>(null);
const timerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const isRunningRef = useRef(false);
const currentUrlRef = useRef('');
useEffect(() => {
const bar = barRef.current;
if (!bar) return;
currentUrlRef.current = location.pathname + location.search;
function clearTimers() {
if (timerRef.current) {
clearTimeout(timerRef.current);
timerRef.current = null;
}
}
function start() {
clearTimers();
isRunningRef.current = true;
bar!.style.transition = 'none';
bar!.style.width = '0%';
bar!.style.opacity = '1';
// Force reflow so browser registers 0% before animating
void bar!.offsetWidth;
bar!.style.transition = 'width 2s cubic-bezier(0.1, 0.5, 0.3, 1)';
bar!.style.width = '80%';
// Safety: auto-complete after 10s
timerRef.current = setTimeout(done, 10000);
}
function done() {
clearTimers();
isRunningRef.current = false;
bar!.style.transition = 'width 200ms ease-out';
bar!.style.width = '100%';
timerRef.current = setTimeout(() => {
if (!barRef.current) return;
barRef.current.style.transition = 'opacity 300ms ease-out';
barRef.current.style.opacity = '0';
}, 250);
}
// Called when URL has changed (navigation completed)
function onUrlChange() {
const newUrl = location.pathname + location.search;
if (newUrl === currentUrlRef.current) return;
currentUrlRef.current = newUrl;
if (isRunningRef.current) {
// Bar was started by click handler — complete it
done();
} else {
// Programmatic navigation (search, filters, router.push/replace)
// Show a quick fill animation as feedback
start();
requestAnimationFrame(() => requestAnimationFrame(() => done()));
}
}
// 1. Intercept <a> tag clicks (for Link components)
const handleClick = (e: MouseEvent) => {
const anchor = (e.target as HTMLElement).closest('a');
if (!anchor) return;
const href = anchor.getAttribute('href');
if (!href) return;
if (
href.startsWith('http') ||
href.startsWith('#') ||
href.startsWith('mailto:') ||
anchor.target === '_blank' ||
e.ctrlKey ||
e.metaKey ||
e.shiftKey
) {
return;
}
if (href === currentUrlRef.current) return;
start();
};
document.addEventListener('click', handleClick, true);
// 2. Monkey-patch pushState/replaceState for programmatic navigation
const origPushState = history.pushState;
const origReplaceState = history.replaceState;
history.pushState = function (
data: unknown,
unused: string,
url?: string | URL | null,
) {
origPushState.call(this, data, unused, url);
onUrlChange();
};
history.replaceState = function (
data: unknown,
unused: string,
url?: string | URL | null,
) {
origReplaceState.call(this, data, unused, url);
onUrlChange();
};
// 3. Back/forward navigation
const handlePopState = () => onUrlChange();
window.addEventListener('popstate', handlePopState);
// 4. Custom event for programmatic navigation (e.g. form submits with router.push)
const handleManualStart = () => {
if (!isRunningRef.current) start();
};
window.addEventListener('progressbar:start', handleManualStart);
return () => {
clearTimers();
document.removeEventListener('click', handleClick, true);
history.pushState = origPushState;
history.replaceState = origReplaceState;
window.removeEventListener('popstate', handlePopState);
window.removeEventListener('progressbar:start', handleManualStart);
};
}, []);
return (
<div
style={{
position: 'fixed',
top: 0,
left: 0,
right: 0,
zIndex: 99999,
pointerEvents: 'none',
height: '3px',
}}
>
<div
ref={barRef}
style={{
height: '100%',
background: 'linear-gradient(90deg, #0284c7, #38bdf8)',
width: '0%',
opacity: '0',
boxShadow: '0 0 8px rgba(2, 132, 199, 0.4)',
}}
/>
</div>
);
}

View File

@@ -0,0 +1,70 @@
'use client';
import { useRouter, useSearchParams } from 'next/navigation';
import { useTransition } from 'react';
interface ReviewedSortSelectorProps {
locale: string;
translations: {
sortBy: string;
reviewDate: string;
aiScore: string;
scoreFilter: string;
scoreAbove50: string;
scoreAll: string;
};
}
export function ReviewedSortSelector({ locale, translations }: ReviewedSortSelectorProps) {
const router = useRouter();
const searchParams = useSearchParams();
const [isPending, startTransition] = useTransition();
const currentSort = searchParams.get('sort') || 'reviewDate';
const currentScore = searchParams.get('score') || '50';
const navigate = (updates: Record<string, string>) => {
const params = new URLSearchParams(searchParams.toString());
for (const [key, value] of Object.entries(updates)) {
params.set(key, value);
}
params.delete('page');
startTransition(() => {
router.push(`/${locale}/reviewed?${params.toString()}`);
});
};
return (
<div className="flex items-center gap-4 flex-wrap">
<div className="flex items-center gap-2">
<label htmlFor="sort-select" className="text-sm text-text-secondary whitespace-nowrap">
{translations.sortBy}
</label>
<select
id="sort-select"
value={currentSort}
onChange={(e) => navigate({ sort: e.target.value })}
disabled={isPending}
className="text-sm border border-border rounded-lg px-3 py-1.5 bg-surface text-text-primary focus:outline-none focus:ring-2 focus:ring-primary-500 disabled:opacity-50"
>
<option value="reviewDate">{translations.reviewDate}</option>
<option value="aiScore">{translations.aiScore}</option>
</select>
</div>
<div className="flex items-center gap-2">
<label htmlFor="score-filter" className="text-sm text-text-secondary whitespace-nowrap">
{translations.scoreFilter}
</label>
<select
id="score-filter"
value={currentScore}
onChange={(e) => navigate({ score: e.target.value })}
disabled={isPending}
className="text-sm border border-border rounded-lg px-3 py-1.5 bg-surface text-text-primary focus:outline-none focus:ring-2 focus:ring-primary-500 disabled:opacity-50"
>
<option value="50">{translations.scoreAbove50}</option>
<option value="0">{translations.scoreAll}</option>
</select>
</div>
</div>
);
}

View File

@@ -0,0 +1,52 @@
'use client';
import { useState, useEffect } from 'react';
import { X, ShieldAlert } from 'lucide-react';
import { useTranslations } from 'next-intl';
const BLOG_POST_URL = 'https://blog.palebluedot.live/2026/03/19/malware-openclaw-skills-security-advisory/';
const STORAGE_KEY = 'security-alert-openclaw-dismissed';
export function SecurityAlertBanner() {
const [isVisible, setIsVisible] = useState(false);
const t = useTranslations('securityAlert');
useEffect(() => {
const dismissed = localStorage.getItem(STORAGE_KEY);
if (!dismissed) {
setIsVisible(true);
}
}, []);
const handleDismiss = () => {
setIsVisible(false);
localStorage.setItem(STORAGE_KEY, 'true');
};
if (!isVisible) return null;
return (
<div className="relative bg-error-bg border-b-2 border-error">
<div className="container-main py-2.5 px-4 flex items-center justify-center gap-2 text-sm">
<ShieldAlert className="w-4 h-4 text-error flex-shrink-0" />
<span className="font-semibold text-error">{t('label')}</span>
<span className="text-text-primary">{t('text')}</span>
<a
href={BLOG_POST_URL}
target="_blank"
rel="noopener noreferrer"
className="text-error underline underline-offset-2 hover:text-error/80 transition-colors font-medium"
>
{t('link')}
</a>
<button
onClick={handleDismiss}
className="absolute end-2 sm:end-4 p-1 hover:bg-error/10 rounded transition-colors"
aria-label={t('dismiss')}
>
<X className="w-4 h-4 text-error" />
</button>
</div>
</div>
);
}

View File

@@ -1,5 +1,5 @@
import Link from 'next/link'; import Link from 'next/link';
import { Star, Download, Shield, CheckCircle, Clock, RefreshCw } from 'lucide-react'; import { Star, Download, Shield, CheckCircle, Clock, RefreshCw, Sparkles, XCircle, ShieldAlert } from 'lucide-react';
import { formatCompactNumber } from '@/lib/format-number'; import { formatCompactNumber } from '@/lib/format-number';
const FORMAT_BADGE_LABELS: Record<string, string> = { const FORMAT_BADGE_LABELS: Record<string, string> = {
@@ -21,15 +21,20 @@ interface SkillCardProps {
ratingCount: number | null; ratingCount: number | null;
securityStatus: string | null; securityStatus: string | null;
isVerified: boolean | null; isVerified: boolean | null;
reviewStatus?: string | null;
aiScore?: number | null;
latestAiScore?: number | null;
sourceFormat?: string | null; sourceFormat?: string | null;
createdAt?: Date | null; createdAt?: Date | string | null;
updatedAt?: Date | null; updatedAt?: Date | string | null;
reviewOutdated?: boolean | null;
isMalicious?: boolean | null;
}; };
locale: string; locale: string;
/** Show time badge (for New Skills page) */ /** Show time badge (for New Skills page) */
showTimeBadge?: 'created' | 'updated' | null; showTimeBadge?: 'created' | 'updated' | null;
/** Format time as "X days ago" */ /** Format time as "X days ago" */
formatTimeAgo?: (date: Date | null, locale: string) => string; formatTimeAgo?: (date: Date | string | null, locale: string) => string;
} }
export function SkillCard({ skill, locale, showTimeBadge, formatTimeAgo }: SkillCardProps) { export function SkillCard({ skill, locale, showTimeBadge, formatTimeAgo }: SkillCardProps) {
@@ -53,6 +58,24 @@ export function SkillCard({ skill, locale, showTimeBadge, formatTimeAgo }: Skill
const showRating = (skill.ratingCount ?? 0) >= 3; const showRating = (skill.ratingCount ?? 0) >= 3;
// AI review score badge (for all reviewed skills)
const getAiScoreBadge = () => {
const rs = skill.reviewStatus;
if (!rs || rs === 'unreviewed' || rs === 'auto-scored') return null;
const score = skill.aiScore ?? skill.latestAiScore;
if (score == null) return null;
if (score === 0) {
return { score: 0, color: 'text-error bg-error/10 border-error/20', rejected: true };
}
const color = score >= 75
? 'text-success bg-success/10 border-success/20'
: score >= 50
? 'text-gold bg-gold/10 border-gold/20'
: 'text-text-muted bg-surface-subtle border-border';
return { score, color, rejected: false };
};
const aiScoreBadge = getAiScoreBadge();
return ( return (
<Link <Link
href={`/${locale}/skill/${skill.id}`} href={`/${locale}/skill/${skill.id}`}
@@ -101,8 +124,30 @@ export function SkillCard({ skill, locale, showTimeBadge, formatTimeAgo }: Skill
{skill.description} {skill.description}
</p> </p>
{/* Metadata Row: Stars + Downloads + Rating */} {/* Metadata Row: Malware / AI Score + Stars + Downloads + Rating */}
<div className="flex flex-wrap items-center gap-4 text-sm text-text-muted mb-2"> <div className="flex flex-wrap items-center gap-4 text-sm text-text-muted mb-2">
{skill.isMalicious && (
<span className="flex items-center gap-1 px-1.5 py-0.5 text-xs font-bold rounded border text-error bg-error/10 border-error/20">
<ShieldAlert className="w-3 h-3" />
<span>{locale === 'fa' ? 'بدافزار' : 'Malware'}</span>
</span>
)}
{!skill.isMalicious && aiScoreBadge && (
<span className={`flex items-center gap-1 px-1.5 py-0.5 text-xs font-medium rounded border ${aiScoreBadge.color}`}>
{aiScoreBadge.rejected ? (
<>
<XCircle className="w-3 h-3" />
<span>{locale === 'fa' ? 'رد شده' : 'Rejected'}</span>
</>
) : (
<>
<Sparkles className="w-3 h-3" />
<span className="ltr-nums">{aiScoreBadge.score}</span>
{skill.reviewOutdated && <span className="text-warning" title="Review based on previous version"></span>}
</>
)}
</span>
)}
<span className="flex items-center gap-1"> <span className="flex items-center gap-1">
<Star className="w-4 h-4" /> <Star className="w-4 h-4" />
<span className="ltr-nums">{formatCompactNumber(skill.githubStars || 0, locale)}</span> <span className="ltr-nums">{formatCompactNumber(skill.githubStars || 0, locale)}</span>

View File

@@ -0,0 +1,46 @@
import { type NextRequest, NextResponse } from 'next/server';
import { auth } from '@/lib/auth';
import { createDb, userQueries } from '@skillhub/db';
/**
* Check if the current request is from an admin user.
* Supports two auth methods:
* 1. API key via Authorization: Bearer <REVIEW_API_KEY> header (for automation/scripts)
* 2. Session-based admin check (for dashboard)
*
* Pass the request object to enable API key auth.
* Returns the user on success, or a NextResponse error to return early.
*/
export async function requireAdmin(request?: NextRequest): Promise<
| { authorized: true; username: string }
| { authorized: false; response: NextResponse }
> {
// Check API key auth first (for automation/scripting)
if (request) {
const authHeader = request.headers.get('authorization');
const apiKey = process.env.REVIEW_API_KEY;
if (apiKey && authHeader === `Bearer ${apiKey}`) {
return { authorized: true, username: 'api-key' };
}
}
// Fall back to session auth
const session = await auth();
if (!session?.user?.githubId) {
return {
authorized: false,
response: NextResponse.json({ error: 'Unauthorized' }, { status: 401 }),
};
}
const db = createDb();
const user = await userQueries.getByGithubId(db, session.user.githubId);
if (!user?.isAdmin) {
return {
authorized: false,
response: NextResponse.json({ error: 'Admin access required' }, { status: 403 }),
};
}
return { authorized: true, username: user.username };
}

View File

@@ -129,16 +129,50 @@ export async function isCacheAvailable(): Promise<boolean> {
} }
} }
/**
* Get cached data or fetch and cache it.
* If Redis is unavailable, falls back to fetcher directly (graceful degradation).
*/
export async function getOrSetCache<T>(
key: string,
ttlSeconds: number,
fetcher: () => Promise<T>
): Promise<T> {
const cached = await getCached<T>(key);
if (cached !== null) return cached;
const data = await fetcher();
// Fire-and-forget cache write
setCache(key, data, ttlSeconds).catch(() => {});
return data;
}
// Cache key builders for consistency // Cache key builders for consistency
export const cacheKeys = { export const cacheKeys = {
stats: () => 'stats:global', stats: () => 'stats:global',
homeStats: () => 'page:home:stats',
homeFeatured: () => 'page:home:featured',
categories: () => 'categories:all', categories: () => 'categories:all',
categoriesHierarchical: () => 'categories:hierarchical',
featuredSkills: () => 'skills:featured', featuredSkills: () => 'skills:featured',
featuredPage: (page: number) => `page:featured:${page}`,
recentSkills: () => 'skills:recent', recentSkills: () => 'skills:recent',
newSkills: (tab: string, page: number) => `page:new:${tab}:${page}`,
newSkillsCounts: () => 'page:new:counts',
ownerStats: (username: string) => `page:owner:${username}:stats`,
ownerRepos: (username: string) => `page:owner:${username}:repos`,
skillDetail: (id: string) => `page:skill:${id.replace(/\//g, ':')}`,
skillRatings: (id: string, limit: number, offset: number) => `ratings:${id.replace(/\//g, ':')}:${limit}:${offset}`,
pageCount: (page: string) => `page:${page}:count`,
searchSkills: (hash: string) => `skills:search:${hash}`, searchSkills: (hash: string) => `skills:search:${hash}`,
skill: (id: string) => `skill:${id.replace(/\//g, ':')}`, skill: (id: string) => `skill:${id.replace(/\//g, ':')}`,
skillView: (skillId: string, ip: string) => `view:${skillId.replace(/\//g, ':')}:${ip}`, skillView: (skillId: string, ip: string) => `view:${skillId.replace(/\//g, ':')}:${ip}`,
skillDownload: (skillId: string, ip: string) => `download:${skillId.replace(/\//g, ':')}:${ip}`, skillDownload: (skillId: string, ip: string) => `download:${skillId.replace(/\//g, ':')}:${ip}`,
skillReview: (id: string) => `review:skill:${id.replace(/\//g, ':')}`,
reviewStats: () => 'review:stats',
reviewedPage: (sort: string, page: number, minScore = 50) => `page:reviewed:${sort}:s${minScore}:${page}`,
reviewStatsPublic: () => 'page:review-stats:public',
malwarePage: (page: number) => `page:malware:${page}`,
}; };
// TTL values in seconds // TTL values in seconds
@@ -149,8 +183,16 @@ export const cacheTTL = {
recent: 60 * 60, // 1 hour recent: 60 * 60, // 1 hour
search: 30 * 60, // 30 minutes search: 30 * 60, // 30 minutes
skill: 60 * 60, // 1 hour skill: 60 * 60, // 1 hour
newSkills: 30 * 60, // 30 minutes
owner: 30 * 60, // 30 minutes
ratings: 15 * 60, // 15 minutes
pageCount: 60 * 60, // 1 hour
view: 60 * 60, // 1 hour - same IP can only count as 1 view per hour view: 60 * 60, // 1 hour - same IP can only count as 1 view per hour
download: 5 * 60, // 5 minutes - same IP can only count as 1 download per 5 min download: 5 * 60, // 5 minutes - same IP can only count as 1 download per 5 min
reviewStats: 60, // 1 minute - admin review stats
reviewed: 60 * 60, // 1 hour - reviewed skills page
reviewStatsPublic: 30 * 60, // 30 minutes - public review stats page
malware: 60 * 60, // 1 hour - malware listing page
}; };
/** /**

View File

@@ -220,12 +220,12 @@ export function buildWelcomeEmail(locale: Locale, username: string, email?: stri
const dir = getDir(locale); const dir = getDir(locale);
const align = getAlign(locale); const align = getAlign(locale);
// Format skill count: dynamic if provided, fallback to "172,000+" // Format skill count: dynamic if provided, fallback to "16,000+"
const skillCountStr = totalSkillCount const skillCountStr = totalSkillCount
? (locale === 'fa' ? (locale === 'fa'
? totalSkillCount.toLocaleString('fa-IR') ? totalSkillCount.toLocaleString('fa-IR')
: totalSkillCount.toLocaleString('en-US')) : totalSkillCount.toLocaleString('en-US'))
: (locale === 'fa' ? '۱۷۲,۰۰۰' : '172,000'); : (locale === 'fa' ? '۱۶,۰۰۰' : '16,000');
const body = ` const body = `
<h1 style="margin: 0 0 16px 0; color: ${COLORS.text}; font-size: 24px; text-align: ${align};"> <h1 style="margin: 0 0 16px 0; color: ${COLORS.text}; font-size: 24px; text-align: ${align};">

50
apps/web/lib/seo.ts Normal file
View File

@@ -0,0 +1,50 @@
import { locales } from '@/i18n';
export const primaryDomain =
process.env.NEXT_PUBLIC_APP_URL || 'https://skills.palebluedot.live';
const DEFAULT_LOCALE = 'en';
/**
* Get canonical URL path for a locale
* (no prefix for default locale, matching localePrefix: 'as-needed')
*/
export function getCanonicalPath(locale: string, route: string): string {
// Normalize route to always start with a slash and not end with a slash
let formattedRoute = route;
if (!formattedRoute.startsWith('/')) {
formattedRoute = `/${formattedRoute}`;
}
if (formattedRoute.length > 1 && formattedRoute.endsWith('/')) {
formattedRoute = formattedRoute.slice(0, -1);
}
if (locale === DEFAULT_LOCALE) {
if (formattedRoute === '/') return '';
return formattedRoute;
}
if (formattedRoute === '/') return `/${locale}`;
return `/${locale}${formattedRoute}`;
}
/**
* Generate Next.js alternates metadata for a specific route
* @param locale Current active locale
* @param route The route path (e.g. '/about', '/categories')
*/
export function getPageAlternates(locale: string, route: string) {
// Ensure we fall back to / if it's empty
const canonicalPath = getCanonicalPath(locale, route) || '/';
const languages: Record<string, string> = {};
for (const l of locales) {
languages[l] = `${primaryDomain}${getCanonicalPath(l, route) || '/'}`;
}
return {
canonical: `${primaryDomain}${canonicalPath}`,
languages,
};
}

View File

@@ -31,9 +31,10 @@
"browse": "Browse", "browse": "Browse",
"categories": "Categories", "categories": "Categories",
"docs": "Docs", "docs": "Docs",
"reviewed": "Reviewed",
"about": "About", "about": "About",
"github": "GitHub", "github": "GitHub",
"sourceCode": "Source Code (Coming Soon)", "sourceCode": "Source Code",
"sourceCodeShort": "Source Code", "sourceCodeShort": "Source Code",
"switchLanguage": "Switch to Farsi" "switchLanguage": "Switch to Farsi"
}, },
@@ -47,10 +48,11 @@
"ctaSecondary": "Getting Started" "ctaSecondary": "Getting Started"
}, },
"stats": { "stats": {
"skills": "Skills", "skills": "Curated Skills",
"downloads": "Downloads", "downloads": "Downloads",
"contributors": "Contributors", "contributors": "Contributors",
"categories": "Categories" "reviewed": "AI Reviewed",
"curationNote": "Every skill is deduplicated and quality-filtered from {totalIndexed}+ indexed repositories"
}, },
"featured": { "featured": {
"title": "Featured Skills", "title": "Featured Skills",
@@ -86,11 +88,13 @@
"allCategories": "All Categories", "allCategories": "All Categories",
"sort": "Sort by", "sort": "Sort by",
"sortOptions": { "sortOptions": {
"recommended": "Recommended",
"lastDownloaded": "Recently downloaded", "lastDownloaded": "Recently downloaded",
"stars": "Most stars", "stars": "Most stars",
"downloads": "Most downloads", "downloads": "Most downloads",
"recent": "Recently updated", "recent": "Recently updated",
"rating": "Highest rated" "rating": "Highest rated",
"aiScore": "AI Review Score"
}, },
"minStars": "Min stars", "minStars": "Min stars",
"minSecurity": "Min security score", "minSecurity": "Min security score",
@@ -177,6 +181,21 @@
"homepage": "Homepage", "homepage": "Homepage",
"lastUpdate": "Last updated", "lastUpdate": "Last updated",
"createdAt": "Created" "createdAt": "Created"
},
"review": {
"title": "AI Review",
"aiScore": "AI Score",
"outOf100": "out of 100",
"instructionQuality": "Instruction Quality",
"descriptionPrecision": "Description Precision",
"usefulness": "Usefulness",
"technicalSoundness": "Technical Soundness",
"audience": "Audience",
"maturity": "Maturity",
"complexity": "Complexity",
"useCases": "Use Cases",
"reviewedBy": "Reviewed by {reviewer}",
"reviewedOn": "on {date}"
} }
}, },
"footer": { "footer": {
@@ -201,9 +220,10 @@
"privacyPolicy": "Privacy Policy", "privacyPolicy": "Privacy Policy",
"termsOfService": "Terms of Service", "termsOfService": "Terms of Service",
"attribution": "Attribution", "attribution": "Attribution",
"manageSkills": "Manage Skills" "manageSkills": "Manage Skills",
"reviewedSkills": "Reviewed Skills"
}, },
"sourceCode": "Source Code (Coming Soon)", "sourceCode": "Source Code",
"copyright": "© {year} SkillHub. Open source under MIT license." "copyright": "© {year} SkillHub. Open source under MIT license."
}, },
"about": { "about": {
@@ -445,7 +465,7 @@
"platformWindsurf": "Windsurf — Windsurf config", "platformWindsurf": "Windsurf — Windsurf config",
"fullDocsLink": "See the full discovery documentation for detailed prompts, search strategies, and advanced configuration.", "fullDocsLink": "See the full discovery documentation for detailed prompts, search strategies, and advanced configuration.",
"cmdInstallDesc": "Install a skill (--platform, --project, --force, --no-api)", "cmdInstallDesc": "Install a skill (--platform, --project, --force, --no-api)",
"cmdSearchDesc": "Search for skills (--platform, --limit, --page, --sort stars|downloads|rating|recent)", "cmdSearchDesc": "Search for skills (--platform, --limit, --page, --sort recommended|aiScore|downloads|stars|rating|recent)",
"cmdListDesc": "List installed skills (--platform, --project, --all)", "cmdListDesc": "List installed skills (--platform, --project, --all)",
"cmdUpdateDesc": "Update a skill or use --all to update all", "cmdUpdateDesc": "Update a skill or use --all to update all",
"cmdUninstallDesc": "Remove an installed skill (--platform, --project)", "cmdUninstallDesc": "Remove an installed skill (--platform, --project)",
@@ -453,7 +473,7 @@
"exInstallGlobal": "Install globally:", "exInstallGlobal": "Install globally:",
"exInstallProject": "Install in current project:", "exInstallProject": "Install in current project:",
"exSearch": "Search for skills:", "exSearch": "Search for skills:",
"exSearchSort": "Search sorted by stars:", "exSearchSort": "Search sorted by AI review score:",
"exUpdateAll": "Update all installed skills:", "exUpdateAll": "Update all installed skills:",
"platformsTitle": "Supported Platforms", "platformsTitle": "Supported Platforms",
"platformsDesc": "SkillHub CLI supports 5 AI coding platforms:", "platformsDesc": "SkillHub CLI supports 5 AI coding platforms:",
@@ -537,6 +557,59 @@
"title": "Featured Skills", "title": "Featured Skills",
"subtitle": "Top skills by popularity and community engagement" "subtitle": "Top skills by popularity and community engagement"
}, },
"reviewed": {
"title": "AI Reviewed Skills",
"subtitle": "Skills evaluated and scored by AI for quality and usefulness",
"explanation": "Each skill has been evaluated by an AI reviewer on a structured rubric: instruction quality, usefulness, technical soundness, and description precision. Scores range from 0 to 100 — skills scoring 75+ (green badge) are high quality, 5074 (gold badge) are solid with targeted value.",
"sortBy": "Sort by",
"sortOptions": {
"reviewDate": "Recently reviewed",
"aiScore": "Highest score"
},
"scoreFilter": "Score",
"scoreOptions": {
"above50": "50+ only",
"all": "All scores"
}
},
"malicious": {
"warningTitle": "Malware Detected",
"warningDescription": "This skill has been flagged as malicious. It contains obfuscated code designed to download and execute harmful payloads. File downloads and installation are blocked.",
"installBlocked": "Installation blocked — this skill contains malware",
"badge": "Malware"
},
"reviewStats": {
"title": "Review Statistics",
"subtitle": "Overview of the AI review pipeline, score distribution, and security status",
"pipelineTitle": "Review Pipeline",
"pipelineDescription": "Status of skills in the review process",
"aiReviewed": "AI Reviewed",
"needsReReview": "Needs Re-review",
"totalReviews": "Total Review Records",
"scoreTitle": "Score Distribution",
"scoreDescription": "Distribution of AI review scores across reviewed skills",
"scoreHigh": "High Quality (75100)",
"scoreMid": "Solid (5074)",
"scoreLow": "Below Average (049)",
"securityTitle": "Security Scan",
"securityDescription": "Automated security scan results for all skills",
"securityPass": "Pass",
"securityWarning": "Warning",
"securityFail": "Fail",
"malwareTitle": "Malware Detection",
"malwareDescription": "Skills flagged as containing malicious code",
"malwareFlagged": "Flagged as Malware",
"malwareViewAll": "View flagged skills",
"skills": "skills"
},
"malwarePage": {
"title": "Flagged Malware Skills",
"subtitle": "These skills have been identified as containing malicious code. Downloads and installation are blocked for your safety.",
"noResults": "No malware-flagged skills found.",
"flaggedOn": "Flagged on",
"downloadBlocked": "Downloads blocked",
"installBlocked": "Installation blocked"
},
"owner": { "owner": {
"title": "@{username}'s Skills", "title": "@{username}'s Skills",
"notFound": "Owner not found", "notFound": "Owner not found",
@@ -544,6 +617,8 @@
"browseAll": "Browse all skills", "browseAll": "Browse all skills",
"claimCta": "Is this your repository?", "claimCta": "Is this your repository?",
"claimButton": "Claim ownership", "claimButton": "Claim ownership",
"ownerManageCta": "You own these skills.",
"ownerManageButton": "Manage / Remove Skills",
"installCta": "Install the most popular skill from this owner:", "installCta": "Install the most popular skill from this owner:",
"stats": { "stats": {
"skills": "{count, plural, one {# skill} other {# skills}}", "skills": "{count, plural, one {# skill} other {# skills}}",
@@ -695,7 +770,8 @@
}, },
"tabs": { "tabs": {
"remove": "Remove Skill", "remove": "Remove Skill",
"add": "Add Skill" "add": "Add Skill",
"removeRepo": "Remove Repository"
}, },
"form": { "form": {
"skillId": "Skill ID", "skillId": "Skill ID",
@@ -718,6 +794,8 @@
"success": { "success": {
"title": "Skill Blocked", "title": "Skill Blocked",
"description": "Your skill has been blocked from indexing. It will no longer appear in SkillHub.", "description": "Your skill has been blocked from indexing. It will no longer appear in SkillHub.",
"pendingTitle": "Request Submitted",
"pendingDescription": "The repository was not found on GitHub. Your removal request has been submitted for admin review.",
"viewRequests": "View My Requests" "viewRequests": "View My Requests"
}, },
"addSuccess": { "addSuccess": {
@@ -729,7 +807,9 @@
"viewRequests": "View My Requests", "viewRequests": "View My Requests",
"foundSkillsIn": "Found skills in:", "foundSkillsIn": "Found skills in:",
"root": "(root)", "root": "(root)",
"andMore": "... and {count} more" "andMore": "... and {count} more",
"reEnabledTitle": "Repository Re-enabled",
"reEnabledDescription": "Your repository has been unblocked and will appear again after the next scan."
}, },
"error": { "error": {
"notOwner": "You are not the owner or admin of this repository. Only repository owners can request removal.", "notOwner": "You are not the owner or admin of this repository. Only repository owners can request removal.",
@@ -741,7 +821,8 @@
"invalidRepo": "The repository was not found or is not accessible. Please check the URL and ensure the repository is public.", "invalidRepo": "The repository was not found or is not accessible. Please check the URL and ensure the repository is public.",
"rateLimitExceeded": "GitHub API rate limit exceeded. Please try again in a few minutes.", "rateLimitExceeded": "GitHub API rate limit exceeded. Please try again in a few minutes.",
"networkTimeout": "Request timed out while checking the repository. Please try again.", "networkTimeout": "Request timed out while checking the repository. Please try again.",
"generic": "An error occurred. Please try again." "generic": "An error occurred. Please try again.",
"repoBlockedByOwner": "This repository was removed by its owner and cannot be re-added."
}, },
"myRequests": { "myRequests": {
"title": "My Requests", "title": "My Requests",
@@ -757,6 +838,25 @@
"showLess": "Show less", "showLess": "Show less",
"showAllPrefix": "Show all ", "showAllPrefix": "Show all ",
"showAllSuffix": " paths" "showAllSuffix": " paths"
},
"removeRepoForm": {
"repoUrl": "GitHub Repository URL or owner/repo",
"repoUrlPlaceholder": "https://github.com/owner/repo",
"repoUrlHelp": "All skills from this repository will be removed from SkillHub and it will not be re-indexed.",
"reason": "Reason for removal",
"reasonPlaceholder": "Why are you removing this repository?",
"submit": "Remove All Skills",
"submitting": "Removing..."
},
"removeRepoSuccess": {
"title": "Repository Removed",
"description": "All {count} skills from {repo} have been removed from SkillHub.",
"descriptionZero": "No active skills were found for this repository. It has been blocked from re-indexing."
},
"removeRepoError": {
"notOwner": "You are not the owner of this repository.",
"invalidRepo": "Repository not found on GitHub.",
"parseError": "Invalid format. Use https://github.com/owner/repo or owner/repo."
} }
}, },
"attribution": { "attribution": {
@@ -872,6 +972,12 @@
"feedback": "Feedback", "feedback": "Feedback",
"dismiss": "Dismiss" "dismiss": "Dismiss"
}, },
"securityAlert": {
"label": "Security Alert:",
"text": "Malware found in 5 skills from openclaw/skills.",
"link": "Read the advisory",
"dismiss": "Dismiss"
},
"email": { "email": {
"welcome": { "welcome": {
"subject": "Welcome to SkillHub!", "subject": "Welcome to SkillHub!",

View File

@@ -31,9 +31,10 @@
"browse": "مرور", "browse": "مرور",
"categories": "دسته‌بندی‌ها", "categories": "دسته‌بندی‌ها",
"docs": "مستندات", "docs": "مستندات",
"reviewed": "بررسی‌شده",
"about": "درباره ما", "about": "درباره ما",
"github": "گیت‌هاب", "github": "گیت‌هاب",
"sourceCode": "کد منبع (به زودی)", "sourceCode": "کد منبع",
"sourceCodeShort": "کد منبع", "sourceCodeShort": "کد منبع",
"switchLanguage": "تغییر به انگلیسی" "switchLanguage": "تغییر به انگلیسی"
}, },
@@ -47,10 +48,11 @@
"ctaSecondary": "راهنمای شروع" "ctaSecondary": "راهنمای شروع"
}, },
"stats": { "stats": {
"skills": "مهارت", "skills": "مهارت برگزیده",
"downloads": "دانلود", "downloads": "دانلود",
"contributors": "مشارکت‌کننده", "contributors": "مشارکت‌کننده",
"categories": "دسته‌بندی" "reviewed": "بررسی‌شده با هوش مصنوعی",
"curationNote": "هر مهارت از میان {totalIndexed}+ مخزن ایندکس‌شده، حذف تکراری و فیلتر کیفیت شده است"
}, },
"featured": { "featured": {
"title": "مهارت‌های ویژه", "title": "مهارت‌های ویژه",
@@ -86,11 +88,13 @@
"allCategories": "همه دسته‌بندی‌ها", "allCategories": "همه دسته‌بندی‌ها",
"sort": "مرتب‌سازی", "sort": "مرتب‌سازی",
"sortOptions": { "sortOptions": {
"recommended": "پیشنهادی",
"lastDownloaded": "اخیراً دانلود شده", "lastDownloaded": "اخیراً دانلود شده",
"stars": "بیشترین ستاره", "stars": "بیشترین ستاره",
"downloads": "بیشترین دانلود", "downloads": "بیشترین دانلود",
"recent": "جدید‌ترین", "recent": "جدید‌ترین",
"rating": "بالاترین امتیاز" "rating": "بالاترین امتیاز",
"aiScore": "امتیاز بررسی هوش مصنوعی"
}, },
"minStars": "حداقل ستاره", "minStars": "حداقل ستاره",
"minSecurity": "حداقل امتیاز امنیتی", "minSecurity": "حداقل امتیاز امنیتی",
@@ -177,6 +181,21 @@
"homepage": "وب‌سایت", "homepage": "وب‌سایت",
"lastUpdate": "آخرین به‌روزرسانی", "lastUpdate": "آخرین به‌روزرسانی",
"createdAt": "تاریخ ایجاد" "createdAt": "تاریخ ایجاد"
},
"review": {
"title": "بررسی هوش مصنوعی",
"aiScore": "امتیاز AI",
"outOf100": "از ۱۰۰",
"instructionQuality": "کیفیت دستورالعمل",
"descriptionPrecision": "دقت توضیحات",
"usefulness": "کاربردی بودن",
"technicalSoundness": "صحت فنی",
"audience": "مخاطبان",
"maturity": "بلوغ",
"complexity": "پیچیدگی",
"useCases": "موارد استفاده",
"reviewedBy": "بررسی توسط {reviewer}",
"reviewedOn": "در {date}"
} }
}, },
"footer": { "footer": {
@@ -201,9 +220,10 @@
"privacyPolicy": "حریم خصوصی", "privacyPolicy": "حریم خصوصی",
"termsOfService": "شرایط استفاده", "termsOfService": "شرایط استفاده",
"attribution": "تقدیر و تشکر", "attribution": "تقدیر و تشکر",
"manageSkills": "مدیریت مهارت‌ها" "manageSkills": "مدیریت مهارت‌ها",
"reviewedSkills": "مهارت‌های بررسی‌شده"
}, },
"sourceCode": "کد منبع (به زودی)", "sourceCode": "کد منبع",
"copyright": "© {year} SkillHub. کد متن‌باز تحت مجوز MIT." "copyright": "© {year} SkillHub. کد متن‌باز تحت مجوز MIT."
}, },
"about": { "about": {
@@ -445,7 +465,7 @@
"platformWindsurf": "Windsurf — پیکربندی Windsurf", "platformWindsurf": "Windsurf — پیکربندی Windsurf",
"fullDocsLink": "مستندات کامل کشف را برای پرامپت‌های دقیق، استراتژی‌های جستجو و پیکربندی پیشرفته ببینید.", "fullDocsLink": "مستندات کامل کشف را برای پرامپت‌های دقیق، استراتژی‌های جستجو و پیکربندی پیشرفته ببینید.",
"cmdInstallDesc": "نصب مهارت (--platform, --project, --force, --no-api)", "cmdInstallDesc": "نصب مهارت (--platform, --project, --force, --no-api)",
"cmdSearchDesc": "جستجوی مهارت‌ها (--platform, --limit, --page, --sort stars|downloads|rating|recent)", "cmdSearchDesc": "جستجوی مهارت‌ها (--platform, --limit, --page, --sort recommended|aiScore|downloads|stars|rating|recent)",
"cmdListDesc": "لیست مهارت‌های نصب‌شده (--platform, --project, --all)", "cmdListDesc": "لیست مهارت‌های نصب‌شده (--platform, --project, --all)",
"cmdUpdateDesc": "به‌روزرسانی مهارت یا استفاده از --all برای به‌روزرسانی همه", "cmdUpdateDesc": "به‌روزرسانی مهارت یا استفاده از --all برای به‌روزرسانی همه",
"cmdUninstallDesc": "حذف مهارت نصب‌شده (--platform, --project)", "cmdUninstallDesc": "حذف مهارت نصب‌شده (--platform, --project)",
@@ -453,7 +473,7 @@
"exInstallGlobal": "نصب سراسری:", "exInstallGlobal": "نصب سراسری:",
"exInstallProject": "نصب در پروژه فعلی:", "exInstallProject": "نصب در پروژه فعلی:",
"exSearch": "جستجوی مهارت‌ها:", "exSearch": "جستجوی مهارت‌ها:",
"exSearchSort": "جستجو بر اساس ستاره‌ها:", "exSearchSort": "جستجو بر اساس امتیاز بررسی:",
"exUpdateAll": "به‌روزرسانی همه مهارت‌ها:", "exUpdateAll": "به‌روزرسانی همه مهارت‌ها:",
"platformsTitle": "پلتفرم‌های پشتیبانی‌شده", "platformsTitle": "پلتفرم‌های پشتیبانی‌شده",
"platformsDesc": "CLI SkillHub از ۵ پلتفرم کدنویسی هوش مصنوعی پشتیبانی می‌کند:", "platformsDesc": "CLI SkillHub از ۵ پلتفرم کدنویسی هوش مصنوعی پشتیبانی می‌کند:",
@@ -537,6 +557,59 @@
"title": "مهارت‌های ویژه", "title": "مهارت‌های ویژه",
"subtitle": "برترین مهارت‌ها بر اساس محبوبیت و مشارکت جامعه" "subtitle": "برترین مهارت‌ها بر اساس محبوبیت و مشارکت جامعه"
}, },
"reviewed": {
"title": "مهارت‌های بررسی‌شده توسط هوش مصنوعی",
"subtitle": "مهارت‌هایی که توسط هوش مصنوعی ارزیابی و امتیازدهی شده‌اند",
"explanation": "هر مهارت در این فهرست توسط یک ارزیاب هوش مصنوعی بر اساس معیارهای ساختاریافته شامل کیفیت دستورالعمل، کاربرد، درستی فنی و دقت توضیحات بررسی شده است. امتیازها از ۰ تا ۱۰۰ هستند — مهارت‌های با امتیاز ۷۵+ (نشان سبز) باکیفیت بالا، ۵۰ تا ۷۴ (نشان طلایی) مستحکم با ارزش هدفمند هستند.",
"sortBy": "مرتب‌سازی",
"sortOptions": {
"reviewDate": "آخرین بررسی‌شده",
"aiScore": "بالاترین امتیاز"
},
"scoreFilter": "امتیاز",
"scoreOptions": {
"above50": "فقط ۵۰+",
"all": "همه امتیازها"
}
},
"malicious": {
"warningTitle": "بدافزار شناسایی شد",
"warningDescription": "این مهارت به عنوان مخرب شناسایی شده است. شامل کد مبهم‌سازی شده برای دانلود و اجرای بارهای مضر است. دانلود فایل و نصب مسدود شده است.",
"installBlocked": "نصب مسدود شده — این مهارت حاوی بدافزار است",
"badge": "بدافزار"
},
"reviewStats": {
"title": "آمار بررسی‌ها",
"subtitle": "نمای کلی از خط لوله بررسی هوش مصنوعی، توزیع امتیاز و وضعیت امنیتی",
"pipelineTitle": "خط لوله بررسی",
"pipelineDescription": "وضعیت مهارت‌ها در فرآیند بررسی",
"aiReviewed": "بررسی‌شده توسط AI",
"needsReReview": "نیاز به بررسی مجدد",
"totalReviews": "کل رکوردهای بررسی",
"scoreTitle": "توزیع امتیاز",
"scoreDescription": "توزیع امتیازهای بررسی هوش مصنوعی در مهارت‌های بررسی‌شده",
"scoreHigh": "کیفیت بالا (۷۵۱۰۰)",
"scoreMid": "قابل قبول (۵۰–۷۴)",
"scoreLow": "زیر متوسط (۰–۴۹)",
"securityTitle": "اسکن امنیتی",
"securityDescription": "نتایج اسکن امنیتی خودکار برای تمام مهارت‌ها",
"securityPass": "تایید",
"securityWarning": "هشدار",
"securityFail": "رد",
"malwareTitle": "شناسایی بدافزار",
"malwareDescription": "مهارت‌هایی که حاوی کد مخرب شناسایی شده‌اند",
"malwareFlagged": "فلگ شده به عنوان بدافزار",
"malwareViewAll": "مشاهده مهارت‌های فلگ‌شده",
"skills": "مهارت"
},
"malwarePage": {
"title": "مهارت‌های فلگ‌شده به عنوان بدافزار",
"subtitle": "این مهارت‌ها حاوی کد مخرب شناسایی شده‌اند. دانلود و نصب آن‌ها برای امنیت شما مسدود شده است.",
"noResults": "هیچ مهارت بدافزاردار فلگ‌شده‌ای یافت نشد.",
"flaggedOn": "فلگ شده در",
"downloadBlocked": "دانلود مسدود شده",
"installBlocked": "نصب مسدود شده"
},
"owner": { "owner": {
"title": "مهارت‌های @{username}", "title": "مهارت‌های @{username}",
"notFound": "مالک یافت نشد", "notFound": "مالک یافت نشد",
@@ -544,6 +617,8 @@
"browseAll": "مرور همه مهارت‌ها", "browseAll": "مرور همه مهارت‌ها",
"claimCta": "آیا این مخزن شماست؟", "claimCta": "آیا این مخزن شماست؟",
"claimButton": "ثبت مالکیت", "claimButton": "ثبت مالکیت",
"ownerManageCta": "این مهارت‌ها متعلق به شما هستند.",
"ownerManageButton": "مدیریت / حذف مهارت‌ها",
"installCta": "نصب محبوب‌ترین مهارت از این مالک:", "installCta": "نصب محبوب‌ترین مهارت از این مالک:",
"stats": { "stats": {
"skills": "{count, plural, one {# مهارت} other {# مهارت}}", "skills": "{count, plural, one {# مهارت} other {# مهارت}}",
@@ -695,7 +770,8 @@
}, },
"tabs": { "tabs": {
"remove": "حذف مهارت", "remove": "حذف مهارت",
"add": "افزودن مهارت" "add": "افزودن مهارت",
"removeRepo": "حذف مخزن"
}, },
"form": { "form": {
"skillId": "شناسه مهارت", "skillId": "شناسه مهارت",
@@ -718,6 +794,8 @@
"success": { "success": {
"title": "مهارت مسدود شد", "title": "مهارت مسدود شد",
"description": "مهارت شما از ایندکس شدن مسدود شد. دیگر در SkillHub نمایش داده نخواهد شد.", "description": "مهارت شما از ایندکس شدن مسدود شد. دیگر در SkillHub نمایش داده نخواهد شد.",
"pendingTitle": "درخواست ثبت شد",
"pendingDescription": "مخزن در GitHub یافت نشد. درخواست حذف شما برای بررسی ادمین ثبت شد.",
"viewRequests": "مشاهده درخواست‌های من" "viewRequests": "مشاهده درخواست‌های من"
}, },
"addSuccess": { "addSuccess": {
@@ -729,7 +807,9 @@
"viewRequests": "مشاهده درخواست‌ها", "viewRequests": "مشاهده درخواست‌ها",
"foundSkillsIn": "مهارت‌ها در این مسیرها یافت شدند:", "foundSkillsIn": "مهارت‌ها در این مسیرها یافت شدند:",
"root": "(root)", "root": "(root)",
"andMore": "... و {count} مورد دیگر" "andMore": "... و {count} مورد دیگر",
"reEnabledTitle": "مخزن فعال‌سازی مجدد شد",
"reEnabledDescription": "مخزن شما رفع انسداد شد و پس از اسکن بعدی دوباره نمایش داده خواهد شد."
}, },
"error": { "error": {
"notOwner": "شما مالک یا ادمین این مخزن نیستید. فقط مالکان مخزن می‌توانند درخواست حذف دهند.", "notOwner": "شما مالک یا ادمین این مخزن نیستید. فقط مالکان مخزن می‌توانند درخواست حذف دهند.",
@@ -741,7 +821,8 @@
"invalidRepo": "مخزن یافت نشد یا قابل دسترسی نیست. لطفاً URL را بررسی کنید و اطمینان حاصل کنید که مخزن عمومی است.", "invalidRepo": "مخزن یافت نشد یا قابل دسترسی نیست. لطفاً URL را بررسی کنید و اطمینان حاصل کنید که مخزن عمومی است.",
"rateLimitExceeded": "محدودیت API GitHub تمام شده است. لطفاً چند دقیقه بعد دوباره تلاش کنید.", "rateLimitExceeded": "محدودیت API GitHub تمام شده است. لطفاً چند دقیقه بعد دوباره تلاش کنید.",
"networkTimeout": "هنگام بررسی مخزن زمان اتصال تمام شد. لطفاً دوباره تلاش کنید.", "networkTimeout": "هنگام بررسی مخزن زمان اتصال تمام شد. لطفاً دوباره تلاش کنید.",
"generic": "خطایی رخ داد. لطفاً دوباره تلاش کنید." "generic": "خطایی رخ داد. لطفاً دوباره تلاش کنید.",
"repoBlockedByOwner": "این مخزن توسط مالکش حذف شده و قابل افزودن مجدد نیست."
}, },
"myRequests": { "myRequests": {
"title": "درخواست‌های من", "title": "درخواست‌های من",
@@ -757,6 +838,25 @@
"showLess": "نمایش کمتر", "showLess": "نمایش کمتر",
"showAllPrefix": "نمایش همه ", "showAllPrefix": "نمایش همه ",
"showAllSuffix": " مسیر" "showAllSuffix": " مسیر"
},
"removeRepoForm": {
"repoUrl": "آدرس مخزن گیت‌هاب یا owner/repo",
"repoUrlPlaceholder": "https://github.com/owner/repo",
"repoUrlHelp": "تمام مهارت‌های این مخزن از SkillHub حذف شده و دیگر ایندکس نخواهد شد.",
"reason": "دلیل حذف",
"reasonPlaceholder": "چرا این مخزن را حذف می‌کنید؟",
"submit": "حذف تمام مهارت‌ها",
"submitting": "در حال حذف..."
},
"removeRepoSuccess": {
"title": "مخزن حذف شد",
"description": "تمام {count} مهارت از {repo} از SkillHub حذف شدند.",
"descriptionZero": "هیچ مهارت فعالی برای این مخزن یافت نشد. مخزن از ایندکس‌سازی مجدد مسدود شده است."
},
"removeRepoError": {
"notOwner": "شما مالک این مخزن نیستید.",
"invalidRepo": "مخزن در گیت‌هاب یافت نشد.",
"parseError": "فرمت نادرست. از https://github.com/owner/repo یا owner/repo استفاده کنید."
} }
}, },
"attribution": { "attribution": {
@@ -872,6 +972,12 @@
"feedback": "بازخورد", "feedback": "بازخورد",
"dismiss": "بستن" "dismiss": "بستن"
}, },
"securityAlert": {
"label": "هشدار امنیتی:",
"text": "بدافزار در ۵ مهارت از openclaw/skills شناسایی شد.",
"link": "مشاهده اطلاعیه",
"dismiss": "بستن"
},
"email": { "email": {
"welcome": { "welcome": {
"subject": "به SkillHub خوش آمدید!", "subject": "به SkillHub خوش آمدید!",

View File

@@ -83,6 +83,9 @@ export const config = {
'/(en|fa)/:path*', '/(en|fa)/:path*',
// Match API routes (for CSRF protection) // Match API routes (for CSRF protection)
'/api/:path*', '/api/:path*',
// Match skill paths explicitly (repo names like "next.js" contain dots
// which would be excluded by the catch-all pattern below)
'/skill/:path*',
// Match other paths (exclude static files and Next.js internals) // Match other paths (exclude static files and Next.js internals)
'/((?!_next|_vercel|.*\\..*).*)', '/((?!_next|_vercel|.*\\..*).*)',
], ],

View File

@@ -27,7 +27,7 @@
"next": "^15.0.0", "next": "^15.0.0",
"next-auth": "5.0.0-beta.30", "next-auth": "5.0.0-beta.30",
"next-intl": "^3.4.0", "next-intl": "^3.4.0",
"next-themes": "^0.4.4", "next-themes": "^0.4.4",
"react": "^18.2.0", "react": "^18.2.0",
"react-dom": "^18.2.0", "react-dom": "^18.2.0",
"resend": "^6.9.1", "resend": "^6.9.1",

View File

@@ -30,12 +30,36 @@ Sentry.init({
replaysOnErrorSampleRate: 1.0, // 100% of sessions with errors replaysOnErrorSampleRate: 1.0, // 100% of sessions with errors
// Filter out common non-actionable errors // Filter out common non-actionable errors
beforeSend(event, hint) { ignoreErrors: [
// Ignore ResizeObserver errors (browser quirk, not actionable) // AbortError during navigation — Next.js RSC prefetch/streaming cancelled mid-flight
if (event.exception?.values?.[0]?.value?.includes("ResizeObserver")) { "AbortError",
return null; // Network stream errors (Firefox)
} "Error in input stream",
// Network fetch failures (Safari / mobile)
"Load failed",
// Generic network errors (client connectivity issues)
"network error",
// Browser extension noise (translation/accessibility extensions)
/Object Not Found Matching Id/,
// ResizeObserver loop limit — browser quirk, not actionable
"ResizeObserver",
// DOM manipulation by browser extensions (translate, font, etc.) conflicts with React
"NotFoundError: Failed to execute 'removeChild' on 'Node'",
"NotFoundError: Failed to execute 'insertBefore' on 'Node'",
// Browser extension internal errors (crypto wallets, etc.)
/func .* not found/,
],
// Ignore errors originating from browser extensions or injected scripts
denyUrls: [
/extensions\//i,
/^chrome:\/\//i,
/^chrome-extension:\/\//i,
/^moz-extension:\/\//i,
/inpage\.js/,
],
beforeSend(event, hint) {
// Log to console in development for debugging // Log to console in development for debugging
if (process.env.NODE_ENV === "development") { if (process.env.NODE_ENV === "development") {
console.error("[Sentry]", hint.originalException || hint.syntheticException); console.error("[Sentry]", hint.originalException || hint.syntheticException);

View File

@@ -34,6 +34,10 @@ export {
// Validator // Validator
export { validateSkill, isValidSkill, formatValidationSummary } from './validator.js'; export { validateSkill, isValidSkill, formatValidationSummary } from './validator.js';
// Review Notes Parser
export type { ParsedReviewNotes } from './review-notes-parser.js';
export { parseReviewNotes } from './review-notes-parser.js';
// Security Scanner // Security Scanner
export { export {
scanSecurity, scanSecurity,

View File

@@ -0,0 +1,141 @@
/**
* Parser for structured review_notes from the AI review pipeline.
*
* Review notes use tagged sections like:
* RATIONALE: ...
* USE-CASES: csv-cleaning, pdf-generation
* AUDIENCE: data-analysts
* MATURITY: production
* COMPLEXITY: complex
* DEPENDENCIES: node, bash
* PLATFORM: cross-platform
* COMPLEMENTS: context-manager
* SEO-EN: One-line summary
* BUNDLE-FIT: productivity
* FRAMEWORK-LOCK: none
* CONTRIBUTING-REPO: none
*/
export interface ParsedReviewNotes {
/** Free-text summary before the first structured tag */
summary: string;
rationale: string | null;
useCases: string[];
audience: string[];
complements: string[];
seoEn: string | null;
bundleFit: string | null;
frameworkLock: string | null;
contributingRepo: string | null;
maturity: 'prototype' | 'beta' | 'production' | null;
complexity: 'simple' | 'moderate' | 'complex' | null;
dependencies: string[];
platform: string | null;
}
const TAG_KEYS = [
'RATIONALE',
'USE-CASES',
'AUDIENCE',
'COMPLEMENTS',
'SEO-EN',
'BUNDLE-FIT',
'FRAMEWORK-LOCK',
'CONTRIBUTING-REPO',
'MATURITY',
'COMPLEXITY',
'DEPENDENCIES',
'PLATFORM',
] as const;
// Build regex that matches any tag at the start of a segment
const TAG_REGEX = new RegExp(`(${TAG_KEYS.join('|')}):\\s*`, 'g');
function splitComma(val: string): string[] {
return val
.split(',')
.map((s) => s.trim())
.filter((s) => s.length > 0);
}
function normalizeNone(val: string): string | null {
const lower = val.trim().toLowerCase();
return lower === 'none' || lower === 'n/a' || lower === '' ? null : val.trim();
}
export function parseReviewNotes(notes: string | null | undefined): ParsedReviewNotes {
const empty: ParsedReviewNotes = {
summary: '',
rationale: null,
useCases: [],
audience: [],
complements: [],
seoEn: null,
bundleFit: null,
frameworkLock: null,
contributingRepo: null,
maturity: null,
complexity: null,
dependencies: [],
platform: null,
};
if (!notes) return empty;
// Normalize escaped newlines to real newlines
const normalized = notes.replace(/\\n/g, '\n');
// Extract tags and their values
const tags: Record<string, string> = {};
let summary = '';
// Find the first tag position
TAG_REGEX.lastIndex = 0;
const firstMatch = TAG_REGEX.exec(normalized);
const firstTagPos = firstMatch ? firstMatch.index : normalized.length;
// Everything before the first tag is the summary
summary = normalized.slice(0, firstTagPos).trim();
// Parse all tag: value pairs
const tagMatches: Array<{ tag: string; start: number; valueStart: number }> = [];
TAG_REGEX.lastIndex = 0;
let match;
while ((match = TAG_REGEX.exec(normalized)) !== null) {
tagMatches.push({
tag: match[1],
start: match.index,
valueStart: match.index + match[0].length,
});
}
for (let i = 0; i < tagMatches.length; i++) {
const current = tagMatches[i];
const nextStart = i + 1 < tagMatches.length ? tagMatches[i + 1].start : normalized.length;
const value = normalized.slice(current.valueStart, nextStart).trim();
tags[current.tag] = value;
}
const maturityVal = tags['MATURITY']?.trim().toLowerCase();
const complexityVal = tags['COMPLEXITY']?.trim().toLowerCase();
return {
summary,
rationale: tags['RATIONALE']?.trim() || null,
useCases: tags['USE-CASES'] ? splitComma(tags['USE-CASES']) : [],
audience: tags['AUDIENCE'] ? splitComma(tags['AUDIENCE']) : [],
complements: tags['COMPLEMENTS'] ? splitComma(tags['COMPLEMENTS']) : [],
seoEn: normalizeNone(tags['SEO-EN'] || ''),
bundleFit: normalizeNone(tags['BUNDLE-FIT'] || ''),
frameworkLock: normalizeNone(tags['FRAMEWORK-LOCK'] || ''),
contributingRepo: normalizeNone(tags['CONTRIBUTING-REPO'] || ''),
maturity: maturityVal === 'prototype' || maturityVal === 'beta' || maturityVal === 'production'
? maturityVal
: null,
complexity: complexityVal === 'simple' || complexityVal === 'moderate' || complexityVal === 'complex'
? complexityVal
: null,
dependencies: tags['DEPENDENCIES'] ? splitComma(tags['DEPENDENCIES']) : [],
platform: normalizeNone(tags['PLATFORM'] || ''),
};
}

View File

@@ -16,6 +16,22 @@ const NAME_PATTERN = /^[a-z0-9][a-z0-9-]*[a-z0-9]$|^[a-z0-9]$/;
const MAX_DESCRIPTION_LENGTH = 1024; const MAX_DESCRIPTION_LENGTH = 1024;
const MAX_NAME_LENGTH = 64; const MAX_NAME_LENGTH = 64;
/**
* Sanitize a string to be valid UTF-8 for JSONB storage.
* Removes incomplete multi-byte sequences and control characters
* that PostgreSQL's jsonb_build_object rejects.
*/
function sanitizeUtf8(input: string): string {
// Remove null bytes and C0 control characters (except \t \n \r)
// eslint-disable-next-line no-control-regex
let result = input.replace(/[\x00-\x08\x0B\x0C\x0E-\x1F]/g, '');
// Encode to buffer and decode back to strip invalid sequences
result = Buffer.from(result, 'utf8').toString('utf8');
// Replace lone surrogates (U+D800-U+DFFF) which are invalid in UTF-8
result = result.replace(/[\uD800-\uDFFF]/g, '');
return result;
}
/** /**
* File patterns for discovering instruction files on GitHub * File patterns for discovering instruction files on GitHub
*/ */
@@ -117,8 +133,8 @@ export function parseSkillMd(content: string): ParsedSkill {
// Build metadata // Build metadata
const metadata: SkillMetadata = { const metadata: SkillMetadata = {
name: String(frontmatter.name || ''), name: sanitizeUtf8(String(frontmatter.name || '')),
description: String(frontmatter.description || ''), description: sanitizeUtf8(String(frontmatter.description || '')),
version: frontmatter.version ? String(frontmatter.version) : undefined, version: frontmatter.version ? String(frontmatter.version) : undefined,
license: frontmatter.license ? String(frontmatter.license) : undefined, license: frontmatter.license ? String(frontmatter.license) : undefined,
author: frontmatter.author ? String(frontmatter.author) : undefined, author: frontmatter.author ? String(frontmatter.author) : undefined,
@@ -387,8 +403,8 @@ export function parseGenericInstructionFile(
const inferredPlatform = pattern?.inferredPlatform || 'claude'; const inferredPlatform = pattern?.inferredPlatform || 'claude';
const metadata: SkillMetadata = { const metadata: SkillMetadata = {
name: derivedName, name: sanitizeUtf8(derivedName),
description: derivedDescription.slice(0, MAX_DESCRIPTION_LENGTH), description: sanitizeUtf8(derivedDescription.slice(0, MAX_DESCRIPTION_LENGTH)),
version: frontmatter.version ? String(frontmatter.version) : undefined, version: frontmatter.version ? String(frontmatter.version) : undefined,
license: frontmatter.license ? String(frontmatter.license) : undefined, license: frontmatter.license ? String(frontmatter.license) : undefined,
author: frontmatter.author ? String(frontmatter.author) : repoMeta.owner, author: frontmatter.author ? String(frontmatter.author) : repoMeta.owner,

View File

@@ -0,0 +1,256 @@
import { describe, it, expect } from 'vitest';
/**
* Threshold-based tie-breaker logic for duplicate detection (T074b).
* These tests validate the decision logic encoded in the SQL query.
*/
const REPO_AGE_THRESHOLD_DAYS = 75;
interface TieBreakCandidate {
id: string;
skillType: 'standalone' | 'collection' | 'aggregator';
repoCreatedAt: Date | null;
githubStars: number;
githubForks: number;
createdAt: Date;
}
/**
* Determine if candidate `a` beats candidate `b` (i.e., `b` should be marked duplicate).
* Returns true if `a` is the canonical (original) skill.
* This mirrors the SQL tie-breaker logic in runPostCrawlCuration().
*/
function isCanonical(a: TieBreakCandidate, b: TieBreakCandidate): boolean {
// Tier 1: standalone/collection beats aggregator
if (a.skillType !== 'aggregator' && b.skillType === 'aggregator') return true;
if (a.skillType === 'aggregator' && b.skillType !== 'aggregator') return false;
// Only compare same-type from here
const sameType = (a.skillType === 'aggregator') === (b.skillType === 'aggregator');
if (!sameType) return false;
// Tier 2: significant age gap (>75 days) → older repo wins
if (a.repoCreatedAt && b.repoCreatedAt) {
const gapMs = Math.abs(a.repoCreatedAt.getTime() - b.repoCreatedAt.getTime());
const gapDays = gapMs / (1000 * 60 * 60 * 24);
if (gapDays > REPO_AGE_THRESHOLD_DAYS) {
return a.repoCreatedAt < b.repoCreatedAt;
}
}
// Tier 3: more stars
if (a.githubStars !== b.githubStars) return a.githubStars > b.githubStars;
// Tier 4: more forks
if (a.githubForks !== b.githubForks) return a.githubForks > b.githubForks;
// Tier 5: older repo_created_at (minor difference)
if (a.repoCreatedAt && b.repoCreatedAt && a.repoCreatedAt.getTime() !== b.repoCreatedAt.getTime()) {
return a.repoCreatedAt < b.repoCreatedAt;
}
// Tier 6: older created_at (SkillHub indexing date)
return a.createdAt < b.createdAt;
}
describe('Duplicate Tie-Breaker Logic', () => {
const baseDate = new Date('2024-01-01');
const daysAgo = (days: number) => new Date(baseDate.getTime() - days * 86400000);
describe('Tier 1: standalone vs aggregator', () => {
it('standalone beats aggregator regardless of stars', () => {
const standalone: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: daysAgo(10),
githubStars: 5, githubForks: 0, createdAt: daysAgo(1),
};
const aggregator: TieBreakCandidate = {
id: 'b', skillType: 'aggregator', repoCreatedAt: daysAgo(200),
githubStars: 5000, githubForks: 500, createdAt: daysAgo(100),
};
expect(isCanonical(standalone, aggregator)).toBe(true);
expect(isCanonical(aggregator, standalone)).toBe(false);
});
it('collection beats aggregator', () => {
const collection: TieBreakCandidate = {
id: 'a', skillType: 'collection', repoCreatedAt: daysAgo(10),
githubStars: 10, githubForks: 1, createdAt: daysAgo(1),
};
const aggregator: TieBreakCandidate = {
id: 'b', skillType: 'aggregator', repoCreatedAt: daysAgo(300),
githubStars: 1000, githubForks: 100, createdAt: daysAgo(200),
};
expect(isCanonical(collection, aggregator)).toBe(true);
});
});
describe('Tier 2: significant age gap (>75 days)', () => {
it('older repo wins when gap > 75 days', () => {
const older: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: daysAgo(200),
githubStars: 10, githubForks: 1, createdAt: daysAgo(5),
};
const newer: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: daysAgo(50),
githubStars: 500, githubForks: 50, createdAt: daysAgo(3),
};
// Gap = 150 days > 75 → older wins despite fewer stars
expect(isCanonical(older, newer)).toBe(true);
expect(isCanonical(newer, older)).toBe(false);
});
it('does NOT apply when gap <= 75 days', () => {
const olderButClose: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: daysAgo(100),
githubStars: 10, githubForks: 1, createdAt: daysAgo(5),
};
const newerButMoreStars: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: daysAgo(50),
githubStars: 500, githubForks: 50, createdAt: daysAgo(3),
};
// Gap = 50 days <= 75 → falls through to Tier 3 (stars)
expect(isCanonical(newerButMoreStars, olderButClose)).toBe(true);
});
it('exactly 75 days falls through to stars', () => {
const a: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: daysAgo(100),
githubStars: 10, githubForks: 0, createdAt: daysAgo(5),
};
const b: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: daysAgo(25),
githubStars: 50, githubForks: 0, createdAt: daysAgo(3),
};
// Gap = exactly 75 days → NOT > 75, so falls to stars
expect(isCanonical(b, a)).toBe(true);
});
});
describe('Tier 3-4: stars and forks (close dates or NULL)', () => {
it('more stars wins when dates are close', () => {
const moreStars: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: daysAgo(100),
githubStars: 500, githubForks: 10, createdAt: daysAgo(5),
};
const fewerStars: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: daysAgo(80),
githubStars: 50, githubForks: 5, createdAt: daysAgo(3),
};
// Gap = 20 days < 75 → stars decide
expect(isCanonical(moreStars, fewerStars)).toBe(true);
});
it('more stars wins when both repo_created_at are NULL', () => {
const moreStars: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: null,
githubStars: 100, githubForks: 10, createdAt: daysAgo(30),
};
const fewerStars: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: null,
githubStars: 50, githubForks: 5, createdAt: daysAgo(20),
};
expect(isCanonical(moreStars, fewerStars)).toBe(true);
});
it('one NULL one not → skip Tier 2, use stars', () => {
const withDate: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: daysAgo(200),
githubStars: 10, githubForks: 0, createdAt: daysAgo(5),
};
const noDate: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: null,
githubStars: 500, githubForks: 50, createdAt: daysAgo(30),
};
// One is NULL → can't compare age → falls to stars
expect(isCanonical(noDate, withDate)).toBe(true);
});
it('forks break ties when stars are equal', () => {
const moreForks: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: daysAgo(50),
githubStars: 100, githubForks: 30, createdAt: daysAgo(5),
};
const fewerForks: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: daysAgo(40),
githubStars: 100, githubForks: 10, createdAt: daysAgo(3),
};
expect(isCanonical(moreForks, fewerForks)).toBe(true);
});
});
describe('Tier 5-6: final tie-breakers', () => {
it('older repo_created_at wins when stars and forks are equal', () => {
const older: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: daysAgo(60),
githubStars: 100, githubForks: 10, createdAt: daysAgo(5),
};
const newer: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: daysAgo(30),
githubStars: 100, githubForks: 10, createdAt: daysAgo(3),
};
// Gap = 30 days < 75 → stars equal, forks equal → Tier 5: older repo wins
expect(isCanonical(older, newer)).toBe(true);
});
it('older created_at wins when everything else is equal or NULL', () => {
const olderIndex: TieBreakCandidate = {
id: 'a', skillType: 'standalone', repoCreatedAt: null,
githubStars: 100, githubForks: 10, createdAt: daysAgo(30),
};
const newerIndex: TieBreakCandidate = {
id: 'b', skillType: 'standalone', repoCreatedAt: null,
githubStars: 100, githubForks: 10, createdAt: daysAgo(5),
};
// All NULL/equal → Tier 6: SkillHub index date
expect(isCanonical(olderIndex, newerIndex)).toBe(true);
});
});
describe('Real-world scenarios', () => {
it('creator wins over older aggregator repo with more stars', () => {
const creator: TieBreakCandidate = {
id: 'creator/tool/skill', skillType: 'standalone',
repoCreatedAt: new Date('2024-06-01'), githubStars: 50,
githubForks: 5, createdAt: daysAgo(30),
};
const aggregator: TieBreakCandidate = {
id: 'collector/awesome/skill', skillType: 'aggregator',
repoCreatedAt: new Date('2020-01-01'), githubStars: 2000,
githubForks: 200, createdAt: daysAgo(60),
};
// Tier 1: standalone beats aggregator
expect(isCanonical(creator, aggregator)).toBe(true);
});
it('old repo copying recently still wins with >75 day gap (accepted trade-off)', () => {
const oldCopier: TieBreakCandidate = {
id: 'old/project/skill', skillType: 'standalone',
repoCreatedAt: new Date('2020-01-01'), githubStars: 30,
githubForks: 2, createdAt: daysAgo(10),
};
const originalCreator: TieBreakCandidate = {
id: 'new/creator/skill', skillType: 'standalone',
repoCreatedAt: new Date('2024-11-01'), githubStars: 100,
githubForks: 15, createdAt: daysAgo(30),
};
// Gap > 75 days → old repo wins (accepted rare trade-off)
expect(isCanonical(oldCopier, originalCreator)).toBe(true);
});
it('recent copier loses to original when gap < 75 days', () => {
const original: TieBreakCandidate = {
id: 'original/skill/md', skillType: 'standalone',
repoCreatedAt: new Date('2024-10-01'), githubStars: 200,
githubForks: 20, createdAt: daysAgo(60),
};
const copier: TieBreakCandidate = {
id: 'copier/skill/md', skillType: 'standalone',
repoCreatedAt: new Date('2024-11-01'), githubStars: 50,
githubForks: 3, createdAt: daysAgo(30),
};
// Gap = 31 days < 75 → stars decide → original (200) > copier (50)
expect(isCanonical(original, copier)).toBe(true);
});
});
});

View File

@@ -28,6 +28,8 @@ export interface MeiliSkillDocument {
securityStatus: 'pass' | 'warning' | 'fail' | null; securityStatus: 'pass' | 'warning' | 'fail' | null;
isFeatured: boolean; isFeatured: boolean;
isVerified: boolean; isVerified: boolean;
reviewStatus: string | null;
aiScore: number;
indexedAt: string; indexedAt: string;
} }
@@ -43,7 +45,7 @@ export interface MeiliSearchOptions {
verified?: boolean; verified?: boolean;
featured?: boolean; featured?: boolean;
}; };
sort?: 'stars' | 'downloads' | 'rating' | 'recent'; sort?: 'stars' | 'downloads' | 'rating' | 'recent' | 'aiScore';
limit?: number; limit?: number;
offset?: number; offset?: number;
} }
@@ -154,6 +156,8 @@ export async function initializeSkillsIndex(): Promise<void> {
'isFeatured', 'isFeatured',
'securityScore', 'securityScore',
'githubStars', 'githubStars',
'reviewStatus',
'aiScore',
]); ]);
// Configure sortable attributes // Configure sortable attributes
@@ -162,6 +166,7 @@ export async function initializeSkillsIndex(): Promise<void> {
'downloadCount', 'downloadCount',
'rating', 'rating',
'indexedAt', 'indexedAt',
'aiScore',
]); ]);
// Configure ranking rules (relevance + custom) // Configure ranking rules (relevance + custom)
@@ -279,6 +284,9 @@ export async function searchSkills(options: MeiliSearchOptions): Promise<MeiliSe
case 'recent': case 'recent':
sort.push('indexedAt:desc'); sort.push('indexedAt:desc');
break; break;
case 'aiScore':
sort.push('aiScore:desc');
break;
} }
const searchResult: SearchResponse<MeiliSkillDocument> = await index.search(options.query, { const searchResult: SearchResponse<MeiliSkillDocument> = await index.search(options.query, {

View File

@@ -6,6 +6,7 @@ import {
ratingQueries, ratingQueries,
installationQueries, installationQueries,
favoriteQueries, favoriteQueries,
discoveredRepoQueries,
} from './queries.js'; } from './queries.js';
import { import {
createTestSkill, createTestSkill,
@@ -22,13 +23,14 @@ import {
// Mock the schema imports // Mock the schema imports
vi.mock('./schema.js', () => ({ vi.mock('./schema.js', () => ({
skills: { id: 'id', name: 'name', description: 'description', githubStars: 'github_stars', downloadCount: 'download_count', rating: 'rating', updatedAt: 'updated_at', isFeatured: 'is_featured', isVerified: 'is_verified', securityScore: 'security_score', viewCount: 'view_count', ratingCount: 'rating_count', ratingSum: 'rating_sum' }, skills: { id: 'id', name: 'name', description: 'description', githubStars: 'github_stars', downloadCount: 'download_count', rating: 'rating', updatedAt: 'updated_at', isFeatured: 'is_featured', isVerified: 'is_verified', securityScore: 'security_score', viewCount: 'view_count', ratingCount: 'rating_count', ratingSum: 'rating_sum', githubOwner: 'github_owner', githubRepo: 'github_repo', isBlocked: 'is_blocked', isOwnerClaimed: 'is_owner_claimed', isDuplicate: 'is_duplicate' },
categories: { id: 'id', name: 'name', slug: 'slug', sortOrder: 'sort_order', skillCount: 'skill_count' }, categories: { id: 'id', name: 'name', slug: 'slug', sortOrder: 'sort_order', skillCount: 'skill_count' },
skillCategories: { skillId: 'skill_id', categoryId: 'category_id' }, skillCategories: { skillId: 'skill_id', categoryId: 'category_id' },
users: { id: 'id', githubId: 'github_id', username: 'username', avatarUrl: 'avatar_url' }, users: { id: 'id', githubId: 'github_id', username: 'username', avatarUrl: 'avatar_url' },
ratings: { id: 'id', skillId: 'skill_id', userId: 'user_id', rating: 'rating', createdAt: 'created_at' }, ratings: { id: 'id', skillId: 'skill_id', userId: 'user_id', rating: 'rating', createdAt: 'created_at' },
installations: { id: 'id', skillId: 'skill_id', platform: 'platform', method: 'method' }, installations: { id: 'id', skillId: 'skill_id', platform: 'platform', method: 'method' },
favorites: { userId: 'user_id', skillId: 'skill_id', createdAt: 'created_at' }, favorites: { userId: 'user_id', skillId: 'skill_id', createdAt: 'created_at' },
discoveredRepos: { id: 'id', isBlocked: 'is_blocked', isArchived: 'is_archived', lastScanned: 'last_scanned', githubStars: 'github_stars', discoveredVia: 'discovered_via', owner: 'owner', repo: 'repo', githubForks: 'github_forks', defaultBranch: 'default_branch', hasSkillMd: 'has_skill_md', skillCount: 'skill_count', scanError: 'scan_error', updatedAt: 'updated_at' },
})); }));
// Helper to create a chainable mock // Helper to create a chainable mock
@@ -659,3 +661,127 @@ describe('favoriteQueries', () => {
}); });
}); });
}); });
describe('skillQueries (repo block/unblock)', () => {
describe('blockByRepo', () => {
it('should call update with isBlocked: true for matching owner/repo', async () => {
const mockDb = createMockDb();
await skillQueries.blockByRepo(mockDb as any, 'rawveg', 'skillsforge-marketplace');
expect(mockDb.update).toHaveBeenCalled();
});
});
describe('unblockByRepo', () => {
it('should call update with isBlocked: false for matching owner/repo', async () => {
const mockDb = createMockDb();
await skillQueries.unblockByRepo(mockDb as any, 'rawveg', 'skillsforge-marketplace');
expect(mockDb.update).toHaveBeenCalled();
});
});
describe('setOwnerClaimed', () => {
it('should call update with isOwnerClaimed: true for matching owner/repo', async () => {
const mockDb = createMockDb();
await skillQueries.setOwnerClaimed(mockDb as any, 'rawveg', 'skillsforge-marketplace');
expect(mockDb.update).toHaveBeenCalled();
});
});
describe('countByRepo', () => {
it('should return count of non-blocked skills for owner/repo', async () => {
const mockDb = createMockDb([{ count: 16 }]);
const result = await skillQueries.countByRepo(mockDb as any, 'rawveg', 'skillsforge-marketplace');
expect(result).toBe(16);
expect(mockDb.select).toHaveBeenCalled();
});
it('should return 0 when no matching skills', async () => {
const mockDb = createMockDb([{ count: 0 }]);
const result = await skillQueries.countByRepo(mockDb as any, 'unknown', 'unknown-repo');
expect(result).toBe(0);
});
it('should return 0 when result is empty', async () => {
const mockDb = createMockDb([]);
const result = await skillQueries.countByRepo(mockDb as any, 'owner', 'repo');
expect(result).toBe(0);
});
});
});
describe('discoveredRepoQueries', () => {
describe('blockRepo', () => {
it('should call update with isBlocked: true for the repo id', async () => {
const mockDb = createMockDb();
await discoveredRepoQueries.blockRepo(mockDb as any, 'rawveg/skillsforge-marketplace');
expect(mockDb.update).toHaveBeenCalled();
});
});
describe('unblockRepo', () => {
it('should call update with isBlocked: false for the repo id', async () => {
const mockDb = createMockDb();
await discoveredRepoQueries.unblockRepo(mockDb as any, 'rawveg/skillsforge-marketplace');
expect(mockDb.update).toHaveBeenCalled();
});
});
describe('getNeedingScanning', () => {
it('should return repos that need scanning', async () => {
const repos = [
{ id: 'owner/repo1', isBlocked: false },
{ id: 'owner/repo2', isBlocked: false },
];
const mockDb = createMockDb(repos);
const result = await discoveredRepoQueries.getNeedingScanning(mockDb as any);
expect(result).toEqual(repos);
expect(mockDb.select).toHaveBeenCalled();
});
it('should return empty array when no repos need scanning', async () => {
const mockDb = createMockDb([]);
const result = await discoveredRepoQueries.getNeedingScanning(mockDb as any);
expect(result).toEqual([]);
});
});
describe('getById', () => {
it('should return repo when found', async () => {
const repo = { id: 'rawveg/skillsforge-marketplace', isBlocked: true };
const mockDb = createMockDb([repo]);
const result = await discoveredRepoQueries.getById(mockDb as any, 'rawveg/skillsforge-marketplace');
expect(result).toEqual(repo);
expect(mockDb.select).toHaveBeenCalled();
});
it('should return null when repo not found', async () => {
const mockDb = createMockDb([]);
const result = await discoveredRepoQueries.getById(mockDb as any, 'nonexistent/repo');
expect(result).toBeNull();
});
});
});

File diff suppressed because it is too large Load Diff

View File

@@ -3,6 +3,7 @@ import {
text, text,
timestamp, timestamp,
integer, integer,
serial,
jsonb, jsonb,
boolean, boolean,
index, index,
@@ -65,8 +66,30 @@ export const skills = pgTable(
isVerified: boolean('is_verified').default(false), isVerified: boolean('is_verified').default(false),
isFeatured: boolean('is_featured').default(false), isFeatured: boolean('is_featured').default(false),
isBlocked: boolean('is_blocked').default(false), // Blocked from re-indexing (owner requested removal) isBlocked: boolean('is_blocked').default(false), // Blocked from re-indexing (owner requested removal)
isMalicious: boolean('is_malicious').default(false), // Flagged as containing malware/malicious code
isDeprecated: boolean('is_deprecated').default(false), // Auto-detected from raw_content (DEPRECATED/ARCHIVED markers)
isStale: boolean('is_stale').default(false), // Skill files no longer accessible on GitHub (confirmed after 3 consecutive 404s)
staleSince: timestamp('stale_since'), // When staleness was first detected
staleCheckCount: integer('stale_check_count').default(0), // Consecutive 404 count (threshold: 3)
lastScanned: timestamp('last_scanned'), lastScanned: timestamp('last_scanned'),
// Curation (populated by batch scripts, not crawler)
qualityScore: integer('quality_score'), // 0-100 from analyzer
qualityDetails: jsonb('quality_details').$type<{
documentation: number;
maintenance: number;
popularity: number;
factors: Array<{ name: string; score: number; weight: number; details?: string }>;
}>(),
skillType: text('skill_type').$type<'standalone' | 'project-bound' | 'collection' | 'aggregator'>(),
isDuplicate: boolean('is_duplicate').default(false),
isOwnerClaimed: boolean('is_owner_claimed').default(false), // Owner explicitly submitted via Add Skill — exempt from duplicate hiding
canonicalSkillId: text('canonical_skill_id'), // points to the "original" if this is a duplicate
repoSkillCount: integer('repo_skill_count'), // cached count of skills in same repo
reviewStatus: text('review_status').default('unreviewed'), // unreviewed → auto-scored → ai-reviewed → verified (or needs-re-review)
latestAiScore: integer('latest_ai_score'), // Denormalized from skill_reviews for efficient sorting
latestReviewDate: timestamp('latest_review_date', { withTimezone: true }), // When last AI-reviewed
// Content (cached) // Content (cached)
contentHash: text('content_hash'), contentHash: text('content_hash'),
rawContent: text('raw_content'), rawContent: text('raw_content'),
@@ -88,6 +111,7 @@ export const skills = pgTable(
// Timestamps // Timestamps
createdAt: timestamp('created_at').defaultNow().notNull(), createdAt: timestamp('created_at').defaultNow().notNull(),
updatedAt: timestamp('updated_at').defaultNow().notNull(), updatedAt: timestamp('updated_at').defaultNow().notNull(),
repoCreatedAt: timestamp('repo_created_at'),
indexedAt: timestamp('indexed_at'), indexedAt: timestamp('indexed_at'),
lastDownloadedAt: timestamp('last_downloaded_at'), lastDownloadedAt: timestamp('last_downloaded_at'),
}, },
@@ -103,6 +127,12 @@ export const skills = pgTable(
updatedIdx: index('idx_skills_updated').on(table.updatedAt), updatedIdx: index('idx_skills_updated').on(table.updatedAt),
sourceFormatIdx: index('idx_skills_source_format').on(table.sourceFormat), sourceFormatIdx: index('idx_skills_source_format').on(table.sourceFormat),
lastDownloadedIdx: index('idx_skills_last_downloaded').on(table.lastDownloadedAt), lastDownloadedIdx: index('idx_skills_last_downloaded').on(table.lastDownloadedAt),
qualityIdx: index('idx_skills_quality').on(table.qualityScore),
skillTypeIdx: index('idx_skills_type').on(table.skillType),
duplicateIdx: index('idx_skills_duplicate').on(table.isDuplicate),
contentHashIdx: index('idx_skills_content_hash').on(table.contentHash),
staleIdx: index('idx_skills_stale').on(table.isStale),
aiScoreIdx: index('idx_skills_ai_score').on(table.latestAiScore),
}) })
); );
@@ -282,6 +312,7 @@ export const discoveredRepos = pgTable(
githubForks: integer('github_forks').default(0), githubForks: integer('github_forks').default(0),
defaultBranch: text('default_branch').default('main'), defaultBranch: text('default_branch').default('main'),
isArchived: boolean('is_archived').default(false), isArchived: boolean('is_archived').default(false),
isBlocked: boolean('is_blocked').default(false), // Owner requested removal — skip re-indexing
scanError: text('scan_error'), // Last scan error if any scanError: text('scan_error'), // Last scan error if any
createdAt: timestamp('created_at').defaultNow().notNull(), createdAt: timestamp('created_at').defaultNow().notNull(),
updatedAt: timestamp('updated_at').defaultNow().notNull(), updatedAt: timestamp('updated_at').defaultNow().notNull(),
@@ -370,12 +401,53 @@ export const addRequests = pgTable(
}) })
); );
/**
* AI review results for skills (Layer 2 — verification pipeline)
*/
export const skillReviews = pgTable(
'skill_reviews',
{
id: serial('id').primaryKey(),
skillId: text('skill_id')
.references(() => skills.id, { onDelete: 'cascade' })
.notNull(),
reviewer: text('reviewer').notNull().default('claude-code'), // 'claude-code' | 'admin'
// Scores (per Skills Guide criteria)
aiScore: integer('ai_score'), // 0-100 overall
instructionQuality: integer('instruction_quality'), // clarity and structure
descriptionPrecision: integer('description_precision'), // description + triggers
usefulness: integer('usefulness'), // real-world value
technicalSoundness: integer('technical_soundness'), // correctness of commands/APIs
// Discoveries during review
reviewNotes: text('review_notes'),
suggestedCategories: jsonb('suggested_categories').$type<string[]>(),
blogWorthy: boolean('blog_worthy').default(false),
collectionCandidate: text('collection_candidate'),
needsImprovement: text('needs_improvement'),
i18nPriority: integer('i18n_priority').default(0), // 0=normal, 1=worth translating, 2=high priority
// Tracking
contentHashAtReview: text('content_hash_at_review'), // skill hash at time of review
reviewedAt: timestamp('reviewed_at').defaultNow(),
reviewVersion: integer('review_version').default(1), // criteria version for recalibration
recommendation: text('recommendation'), // 'flag-malicious' | null — reviewer recommendation for admin action
},
(table) => ({
skillIdx: index('idx_reviews_skill').on(table.skillId),
scoreIdx: index('idx_reviews_score').on(table.aiScore),
blogIdx: index('idx_reviews_blog').on(table.blogWorthy),
})
);
// Relations // Relations
export const skillsRelations = relations(skills, ({ many }) => ({ export const skillsRelations = relations(skills, ({ many }) => ({
categories: many(skillCategories), categories: many(skillCategories),
ratings: many(ratings), ratings: many(ratings),
installations: many(installations), installations: many(installations),
favorites: many(favorites), favorites: many(favorites),
reviews: many(skillReviews),
})); }));
export const categoriesRelations = relations(categories, ({ many, one }) => ({ export const categoriesRelations = relations(categories, ({ many, one }) => ({
@@ -451,6 +523,13 @@ export const addRequestsRelations = relations(addRequests, ({ one }) => ({
}), }),
})); }));
export const skillReviewsRelations = relations(skillReviews, ({ one }) => ({
skill: one(skills, {
fields: [skillReviews.skillId],
references: [skills.id],
}),
}));
/** /**
* Email subscriptions for newsletter and marketing emails * Email subscriptions for newsletter and marketing emails
*/ */

View File

@@ -46,6 +46,7 @@ export function createTestSkill(overrides: Partial<SkillInsert> = {}): SkillInse
contentHash: 'abc123', contentHash: 'abc123',
createdAt: new Date(), createdAt: new Date(),
updatedAt: new Date(), updatedAt: new Date(),
repoCreatedAt: null,
indexedAt: new Date(), indexedAt: new Date(),
...overrides, ...overrides,
}; };

3
pnpm-lock.yaml generated
View File

@@ -393,6 +393,9 @@ importers:
typescript: typescript:
specifier: ^5.3.0 specifier: ^5.3.0
version: 5.9.3 version: 5.9.3
vitest:
specifier: ^1.2.0
version: 1.6.1(@types/node@20.19.27)(jsdom@27.4.0)(terser@5.46.0)
packages: packages:

View File

@@ -51,6 +51,9 @@ CREATE TABLE IF NOT EXISTS skills (
is_blocked BOOLEAN DEFAULT FALSE, -- Blocked from re-indexing (owner requested removal) is_blocked BOOLEAN DEFAULT FALSE, -- Blocked from re-indexing (owner requested removal)
last_scanned TIMESTAMP WITH TIME ZONE, last_scanned TIMESTAMP WITH TIME ZONE,
-- Review pipeline
review_status TEXT DEFAULT 'unreviewed', -- 'unreviewed', 'auto-scored', 'ai-reviewed', 'verified', 'needs-re-review'
-- Content -- Content
content_hash TEXT, content_hash TEXT,
raw_content TEXT, raw_content TEXT,
@@ -62,6 +65,7 @@ CREATE TABLE IF NOT EXISTS skills (
-- Timestamps -- Timestamps
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL, created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL, updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
repo_created_at TIMESTAMP WITH TIME ZONE,
indexed_at TIMESTAMP WITH TIME ZONE, indexed_at TIMESTAMP WITH TIME ZONE,
last_downloaded_at TIMESTAMP WITH TIME ZONE last_downloaded_at TIMESTAMP WITH TIME ZONE
); );
@@ -160,6 +164,7 @@ CREATE TABLE IF NOT EXISTS discovered_repos (
github_forks INTEGER DEFAULT 0, github_forks INTEGER DEFAULT 0,
default_branch TEXT DEFAULT 'main', default_branch TEXT DEFAULT 'main',
is_archived BOOLEAN DEFAULT FALSE, is_archived BOOLEAN DEFAULT FALSE,
is_blocked BOOLEAN NOT NULL DEFAULT FALSE,
scan_error TEXT, scan_error TEXT,
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL, created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL updated_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL
@@ -348,6 +353,7 @@ BEGIN
-- Excluded: cached_files (download cache, not content change) -- Excluded: cached_files (download cache, not content change)
-- Excluded: indexed_at, last_scanned (indexer bookkeeping) -- Excluded: indexed_at, last_scanned (indexer bookkeeping)
-- Excluded: github_stars, github_forks (popularity metrics, not content) -- Excluded: github_stars, github_forks (popularity metrics, not content)
-- Excluded: quality_score, quality_details, skill_type, is_duplicate, canonical_skill_id, repo_skill_count (curation metadata)
IF ROW(NEW.name, NEW.description, NEW.github_owner, NEW.github_repo, NEW.skill_path, IF ROW(NEW.name, NEW.description, NEW.github_owner, NEW.github_repo, NEW.skill_path,
NEW.branch, NEW.commit_sha, NEW.source_format, NEW.version, NEW.license, NEW.author, NEW.homepage, NEW.branch, NEW.commit_sha, NEW.source_format, NEW.version, NEW.license, NEW.author, NEW.homepage,
NEW.compatibility, NEW.triggers, NEW.compatibility, NEW.triggers,
@@ -461,6 +467,124 @@ CREATE INDEX IF NOT EXISTS idx_skills_source_format ON skills(source_format);
ALTER TABLE skills ADD COLUMN IF NOT EXISTS last_downloaded_at TIMESTAMP WITH TIME ZONE; ALTER TABLE skills ADD COLUMN IF NOT EXISTS last_downloaded_at TIMESTAMP WITH TIME ZONE;
CREATE INDEX IF NOT EXISTS idx_skills_last_downloaded ON skills(last_downloaded_at DESC NULLS LAST); CREATE INDEX IF NOT EXISTS idx_skills_last_downloaded ON skills(last_downloaded_at DESC NULLS LAST);
-- Curation columns (Phase 2 - February 2026)
ALTER TABLE skills ADD COLUMN IF NOT EXISTS quality_score INTEGER;
ALTER TABLE skills ADD COLUMN IF NOT EXISTS quality_details JSONB;
ALTER TABLE skills ADD COLUMN IF NOT EXISTS skill_type TEXT; -- 'standalone', 'project-bound', 'collection', 'aggregator'
ALTER TABLE skills ADD COLUMN IF NOT EXISTS is_duplicate BOOLEAN DEFAULT FALSE;
ALTER TABLE skills ADD COLUMN IF NOT EXISTS is_owner_claimed BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE skills ADD COLUMN IF NOT EXISTS canonical_skill_id TEXT; -- points to original if duplicate
ALTER TABLE skills ADD COLUMN IF NOT EXISTS repo_skill_count INTEGER; -- cached count of skills in repo
ALTER TABLE discovered_repos ADD COLUMN IF NOT EXISTS is_blocked BOOLEAN NOT NULL DEFAULT FALSE;
CREATE INDEX IF NOT EXISTS idx_skills_quality ON skills(quality_score DESC NULLS LAST);
CREATE INDEX IF NOT EXISTS idx_skills_type ON skills(skill_type);
CREATE INDEX IF NOT EXISTS idx_skills_duplicate ON skills(is_duplicate);
CREATE INDEX IF NOT EXISTS idx_skills_content_hash ON skills(content_hash);
CREATE INDEX IF NOT EXISTS idx_skills_owner_claimed ON skills(is_owner_claimed) WHERE is_owner_claimed = TRUE;
CREATE INDEX IF NOT EXISTS idx_discovered_repos_blocked ON discovered_repos(is_blocked) WHERE is_blocked = TRUE;
-- Review pipeline columns (Phase 4+5 - February 2026)
ALTER TABLE skills ADD COLUMN IF NOT EXISTS review_status TEXT DEFAULT 'unreviewed';
CREATE INDEX IF NOT EXISTS idx_skills_review_status ON skills(review_status);
-- Deprecated detection (Phase 7.4 - February 2026)
ALTER TABLE skills ADD COLUMN IF NOT EXISTS is_deprecated BOOLEAN DEFAULT FALSE;
-- Safe pre-filter function for raw_content (handles invalid UTF-8 gracefully)
-- Used by review pipeline queries (getPending, countPending, countReReviews)
-- NOTE: EXCEPTION handler returns TRUE (benefit of the doubt) — CJK skills with
-- encoding edge cases should not be silently excluded from the review pipeline.
-- Genuine content issues will be caught during the actual review.
CREATE OR REPLACE FUNCTION raw_content_passes_prefilter(content TEXT) RETURNS BOOLEAN AS $$
BEGIN
-- Length check (octet_length works on bytes, safe for any encoding)
IF octet_length(content) < 200 THEN RETURN FALSE; END IF;
-- Content checks (may fail on encoding edge cases)
BEGIN
IF position('<!-- generated' in content) > 0 THEN RETURN FALSE; END IF;
IF position('/Users/' in LEFT(content, 1000)) > 0 THEN RETURN FALSE; END IF;
IF position('C:\Users\' in LEFT(content, 1000)) > 0 THEN RETURN FALSE; END IF;
EXCEPTION WHEN OTHERS THEN
-- Encoding error in string checks — content is long enough, let it through
RETURN TRUE;
END;
RETURN TRUE;
END;
$$ LANGUAGE plpgsql IMMUTABLE;
-- Batch populate is_deprecated for existing rows (one-time migration)
-- Uses PL/pgSQL with exception handling to skip rows with invalid UTF-8
DO $$
DECLARE
r RECORD;
rc TEXT;
cnt INT := 0;
BEGIN
FOR r IN SELECT id, raw_content FROM skills WHERE is_deprecated = false LOOP
BEGIN
rc := LEFT(r.raw_content, 1000);
IF rc ~* '(DEPRECATED|ARCHIVED|NO LONGER MAINTAINED|THIS PROJECT IS ABANDONED)' THEN
UPDATE skills SET is_deprecated = true WHERE id = r.id;
cnt := cnt + 1;
END IF;
EXCEPTION WHEN OTHERS THEN
NULL; -- Skip rows with encoding errors
END;
END LOOP;
RAISE NOTICE 'Marked % skills as deprecated', cnt;
END;
$$;
-- Repo creation date for duplicate tie-breaking (T074b)
ALTER TABLE skills ADD COLUMN IF NOT EXISTS repo_created_at TIMESTAMP WITH TIME ZONE;
-- AI review results table (Phase 7.1 - February 2026)
CREATE TABLE IF NOT EXISTS skill_reviews (
id SERIAL PRIMARY KEY,
skill_id TEXT NOT NULL REFERENCES skills(id) ON DELETE CASCADE,
reviewer TEXT NOT NULL DEFAULT 'claude-code', -- 'claude-code' | 'admin'
-- Scores (per Skills Guide criteria)
ai_score INTEGER, -- 0-100 overall
instruction_quality INTEGER, -- clarity and structure
description_precision INTEGER, -- description + triggers
usefulness INTEGER, -- real-world value
technical_soundness INTEGER, -- correctness of commands/APIs
-- Discoveries during review
review_notes TEXT,
suggested_categories JSONB, -- text[] stored as JSONB for Drizzle compatibility
blog_worthy BOOLEAN DEFAULT FALSE,
collection_candidate TEXT,
needs_improvement TEXT,
i18n_priority INTEGER DEFAULT 0, -- 0=normal, 1=worth translating, 2=high priority
-- Tracking
content_hash_at_review TEXT, -- skill hash at time of review
reviewed_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
review_version INTEGER DEFAULT 1 -- criteria version for recalibration
);
CREATE INDEX IF NOT EXISTS idx_reviews_skill ON skill_reviews(skill_id);
CREATE INDEX IF NOT EXISTS idx_reviews_score ON skill_reviews(ai_score);
CREATE INDEX IF NOT EXISTS idx_reviews_blog ON skill_reviews(blog_worthy) WHERE blog_worthy = true;
-- Stale skill detection (March 2026)
ALTER TABLE skills ADD COLUMN IF NOT EXISTS is_stale BOOLEAN DEFAULT FALSE;
ALTER TABLE skills ADD COLUMN IF NOT EXISTS stale_since TIMESTAMP WITH TIME ZONE;
ALTER TABLE skills ADD COLUMN IF NOT EXISTS stale_check_count INTEGER DEFAULT 0;
CREATE INDEX IF NOT EXISTS idx_skills_stale ON skills(is_stale) WHERE is_stale = TRUE;
-- Denormalized AI review scores (March 2026)
ALTER TABLE skills ADD COLUMN IF NOT EXISTS latest_ai_score INTEGER;
ALTER TABLE skills ADD COLUMN IF NOT EXISTS latest_review_date TIMESTAMPTZ;
CREATE INDEX IF NOT EXISTS idx_skills_ai_score ON skills(latest_ai_score) WHERE latest_ai_score IS NOT NULL;
-- Malicious skill detection (March 2026)
ALTER TABLE skills ADD COLUMN IF NOT EXISTS is_malicious BOOLEAN DEFAULT FALSE;
ALTER TABLE skill_reviews ADD COLUMN IF NOT EXISTS recommendation TEXT;
-- Grant permissions -- Grant permissions
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO postgres; GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO postgres;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO postgres; GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO postgres;

View File

@@ -44,6 +44,11 @@ RUN adduser --system --uid 1001 indexer
# Copy only the bundled dist (no node_modules needed - everything is bundled) # Copy only the bundled dist (no node_modules needed - everything is bundled)
COPY --from=builder --chown=indexer:nodejs /app/services/indexer/dist ./dist COPY --from=builder --chown=indexer:nodejs /app/services/indexer/dist ./dist
# Curation scripts + skillhub-core (for batch-security, batch-score, curate)
COPY --from=builder --chown=indexer:nodejs /app/scripts/curation ./scripts/curation
COPY --from=builder --chown=indexer:nodejs /app/packages/core/dist ./packages/core/dist
RUN npm install --no-save pg gray-matter yaml
USER indexer USER indexer
CMD ["node", "dist/worker.js"] CMD ["node", "dist/worker.js"]

View File

@@ -9,7 +9,8 @@
"worker": "tsx src/worker.ts", "worker": "tsx src/worker.ts",
"crawl": "tsx src/crawl.ts", "crawl": "tsx src/crawl.ts",
"lint": "eslint src/", "lint": "eslint src/",
"typecheck": "tsc --noEmit" "typecheck": "tsc --noEmit",
"test": "vitest"
}, },
"dependencies": { "dependencies": {
"@octokit/rest": "^20.0.2", "@octokit/rest": "^20.0.2",
@@ -26,6 +27,7 @@
"@types/node": "^20.10.0", "@types/node": "^20.10.0",
"tsup": "^8.0.1", "tsup": "^8.0.1",
"tsx": "^4.7.0", "tsx": "^4.7.0",
"typescript": "^5.3.0" "typescript": "^5.3.0",
"vitest": "^1.2.0"
} }
} }

View File

@@ -0,0 +1,55 @@
import { describe, it, expect } from 'vitest';
import { parseGitHubRepoUrl } from './crawl.js';
describe('parseGitHubRepoUrl', () => {
it('parses plain repo URL', () => {
const result = parseGitHubRepoUrl('https://github.com/nuxt/ui');
expect(result).toEqual({ owner: 'nuxt', repo: 'ui', branch: null });
});
it('parses URL with /tree/branch', () => {
const result = parseGitHubRepoUrl('https://github.com/nuxt/ui/tree/v4');
expect(result).toEqual({ owner: 'nuxt', repo: 'ui', branch: 'v4' });
});
it('parses URL with /tree/branch and subdirectory path', () => {
const result = parseGitHubRepoUrl('https://github.com/nuxt/ui/tree/v4/skills/nuxt-ui');
expect(result).toEqual({ owner: 'nuxt', repo: 'ui', branch: 'v4' });
});
it('parses URL with trailing slash', () => {
const result = parseGitHubRepoUrl('https://github.com/owner/repo/');
expect(result).toEqual({ owner: 'owner', repo: 'repo', branch: null });
});
it('parses URL with /tree/ but no branch (just tree segment)', () => {
const result = parseGitHubRepoUrl('https://github.com/owner/repo/tree/');
// parts[3] would be undefined since there's no branch after /tree/
expect(result).toEqual({ owner: 'owner', repo: 'repo', branch: null });
});
it('handles http URLs', () => {
const result = parseGitHubRepoUrl('http://github.com/owner/repo');
expect(result).toEqual({ owner: 'owner', repo: 'repo', branch: null });
});
it('handles release branches with slashes in name', () => {
// Note: only first segment after /tree/ is captured as branch
const result = parseGitHubRepoUrl('https://github.com/owner/repo/tree/release');
expect(result).toEqual({ owner: 'owner', repo: 'repo', branch: 'release' });
});
it('throws on URL with only owner', () => {
expect(() => parseGitHubRepoUrl('https://github.com/onlyone')).toThrow('Invalid GitHub URL');
});
it('throws on URL with no path', () => {
expect(() => parseGitHubRepoUrl('https://github.com/')).toThrow('Invalid GitHub URL');
});
it('handles repo URL with blob segment (not tree)', () => {
const result = parseGitHubRepoUrl('https://github.com/owner/repo/blob/main/README.md');
// /blob/ is not /tree/, so no branch extraction
expect(result).toEqual({ owner: 'owner', repo: 'repo', branch: null });
});
});

View File

@@ -8,11 +8,39 @@ import { INSTRUCTION_FILE_PATTERNS, type SourceFormat } from 'skillhub-core';
import { scheduleFullCrawl, scheduleIncrementalCrawl, getQueueStats, getQueue } from './queue.js'; import { scheduleFullCrawl, scheduleIncrementalCrawl, getQueueStats, getQueue } from './queue.js';
import { syncAllSkillsToMeilisearch, checkMeilisearchHealth } from './meilisearch-sync.js'; import { syncAllSkillsToMeilisearch, checkMeilisearchHealth } from './meilisearch-sync.js';
import { createDb, skillQueries, categoryQueries, discoveredRepoQueries, awesomeListQueries, addRequestQueries, userQueries, sql } from '@skillhub/db'; import { createDb, skillQueries, categoryQueries, discoveredRepoQueries, awesomeListQueries, addRequestQueries, userQueries, sql } from '@skillhub/db';
import { createStrategyOrchestrator, createDeepScanCrawler, createAwesomeListCrawler } from './strategies/index.js'; import { createStrategyOrchestrator, createDeepScanCrawler, createAwesomeListCrawler, createPopularReposCrawler, createCommitsSearchCrawler } from './strategies/index.js';
import { createCrawler } from './crawler.js'; import { createCrawler } from './crawler.js';
import { indexSkill } from './skill-indexer.js'; import { indexSkill } from './skill-indexer.js';
import { TokenManager } from './token-manager.js'; import { TokenManager } from './token-manager.js';
/**
* Parse a GitHub repository URL into owner, repo, and optional branch.
* Handles: github.com/owner/repo, github.com/owner/repo/tree/branch,
* github.com/owner/repo/tree/branch/some/path
*/
export function parseGitHubRepoUrl(urlStr: string): {
owner: string;
repo: string;
branch: string | null;
} {
const url = new URL(urlStr);
const parts = url.pathname.split('/').filter(Boolean);
const owner = parts[0];
const repo = parts[1];
if (!owner || !repo) {
throw new Error(`Invalid GitHub URL: ${urlStr}`);
}
// Check for /tree/{branch} path segment
if (parts[2] === 'tree' && parts[3]) {
return { owner, repo, branch: parts[3] };
}
return { owner, repo, branch: null };
}
const command = process.argv[2] || 'stats'; const command = process.argv[2] || 'stats';
async function main() { async function main() {
@@ -221,8 +249,17 @@ async function main() {
case 'discover-repos': { case 'discover-repos': {
console.log('Running all discovery strategies...\n'); console.log('Running all discovery strategies...\n');
const discoverDb = createDb(process.env.DATABASE_URL); const discoverDb = createDb(process.env.DATABASE_URL);
const discoverBudgetCrawler = createCrawler();
const orchestrator = createStrategyOrchestrator(); const orchestrator = createStrategyOrchestrator();
// Check budget before starting (reserve 20% for website users)
const discoverBudget = await discoverBudgetCrawler.checkBudget(0.20);
console.log(`API Budget: ${discoverBudget.remaining}/${discoverBudget.limit} remaining (reserve 20%)`);
if (!discoverBudget.ok) {
console.log(`\nAPI budget too low. Waiting for reset...`);
await discoverBudgetCrawler.waitForBudget(0.20);
}
const { repos: discoveredRepos, stats: discoverStats } = await orchestrator.runAllStrategies(); const { repos: discoveredRepos, stats: discoverStats } = await orchestrator.runAllStrategies();
console.log(`\nSaving ${discoveredRepos.length} discovered repos to database...`); console.log(`\nSaving ${discoveredRepos.length} discovered repos to database...`);
@@ -249,8 +286,17 @@ async function main() {
case 'awesome-lists': { case 'awesome-lists': {
console.log('Running awesome list discovery...\n'); console.log('Running awesome list discovery...\n');
const awesomeDb = createDb(process.env.DATABASE_URL); const awesomeDb = createDb(process.env.DATABASE_URL);
const awesomeBudgetCrawler = createCrawler();
const awesomeCrawler = createAwesomeListCrawler(); const awesomeCrawler = createAwesomeListCrawler();
// Check budget before starting (reserve 20% for website users)
const awesomeBudget = await awesomeBudgetCrawler.checkBudget(0.20);
console.log(`API Budget: ${awesomeBudget.remaining}/${awesomeBudget.limit} remaining (reserve 20%)`);
if (!awesomeBudget.ok) {
console.log(`\nAPI budget too low. Waiting for reset...`);
await awesomeBudgetCrawler.waitForBudget(0.20);
}
// Save known lists to DB // Save known lists to DB
for (const list of awesomeCrawler.getKnownLists()) { for (const list of awesomeCrawler.getKnownLists()) {
await awesomeListQueries.upsert(awesomeDb, { await awesomeListQueries.upsert(awesomeDb, {
@@ -291,12 +337,125 @@ async function main() {
break; break;
} }
case 'discover-popular': {
console.log('Discovering popular GitHub repositories...\n');
const popDb = createDb(process.env.DATABASE_URL);
const popCrawler = createPopularReposCrawler();
const popBudgetCrawler = createCrawler();
// Parse min-stars option (default 1000)
const minStarsArg = process.argv.find(a => a.startsWith('--min-stars='));
const minStars = minStarsArg ? parseInt(minStarsArg.split('=')[1]) : 1000;
// Check budget
const popBudget = await popBudgetCrawler.checkBudget(0.20);
console.log(`API Budget: ${popBudget.remaining}/${popBudget.limit} remaining`);
if (!popBudget.ok) {
console.log(`\nAPI budget too low. Waiting for reset...`);
await popBudgetCrawler.waitForBudget(0.20);
}
const repos = await popCrawler.discoverPopularRepos(minStars);
console.log(`\nSaving ${repos.length} popular repos to discovered_repos...`);
let popSaved = 0;
for (const repo of repos) {
try {
await discoveredRepoQueries.upsert(popDb, {
id: `${repo.owner}/${repo.repo}`,
owner: repo.owner,
repo: repo.repo,
discoveredVia: 'popular-scan',
githubStars: repo.stars,
githubForks: repo.forks,
defaultBranch: repo.defaultBranch,
isArchived: repo.isArchived,
});
popSaved++;
} catch {
// Skip errors
}
}
console.log(`Saved ${popSaved} repositories`);
console.log('Run "deep-scan" next to scan their branches for skills');
break;
}
case 'commits-search': {
console.log('Searching for repos with recent SKILL.md commits...\n');
const csDb = createDb(process.env.DATABASE_URL);
const csCrawler = createCommitsSearchCrawler();
const csBudgetCrawler = createCrawler();
// Parse days-back option (default 30)
const daysBackArg = process.argv.find(a => a.startsWith('--days='));
const daysBack = daysBackArg ? parseInt(daysBackArg.split('=')[1]) : 30;
// Check budget
const csBudget = await csBudgetCrawler.checkBudget(0.20);
console.log(`API Budget: ${csBudget.remaining}/${csBudget.limit} remaining`);
if (!csBudget.ok) {
console.log(`\nAPI budget too low. Waiting for reset...`);
await csBudgetCrawler.waitForBudget(0.20);
}
const csRepos = await csCrawler.discoverReposFromCommits(daysBack);
console.log(`\nSaving ${csRepos.length} repos to discovered_repos...`);
let csSaved = 0;
for (const repo of csRepos) {
try {
await discoveredRepoQueries.upsert(csDb, {
id: `${repo.owner}/${repo.repo}`,
owner: repo.owner,
repo: repo.repo,
discoveredVia: 'commits-search',
githubStars: repo.stars,
});
csSaved++;
} catch {
// Skip errors
}
}
console.log(`Saved ${csSaved} repositories`);
console.log('Run "deep-scan" next to scan their branches for skills');
break;
}
case 'deep-scan': { case 'deep-scan': {
console.log('Deep scanning discovered repositories...\n'); console.log('Deep scanning discovered repositories...\n');
const deepDb = createDb(process.env.DATABASE_URL); const deepDb = createDb(process.env.DATABASE_URL);
const deepCrawler = createDeepScanCrawler(); const deepCrawler = createDeepScanCrawler();
const skillCrawler = createCrawler(); const skillCrawler = createCrawler();
// Parse budget option (default 20% reserve for website users)
const deepBudgetArg = process.argv.find(a => a.startsWith('--budget='));
const deepBudgetPct = deepBudgetArg ? parseInt(deepBudgetArg.split('=')[1]) / 100 : 0.20;
// Parse branch scanning options
const allBranchesFlag = process.argv.includes('--all-branches');
const branchesArg = process.argv.find(a => a.startsWith('--branches='));
const extraBranchPatterns: string[] = branchesArg
? branchesArg.split('=')[1].split(',').map(p => p.trim()).filter(Boolean)
: [];
const deepScanOptions = { allBranches: allBranchesFlag, extraBranchPatterns };
if (allBranchesFlag) {
console.log('Branch mode: all branches');
} else if (extraBranchPatterns.length > 0) {
console.log(`Branch mode: smart + extra patterns [${extraBranchPatterns.join(', ')}]`);
} else {
console.log('Branch mode: smart (default + version/release branches)');
}
// Check initial budget
const deepInitialBudget = await skillCrawler.checkBudget(deepBudgetPct);
console.log(`API Budget: ${deepInitialBudget.remaining}/${deepInitialBudget.limit} remaining (reserve ${Math.round(deepBudgetPct * 100)}%)`);
if (!deepInitialBudget.ok) {
console.log(`\nAPI budget too low. Waiting for reset...`);
await skillCrawler.waitForBudget(deepBudgetPct);
}
// Get repos that need scanning (never scanned or stale) // Get repos that need scanning (never scanned or stale)
const oneWeekAgo = new Date(); const oneWeekAgo = new Date();
oneWeekAgo.setDate(oneWeekAgo.getDate() - 7); oneWeekAgo.setDate(oneWeekAgo.getDate() - 7);
@@ -307,16 +466,25 @@ async function main() {
break; break;
} }
console.log(`Found ${reposToScan.length} repositories to scan`); console.log(`Found ${reposToScan.length} repositories to scan\n`);
let scannedCount = 0; let scannedCount = 0;
let skillsDiscovered = 0; let skillsDiscovered = 0;
let skillsIndexed = 0; let skillsIndexed = 0;
for (const repo of reposToScan) { for (const repo of reposToScan) {
// Check budget every 10 repos
if (scannedCount > 0 && scannedCount % 10 === 0) {
const midBudget = await skillCrawler.checkBudget(deepBudgetPct);
if (!midBudget.ok) {
console.log(`\n Budget low (${midBudget.remaining}/${midBudget.limit}). Pausing for reset...`);
await skillCrawler.waitForBudget(deepBudgetPct);
}
}
try { try {
console.log(`\nScanning ${repo.owner}/${repo.repo}...`); console.log(`\nScanning ${repo.owner}/${repo.repo}...`);
const skills = await deepCrawler.scanRepository(repo.owner, repo.repo); const skills = await deepCrawler.scanRepository(repo.owner, repo.repo, deepScanOptions);
// Mark as scanned // Mark as scanned
await discoveredRepoQueries.markScanned( await discoveredRepoQueries.markScanned(
@@ -367,6 +535,10 @@ async function main() {
} }
} }
// Show final API status
const deepFinalBudget = await skillCrawler.checkBudget(deepBudgetPct);
console.log(`\n API remaining: ${deepFinalBudget.remaining}/${deepFinalBudget.limit}`);
console.log(`\nDeep scan complete:`); console.log(`\nDeep scan complete:`);
console.log(` Repositories scanned: ${scannedCount}`); console.log(` Repositories scanned: ${scannedCount}`);
console.log(` Skills discovered: ${skillsDiscovered}`); console.log(` Skills discovered: ${skillsDiscovered}`);
@@ -494,6 +666,15 @@ async function main() {
case 'full-enhanced': { case 'full-enhanced': {
console.log('Running full enhanced crawl (discovery + scan + index)...\n'); console.log('Running full enhanced crawl (discovery + scan + index)...\n');
const enhancedDb = createDb(process.env.DATABASE_URL); const enhancedDb = createDb(process.env.DATABASE_URL);
const enhancedBudgetCrawler = createCrawler();
// Check budget before starting (reserve 20% for website users)
const enhancedBudget = await enhancedBudgetCrawler.checkBudget(0.20);
console.log(`API Budget: ${enhancedBudget.remaining}/${enhancedBudget.limit} remaining (reserve 20%)\n`);
if (!enhancedBudget.ok) {
console.log(`API budget too low. Waiting for reset...`);
await enhancedBudgetCrawler.waitForBudget(0.20);
}
// Step 1: Run all discovery strategies // Step 1: Run all discovery strategies
console.log('Step 1: Running discovery strategies...'); console.log('Step 1: Running discovery strategies...');
@@ -594,19 +775,15 @@ async function main() {
} }
try { try {
// Parse repository URL // Parse repository URL (may include branch in /tree/{branch} path)
const url = new URL(request.repositoryUrl); const { owner, repo, branch: urlBranch } = parseGitHubRepoUrl(request.repositoryUrl);
const pathParts = url.pathname.split('/').filter(Boolean);
const owner = pathParts[0];
const repo = pathParts[1];
if (!owner || !repo) { // Get repo metadata for default branch
throw new Error('Invalid repository URL');
}
// Get default branch
const repoMeta = await addCrawler.getRepoMetadata(owner, repo); const repoMeta = await addCrawler.getRepoMetadata(owner, repo);
const branch = repoMeta.defaultBranch; const branch = urlBranch ?? repoMeta.defaultBranch;
if (urlBranch) {
console.log(` Branch: ${branch} (from URL)`);
}
// Parse skill paths (comma-separated) // Parse skill paths (comma-separated)
const skillPaths = request.skillPath.split(',').map((p: string) => p.trim()); const skillPaths = request.skillPath.split(',').map((p: string) => p.trim());
@@ -693,6 +870,9 @@ async function main() {
status: 'indexed', status: 'indexed',
indexedSkillId: allSkillIds.join(','), indexedSkillId: allSkillIds.join(','),
}); });
// Mark all skills from this repo as owner-claimed so they're
// exempt from duplicate hiding on the owner page and browse pages
await skillQueries.setOwnerClaimed(addDb, owner, repo);
const newCount = indexedSkillIds.length; const newCount = indexedSkillIds.length;
const existingCount = existingSkillIds.length; const existingCount = existingSkillIds.length;
if (newCount > 0 && existingCount > 0) { if (newCount > 0 && existingCount > 0) {
@@ -749,6 +929,170 @@ async function main() {
break; break;
} }
case 'add-repo': {
const input = process.argv[3];
if (!input) {
console.error('Usage: node dist/crawl.js add-repo <owner/repo or https://github.com/owner/repo>');
process.exit(1);
}
let arOwner: string;
let arRepo: string;
try {
if (input.startsWith('https://') || input.startsWith('http://')) {
const arUrl = new URL(input);
const arParts = arUrl.pathname.split('/').filter(Boolean);
arOwner = arParts[0];
arRepo = arParts[1];
} else {
const arParts = input.split('/').filter(Boolean);
arOwner = arParts[0];
arRepo = arParts[1];
}
} catch {
console.error('Could not parse owner/repo from input:', input);
process.exit(1);
}
if (!arOwner! || !arRepo!) {
console.error('Could not parse owner/repo from input:', input);
process.exit(1);
}
console.log(`Validating ${arOwner}/${arRepo} on GitHub...`);
const arGhRes = await fetch(`https://api.github.com/repos/${arOwner}/${arRepo}`, {
headers: {
Authorization: `token ${process.env.GITHUB_TOKEN}`,
'User-Agent': 'SkillHub',
Accept: 'application/vnd.github.v3+json',
},
signal: AbortSignal.timeout(10000),
});
if (!arGhRes.ok) {
console.error(`Repo ${arOwner}/${arRepo} not found on GitHub (HTTP ${arGhRes.status})`);
process.exit(1);
}
const arGhData = await arGhRes.json() as {
stargazers_count: number;
forks_count: number;
default_branch: string;
archived: boolean;
};
const arDb = createDb(process.env.DATABASE_URL);
await discoveredRepoQueries.upsert(arDb, {
id: `${arOwner}/${arRepo}`,
owner: arOwner,
repo: arRepo,
discoveredVia: 'manual',
sourceUrl: `https://github.com/${arOwner}/${arRepo}`,
githubStars: arGhData.stargazers_count ?? 0,
githubForks: arGhData.forks_count ?? 0,
defaultBranch: arGhData.default_branch ?? 'main',
isArchived: arGhData.archived ?? false,
});
console.log(`✅ Added ${arOwner}/${arRepo} to discovered_repos.`);
console.log(` Run 'deep-scan' to index skills from this repository.`);
break;
}
case 'stale-check': {
console.log('Checking for stale skills (removed from GitHub)...\n');
const staleDb = createDb(process.env.DATABASE_URL);
const staleCrawler = createCrawler();
// Parse --limit=N option
const staleLimitArg = process.argv.find(a => a.startsWith('--limit='));
const staleLimit = staleLimitArg ? parseInt(staleLimitArg.split('=')[1]) : 500;
// Check GitHub rate limit
try {
const rateLimitInfo = await staleCrawler.getRateLimitStatus();
console.log(`GitHub API: ${rateLimitInfo.remaining}/${rateLimitInfo.limit} requests remaining`);
if (rateLimitInfo.remaining < 50) {
console.log('Not enough API quota for stale checking. Try again later.');
break;
}
console.log('');
} catch {
console.log('Could not check GitHub rate limit. Proceeding anyway...\n');
}
// Get skills to check (oldest scanned first)
const skillsToCheck = await skillQueries.getSkillsToStaleCheck(staleDb, staleLimit);
console.log(`Checking ${skillsToCheck.length} skills...\n`);
let staleChecked = 0;
let stillAlive = 0;
let newlyStale = 0;
let staleErrors = 0;
for (const skill of skillsToCheck) {
// Rate limit check every 50 skills
if (staleChecked > 0 && staleChecked % 50 === 0) {
try {
const midCheck = await staleCrawler.getRateLimitStatus();
if (midCheck.remaining < 20) {
console.log(`\n API quota low (${midCheck.remaining} remaining). Stopping.`);
break;
}
} catch {
// Continue if rate limit check fails
}
}
try {
const sourceFormat = skill.sourceFormat || 'skill.md';
const pattern = INSTRUCTION_FILE_PATTERNS.find(p => p.format === sourceFormat);
const filename = pattern?.filename || 'SKILL.md';
const filePath = skill.skillPath === '.' ? filename : `${skill.skillPath}/${filename}`;
const exists = await staleCrawler.checkFileExists(
skill.githubOwner,
skill.githubRepo,
filePath,
skill.branch || 'main'
);
if (exists) {
stillAlive++;
// Clear any partial stale state
if ((skill.staleCheckCount ?? 0) > 0) {
await skillQueries.clearStaleState(staleDb, skill.id);
}
} else {
await skillQueries.incrementStaleCheck(staleDb, skill.id);
newlyStale++;
console.log(` STALE: ${skill.id}`);
}
} catch {
staleErrors++;
}
staleChecked++;
// Progress every 100
if (staleChecked % 100 === 0) {
console.log(` Progress: ${staleChecked}/${skillsToCheck.length} checked`);
}
}
console.log(`\nStale check complete:`);
console.log(` Checked: ${staleChecked}`);
console.log(` Still alive: ${stillAlive}`);
console.log(` Stale detected: ${newlyStale}`);
console.log(` Errors: ${staleErrors}`);
// Show total stale count
const totalStale = await skillQueries.countStale(staleDb);
console.log(` Total stale skills in DB: ${totalStale}`);
break;
}
default: default:
console.log('Usage: pnpm crawl <command>\n'); console.log('Usage: pnpm crawl <command>\n');
@@ -767,6 +1111,7 @@ async function main() {
console.log(' sync-meili - Sync all skills from database to Meilisearch'); console.log(' sync-meili - Sync all skills from database to Meilisearch');
console.log(' recategorize - Re-categorize all skills with 23 categories (alias: link-categories)'); console.log(' recategorize - Re-categorize all skills with 23 categories (alias: link-categories)');
console.log(' process-add-requests - Process pending add requests (auto-index skills)'); console.log(' process-add-requests - Process pending add requests (auto-index skills)');
console.log(' stale-check - Check for stale skills removed from GitHub (--limit=N)');
process.exit(1); process.exit(1);
} }

View File

@@ -45,6 +45,7 @@ export interface RepoMetadata {
license: string | null; license: string | null;
description: string | null; description: string | null;
updatedAt: string; updatedAt: string;
createdAt: string;
defaultBranch: string; defaultBranch: string;
topics: string[]; topics: string[];
} }
@@ -109,14 +110,10 @@ export class GitHubCrawler {
this.lastCodeSearchTime = Date.now(); this.lastCodeSearchTime = Date.now();
} }
private async getOctokit(): Promise<Octokit> { private async getOctokit(): Promise<{ octokit: Octokit; token: string }> {
return this.octokitPool.getBestInstance(); return this.octokitPool.getBestInstance();
} }
private getCurrentToken(): string {
return this.tokenManager.getBestToken();
}
/** /**
* Discover skills from all sources: official repos, community repos, and GitHub search * Discover skills from all sources: official repos, community repos, and GitHub search
*/ */
@@ -168,8 +165,7 @@ export class GitHubCrawler {
const branch = repoMeta.defaultBranch; const branch = repoMeta.defaultBranch;
// List contents of skills directory // List contents of skills directory
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.getContent({ const response = await octokit.repos.getContent({
owner, owner,
repo, repo,
@@ -219,10 +215,9 @@ export class GitHubCrawler {
/** /**
* Check if a file exists in a repository * Check if a file exists in a repository
*/ */
private async checkFileExists(owner: string, repo: string, path: string, ref: string): Promise<boolean> { async checkFileExists(owner: string, repo: string, path: string, ref: string): Promise<boolean> {
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.getContent({ const response = await octokit.repos.getContent({
owner, owner,
repo, repo,
@@ -319,8 +314,7 @@ export class GitHubCrawler {
// Enforce code search secondary rate limit delay // Enforce code search secondary rate limit delay
await this.waitForCodeSearchSlot(); await this.waitForCodeSearchSlot();
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.search.code({ const response = await octokit.search.code({
q: query, q: query,
per_page: perPage, per_page: perPage,
@@ -352,7 +346,7 @@ export class GitHubCrawler {
owner: item.repository.owner.login, owner: item.repository.owner.login,
repo: item.repository.name, repo: item.repository.name,
path: skillPath, path: skillPath,
branch: 'main', branch: '', // empty → fetchSkillContent falls back to repoMeta.defaultBranch
sourceFormat: expectedFormat, sourceFormat: expectedFormat,
}); });
} }
@@ -440,8 +434,7 @@ export class GitHubCrawler {
* Get repository metadata * Get repository metadata
*/ */
async getRepoMetadata(owner: string, repo: string): Promise<RepoMetadata> { async getRepoMetadata(owner: string, repo: string): Promise<RepoMetadata> {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.get({ owner, repo }); const response = await octokit.repos.get({ owner, repo });
this.octokitPool.updateStats(token, response.headers); this.octokitPool.updateStats(token, response.headers);
@@ -451,6 +444,7 @@ export class GitHubCrawler {
license: response.data.license?.spdx_id || null, license: response.data.license?.spdx_id || null,
description: response.data.description, description: response.data.description,
updatedAt: response.data.updated_at, updatedAt: response.data.updated_at,
createdAt: response.data.created_at,
defaultBranch: response.data.default_branch, defaultBranch: response.data.default_branch,
topics: response.data.topics || [], topics: response.data.topics || [],
}; };
@@ -463,8 +457,7 @@ export class GitHubCrawler {
async fetchOwnerEmail(username: string): Promise<{ email: string | null; source: string | null }> { async fetchOwnerEmail(username: string): Promise<{ email: string | null; source: string | null }> {
try { try {
// Step 1: GitHub Profile API // Step 1: GitHub Profile API
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const userResponse = await octokit.users.getByUsername({ username }); const userResponse = await octokit.users.getByUsername({ username });
this.octokitPool.updateStats(token, userResponse.headers); this.octokitPool.updateStats(token, userResponse.headers);
@@ -524,8 +517,7 @@ export class GitHubCrawler {
ref: string ref: string
): Promise<string> { ): Promise<string> {
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.getContent({ const response = await octokit.repos.getContent({
owner, owner,
repo, repo,
@@ -558,8 +550,7 @@ export class GitHubCrawler {
ref: string ref: string
): Promise<FileInfo[]> { ): Promise<FileInfo[]> {
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.getContent({ const response = await octokit.repos.getContent({
owner, owner,
repo, repo,

View File

@@ -0,0 +1,157 @@
/**
* Email notification for skill indexing completion
* Sends notification to users who submitted add-requests
*/
import { Resend } from 'resend';
const SITE_URL =
process.env.NEXT_PUBLIC_APP_URL || 'https://skills.palebluedot.live';
const FROM_EMAIL =
process.env.RESEND_FROM_EMAIL || 'SkillHub <onboarding@resend.dev>';
// Lazy-init Resend client (returns null if API key not configured)
let resendClient: Resend | null = null;
function getResend(): Resend | null {
if (resendClient) return resendClient;
const apiKey = process.env.RESEND_API_KEY;
if (!apiKey) return null;
resendClient = new Resend(apiKey);
return resendClient;
}
interface SkillIndexedDetails {
skillId: string;
skillName: string;
repositoryUrl: string;
}
const translations = {
en: {
subject: 'Your skill has been indexed on SkillHub!',
greeting: 'Good news!',
intro: (name: string, repo: string) =>
`Your skill <strong>${name}</strong> from <strong>${repo}</strong> has been successfully indexed on SkillHub.`,
status:
'Your skill is now searchable and installable by developers.',
cta: 'View Skill',
footer:
'You received this because you submitted an add request on SkillHub.',
},
fa: {
subject: 'مهارت شما در SkillHub ایندکس شد!',
greeting: 'خبر خوب!',
intro: (name: string, repo: string) =>
`مهارت <strong>${name}</strong> از مخزن <strong>${repo}</strong> با موفقیت در SkillHub ایندکس شد.`,
status:
'مهارت شما اکنون قابل جستجو و نصب توسط توسعه‌دهندگان است.',
cta: 'مشاهده مهارت',
footer:
'این ایمیل به دلیل ثبت درخواست افزودن مهارت شما در SkillHub ارسال شده است.',
},
} as const;
function buildHtml(
locale: 'en' | 'fa',
details: SkillIndexedDetails
): string {
const t = translations[locale];
const isRtl = locale === 'fa';
const dir = isRtl ? 'rtl' : 'ltr';
const fontFamily = isRtl ? 'Tahoma, Arial, sans-serif' : 'Arial, sans-serif';
const encodedId = details.skillId
.split('/')
.map(encodeURIComponent)
.join('/');
const skillUrl = `${SITE_URL}/${locale}/skill/${encodedId}`;
return `<!DOCTYPE html>
<html lang="${locale}" dir="${dir}">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
</head>
<body style="margin:0;padding:0;background-color:#f4f4f5;font-family:${fontFamily};">
<table width="100%" cellpadding="0" cellspacing="0" style="background-color:#f4f4f5;padding:32px 0;">
<tr>
<td align="center">
<table width="600" cellpadding="0" cellspacing="0" style="max-width:600px;width:100%;">
<!-- Header -->
<tr>
<td style="background-color:#0284c7;padding:24px 32px;border-radius:8px 8px 0 0;">
<h1 style="margin:0;color:#ffffff;font-size:22px;font-family:${fontFamily};">SkillHub</h1>
</td>
</tr>
<!-- Body -->
<tr>
<td style="background-color:#ffffff;padding:32px;border-radius:0 0 8px 8px;">
<h2 style="margin:0 0 16px;color:#18181b;font-size:20px;font-family:${fontFamily};">
${t.greeting}
</h2>
<p style="margin:0 0 16px;color:#3f3f46;font-size:16px;line-height:1.6;font-family:${fontFamily};">
${t.intro(details.skillName, details.repositoryUrl)}
</p>
<p style="margin:0 0 24px;color:#3f3f46;font-size:16px;line-height:1.6;font-family:${fontFamily};">
${t.status}
</p>
<!-- CTA Button -->
<table cellpadding="0" cellspacing="0" style="margin:0 0 24px;">
<tr>
<td style="background-color:#0284c7;border-radius:6px;padding:12px 24px;">
<a href="${skillUrl}" style="color:#ffffff;text-decoration:none;font-size:16px;font-weight:bold;font-family:${fontFamily};">
${t.cta}
</a>
</td>
</tr>
</table>
<!-- Footer -->
<p style="margin:0;color:#a1a1aa;font-size:13px;line-height:1.5;font-family:${fontFamily};">
${t.footer}
</p>
</td>
</tr>
</table>
</td>
</tr>
</table>
</body>
</html>`;
}
/**
* Send email notification when a skill has been indexed
*/
export async function sendSkillIndexedEmail(
to: string,
locale: 'en' | 'fa',
details: SkillIndexedDetails
): Promise<boolean> {
const resend = getResend();
if (!resend) return false;
try {
const t = translations[locale];
const html = buildHtml(locale, details);
const result = await resend.emails.send({
from: FROM_EMAIL,
to,
subject: t.subject,
html,
});
if (result.error) {
console.error('[Email] Resend error (indexed):', result.error);
return false;
}
console.log(
`[Email] Sent skill-indexed notification to ${to} for ${details.skillId}`
);
return true;
} catch (error) {
console.error('[Email] Failed to send indexed email:', error);
return false;
}
}

View File

@@ -92,6 +92,8 @@ async function initializeIndex(): Promise<void> {
'isFeatured', 'isFeatured',
'securityScore', 'securityScore',
'githubStars', 'githubStars',
'reviewStatus',
'aiScore',
]); ]);
// Configure sortable attributes // Configure sortable attributes
@@ -100,6 +102,7 @@ async function initializeIndex(): Promise<void> {
'downloadCount', 'downloadCount',
'rating', 'rating',
'indexedAt', 'indexedAt',
'aiScore',
]); ]);
// Configure ranking rules // Configure ranking rules
@@ -113,6 +116,12 @@ async function initializeIndex(): Promise<void> {
'githubStars:desc', 'githubStars:desc',
]); ]);
// Configure separator tokens so hyphens split into individual words
// This means "pdf-converter" is searchable as "pdf converter"
await index.updateSettings({
separatorTokens: ['-'],
});
indexInitialized = true; indexInitialized = true;
console.log('Meilisearch skills index initialized'); console.log('Meilisearch skills index initialized');
} catch (error) { } catch (error) {
@@ -138,6 +147,8 @@ export async function syncSkillToMeilisearch(skill: {
securityStatus?: 'pass' | 'warning' | 'fail' | null; securityStatus?: 'pass' | 'warning' | 'fail' | null;
isFeatured?: boolean | null; isFeatured?: boolean | null;
isVerified?: boolean | null; isVerified?: boolean | null;
reviewStatus?: string | null;
latestAiScore?: number | null;
indexedAt?: Date | null; indexedAt?: Date | null;
}): Promise<boolean> { }): Promise<boolean> {
const meili = getMeilisearchClient(); const meili = getMeilisearchClient();
@@ -165,6 +176,8 @@ export async function syncSkillToMeilisearch(skill: {
securityStatus: skill.securityStatus || null, securityStatus: skill.securityStatus || null,
isFeatured: skill.isFeatured || false, isFeatured: skill.isFeatured || false,
isVerified: skill.isVerified || false, isVerified: skill.isVerified || false,
reviewStatus: skill.reviewStatus || null,
aiScore: skill.latestAiScore || 0,
indexedAt: skill.indexedAt?.toISOString() || new Date().toISOString(), indexedAt: skill.indexedAt?.toISOString() || new Date().toISOString(),
}; };
@@ -248,6 +261,8 @@ export async function syncAllSkillsToMeilisearch(
securityStatus?: 'pass' | 'warning' | 'fail' | null; securityStatus?: 'pass' | 'warning' | 'fail' | null;
isFeatured?: boolean | null; isFeatured?: boolean | null;
isVerified?: boolean | null; isVerified?: boolean | null;
reviewStatus?: string | null;
latestAiScore?: number | null;
indexedAt?: Date | null; indexedAt?: Date | null;
}> }>
): Promise<{ success: number; failed: number }> { ): Promise<{ success: number; failed: number }> {
@@ -278,6 +293,8 @@ export async function syncAllSkillsToMeilisearch(
securityStatus: skill.securityStatus || null, securityStatus: skill.securityStatus || null,
isFeatured: skill.isFeatured || false, isFeatured: skill.isFeatured || false,
isVerified: skill.isVerified || false, isVerified: skill.isVerified || false,
reviewStatus: skill.reviewStatus || null,
aiScore: skill.latestAiScore || 0,
indexedAt: skill.indexedAt?.toISOString() || new Date().toISOString(), indexedAt: skill.indexedAt?.toISOString() || new Date().toISOString(),
})); }));

View File

@@ -29,9 +29,9 @@ export class OctokitPool {
return instance; return instance;
} }
async getBestInstance(): Promise<Octokit> { async getBestInstance(): Promise<{ octokit: Octokit; token: string }> {
const token = await this.tokenManager.checkAndRotate(); const token = await this.tokenManager.checkAndRotate();
return this.getInstance(token); return { octokit: this.getInstance(token), token };
} }
updateStats(token: string, headers: Record<string, unknown>): void { updateStats(token: string, headers: Record<string, unknown>): void {

View File

@@ -4,7 +4,8 @@
*/ */
import type { SourceFormat } from 'skillhub-core'; import type { SourceFormat } from 'skillhub-core';
import { createDb, skillQueries, categoryQueries, type Database } from '@skillhub/db'; import { createDb, skillQueries, categoryQueries, addRequestQueries, userQueries, type Database } from '@skillhub/db';
import { sendSkillIndexedEmail } from './email-notify.js';
import type { GitHubCrawler } from './crawler.js'; import type { GitHubCrawler } from './crawler.js';
import { SkillAnalyzer } from './analyzer.js'; import { SkillAnalyzer } from './analyzer.js';
import { syncSkillToMeilisearch } from './meilisearch-sync.js'; import { syncSkillToMeilisearch } from './meilisearch-sync.js';
@@ -32,7 +33,28 @@ export async function indexSkill(
// Fetch skill content // Fetch skill content
console.log(`Fetching ${source.owner}/${source.repo}/${source.path} [${sourceFormat}]...`); console.log(`Fetching ${source.owner}/${source.repo}/${source.path} [${sourceFormat}]...`);
const content = await crawler.fetchSkillContent(source); let content;
try {
content = await crawler.fetchSkillContent(source);
} catch (error) {
const errorMsg = error instanceof Error ? error.message : String(error);
// Detect 404 (skill removed from GitHub) — increment stale check
if (errorMsg.includes('not found') || errorMsg.includes('Not Found') || errorMsg.includes('404')) {
const skillName = source.path.split('/').pop() || 'skill';
const formatSuffix = sourceFormat !== 'skill.md' ? `~${sourceFormat.replace('.', '')}` : '';
const skillId = `${source.owner}/${source.repo}/${skillName}${formatSuffix}`;
await skillQueries.incrementStaleCheck(database, skillId).catch((err) => {
console.warn(` -> Failed to increment stale check for ${skillId}:`, err);
});
console.log(` -> ${skillId}: GitHub 404 (stale check incremented)`);
return null;
}
// For non-404 errors, rethrow
throw error;
}
// Analyze the skill with format awareness // Analyze the skill with format awareness
const analysis = await analyzer.analyze(content, sourceFormat); const analysis = await analyzer.analyze(content, sourceFormat);
@@ -58,10 +80,18 @@ export async function indexSkill(
// Check if we need to update (unless force) // Check if we need to update (unless force)
if (!force) { if (!force) {
if (existing && existing.contentHash === analysis.meta.contentHash) { if (existing && existing.contentHash === analysis.meta.contentHash) {
// Content unchanged, but check if path/branch changed
// (e.g., repo was restructured, file moved to a different directory)
const actualBranch = source.branch || content.repoMeta.defaultBranch;
if (existing.skillPath !== source.path || existing.branch !== actualBranch) {
console.log(`Skill ${skillId} path changed: ${existing.skillPath}${source.path}`);
// Don't skip - fall through to upsert which now updates skillPath/branch
} else {
console.log(`Skill ${skillId} unchanged, skipping`); console.log(`Skill ${skillId} unchanged, skipping`);
return null; return null;
} }
} }
}
// Upsert to database // Upsert to database
await skillQueries.upsert(database, { await skillQueries.upsert(database, {
@@ -81,7 +111,16 @@ export async function indexSkill(
triggers: analysis.skill.metadata.triggers, triggers: analysis.skill.metadata.triggers,
githubStars: content.repoMeta.stars, githubStars: content.repoMeta.stars,
githubForks: content.repoMeta.forks, githubForks: content.repoMeta.forks,
repoCreatedAt: new Date(content.repoMeta.createdAt),
securityScore: analysis.security.score, securityScore: analysis.security.score,
securityStatus: analysis.security.status,
qualityScore: analysis.quality.overall,
qualityDetails: {
documentation: analysis.quality.documentation,
maintenance: analysis.quality.maintenance,
popularity: analysis.quality.popularity,
factors: analysis.quality.factors,
},
contentHash: analysis.meta.contentHash, contentHash: analysis.meta.contentHash,
rawContent: content.skillMd, rawContent: content.skillMd,
indexedAt: new Date(), indexedAt: new Date(),
@@ -100,6 +139,41 @@ export async function indexSkill(
indexedAt: new Date(), indexedAt: new Date(),
}); });
// Check for matching add-requests and mark as indexed
try {
const matchingRequests = await addRequestQueries.findApprovedByRepo(
database,
source.owner,
source.repo
);
for (const request of matchingRequests) {
await addRequestQueries.updateStatus(database, request.id, {
status: 'indexed',
indexedSkillId: skillId,
});
// Send email notification
const user = await userQueries.getById(database, request.userId);
if (user?.email) {
const locale = (user.preferredLocale === 'fa' ? 'fa' : 'en') as 'en' | 'fa';
await sendSkillIndexedEmail(
user.email,
locale,
{
skillId,
skillName: analysis.skill.metadata.name,
repositoryUrl: `https://github.com/${source.owner}/${source.repo}`,
}
).catch((err: unknown) => {
console.warn(`[Indexer] Failed to send indexed email for ${skillId}:`, err);
});
}
}
} catch (error) {
console.warn(`[Indexer] Failed to check add-requests for ${skillId}:`, error);
}
// Link skill to categories based on keywords // Link skill to categories based on keywords
try { try {
const categories = await categoryQueries.linkSkillToCategories( const categories = await categoryQueries.linkSkillToCategories(

View File

@@ -32,7 +32,7 @@ export class AwesomeListCrawler {
this.octokitPool = new OctokitPool(this.tokenManager); this.octokitPool = new OctokitPool(this.tokenManager);
} }
private async getOctokit(): Promise<Octokit> { private async getOctokit(): Promise<{ octokit: Octokit; token: string }> {
return this.octokitPool.getBestInstance(); return this.octokitPool.getBestInstance();
} }
@@ -73,7 +73,7 @@ export class AwesomeListCrawler {
readmePath = 'README.md' readmePath = 'README.md'
): Promise<RepoReference[]> { ): Promise<RepoReference[]> {
try { try {
const octokit = await this.getOctokit(); const { octokit } = await this.getOctokit();
const response = await octokit.repos.getContent({ const response = await octokit.repos.getContent({
owner, owner,
repo, repo,
@@ -165,7 +165,7 @@ export class AwesomeListCrawler {
for (const query of searchQueries) { for (const query of searchQueries) {
try { try {
const octokit = await this.getOctokit(); const { octokit } = await this.getOctokit();
const response = await octokit.search.repos({ const response = await octokit.search.repos({
q: query, q: query,
sort: 'stars', sort: 'stars',

View File

@@ -0,0 +1,146 @@
import type { Octokit } from '@octokit/rest';
import { TokenManager } from '../token-manager.js';
import { OctokitPool } from '../octokit-pool.js';
export interface CommitSearchRepoResult {
owner: string;
repo: string;
stars?: number;
}
/**
* Commits Search Discovery Strategy
* Searches GitHub for recent commits mentioning "SKILL.md" in their messages.
* This finds repos where someone recently added or modified a SKILL.md file,
* even on non-default branches (unlike Code Search which only indexes default branches).
*
* Limitation: Searches commit messages, not file paths. Only catches commits
* where the author mentioned "SKILL.md" in the message.
*/
export class CommitsSearchCrawler {
private octokitPool: OctokitPool;
private tokenManager: TokenManager;
constructor(tokenManager?: TokenManager) {
this.tokenManager = tokenManager || TokenManager.getInstance();
this.octokitPool = new OctokitPool(this.tokenManager);
}
private async getOctokit(): Promise<{ octokit: Octokit; token: string }> {
return this.octokitPool.getBestInstance();
}
/**
* Search for recent commits mentioning SKILL.md in their messages.
* Returns unique repos found.
*/
async discoverReposFromCommits(daysBack: number = 30): Promise<CommitSearchRepoResult[]> {
const sinceDate = new Date();
sinceDate.setDate(sinceDate.getDate() - daysBack);
const dateStr = sinceDate.toISOString().split('T')[0]; // YYYY-MM-DD
const queries = [
`SKILL.md committer-date:>${dateStr}`,
`"add SKILL.md" committer-date:>${dateStr}`,
`"SKILL.md" path:skills committer-date:>${dateStr}`,
];
const allRepos = new Map<string, CommitSearchRepoResult>();
console.log(`Searching commits from last ${daysBack} days (since ${dateStr})...`);
for (const query of queries) {
try {
console.log(`\n Query: ${query}`);
const repos = await this.searchCommits(query);
for (const repo of repos) {
const key = `${repo.owner}/${repo.repo}`.toLowerCase();
if (!allRepos.has(key)) {
allRepos.set(key, repo);
}
}
console.log(` Found ${repos.length} repos (total unique: ${allRepos.size})`);
} catch (error) {
console.warn(` Failed for query "${query}":`, error instanceof Error ? error.message : error);
}
}
console.log(`\nTotal unique repos from commits search: ${allRepos.size}`);
return Array.from(allRepos.values());
}
/**
* Execute a single commits search query and extract unique repos.
*/
private async searchCommits(query: string): Promise<CommitSearchRepoResult[]> {
const repos = new Map<string, CommitSearchRepoResult>();
const maxPages = 5;
const perPage = 100;
for (let page = 1; page <= maxPages; page++) {
try {
const { octokit, token } = await this.getOctokit();
const response = await octokit.search.commits({
q: query,
sort: 'committer-date',
order: 'desc',
per_page: perPage,
page,
});
this.octokitPool.updateStats(token, response.headers);
if (page === 1) {
console.log(` Total results: ${response.data.total_count}`);
}
for (const item of response.data.items) {
const repoData = item.repository;
const key = `${repoData.owner.login}/${repoData.name}`.toLowerCase();
if (!repos.has(key)) {
repos.set(key, {
owner: repoData.owner.login,
repo: repoData.name,
});
}
}
if (response.data.items.length < perPage) break;
} catch (error) {
if (this.isBeyondResultsLimit(error)) {
console.log(` Reached 1000-result limit`);
break;
}
if (this.isRateLimitError(error)) {
console.log(` Rate limit hit, rotating token...`);
await this.tokenManager.checkAndRotate();
page--; // Retry same page
continue;
}
throw error;
}
}
return Array.from(repos.values());
}
private isRateLimitError(error: unknown): boolean {
if (!(error instanceof Error)) return false;
const status = (error as { status?: number }).status;
return status === 403 || status === 429;
}
private isBeyondResultsLimit(error: unknown): boolean {
return (
error instanceof Error &&
'status' in error &&
(error as { status: number }).status === 422
);
}
}
export function createCommitsSearchCrawler(tokenManager?: TokenManager | string): CommitsSearchCrawler {
if (typeof tokenManager === 'string') {
return new CommitsSearchCrawler(new TokenManager([tokenManager]));
}
return new CommitsSearchCrawler(tokenManager);
}

View File

@@ -0,0 +1,147 @@
import { describe, it, expect } from 'vitest';
import { filterAndSortBranches } from './deep-scan.js';
describe('filterAndSortBranches', () => {
it('returns only defaultBranch when no matching branches exist', () => {
const result = filterAndSortBranches(
['main', 'feature/foo', 'bugfix/bar', 'experiment'],
'main'
);
expect(result).toEqual(['main']);
});
it('includes well-known branch names (up to 5 non-default cap)', () => {
const result = filterAndSortBranches(
['main', 'stable', 'next', 'latest', 'canary', 'dev', 'develop', 'feature/x'],
'main'
);
expect(result[0]).toBe('main');
expect(result).toContain('stable');
expect(result).toContain('next');
expect(result).toContain('dev');
// 6 well-known names but only 5 non-default slots, so some get dropped
expect(result.length).toBe(6); // main + 5
expect(result).not.toContain('feature/x');
});
it('includes version branches matching /^[vV]\\d/', () => {
const result = filterAndSortBranches(
['main', 'v4', 'v3', 'v2', 'v1'],
'main'
);
expect(result[0]).toBe('main');
expect(result).toContain('v4');
expect(result).toContain('v3');
expect(result).toContain('v2');
expect(result).toContain('v1');
});
it('sorts version branches by semver descending', () => {
const result = filterAndSortBranches(
['main', 'v1', 'v3', 'v2', 'v10', 'v4'],
'main'
);
const versions = result.filter(b => b.startsWith('v'));
expect(versions[0]).toBe('v10');
expect(versions[1]).toBe('v4');
expect(versions[2]).toBe('v3');
expect(versions[3]).toBe('v2');
});
it('handles complex version numbers (v2.1, v3.0.1)', () => {
const result = filterAndSortBranches(
['main', 'v2.1', 'v2.0', 'v3.0.1', 'v1.5'],
'main'
);
const versions = result.filter(b => b.startsWith('v'));
expect(versions[0]).toBe('v3.0.1');
expect(versions[1]).toBe('v2.1');
expect(versions[2]).toBe('v2.0');
expect(versions[3]).toBe('v1.5');
});
it('takes only top 5 version branches', () => {
const result = filterAndSortBranches(
['main', 'v1', 'v2', 'v3', 'v4', 'v5', 'v6', 'v7', 'v8'],
'main'
);
// defaultBranch + 5 non-default max
expect(result.length).toBeLessThanOrEqual(6);
// Should have the top 5 versions (v8, v7, v6, v5, v4)
expect(result).toContain('v8');
expect(result).toContain('v7');
expect(result).toContain('v6');
expect(result).toContain('v5');
expect(result).toContain('v4');
expect(result).not.toContain('v1');
});
it('caps total non-default at 5 when mixing well-known + versions', () => {
const result = filterAndSortBranches(
['main', 'stable', 'next', 'dev', 'v4', 'v3', 'v2', 'v1'],
'main'
);
// 1 default + max 5 non-default
expect(result.length).toBeLessThanOrEqual(6);
expect(result[0]).toBe('main');
});
it('includes release/ and releases/ prefixed branches', () => {
const result = filterAndSortBranches(
['main', 'release/3.0', 'releases/v2', 'feature/x'],
'main'
);
expect(result).toContain('release/3.0');
expect(result).toContain('releases/v2');
expect(result).not.toContain('feature/x');
});
it('applies extra patterns from CLI (exact and prefix match)', () => {
const result = filterAndSortBranches(
['main', 'alpha', 'beta', 'alpha/test', 'feature/x'],
'main',
['alpha', 'beta']
);
expect(result).toContain('alpha');
expect(result).toContain('beta');
expect(result).not.toContain('feature/x');
});
it('excludes defaultBranch from filtered list even if it matches a pattern', () => {
const result = filterAndSortBranches(
['dev', 'v4', 'stable'],
'dev'
);
expect(result[0]).toBe('dev');
// dev should appear only once
const devCount = result.filter(b => b === 'dev').length;
expect(devCount).toBe(1);
});
it('puts defaultBranch first always', () => {
const result = filterAndSortBranches(
['v10', 'master', 'v5', 'stable'],
'master'
);
expect(result[0]).toBe('master');
});
it('handles V (uppercase) version branches', () => {
const result = filterAndSortBranches(
['main', 'V4', 'V3'],
'main'
);
expect(result).toContain('V4');
expect(result).toContain('V3');
});
it('returns [defaultBranch] for empty branch list', () => {
const result = filterAndSortBranches([], 'main');
expect(result).toEqual(['main']);
});
it('returns [defaultBranch] when only default is in the list', () => {
const result = filterAndSortBranches(['main'], 'main');
expect(result).toEqual(['main']);
});
});

View File

@@ -3,10 +3,92 @@ import { TokenManager } from '../token-manager.js';import { OctokitPool } from '
import type { SkillSource } from 'skillhub-core'; import type { SkillSource } from 'skillhub-core';
import { INSTRUCTION_FILE_PATTERNS, type SourceFormat } from 'skillhub-core'; import { INSTRUCTION_FILE_PATTERNS, type SourceFormat } from 'skillhub-core';
export interface DeepScanOptions {
allBranches?: boolean;
extraBranchPatterns?: string[];
}
/** Well-known branch names that indicate important non-default branches. */
const IMPORTANT_BRANCH_NAMES = ['stable', 'next', 'latest', 'canary', 'dev', 'develop'];
/** Prefixes that indicate release/version branches. */
const IMPORTANT_BRANCH_PREFIXES = ['release/', 'releases/'];
/**
* Pure helper: filter and sort branches to find important ones worth scanning.
*
* Rules:
* 1. Always include defaultBranch first
* 2. Include exact name matches from IMPORTANT_BRANCH_NAMES
* 3. Include prefix matches from IMPORTANT_BRANCH_PREFIXES
* 4. Include version branches matching /^[vV]\d/ — sorted by semver desc, top 5
* 5. Include branches matching extraPatterns (exact or prefix match)
* 6. Cap non-default branches at 5
*/
export function filterAndSortBranches(
allBranchNames: string[],
defaultBranch: string,
extraPatterns: string[] = []
): string[] {
const versionBranches: string[] = [];
const otherImportant: string[] = [];
for (const name of allBranchNames) {
if (name === defaultBranch) continue;
// Check well-known exact names
if (IMPORTANT_BRANCH_NAMES.includes(name)) {
otherImportant.push(name);
continue;
}
// Check prefix patterns (release/, releases/)
const lower = name.toLowerCase();
if (IMPORTANT_BRANCH_PREFIXES.some(p => lower.startsWith(p))) {
otherImportant.push(name);
continue;
}
// Version branches: v* or V* followed by a digit (e.g., v4, v2.1, V3)
if (/^[vV]\d/.test(name)) {
versionBranches.push(name);
continue;
}
// Check extra patterns from CLI (exact or prefix match)
if (extraPatterns.length > 0) {
const matched = extraPatterns.some(p => name === p || name.startsWith(p + '/'));
if (matched) {
otherImportant.push(name);
}
}
}
// Sort version branches by semver descending, take top 5
const sortedVersionBranches = versionBranches
.sort((a, b) => {
const normalize = (s: string) =>
s.replace(/^[vV]/, '').split(/[.\-x]/).map(n => parseInt(n) || 0);
const aN = normalize(a);
const bN = normalize(b);
for (let i = 0; i < Math.max(aN.length, bN.length); i++) {
const diff = (bN[i] || 0) - (aN[i] || 0);
if (diff !== 0) return diff;
}
return 0;
})
.slice(0, 5);
// Combine: other important first, then version branches, cap at 5 non-default total
const nonDefault = [...otherImportant, ...sortedVersionBranches].slice(0, 5);
return [defaultBranch, ...nonDefault];
}
/** /**
* Deep Scan Strategy * Deep Scan Strategy
* Uses Git Trees API to recursively scan entire repositories for SKILL.md files * Uses Git Trees API to recursively scan entire repositories for SKILL.md files
* Can discover skills that aren't found by code search * Can discover skills that aren't found by code search
* Supports scanning multiple branches per repository
*/ */
export class DeepScanCrawler { export class DeepScanCrawler {
private octokitPool: OctokitPool; private octokitPool: OctokitPool;
@@ -17,45 +99,103 @@ export class DeepScanCrawler {
this.octokitPool = new OctokitPool(this.tokenManager); this.octokitPool = new OctokitPool(this.tokenManager);
} }
private async getOctokit(): Promise<Octokit> { private async getOctokit(): Promise<{ octokit: Octokit; token: string }> {
return this.octokitPool.getBestInstance(); return this.octokitPool.getBestInstance();
} }
private getCurrentToken(): string { /**
return this.tokenManager.getBestToken(); * List important branches worth scanning beyond the default branch.
* Calls octokit.repos.listBranches (1 API call).
*/
private async listImportantBranches(
owner: string,
repo: string,
defaultBranch: string,
extraPatterns: string[] = []
): Promise<string[]> {
try {
const { octokit, token } = await this.getOctokit();
const response = await octokit.repos.listBranches({
owner,
repo,
per_page: 100,
});
this.octokitPool.updateStats(token, response.headers);
const allBranchNames = response.data.map(b => b.name);
return filterAndSortBranches(allBranchNames, defaultBranch, extraPatterns);
} catch {
console.log(` Could not list branches for ${owner}/${repo}, using default only`);
return [defaultBranch];
}
} }
/** /**
* Deep scan a repository for all SKILL.md files using Git Trees API * List all branches in a repository (for --all-branches mode).
* This is more thorough than code search as it scans the entire repo * Handles pagination for repos with many branches.
*/ */
async scanRepository(owner: string, repo: string): Promise<SkillSource[]> { private async listAllBranches(
owner: string,
repo: string,
defaultBranch: string
): Promise<string[]> {
const allBranches: string[] = [defaultBranch];
let page = 1;
for (;;) {
try {
const { octokit, token } = await this.getOctokit();
const response = await octokit.repos.listBranches({
owner,
repo,
per_page: 100,
page,
});
this.octokitPool.updateStats(token, response.headers);
for (const branch of response.data) {
if (branch.name !== defaultBranch) {
allBranches.push(branch.name);
}
}
if (response.data.length < 100) break;
page++;
} catch {
break;
}
}
console.log(` Found ${allBranches.length} total branches for ${owner}/${repo}`);
return allBranches;
}
/**
* Scan a single branch for instruction files using the Git Trees API.
* Returns [] silently for truncated trees (fallback handles those at repo level).
*/
private async scanSingleBranch(
owner: string,
repo: string,
branch: string
): Promise<SkillSource[]> {
const skills: SkillSource[] = []; const skills: SkillSource[] = [];
try { try {
// Get repository info for default branch const { octokit, token } = await this.getOctokit();
const octokit = await this.getOctokit();
const token = this.getCurrentToken();
const repoInfo = await octokit.repos.get({ owner, repo });
this.octokitPool.updateStats(token, repoInfo.headers);
if (repoInfo.data.archived) {
console.log(` Skipping archived repo: ${owner}/${repo}`);
return [];
}
const defaultBranch = repoInfo.data.default_branch;
// Get the full tree recursively
const tree = await octokit.git.getTree({ const tree = await octokit.git.getTree({
owner, owner,
repo, repo,
tree_sha: defaultBranch, tree_sha: branch,
recursive: 'true', recursive: 'true',
}); });
this.octokitPool.updateStats(token, tree.headers); this.octokitPool.updateStats(token, tree.headers);
// Find all instruction files (SKILL.md, .cursorrules, .windsurfrules, AGENTS.md, etc.) if (tree.data.truncated) {
// Truncated — skip this branch silently
return [];
}
const instructionFiles = tree.data.tree.filter( const instructionFiles = tree.data.tree.filter(
(item) => { (item) => {
if (item.type !== 'blob' || !item.path) return false; if (item.type !== 'blob' || !item.path) return false;
@@ -83,13 +223,81 @@ export class DeepScanCrawler {
owner, owner,
repo, repo,
path: skillPath, path: skillPath,
branch: defaultBranch, branch,
sourceFormat: matchedPattern.format, sourceFormat: matchedPattern.format,
}); });
} }
} catch (error) {
if (this.isNotFoundError(error)) return [];
// Rate limit and other errors propagate to scanRepository()
throw error;
}
return skills;
}
/**
* Deep scan a repository for all instruction files using Git Trees API.
* Scans multiple important branches (version, release, well-known names).
* Deduplicates skills across branches, preferring the default branch.
*/
async scanRepository(owner: string, repo: string, options: DeepScanOptions = {}): Promise<SkillSource[]> {
try {
const { octokit, token } = await this.getOctokit();
const repoInfo = await octokit.repos.get({ owner, repo });
this.octokitPool.updateStats(token, repoInfo.headers);
if (repoInfo.data.archived) {
console.log(` Skipping archived repo: ${owner}/${repo}`);
return [];
}
const defaultBranch = repoInfo.data.default_branch;
// Determine which branches to scan
let branchesToScan: string[];
if (options.allBranches) {
branchesToScan = await this.listAllBranches(owner, repo, defaultBranch);
} else {
branchesToScan = await this.listImportantBranches(
owner, repo, defaultBranch, options.extraBranchPatterns
);
}
if (branchesToScan.length > 1) {
console.log(` Scanning ${branchesToScan.length} branches: ${branchesToScan.join(', ')}`);
}
// Collect skills from each branch, deduplicating by path::format
const skillsByKey = new Map<string, SkillSource>();
for (const branch of branchesToScan) {
const branchSkills = await this.scanSingleBranch(owner, repo, branch);
for (const skill of branchSkills) {
const key = `${skill.path}::${skill.sourceFormat || 'skill.md'}`;
const existing = skillsByKey.get(key);
if (!existing) {
skillsByKey.set(key, skill);
} else if (existing.branch !== defaultBranch && skill.branch === defaultBranch) {
// Prefer default branch version
skillsByKey.set(key, skill);
}
// Otherwise keep existing (first non-default wins if default doesn't have it)
}
}
const skills = Array.from(skillsByKey.values());
if (skills.length > 0) { if (skills.length > 0) {
const nonDefault = skills.filter(s => s.branch !== defaultBranch);
if (nonDefault.length > 0) {
console.log(` Found ${skills.length} skills in ${owner}/${repo} (${nonDefault.length} on non-default branches)`);
} else {
console.log(` Found ${skills.length} skills in ${owner}/${repo}`); console.log(` Found ${skills.length} skills in ${owner}/${repo}`);
} }
}
return skills; return skills;
} catch (error) { } catch (error) {
@@ -97,30 +305,59 @@ export class DeepScanCrawler {
return []; return [];
} }
if (this.isTruncatedError(error)) { if (this.isTruncatedError(error)) {
// Tree is truncated (>100k files), fall back to directory listing
console.log(` Repository ${owner}/${repo} is too large, using fallback scan`); console.log(` Repository ${owner}/${repo} is too large, using fallback scan`);
return this.fallbackScan(owner, repo); return this.fallbackScanWithDefaultBranch(owner, repo);
}
if (this.isRateLimitError(error)) {
console.log(` Rate limit hit scanning ${owner}/${repo}, waiting for token rotation...`);
await this.tokenManager.checkAndRotate();
return this.scanRepository(owner, repo, options);
} }
throw error; throw error;
} }
} }
private isRateLimitError(error: unknown): boolean {
if (!(error instanceof Error)) return false;
const status = (error as { status?: number }).status;
if (status === 403 || status === 429) return true;
return error.message.includes('rate limit') || error.message.includes('secondary rate limit');
}
/** /**
* Fallback scan for large repositories - scan known skill directories * Wrapper: fetches defaultBranch then runs fallback scan.
* Used when getTree is truncated for repos with >100k files.
*/ */
private async fallbackScan(owner: string, repo: string): Promise<SkillSource[]> { private async fallbackScanWithDefaultBranch(owner: string, repo: string): Promise<SkillSource[]> {
let defaultBranch = 'main';
try {
const { octokit, token } = await this.getOctokit();
const repoInfo = await octokit.repos.get({ owner, repo });
this.octokitPool.updateStats(token, repoInfo.headers);
defaultBranch = repoInfo.data.default_branch;
} catch {
// Use 'main' as safe fallback
}
return this.fallbackScan(owner, repo, defaultBranch);
}
/**
* Fallback scan for large repositories - scan known skill directories.
* Uses the actual defaultBranch instead of hardcoding 'main'.
*/
private async fallbackScan(owner: string, repo: string, defaultBranch: string): Promise<SkillSource[]> {
const skills: SkillSource[] = []; const skills: SkillSource[] = [];
const knownPaths = ['skills', '.claude/skills', '.github/skills', '.codex/skills', '']; const knownPaths = ['skills', '.claude/skills', '.github/skills', '.codex/skills', ''];
for (const basePath of knownPaths) { for (const basePath of knownPaths) {
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.getContent({ const response = await octokit.repos.getContent({
owner, owner,
repo, repo,
path: basePath || '.', path: basePath || '.',
ref: defaultBranch,
}); });
this.octokitPool.updateStats(token, response.headers); this.octokitPool.updateStats(token, response.headers);
@@ -134,7 +371,7 @@ export class DeepScanCrawler {
owner, owner,
repo, repo,
path: basePath || '.', path: basePath || '.',
branch: 'main', branch: defaultBranch,
sourceFormat: 'skill.md' as SourceFormat, sourceFormat: 'skill.md' as SourceFormat,
}); });
} }
@@ -143,12 +380,12 @@ export class DeepScanCrawler {
const dirs = response.data.filter((item) => item.type === 'dir'); const dirs = response.data.filter((item) => item.type === 'dir');
for (const dir of dirs) { for (const dir of dirs) {
try { try {
const subOctokit = await this.getOctokit(); const { octokit: subOctokit, token: subToken } = await this.getOctokit();
const subToken = this.getCurrentToken();
const subDir = await subOctokit.repos.getContent({ const subDir = await subOctokit.repos.getContent({
owner, owner,
repo, repo,
path: dir.path, path: dir.path,
ref: defaultBranch,
}); });
this.octokitPool.updateStats(subToken, subDir.headers); this.octokitPool.updateStats(subToken, subDir.headers);
@@ -159,7 +396,7 @@ export class DeepScanCrawler {
owner, owner,
repo, repo,
path: dir.path, path: dir.path,
branch: 'main', branch: defaultBranch,
sourceFormat: 'skill.md' as SourceFormat, sourceFormat: 'skill.md' as SourceFormat,
}); });
} }
@@ -184,7 +421,7 @@ export class DeepScanCrawler {
owner, owner,
repo, repo,
path: '.', path: '.',
branch: 'main', branch: defaultBranch,
sourceFormat: pattern.format, sourceFormat: pattern.format,
}); });
} }
@@ -200,14 +437,15 @@ export class DeepScanCrawler {
* Scan multiple repositories * Scan multiple repositories
*/ */
async scanRepositories( async scanRepositories(
repos: Array<{ owner: string; repo: string }> repos: Array<{ owner: string; repo: string }>,
options: DeepScanOptions = {}
): Promise<Map<string, SkillSource[]>> { ): Promise<Map<string, SkillSource[]>> {
const results = new Map<string, SkillSource[]>(); const results = new Map<string, SkillSource[]>();
for (const { owner, repo } of repos) { for (const { owner, repo } of repos) {
try { try {
console.log(`Deep scanning: ${owner}/${repo}`); console.log(`Deep scanning: ${owner}/${repo}`);
const skills = await this.scanRepository(owner, repo); const skills = await this.scanRepository(owner, repo, options);
results.set(`${owner}/${repo}`, skills); results.set(`${owner}/${repo}`, skills);
} catch (error) { } catch (error) {
console.warn(` Failed to scan ${owner}/${repo}:`, error); console.warn(` Failed to scan ${owner}/${repo}:`, error);
@@ -223,8 +461,7 @@ export class DeepScanCrawler {
*/ */
async getFileContent(owner: string, repo: string, path: string): Promise<string | null> { async getFileContent(owner: string, repo: string, path: string): Promise<string | null> {
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.getContent({ const response = await octokit.repos.getContent({
owner, owner,
repo, repo,

View File

@@ -24,14 +24,10 @@ export class ForkNetworkCrawler {
this.octokitPool = new OctokitPool(this.tokenManager); this.octokitPool = new OctokitPool(this.tokenManager);
} }
private async getOctokit(): Promise<Octokit> { private async getOctokit(): Promise<{ octokit: Octokit; token: string }> {
return this.octokitPool.getBestInstance(); return this.octokitPool.getBestInstance();
} }
private getCurrentToken(): string {
return this.tokenManager.getBestToken();
}
/** /**
* Get all forks of a repository * Get all forks of a repository
*/ */
@@ -41,8 +37,7 @@ export class ForkNetworkCrawler {
while (page <= maxPages) { while (page <= maxPages) {
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.listForks({ const response = await octokit.repos.listForks({
owner, owner,
@@ -121,8 +116,7 @@ export class ForkNetworkCrawler {
*/ */
async getParentRepo(owner: string, repo: string): Promise<{ owner: string; repo: string } | null> { async getParentRepo(owner: string, repo: string): Promise<{ owner: string; repo: string } | null> {
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.repos.get({ owner, repo }); const response = await octokit.repos.get({ owner, repo });
this.octokitPool.updateStats(token, response.headers); this.octokitPool.updateStats(token, response.headers);
@@ -152,8 +146,7 @@ export class ForkNetworkCrawler {
// Add root repo // Add root repo
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const rootInfo = await octokit.repos.get({ owner: rootOwner, repo: rootRepo }); const rootInfo = await octokit.repos.get({ owner: rootOwner, repo: rootRepo });
this.octokitPool.updateStats(token, rootInfo.headers); this.octokitPool.updateStats(token, rootInfo.headers);

View File

@@ -7,6 +7,7 @@ import { createAwesomeListCrawler } from './awesome-list.js';
import { createTopicSearchCrawler } from './topic-search.js'; import { createTopicSearchCrawler } from './topic-search.js';
import { createDeepScanCrawler } from './deep-scan.js'; import { createDeepScanCrawler } from './deep-scan.js';
import { createForkNetworkCrawler } from './fork-network.js'; import { createForkNetworkCrawler } from './fork-network.js';
import { createCommitsSearchCrawler } from './commits-search.js';
import { TokenManager } from '../token-manager.js'; import { TokenManager } from '../token-manager.js';
import type { SkillSource } from 'skillhub-core'; import type { SkillSource } from 'skillhub-core';
@@ -41,6 +42,7 @@ export class StrategyOrchestrator {
private topicCrawler: ReturnType<typeof createTopicSearchCrawler>; private topicCrawler: ReturnType<typeof createTopicSearchCrawler>;
private deepScanCrawler: ReturnType<typeof createDeepScanCrawler>; private deepScanCrawler: ReturnType<typeof createDeepScanCrawler>;
private forkCrawler: ReturnType<typeof createForkNetworkCrawler>; private forkCrawler: ReturnType<typeof createForkNetworkCrawler>;
private commitsCrawler: ReturnType<typeof createCommitsSearchCrawler>;
private tokenManager: TokenManager; private tokenManager: TokenManager;
constructor(tokenManager?: TokenManager) { constructor(tokenManager?: TokenManager) {
@@ -49,6 +51,7 @@ export class StrategyOrchestrator {
this.topicCrawler = createTopicSearchCrawler(this.tokenManager); this.topicCrawler = createTopicSearchCrawler(this.tokenManager);
this.deepScanCrawler = createDeepScanCrawler(this.tokenManager); this.deepScanCrawler = createDeepScanCrawler(this.tokenManager);
this.forkCrawler = createForkNetworkCrawler(this.tokenManager); this.forkCrawler = createForkNetworkCrawler(this.tokenManager);
this.commitsCrawler = createCommitsSearchCrawler(this.tokenManager);
} }
/** /**
@@ -169,6 +172,32 @@ export class StrategyOrchestrator {
}; };
} }
/**
* Run commits search discovery (finds repos with recent SKILL.md commits)
*/
async runCommitsSearchStrategy(): Promise<DiscoveryResult> {
console.log('\n=== Running Commits Search Strategy ===');
const startTime = Date.now();
const repoResults = await this.commitsCrawler.discoverReposFromCommits(30);
const repos = repoResults.map((r) => ({
owner: r.owner,
repo: r.repo,
stars: r.stars,
discoveredVia: 'commits-search',
}));
console.log(`Commits search strategy completed in ${Date.now() - startTime}ms`);
console.log(`Discovered ${repos.length} repositories`);
return {
source: 'commits-search',
repos,
skills: [],
};
}
/** /**
* Run all strategies and return combined results * Run all strategies and return combined results
*/ */
@@ -230,7 +259,24 @@ export class StrategyOrchestrator {
console.error('Topic search strategy failed:', error); console.error('Topic search strategy failed:', error);
} }
// 3. Fork network (for high-star repos from above) // 3. Commits search (find repos with recent SKILL.md commits)
try {
const commitsResult = await this.runCommitsSearchStrategy();
stats.byStrategy['commits-search'] = {
repos: commitsResult.repos.length,
skills: 0,
};
for (const repo of commitsResult.repos) {
const key = `${repo.owner}/${repo.repo}`.toLowerCase();
if (!allRepos.has(key)) {
allRepos.set(key, repo);
}
}
} catch (error) {
console.error('Commits search strategy failed:', error);
}
// 4. Fork network (for high-star repos from above)
try { try {
const highStarRepos = Array.from(allRepos.values()) const highStarRepos = Array.from(allRepos.values())
.filter((r) => (r.stars || 0) >= 10) .filter((r) => (r.stars || 0) >= 10)
@@ -283,10 +329,15 @@ export function createStrategyOrchestrator(tokenManager?: TokenManager | string)
// Re-export strategy creators // Re-export strategy creators
export { createAwesomeListCrawler } from './awesome-list.js'; export { createAwesomeListCrawler } from './awesome-list.js';
export { createTopicSearchCrawler } from './topic-search.js'; export { createTopicSearchCrawler } from './topic-search.js';
export { createDeepScanCrawler } from './deep-scan.js'; export { createDeepScanCrawler, filterAndSortBranches } from './deep-scan.js';
export type { DeepScanOptions } from './deep-scan.js';
export { createForkNetworkCrawler } from './fork-network.js'; export { createForkNetworkCrawler } from './fork-network.js';
export { createPopularReposCrawler } from './popular-repos.js';
export { createCommitsSearchCrawler } from './commits-search.js';
// Re-export types // Re-export types
export type { RepoReference } from './awesome-list.js'; export type { RepoReference } from './awesome-list.js';
export type { RepoResult } from './topic-search.js'; export type { RepoResult } from './topic-search.js';
export type { ForkInfo } from './fork-network.js'; export type { ForkInfo } from './fork-network.js';
export type { PopularRepoResult } from './popular-repos.js';
export type { CommitSearchRepoResult } from './commits-search.js';

View File

@@ -0,0 +1,163 @@
import type { Octokit } from '@octokit/rest';
import { TokenManager } from '../token-manager.js';
import { OctokitPool } from '../octokit-pool.js';
export interface PopularRepoResult {
owner: string;
repo: string;
stars: number;
forks: number;
defaultBranch: string;
isArchived: boolean;
}
/**
* Popular Repos Discovery Strategy
* Discovers popular GitHub repos (by stars) and adds them to discovered_repos
* so deep-scan can check their branches for SKILL.md files.
*
* This catches repos that add SKILL.md on non-default branches
* which GitHub Code Search doesn't index.
*/
export class PopularReposCrawler {
private octokitPool: OctokitPool;
private tokenManager: TokenManager;
constructor(tokenManager?: TokenManager) {
this.tokenManager = tokenManager || TokenManager.getInstance();
this.octokitPool = new OctokitPool(this.tokenManager);
}
private async getOctokit(): Promise<{ octokit: Octokit; token: string }> {
return this.octokitPool.getBestInstance();
}
/**
* Discover popular repos by star count ranges.
* Segments by star ranges to bypass GitHub's 1000-result-per-query limit.
*/
async discoverPopularRepos(minStars: number = 1000): Promise<PopularRepoResult[]> {
const starRanges = this.buildStarRanges(minStars);
const allRepos = new Map<string, PopularRepoResult>();
console.log(`Discovering popular repos (${minStars}+ stars) across ${starRanges.length} ranges...`);
for (const range of starRanges) {
try {
console.log(`\n Searching: stars:${range}`);
const repos = await this.searchByStarRange(range);
for (const repo of repos) {
const key = `${repo.owner}/${repo.repo}`.toLowerCase();
if (!allRepos.has(key)) {
allRepos.set(key, repo);
}
}
console.log(` Found ${repos.length} repos (total unique: ${allRepos.size})`);
} catch (error) {
console.warn(` Failed for range stars:${range}:`, error instanceof Error ? error.message : error);
}
}
console.log(`\nTotal unique repos discovered: ${allRepos.size}`);
return Array.from(allRepos.values());
}
/**
* Build star ranges to segment search queries.
* GitHub search returns max 1000 results per query.
*/
private buildStarRanges(minStars: number): string[] {
const ranges: string[] = [];
const breakpoints = [500, 1000, 2000, 5000, 10000, 25000, 50000, 100000];
// Filter breakpoints based on minStars
const relevantBreaks = breakpoints.filter(b => b >= minStars);
for (let i = 0; i < relevantBreaks.length; i++) {
const low = i === 0 ? minStars : relevantBreaks[i - 1];
const high = relevantBreaks[i];
if (low < high) {
ranges.push(`${low}..${high}`);
}
}
// Add the final "greater than" range
const lastBreak = relevantBreaks[relevantBreaks.length - 1] || minStars;
ranges.push(`>${lastBreak}`);
return ranges;
}
/**
* Search repos within a specific star range.
*/
private async searchByStarRange(starRange: string): Promise<PopularRepoResult[]> {
const results: PopularRepoResult[] = [];
const maxPages = 10;
const perPage = 100;
for (let page = 1; page <= maxPages; page++) {
try {
const { octokit, token } = await this.getOctokit();
const response = await octokit.search.repos({
q: `stars:${starRange}`,
sort: 'stars',
order: 'desc',
per_page: perPage,
page,
});
this.octokitPool.updateStats(token, response.headers);
for (const repo of response.data.items) {
if (!repo.archived) {
results.push({
owner: repo.owner!.login,
repo: repo.name,
stars: repo.stargazers_count,
forks: repo.forks_count,
defaultBranch: repo.default_branch,
isArchived: repo.archived,
});
}
}
if (response.data.items.length < perPage) break;
} catch (error) {
if (this.isBeyondResultsLimit(error)) {
console.log(` Reached 1000-result limit for stars:${starRange}`);
break;
}
if (this.isRateLimitError(error)) {
console.log(` Rate limit hit, rotating token...`);
await this.tokenManager.checkAndRotate();
page--; // Retry same page
continue;
}
throw error;
}
}
return results;
}
private isRateLimitError(error: unknown): boolean {
if (!(error instanceof Error)) return false;
const status = (error as { status?: number }).status;
return status === 403 || status === 429;
}
private isBeyondResultsLimit(error: unknown): boolean {
return (
error instanceof Error &&
'status' in error &&
(error as { status: number }).status === 422
);
}
}
export function createPopularReposCrawler(tokenManager?: TokenManager | string): PopularReposCrawler {
if (typeof tokenManager === 'string') {
return new PopularReposCrawler(new TokenManager([tokenManager]));
}
return new PopularReposCrawler(tokenManager);
}

View File

@@ -96,14 +96,10 @@ export class TopicSearchCrawler {
this.octokitPool = new OctokitPool(this.tokenManager); this.octokitPool = new OctokitPool(this.tokenManager);
} }
private async getOctokit(): Promise<Octokit> { private async getOctokit(): Promise<{ octokit: Octokit; token: string }> {
return this.octokitPool.getBestInstance(); return this.octokitPool.getBestInstance();
} }
private getCurrentToken(): string {
return this.tokenManager.getBestToken();
}
/** /**
* Search by all known skill-related topics * Search by all known skill-related topics
*/ */
@@ -117,8 +113,7 @@ export class TopicSearchCrawler {
try { try {
console.log(` Searching topic: ${topic}`); console.log(` Searching topic: ${topic}`);
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.search.repos({ const response = await octokit.search.repos({
q: `topic:${topic}`, q: `topic:${topic}`,
sort: 'stars', sort: 'stars',
@@ -175,8 +170,7 @@ export class TopicSearchCrawler {
try { try {
console.log(` Query: ${query}`); console.log(` Query: ${query}`);
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.search.repos({ const response = await octokit.search.repos({
q: query, q: query,
sort: 'stars', sort: 'stars',
@@ -232,8 +226,7 @@ export class TopicSearchCrawler {
for (let page = 2; page <= maxPages; page++) { for (let page = 2; page <= maxPages; page++) {
try { try {
const octokit = await this.getOctokit(); const { octokit, token } = await this.getOctokit();
const token = this.getCurrentToken();
const response = await octokit.search.repos({ const response = await octokit.search.repos({
q: query, q: query,

Some files were not shown because too many files have changed in this diff Show More