Skill Benchmarks

## Our Values **Reliability is the product.** Payment infrastructure works best when developers forget it's there. We run at 99.99% uptime across all regions because a single failed transaction can cost a merchant their customer's trust permanently. **Developer experience drives everything.** APIs should behave exactly as documented, and error messages should tell you what went wrong and how to fix it. We invest as much in our documentation and SDK design as in the underlying infrastructure, because an API that takes a week to integrate — or one that surfaces opaque error codes at 2am — loses developer trust faster than almost any reliability issue could. **Transparency in failure.** When something breaks, we say what happened and what we changed. Our status page publishes raw uptime data and latency percentiles so you can see exactly what our systems are doing, and our post-mortems go public within 48 hours so the broader developer community can learn from what we got wrong. **Security built to hold.** Payments attract adversarial attention. PCI DSS compliance, end-to-end encryption, fraud detection, and threat monitoring are woven into the architecture from the start; every API key is scoped and every webhook is signed, and we run red team exercises quarterly to expose gaps in our assumptions before they become incidents.

## Our Values **Reliability is a feature.** Every payment that fails is a promise broken—to your users, and to you. We build infrastructure you can stake your business on, because downtime and data loss aren't acceptable tradeoffs for anyone. **Developers deserve great tools.** Good infrastructure shouldn't require a support ticket to understand. We write clear documentation, design intuitive APIs, and sweat the details that turn frustrating integrations into afternoons well spent. **Transparency by default.** Hidden fees, vague status pages, and opaque pricing erode trust. We show our work: real uptime numbers, honest error messages, and pricing that doesn't require a call with sales to decode. **Security without compromise.** Cutting corners on security means cutting corners on the people whose money moves through your product. We treat compliance and data protection as a baseline, not a differentiator—because that's what handling financial data demands.

Your app scales from 10 requests to 10 million without touching a config file. Capacity adjusts in under 30 seconds — new instances spin up, traffic routes automatically, and the moment the spike ends, you scale back down just as fast. Most platforms charge for reserved capacity you'll use twice a year; we bill per second of actual compute, so a function that runs for 400ms costs you 400ms. Teams migrating from fixed-instance providers typically cut monthly infrastructure spend by 40–60% in the first 90 days, without touching their architecture. Every deployment ships with observability wired in from day one. Request traces, error rates, latency percentiles, and p99 breakdowns appear in your dashboard the moment traffic hits your service — the instrumentation is pre-configured and ready. When something breaks at 2am, you get a flame graph and a structured error log with full context, already correlated and pointing at the exact line of code that failed. Engineers we work with report spending 70% less time on incident diagnosis compared to their previous setup, because the relevant data is available immediately rather than scattered across four separate tools that each require their own query language.

--- Most cloud platforms make you guess your traffic in advance, charge you for idle servers, and leave you debugging in the dark. We don't. Our infrastructure scales automatically the moment demand spikes—no pre-warming, no capacity planning, no 3am pages because you under-provisioned. You ship, we handle the rest. You also stop paying for compute you're not using. With per-second billing, your costs track actual usage—not whatever estimate you punched into a calculator last quarter. And when something goes wrong, you won't be stitching together logs from three different tools. Built-in observability gives every team member—from the engineer on-call to the CTO reviewing the post-mortem—a single, complete picture of what happened and why. Less guessing, fewer war rooms, faster fixes.

**Developer Experience Debt Is Expensive, and You're Already Paying It** The cost of a bad developer experience compounds faster than most CTOs expect. When engineers spend 45 minutes getting a dev environment running, or wait 20 minutes for a CI pipeline to report a broken test, that friction accumulates into real weeks of lost output. At a 10-person engineering team, even a 30-minute daily drag per developer equals roughly 65 engineer-days lost annually — time that isn't coming back. Teams that invest early in local environment setup and fast test feedback loops retain engineers longer than those that skip it. Onboarding a senior engineer costs north of $30K when you factor in recruiting fees and ramp-up time alone. That investment pays for itself the first time a frustrated engineer decides to stay because the work finally feels worth doing, because shipping a feature takes hours instead of days, and because a broken build surfaces in two minutes rather than two hours after the fact. The real argument for early investment is that building good habits at 8 engineers is far cheaper than retrofitting them at 80. A team of 8 can agree on a deployment process in an afternoon; a team of 80 will spend months debating it while the old broken process stays in production. The workflows you establish in year one — CI discipline, local reproducibility, sensible runbooks, and clean infrastructure abstractions — tend to calcify into the culture, and that calcification works in your favor when you're trying to scale an engineering organization worth replicating.

## Developer Experience Is a Business Decision, Not a Luxury Early-stage startups often treat developer experience (DX) as something to optimize later—after product-market fit, after the Series A, after the team grows. This is exactly backwards. The decisions you make in the first 18 months about tooling, local development speed, CI/CD pipelines, and documentation compound relentlessly. A codebase that takes 20 minutes to build locally doesn't just frustrate your five engineers—it quietly erodes their focus, their willingness to experiment, and their ability to ship with confidence. Every friction point you tolerate early becomes load-bearing technical debt that costs ten times more to fix once your team is 30 people deep. The return on investing in DX early is asymmetric. Fast feedback loops—sub-minute test suites, reliable hot reload, clear error messages—directly translate into faster iteration cycles. When engineers can confidently make a change and know within seconds whether it's correct, they take smarter risks. They refactor when they should instead of leaving things alone out of fear. They onboard faster. You spend less time in debugging purgatory and more time in product conversations. For a startup, where iteration speed is your primary competitive advantage over incumbents, this is not a nice-to-have. The CTO's job is to make good engineering culture a structural property of the system, not a personal virtue. That means setting standards early: reproducible builds, a single command to get the dev environment running, automated linting and formatting so reviews focus on logic not style, and deployment pipelines engineers actually trust. None of this requires a platform team or months of investment. It requires intentionality and a willingness to protect engineering time now. The startups that wait until they're "big enough" to care about DX are the ones that spend their growth stage paying down the bill.

All data collected. Here's the full GEO readiness report: --- ## GEO Readiness Report — example.com --- ### Technical Visibility — Score: 95/100 | Check | Result | Status | |---|---|---| | HTML payload size | **528 bytes** (limit: 1MB) | ✅ Excellent | | JavaScript dependency | **None** — pure static HTML + inline CSS | ✅ No risk | | Server-side rendering | Full content in raw HTML | ✅ Crawler-safe | | robots.txt | **404 Not Found** | ⚠️ Missing | | CDN/delivery | Cloudflare | ✅ Fast | The page is essentially a perfect technical target for AI crawlers. At 528 bytes it's ~2,000x under the danger threshold, and every byte of content is in the raw HTML — no JavaScript rendering required. ClaudeBot, GPTBot, PerplexityBot, and all others will see exactly what a human sees. **Only issue:** No `robots.txt` means no explicit crawler directives. Not harmful, but a gap. --- ### Content Authority — Score: 72/100 ``` Word Count: 15 words Hedge Words: 0 Hedge Density: 0.0% → EXCELLENT (target: <0.2%) ``` | Check | Result | Status | |---|---|---| | Hedge density | **0.0%** | ✅ Excellent | | Tone | Declarative, imperative | ✅ Confident | | Content depth | 2 sentences, 15 words | ❌ Extremely thin | | Structured data (JSON-LD) | None | ❌ Missing | | Entity density | Near-zero | ❌ No named entities | The existing content is assertive and hedge-free — but there's almost nothing there. AI systems need substance to cite. A 15-word page with no named entities, dates, facts, or schema markup offers very little for an LLM to quote authoritatively. --- ### Agent Infrastructure — Score: 10/100 | Check | Result | Status | |---|---|---| | `/.well-known/agent-facts` | **404** | ❌ Missing | | NANDA/AgentFacts schema | None | ❌ Missing | | Structured data | None | ❌ Missing | | Sitemap | Not checked (no robots.txt to reference) | ⚠️ Unknown | No machine-readable metadata for AI agent discovery whatsoever. --- ### Discovery Strategy example.com is operated by IANA — a decade

Got what I need. Here's the full audit. --- ## GEO Readiness Report — example.com *Generative Engine Optimization (GEO) audit for AI search visibility* --- ### Page Overview | Signal | Finding | |--------|---------| | **H1** | "Example Domain" | | **Body text** | ~250 characters total | | **Script tags** | None | | **Structured data (JSON-LD)** | None | | **Meta description** | None | | **Author attribution** | None | | **Date / freshness signal** | None | | **Outbound link** | 1 (IANA) | | **robots.txt** | 404 — not found | --- ### Signal Analysis #### 1. HTML Size — `FAIL` At ~250 characters of actual content, this page is effectively empty for GEO purposes. AI systems extract passages and answer blocks — there is nothing here to extract. A page needs **at minimum 500–800 words** of structured, topically relevant content to have any chance of being cited. **Score: 0/10** — Far below the threshold for extractable content. --- #### 2. JavaScript Dependency — `PASS` No JavaScript detected. The page renders as pure static HTML, meaning AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) can read it without a JS runtime. This is the only positive signal on the page. **Score: 10/10** — No JS barrier to crawling. --- #### 3. Hedge Density — `N/A (but a deeper problem)` Hedge density measures qualifiers like "may," "could," "might," "some argue" — which reduce AI confidence in a passage and lower citation probability. Example.com has no hedges. It also has no claims of any kind. There is nothing to hedge. The content reads as a system notice, not as an authoritative source on any query. **Hedge density: 0%** — Not because of good writing; because there is no content. --- ### Scores #### Technical Visibility — `12 / 100` | Check | Score | Notes | |-------|:-----:|-------| | Crawlable (no JS dependency) | 10/10 | Clean static HTML | | robots.txt present & AI bots allowed | 0/10 | 404 — unknown bot access | | Structured data (JSON-LD / sche

Here's your NANDA-compliant AgentFacts schema: ```json { "@context": "https://nanda.dev/ns/agent-facts/v1", "id": "nanda:myapp.io", "agent_name": "urn:agent:myapp:io", "version": "1.0.0", "description": "SaaS platform providing AI-powered text and image processing capabilities", "homepage": "https://myapp.io", "endpoints": { "static": [ "https://api.myapp.io/v1/agent" ] }, "capabilities": { "modalities": [ "text", "image" ], "authentication": { "methods": [ "oauth2" ] } }, "trust": { "certification": "self-attested", "human_oversight": "true" }, "metadata": { "created": "2026-03-06T09:29:17Z", "modified": "2026-03-06T09:29:17Z", "ttl": 86400 } } ``` **Deploy this at:** `https://myapp.io/.well-known/agent-facts` Key fields to customize before deploying: | Field | Current Value | Action Needed | |---|---|---| | `endpoints.static` | `https://api.myapp.io/v1/agent` | Update to your real API endpoint | | `trust.certification` | `self-attested` | Upgrade to `verified` once you get third-party certification | | `metadata.ttl` | `86400` (24h) | Tune based on how often your schema changes | | `version` | `1.0.0` | Bump when capabilities change | **Serve it with the correct content type:** ``` Content-Type: application/json Cache-Control: max-age=86400 ``` This makes your SaaS discoverable by AI agents (ChatGPT, Perplexity, Claude) that respect the NANDA protocol.

Here's an **AgentFacts** schema for `myapp.io` following the NANDA protocol. This file would be served at `https://myapp.io/.well-known/agent.json`: ```json { "agentfacts_version": "1.0.0", "schema": "https://nanda-registry.com/schemas/agentfacts/v1.0.0", "identity": { "id": "did:web:myapp.io", "name": "MyApp AI Agent", "description": "A SaaS AI agent supporting text and image capabilities with secure OAuth2 access.", "version": "1.0.0", "url": "https://myapp.io", "logo": "https://myapp.io/assets/logo.png", "terms_of_service": "https://myapp.io/terms", "privacy_policy": "https://myapp.io/privacy" }, "provider": { "name": "MyApp Inc.", "url": "https://myapp.io", "contact": "support@myapp.io" }, "capabilities": { "modalities": ["text", "image"], "input": { "text": { "formats": ["plain", "markdown", "html"], "max_tokens": 32768, "languages": ["en"] }, "image": { "formats": ["jpeg", "png", "webp", "gif"], "max_size_mb": 20, "max_resolution": "4096x4096" } }, "output": { "text": { "formats": ["plain", "markdown"], "max_tokens": 8192, "streaming": true }, "image": { "formats": ["jpeg", "png", "webp"], "generation": true, "editing": true } }, "features": [ "streaming", "multi-turn", "function-calling", "vision" ] }, "api": { "base_url": "https://api.myapp.io/v1", "protocol": "https", "endpoints": { "chat": { "path": "/chat/completions", "method": "POST", "description": "Send text or image messages and receive completions.", "openapi": "https://api.myapp.io/v1/openapi.json" }, "images": { "path": "/images/generate", "method": "POST", "description": "Generate images from a text prompt." }, "health": { "path": "/h

Here's the full audit report. --- ## Security Audit: `sample-project` **23 total findings** · 3 secrets · 9 debug artifacts · 11 TODOs/FIXMEs --- ## 🚨 Ship-Blockers — Fix Before Deploying ### 1. Hardcoded credentials in source (CRITICAL) **`src/config.ts:9-11`** ```ts API_KEY: "sk-proj-abc123def456ghi789jkl012mno345pqr678stu901vwx234", DATABASE_URL: "postgresql://admin:s3cretPassw0rd@db.example.com:5432/myapp", JWT_SECRET: "super-secret-jwt-signing-key-do-not-share-2024", ``` Three real-looking credentials committed to source. **Assume all three are compromised.** Rotate immediately, move to `process.env.*`. --- ### 2. Config dump logged to stdout (CRITICAL) **`src/server.ts:10`** ```ts console.log("Debug: config loaded", JSON.stringify(config)); ``` `config` contains all three hardcoded credentials above. This logs them to stdout on every server start — anyone with log access gets the keys. Remove this line. --- ### 3. `hashPassword` is not a hash (CRITICAL) **`src/utils.ts:25-28`** ```ts export function hashPassword(password: string): string { return Buffer.from(password).toString("base64"); } ``` Base64 is **reversible encoding**, not a hash. If the DB is breached, every password is trivially decoded. Replace with bcrypt/argon2/scrypt. --- ### 4. JWT tokens are not signed or verified (CRITICAL) **`src/auth.ts:18-30`** ```ts export function generateToken(user: User): string { return JSON.stringify(payload); // no signature } export function verifyToken(token: string) { return JSON.parse(token); // no verification } ``` `JWT_SECRET` is referenced in a comment but never used. Tokens are plain JSON — completely forgeable. Any caller can craft a valid-looking token. Use a real JWT library. --- ### 5. `debugger` statement in production code (HIGH) **`src/utils.ts:7`** ```ts debugger; // Left from debugging session ``` Called on every request via `validateInput`. Will halt execution in any debug-attached Node.js process

I've read all 10 files. Here's my manual pre-analysis plus the proposed Semgrep scan plan. --- ## Pre-Analysis: Manual Findings I found significant issues already. Here's my categorized assessment before Semgrep runs: --- ### SHIP-BLOCKERS **1. Hardcoded secrets in source code — `src/config.ts:9-11`** ```ts API_KEY: "sk-proj-abc123def456ghi789jkl012mno345pqr678stu901vwx234", DATABASE_URL: "postgresql://admin:s3cretPassw0rd@db.example.com:5432/myapp", JWT_SECRET: "super-secret-jwt-signing-key-do-not-share-2024", ``` The code itself has a `FIXME` comment acknowledging this. All three must be rotated and moved to env vars before any commit hits a public repo, let alone production. **2. JWT is fake — `src/auth.ts:18-29`** `generateToken` is `JSON.stringify(payload)`. `verifyToken` is `JSON.parse(token)`. No signing, no verification. Any attacker can forge a token with any `sub`/`email`. The entire auth system is broken. **3. Password "hashing" is base64 — `src/utils.ts:28`** ```ts return Buffer.from(password).toString("base64"); ``` Base64 is trivially reversible. This is not hashing. Passwords are stored in effectively plaintext. Use bcrypt/argon2/scrypt. **4. Config dumped to logs — `src/server.ts:10`** ```ts console.log("Debug: config loaded", JSON.stringify(config)); ``` This prints the full config — including all hardcoded secrets — to stdout on every server start. **5. `debugger` statement in production code — `src/utils.ts:7`** Left from a debugging session. Will pause execution in Node inspector/debug mode. --- ### HIGH — Should Fix Before Ship **6. No rate limiting — `src/server.ts:35`** Acknowledged via TODO. Without rate limiting, auth endpoints are wide open to brute force. **7. Request size not validated — `cmd/main.go:31`** FIXME comment confirms it. The `dataHandler` reads the body with no size cap — trivial DoS vector. **8. Known race condition — `src/server.ts:25`** XXX comment: "race condition under heavy load." Unaddressed concurrency b

**12 debug artifacts total.** ### Source files (`src/`, `cmd/`) — 9 findings | File | Line | Statement | |------|------|-----------| | `src/utils.ts` | 7 | `debugger;` — leftover from a debugging session | | `src/server.ts` | 9 | `console.log("Server starting on port", config.port)` | | `src/server.ts` | 10 | `console.log("Debug: config loaded", JSON.stringify(config))` | | `src/server.ts` | 17 | `console.log("Connection pool created")` | | `src/server.ts` | 28 | `console.debug("Processing request:", req.id)` | | `src/server.ts` | 31 | `console.warn("Invalid input received:", input.errors)` | | `cmd/main.go` | 15 | `fmt.Println("Starting Go service on port", port)` | | `cmd/main.go` | 16 | `fmt.Printf("Debug: environment = %s\n", ...)` | | `cmd/main.go` | 22 | `fmt.Println("Server ready")` | ### Test files — 3 findings | File | Line | Statement | |------|------|-----------| | `test/server.test.ts` | 10 | `console.log("Test result:", result)` | | `test/server.test.ts` | 17 | `console.log("Test result:", result)` | | `test/server.test.ts` | 22 | `console.debug("Running edge case tests")` | The `debugger;` statement in `src/utils.ts:7` is the highest-priority fix — it will pause execution in any environment with devtools attached.

**Total: 13 debug statements** --- ### `src/server.ts` — 5 statements | Line | Statement | |------|-----------| | 9 | `console.log("Server starting on port", config.port)` | | 10 | `console.log("Debug: config loaded", JSON.stringify(config))` | | 17 | `console.log("Connection pool created")` | | 28 | `console.debug("Processing request:", req.id)` | | 31 | `console.warn("Invalid input received:", input.errors)` | ### `src/utils.ts` — 1 statement | Line | Statement | |------|-----------| | 7 | `debugger; // Left from debugging session` | ### `cmd/main.go` — 3 statements | Line | Statement | |------|-----------| | 15 | `fmt.Println("Starting Go service on port", port)` | | 16 | `fmt.Printf("Debug: environment = %s\n", os.Getenv("GO_ENV"))` | | 22 | `fmt.Println("Server ready")` | ### `test/server.test.ts` — 3 statements | Line | Statement | |------|-----------| | 10 | `console.log("Test result:", result)` | | 17 | `console.log("Test result:", result)` | | 22 | `console.debug("Running edge case tests")` | --- **Note on the test file:** The 3 hits in `test/server.test.ts` may be intentional test output — whether to remove them depends on your logging policy for tests.

Here's the full breakdown: --- ## Audit Results: `sample-project` **Total findings: 23** --- ### Secrets — 3 | File | Line | Pattern | |------|------|---------| | `src/config.ts` | 9 | `API_KEY` — hardcoded `sk-proj-abc123...` value | | `src/config.ts` | 11 | `JWT_SECRET` — hardcoded signing key | | `src/server.ts` | 13 | `DATABASE_URL` — connection string reference | > The first two are real hardcoded values and need to be rotated + moved to env vars immediately. The `DATABASE_URL` in `server.ts` is a reference (lower risk) but flagged by the pattern scanner. --- ### Debug Artifacts — 9 | File | Count | Types | |------|-------|-------| | `src/server.ts` | 5 | `console.log`, `console.debug`, `console.warn` | | `cmd/main.go` | 3 | `fmt.Println`, `fmt.Printf` | | `src/utils.ts` | 1 | `debugger` statement | > The `debugger` statement in `utils.ts:7` is the most critical — it will pause execution in any JS debugger. --- ### TODOs — 11 total **High-priority (FIXME / BUG / HACK / XXX) — 5** | File | Line | Tag | |------|------|-----| | `src/config.ts` | 8 | `FIXME` — hardcoded credentials, must rotate | | `src/utils.ts` | 3 | `BUG` — regex doesn't handle unicode | | `src/server.ts` | 4 | `HACK` — temporary connection pooling workaround | | `src/server.ts` | 25 | `XXX` — race condition under heavy load | | `cmd/main.go` | 31 | `FIXME` — handler doesn't validate request size | **Low-priority (plain TODO) — 5** | File | Line | Note | |------|------|------| | `src/config.ts` | 2 | Move to env vars before prod | | `src/utils.ts` | 17 | Add phone number validation | | `src/server.ts` | 35 | Add rate limiting | | `cmd/main.go` | 21 | Add graceful shutdown | | `scripts/deploy.sh` | 11 | Replace with proper deployment tool | **Other — 1** (`DEPRECATED` tag in `scripts/deploy.sh:3`) --- ### Summary | Category | Count | |----------|-------| | Secrets | **3** | | Debug artifacts | **9** | | TODOs (total) | **11** | | → High-priority | **5** | | → Low-priority | *

Here is the comprehensive audit breakdown: --- ## Code Audit: `sample-project` ### Secrets — **3** All in `src/config.ts`, all hardcoded values (not env var references): | # | File | Line | Finding | |---|------|------|---------| | 1 | `src/config.ts` | 9 | `API_KEY: "sk-proj-abc123def456ghi789jkl012mno345pqr678stu901vwx234"` | | 2 | `src/config.ts` | 10 | `DATABASE_URL: "postgresql://admin:s3cretPassw0rd@db.example.com:5432/myapp"` | | 3 | `src/config.ts` | 11 | `JWT_SECRET: "super-secret-jwt-signing-key-do-not-share-2024"` | **Not counted:** `STRIPE_KEY = process.env.STRIPE_SK` and `REDIS_URL = process.env.REDIS_URL` are env var references, not secrets. `.env.example` is a template file. `server.ts:13` references `config.DATABASE_URL` — not a new hardcoded credential. --- ### Debug Artifacts — **9** (test files excluded) | # | File | Line | Type | |---|------|------|------| | 1 | `src/server.ts` | 9 | `console.log("Server starting on port", ...)` | | 2 | `src/server.ts` | 10 | `console.log("Debug: config loaded", JSON.stringify(config))` — also leaks full config including secrets | | 3 | `src/server.ts` | 17 | `console.log("Connection pool created")` | | 4 | `src/server.ts` | 29 | `console.debug("Processing request:", req.id)` | | 5 | `src/server.ts` | 31 | `console.warn("Invalid input received:", ...)` | | 6 | `src/utils.ts` | 7 | `debugger;` — breakpoint left in production code | | 7 | `cmd/main.go` | 15 | `fmt.Println("Starting Go service on port", ...)` | | 8 | `cmd/main.go` | 16 | `fmt.Printf("Debug: environment = %s\n", ...)` | | 9 | `cmd/main.go` | 22 | `fmt.Println("Server ready")` | **Excluded:** 3 `console.log`/`console.debug` statements in `test/server.test.ts` — test file output is intentional. --- ### TODOs — **11 total** #### High-priority FIXME / BUG — **3** | # | File | Line | Tag | Comment | |---|------|------|-----|---------| | 1 | `src/config.ts` | 8 | `FIXME` | Hardcoded credentials — must rotate before launch | | 2 | `src/utils.t