Public evaluation

This is what visitors to your repository can see.

Latest score

78/100

Commit
e391c68
Method
v1.1.0
Reviewed

Six axes

Grades out of 10 at commit e391c68, method v1.1.0.

  • architecture 8/10
  • performance 7/10 weakest axis
  • consistency 8/10
  • security 8/10
  • documentation 8/10
  • correctness 8/10

Findings by detectability

Defects

tool-detectable
5
found by review
4

Check gaps

check gaps
7

Check gaps count quality checks that are missing, not run, failing or too loose; they are not added to the defects above. 18 review notes are not defects and are not counted.

Findings by axis

Finding counts per axis for the latest evaluation
Axistool-detectablefound by reviewcheck gaps
architecture000
performance2 performance findings, tool-detectable01 performance findings, check gaps
consistency002 consistency findings, check gaps
security3 security findings, tool-detectable2 security findings, found by review4 security findings, check gaps
documentation000
correctness02 correctness findings, found by review0

In this sample, counts link to the detailed findings below. On public evaluations, only the counts are visible. Finding details remain private.

Badge

KwalitAI score badge of this sample evaluation

The badge of this sample evaluation, as the repository README shows it.

Private audit report

This is what you receive with the paid audit.

This report of arhuman/kwalitai, the repository behind this site, is published with the owner's consent. Its text is the report's; the status of each finding comes from the follow-up review review_kwalitai_20261017_4 at commit 9f959fe.

Summary

Axis grades of the review
AxisGrade
architecture8/10
performance7/10
consistency8/10
security8/10
documentation8/10
correctness8/10
Overall8/10

Synthesis

The codebase is uniformly good, and the one movement since review_kwalitai_20261017_2 is real. Commit 9f47c79 made the complexity gate measure what the thresholds say, and the five cyclomatic-11 functions came down to 10 or below, so Consistency rises from 7 to 8.

Performance is the weakest axis at 7. That is not because of a slow path: the directory query is unbounded and production cannot be measured.

The single biggest operational risk is new and cheap to close. make up will start production without TRUSTED_PROXIES, and then every visitor shares one contact-form limit, so one sender can lock the sales contact channel.

The biggest process risk is recurrence: 14 of the 16 defect and gate rows in this review were raised in earlier reviews, and each is a missing detector, not a missing fix.

Confidence limits: branch protection could not be read (there is no origin remote), so every wired CI job is treated as blocking. The production .env was not inspected. The new ingest finding is confirmed by reading the code, not by execution. semgrep and tree-sitter did not run, so the resource-safety and structural checks rest on reading the source.

Recommendations

In priority order.

  1. Medium Wire the detectors behind today's 5 tool-detectable defects. (a) Add a TRUSTED_PROXIES check to make preflight (Makefile:106-113): without it, production can start with every visitor sharing one contact limit. (b) Add the shared semgrep pack, at least go.sql-select-no-limit and go.query-in-loop, to the repo's semgrep config, and run it from make audit (Makefile:53). (c) Pin all 19 uses: to commit SHAs and add zizmor .github/workflows to the CI lint job (ci.yml:43). (Security, Performance, Architecture)
  2. Medium Reject or warn when an identical report is re-ingested under another repository. In existingEvaluation (internal/ingest/ingest.go:181), compare the found row's repo_id with the requested repository. Today the operator is told "already ingested", and the report stays attached to the wrong repository. (Correctness)
  3. Medium Cover the production ingest wiring: TestServeIngestEndToEnd (POST /ingest through poolIngester, serve.go:101, now 0%) and TestRunIngest (main.go:261, now 33.3%). A mistake between serve and ingest.Run passes make ci today. (Correctness)
  4. Medium Give the contact limiter a global ceiling, and key IPv6 clients on their /64 (internal/web/contact.go:167). (Security)
  5. Medium Log the repo value only after the token check, or truncate it to 200 bytes (internal/web/ingest.go:46, publish.go:35). (Security)
  6. Medium Make formatting and error checking failing gates: the gofmt formatter, errcheck and unused in .golangci.yml:13, and drop -U1000 at :43. Expected new findings: 0. (Consistency)
  7. Medium Add gitleaks detect --redact to make audit (Makefile:53). Nothing stops a committed credential today. (Security)
  8. Medium In storeReport (internal/ingest/report.go:83), fsync before the rename, and verify an existing copy against report_sha256 instead of trusting os.Stat. (Correctness)
  9. Medium Add a page size to PGStore.Directory (internal/web/store.go:94) and a request logger in newRouter (cmd/kwalitai/serve.go:33). (Performance)
  10. Medium Replace the template text in SECURITY.md:5-6 and CONTRIBUTING.md:3 with the project's actual policy. (Documentation)
  11. Low Add FuzzUnpack seeded from TestUnpackRejects (internal/ingest/archive_test.go). (Performance, Security)

Findings

Every finding the report counts as a defect or a check gap; review notes that are not defects are left out. Status at the follow-up review: 15 of 16 findings fixed, 1 still open. 1 security finding still open at the follow-up review is withheld: a Security finding is shown only once fixed.

Performance

Severity
Medium
Axis
performance
Class
check gap
Location
Makefile:53
Finding
recurrence since review_kwalitai_20261016, review_kwalitai_20261017 and review_kwalitai_20261017_2: shared semgrep pack absent from make audit and CI, semgrep not installed. RC11, RC13, RC15 to RC19 and RC24 rules never run; 2 live instances found by reading (ingest.go:207, store.go:94)
Cause
The shared semgrep rules for queries in loops and unbounded selects are not configured, so those shapes are found only by reading.
Recommendation
Add the semgrep rules to the repo and run them from make audit and CI (recommendation 1b).
Status
Fixed in 840cdcc
Severity
Medium
Axis
performance
Class
tool-detectable
Location
internal/ingest/ingest.go:207
Finding
recurrence since review_kwalitai_20261016, review_kwalitai_20261017 and review_kwalitai_20261017_2: one INSERT per finding_counts key inside a loop in the ingest transaction (bounded by axis x severity x class, at most 7x4x6 = 168 rows)
Cause
Finding counts are inserted one row per statement inside a loop in the ingest transaction.
Recommendation
Insert every finding count in one statement (recommendation 1b names the rule that catches it).
Status
Fixed in 39ac874
Severity
Medium
Axis
performance
Class
tool-detectable
Location
internal/web/store.go:94
Finding
recurrence since review_kwalitai_20261017 and review_kwalitai_20261017_2: Directory selects every published repository with no LIMIT and GET /r renders all rows, so page size and query time grow with the catalogue
Cause
The directory query selects every published repository with no LIMIT.
Recommendation
Add a page size to PGStore.Directory (recommendation 9).
Status
Fixed in dcbbdb0

Consistency

Severity
Medium
Axis
consistency
Class
check gap
Location
Makefile:125
Finding
recurrence since review_kwalitai_20261016, review_kwalitai_20261017 and review_kwalitai_20261017_2: no format gate. gofmt runs only inside make tidy, which rewrites files; .golangci.yml has no formatters section and CI never checks formatting. Tree is gofmt-clean at HEAD (checks/format_tidy.txt), so the gate is absent, not red
Cause
gofmt runs only inside make tidy, which rewrites files; no lint configuration or CI step fails on unformatted code.
Recommendation
Enable the gofmt formatter in .golangci.yml so make audit and CI fail on unformatted code (recommendation 6).
Status
Fixed in 23d9c88
Severity
Medium
Axis
consistency
Class
check gap
Location
.golangci.yml:13
Finding
recurrence since review_kwalitai_20261017 and review_kwalitai_20261017_2: errcheck and unused are not enabled (default: none), staticcheck U1000 is disabled at .golangci.yml:43, and the errcheck exclusion at .golangci.yml:47 targets a linter that never runs. No dead-code or discarded-error check exists; 0 non-deliberate instances today
Cause
errcheck and unused are not enabled and staticcheck U1000 is disabled, so no check looks for dead code or discarded errors.
Recommendation
Enable errcheck and unused and drop -U1000 from staticcheck (recommendation 6).
Status
Fixed in 23d9c88

Security

1 open finding on this axis withheld until fixed.

Severity
Medium
Axis
security
Class
check gap
Location
cmd/kwalitai/serve.go:68
Finding
recurrence since review_kwalitai_20261016, review_kwalitai_20261017 and review_kwalitai_20261017_2: hardening presence probes absent (spproof installed, no .spproof.yml). Server timeouts, SetTrustedProxies and the SMTP deadline are in place by hand; no check fails if one is removed
Cause
Server timeouts, trusted proxies and the SMTP deadline are set by hand with no policy that fails when one is removed.
Recommendation
Add a .spproof.yml policy proving each hardening setting and run it in make audit.
Status
Fixed in 8672022
Severity
Medium
Axis
security
Class
check gap
Location
.github/workflows/ci.yml:30
Finding
recurrence since review_kwalitai_20261016, review_kwalitai_20261017 and review_kwalitai_20261017_2: no workflow security linter (zizmor or pinact) in make audit or CI; 19 of 19 action references across both workflows are pinned by mutable tag
Cause
No workflow security linter runs, so actions pinned by mutable tag go unnoticed.
Recommendation
Run zizmor on .github/workflows in make audit and CI (recommendation 1c).
Status
Fixed in 6a53ace
Severity
Medium
Axis
security
Class
check gap
Location
.github/workflows/ci.yml:43
Finding
recurrence since review_kwalitai_20261017 and review_kwalitai_20261017_2: no secret scanner (gitleaks or trufflehog) in make audit or CI; 0 hits at HEAD per a local trufflehog filesystem run (checks/trufflehog.txt)
Cause
No secret scanner runs in make audit or CI.
Recommendation
Add gitleaks to make audit (recommendation 7).
Status
Fixed in df25149
Severity
Medium
Axis
security
Class
tool-detectable
Location
.github/workflows/release.yml:27
Finding
recurrence since review_kwalitai_20261016, review_kwalitai_20261017 and review_kwalitai_20261017_2: goreleaser-action, checkout and setup-go pinned by mutable tag in the contents: write release job that receives GITHUB_TOKEN
Cause
Actions in the release job, which holds a write token, are pinned by mutable tag.
Recommendation
Pin every uses: to a commit SHA (recommendation 1c).
Status
Fixed in 6a53ace
Severity
Medium
Axis
security
Class
tool-detectable
Location
.github/workflows/ci.yml:136
Finding
recurrence since review_kwalitai_20261016, review_kwalitai_20261017 and review_kwalitai_20261017_2: 16 action references pinned by mutable tag, including third-party trivy-action in the security-events: write scan job
Cause
16 action references, including a third-party action in a job with security-events: write, are pinned by mutable tag.
Recommendation
Pin every uses: to a commit SHA (recommendation 1c).
Status
Fixed in 6a53ace
Severity
Medium
Axis
security
Class
tool-detectable
Location
Makefile:106
Finding
make preflight gates make up on POSTGRES_PASSWORD and APEX_DOMAIN only; an empty TRUSTED_PROXIES boots prod behind Traefik with every visitor sharing one contact bucket (3 messages, then 1 per 10 minutes), so one sender can keep POST /contact at 429 for everyone. README and env.sample document the risk; nothing checks it
Cause
make preflight checks two settings only; an empty TRUSTED_PROXIES lets production start with every visitor sharing one contact limit.
Recommendation
Refuse an empty TRUSTED_PROXIES in make preflight (recommendation 1a).
Status
Fixed in fec586f
Severity
Medium
Axis
security
Class
found by review
Location
internal/web/contact.go:167
Finding
recurrence since review_kwalitai_20261017_2: contact limiter keys each bucket on the full client address with no global ceiling, so the number of emails sent is bounded only by how many addresses a sender controls (an IPv6 /64 holds 2^64); whether the edge accepts IPv6 was not verified
Cause
The contact limiter keys each bucket on the full client address and has no global ceiling.
Recommendation
Key IPv6 clients on their /64 and add a global ceiling (recommendation 4).
Status
Fixed in 2cae8cb
Severity
Medium
Axis
security
Class
found by review
Location
internal/web/ingest.go:46
Finding
recurrence since review_kwalitai_20261017_2: unauthenticated POST /ingest and POST /publish (publish.go:35) are logged at Warn with the caller-chosen repo query value before the token check, up to the 1 MB default header limit, so an anonymous client controls log volume within the Traefik limit of 10 per minute per IP
Cause
The caller-chosen repo value is logged before the token check.
Recommendation
Log the repo value only after the token check (recommendation 5).
Status
Fixed in d72e8d6

Correctness

Severity
Medium
Axis
correctness
Class
found by review
Location
internal/ingest/ingest.go:181
Finding
re-ingesting an identical report directory under another --repo (schema 1, or schema 2 with a null repo; CLI and POST /ingest ?repo=) conflicts on report_sha256, and existingEvaluation returns the first repo's evaluation with no warning because the hashes match; the operator sees 'already ingested' and a later publish of the intended repo fails with 404. Confirmed by reading ingest.go:154 and 181-193, not executed
Cause
existingEvaluation matches on the report hash alone and never compares the repository.
Recommendation
Refuse an identical report under another repository (recommendation 2).
Status
Fixed in 9bded8d
Severity
Medium
Axis
correctness
Class
found by review
Location
internal/ingest/report.go:83
Finding
recurrence since review_kwalitai_20261016, review_kwalitai_20261017 and review_kwalitai_20261017_2: report copy is not fsynced before the rename and an existing destination is trusted on os.Stat alone, although report_sha256 is stored and could verify it; documented in the doc comment, not fixed, not tested
Cause
The report copy is renamed without fsync, and an existing copy is trusted without checking its stored hash.
Recommendation
Fsync before the rename and verify an existing copy against report_sha256 (recommendation 8).
Status
Fixed in 3b7ffa0

The same audit for your repository

Get your repository audit, 490 EUR

The price includes one follow-up verification after your fixes.