To: Andrew @corney, @thomthom, @eneroth3, @jiminybillybob, @Matheron
TL;DR
I’d like to propose two complementary tools that move first-pass validation out of your queue and into developers’ local workflows — before the Submit button is ever pressed. Both are designed to be community-built, open-source, and low-maintenance for your team.
Context: How Other Ecosystems Handle This
You’re not alone in this challenge. Every major extension/plugin marketplace has wrestled with the same surge-quality-security trilemma:
-
VS Code Marketplace (Microsoft): Runs automated malware scans on every submission and recently added secret scanning in their vsce CLI tool so developers catch credential leaks before upload. They also introduced a Trusted Publisher badge system to fast-track trusted vendors — directly analogous to your paid/subscriber prioritization.
-
Open VSX Registry (Eclipse Foundation): Just rolled out mandatory pre-publication security checks in early 2026 — scanning for namespace impersonation, accidental secrets, and known malicious patterns. Suspicious submissions are quarantined for manual review rather than blocked entirely. Their key insight: shift from reactive to proactive.
-
npm: The publish-please / npm audit ecosystem lets developers dry-run a publish and catch vulnerabilities, uncommitted changes, version mismatches, and sensitive files before anything reaches the registry. The prepublish hook enforces this automatically.
-
GitHub Marketplace Actions: Enforces automatic version-tag-to-manifest matching as a publish prerequisite. No mismatch, no publish. Simple, zero-review-team overhead.
The pattern is consistent: the most effective triage is the triage developers do themselves, when given good tooling and fast feedback loops.
Proposal 1 — ew-preflight: A Local Pre-Submit CLI
A command-line tool (Ruby gem) that developers run before packaging their .rbz. It wraps existing tools and adds EW-specific checks.
Tier 1 — Instant checks (< 5 seconds)
These run on every invocation and catch the most common rejection reasons:
| Check |
What it catches |
rubocop-sketchup --only SketchupRequirements |
Namespace pollution, exit!, $LOAD_PATH modification, Gem.install, missing SketchupExtension registration, SketchupRequire with file extension |
| RBZ structure validation |
Single root .rb + matching subfolder; root file only registers, doesn’t load |
| Version consistency check |
SketchupExtension.new version string vs. EW submission metadata — catches the 1.3.2/1.4.6 class of mismatch your team flagged |
| Metadata completeness |
Extension name uniqueness within local namespace; description non-empty; support URL present |
| Binary / compiled file detection |
Flags .so, .bundle, .dll files and prompts developer to confirm they have trusted-vendor status |
| Secret scanning |
Regex sweep for API keys, tokens, passwords in source (mirrors what VS Code’s vsce now does) |
Tier 2 — Deep scan (opt-in, --deep flag, ~30–60 seconds)
Runs an LLM-assisted review against an EW-specific system prompt. Think of it as a local CodeRabbit for SketchUp extensions:
- Checks for AI-generated “slop” patterns: generic boilerplate, lack of SketchUp API usage, copy-paste from tutorials
- Flags use of undocumented SketchUp APIs
- Detects obfuscated code blocks that would trigger your security concern about compiled/encrypted submissions
- Reviews observer callbacks for missing
start_operation with transparent flag
- Can run locally via Ollama (no cloud dependency) or OpenAI-compatible endpoint
Output example
$ ew-preflight build/my_extension.rbz
EW Preflight v1.0 — AgiliCity
================================
✅ RBZ structure valid
✅ Single root .rb loader
✅ SketchupExtension registered (load-default: true)
✅ No global namespace pollution detected
⚠️ VERSION MISMATCH — loader: 2.1.0 / EW metadata: 2.0.9
❌ SketchupRequire: remove .rb extension on line 14 of main.rb
❌ Binary detected: libs/x64/my_solver.so — confirm trusted-vendor status
ℹ️ Run with --deep for AI-assisted quality review
2 errors, 1 warning. Fix errors before submitting.
Distribution
- Ruby gem:
gem install ew-preflight
- GitHub Action:
uses: sketchup-community/ew-preflight@v1 — runs on every push/tag
- Works offline; no EW account required
Proposal 2 — Open a Subset of ew-review to the Community
We understand your internal ew-review tool is the gold standard, and we’re not asking for full access. A more modest ask:
Publish a read-only “developer preview” mode — a subset of your review checklist as a structured JSON/YAML spec that community tools (including ew-preflight) can consume.
Specifically:
-
A machine-readable list of auto-rejection rules — the things that are always a hard NO, expressed as a schema developers can validate against locally. Version mismatch format, binary restrictions, namespace requirements, etc.
-
A versioned “what changed” log — when rules change (like the current version-string policy), developers get notified via gem update rather than finding out after a 3-week queue wait.
-
Optional: a sandboxed API endpoint — POST /ew-review/preflight that accepts an .rbz and returns the Tier 1 results your automated pipeline already produces, without entering the human-review queue. This is essentially what you described as “AI review on submission with feedback visible within a few minutes” — we’d just make it callable by developers before submission too.
This is the model the Open VSX Registry is moving toward, and it has the significant advantage that you control the spec — community tools stay in sync automatically when you update it.
What This Gives Your Team
| Current situation |
With this tooling |
| Version mismatches discovered in review |
Blocked at preflight, developer fixes in minutes |
| Binary files from unknown developers enter queue |
Flagged locally, developer aware before submitting |
| AI slop detected manually |
Tier 2 scan surfaces low-effort submissions |
| New team member doesn’t know unwritten rules |
Rules are in the gem; always current |
Human reviewer reads every SketchupRequire offense |
rubocop-sketchup already catches this; just needs a wrapper |
The goal is not to replace your review — it’s to ensure that what reaches your queue has already passed the mechanical checks, leaving your reviewers free to focus on the things that actually require human judgment: originality, safety intent, UX quality.
Proposed Next Steps
-
We build ew-preflight v0.1 (Tier 1 only) — open source, MIT license, community-maintained. We’re happy to build this without any commitment from your team.
-
We’d welcome a 30-minute call to understand which of your current rejection reasons could be expressed as machine-readable rules. Even a rough list shared informally would let us cover 80% of the cases.
-
If the sandboxed API endpoint (Proposal 2, item 3) is on your roadmap anyway, we’d love to be early testers.
We recognize you’re stretched thin right now, so we’ve deliberately designed this as something the developer community builds for you — not something that requires your team’s engineering time to maintain.
Thanks for all you do keeping the ecosystem healthy!
Disclaimer: I’m posting this on behalf of AgiliCity | Modelur, not as Extensioneers Guild member. But I’m pretty sure some Extensioneers would be interested in contributing if we are to move forward with this.
