Crowdsourcing product development

Imagine you're using a software product and you think, "it'd be great if there was a feature for X." You click the "Bug/Improvement" button, type what you want and why, and hit submit. Ten minutes later, a toast pops up saying that your feature is live and you just need to refresh.

This isn't a thought experiment. It's been running in my product, Polytask, for a few weeks now, and it's already built over a dozen features this way.

Why do this

1. Customers never hit a wall. Normally when a product has a limitation, you put in a request and wait six months, hoping it makes the roadmap. Here, it just gets built.
2. Customers become contributors. When their idea ends up in the product, they feel more ownership of it and more committed to it.
3. It unlocks ideas that would otherwise stay hidden. There's so much knowledge and creativity sitting in customers' heads, and normally they keep it to themselves. If they know it'll get built straight away, they're far more likely to share it.
4. It saves me a huge amount of time. Realistically, triaging a feature request means babysitting Claude while it builds it, then handling the comms myself. The pipeline does both.
5. It's a pipeline, not just a feature. Once I trust the process, I can send almost anything through it: exceptions in prod, a cron job that improves the product on a regular schedule, and so on.

How it works

First, you need a stack that AI can work inside super reliably. The short version: you need to get to the point where if you ask Opus 5.5 to "build X", 95% of the time your next prompt is "ok, ship it to prod." If you're there, your stack is ready. If you're not, whatever is stopping you needs to be solved first.

Once you have that, the rest is straightforward and here's how I've implemented it:

1. The user enters a feature request in a modal in the app.
2. The request is posted to a private Discord server.
3. A server at home listens on Discord and picks it up.
4. A sandbox spins up and decides what to do with it: build it, or escalate to a human.
5. If it decides to build, it builds the feature, reviews it, tests it and pushes to prod.
6. Progress is posted back to the Discord thread along the way.
7. The user gets a toast notification letting them know their request is done.

Concern 1: Security

Isn't someone just going to use this to hack the site? Not if you put the right safeguards in place. Here's what I'd recommend:

• Use a top model. Only use Opus 5.5 or better to handle requests.
• Only accept requests from known users. Your bot can check whether the user is a known, paying customer. Start with an allow-list of the exact people you trust.
• Add an approval step if you want one. A human "Approve" before each request gets built is a cheap extra layer.
• Run every request through a security filter first. Could the user be trying to exploit the product with this request? Is it something a normal user would actually run into, written in a normal way? If it contains weirdly specific details or programming references (libraries, code snippets, etc.), escalate it to a human instead of building it.

Concern 2: Building the wrong thing

A request can be perfectly safe and still be a bad idea. So you'll want product management knowledge baked into the bot: the change must be generic and useful across all users, it must align with the strategic direction of the product, and so on.

This is where you need some good old-fashioned testing. Encode your product decision-making into an agent, fire a bunch of feature requests at it, and see how it handles them. Make sure to include some you'd say no to.

Run an experiment

You don't have to go all in on day one. Find a way to add auto-build to your product that fits your risk appetite. The easiest way to reduce risk is to hand-pick who gets access. That gives you a feel for how people actually use it, and shows you where the risks are before you open it up more widely.